MV3D Development Blog

July 3, 2013

MV3D 0.80 Released!

Filed under: Uncategorized — Tags: , , , — SirGolan @ 12:24 pm

We are pleased to announce the release of MV3D 0.80! The highlights of this version are a revamped toolchain which should enable a better workflow for building new content. Some of the improvements include tabbed tools instead of docked windows, a property grid for editing, and support for 3rd party asset types. Also in this version is the first example gameplay system– the stats system. To support it is a new component system that loads objects from an xml template and attaches components such as the new stats system. Finally, many improvements to the low level server code were made including initial support for asynchronous database connections, and a new JSON based protocol which can be used instead of PB to support clients written in languages other than Python.

Contributors to this release include:

  • Mike Handverger
  • Judd Cohen
  • johne5
  • Mythros

MV3D is an open source multiplayer game and virtual world framework written in Python. It encompasses a scalable server with dynamic load balancing, a robust set of content creation tools, and an extensible 3D client application. MV3D provides the foundation to build anything from 3D chat rooms to full MMORPGs with the goal of letting you concentrate on creating a unique virtual world instead of the infrastructure to support it.

The next release will be 0.85 and will focus on tools, usability, and example gameplay.

For more information on MV3D and this or future releases, please visit the website. The full release notes for version 0.80 are available online. For further inquiries, feel free to stop by our IRC channel on irc.freenode.net #MV3D.

Retrospective

What went well?

  • py2exe client and tool releases include support for both Panda3D and Ogre3D.
  • Datastore load test showed excellent performance. 
  • Multiple projects are now using MV3D in various forms.
  • Relaxed release process. Otherwise it would have been another 3 months! Time will tell if this is a “well” or “less well” though.
  • Improved support for traditional static world MMOs.

What went less well? 

  • 18 Months is way too long in between releases.
  • MV3D is still not very user friendly.
  • Not many flashy features were added.

March 27, 2012

Windows, Linux, and Mac OS X support!

Filed under: Uncategorized — SirGolan @ 10:12 pm

In svn trunk, MV3D now supports all three platforms! That includes server, client, and tools. This is a pretty big deal as until fairly recently, MV3D didn’t support running anything but the server on Linux. Both Ogre3D and Panda3D are available on Windows and Linux. OS X only supports Panda3D at the moment. There’s a build slave configured for each supported OS combo, and there’s now a wiki page showing supported platforms.

Panda 3D on OS X:

 

Ogre3D on Linux:

 

The 0.75 release is wrapping up now, but we’ll need as much testing on all supported OSes as we can get. Unfortunately, there isn’t an easy install for Ogre3D on Linux just yet, but that’s something we’ll work on in coming releases. The first step here is to get it all working. Next up is making it easier to use. On the other hand, Panda3D is a breeze to install for any supported platform.

I’m currently working out what the next release will bring. There are a lot of options and a lot that needs to get done before we hit 1.0. Stop by IRC or the forums if you are interested in helping out or finding out more about MV3D.

September 19, 2011

Dev Environment Installer Available!

Filed under: Uncategorized — SirGolan @ 5:22 pm

Probably the biggest barrier people have to helping out with MV3D is getting a development environment set up. It’s pretty easy (on Windows) to run from the prebuilt binaries, but those don’t allow you to edit the source code. As of today, a Windows install package for the development environment is now available! This includes:

  • Python
  • Panda3D
  • Ogre3D
  • All of the prerequisite Python libs.
  • Combinator - a tool used to manage development with MV3D’s dev process.
  • Directx and vc redistributables.

There are a few caveats though. You must pick either Panda or Ogre3D. Although it will install both, one of them won’t be fully configured since there’s only one active Python install. Also, there will be many sub-installers that come up. Make sure to choose the default options on everything with those.

Finally, and most important. Do not use this if you have an existing Python installation other than Python 2.6 in C:\Python26! It will mess up your current install otherwise.

Thanks,

Mike

August 7, 2011

MV3D 0.60 Released!

Filed under: Uncategorized — SirGolan @ 3:12 pm

We are pleased to announce the release of MV3D version 0.60! This release focused on scalability of worlds and includes support for splitting a single area across multiple server processes with automatic load balancing and redundancy. Areas can now be connected together using gateways to build worlds limited in size only by the amount of available hardware. The Overseer Cluster Management tool was upgraded in order to better handle many processes across multiple physical servers. Camera controls across all content tools have been unified. Better support for Panda3D clients was added as well.

MV3D is an open source virtual world and multiplayer game framework written in Python. It was designed with scalability in mind and is able to distribute a world across as many servers as needed while dynamically balancing the load. The simulation framework is not specifically slanted towards any one genre of online game, and can just as easily be used for a space game as a fantasy setting. Objects on an MV3D server are simulated using the ODE physics engine. A single server can host thousands of simulated objects. MV3D can use either Python-Ogre or Panda3D on the client for visuals.

For more information on MV3D and this or future releases, please visit the website. The full release notes for version 0.6 are available online as well. For further inquiries, feel free to stop by our IRC channel on irc.freenode.net #MV3D.

Video and screenshots of our new Demo World, the Town of Wellston:



Retrospective:

What Went Well:

  • Last minute shader and dynamic light support!
  • The client is now authoritative for its movement which means things are much smoother.
  • Fixed some crazy bugs including a few with the persist code which were pretty serious.
  • The persist upgrade system works as designed! The old demo world’s databases were automatically updated with no intervention or errors!
  • New Ogre .scene import functionality in Builder is pretty awesome and saved hours of work on the demo world.
  • New triangle mesh colliders including automatic converter from Ogre/Panda meshes makes indoor world building much easier.
  • The new representation file format is a definite win!
  • Official support for Panda3D client and tools on Linux.

What Went Less Well:

  • Client movement is still jerky (but at least we know it’s a client only problem).
  • The release took 2 months longer than expected.
  • Panda3D support is still not 100% there. Missing: sky, animation, and terrain texture splatting.
  • The tool workflow needs to be simplified.
  • Linux support for Ogre3D is still lacking due to inability to successfully build Python-Ogre.

Release 0.70 Planning:

The next release will be 0.70 and is focused on tightening up the toolchain as well as adding new client side features. The current list of tickets planned is available here. We will be meeting on #MV3D to go over the plans for 0.70. I’ll post a comment here when we decide on the time and date.

July 18, 2011

Testing MV3D v0.60

Filed under: Uncategorized — SirGolan @ 9:25 pm

The next release is nearly ready. We’re looking to get some people to help with testing before officially giving it the thumbs up. You can grab the latest binaries and check it out. The demo server will be updated over the next week, and expect to see a new and improved demo world. Here’s a little preview:

One exciting thing you’ll notice with the binaries is that there are now Panda client and tool binaries for Windows. You’ll need the Panda3D runtime installed to use those.

Look for the new release in the next week or so if all goes well!

February 19, 2011

MV3D Goes Beta!

Filed under: Uncategorized — SirGolan @ 3:25 pm

MV3D 0.50 has just been released! This release took quite a bit longer than intended and ended up being 2 months late. On the flip side, there’s a lot of new functionality such as support for the Panda3D rendering engine and a path-finding system for NPCs. In addition, many bugs were squashed and the performance problems of the last release are pretty much gone.

What Went Right:

  • Panda3D support is great!
  • The NavMesh + A* path-finding system is pretty much industry standard and includes automatic NavMesh generation.
  • A client Patching System is available complete with patch building tool.
  • Documentation. There is now a User Manual.
  • 50 bug tickets closed! This is the most stable release to date.

What Went Wrong:

  • Underestimated the time it would take to write a fully functional NavMesh generator, which extended the release by 2 months.
  • Started documentation too late in the project (this isn’t specific to this release).
  • It is still pretty hard to configure a whole cluster. This will be addressed in the next release.
  • MV3D’s tools could use a polish pass and more documentation.

I’m very excited about the next version as well. One of the planned features is splitting up of areas across multiple servers. Another big push is for making MV3D more accessible to new users. Part of that is expanding the user manual as well as simplifying configuration.

For more information on MV3D, see the website at www.mv3d.com, or the release notes. There is also a #MV3D chat room on irc.freenode.net which has been getting more use lately. If you don’t have an IRC client such as mIRC, Pidgin, or Trillian, you can use the web interface.

September 27, 2010

MV3D 0.44, 0.50, and beyond.

Filed under: Uncategorized — SirGolan @ 11:03 pm

MV3D 0.44 is ready, and 2 days ahead of schedule! The release was focused around building an asset tool-chain to facilitate adding content to MV3D worlds. In that respect, it was an outstanding success. The sprint only lasted about 3 months, which is a lot shorter than most MV3D releases. I’d like to keep up that pace moving forward. The most amazing thing to me for this release is that for the first time, you can spin up an server and populate it completely using tools!

The Good:

  • Solid asset tool-chain!
  • Much more stable than 0.42.
  • Guide (the GUI data-binding library) is awesome!
  • Overseer cluster manager really speeds up iteration time and should really help managing servers.
  • MV3D’s Client now works on Linux!
  • Users can download server, client, and tool binaries for Windows which makes it incredibly easy to try out MV3D.

The Bad:

  • Performance is pretty horrible on the client and server.
  • More tools still needed. Plus, the new tools are just at a bare minimum state.
  • Even though there were many bugs fixed, quite a few big ones remain.
  • Not enough work was done on guidece and guideweb, the CEGUI (in game) and web versions of guide respectively.
  • Even with the GUI, it’s still really hard to set up a cluster of MV3D servers. Partly due to bugs but mostly due to all the settings that have to line up exactly.
  • Still some bugs that can cause datastore corruption on upgrades.

All in all, it was a solid release and a lot was done in a short time. For next release, which will be 0.50, the current plan is bugfixing, performance, and properly functioning areas with gateways.

Including the plans for next release, there are a few major features missing which are required for a 1.0 release. The ones I am currently aware of are load balancing an area across multiple servers and pathfinding AI. I’m starting to put together a roadmap for these. It will be updated more soon.

Head on over to MV3D and get the new version! Binaries will be up there tonight. If you are in a hurry, you can grab them hot off the press from the Hudson builder.

June 18, 2010

New Release and Moar Content

Filed under: Uncategorized — SirGolan @ 11:21 pm

MV3D 0.42 released today! Check out the release notes or download the MV3D Windows Client! In this release, the core systems were made ready for tool building and content creation. This included a redesigned persistence mechanism and a cross system GUI layout.

The Good:

  • New persistence system is pretty awesome.
  • For the first time ever, you can export or import sections of the game world!
  • Pretty cool start to a new way to build UIs which should save tons of time and allow for quicker iteration as well as reusability.
  • Made an unreleased play in browser app for MV3D.

The Bad:

  • New persistence system may have some performance issues.
  • Tons of bugs still need to be squashed.
  • Didn’t get to any tools which was the original focus of the release.
  • No visual features added– nothing cool to show off really.

What’s next? Well, the next release will focus on the content pipeline. This means upgrading Ogre (since it’s long overdue) and tools, tools, and more tools. The one major unknown is how the content pipeline will actually work. Ogre has a lot of half functional or very basic exporters for various 3D modeling packages, which is an unfortunate start to the pipeline. That’s not so much something that MV3D can or should solve, but it’s fairly disappointing.

After exporting the raw files, they have to be added as assets in MV3D. Currently, RED can do this in a very basic way. It does include the ability to SFTP the asset files to the server that will be distributing them which is handy. This will need to be extended and made smarter and easier to use. RED should definitely get in app previewing of assets. It’d also be nice to add a preview image to assets in general. This could be displayed in RED, in the IGE, or on the web editor. Speaking of the web editor, it’s completely busted more or less. That’ll need fixing, which may come in the way of guide (the GUI framework which also supports web GUIs). What’s awesome about fixing it with guide is that the same GUI can be used in IGE or RED without any modifications.

With the assets imported, the real fun begins. RED will need a plugin that lets you add and arrange objects in a vacuum (i.e. not anywhere live in the game). Then you can use Freeze (the importer/exporter built in 0.42) to save off pieces of the game world. An additional RED plugin should be able to connect to a server and edit existing areas. Here you could import the stuff you created before and place it in game. This should also have the same functionality of the other plugin so that you can edit anything you want.

One other missing piece is the collision volumes. These need to be manually placed (or generated from static geometry in some cases). This means the other required tool will be something which is able to position colliders for models.

But wait, there’s more! There needs to be a way to edit bodies of water, and in fact, the current ocean object should probably be made into a water volume since right now it’s just an infinite plane. Similar to water, the sky should be editable, and if you are editing the sky, physics properties of the realm need some love too.

Finally, it’d be nice to be able to have more than one area in the game at some point. Part of the RED area editor should include connecting areas together. At this point, it’s been so long since this was tested, that it may not even work any more.

Sounds like I have quite a few tools to write. Any volunteers to help?

January 31, 2010

Storage Space

Filed under: Uncategorized — SirGolan @ 3:04 pm

I’ve been trying to come up with a better way to persist MV3D state info. Basically, I need a solution that meets the following:

  • Synchronous when saving data
  • Transactional
  • Extremely Fast when saving data
  • Maintains data consistency
  • Supports queries
  • Externally available (i.e. you can view/modify/query the data outside of MV3D)
  • Low CPU overhead

    The current solution does support most of those. The four it’s probably worst about are queries, data consistency, CPU overhead, and external availability. It’s also just kind of wacky. You define a list of properties in your class that should be stored along with which of those properties you’d like to generate a search index for. When it’s time to save an object, it generates a uuid for it, then it stuffs all the properties you defined into a special object, pickles it, and then uses Axiom to persist it to sqlite. It uses index objects stored in Axiom which match the uuid with the value of the property in the object. Querying is also a little odd since you use magical properties on a query object (q.a == 12). Unfortunately, the object type you are querying for may not have an a attribute since the query object doesn’t really care. Another interesting part of the system is that it requires all servers to store their data locally. I’m not sure how I feel about this since some MV3D data really isn’t useful to other servers– at least as the design stands now. However, it does require that there be two servers for every item stored in order to recover from a catastrophic failure of a single server.

    I’ve tried in the past to go with a completely Axiom based solution, but that didn’t work well because of the restrictions Axiom puts on any classes you mark up as Items that can be persisted. One thing I’d been thinking about was a dual object approach where you have the in memory object, and when it needs to persist, it stores all its persistable attributes into a specific Axiom class. This system could even use powerUps. This had a couple of downsides, one of which being that you had to define twice as many classes. It also ends up that you’d have to define your datastructure twice as well. The other option is to have a utility to generate that extra code, but I’m not a big fan of code generation, so I’d like to avoid that.

    What I’m currently thinking of is adding a new service type to MV3D that would generally act like a data access layer (DAL). You’d mark up classes in Python with which attributes should be stored– and I’m thinking of combining this with the attributes that are sent over the network. In order to persist an object that had this mark up, you’d hand it over to this service. It’d then store the marked up attributes and return a unique ID you can use to retrieve the object later. While technically at this point, it really doesn’t matter where or how the data is stored, I’m going to go into that a bit. There would be several low level functions: register schema, upgrade schema, add object, update object, get object, and query. Basically, schema in this sense is the markup of a particular class which defines attributes to store and what type they are. For SQL based stores, this will be directly associated with a table. Schemas will be versioned so that when a newer version is registered, all data is upgraded. I see this happening by renaming the existing table, creating a new one with the new schema, and running code on each element to upgrade it.

    After the schema is created, adding an object would just be a SQL insert and updating would be similar. Both of those could return the primary key of the row as an ID. Adding the primary key to the object’s class and the service’s location would give you a method of retrieving that object from anywhere. Querying for the user could make use of the class markup objects to give them the ability to do something like: store.query(Person, Person.name == “mike”).

    Looking at the requirements I mentioned, the major ones that it fails are being synchronous and maintaining data consistency. I put synchronous when saving up there because I like to be able to wrap methods that modify data in a @autoStore decorator which stores the object after the method runs. If this were async, either that function would have to return a deferred, or you’d lose data consistency if there was a failure. It would also be sweet to be able to define the markup classes as descriptors which would a) store the object and b) update any network clients of the changes. A possible solution to this would be to strategically limit the remotely available functionality of the service to the low level operations I mentioned previously. Then to make them have nothing to do with converting objects into persistable data. This would require a local object/service to interact with the remote store. The local code could build a queue of transactions to send to the master store, and this queue could be persisted via sqlite. This way, the data would be synchronously stored to disk locally and then sent off to the remote store whenever.

    One other issue here is that theoretically multiple servers could access the same object in the store at the same time. Yes, that sounds like a feature, but the aforementioned queue causes some issues with doing this. The data in the master store may not be 100% up to date. What might be interesting is to create a locking mechanism whereby when loading an object from the store, you acquire a lock on it. The hard part will be making sure that the lock goes away whenever you stop using the object and that failure conditions (such as your server crashing while holding locks on multiple objects) are properly handled.

    Another issue is what to do if the store rejects a change that seemed like it would succeed locally. If we go with writes that return immediately and don’t wait for success, it would already be too late to tell the original caller that something went wrong. What are some of the reasons this would happen?

  • Remote storage server is down. Ok, just keep it in the queue until it’s back up.
  • Remote storage server is out of space. Keep it in the queue is probably best.
  • Schema mismatch or invalid data. This seems like the worst failure mode.

    What do we do with a schema mismatch or invalid data? This indicates a fairly serious problem, so maybe it deserves a serious resolution– revert the object and all other objects related to the transaction that failed both on the remote store and in memory. That still doesn’t seem like a great way to resolve the issue, but it would maintain data consistency.

    All in all, this seems like it’d be pretty challenging to implement, and it’d mean that I’d be maintaining a DAL/ORM as opposed to using a pre-existing one. I’m generally against re-inventing the wheel like that, but all the Python ORMs I know about are designed for webapps and don’t translate well to MMOGs.

    Really at this point, I’m looking for someone to talk me out of this and tell me why it’s a horrible idea. Otherwise, I might just be crazy enough to try it out. One thing that’s bugging me is that the current mechanism works. I haven’t had any lost data issues or anything; however, I’ve found that sometimes it’s just easier to blow away the whole store than to say revert a change I made.

    At the very least, I like how the new system abstracts out the DAL to a service that can optionally be on a remote server. This makes it so the underlying persistence technology can be pretty much anything. Another major benefit would be the ability to freeze objects by storing them and stopping simulation of them. Then you could unfreeze them later on a different server. What’s this good for? Well, making it so your character leaves the world when you log out for one. This isn’t currently possible without a load of hacks.

  • January 24, 2010

    Ship it!

    Filed under: Uncategorized — SirGolan @ 3:09 pm

    MV3D Version 0.40 will be officially released today. I’m building the Windows binaries and source release now. I’m happy with the release, but visually, it’s a bit of a step backwards. I don’t have any good screenshots to show, or a movie detailing the new features. There are new features though. They just aren’t visible, and if they are working properly, you should never need to worry about them.

    The main theme of this release was completely refactoring the scalability and redundancy of MV3D at the core. This was a very big success. I now feel comfortable saying that MV3D is ready to scale. In a properly configured cluster, items will be both spread across the servers and have redundancy in the case of a failure. Items can be transferred from server to server based load balancing needs.

    Looking at MV3D as a whole, there are still a few elements that need major work. The most obvious right now are tools, the client, and possibly data storage (sadly). It seems to me that tools to generate content are less useful if the data storage mechanism changes significantly (since either I’ll have to write a converter or you’ll have to lose all your stuff). On top of that, making the client look good and perform well is hard to do and mostly pointless without any content.

    With that said, the logical order is to work on data storage first, then tools, and finally fix up the client. I’m hoping to not spend too much time on data storage. I have some ideas, and generally want to make whatever I come up with retain the same API. For tools, RED needs to be extended and generally made useful. There also needs to be a way to more or less export and import whole areas (or even whole realms) at a time. Any project bigger than a single person messing around will want to have this ability so that they can version control their world.

    While fixing a recent bug, I came to realize that a tool for managing MV3D clusters is desperately needed. My repro steps for that bug involved checking in to SVN, updating two servers, restarting them, and then connecting to their SSH console. It would have been nice to be able to run 2 or more servers on my desktop for testing. So, this is another tool that should be forthcoming.

    While I’m talking about that, the reason I had to repro the bug on the servers as opposed to in unit tests is because while there are unit test facilities for testing a full server against a real-ish client, there isn’t much in the way of full integration tests between multiple servers. Some of the problems I was tracking down ended up being issues that came up only when you started two servers, created some stuff, shut them down, restarted them, and tried to create more stuff. Integration test helpers for scenarios like this would be pretty awesome.

    Another headache I ran into while rebuilding the demo servers was that since the starter world initialization code is written to work on a single server that provides all services, it doesn’t work well when you have a separate directory or account server. This made it so that I was basically typing the code on the console to create the world. No one should be expected to have to do that– a tool is needed that can bootstrap a set of servers to get them to the point where other tools like RED can be used to create a world.

    There are plenty more things that need tools, but so I don’t bore people more than usual, I’ll move on to the client. The client seems slow. In reality, it gets 150-160 FPS, but it still feels sluggish for some reason. This is probably physics related, but it makes things look bad. The other big issue on the client is that since the server sends position corrections to the client’s avatar instead of the other way around, moving around can be a little clunky (even with the smoothing that’s done now if there’s a discrepancy). The UI on the client is very clunky and ugly. I’d really like to get that looking better. There’s tons of stuff really that needs help.

    Sounds like a lot of work, doesn’t it? It definitely is. If you’ve gotten this far, you must be interested in MV3D, so why not contribute? There are a ton of open tickets now that would be excellent for someone looking to get started.

    That’s all for now! If you try out the new release, just be warned that it’s visually the same as the last release. I’m hoping that as tools come online, I’ll be able to add more content to the demo world, but that depends on what code changes I have to make on the server to support the tools.

    Newer Posts »

    Powered by WordPress