MV3D Development Blog

October 20, 2008

Nevow + Axiom website: finished product (sort of)

Filed under: Uncategorized — SirGolan @ 10:48 pm

First off, websites are hardly ever “finished” as I’m sure many of you know. Anyway, I’ve got the site up for real now although it’s still lacking in content (Corey, put some stuff up there, hint hint). Working with Nevow and Axiom was great and enabled me to build the site in a very short amount of time. The brunt of the work was done in about a month. It’s got a complete admin interface to it including image uploading. I also wrote a full featured shopping cart for the first time. Check it out:

http://www.cescenics.com

By the way, don’t those calendars look nice? I know you want to buy one.

Another cool thing– got to give some credit to PayPal. Their sandbox is a great tool, but also they’ve added some really neat stuff since I last used their API. You can have them notify you via a hitting a URL of your choosing when the status of a transaction changes. So, I’ve used that to verify that a purchase has actually gone through.

This website plus lots of extra hours at my job have really slowed down MV3D work. I’m currently finishing up the in game ui for adding and removing objects. Unfortunately, for every in game editor piece I finish, I find about 5 more that need to be done. One I just thought of today are body modifiers– so you can modify the physics properties including colliders and also the visual properties like models, textures, scaling and placement.

I have found a new web host though. And I’ll be attempting to move trac over there shortly.

July 6, 2008

Nevow Complaints Vol 1

Filed under: Uncategorized — SirGolan @ 12:55 pm

I’ve heard a good amount of complaints made against Nevow, and one of them was that templates can’t include other templates. I’ve been working on a web based resource management system for MV3D this weekend and of course, it uses Nevow. I decided it would be nice to have a main layout template that included other templates for navagation and the content of the page. But that can’t be done with Nevow, right? Wrong. It’s pretty easy to include other templates. Just use rend.Fragment– it even suggests using it that way in its docstring.

Before I get in to specifics, I’d like to just comment on what I just said. Another complaint that I’ve heard about Nevow is that there is barely any documentation. In some cases, I can’t disagree– if you are talking about missing docstrings or an extensive manual. One thing I’ve learned, though, is that if you want to know how to use something, just read the code. The only Divmod code I’ve found so far that was hard to read was deep inside Axiom. Definitely part of that was because I didn’t really understand metaclasses at the time. What I’m saying is that I’ve always found the quickest way to answer a question is to just get out the code.

Anyway, rend.Fragment isn’t mentioned anywhere in the Nevow getting started manual that I know, but it took me 20 seconds of skimming rend.py to find it and even before reading the docstring, I knew it was what I was looking for. I also found that there’s a nice example of how to use it in exactly the manner I wanted in the examples folder.

What’s neat is that you can use it to cache chunks of your pages really easily. Say you have a header bar on the top of all your pages that is always the same for users who aren’t authenticated. Just make a subclass of rend.Fragment to render it and use the same instance of that everywhere. If that header changes for logged in users, then you can easily switch which rend.Fragment subclass you display in the header section depending on whether the user is logged in or not.

I could give an example usage here, but the best place to look is in Nevow/examples/fragments/.

In other news, there was something strange going on with the MV3D alpha server where it wouldn’t allow people to log in after creating an account. I’m not sure what the deal is, but somehow, I’ve fixed it. So, please give the new release a try if you haven’t already.

July 3, 2008

Release time!

Filed under: Uncategorized — SirGolan @ 12:23 am

I just released MV3D 0.32 today. It has all the features I mentioned in a previous post plus now a load tester. There’s a slight issue with the load tester, though. It can’t handle load very well. The load tester, not MV3D. I had it going with about 75 clients connected and moving around, but the load tester was going very slowly. The problem is that currently, the client has no choice on which objects it gets updates for. The server decides which objects are within view range and automatically makes sure those exist on the client. Since the load tester impersonates any number of clients (75 in the example above), that means 75 player objects get updated 75 times. I have a ticket for the next release that will stop this behavior and allow the client to choose which objects to receive updates on (of course limited to ones the server deems are in its view range). I suspect that will fix the load tester and allow it to test more effectively.

However, 75 clients connected is pretty impressive. I was even able to log in via a real graphical client and walk around a bit. It got choppy now and then, but for the most part, other than a low framerate (30-50ish), it was pretty responsive. Impressive considering there were three apps running on my system that wanted 100% of a single core, and I only have 2 cores. I didn’t end up making any changes based on the load testing I did. That was mostly due to not being able to stress the server enough to make it matter.

My next task is to figure out what the 0.34 and 0.36 releases mean. Now that I have persistence solved for the moment, there are two areas that need attention. One is the world editor, and the other is load balancing and redundancy. My current thinking is that since load balancing and redundancy worked previously and the framework for them is still in place, it should be ok to focus on world building first. I figure that there won’t be any need for load balancing if no one can make worlds other than the test world. In any case, it’s time to do the ticket shuffle.

June 14, 2008

Better late than never.

Filed under: Uncategorized — SirGolan @ 12:25 pm

It took a bit longer than expected, but MV3D version 0.32 is almost ready to go out. At this time, I’m looking for people to test and report any bugs you may find. I’ve upgraded the alpha server (alpha.mv3d.com) to this version and have it up and running now. You can grab the client and server from SVN if you’d like. Note that a new Media package is available in the downloads section of the site. If you are running Windows and just want to try out the client, I have a zip file with everything you need (including media) located here:

http://download.mv3d.com/MV3D-0.32-rc2-WindowsClient.zip

You may also require:

http://download.mv3d.com/vcredist_x86.exe (Visual C++ Redistributable)
http://download.mv3d.com/dxwebsetup.exe (Direct X)

Before logging in to the alpha server, you will have to create yourself an account using the login server:

https://login.mv3d.com:8080/

I hope to wrap up this release soon.

New features in this release:

  • Graphics
    • Trees
    • Grass
    • Clouds
    • Multiple player models
  • Basis for the new in game editor
  • Server
    • Axiom based persistence engine
    • Proper logging
    • SSH REPL for admin/debugging
  • In Game
    • Character generator including model selection
    • Chat works
    • Enhanced biped physics
    • Working water physics

Here’s a couple of screenshots:

May 31, 2008

WIN!

Filed under: Uncategorized — SirGolan @ 12:17 am

We have a WINNAR here. I did what I was talking about in my last post and made a way to keep track of values that changed often and to just automatically save them every once in a while. Basically, I made an Axiom Item for storing Vectors and Quaternions, then I created a new type of attribute for the MV3D datastore, which specifies that the value of that attribute is to be stored via an Axiom Item class. Since the only place that position/rotation is stored for objects is in the ODE body, I also made that attribute type able to use a getter/setter function to get/set the value to store.

Then from there, I added another option to them to specify that you want the attribute persisted automatically at regular intervals. If that’s on, it gets added to a list that the datastore keeps and every 15 seconds (currently), it stores the value again. It also automatically stores the value when the object it’s in is stored or when the store is closed. This seems to work great. It gave me a scare when I first got it working though–

Synced 480 autoStore objects in 19.34 seconds

Yeah, that was bad. But then I realized that I didn’t have it running in a transaction. I also added the smarts to it so that it doesn’t store a value if it hasn’t changed. That leads me to:

2008-05-31 00:09:43-0400 [-] Synced 6480 autoStore objects in 0.09 seconds

That was with not much changing.. Here’s two with about 800 objects all moving:

2008-05-31 00:00:04-0400 [-] Synced 9720 autoStore objects in 0.23 seconds
2008-05-31 00:00:17-0400 [-] Synced 9720 autoStore objects in 0.28 seconds

Those are all perfectly fine and don’t interrupt the server enough for it to be noticed (especially the 0.09 second value– the server only updates every 0.05 seconds).

There are a couple of bugs that I noticed, though. When the octree based area saves itself (which happens when something passes between nodes), it takes a while. Most likely, it’s saving way more data than it needs to, though. Also, (and this seems like it was a bug before I started messing with the datastores) after loading up a world, the physics is a little off from what the client expects. My theory is that there is some value for biped movement speed that isn’t being persisted or something isn’t being put in the right bucket when loaded.

All in all, though, I’m feeling pretty good about this.

May 28, 2008

FAIL

Filed under: Uncategorized — SirGolan @ 9:59 pm

Ok, so I made a good attempt at a store that allows you to queue objects up to be written to it asynchronously. There were a few tricky spots such as getting queries to use the correct version of the data. Basically, the store had a global revision that would get increased every time you called “sync”. The item in the store with the greatest id that was less than or equal to the store revision was considered to be the most accurate. After every sync, it would clean up old items.

Everything worked great except for the speed. Storing to an empty store took no time at all, but after that, even queuing an item to be stored started to take time. That meant that storing about 500 items took 2 seconds. Then the sync afterward took forever. The first time, it was about 40 seconds, then 120 other times. I’d be ok with the asynchronous part of things taking a bit longer, but 120 seconds was way too long. I’m scrapping this idea.

I started to think about the data I’m storing more closely though and have some ideas. Really, the only things that change very often are the physics attributes. Position, velocity, etc. I bet that I could store those directly to axiom pretty darn quickly. The fun part of those values is that they aren’t attributes of the objects that are being stored. They are attributes of the ODE objects. Currently, when storing them, a trigger gets called that plops them into the containing object as things like “initial_velocity”.

One thing I may try is making those things python properties that map to get/set on on the ODE object and then telling the datastore that they are values that rapidly change. The store could then add them to a big list of similar things that could be persisted at a separate time than the object as a whole (though persisting the object should also persist those values immediately). Also, I wonder if they should also be stored as a normal Axiom Item. That could make updating them easier.

So, this is my next thing to try. I’m feeling pretty positive about it, but I was feeling the same way about my last attempt as well.

May 26, 2008

Fun with Axiom

Filed under: Uncategorized — SirGolan @ 8:57 pm

So, I’ve migrated all of MV3D to using a new type of datastore that has Axiom as its back end. It’s pretty sweet and really wasn’t a lot of code. The interface is the same as the other MV3D datastores, so it was pretty much a drop in replacement. It is very fast and uses transactions for extra speed. I haven’t compared, but I think it’s slightly faster than my SyncFile datastore, which is what everything in MV3D used except the Sim service. The Sim service has the unique requirement that the data it needs to save changes 20x per second. In practical use situations, it’ll also have tons of data to store. Clearly, saving 20x per second isn’t an option, so instead, it saves everything every so often. One problem with doing that is that if you have 500 objects in an area, you have to save all 500 in the same physics iteration or else your simulation is out of whack when you load it. This makes for a pretty tough challenge, and I’d venture a guess that it’s one of the big reasons why MMORPGs don’t have completely dynamic worlds like MV3D.

The solution I came up with before Axiom was a memory based datastore that could sync to disk in its own thread. What would happen is that it would store all the objects in a big python dictionary in memory synchronously. With 700 objects, this takes a little less than 1/2 a second on my system. Then, it stores that dictionary to a flat file in a thread. This means that the simulation is only interrupted for 1/2 a second. That’s still not perfect, but I figured it may work to cheat a little bit and have say 1/2 of the objects be from t, and the other from t+1. That way, it’d only be 1/4 of a second hold up.

Unfortunately, I haven’t come up with a way to do something similar with the Axiom based store. Right now, it starts a big transaction and stores all the objects using Axiom which takes about 5 seconds. Having everything stopped for 5 seconds on the server is definitely not something I can do.

The best idea I’ve come up with so far is to grab all the attributes of the objects I need to save synchronously (which should take < 0.5 seconds) and then stuff them into a cooperator that stores a few at a time through Axiom. My only fear with that approach is that it’ll take a lot more time to store everything. Actually, this could be done in a nice way. I could make a new Axiom based datastore that stores things to memory initially, but when you call it’s ’sync’ method, it iterates through all the things you’ve stored and saves them through Axiom.

Of course, there is a relatively big issue with that approach. There is no way to do everything in a single transaction, so if something went wrong while it was half way through storing in Axiom, it can’t be rolled back. I could use a trick I learned from a DBA at my job, though. Add an extra attribute that signifies that an object in the database is active, and set that attribute on all the objects if everything succeeds. I’m not sure that would save me any time, but what I can do is to add an attribute for revision number (or timestamp) which would be the same for all objects and have a separate Axiom object that specifies the most recent revision that was successfully stored.

April 22, 2008

Sleep? What?

Filed under: Uncategorized — SirGolan @ 1:32 am

It’s really odd to be just heading to bed when my wife’s alarm clock is going off. She’s had early morning schedules and I’ve had late nights trying to just get everything done.

I’ve been making slow progress with MV3D, but still progress. One person has been helping me out with the visual side of things, which is great. He’s gotten a few PC models in place that are borrowed from Ogre. One of them is a dragon model which I’d like to make fly around (instead of walk) because it’d just be helpful in testing to be able to fly around the map. I’ve added a very simple character generator that now allows you to select which PC model you want to use. So, next release, I’ll set up the demo server with no PCs on it and let people create their own account/pc.

I’ve done a few other minor tweaks and stuff, but what I really want to ramble about is the new world editor. I have a few vague ideas, but nothing too concrete. I know that I want it to be completely plug-in based. It also needs to adhere to the MV3D security model, which I don’t believe the old one did. Mostly I’m thinking that it may not offer the user the same set of plugins depending on the area/realm/server they’re on.

That may sound strange, but when’s the last time you needed to adjust terrain– in space. It also has to be fairly security conscious, so sim services may want to have a list of editor plugins that are allowed. Maybe realms/areas/objects/bodies provide a lists of editor plugins that work on the thing.

So, a body may have move, scale, modify material. An object would have name, stats, add/remove body. Areas would have add/remove/select object/link (to another area). Realms would have settings for weather, add/remove/select area.

It would be nice if the different services running could add types to that. So, running a realm service would give you the realm editor, running a sim service would give you area/object/body editor. NPC services could let users add and edit NPCs in game.

Maybe this should be broader in scope than a world editor. Maybe it should really be a server control panel type thing. Don’t know. Have to think more about it, but I want to get this started.

Though, now, it is time for sleep.

March 31, 2008

Video

Filed under: Uncategorized — SirGolan @ 9:58 pm

I’ve been working on this for quite some time, and I decided that the 0.3 release was a good time to go back and finally finish it up. Here’s a video of MV3D:

Note that it shows features that are not in V0.3, but are in trunk (or have been temporarily disabled like the in game editor).

March 28, 2008

This makes no sense.

Filed under: Uncategorized — SirGolan @ 8:41 pm

So, I was browsing around on some forums for an MMORPG “construction kit” and saw someone from that company making a post about how they currently don’t support collision detection at all on the server, and they don’t have any plans for it other than some very basic ability for game designers to check if two bounding boxes are intersecting. That wasn’t planned until at least two releases out, and this product is >1.0. Anyway, from what the guy said, it sounded like they did all their collision detection on the client. What an amazing idea! Surely no one would ever want to make a client hack to let you walk through walls. Inconceivable! Just makes me wonder what other decisions they’ve passed off to the client.

I’m not sure if I ever mentioned it here, but Second Life does physics / collisions similar to how MV3D does. The physics engine runs on the server (though possibly not on the client, which accounts for some amount of chunkiness, I’d guess).

In other news, here’s a screenshot of trees that I’ve put back into MV3D. This time in a non hacky manner, and you can’t walk through them on the client OR the server.

Up next is putting grass back in. Hopefully it won’t slow everything down this time.

« Older PostsNewer Posts »

Powered by WordPress