Statistics

Users
3486
Articles
156
Articles View Hits
1529693
It's been almost 3 years since we started working on this title.  Team members have come and gone, but the game still goes on.  We're getting close and everyone is getting that second wind, working crazy hours to get all the features in the game.  ADukes is from Germany and he's even staying up to be online for at least a little while when we all start our nightly work around 8-9pm EST.  I'm too tired to figure out what time that is in Germany :)

We've finally got some solid multiplayer!  The last bug was so hard to find.  Some maps ran perfectly and some would crash after a while.  We could not figure out what it could be.  It usually started with one unit being slightly out of sync on one machine.  This would seem that the math was bad, but we replaced all the math routines with fixed point so that they would be the same across different processors.  We've had this bug in there for over a year, but we just could not make the connection between what was different about the maps.  Sometimes it ran great on a map, and other times it had a bad sync on the same map!

 The test team had moved their mp focus to this last bug over the past month.  I've been adding logging, 12mb logs for 5 minutes of testing.  They would send me log files, I would track it to where I could, add more logging, new exe, more testing....  Finally the light bulb went on over my head, I swear it was there :)  I noticed in the last round of testing that the Devil's Den map kept crashing while our PPT map didn't.  I guess it was since it was Devil's Den that it finally came to me, lots of big rocks!  There was a ton of collision detection on the DD map.  Men walk around some rocks, over others, but all were collidable. 

 So figuring it was a math issue, we dumped out the locations of all the objects into the log and there it was!  Although all the calculations were the same, the starting locations of the objects were still being done in the PowerRender graphics engine which does not use fixed point math.  Why would they?  No one but this team is crazy enough to try and lockstep a game of this magnitude.  So I took a look at the code that figured out where the objects were once they were read in from the map file.  It was way about my head :)  Not a clue about transformation stuff, be nice to have a real graphics coder on this project.  I've been holding my own, but graphics coding is certainly not my specialty.  So I did the brute force approach, that I can do.  Pound a nail with a sledgehammer.  Since we have very little communication during map load, it was no problem sending the location of every object across the wire.  The server figures it all out, then sends the coordinates to all other machines.  This way I didn't have to go into PR and try to convert it's transformation matrix stuff into fixed point math.  One great thing about RakNet, which we use for our multiplayer code, is that it handles splitting up the packets if they are too big.  So I was able to just send this giant glob of floats across the wire.  Once that was in, the coordinates matched and the test team started their first night of crash free mp testing!  That was the last piece of the puzzle.  We're all feeling slightly better about getting this thing out the door.

 Fixed point match basically converts and stores all floating point numbers in integers.  It does all it's math in integer format and converts it back to float only when necessary.  This is because different processors can have different results for floating point math, which just cannot happen during lockstep multiplayer.  Which means that every client runs the game independently, but starts at the same point so that all clients are running the same exact game.  It's pretty cool and makes it so that very little data is necessary to send.  If a user clicks a button, that press is not recognized right away, but a few frames ahead instead.  First it sends the button press out to all machines and says "Hey, here's a button press, lets do it on frame 20099, ready go!"  This way all machines stay perfectly in sync.  It was a bear to get working, but now that it's in, it's great.

 Other things going on are that the test team has also started alpha scenario testing.  That's to get the big bugs out before we get to beta.  We consider alpha the point at which every feature is either stubbed or coded.  No new features will be added.  Beta is when all features are coded and the only new code that will be added is to fix a bug with a documented bug report.  That's a pain, but it works.  When we reach beta we'll bring in some new faces, there's just a time when you've looked at the same screen for so long that you can't see the obvious.  So the beta test team will get assigned specific parts of the game to test and run it through it's paces.

We're looking good for a late 2009 early 2010 release.  We did hope to get it done by the end of this year, but we're not sure when we'll get to beta.  There is still alpha scenario testing to do and that may require some AI work, which always takes too long.  Our preliminary specs are posted in our FAQ and we try to stay very active on our forums if you have questions.