Jeff Kesselman on Sun's Project Darkstar -- Game Infrastructure

Discussions

News: Jeff Kesselman on Sun's Project Darkstar -- Game Infrastructure

  1. Java.sun.com has an interesting interview with Sun's Jeff Kesselman, "chief instigator" of the recently released open source Project Darkstar. Here's the basic claim: Darkstar "offers Java technology-based infrastructure software designed for latency-critical, massively scaled online applications such as online games." Which means: "...not only will development costs go down, but the technical challenges of scaling, load balancing, data consistency, and failure recovery currently faced by developers of multiplayer games will be radically reduced or even eliminated." Kesselman claims that developers can now write big multiplayer online games without having to learn either distributed computing or multi-threaded programming. Sun hopes that by releasing software that reduces development costs, they enable more developers to enter the market for massively multiplayer online games, which they hope will in turn grow the market for their hardware and service business. Several questions: Do we really know what the market for these multiplayer games is? Does anyone have experience with Darkstar -- Sun claims that big mostly unnamed industry players are interested. Are Sun's claims credible? And what else should we be thinking about? Is there a use for something like Darkstar in applications outside of the purely gaming arena? Consider the transactional capabilities: is there a business application here?
  2. Is there really a need for this type of middleware? I'm quite skeptical. I'll believe it when it is behind the scenes of several successful projects.
  3. Absolutely there is a need for this kind of middelware. There are several companies in the gaming space claiming similar capabilities including: BigWorld, Dimps, Icarus, HeroEngine, Multiverse, others. Most of these are not Java based, so Sun is providing a particular value here to the Java community. The issue with these systems is that they have large data, are highly concurrent, are near realtime ( i.e. not deterministic, but very low latency ). Popular online games (MMOGs), are know to have millions of users and run on 1000's of CPUs. These are read/write intensive, highly concurrent....Absolutely NON TRIVIAL! Plus, the gaming industry has a very long history of developing console type games and are just now learning about what it takes to build a true horizontally scalable MMOG. The development cycle for an MMOG is 3+ years and they've only got one shot to make it right, so using a proven system will substantially reduce their risk and their development cycle. So far, none of the vendors mentioned above has deployed a successful game, so only time will tell who did a good design and implementation. It will be interesting to see how this Sun system is leveraged by the Java game developers. -Robert Versant Corporation
  4. Has Darkstar found a generic solution for EVE-scale fleet battles without lag?
  5. Hi Guys, I responded to a bunch of the very intelligent questions in the main article over in my blog at http://blogs.sun.com/gameguy Some of you have already given answers I'd give here, and done maybe better then I would, so I won't reiterate on those. On the EVE questions. Yes, and no. We aren't talking magic pixie dust. You cant just sprinkle it over an existing server architecture and have it solve your problems. Think of Project Darkstar like you think of an app server-- but one designed specifically around the constraints and requirements of massive scale real-time games. COULD you implement a combat system for something like EVE that would scale? I'm absolutely positive you could, and fairly easily. However, it would be a separate back-end from the rest of EVE unless you were to re-architect and re-implement all of it. Jeff Kesselman
  6. By the way.... None of the companies you mentioned above (BigWorld, Multiverse, etc) are really doing what we are doing with Darkstar. Those are game engine companies. We are not producing a game engine. Project Darkstar is infrastructure software on which game engines may be easily written. Think of it this way, Project Darkstar is to Big World or Multiverse what Glassfish is to JBilling.
  7. By the way....

    None of the companies you mentioned above (BigWorld, Multiverse, etc) are really doing what we are doing with Darkstar.

    Those are game engine companies.
    Hi Jeff, Well, I agree and disagree here... I agree what you are doing is different in scope ( and applicability - I'll assume Darkstar has other use besides gaming ), but those "game engines" proport to be providing what you provide + more. But...as I've suggested, the jury is not out yet, until one of them deploys a game with a few 100,000+ users, nobody will know if they are doing what you are doing ;-) Hopefully, they are making the right implementation decisions, otherwise there will be some very public, visible, crash and burns and some of those companies and their users will disappear. Cheers, -Robert Versant Corporation
  8. [quote] Well, I agree and disagree here... I agree what you are doing is different in scope ( and applicability - I'll assume Darkstar has other use besides gaming ), but those "game engines" proport to be providing what you provide + more [/quote] Not really, if you look under the hood, any more then an old style monolithic business app does what a J2EE server does. Their focus is on client and tool chain. Their server technologies are all really highly limited and built around the same old model that been in use in the MMORPG space for quite awhile-- the geogrphic distribution of load. Darkstar is a fundamental paradigm shift doing distribution by CPU usage and object access not by character location in the virtual world. (A great scheme if people were gaussian... but people are about as ungaussian as you can get.) Our achitecture gives us maximally effective use of the back end at all times regardless of where people are in the world. It also gives us automatic persistence of the entire game state and every game object which both allows us to fail-over and hide server failures as well as eliminating game "rollbacks." Finally, because we are a transactional technology, we eliminate race conditions (and deadlocks, we do deadlock avoidance). Race conditions are the primary cause of "dupe bugs." What we do is allow the game programmer to program his or her game as if it was running event-driven/monothreaded ona single core on a single machien adn we make ti scale out massively in parallel. None of this are things offered by the game engines you mention. In fact, it would benefit them to rebuild their backends on to of us and we've talked with some of them about that. Finally, a game engine contains many assumptions about the structure and kind of game your building. We do not. The architecture works as well for hundreds of thousands of people playing cards in groups of 4 as it does for tens of thousands of players all playing in the same MMO or in 64 player groups in an FPS. There are a bunch of slide presentations available at www.projectdarkstar.com. I'd recommend Jim Waldo's to start with... http://www.projectdarkstar.com/index.php?option=com_content&task=view&id=53&Itemid=34 I am going to be putting up a new talk I did at a conference a few weeks ago within the next week or two. That one really focuses on how darkstar differes from the way MMOs are built today. JK
  9. Btw.. one of our developers went to the Multiverse talk at AGC and he told me, when they got to the server design they actually said, "Now, we're not doing anything as cool as Darkstar does. " 8) JK
  10. Their server technologies are all really highly limited and built around the same old model that been in use in the MMORPG space for quite awhile
    Yes, that was exactly my point. They have excellent tooling, founded on the heritage of console gaming focus and first generation MMO's. However, none of those MMO game engine vendors are going around saying, "gee we probably don't really scale well". On the contrary, they are saying "we will scale well for all your needs". Only once purchased and pushed under load, customers discover they have a lot of re-work to do, especially in the data layer for many of the reasons you've stated in your post. Thankfully, there has been enough dialog in these areas over the last 18 months that some of the vendors are taking action to fix the problem before it's critical...or dare I say, too late. I know at least one game server which has been loaded to 160,000 concurrent users in a single world .... and a 4,000 concurrent user event in an area the size of 1 block. So, they are not all doing things the old way ;-) I look forward to poking around under the covers of Darkstar now that it's open. Cheers, -Robert Versant Corporation
  11. Other uses for Darkstar?[ Go to top ]

    This seems like a very cool piece of software, but MMOGs, as cool as they are, are still a very narrow market. I mean, I'd love to take a couple of years off, hire a dozen people and build one, but I just don't have the budget for it at the moment ;-). What I'm wondering is if this something that might be worth looking into to use on my day job. Are there other applications envisioned for the platform? Seems like you could use this as the basis for a very scalable Coherence or GigaSpaces-type of product. Anyone given any thought to that? Am I off the mark? Thanks, Bruce
  12. Re: Other uses for Darkstar?[ Go to top ]

    Hi bruce! I'm gonna be lazy and say I just adressed this somewhat in my own blog: http://blogs.sun.com/gameguy Also keep in mind that Darkstar is for more then just MMOs. We have people doing casual games and other kinds of games with it to. Game-like apps it can also potentially be a good fit for, see the above blog for more info on that and what the limits are.