Home

News: High Performance Computing Panel from TSSJS 2007

  1. Nati Shalom of Gigaspaces, Bob Lozano of Appistry, Gil Tene of Azul, John Davies of IONA and Kirk Pepperdine of TSS discussed high performance computing on a panel at TheServerSide Symposium 2007 in Las Vegas. This is a video of that panel. One question posed to the panel questioned J2EE's usefulness in high performance computing, sort of:
    Moderator: But one last question before I turn it over to the audience. In fact it is two questions, I will combine them into one. ... Basically, I think all of us certainly here have seen sort of the end - you can see the end of J2EE, and I am claiming you can start to see the end of relational databases and all of this stuff as well because we are starting to be able to do this in memory... What do you think? Answer either way. We have got J2EE - what are your opinions about the design? How much does J2EE come into this [high performance computing, enabled by grids] and databases? Bob Lozano: In particular with J2EE I think that a little bit of what Nati [Shalom] was saying was that it builds in, you really think about the designs that where it came from right, and it builds in a relatively small cluster notion. It builds in a lot of the infrastructure. I mean it came infrastructure up and so I think building... one of the things I like about what is happening with Spring and other abstractions is that you really can use the opportunity to separate out the infrastructure assumptions from what your code is supposed to be doing, I mean the truth is, in a lot of applications, the responsibility to have to manage messaging, to have to manage failure, and manage scale, and do all that from the infrastructure up is what keeps them from scaling, it is what makes them cranky and hard to operate, it is what makes you have to stay up late at night wondering what is going to happen. And so I think that if the middleware, if you will, or the infrastructure abstraction can become a simple one instead of thinking of all the moving parts, instead of thinking of what might break, what might not, if you can just think of the logic you are trying to do, independent, and assume that the infrastructure will successfully execute it, and that if you need more capacity you can grow it with whatever box size you want to grow it with, so on so forth, that I think will be a very thrilling assumption. So, as a consequence I think that the days of a big heavy app server though they’re there for to continue to operate things built that way, but it is not going to be the real enablement of a much lower cost or more reliable infrastructure, it just cannot.
    Of course, one has to remember that that doesn't mean J2EE is inappropriate, just not quite the right tool for "high performance computing" as defined by the panel. Bob defined high performance computing systems as systems that "just naturally scale without limit and they can count on them." Nati Shalom said this: "I think that a lot of the ideas of high performance computing came from the computational tasks and taking a computational task and then running it faster by distributing over a large number of machines. High performance computing is actually defining a new type of architecture, building business critical application, not just the classic computational task like Monte Carlo Simulation. So, my view of high performance computing is that you can take an application and scale it out as you rightly said across multiple machines not on a physical machine with large CPUs." What do you think?
  2. What about clustering? It can basically achieve the same job for HPC and we have far less questions about its integration with J2EE. What exactly are the advantages of grid over clustering?
  3. What about clustering? It can basically achieve the same job for HPC and we have far less questions about its integration with J2EE.
    What exactly are the advantages of grid over clustering?
    They fundamentally address different types of work-load. Java EE is designed to address OLTP work-loads, while grid infrastructures have traditionally focused on job scheduling. We're starting to see "the two become one" in many cases (shared infrastructure, capacity on demand, SOA, etc.), but the architecture of Java EE is not conducive to the types of applications that we tend to see being deployed on grid architectures, and vice versa, so both are going to have to evolve in order to work together well. The work that we do within grid infrastructures is primarily highly parallel calculations, aggregations, data analysis, etc., and mostly for financial services and [I could tell you but then I'd have to kill you]. We're starting to see more and more grid-based EDA and CEP apps being built, which are basically the next level of capability bridging across XTP (eXtreme Transaction Processing) and grid, and that's a pretty exciting area. We're also seeing more and more grid apps with Java EE front ends, e.g. to provide the SOA interface stacks necessary to front-end large calculations. (The "real" front ends, i.e. the GUIs, are often still Excel. :-) Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  4. I talked about it and had an interesting discussion with Cameron about DataGrid and beyond in this thread (a few TSS posts back). It explains a bit about the position of high performance computing in the Java world (or at least my view of it). Cheers, Shay Banon System Architect at GigaSpaces Compass Founder and Lead Developer
  5. Hi Cameron, I have copied/pasted part of your reply to ask you a question: The work that we do within grid infrastructures is primarily highly parallel calculations, aggregations, data analysis, etc., and mostly for financial services and [I could tell you but then I'd have to kill you]. We're starting to see more and more grid-based EDA and CEP apps being built, which are basically the next level of capability bridging across XTP (eXtreme Transaction Processing) and grid, and that's a pretty exciting area. ------------------------------------------------------------------------------------------------------------------------------------------------------------ Do you foresee grid-based EDA and CEP architecture converging into one or CEP architecture on top of EDA ? Ingesting 100s of Terabytes a day of satellite data does not leave a choice but use CEP architecture. In the mean time I found grid-based EDA highly attractive as it, beyond being gridded, provides high-level of parallelization, which I foresee would remove a need for complex multi-threaded programming. Thanks in advance, Arkady
  6. Memory != persistent DB[ Go to top ]

    Ridiculous statement. Perhaps the context here is grid and scifi computing, but this just reeks of the pointless war Sun tried to pick with RDBMS's. Do EJBs in memory, buy our servers and application containers, and turn your back on proven database technologies. Scifi computing can rerun simulations and the like if results get corrupted in memory or one of the nodes goes down. Whatever. That is a totally different use case than transactional processing and enterprise computing.
  7. Re: Memory != persistent DB[ Go to top ]

    I don't think anyone claimed "Memory == persistent DB", a statement closer to my claim is that complex hierarchical models do not fit into traditional relational databases. At the end of a process there will nearly always be the need for persistence, it is not true however that the persistence can only be in the form of a "classic" relation database. In the past a lot of data was loaded into a relational database and the application ran on the data from the database. Of course there was no need to store the results because everything was effectively persistent already and if the system crashed half way through you always knew where you were (assuming it was reasonably well designed), safe but very slow. In the heyday of databases data models were designed around the enterprise database. With today's global businesses the standards are no longer based on just one enterprise database and as a result many of today's data models are simply too complex for the proven but now out of date relational database technology. So, databases had their time? no, relational databases on the other hand just don't fit with many of today's needs, we need a better model, something that will handle the complex hierarchical models like FpML. This isn't SciFi computing, we've been doing this for years in the banking world and I've started to see other verticals looking into the same ideas, on-line betting, telcos, government, healthcare etc. Your classic RDBMS is here to stay, as is COBOL. -John-
  8. Re: Memory != persistent DB[ Go to top ]

    This isn't SciFi computing, we've been doing this for years in the banking world
    Sure, but only for a very small subset of problems. The absolute majority of all transactions are still handled by TP monitors and RDBMS, and the absolute majority of systems under development is still using this type of setup.
  9. Depends wat J2EE mean[ Go to top ]

    Whether J2EE is suitable for high performance computing depends on what you mean by J2EE. If you mean EJBs, Web Services, JSF, JNDI then probably not, but if you mean JDBC, Servlets, JMS, RMI, CORBA/IIOP then sure it is.