Discussions

News: Article: Distributed Computing Economics

  1. Article: Distributed Computing Economics (32 messages)

    How do economic considerations play a role in designing distributed systems? How do you decide when to move the data to the computation, or the computation to the data?These are some the questions Jim Gray, director of Microsoft's Bay Area Research Lab, discusses in a new ClusterComputing.org article:

    Distributed Computing Economics.

    Here's an excerpt:<br>
    Computing economics are changing. Today there is rough price parity between (1) one database access, (2) ten bytes of network traffic, (3) 100,000 instructions, (4) 10 bytes of disk storage, and (5) a megabyte of disk bandwidth. This has implications for how one structures Internet-scale distributed computing: one puts computing as close to the data as possible in order to avoid expensive network traffic.

    Threaded Messages (32)

  2. It's an oddly written paper, but I do agree with the conclusion: Put the computation near the data. That's why stored procedures are faster than pulling the data a row at a time into an entity EJB and working on it there, and why caching makes such a difference when the business logic is in the application tier.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  3. Cameron, I agree with you about stored procedures. I sometimes recommend stored procedures as spot optimizations, but rarely as a general purpose architecture. It's a difficult way to build software, but the performance benefits cannot be ignored.

    Other interesting points in the article:

    - Outsourcing is hard. The IBM on-demand strategy is a gamble that IBM can efficiently do what most others have failed to do: provide an outsourcing model with predictable and economical costs. They may well pull it off.

    - Management is hard. Hardware costs are no longer the overriding costs of doing computing. The most successful Internet companies know how to manage their systems better than the rest of us. That's also the Microsoft achiles heel, so far, on the server tier.

    Take care.
    -bt
  4. very interesting[ Go to top ]

    This is an excellent analysis. Makes an excellent point about grid computing, that computational grids make more sense than storage/data grids.

    Utility based computing isn't a very good business model, many business applications are not compute intensive. So, whats the future of IBM's OnDemand strategy if there isn't much utility accessing applications on demand.
  5. very interesting[ Go to top ]

    Utility based computing isn't a very good business model, many business applications are not compute intensive.

    Sure, CPU is seldom the bottleneck for business software. Grid is more aimed at e-science, where number crunching never stops. Of course e-science is both academic and commercial. Interestingly the dominant grid platform (Globus on Linux) is freeware. And while grids tend to be viewed as a way to recycle wasted capacity, one consultancy foresees it stimulating sales:

    "By 2005, Grid Technology Partners expects the overall grid computing market to grow as high as $4.1 billion. It is probable that this number includes hardware and software elements from all layers of the grid architecture."
    -- http://www.lightreading.com/document.asp?doc_id=33405&page_number=5
  6. very interesting[ Go to top ]

    Brian,

    FWIW - we see the most demand in financial applications, that gobble unbelievable amounts of memory and CPU power ... and have problems sets that theoretically will scale well into the thousands of CPUs in a grid. I'm talking about existing C/C++ applications that run today on large Unix systems and being moved to Java grid architectures.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  7. very interesting[ Go to top ]

    \Cameron\
    FWIW - we see the most demand in financial applications, that gobble unbelievable amounts of memory and CPU power ... and have problems sets that theoretically will scale well into the thousands of CPUs in a grid. I'm talking about existing C/C++ applications that run today on large Unix systems and being moved to Java grid architectures.
    \Cameron\

    Sounds like someone that's heard of fixed income pricing models, black shoals, monte carlo, and other black magic subjects :-)

        -Mike
  8. ka-ching[ Go to top ]

    FWIW - we see the most demand in financial applications, that gobble unbelievable amounts of memory and CPU power and have problems sets that theoretically will scale well into the thousands of CPUs in a grid.

    Fascinating stuff. I once worked on stochastic software for project portfolio analysis, so I've an inkling what you mean about number-crunching scalability in finance.

    When you say "a grid", are you refering to a mere cluster united by NFS/J2EE/Tangosol? Or do you truly mean fabric with metacomputing system software, such as Globus?
  9. ka-ching[ Go to top ]

    Several of our customers use various grid management packages, such as Platform Computing (I think that's the name?!?!) and obviously they use Tangosol Coherence for the state management / event flow etc.

    Because of the nature of the data, they don't farm it out to PCs in the company, they actually run it in a cluster in a datacenter. It's a fascinating topic, because you end up with blade frames with very high speed interconnects (e.g. 80 blades with 2x CPUs each on very high speed proprietary back plane) and then you connect the blade frames with relatively "low speed" ;-) standard 10Gbps connections ... so we're actually working to make the clustering "topology aware" so it avoids using the "slow" 10Gpbs connections when possible.

    The amazing thing is that a single blade frame will only run you $1MM or so pretty well decked out, and it is (for many tasks) several times faster than a $10MM high-CPU-count box that uses twice as much floor space. The difference is that you have to "emulate" shared memory with a cluster, while with a big Unix box you have a real (and simple) single memory space (albeit typically NUMA). (Some companies run stuff on high end Unix boxes, with many JVMs clustered on just one box, but they architect that way so they are able to move to commodity servers later, should they need to, or in some cases, if they are allowed to ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  10. ka-ching[ Go to top ]

    The difference is that you have to "emulate" shared memory with a cluster...

    I was at Super Computer 2002, and it seemed to me that, sadly, most grid applications use MPI for scalability rather than emulating shared memory. Too bad there's no obvious way to share a Java object amongst several JVMs. Tuple spaces are a nifty alternative, but they're based on object serialization, which is slow.

    I think even emulated shared memory layered above message passing would be faster than a tuple space. And shared core, even NUMA as you mention, would surely scream.

    So has Sun's Grid Engine worked out some way for JVMs to share Java objects in memory? I haven't seen anything like that on Globus. But then come to think of it, most grid number-crunching doesn't use Java anyway. Maybe Java should stick to coarse grained communication, as with J2EE or Jini. Hmmm.
  11. ka-ching[ Go to top ]

    The real problem is object input and output streams. They're dogs. Like very slow dogs with bad hearts and clogged veins with their fur covered with frozen molasses. You know it's bad when you profile Java on the network and the serialization times are several times as bad as the network latencies.

    That's why Coherence supports several serialization optimizations, and it's not an unusual approach. On the other hand, I haven't written object graph serialization before, so maybe it's just hard and naturally slow. (?)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  12. ka-ching[ Go to top ]

    Part of it is the cost of reflection on Serializable, part of it is the need to constantly flush the internal cache of written objects. If you ensure all your objects implement Externalizable and hand code the serialization, things can be sped up by quite a bit. But even then, the OOS has to do alot of instanceof operators to figure out what it's dealing with, and you still have to deal with cache flushing at the end. I understand the Java 1.4 OOS/OIS implementation is quicker and gives finer control, but I haven't had the opportunity to look into it in detail myself.

    Certainly writing out bytes yourself is usually faster, since you're hard coding the read/write code for a fixed format. OOS/OIS is generalized, and you most definitely pay for the generality.

        -Mike
  13. ka-ching[ Go to top ]

    Certainly writing out bytes yourself is usually faster, since you're hard coding the read/write code for a fixed format.

    Yet another nifty trick, but still burdened by excessive copying. ForTran and C/++ can readily map objects into memory. Java hasn't figured out how to do this. And this fundamentally hurts Java's scalability for structured numerics, whether the memory is physically shared core, pages virtually shared by emulation, or buffers exchanged via MPI.

    Sure, java.nio can map an array of ints or doubles, but it can't map a mixed structure of ints and doubles. When it comes to sharing structured data, it seems to me that Java simply isn't ready to go beyond coarse grained communication. The irony is that even though Java, unlike C/++, actually went to the trouble to standardize the layout of primitives types in bytes, JVMs still can't share mixed structures of primitives.
  14. ka-ching[ Go to top ]

    \Brian Miller\
    Yet another nifty trick, but still burdened by excessive copying. ForTran and C/++ can readily map objects into memory. Java hasn't figured out how to do this. And this fundamentally hurts Java's scalability for structured numerics, whether the memory is physically shared core, pages virtually shared by emulation, or buffers exchanged via MPI.
    \Brian Miller\

    No, C/C++ cannot readily map objects into memory. Yes, you can cast to a struct - and break when you upgrade compilers, change compiler switches, or try to interchange with another compiler or OS. In practice the vast majority of C/C++ code which needs to "talk" with the outside world does it a field at a time, dealing with byte ordering issues, struct holes, etc by not trying to use structs, but instead converting each field out of a (char *). The minute you try to use just structs your environment and data store becomes _incredibly_ fragile.

    \Brian Miller\
    Sure, java.nio can map an array of ints or doubles, but it can't map a mixed structure of ints and doubles. When it comes to sharing structured data, it seems to me that Java simply isn't ready to go beyond coarse grained communication. The irony is that even though Java, unlike C/++, actually went to the trouble to standardize the layout of primitives types in bytes, JVMs still can't share mixed structures of primitives.
    \Brian Miller\

    Few Java applications would benefit from being able to map char/byte/int/float/double/boolean alone. Very, very few. If nothing else, you pretty much have to have Java Strings, and that one requirement alone will blow a hole into any mapping scheme like you're talking about.

        -Mike
  15. ka-ching[ Go to top ]

    No, C/C++ cannot readily map objects into memory. Yes, you can cast to a struct - and break when you upgrade compilers, change compiler switches, or try to interchange with another compiler or OS.

    Cameron described Sun's superclusters. SGI just sold Oak Ridge "the first Linux cluster that scales up to 64 processors within each node and the first cluster ever to allow global shared memory access across nodes". Clearly industry wants shared memory. When it comes to homogeneous clustering such as that, C/++ and ForTran can exploit shared memory in a straightforward fashion. I kinda hope Sun's Grid Engine folks are trying the same for Java, but Google reports nothing tangible.

    If nothing else, you pretty much have to have Java Strings...

    Maybe object sharing is important, in which case Java and multiprocessing needs more explanation than I've been casually able to Google -- especially garbage collection of a shared heap. As if compaction of a shared heap weren't seemingly enough of a headache, then there's likely also the matter of keeping each node's working set of pages small during heap management. Would shared heap be feasible?
  16. ka-ching[ Go to top ]

    I think some of these ideas need to be challenged a little. With objects, is page sharing (mapping pages in and out of various servers' physical address spaces using a "shared memory over the network" approach) the right approach? It was hard enough with relatively well-architected solutions in the C/C++ world (and I should add Smalltalk here as well).

    With my admittedly limited experience on the topic, I thus far prefer solutions that are "smalltalk like" in their approach, in that the "messages" are separated from the "objects". The more you can leave the "objects" where they are and move the "messages", the more scalable these systems appear to be, at least for the cases that I have seen. (We use the term "events" instead of "messages", but the idea is the same. It seems, since objects have interdependencies that tend to localize, and are typically "heavier" than the data that drives the events, that instead of trying to move the objects around to work on them, it is better to move the event to where the object lives.)

    Besides, from what I've seen, if you are trying to scale, you have to have a way to partition the load across the servers, and if the load is represented by events, and events gravitate to their target objects, and the objects are well-partitioned, then you can accomplish reasonably good scale with this approach. OTOH, if you move the objects to where the processing is occurring, you can quickly end up with a single-threaded cluster or grid, which isn't much of a cluster / grid at all.

    As a disclaimer, each problem set is different; there are many classes of complex problems that cannot be solved more efficiently in a distributed environment, and even some problems that are best solved on a single fast CPU.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  17. ka-ching[ Go to top ]

    Um, your arguments are rather all over the map on this one. You're arguing about excessive copying and homegenous shared memory clusters and layouts of primitives. It's a dizzying array of very different concepts!

    The fact is - C, C++, Fortran, or Java - 99.9% of what people do is homogenous. Various systems are intercommunicating. Data _must_ be translated between systems - you simply cannot use it in "raw" form, not even in C (the old cast-to-struct trick breaks constantly in reality). I will acknowledge that no one will be writing a shared-memory cluster virtual memory system in Java anytime soon. But for normal applications, even some very unusual, very highly distributed applications, the problems you're pointing out are not the main bottlenecks at all.

        -Mike
  18. ka-ching[ Go to top ]

    <cameron>
    That's why Coherence supports several serialization optimizations, and it's not an unusual approach. On the other hand, I haven't written object graph serialization before, so maybe it's just hard and naturally slow. (?)
    </cameron>

    There are multiple strategies available to deal with it. In our product we produce 40 times (4000%) faster serialization/deserialization in out metadata service comparing to standard Java/.NET serialization routines.

    Not that we are smarter that Sun/MS’s guys, but this is a broad problem that can be approached from many different directions.

    Regrads,
    Nikita Ivanov.
    Fitech Labs.
  19. ka-ching[ Go to top ]

    \Nikita Ivanov\
    There are multiple strategies available to deal with it. In our product we produce 40 times (4000%) faster serialization/deserialization in out metadata service comparing to standard Java/.NET serialization routines.
    \Nikita Ivanov\

    Is this compared to plain old Serializable objects, or Externalizable objects?

        -Mike
  20. ka-ching[ Go to top ]

    <mike>
    Is this compared to plain old Serializable objects, or Externalizable objects?
    </mike>
    Serialization that is.

    Regards,
    Nikita Ivanov.
    Fitech Labs.
  21. ProActive[ Go to top ]

    For this area check out:

    one of Open Source project from ObjectWeb (www.objectweb.org): ProActive
    http://www-sop.inria.fr/oasis/ProActive/

    <ProActive>
    ProActive is a Java library (Source code under LGPL licence) for parallel, distributed, and concurrent computing, also featuring mobility and security in a uniform framework. With a reduced set of simple primitives, ProActive provides a comprehensive API allowing to simplify the programming of applications that are distributed on Local Area Network (LAN), on cluster of workstations, or on Internet Grids.
    -> Interfaced with Globus, Jini, LSF, rsh, ssh, rlogin, etc.
    </ProActive>

    Lofi.
    http://openuss.sourceforge.net
  22. very interesting[ Go to top ]

    FWIW - we see the most demand in financial applications, that gobble unbelievable amounts of memory and CPU power ... and have problems sets that theoretically will scale well into the thousands of CPUs in a grid. I'm talking about existing C/C++ applications that run today on large Unix systems and being moved to Java grid architectures.


    Do you think this would apply to industries like telecom where they still do billing cycles? Of course this might just be solved with proper use of current technology (JMS/...).
  23. It's an oddly written paper, but I do agree with the conclusion: Put the computation near the data.

    That begs code mobility rather than data transmission, and that favors portable executables, as with Java or Python.

    That's why stored procedures are faster than pulling the data a row at a time into an entity EJB...


    A J2EE standard for stored procedures would be nice.
  24. It's an oddly written paper, but I do agree with the conclusion: Put the computation near the data.

    >
    > That begs code mobility rather than data transmission, and that favors portable executables, as with Java or Python.
    >
    > That's why stored procedures are faster than pulling the data a row at a time into an entity EJB...
    >
    > A J2EE standard for stored procedures would be nice.

    there already is an ANSI SQL-1999 (SQL3 or SQL-99) "SQLJ Part 1" (a.k.a. "SQL Routines Using Java") and all majors database vendors furnish both a proprietary language for stored procedures (such as Oracle's PL/SQL) and Java Stored PROcedures -- check out Oracle's implementation @
    http://otn.oracle.com/tech/java/jsp/content.html
    and read the related white paper "Unleash the Power of Java Stored PRocedures"

    Kuassi
  25. That begs code mobility rather than data transmission, and that favors portable executables, as with Java or Python.


    That doesn't seem to be where the author is going--he seems to be arguing for service-oriented architectures rather than for portable agents.
  26. Hi Cameron,

    "Put the computation near the data"

    Agreed.

    "That's why stored procedures are faster than pulling the data a row at a time into an entity EJB and working on it there, and why caching makes such a difference when the business logic is in the application tier."

    I am not going to debate the caching aspect, ;-), but can we agree on typically data characteristics of "EJB" applications. More "read-only" access of operational data, "read-only" access of reference data and in general the data read is presented to the client with hardly the same level of aggregation that this report talks about.

    Where there is aggregation the detail (rows of data) tends to be displayed alongside it.

    "read-only" because is all depends on the context - time frame,operation/usage etcs.

    So with that in mind, does what the well respected Mr Gray have to say, that relevant to the solutions we (the general J2EE community) are building these days. If I could aggregate and filter data at the database using Stored Procedures I probably would, of course after considering many other boring issues, but I do not always find this to be an option. What is retrieved from the database is what is displayed.

    One last thought how many EJB systems pull data from various datasources via RDBMS and MSGings backends. In the finanical sector you will notice that data that requires business logic to be executed against it, gets pulled from the most unusual places and can have various computation dependencies. So if I put all the code in the backends I might now need to transmit data back and forth between them. Architects need to strike a balance between data, code and execution mobility.

    All the best,

    William Louth
    JDBInsight Product Architect
    www.jinspired.com
  27. "Mobile threads"[ Go to top ]

    One of the more interesting scenarios is when the code needs to perform calculations on data stored at a few different locations and when those calculations depend one upon another. Having a software agent do this solves good part of the problem. Having a continuations based software agent is even more interesting as it allows you to preserve the state of your thread as soon as you finish with one location, "serialize" your thread, move it onto the next server and continue computation on that other data source. Java does not offer ability to "serialize" threads, however, package that my company developed makes it super easy.

    You can find more information on "threads serialization" on my blog at
    www.freeroller.net/page/alexkrut or
    www.velare.com/cgi/alexkrut-blosxom.cgi

    Cheers,
    alex krut
  28. Interesting article... I am just glad to observe always growing interest in area of grid computing. Having attended several grid computing conferences I have witnessed a vast variety of testimonies for and against grid technology. I guess by now industry has agreed that storage grids hold almost no commercial value. WAN grids, albeit being useful for academic purposes, can hardly ever be utilized by enterprises due to security issues of resource sharing and passing sensitive information across the WAN to be processed at almost unknown to the company endpoint.

    In my opinion the future of grid computing lies within clustered enterprise grids, i.e. companies utilizing all unused desktops standing around or servers during off-peak hours to perform time-constraining CPU-intensive calculations. This could be a financial company performing portfolio analysis at night before the next trading day starts, or a biotech company analyzing DNA-related data. Within enterprise, issues of resource sharing and security take a totally different meaning from WAN grids and the idea of utilizing existing CPU power opposed to purchasing more hardware can always be sold to any cost-sensitive business.

    Regards,
    Dmitriy Setrakyan
    Fitech Labs, Inc.
    xNova™ - Service Oriented Technology for Java and .NET
  29. WAN grids, albeit being useful for academic purposes, can hardly ever be utilized by enterprises due to security issues of resource sharing and passing sensitive information across the WAN to be processed at almost unknown to the company endpoint.

    You're describing a public grid, which is something quite different from a virtual organization. The Globus Security Infrastructure makes single sign-on for a WAN of trusted sites amply safe.

    In my opinion the future of grid computing lies within clustered enterprise grids, i.e. companies utilizing all unused desktops standing around or servers during off-peak hours to perform time-constraining CPU-intensive calculations.

    Of course. But that isn't a grid. You describe a private resource pool to be scheduled, ala Condor.
  30. WAN grids, albeit being useful for academic purposes, can hardly ever be utilized by enterprises due to security issues of resource sharing and passing sensitive information across the WAN to be processed at almost unknown to the company endpoint.

    <Brian>
    You're describing a public grid, which is something quite different from a virtual organization. The Globus Security Infrastructure makes single sign-on for a WAN of trusted sites amply safe.


    I think I failed to convey my point. It is a known fact that you can implement secure protocol for WAN grids. What I was relating to is that companies will be reluctant to allow remote (across WAN) computers to work with sensitive data. Moreover, sometimes remote computational tasks will need access to within-company resources and businesses will rarely allow such access from outside the firewall.


    In my opinion the future of grid computing lies within clustered enterprise grids, i.e. companies utilizing all unused desktops standing around or servers during off-peak hours to perform time-constraining CPU-intensive calculations.

    <Brian>
    Of course. But that isn't a grid. You describe a private resource pool to be scheduled, ala Condor.


    There is no precise definition of what a grid is and limiting grid to WAN is somewhat impractical. Grid computing is an ability to distribute a task among multiple resources and ability to aggregate results upon task completion. Enterprise clustered grid very well falls into this category. Notion of enterprise grid (or cluster grid) is nothing new. Sun has been pushing the idea of enterprise grid for quite some time with their Sun ONE Grid Engine product. There are also other companies that already have or plan to implement enterprise grid which shows that industry has no doubts about commercial growth potential for enterprise grid computing.

    Regards,
    Dmitriy Setrakyan
    Fitech Labs, Inc.
    xNova™ - Service Oriented Technology for Java and .NET
  31. There is no precise definition of what a grid is and limiting grid to WAN is somewhat impractical.

    "The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations." -- The Anatomy of The Grid, section 1, "Introduction"
  32. You caught me on a typo;-) What I meant to say is: "There is no unique definition of what a grid is".

    Here is another definition:
    "A computational grid is a hardware and software infrastructure that provides dependable consistent, pervasive, and inexpensive access to high-end computational capabilities" -- The Grid: Blueprint for a New Computing Infrastructure", section 2.1.2, "Definition of Computational Grids".

    I can find many other definitions. Without playing word games, the point I am trying to convey is that enterprise clustered grid is most prominent area for commercial development of grid computing technology and is no less of a grid than academically used WAN grids.

    Kind regards,
    Dmitriy Setrakyan
    Fitech Labs, Inc.
    xNova™ - Service Oriented Technology for Java and .NET
  33. I've always felt skeptical about this "computing on demand" stuff. Looks like someone has articulated the problems far better than I could.