Concurrency: Next New Major Language Feature?

Discussions

News: Concurrency: Next New Major Language Feature?

  1. Concurrency: Next New Major Language Feature? (31 messages)

    There has been a lot of talk around concurrency. We bow to Doug Lea for his contributions in java.util.concurrency.*, and Herb Sutter wrote an interesting piece entitled The Free Lunch Is Over. Where are we going next? Microsoft's Cw language has support for the idea of asynchronous methods and something called chords.

    Herb Sutter: The Free Lunch is Over

    Harry Pierson: Concurrency: Next New Major Language Feature?

    Threaded Messages (31)

  2. Great Article[ Go to top ]

    Herb's article is excellent reading and presents a convincing case for why exploiting concurrency is becoming so important.
  3. Great Article[ Go to top ]

    Herb's article is excellent reading and presents a convincing case for why exploiting concurrency is becoming so important.

    I thought the author was going to announce a major breakthrough in Computer Science to make general concurrent programming easy and less error prune. But we are once again disappointed.

    Nice written article, but no new substance. Period.
  4. Great Article[ Go to top ]

    I thought the author was going to announce a major breakthrough in Computer Science to make general concurrent programming easy and less error prone. But we are once again disappointed.Nice written article, but no new substance. Period.
    Well, I think increasing the awareness in the programming community on this subject is in itself of great value. My personal experience is that the percentage of programmers who automatically consider concurrency issues in their work is still fairly small, for exactly the reasons outlined in the article.
    Randy
  5. persistence is the real revolution[ Go to top ]

    Great article! I've been thinking about this for awhile now since I read that Intel et al reached the clockspeed barrier.

    I disagree with this statement by the authoer but we seem to have reached the end of the “applications are increasingly I/O-bound or network-bound or database-bound” trend I don't know about you guys, but the majority of problems I see out there are database bottlenecks.

    Even for development, I yearn for the day when my disk is not a disk, but rather solid-state RAM that is orders of magnitutude faster than current hard-disks.

    IMO, the J2EE guys have been realizing the benefits of concurrent multi-threaded programs on multi-cpu machines for years. Multi-cpu desktops are cheap. I don't think the concurrency revolution can really start until large, fast RAM-disks are integrated and available for cheap in desktops and servers.

    Bill
  6. persistence is the real revolution[ Go to top ]

    I disagree with this statement by the authoer but we seem to have reached the end of the “applications are increasingly I/O-bound or network-bound or database-bound” trend I don't know about you guys, but the majority of problems I see out there are database bottlenecks. Even for development, I yearn for the day when my disk is not a disk, but rather solid-state RAM that is orders of magnitutude faster than current hard-disks.

    I agree. More CPUs on a DBMS typically means more data to be written to the transaction log which in turn means an I/O bottleneck writing it out (not withstanding SANs with huge caches etc etc).

    More CPUs doing work against a given set of tables can also mean more contention (for locks).

    I would imagine that in most database apps today (particularly those that don't run on an DBMS that has MVCC) there is a lot of work to be done before they become CPU bound.

    Having said that, maybe this is where a DBMS like ANTS, which has a rather cunning lock free mechanism, will come to shine when compared with "traditional" RDBMSs that use locks (as we know them).

    Robert
  7. Oracle does not have a transaction manager, like MS-SQL has.
    PostgreSQL has MVCC.
    Transaction log throughput is fast given the devices match up to the CPU power - and they really should match it in any sane setup.
    See this for an explanation of locking in Oracle, and the lack of a locking manager.
    I think this may put a different light on your reasoning.
  8. persistence is the real revolution[ Go to top ]

    Bill,
    I don't know about you guys, but the majority of problems I see out there are database bottlenecks.

    From our own experience, the database is the bottleneck in over 90% of the time.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  9. Hyper-threading == Multi-cpu?[ Go to top ]

    I'm getting a laptop with a hyper-threading enabled chip next week.

    Do the concurrency problems that show up with buggy multi-threaded code show up just as well on hyper-threaded CPUs as on multi-cpu machines?

    Thanks,

    BIll
  10. Hyper-threading == Multi-cpu?[ Go to top ]

    Well, it sort of is and sort of isn't the same.

    HT is exposed as multiple CPUs at the operating system level, and you therefore have to have a multi-threaded program to take advantage of it.

    However, most of the CPU hardware (and all of the mainboard hardware) is shared between the concurrent threads of execution. This affects whole classes of concurrency-related issues. For example, since there is only one data cache you would not need to worry about the same set of memory coherence issues between processors. But there really are two independent streams of processor execution going on, while there aren't in a regular uniprocessor, so things like spin locks start to gain appeal (in limited situations of course). It's a new animal, but fortunately the set of concurrency problems is a subset.

    I feel obligated to point out that the jury is still out on hyper-threading. The whole assumption is that there are lots of unused resources in a processor during typical execution, and that those could be used by another thread. So you could run two instruction streams on the same hardware and get more speed. There are at least a few snags in the promise of HT:
    1. Both threads might be trying to use the same hardware at the same time (e.g. if both are doing lots of memory reads, all the load units will be busy), so the processor will never be quite as powerful as two independent processors. Especially for applications where all the threads heavily use the same resources.
    2. You'll have to make some concurrency changes that will add overheads.
    3. Not all CPU architectures are the same with regard to availability of free resources when executing any given program (initially written in Java, for example).
    4. Given #1, #2, and #3, to get the most benefit you'll need to be tuning your code to a specific CPU architecture. You want to make sure your threads use different stuff in the processor, and you don't want the full multi-processor concurrency overheads to be in effect.
    5. Given the complexity involved in #4, by the time you are done optimizing, the next CPU architecture will be out!
    6. HT is more limited in # of "CPUs" than dual-core and other stuff that will be coming out real soon now, or just garden variety multiprocessor systems.

    So most people will just ignore all of these details, flip on HT, and measure whether they got a big boost, little boost, nothing, or perhaps even a performance cut due to multi-cpu overheads. So far, people basically seem to be running the SMP version of code on an SMP version of the operating system that has HT activated, but is by no means optimized.

    Needless to say, I have not heard any reports of people doing this and getting the 60-80% speedups from their Pentium 4's I read about in the theoretical/academic papers from Intel in the late 90's. For many reasons.

    Most people I know simply gave up trying, though the best I've heard (in my admittedly limited experience) was a 15-20% speedup in an application that had long-running independent threads that did a full mix of memory read, computation, and memory write. And IMHO if any CPU architecture is likely to benefit from HT, the P4 (with its extremely deep pipeline, proliferation of independent hardware units, and general register shortage) is it.

    So I think we still don't know if developing HT to full potential will be worth it. If the hype about it is replaced by hype about dual-core, I think you can write it off real soon.
  11. Hyper-threading == Multi-cpu?[ Go to top ]

    Well, it sort of is and sort of isn't the same.HT is exposed as multiple CPUs at the operating system level, and you therefore have to have a multi-threaded program to take advantage of it.

    Lo, just when i thought i got some bit of concurrency control,am bitten up by some more confusion.confusion is fun.
    would be real enlightenment for me, if the following doubts are cleared.
     
    1) "HT is exposed as multiple CPUs at the operating system
       level"
        Does that mean we will have multiple operating system on a HT chip,instead of operating system handling multiple CPU?

    2) How does it affect Java threads?Is my assumption true that ,with a Java multi-threaded program,it does not matter what architecture is used under the process.In fact it is dependent on performance requirements based on the usage of the application, which is a JVM to OS mapping problem(just like currently,some JVM's map 1-to-1 to native process or maps 1-to-1 to native threads,native threads being ,Threads at process level).Essentially,what matters is if the Java application is designed for multi-threading or not. What pattern of multi-threading is used for the implementation in the underlying layers, should be configurable during deployment time based on the horizontal requirements of the application.

    Sensible?

    Regards
    Surajeet
  12. Hyper-threading == Multi-cpu?[ Go to top ]

    "HT is exposed as multiple CPUs at the operating system level" -- Does that mean we will have multiple operating system on a HT chip,instead of operating system handling multiple CPU?

    No. There is only one operating system. Even though there is only one physical CPU chip the operating system will handle it as if it were two, because it is capable of running multiple instruction streams at the same time. Applications that ask how many processors there are will get 2 for an answer. Two processes (or perhaps threads within a process) can be running at the same time, just like in a dual-cpu.
    2) How does it affect Java threads?Is my assumption true that ,with a Java multi-threaded program,it does not matter what architecture is used under the process. ... Essentially, what matters is if the Java application is designed for multi-threading or not. What pattern of multi-threading is used for the implementation in the underlying layers, should be configurable during deployment time based on the horizontal requirements of the application.

    Your assumption is sort of true. If your application is _truly_ multithreaded properly, correct function is independent of underlying architecture (as long as it is UMA, anyhow). However the exception to your assumption is that common (anti)patterns lead developers to write multithreaded Java programs that work fine on a uniprocessor but not on true multiprocessor boxes. And the concern your assumption doesn't address is whether or not your application will actually run faster or scale better.
  13. Hyper-threading == Multi-cpu?[ Go to top ]

    Hyper-threading == Multi-cpu?

    Most definitely not. All hyperthreading does is make use of some extra idle functional units on the processors.

    > Do the concurrency problems that show up with buggy multi-threaded code show up just as well on hyper-threaded CPUs as on multi-cpu machines?

    I wouldn't think so. For example, on a multi-cpu machine, you have per-cpu caches to worry about, and if you get your synchronization wrong, you can experience all sorts of issues. On a hyper-threaded CPU, I wouldn't expect to see that problem, since you don't have multiple CPU caches to synchronize.

    God bless,
    -Toby Reyelts
  14. Hyper-threading == Multi-cpu?[ Go to top ]

    I'm getting a laptop with a hyper-threading enabled chip next week. Do the concurrency problems that show up with buggy multi-threaded code show up just as well on hyper-threaded CPUs as on multi-cpu machines?Thanks,BIll

    Unfortunately hyper-threading is mostly useful for multi CPU machines only. Boxes with only 1 CPU won't notice any advantage.<br>

    The reason for this is that hyper-threading is just a sorting technology: It sorts its input so that it does all reads at once and all writes at once, so it improves the use of the bus. If there are no other CPU to take advantage of that... well, it is cool, but your laptop won't run faster with it.
  15. Hyper-threading == Multi-cpu?[ Go to top ]

    I had to disable HT (in BIOS) b/c I was getting odd behavior while debugging in Eclipse in our product. The performance didn't drop a bit (not visually, not subjectively like doing the product build).

    My 2 cents,
    Nikita.
  16. re: Hyper-threading == Multi-cpu?[ Go to top ]

    I had to disable HT (in BIOS) b/c I was getting odd behavior while debugging in Eclipse in our product. The performance didn't drop a bit (not visually, not subjectively like doing the product build).My 2 cents,Nikita.

    Nikita, are you sure that you would have experienced a major performance boost of you did a build on a 2 processor machine?

    Personally I find things like jsp compilation etc, dramatically faster on my HT laptop, but then again, thats just a subjective view, without any real value.

    Just like yours.
  17. Hyper-threading == Multi-cpu?[ Go to top ]

    Bill,
    Do the concurrency problems that show up with buggy multi-threaded code show up just as well on hyper-threaded CPUs as on multi-cpu machines?

    If the problems are within a single process, then yes it is possible. It is not as likely, though.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  18. Thanks for the insight.

    But I find it suprising that the "Distributed Shared Memory" did not find its mention as the next big thing along with threads. In fact, as we move towards a thread oriented architecture (TOA) we will need SM even more.

    Without hyping, Jini may become a realy useful technology after all.

    -Abhijit
  19. Abhijit,
    Thanks for the insight. But I find it suprising that the "Distributed Shared Memory" did not find its mention as the next big thing along with threads. In fact, as we move towards a thread oriented architecture (TOA) we will need SM even more. Without hyping, Jini may become a realy useful technology after all.

    Jini does not provide distributed shared memory, any more than RMI does.

    I'd say that distributed shared memory is not the next big thing, but one of the current big things. There are thousands of large J2EE application deployments using reliable distributed shared memory today (with products including our Tangosol Coherence).

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  20. I think concurrent programming drives people - including myself - nuts. In my ideal world I could do my Java stuff in a transaction comparable to a database transaction. This will of course will never be possible as certain things simply are not undoable.
  21. Avg Joe Java?[ Go to top ]

    What does this mean for your average journeyman developer. Do they need to learn concurrent programming? (By the way, thanks Doug Lea, the concurrency library really does make things simpler !) Or will this all be handled by containers, development tools, etc. Have we abandoned the idea of making programming simpler (i.e. concentrating on business logic) and letting the infrastructure handle the technical details?

    Just wondering....

    Gordon
  22. Avg Joe Java?[ Go to top ]

    What does this mean for your average journeyman developer. Do they need to learn concurrent programming?
    Had not Java have Thread, Runnable, synchronized, wait and sleep from day one? Despite some recent bashing of Allen Holub, his articles on threading helped me out a lot, check them out on Javaworld, this page has all the links: http://www.holub.com/publications/articles/index.html#javaworld
  23. There are other ways than concurrency to get improved performance, in general. So we might be seeing more of those too.

    Of course wastefulness was mentioned in the article, but instead of chewing on ever-more-complicated WS protocols, we could do something more binary, thus freeing networks and CPUs from a lot of burden. And we could get rid of all the adware and spam that clogs systems in so many ways.

    I/O improvements such as disks, networks, and storage/database design are still going ahead full steam (unless you are an HP customer :-P), so for some of our apps we might just sit around and wait for them to get better.

    There is also special-purpose hardware, such as video accelerators for games, that take hold in some applications.

    In fact I wonder if the slowdown in CPUs is more technological or more market-driven? Once you cut away the people who are disk-bound, network-bound, graphics-bound, or have already made their applications massively distributed for grid computing, which consumers are left to justify the huge chip design and fabrication plant outlays for getting us to 10GHz? After all, I've heard many wild corrolaries for Moore's Law. While chip complexity doubles every 18 months, does chip designer productivity double every 18 months? Or do you need more designers? (I've heard the latter...) I've also heard that fabrication equipment costs were quadrupling every two years, or something like that...

    Or maybe Intel just rolled snake eyes. They ran out of headroom in the P4 design before they expected, and uptake of the Itanium/IA64 has been slow so it's not on the graph yet.
  24. Another interesting read[ Go to top ]

    ftp://ftp.cs.wits.ac.za/pub/research/reports/TR-Wits-CS-2002-3.pdf
  25. The Next Big Thing[ Go to top ]

    I had some thoughts in this area in November 2003; I tried to post them to The Server Side but wasn't able to for some reason. Anyway, here are my original thoughts, which I have not updated.

    "
    I've had a few thoughts on the natural successor to Java and C#, which I'd like to share, to see what others think.

    ---

    In a nutshell, I think the Next Big Thing might be a return to parallelism, at the language level.

    First, some observations:

    o Hardware has been naturally parallel for many decades now.

    o We are getting closer to the limits of serial processing in silicon, we will reach the limit in less than 20 (say) years. [OK, a lot of people have said this before, but it will happen eventually.]

    o Software has been a von Neuman bottleneck for some time now.
    We are not exploiting the natural parallelism of hardware, we are not even close.

    o Hardware is becoming even more parallel, with Intel hyperthreading, and increasing numbers of execution units on chips.

    o There is a convergence of Model Driven Architecture, for automation of design-to-code, and modelling of executable Business Processes with specifications such as BPEL4WS, the convergence point is Activity Diagrams. Both are fashionable but embryonic. We are at beginning to learn how to design and code software which does more than one thing at a time, in a standard way.

    o The BPEL4WS specification shows how interfaces to Business Processes, and their implementation, can be defined in XML.

    o XML syntax is lousy for programming languages, it is way too clunky.

    o Service Oriented Architecture is growing in popularity, and has asynchronous messaging at its core.

    o The technology for vectorising loops has been well known for at least 25 years, and proven in practice many times ouver. Some of this can be done automatically by compilers, however it may require additional input from the developer. It is not fashionable at the moment.

    o We know how to design platform-independent languages, which map onto platform-independent byte code, which can be mapped efficiently to hardware using JIT and hotspot dynamic compilation techniques.

    o We know how to do continuous integration of UML and Java code for class diagrams and sequence diagrams with tools like TogetherJ.
    But not activity diagrams. Closer coupling of design and code is the future.

    o The long-running and cross-system nature of business processes requires a non-stack based, non-platform specific approach to managing process state. We know how represent this state with XML.

    o Agent echnology, which allows running code to be moved from one machine to another, has not really caught on yet, and is not in fashion. It does, however, have something in common with Business Processes – it requires the execution state to be convertible to a platform-independent format between execution steps. Again, we know how to do this with XML.

    So now for my predictions:

    o The Next Big Thing will be a language which supports fine-grained parallelism naturally for invocations, loops, array/collection/relationship/resultset processing, IO etc.

    o It will be strongly-typed, with exceptions.

    o It will not be stack-based, it can’t be.

    o It will run on current stack-based hardware.

    o There will be a parallel extension of Java – Kava? [I see from Google that there already is a Kava - whatever.]

    o The byte code for Kava will be a superset of Java byte code.

    o There will be a parallel extension of C# - C|| ?

    o The Microsoft machine-independent language for C|| will be a superset of that for C# and CLR, namely MSIL.

    o Stack-based optimisation and mapping to hardware will be performed at runtime from byte code to machine code using hotspot- type techniques.

    o Service Oriented Architecture will be driven down to a very fine grained level, with something like individual classes made available as services.

    o Activity diagrams will have many “swim lanes”, each of which may be very small, supporting many small interacting processes.

    o There will be tools like TogetherJ which allow continuous integration of design including Activity Diagrams/BPEL etc with Kava/C|| code. Both Activity Diagrams and BPEL will have to evolve.

    o There will be further blurring of the boundaries between design and development.

    o The ever-increasing natural parallelism of hardware will be better exploited, but still not fully.

    o The new languages will naturally support Agent technology. This will allow mobile code to move to where the data is for much more efficient processing. We still don’t know how to secure this, but we aren’t far off with Java sandpit and other security technology built into the JVM.

    o The code will run on many different kinds of devices including phones, cards, ubiquitous devices. Well we can do most of that already with Java – it is just a question of getting the cost down.

    o The code will support efficient processing of images, graphics, video, speech etc. using the high-level languages, using the inherent parallelism, on small devices. [Though I don't know to what extent we will still use specialised code for these applications, some commodity code will still be implemented in C etc.]

    Maybe.
    "

    IBM plans for a cell processor seem quite relevant: see http://www.theregister.com/2005/01/13/ibm_cell_chip/ for example.

    I haven't read Herb or Harry's pieces yet.

    Andrew Harris
  26. The Next Big Thing[ Go to top ]

    In technology areas where Moore's law applies it can happen in 2 ways or a combination thereof.
    1) Increased performance.
    2) Lower cost.

    My guess is development will flow in direction with least resistance, Moore's law will simply make CPU's cheaper instead of more advanced (when physics slow down MHz race and if concurency is too complicated to utilize).

    Granted Moore's law driving prices down is usually a property of young areas of technology, but I won't exclude possibility that we can return to that again when we hit physics challenges.
  27. The Next Big Thing[ Go to top ]

    One of the points I was trying to make was that we are indeed going to see an increase in the number of CPU cores, and that individual core performance will not continue to increase at the same rate. Who knows, perhaps the number of cores will double every m months.

    So to keep our new applications performing, we need to learn how to increase the amount of parallelism, in fact we need an application programming language which will facilitate this.

    There is another reason why we need better support for parallelism, namely the support for business process applications. There is also a need for the ability to serialise, persist and migrate high level processing state.

    Andrew
  28. The Next Big Thing[ Go to top ]

    Let me play devil's advocate for a second.

    Even if forward development occurs in number of cores and concurrency, maybe the average developer can still just coast along. If the DB is a bottleneck, maybe I can just throw more cores at it, and expect Oracle will do what it takes to make sure their product makes effective use of oodles of them. Maybe I'll just use someone else's libraries/components that optimize for cuncurrency and glue them in where CPU performance is critical, such as a sort algorithm that uses multi-core, or whatever; that was the strategy some people adopted for making use of SIMD instructions, as I recall. And maybe my framework will parallelize everything for me, or maybe it will just parallelize all the I/O leaving my business logic to run sequentially on a single CPU.

    After all, if my business logic usually updates a bank balance or something, a 3GHz processor that does multiple instructions per clock ought to be able to handle, oh, a billion transactions per second if it weren't for the I/O, persistence, ACID, and so on...

    But seriously, most code still follows a 90/10 rule, with very few points in the code comprising the bulk of the intensive compute work. This means performance can be "someone else's job" for lots of developers out there for a good time to come.
  29. The Next Big Thing[ Go to top ]

    Let me play devil's advocate for a second.Even if forward development occurs in number of cores and concurrency, maybe the average developer can still just coast along. If the DB is a bottleneck, maybe I can just throw more cores at it, and expect Oracle will do what it takes to make sure their product makes effective use of oodles of them.



    Seriously, I don't know about you guys, but the J2EE applications I've seen rarely peak the CPU. They're all waiting on lock contention or I/O. The thing is, there already are multi-CPU machines out there that Oracle can run upon. Multi-core isn't really gonna do anything except make it a lot cheaper to buy a multi-cpu machine.

    Bill
  30. The Next Big Thing[ Go to top ]

    Maybe one way we can better e more easily leverage the increased number of cores is using Distributed Systems paradigms within a single multicore computer, in a similar way Jini/Javaspaces framework works with miltiple networked comps. There are decades of studies and existing APIs/frameworks/patterns for distributed systems and grid computing, and some of this knowledge may help us facilitate the creation of more efficient solutions for multicore boxes.

    Regards,
    Henrique Steckelberg
  31. The Next Big Thing[ Go to top ]

    In a nutshell, I think the Next Big Thing might be a return to parallelism, at the language level.

    If so, I for myself would like that constructs like those found in the now almost forgotten OCCAM language (PAR, ALT, channels, etc) could serve as an inspiration for those upcoming changes.

    BTW, ladder diagrams are also a powerful representation for parallel computations. It has been in use for decades in the industrial control area, particulary as a programming "language" for PLCs.
  32. "There is also lock-free programming, supported most directly at the language level in Java 5..."
    Do you know anything about that?

    Thanks.