OIL 1.0 Released: A Queue API

Discussions

News: OIL 1.0 Released: A Queue API

  1. OIL 1.0 Released: A Queue API (11 messages)

    OIL (Objects in a Line) provides a queue API and its implementation which is required by many server applications which stores client requests permanently in order. OIL provides:

        * Queue which provides sequential access.
        * Index which provides random access based on java.util.Map

    With OIL, you can:

        * Random-access queues: Useful for
              o Canceling scheduled requests
              o Searching queue items by specific key
        * Embed persistent object store:
              o The default implementation provides thread-safe WAL (Write-Ahead Logging) persistance with high performance.
              o You can create your own implementation.

    Check OIL at http://gleamynode.net/dev/projects/oil/

    Threaded Messages (11)

  2. And it'd be great if the queued element can be prioritized :)
  3. OIL is concurrent capable and thread-safe of course. But there is no transaction support such as isolation level, commit and rollback.

    Prioritization is not yet supported, but I believe I can implement it in the future.
  4. I looked at the source. It appears to be a cross between a devolved set of Java Collections interfaces and a devolved set of JMS interfaces, and includes a sample implementation. I have some major concerns with it:

    - It fails to implement either standard API (Collections, JMS) that it somewhat exposes.

    - It hardly seems appropriate to "inline" a custom "recoverable" database (i.e. persistent queue) implementation into a simple queueing project.

    - IMHO By calling it recoverable, you're making a promise that you can't actually keep. The term "recoverable" has a specific meaning when it comes to message stores.

    - Since it's Apache licensed, why not use one of the Apache database implementations? Why pretend to build a recoverable database?

    - Why not build off of ActiveMQ or OpenMQ or something like that? Or at least use one of the existing logs (Howl? Pyralog?) used by an open source tx manager.

    Honestly, I don't mean to sound negative .. it's obvious that someone put a lot of work into this with the best of intentions. However, by inlining common functionality, the resulting software reliability is dramatically dimished; and by failing to use standard interfaces, the package as a whole represents entropy, which increases the cost of building and maintaining software.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  5. Hi,

    Actually I didn't know the existance of Hown or Pyralog at the moment of implementation, so I had to create my own, and I admit it was 'inlining'. But this API and its implementation was being used by my colleagues and company project, so I could not abandon this right now.

    <quote>IMHO By calling it recoverable, you're making a promise that you can't actually keep. The term "recoverable" has a specific meaning when it comes to message stores.</quote>

    I might be missing something here. Could you explain a bit more or give me the URL related with it?

    <quote>It fails to implement either standard API (Collections, JMS) that it somewhat exposes.</quote>

    The intention of OIL API is somewhat different from JMS. Let's assume you wrote a messaging application which have to wait for a reply (e.g. the result of request) which comes in a random order and random time (e.g. within 24 hrs). You'll have to remember what you have sent to other components until the reply comes in. I thought JMS API doesn't handle this kind of situation. JMS just passes the received messages to user applications and that's all; it doesn't provide any user space to store them in the long term. And, OIL does that for those kinds of applications (e.g. SMS sender).

    Please let me know what I'm thinking about JMS is wrong. Actually I'm not a JMS professional; I've just read Oreilly JMS book once.

    And I forgot to say OIL could work as a backing storage of JMS implementations in the future. Currently it can't because it lacks of transaction, hierarchy, and priority.

    Thank you for having interests on OIL and nice critiques most of all. I appreciate that. :)
  6. Hi Trustin,
    Hi,Actually I didn't know the existance of Hown or Pyralog at the moment of implementation, so I had to create my own, and I admit it was 'inlining'.

    I totally understand .. it's hard to keep track of (or even find or know what to look for) everything that is available. HOWL is an ObjectWeb project at http://howl.objectweb.org/ and PyraLog is at http://www.jroller.com/page/pyrasun/20040331 (etc.) There's also something called JOTM .. try Google for these.
    JMS just passes the received messages to user applications and that's all; it doesn't provide any user space to store them in the long term.

    Better to _extend_ than come out with an entirely different API .. just my $.02 :-)
    Thank you for having interests on OIL and nice critiques most of all. I appreciate that. :)

    N/P .. I'm glad you understand that I meant my comments to be constructive.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  7. OIL 1.0 Released: A Queue API[ Go to top ]

    I can not agree with you. Why are you asking "why to develop"? It's already developed and released
    as open source, free of charge software library, and at least because of that it has right to live.
    Another point is why to use J2EE all around the world? Of course J2EE is a standard, but don't we
    need any alternatives? For example, I am writing not too simple but limited in size application that must have queuing capability. Do I need to use ActiveMQ or JbossMQ in this case? I say no. I need a really lightweight
    option without tens of megabytes of dependencies, JAAS authentication, transaction manager and so on.
    And in this case I prefer to choose projects like OIL rather than ActiveMQ and others. In applications like this many cute J2EE features are becoming an unusual overhead, not advantage...

    I want to express my gratitude to Trustin for his work (he has really interesting projects,
    for example "tl-netty").

    I think we really need choices, thank you!

    Best regards
    Kamil Shamgunov
    EISST LTD
    www.eisst.com
  8. OIL 1.0 Released: A Queue API[ Go to top ]

    I can not agree with you. Why are you asking "why to develop"? It's already developed and released as open source, free of charge software library, and at least because of that it has right to live.

    Yes, of course. Everything, good or bad, has its right to exist. Especially if it already exists, or is free. How can one argue with such a truism? ;-)
    Another point is why to use J2EE all around the world? Of course J2EE is a standard, but don't we need any alternatives?

    Not what I meant, at all. I like the ideas in the project, but I am a sworn enemy of entropy in software, and new APIs that look kind of like other APIs represent unnecessary entropy, so I am forced to speak my mind.
    For example, I am writing not too simple but limited in size application that must have queuing capability. Do I need to use ActiveMQ or JbossMQ in this case? I say no. I need a really lightweight option without tens of megabytes of dependencies, JAAS authentication, transaction manager and so on.

    Well, 95% of the time that people make these claims, they are simply giving their preference and not their requirement .. and the other 5% it is actually size-limited (e.g. J2ME stuff.)

    When you get in your car to go to the store, do you rip out the passenger seat and the back seat because you are driving by yourself? I think the "lightweight hysteria" has done its damage already and it's time to move on to the next well-intentioned but poorly delineated idea ;-)
    I think we really need choices, thank you!

    Of course. But the desire for choice should not be the end of constructive criticism. Otherwise we might as well just embrace every bad idea and good idea equally, as they represent choice, right?

    My comments were never intended to dispirit Trustin .. rather, I'd hope that whatever kernel of truth there is in what I said will affect his future design choices, and those of anyone else reading this.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  9. Thank you Kamil first of all. I'm so happy that Netty helped you. Please look at SEDA project in ASF, too. I'm working on it now, and it will replace Netty in the near future.

    I want to explain why I had to reinvent similar API to Cameron. I wanted a queue which provides an random access which takes O(1) time. Providing random access was not so easy because I had to point the exact item in the queue even if the content of queue is removed or realigned by expansion. Java Collections API provides random access by integer-based position, but it doesn't work at all in the case I mentioned. Please let me know if you know how to locate items in a queue in an easy way when one or more items in it is removed.

    Plus, I think similar-but-different API is not always bad. We all are accustomed to Collections API. I modified it, but it is still easy to learn. It could lower the learning curve like the case of JDOM. But it is my fault that no easy way to interface with standard API is provided.

    Cheers,
    Trustin Lee
  10. Thank you, Trustin! Of course I'll look at SEDA, as far as I get a little time for it, could you, please, shortly outline differences between SEDA and Netty2?
  11. You'll get an information about that here:

    http://www.theserverside.com/news/thread.tss?thread_id=30170

    Thanks. :)
  12. OIL 1.0 Released: A Queue API[ Go to top ]

    I like the ideas in the project, but I am a sworn enemy of entropy in software,
    >and new APIs that look kind of like other APIs represent unnecessary entropy,
    >so I am forced to speak my mind.

    As far as I understand you, there is some reference point where software
    with correct API ends and software with incorrect API starts? Where can I
    find more info about it? :) My experience shows that changing API
    is not always bad. And I have examples, we have JDO and EJB entity beans standards,
    but there is Hibernate, that have different but "kind of" API. Are you still gonna be the
    sworn enemy of it? Many applications have their own APIs that kind of other APIs(JGroups, Struts, WebWork, Ubik, etc,etc.)

    >When you get in your car to go to the store, do you rip out the passenger seat and the back seat
    >because you are driving by yourself? I think the "lightweight hysteria" has done its damage
    >already and it's time to move on to the next well-intentioned but poorly delineated idea ;-)

    Of course, but I'll never take my gardener to the store, not because he is bad gardener. He
    is great gardener, but I don't need his functionality there. :)

    >Well, 95% of the time that people make these claims, they are simply giving their preference and
    >not their requirement.. and the other 5% it is actually size-limited (e.g. J2ME stuff.)

    Where people get this numbers to make these claims? :) And in case of preference, is it so bad?
    Preferences are moving business. I think that not too much people buy tracks to go to work,
    because they PREFER a little bit smaller and prettier cars, that feet their PREFERENCES in design,
    color and technical solutions.

    Best regards
    Kamil Shamgunov
    EISST LTD
    www.eisst.com