HiveMind: What's it all about?

Discussions

News: HiveMind: What's it all about?

  1. HiveMind: What's it all about? (29 messages)

    HiveMind is a project that helps implement systems as small, testable services which interact with one another to make up an application. In this article, James Carman explores some of the key features of HiveMind by developing a user registration service to be used by an online application.

    Read: HiveMind: What's it all about?

    Threaded Messages (29)

  2. I know this is quibbling, but I named the project HiveMind, with a captial M. And it looks like TechTarget finally realized I no longer need admin access, so I can't fix it myself!
  3. Need example Tapestry + Hivemind[ Go to top ]

    I am using Tapestry for my project about BPM for China Telecom. I need help for integraing hivemind. I known that the Tapestry alpa 1 have released, but not example. need help thanks!
  4. HiveMind: What's it all about?[ Go to top ]

    Looks nice, but I notice that you require separate declarations for both your interface and your implementation of a given service. I am used to Spring, where the interface declaration is implicit.

    What benefits do you gain from requiring an explicit interface declaration for every service in the app?
  5. program to Interface[ Go to top ]

    Looks nice, but I notice that you require separate declarations for both your interface and your implementation of a given service. I am used to Spring, where the interface declaration is implicit.What benefits do you gain from requiring an explicit interface declaration for every service in the app?

    program to Interface not class.
  6. program to Interface[ Go to top ]

    program to Interface not class.

    Yes, I think he realizes that, but the question is not why to use interfaces in the first place, but rather why they have to be explicitly declared to HiveMind.
  7. HiveMind: What's it all about?[ Go to top ]

    Looks nice, but I notice that you require separate declarations for both your interface and your implementation of a given service. I am used to Spring, where the interface declaration is implicit.What benefits do you gain from requiring an explicit interface declaration for every service in the app?

    I found that odd, too; the ability to seperate the service-point from its implementation is not widely used in HiveMind. It's good to demonstrate that it can be done, but not something you do every day ... generally, you define the service point and provide the implementation (<invoke-factory>, <create-instance>, <interceptor>) within the <service-point>. James didn't really mention the idea of modules, central to HiveMind, which is where you might use the longer version: service-point in one module, implementation in another, as a kind of plug in.

    HiveMind 1.0 is very dogmatic that services must have an interface. HiveMind 1.1 is less so, services can have an interface that is a JavaBean class instead. Internally, HiveMind actually builds a synthetic interface from the public methods of the class (and the interfaces implemented by the class). I don't use this approach, but it can be done.

    1.1 also allows you to abbreviate class names, and omit the interface when its name matches the service id (very common), and a bunch of other ways to reduce the XML clutter.

    In terms of testing ... having a seperate interface and implementation is a god send. The unit tests for HiveMind get over 91% code coverage without breaking a sweat ... because we can easily plug mock objects (via EasyMock) into our code to run sophisicated tests.
  8. I didn't want to confuse things by showing the two mechanisms for defining a service's implementation. For consistency, I like to use implementation blocks as opposed to in-lining the implementation. That's just my personal preference, though.

    One benefit of HiveMind being interface-based is that you can declare exactly how you want the outside world to see your service implementation logic, the service interface. With Spring (I like Spring, so this is not a knock), client code can access the bean via any interface that it implements. This could be dangerous if the bean implements some lifecycle interface and a client calls the destroy() method for instance. Since HiveMind actually hands you a lightweight proxy that ONLY implements the service interface, you can't downcast the service object to another interface and start calling methods on it.
  9. Not sure of the benefits.[ Go to top ]

    Great article. I loaded up HiveMind and had a play.

    I realise that in the IoC world there are great gains to having services available via lookup, effectively providing an appropriate service at runtime. But I can't see the benefit in configuring the services as jar files. Isn't changing an application's libraries effectively the same as a code change? With the HiveMind approach wouldn't you be expected to retest and redeploy your application every time you changed what service configurations were to be classloaded? If that's the case, you might well just hardcode your services instead of hardcoding the configuration file that references the services.

    This isn't a knock, it's a request for more information.

    Cheers
    Jem
  10. Levels of indirection[ Go to top ]

    One of the key concepts behind dependency injection is setting up appropriate levels of indirection. Of course you'll want to test your system with all the different service implementations that you plan to support, but by isolating that service-level configuration to a dependency injection area of your source (be it XML, Java, properties file, etc.), you achieve a comfortable level of indirection. This lets you change things in one place for different configurations (either statically or dynamically, at the user's behest).

    If your code is sprinkled with 'new NYSEOrderProcessingImpl()', things get a bit trickier to reconfigure when it comes to reusing the code for NASDAQ as well as NYSE.

    The IoC folks often beat up the service-lookup approach to things, and in particular give JNDI lookups a hard time. But if you think about it, a JNDI lookup is in a sense very similar from an indirection standpoint. Essentially, IoC frameworks give you a means to inject directly into your business code, whereas JNDI gives you a means to inject into a shared namespace, from which the business code can look up the (indirected) dependency. What the IoC frameworks give you that JNDI doesn't is simplicity -- no PortableRemoteObject, no JNDI string memorization, etc.

    And, as with many types of third-party frameworks, the various dependency injection frameworks out there also support random goodies that you probably wouldn't code up yourself, such as per-instance injection and varying levels of isolation between different contexts.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  11. Thanks Patrick[ Go to top ]

    Right! So the primary benefit is flexibility at runtime or deploytime, not, as I had originally thought, during development. Thanks.
  12. Thanks Patrick[ Go to top ]

    ... and also during test time, as it makes it trivial to swap out different implementations (including broken ones) for different parts.

    For most apps, test time is often the most common (if not the only) place where changes of configuration happen.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  13. Re: Not sure of the benefits.[ Go to top ]

    Thank you! What a lot of folks have pointed out here and I conceded in my article is that I neglected to include information (for brevity) about one of HiveMind's most important features, configuration points. With configuration points, the module discovery mechanism really shines. New "contributions" to the configuration point can be picked up dynamically at runtime (think of plugins) merely by including a new JAR file containing a module descriptor file in the classpath. Of course, there are also other ways in which you can provide module descriptors to the registry (HiveMind 1.1 includes the ModuleDescriptorProvider interface). This is not to say that it's useless to "hard-code" the services in the configuration file. Structuring your implementation classes so that their dependencies can be injected into them (and using interfaces) greatly improves the testability of your code. Using the configuration file is an easy way to tell HiveMind how to "weave together" your implementation objects so that they can collaborate with one another. Also, HiveMind's smart proxies allow other features that would be difficult to hard-code yourself.

    Maybe I'll see if I can write a follow-up article showing off some of the more advanced features of HiveMind. Again, thank you for your kind words.
  14. New wheel, old wheel[ Go to top ]

    A lot of people have mentioned here that Spring is a framework built on top of an IoC container for integrating with third-party application services. While that definition is true, Spring is very carefully packaged so the IoC container can be used standalone, and it represents a very tight, defect-free, dependency injection container.

    As I see the latest Tapestry release held up by a desire for a more stable hiveMind release (http://www.javablogs.com/Jump.action?id=208999), I begin to wonder what the wisdom is here? Again, why reinvent?
    ...HiveMind's most important features, configuration points. With configuration points, the module discovery mechanism really shines. New "contributions" to the configuration point can be picked up dynamically at runtime (think of plugins) merely by including a new JAR file containing a module descriptor file in the classpath.

    Spring does the same thing. It can auto-discover bean context xml in pluggable ways by searching the classpath for jar files with the appropriate descriptors. I haven't heard any capabilities mentioned that HiveMind has that Spring doesn't.
  15. New wheel, old wheel[ Go to top ]

    The "stable HiveMind release" is just about Tapestry CVS head using some features of HiveMind CVS head. Before we can release a new alpha of Tapestry, we need the new alpha of HiveMind so that end users can build the Tapestry source they download.

    It is very strange that you feel that Spring offers everything HiveMind does, when the Spring developers I've talked to personally know that not to be the case. Spring is a solid IoC container at the center a large collection of libraries and solutions -- and that's great, I wouldn't want to try and compete with all that. HiveMind is another solid IoC container that includes very sophisticated configuration approach and the ability to represent services in configurations, leading to very powerful data-driven solutions. The modules machanism means that extensibility of core services and configurations is an inherent feature of the framework, something that's critical to Tapestry, allowing end users to customize the behavior of the framework in precise ways.
  16. Re: New wheel, old wheel[ Go to top ]

    Configuration points are something Spring does not have and HiveMind does. The dynamic discovery is something that can be done in Spring quite easily and isn't exactly a novel technology, but it does prove to be quite useful.
  17. Service Lifecycle Models[ Go to top ]

    I thought that the "Service Lifecycle Models" features seemed the most interesting. It's something that I've wanted for a while.

    Q1: Is this feature unique to HiveMind?

    Q2: Does Spring have something similar?

    Dave Ford
    http://www.smart-soft.com
  18. I think Catalin's circuit analogy clearly states the concern that I've had for a while with IoC. That is, with more and more fine grained services, the application slowly starts moving out of Java and into very complex, vendor-specific XML files.

    I have come to appreciate the compile time checking that languages like Java give me. Even more, I like the assistance my Java IDE (IntelliJ) gives me. Also, the fact that I know Java very well doesn't hurt. I don't see the point of using XML files for everything. I don't mind XML (or text file configuration), if there is a need for it. But I don't see the need most of time.

    Take Hibernate, for example. In Hibernate's config file, when I type this:

    <property name="

    and then press Ctrl-Space in my IDE, I get *nothing*. I don't see why Hibernate would even use XML for this. What's the point? Do they really think some non-programmer is going to need to tweak these properties? I would prefer something like this:

    1. I type: cfg.setD + Ctrl-Space

    2. My IDE then completes the method name: cfg.setDialect(

    3. I type My + Ctrl-Space

    4. My IDE fills in the rest: cfg.setDialect(MysqlDialect.class);

    There are so many advantages to this type of configuration (over XML):

    1. The obvious InteliSence IDE completion.
    2. The refactoring features of my IDE do a much better job than with XML.
    3. I can use things like inheritance and composition in Java.
    4. My App Server detects changes in class files

    I've seen dozens of projects that invent their own XML dialect. Then that dialect slowly evolves into a "programming language" by adding loops, condionals, variable declarations, etc. Maven even has an XML dialect that supports inheritance. I'm all for domain specific languages - but only where appropriate.

    So, in summary, I think dependency injection is a great way to build big services by wiring together small ones, but if the wireing becomes a large part of the app, why not leave it in Java? As just one possible example, you could do something like this:

    public interface ProjectXyzAbstractConfigSpec{
        Class<? extends DataSource> getDataSourceImplClass();
        Class<? extends IMyService> getMyServiceImplClass();
    }

    public interface ProjectXyzConfigImpl implements ProjectXyzAbstractConfigSpec{
        @singleton Class<? extends DataSource> getDataSourceImplClass(){ return MysqlDataSource.class; }
        @pooled Class<? extends IMyService> getMyServiceImplClass() { return MyServiceImpl.class; }
    }
  19. Dave,

    As for your Hibernate gripe, have you tried XDoclet? Most folks don't actually type their Hibernate config files.

    Now for your ProjectXyzConfigImpl example, how would you actually use this class (I assume you meant for it to be a class and not an interface, since you provided implementations of the methods) in a project?

    Do you agree that IoC is a useful paradigm? Do you just not agree with how most IoC containers have gone about implementing automatic IoC? Of course, you can implement IoC "by hand", but the idea is to automate it somehow, thereby eliminating the burden. Are you saying that you would like to provide some descriptor object to the IoC container which describes all of the services and their implementations (a la EJB 1.0)? I'm not saying any of this is wrong, per se. I'm just trying to pick your (or some other reader's) brain. In fact, the HiveMind team (at least some of us) is trying to come up with an easy way to build HiveMind registries using Java code.
  20. Do you agree that IoC
    > is a useful paradigm?
    Yes. I like (and use) IoC. The part I don't get is why the "wiring" needs to be in a different language. I saw a long discussion on the HiveMind web site about what language to use for the wiring. (http://wiki.apache.org/jakarta-hivemind/NotXMLProposal) But they never mention using *Java* for the wiring. In Pico container, there are no XML files - just Java. Here is an example:

    MutablePicoContainer reg = new DefaultPicoContainer();
    reg.registerComponentImplementation(MyTransformer.class);
    //Note: MyTransformer's constructor takes: (DataSource ds, Foo f)
    reg.registerComponentImplementation(MysqlDataSource.class);
    reg.registerComponentImplementation(Foo.class);
    //Note: Foo's constructor takes: (DataSource ds)
            
    Foo foo = (Foo) reg.getComponentInstance(Foo.class);
    Transformer transformer = (Transformer) reg.getComponentInstanceOfType(Transformer.class);

    I don't see the value of forcing XML for this type of thing. It's a co-mingling of concerns. "Dependency Injection" is one concern. And "externalizing configuration" is a different concern. There already exist dozens of tools to turn xml into java objects. And dozens more to define xml schemas and validate. If one wants to move large chucks of their app into a config file, that's fine. But it really should be a separate layer. Martin Fowler has a blog on the same issue:

    http://www.martinfowler.com/articles/injection.html#CodeOrConfigurationFiles

    To be honest, this is a general pet peeve of mine and it's not specific to HiveMind. Spring has the same issues.

    > the HiveMind team (at least some of us)
    > is trying to come up with an easy way to
    > build HiveMind registries using Java code.
    Yes. This is what I would like. In fact, I'm surprised that wasn't the first step. Phase one should be to populate the registry programmatically. Then, *on top of that*, add an XML layer or groovy-builder layer or YAML layer, etc.

    > Now for your ProjectXyzConfigImpl
    > example, how would you actually
    > use this class?
    The ProjectXyzConfigImpl serves the same purpose as the first 4 lines of the Pico example above. That is to populate the registry. The AbstractConfig and ConfigImpl is my (lame??) attempt to "come up with an easy way to build the registry using Java".

    The AbstractConfigSpec is analogous to "service-point" tags:

    public interface ProjectXyzAbstractConfigSpec{
        Class<? extends DataSource> getDataSourceImplClass();
        Class<? extends IMyService> getMyServiceImplClass();
    }

    The ConfigImpl is analogous to the Hive minds "implementation" tags

    public class ProjectXyzConfigImpl implements ProjectXyzAbstractConfigSpec{
        @singleton Class<? extends DataSource> getDataSourceImplClass(){ return MysqlDataSource.class; }
        @pooled Class<? extends IMyService> getMyServiceImplClass() { return MyServiceImpl.class; }
    }

    You would use it like this:

    Registry reg = new Registry();
    reg.add(new ProjectXyzConfigImpl());

    DataSource ds = (DataSource) reg.get(DataSource.class)

    > As for your Hibernate gripe,
    > have you tried XDoclet?
    Yes. But, I would rather use:
    cfg.setDialect(MysqlDialect.class);

    It's much easier and type safe. Especially for IDE users. Also, XDoclet tags (or Annotations) are more appropriate to meta-data that is associated with a Class, Field or Method. But *dialect* isn't associated with a Class, Field or Method. Dialect is associated with a SessionFactory instance.
  21. Dave,

    I see your (and Mr. Fowler's) point about config files vs. code. However, HiveMind does provide a way to set up your registry "by hand." Instead of a ProjectXyzAbstractConfigSpec class, you might have a ProjectXyzModuleDescriptorProvider class...

    RegistryBuilder builder = new RegistryBuilder();
    builder.addModuleDescriptorProvider( new ProjectXyzModuleDescriptorProvider() );
    builder.addDefaultModuleDescriptorProvider();
    Registry registry = builder.constructRegistry( Locale.getDefaultLocale() );

    The problem is that we need to come up with a framework that makes it a bit easier to write a ModuleDescriptorProvider. The only implementing class included with HiveMind now is the XmlModuleDescriptorProvider, which is used to parse XML descriptor files. The ModuleDescriptorProvider idea was introduced in HiveMind 1.1 and we (actually Knut, I believe) adapted the previous XML parsing logic into the new ModuleDescriptorProvider paradigm. Perhaps we should come up with an AbstractModuleDescriptorProvider which would facilitate many of the activities that you need to do in order to build a ModuleDescriptor. Or, perhaps adding some shortcut methods to the descriptor classes might help things a bit. I don't know. As I said, we're actively trying to come up with ways to make it easier.
  22. Service Lifecycle Models[ Go to top ]

    I thought that the "Service Lifecycle Models" features seemed the most interesting. It's something that I've wanted for a while. Q1: Is this feature unique to HiveMind? Q2: Does Spring have something similar?Dave Fordhttp://www.smart-soft.com

    I believe it does, but it requires a bit more XML. I don't know how Spring handles end-of-request cleanup notification. I don't know if Spring has a pooled model (for expensive to initialize services).

    It is really quite powerful to know that the service model is enapsulated inside the proxy; you code can freely mix references to pure singletons, and per-thread singletons, and they all look the same (and all work properly at runtime) ... at least, until your code mistakenly stores some thread-specific information obtained from a threaded service as if it was truly global.

    I often combine a threaded service model with the hivemind.lib.ServicePropertyFactory. This factory creates a services that is a projection of a property of another service.

    So, for instance, the RequestGlobals service is threaded, and has a request property of type HttpServletRequest.

    I can then generate an HttpServletRequest service as the projection of RequestGlobals.request.

    I can then inject HttpServletRequest into my other services. An method invokations on this proxy will ultimately be invoked on the HttpServletRequest instance stored inside RequestGlobals.

    Again, easy testing, less code.
  23. OK, I use Spring. Quite like it, don't quite trust autowiring because I like to see how my code works, but I'm getting there.

    Anyway: config files bloat in spring. The solution is to be clever about the partitions within the system and to layer config seperatly from wiring - lets the support guys have very simple configs to play with.

    The example in the article ends up using AOP, a large config file, seperation of functionality into numerous subclasses, many with interfaces. A hibernate config requiring a factory and a session, both explicit in the xml.

    A traditional OO java approach would simply have DAO, inline SQL using autocommit jdbc. Maybe 3 classes would do the trick, with a 3 line property file for DB config (location, user, passwd).

    The technology may be useful, but the example is not appropriate. It bloats a very simple problem into a coding and deployment nightmare.

    Be nice to see a more pragmatic example, one which competes with the traditional approach in terms of simplicity of development and support. If that is possible - and I'm not convinced that it is.

    Jonathan
  24. Is this the ultimate in bloat?[ Go to top ]

    OK, I use Spring. Quite like it, don't quite trust autowiring because I like to see how my code works, but I'm getting there.Anyway: config files bloat in spring. The solution is to be clever about the partitions within the system and to layer config seperatly from wiring - lets the support guys have very simple configs to play with.The example in the article ends up using AOP, a large config file, seperation of functionality into numerous subclasses, many with interfaces.

    That's the basic approach I take. I've got a config file for application configuration, for example, a temp directory or some such. I have one for the presentation, middle, and persistence tiers. I have one for security tier and one for the roles information(if the application in question uses roles stored in XML as opposed to the database). We have to integrate with a content managment system, so we have one for that module. We have one for the notification system.

    Now, bear in mind, before spring we still had all of these,but some used different code to read and process them. With Spring, we've (almost) brought them all together under one consisten umbrella. There is one remaining module that will require substantial recoding fix its dependencies and this is because this particular module needs to be revamped anyway. But since it works pretty well, it is not a high priority.

    But as I said, now most of our config files look and act the same. I think it has worked quite well.
  25. My questions about HiveMind[ Go to top ]

    It seems it is a mini-scale SOA framework based on IOC.
    It seems to adopt the SOA approach. But this service is at inner-application level. To achieve independency, it use interface (service-point) declaration and use IOC to establish the relationship in service/implementation.
    this approach is questionable:
    1. To what extent can this mini-scale SOA framework be used?
    2. Why to use HiveMind instead of using IOC framework directly?
  26. Echo some other's sentiments[ Go to top ]

    As a user of Spring, I read the article trying to discern differences between the two products. I suppose the only thing that I could pick out (of the article) that HiveMind does much differently than Spring is the concept of a Threaded model in the service lifecycle.

    I would assume that HiveMind has many more features/capabilities than touched upon for the article, since nearly every other IoC container provides the "key features" discussed in the article. What has been mentioned so far would comprise only one chapter in Spring's very large table of contents.

    So what are the key differentiators? Why reinvent the wheel?
  27. Echo some other's sentiments[ Go to top ]

    I don't know spring very much, but I would summarize my opinion like this :
      although the two frameworks share some common technologies ( IOC, services models, xml configuration, AOP capabilities ...) the main differences is in the way you use them.

       I think Spring is a framework that you can leverage to quickly hook your application into other existing frameworks/libraries. One of the key notions - AFAIK - is the notion of template.

       On the other hand, Hivemind is much more about instantiating, configuring and making cooperate services - or components if you prefer the name.

      One of the thing I like very much in Hivemind are the contributions points. If you have a look at the Eclipse plugins system, you would find that there is the notion of extension points which are very similar. But Hivemind goes farther, and this is one of it's strengths. You can use a self describing schema to validate the contributions, you can use simple transformations rules to adapt the contributions. And this let you change the behaviour of your application just by changing the configuration.

      I don't think Hivemind reinvent the wheel here, this is a matter of what you want to do. If you want to use quickly a lot of other libraries, then Spring seems better. But if you want to build a homemade suite of reusable components, then Hivemind seems better.

    There is also a complete comparison between the two frameworks on Howard's blog :

    http://javatapestry.blogspot.com/archives/2004_02_01_javatapestry_archive.html#107757766902671389

    cheers.
  28. Hivemind / Spring / JNDI[ Go to top ]

    Hivemind is a pure IoC container. It's focused on one thing: being a container.

    Spring is a application framework built on a IoC container. It's a larger effort with a intentionally larger scope: it can serve as a one-stop shop for app development (addressing nearly the entire Java/J2EE stack), or you can choose to use it's container in standalone. You use what you need.

    The two products can actually be used in combination. If a shop was already using Hivemind for IoC, they could still leverage Spring's rich J2EE support libraries. You could use both a Hivemind registry and a Spring bean factory to host services or modules within the same app if you had a good reason to.

    Long before I was a Spring Developer, I used Spring because mainly for the synergy I and my team experienced using it. I knew I could turn to Spring and its huge community for an answer on just about anything.

    Just one further, unrelated note on Patrick's JNDI comment: a container like Hivemind or Spring can easily leverage JNDI lookups without exposing client services to JNDI lookup code. In this case, a Factory is responsible for encapsulating the lookup and the container is responsible for handing the retrieved object to clients that need it. You get much more flexibility that way, and all the power of JNDI.

    Keith
  29. Hivemind / Spring / JNDI[ Go to top ]

    You use what you need.

    Hey!! That's our line :-p
  30. Likewise, in HiveMind, we can set to task a good number of simple services, and by letting them collaborate and share information, they can produce the desired application behavior. But at its heart, HiveMind is about the simplicity that comes from breaking complex problems into small pieces -- the heart of Object Oriented development.
    This IOC approach is very tempting, and really great for tests, and enforces good programming style.
    But I'm wondering, taking it too far (too fine-grained granularity) couldn't be equally dangerous?
    If we break a system into lots a services, we would move the entire complexity of the application into wireing those services. The services themselves are very simple and easy to test, but for the entire application we have a different situation. And debugging the wireing in a complex application might not be a trivial task, especially when you have decalative wireing, so you will have to enter in the guts of the framework.
    Taking to the extreme, imagine an electronic circuit, where you have only 3 simple components: transistors, resistors and capacitors. While testing these components is very easy, testing a circuit with a dozen of these is not quite trivial.
    These in an extreme scenario, of course, but I fear that taking this approach too far may be dangerous, and IOC concept, although a very good concept, should not be considered a general programming technique, but a useful technique when used with moderation.