News: How Do Java Developers Edit-In-Place and Monitor?

  1. PushToTest recently provided root cause analysis and mitigation services to a medium-size organization using JBoss/Seam/Hibernate/Oracle. When we arrived on the project we found a very professional team of Java developers, and a management team dedicated to improving the system and development environment. While everyone seemed reasonable and professional it was suprising how difficult it was to effect a change in the system. Two problems were apparent: - Serious issues in developer productivity caused by a lack of “edit in place” techniques. For example, a single word change took more than 10 minutes to appear. - There seemed to be no real way to monitor the actual system. For example, while they have a Nagios set-up it is mostly configured for "alerts." Tuning the development environment is a matter of optimizing the time between when a developer makes a change and they are able to see it in the runtime. The ideal development environment allows the developer to “edit in place” and see their changes immediately. This is difficult, but not impossible to achieve in an Enterprise Java environment where specific packaging is required. Many times we find developers wait a considerable amount of time between when they try to edit a file, it checks out and becomes writable, they edit the file, build and deploy. Decreasing the amount of time it takes to run the changed software on the machines or increasing the resources available on the machines directly improves productivity. Revising the configuration of the machines or upgrading the machines in terms of memory and CPU power to run the software configured on the machines increases productivity. The other thing to consider is how the developer team monitors the actual system. Our customer is currently using a Nagios set-up but this is mostly configured for “alerts.” They need something like PushToTest’s integration with Glassbox to correlate backend, frontend, and JBoss performance over time. JBoss ON, Hyperic, Wily Interscope and other solutions can be used to do this.  We do not highly recommend JBossON for this purpose, as it is basically a fork of Hyperic. You can make JBossON work with a significant effort. Wily is the gold standard here but comes with a staggering cost. What is your organization using to accomplish "edit in place" and monitoring? -Frank

    Threaded Messages (20)

  2. Edit in place is the main goal that we try to achieve within the Play framework (http://www.playframework.org). All the framework design is built around the 'fix the bug and hit reload' idea ... But I think that bad error reports are often another productivity issue in Java frameworks. Especially for source files that are automatically translated in Java like the JSP ...
  3. Edit in place[ Go to top ]

    If you want edit-in-place just get JavaRebel.2.0 allows to skip both build and deployment.
  4. What will happen if you have some object created already and it is composed object with in another object. It has some state(different values for members) right? I mean with in another object, How JavaRebel handles it?
  5. What will happen if you have some object created already and it is composed object with in another object. It has some state(different values for members) right? I mean with in another object, How JavaRebel handles it?
    JavaRebel preserves object state. If there is an instance of a class and you add a method to the class the old instance will get that new method. You do have to take under consideration that no constructor is re-invoked. So if you add a member to a class, the old instance will get the new member but if you set the member to a new value only in the constuctor then it will stay null for the old instance.
  6. I don't want to sound like a sycophant, but I have been using JavaRebel for years now, and I practically can't do any web programming without it, that's how spoiled I am. It does have minor problems (hard to read class names, restarts of the container still present, if rare) but overall, its AMAZING how much faster I code. They have a Spring plugin that works nicely with XML declared beans, but I couldn't get it to work with spring annotations. Now only if the guy that writes the Struts plugin would release a version taht works with 2.1....
  7. Re: JavaRebel[ Go to top ]

    I had different experience. I started to use JavaRebel several months ago. At first, it works great, even it takes a bit longer to load (I guess that's the time JavaRebel doing its magic). But recently, I downloaded the latest version, it takes a very long time to load (somethings even 5-10 minutes), that the time saved are pretty much waste at initial start. Since our have to restart a lot, we pretty much abandon the JavaRebel. Many of our team member had the similar experience and we decide not to use it. So edit in place is not solved for us.
  8. Re: JavaRebel[ Go to top ]

    Any open source equivalent to JavaRebel? Or atleast in development stage?
  9. Re: JavaRebel[ Go to top ]

    But recently, I downloaded the latest version, it takes a very long time to load (somethings even 5-10 minutes), that the time saved are pretty much waste at initial start.
    It's likely you hit the deadlock bug in JavaRebel 2.0. It is fixed in the nightly build.
  10. JavaRebel[ Go to top ]

    I've been using JavaRebel for about a month now. It has saved me hours of time by allowing me to make a change, reload a JSP and see the change reflected instantly.
  11. Re: Edit in place[ Go to top ]

    If you want edit-in-place just get JavaRebel.2.0 allows to skip both build and deployment.
  12. Prefer TDD[ Go to top ]

    I guess it depends on your work habits. I do like "edit-in-place" for visual layout changes, but for component changes I think creating a unit test for the bug and working against that (test driven style) is far more efficient than even the fastest "fix-publish-hand test" cycle. JavaRebel is very cool if you do want this kind of feature, I'm not sure how well it works with Seam right now.
  13. I don't know if I'm being snarky or if I am just misunderstanding the author, but fixing a bug should just be a matter of: 1. Writing a unit test, prove the bug 2. Fix bug until unit test passes 3. (optional refactor if necessary) 4. Run suite of unit tests It kinda sounds like the author is looking for some kind of dynamic language like Ruby, Groovy, PHP or something similar, where you edit files in-place in production to fix an issue. And that, IMHO, is not a good practice even if you are using some kind of dynamic language platform. It's amazing how easy one's life can be with unit tests.
  14. Length of the cycle[ Go to top ]

    What happens when step #4 (run the suite of unit tests) takes 10 minutes or longer. The problem we found was having to make small text or formatting changes and having to wait around 10 minutes while the build did its thing. A better solution would be a development build that I can tweek on the app server itself. So the changes I make are immediate. Then later I can merge the changes into the source repository. I'm not arguing for one path versus an other, I'm wanting to learn what developers are using to solve this problem. Also, I don't see unit testing as beneficial for the entire application. Functional modules, core logic, yes. Testing the "Have someone call me" button on the customer service page, probably not. How long does your bug fixing cycle (#1 to #4) typically take? -Frank
  15. Be realistic[ Go to top ]

    What happens when step #4 (run the suite of unit tests) takes 10 minutes or longer.
    I've never seen a project where running a suite of _unit_ tests takes 10 minutes or longer. And I work on projects that have thousands of unit tests. Regression tests or functional tests perhaps, but unit tests should run very quickly. Try and short-circuit the process and you'll probably fix this bug but introduce two more. So if the cost to avoiding that is waiting a few minutes to run some tests, it seems like a fair tradeoff to me.
  16. Re: Length of the cycle[ Go to top ]

    10 minutes is a long time for anyone to sit around doing nothing useful, it reminds me of what it was like to do big C compiles with Turbo C on and 8086 and swapping floppy disks . It sounds like there is something wrong with the build process if a person can't do a incremental build and deploy to an exploded war or ear in a few seconds. I do this with Seam apps all the time, and if you are talking about a JSF change it is possible to set up eclipse to push those over with minimal fuss and no app server restart required. Changing java classes requires a restart (usually) but those should be vetted with unit tests anyway. I'm guessing that the 10 minute figure is coming from a full clean build plus an app server restart, which is overkill for a local development cycle. I usually only do a full clean build once a day, unless major changes are happening in the code. It sounds like it would be worth the effort to take a look at the build and see what can be done to improve it. The full build plus full unit plus integration tests should probably be pushed off on to an integration server like Hudson. I usually just run the one or two unit tests I care about directly from the IDE make sure they work, do a sanity check in the UI on my local instance if applicable, and then check it in. I'll hear about it from the continuous integration server if I broke one of the longer running integration tests.
  17. Unit tests don't work for UI related stuff, or at least very bad. One reason is frameworks usually have quite bad support for that, other reason is simply that you only know the result your code is good when you actually see it on your screen. Also, as mentioned before, many JEE servers such as JBoss make it very hard to test logic related to EJB's, persistence, database outside the container (and inside unit tests). And most developers just have to deal with that software, any yes-but and no-buts aside.
  18. Minor note: this message was a reply to Michael Dowling, on using unit tests (I didn't notice theserverside doesn't support threading)
  19. Either the author has never tried developing in Java or he's just out here selling. JVM and debugger allow on-the-fly recompilation and hot-swap debugging for non-structural changes. So "in-place-editing" works for an extent. JSP also allow run-time editing as well. Note: he's talking about "developers" yet it sounds like he wants this for "operation management". On-the-fly messing up with production system is a bad idea in the first place, no matter what language system one deploys!
  20. The author is talking about Enterprise software. We have complicated build processes there, packaging. From one side, the project should be able to generate a deployable artifact in just one step. On the other hand, each change should be immediately updated in the developer's own environment (for example, local tomcat). I use the omnipotent FileSync eclipse plug-in ( http://andrei.gmxhome.de/filesync/index.html ). Eclipse has build-on-save, and all you have to do is setup the file synchronizer to send the recompiled files to the runtime (Tomcat, or any app server). Thus you have both one-step build process and hot-swapping.
  21. A very good question[ Go to top ]

    What is your organization using to accomplish "edit in place" and monitoring?

    Frank asks a great question as usual, but it is really two questions around one very important topic: Developer productivity. Edit in place: In this economy we can hardly continue to have slow performing things like J2EE packaging and deployment. Frankly JBossAS, Weblogic and WebSphere are the worst for this. There are hacky solutions to pull it off but they aren't great. We also have little time for diagnostics by hand (ala jboss's twiddle or jmx-console or oracle's crazy statistic tables). A Spring/Tomcat or Spring/Jetty environment (IMO jetty is better for development), you can come far far closer. Especially when you combine stuff like JavaRebel. You can get most of the way there with Java's agent system but it is far from perfect, you still have to restart sometimes (I committed some stuff like this for working with Spring in Appcelerator: http://community.appcelerator.org/thread/2371). On the other hand, I really think that JavaRebel should be pretty easy to replicate in an open source solution (hard for our company to justify spending much time on at $60-$150 bucks). It basically works like JBossAOP or AspectJ's "advisable" class manipulation. If Rails would grow up, Ruby, Perl and PHP just don't have THIS problem (ruby has much worst production environment deployment semantics but that is another story). Monitoring/Diagnostics: We really have two types of monitoring: production and test/development. Ideally you have a scalable solution for both. The trouble with solutions like JBossON is they are part of a single vendor's vertical pipe dream. The idea that you will choose Red Hat for everything (including things they don't make). Sure it has plugins for various databases, but it becomes readily apparent that the focus of JBossON is unsurprisingly JBoss. I've never been greatly impressed with Hyperic outside of a NOC solution (like Nagios). I'm hoping the Spring acquisition changes this. Springsource tends to focus on developer productivity generally. Wily is a much better solution for diagnostics (which hibernate session executing which query is taking so long and why) but is expensive and isn't exactly a turn-key solution. Wily can be pretty invasive and laborious for more simple load test monitoring. We still need something more comprehensive and scalable solution that we can both on the desktop and in the NOC like what Helios is heading to offer: https://helios.dev.java.net/. End to End (an ideal environment): 1. Something like Appcelerator+JavaInstrumentation and Jetty or Tomcat + Spring/Annotations on the developer's desktop. Scale this to JBoss or something else if you need two-phase commit but only in production, test and continuous integration environments. Risk: works on my machine. 2. Use Hibernate/JPA for persistence and have a local desktop database for development (shared dev databases are problematic) 3. A continuous integration system: cruise control, Maestro, or cron + ant if you must which deploys to an environment that looks exactly like prod. 4. parameterize all of your descriptors so that one deployment works in every environment. 5. Unit tests including run-time asserts for component performance tests (meaning your DAO or Home class "findCustomer" should not take
    1s once the environment is spun up).
    6. A monitoring tool like Nagios or Helios with plugins for network, database (top 10 most run queries, top 10 heaviest), application server, major application component (JSR-177 stats on major EJBs, connection pools for instance), web server (if you still have one), and load balancer. 7. Regular load and functional testing with something great and high value like Testmaker and Selenium tests.. Should be easy to run once you have the scripts and you should see a big red line. You shouldn't have to spend as much to develop this environment as the application costs (aka Mercury). How you know you're good: 1. time to edit a single .java file and see that change in practice <10s 2. time to change a method signature and see that in practice <30s 3. if you screw up your fetch strategy something pops up to tell you that ORDER->ORDER LINE ITEM runs 100s of times per session without someone digging too deep. The sooner you catch this, the cheaper it is to fix and less of a chance it makes it to prod. 4. If your hibernate session contains 20mb of data per user, something pops up and let's you know. 5. Load tests to compare each major build or at least release before it gets to prod. Sadly I've never seen this put fully into practice anywhere. I've seen places achieve bits and pieces of it, but often some technology choices (Sybase or Informix or ...) prevent a full stack edit-in-place with full stack monitoring from DEV to NOC and diagnostics. -Andy PS. Fair notice: We work with Push to Test and Appcelerator.