Hot Deployment: How hot is it?

Discussions

News: Hot Deployment: How hot is it?

  1. Hot Deployment: How hot is it? (19 messages)

    Debu Panda of Oracle has written about the role of hot deployment. He discusses what it is, what it is not, and the various issues that you have to think about if you use it.

    Conclusion
    Looking at the hot deployment features supported by various vendors everybody thinks it’s “cool idea” but not robust as it appears. None of the vendors recommend wholeheartedly these in production environment, so you have to be careful when you decide to use this in your production server running your website or mission critical applications.
    Read Debu in Hot Deployment – How hot is this!

    Threaded Messages (19)

  2. Hot Deployment: How hot is it?[ Go to top ]

    Hot deployment is VERY hot for development. The time savings realized when your developers can simply run their build and have the new code auto-deploy instead of build, shutdown, startup is massive.

    In production, I wouldn't worry so much about the time spent polling for changes as much as I'd worry about the infrastructure. If the system isn't critical, you should be able to survive a server reboot without much impact. If the system is critical, you should have failover in place, and once again, you can survive a reboot. If your system REQUIRES production hot redeploy, you probably need to examine your system's design.

    I would go so far as to say that hot redeploy should be disabled (by each company's internal policy) in production. The potential for abuse, error, and poor practices outweighs any benefits. A simple set of procedures and infrastructure can pretty much eliminate any need for it in production.
  3. Hot Deployment: How hot is it?[ Go to top ]

    It also helps to determine if your "application" is serializable (read clusterable) during development. That way you can fix as you go.
  4. A few problems I encountered with hot deployment.

    * In web application environment preserving a session state is problematic if you store application objects in session scope. Also, locale-specific resource bundles referenced in application code (and usually stored in session scope as well) need special treatment for proper serialization/deserialization, which is performed by the application server at the time of hot re-deployment. Java i18n is not new, but this side barely covered.

    * When debugging webapps with IDEA 4.0, there is a limitation to hot re-deployment. The following is a copy from IDEA docs:
    <quote>
    The HotSwap mechanism enables you to reload classes changed during debugging without need to restart the entire application.

    At the moment due to original limitations of Java SDK 1.4.x the HotSwapping is possible ONLY if a method is altered. In all other cases (like changing method or class signature), the class reload is impossible and the corresponding error message appears.
    </quote>

    When fully functional, though, hot deployment is quite handy.
  5. Hot deployment and hot swap are completly different beasts, Valeri. Hot swap is the ability to change class definitions while a VM is running, without the application ever noticing that. This is provided so you can reload quick changes to a class while you're debugging. Hot swap is provided by the JVM, and while quite useful in server-side development, has no direct link to J2EE or application servers. Hot swap also has a number of limitations, as you already discovered.

    Now, hot deployment is the ability to reload an entire J2EE application without bringing down the container. This is implemented by application servers is very different ways, and has no direct links to the JVM.
  6. Thanks, I understand the difference.
    Hot swap is provided by the JVM, and while quite useful in server-side development, has no direct link to J2EE or application servers.
    Well, I guess IDEA made that link: I am running Orion application server while debugging (and how swapping) the J2EE application in IDEA.
    Now, hot deployment is the ability to reload an entire J2EE application without bringing down the container. This is implemented by application servers is very different ways, and has no direct links to the JVM.
    AFAIK, all J2EE servers serialize application state when trying to hot re-deploy the application. Process of serialization is performed directly by JVM.
  7. The libraries at the container level cannot be hot deployed and requires restart. If your newer version of the application is dependent upon a newer library you need to restart your container.
    The interfaces of our business objects are in the classpath. So often I have to shutdown the application server and need to restart it. This happens for example if I add a method to the business object.

    Is there really no way to hot deploy everything? Its very much time over the long run which I lose by having to restart the application server.

    You maybe wonder why we put the interfaces of our business objects in the application server's classpath or why we have interfaces for our business objects (like article, business partner and so on). But we must do it because we use a framework which requires this design.
  8. hot deployment != hot swap[ Go to top ]

    You maybe wonder why we put the interfaces of our business objects in the application server's classpath or why we have interfaces for our business objects (like article, business partner and so on). But we must do it because we use a framework which requires this design.
    May be you should move that framework to your EAR.
    If you cannot move it into the EAR, I am guessing that could be because multiple clients are using it. Maybe you should treat it as a service and take care of it.
  9. hot deployment != hot swap[ Go to top ]

    We don't use EAR files in development, only in production. Is this wrong? I never understood why we do this with Weblogic.

    Besides this, I don't understand why it would help.
  10. hot deployment != hot swap[ Go to top ]

    We don't use EAR files in development, only in production. Is this wrong? I never understood why we do this with Weblogic.Besides this, I don't understand why it would help.
    I think it is not wrong and I think the best container for development is a unit test or "main" method for dirty tests. Probably the main problem is with
    JSP pages, but I can recommend resin for JSP developmet, you can configure it as you want and to use any directory structure you like, it works very good for me without IDE integration.
  11. hot deployment != hot swap[ Go to top ]

    OC4J allows you deploy your application in a open expanded directory structure. However your directory structure have to similar to structure like EAR/WAR and you have to have deployment descriptors.

    This is really useful in a development environment when your programs continuosly change.

    If your library also changes frequently during development phase you can put this as a part of application and add that to the Manifest Class-Path of your WAR or EJB-JAR. In OC4J, if you want to automatically deploy an application without having to restart OC4J (and hence the new library) you can achieve that just changing the timestamp of application.xml (UNIX touch command) and the application will redeploy and hence the library that is part of the application
     

    regards
    Debu Panda
  12. Hot Deployment on JBoss[ Go to top ]

    Hot deployment is very important to development tools like Skyway Builder. However there are some issues with Weblogic and JBoss freeing up memory from previous deployments. These issues are well-document on the respective support forums. Every deploy causes the memory utilization to go up. When you reach the limit, you get an "out of memory" exception. Depending on the configuration of the development environment, we can usually get through 15 to 20 hot deploys before we need to restart the server.

    If we couldn't hot deploy, it would add a tremendous amount of complexity to codeless development environments, like Skyway Builder. The users of such a development environment don't need any knowledge of application servers, or J2EE for that matter. The hot deploy feature shields them from having to understand the details of J2EE application deployment and application server startup/shutdown procedures.
  13. Container Restart is always better than hot deploy. So choose hot deployment only when container restart is not a possibility! Choose a time when not too many users are in the system.
    The libraries at the container level cannot be hot deployed and requires restart. If your newer version of the application is dependent upon a newer library you need to restart your container.
    Hot deployment is a REAL timesaver in development.

    May be Container restart may be better than hot deploy.
    But the restart of a production system (even when there not a lot of users) involves a lot of Bureaucratic process. Some of them are here:

    1. First disable the monitoring tools (otherwise you are bound to send millions of pager messages in the middle of the night).
    2. This has to be done not only for the application you want to change but also others - which technically do not belong into your domain, just because they are hosted on the same system. In other words, you are affecting the availability QoS level contracts of other hosted applications.
    3. Just think about if every application in a web/app server farm starts stopping the application server/farm for their needs.
    4. And then archive logs of not only your application, but others as well.
    5. Then Restart all of them and ensure if the "other" applications started as well (in addition to yours).

    The sheer economics of the above exercise demands that the hot deployment is not only HOT but also a necessity in production environments. I am not talking about one or two J2EE apps hosted, I am talking about large systems hosting dozens of applications.
  14. The libraries at the container level cannot be hot deployed and requires restart. If your newer version of the application is dependent upon a newer library you need to restart your container.
    Be honest - how many times do you really change libraries in the lifetime of a system?

    Compare that with the number of times you roll out new versions of your own applications.

    Library changes is not a reason to rule out hot deployment.

    Oh, I bundle most of the libraries with the EAR except for some (like Database drivers etc.., Third party Service Providers for App server).

    For example: In the systems I have built it does not make sense to get log4j loaded by the system class loader. We have applications using Log4J from stone age to the applications that absolutely need the latest log4j features all hosted on the same system.
  15. Looking at the hot deployment features supported by various vendors everybody thinks it’s “cool idea” but not robust as it appears. None of the vendors recommend wholeheartedly these in production environment, so you have to be careful when you decide to use this in your production server running your website or mission critical applications.
    It is in the best interest of the developers, business and the vendors that they get this robust enough.

    Applications are called "mission critical" for a reason, one of them being their availability!!!!
  16. I think the whole area of deployment in J2EE is really ill thought out. The design caters to one specific model of development - deployment - change life cycle but makes all other patterns unnecessarily complicated. Just one example: In portal/content management systems, people frequently upload fragments of web pages or files for download. Directories are created, files are edited online and stored again, JSPs are generated on the fly and so on. Now, if you have all the pages in a .war, which is _the only_ J2EE standardised deployment format for web applications, you have a problem. Of course there are ways to work around it, but it usually means that you are forced to store documents, fragments, files, etc in the database instead of the file system.

    In general, all situations where you don't have a clear separation between the development stage and the running application are difficult to handle. A J2EE application cannot modify its own configuration at runtime. The deployment descriptors are parsed and interpreted at the time of deployment but afterwards they are cast in stone. There's no API to modify those aspects of the system that are in the deployment descriptors. If you want to dynamically change any of those aspects at runtime, you have to work around J2EE or resort to coding to extremely low level J2EE SPI interfaces like writing your own JCA resource adapter.

    I don't agree with the poster who suggested that it was a problem with the design of your application when you run into such problems. Not at all! It's a problem with the design of J2EE. J2EE deployment works as if we were talking about desktop applications. Write your app, compile it, install it and when you update it, you shut it down, install the update and start it up again. That's not the right model for 24x7 enterprise applications.

    And as an example of how it should be, I don't have to look to something tremendously new or innovative. Database systems have always had alter statements. alter system, alter session, alter just about anything. In Oracle, you can change 99.9% of the settings without shutting down anything or even interrupting a single transaction. Ask any DBA what he would think about a DBMS that had to be shut down to add a security role or a stored procedure.

    Of course I know, that there are workarounds for many problems connected to the static nature of J2EE applications. But it's a lot of work and you need to know J2EE inside out. It's a small wonder that app server vendors differentiate themselves by offering management and configuration facilities that enable all the dynamic modifications that you inevitably get in an enterprise application. We pay them a lot of money to fix what they have screwed up in the J2EE specs.
  17. clustered prod envs[ Go to top ]

    this has been my experience so far with clustered production environments on bea (6.1, 7.0, 8.1):

    - hot deploy is useless in production environments, because when the app reloads, it is inaccessible across the whole cluster
    - if the shared components have not changed (session contents, etc), then one could start bouncing individual containers, relying on session replication to fail over the sessions
    - if the definitions did indeed change (or server classpath libs, etc), and downtime is not an option, then one needs a two-cluster setup, where sessions are serialized in the central storage (not in-memory). this way, one of the clusters could be brought down, while all the users fail over to the next cluster. however, database-based session persistence could be prohibitevely expensive. ideally, some sort of in-memory failover between clusters wold be desirable.

    to summarize, hot deployment in production could cost you lost in-memory sessions. to mitigate that you can either partition your customer base into "valuable" and "not-so-valuable" sessions, with the former being persisted to the database, or just use database persistence altogether (if possible), or do a rolling container restart, if changes are insignificant.

    as for development, obviously hot deploy is invaluable.
  18. edge computing[ Go to top ]

    akamai's edge computing could be an alternative here, although it is not available for bea (and i doubt it ever will be), they can work w/ wsphere to persist your in-memory cache on the edge, as well as some of the j2ee workload - this way you can architect your infrastructure to avoid downtime. of course, this is just an isolated example with very limited applicability.
  19. thank you[ Go to top ]

    Really nice application. Thank you for giving you ideas. -------------------------------------------------------- muzik dinle
  20. hot deployment[ Go to top ]

    Hot deployment sometimes can be messy, I have many a times seen that server didn't detect changes done in the code. Probably servers run some thread which checks for changes at regular intervals. But the timestamp changes are simply ignored by Tomcat, WebLogic or WebSphere and specially when the changes are very small. Anyways I have written about enabling hot deployment in various servers as not all servers come configured for hot deployment out of the box.

    Hot Deployment in Tomcat