Discussions

EJB design: Any Patterns for Audit Logging in J2EE Applications?

  1. We are using EJB Session beans and DAOs in our application and there is requirement for Audit Logging. Basically we want to do a Audit Log of changes in the business object attibutes. What is the best way to implement this.


    Thanks

    -Ramesh
  2. It depends. There are two basic options:

    1) Log the database updates, using triggers in the database or some other database feature (most RDBMS software has some kind of audit feature).

    2) Log in your DAO, using some sort of logging API like Log4J.

    Do whichever option works best for you, based on the skill sets of your team and the kind of information you need to capture. For example, if you using connection pooling, the DAO-to-database connection is probably using a generic user. If that is the case, and you need to log user information, you will need to log it in the EJB layer, where you use methods like ejbContext.getCallerPrincipal().getName() to get the userId.
  3. Some Suggestions[ Go to top ]

    Some ways of implementing audit logging include using triggers at the database or using Log4j.

    however, each of these have brought issues for me in the past:
    1. Triggers means I need to, well, write PL/SQL code to perform audit logging to database tables. This is added hassle, as I need to work in two domains now - J2EE and relational database.
    2. All loggers I know of, including Log4j and JDK1.4 Logger, use synchronized methods to control access to shared resources such as the log files. This is technically a no-no for EJB applications - see http://theserverside.com/discussions/thread.tss?thread_id=22414.

    For audit logging using Log4j - you can define an audit category for your application and use that wherever you want to write to the audit log. You can control in log4j config where this log is written to, what levels are logged etc.

    A scalable approach, if you really need it, and one that is completely in line with the J2EE specification, is to use JMS. That is, publish your audit log messages to a message queue, and another, seperate process, can take them off the queue and log them to wherever. OpenJMS is an excellent open-source implementation if you are considering this, though it may be overkill.

    Hope this helps,
    Regards,
     Trev
  4. JMS + Elbow Grease[ Go to top ]

    JMS is a way to do it, but doing it in a DAO or in JMS, it's still the same amout of work, and it's shameful that J2EE has no facility for this, as it's something that could be easily implemented as part of CMP. If you do it with JMS, you have to use durable subscribers and all the other mumbo jumbo to ensure JMS messages don't get lost on a server crash/reboot.

    Doing it with triggers is slightly more transparent, but unless your usernames to be associated with audit log entries are database users, triggers will not really work.

    What we do where I work is we have a code generator generate our DAOs. The generated code assumes an audit table that follows certain conventions and puts auditing code in there.
  5. Thanks for your replies.


    We don't want to use DB Triggers to do audits.

    In Our current design we pass Value Objects (DTOs) as shown below along with the login user details.

    WEB LAYER -> EJB LAYER - DATA ACCESS LAYER

    Currently what we are planning to do is, in the EJB layer when an event occurs that causes a change in the Business Object data, we will get old data of the business object data as a Values Object (from DB) and generate a diff between Old Value Object and New Value Object and do a audit log of all the changes into a separate table along with EVENT NAME, USER NAME, TIMESTAMP and Business Object changes.

    The question is if we do all this as part of current TXN, do we end up with performance issues?

    Does using JMS for doing audit logging will help improving the performance of the App and improving the current TXN response time?


    Thanks

    -Ramesh
  6. Performing the audit log as part of the current transaction will have a significant negative impact on performance. Assuming you are logging every operation, you are doubling your number of SQL calls.

    I would suggest you log using some other mechanism. JMS is a good choice. Send log data in a JMS message, and process it elsewhere (maybe even on a different server). This should minimize the performance cost.

    Also, for security related logging, I suggest you ultimately store your audit log in a database other than your production database. If your production database were ever compromised, the hacker could clear out the log. This will also improve performance, since logging will not increase the load on your production database.

    Another suggestion: rather than logging the diff of value objects, I suggest you log the following: EVENT, USER, TIMESTAMP, CLASS, OLD-VALUE, NEW-VALUE, printing values using the DTO's toString() method. This should allow you to make the logging operation completely generic, so that you can write a single method in your DAO superclass or a utility logging class. This will give a lot of flexibility in tweaking your logging process to opmitize for performance, etc.
  7. Paul,

           Thank you very much for pointing out nice points w.r.t design and performance.

     "store your audit log in a database other than your production database"

    But, in our web applicatin on every page we need to show a Audit Trail button, when user clicks on it we have to show paginated audit logs of the business object.

    "printing values using the DTO's toString() method"

     Keeping end-user in mind, I think audit logging DTO's toString() method causes readability issues.
     
     In out current design we want to maintain list of attributes for each page that needs to be tracked in audit logs, but this design will cause maintenance issues over the time. Do you think is there any better approach to do this.


    Thanks,

    Ramesh
  8. But, in our web application on every page we need to show a Audit Trail button, when user clicks on it we have to show paginated audit logs of the business object.
    In that case, it is probably more effecient to store the audit log in the same database. I suggest you also log the OBJECT_ID (primary key) and TABLE_NAME to the values you are logging. That will only work if all your primary keys are the same type.
    Keeping end-user in mind, I think audit logging DTO's toString() method causes readability issues.

    In our current design we want to maintain list of attributes for each page that needs to be tracked in audit logs, but this design will cause maintenance issues over the time. Do you think is there any better approach to do this.
    It depends. If it is just a matter formatting, I suggest you log data as an XML string, which would be easier to display in a variety of ways. You can add a toXMLString() method to your DAO.

    However, if the requirement is that you can, on a object-by-object basic, track the changes in individual fields, then things are more complicated. You might want to log individual attributes, or maybe even create a separate audit table to match each production table. The audit table should have the same columns as the production table with an extra TIMESTAMP, EVENT and USER_ID fields for auditing purpose.

    Be sure you really need this level of detail before you implement this. If you do, I suggest you go with the duplicate audit tables, which will have the most flexibility in the long run, and will probably be more performant than a generic AUDIT_ATTRIBUTE table.

    If you take this route, you may want to reconsider using database triggers; it would be easier and effecient to set up a trigger to insert and timestamp a new record in the audit tables every time you did an update. For the trigger approach to work, though, you would need to add EVENT and USER_ID fields to every table.
  9. Be sure you really need this level of detail before you implement this. If you do, I suggest you go with the duplicate audit tables, which will have the most flexibility in the long run, and will probably be more performant than a generic AUDIT_ATTRIBUTE table.

    In our project, we have similar requirements and planning to write DB Triggers on Main tables to log data changes into Audit Log tables. Are there any pitfalls with this approach? The motivation for going with this approach is to decouple data changes logging from the main transactions.
  10. In our project, we have similar requirements and planning to write DB Triggers on Main tables to log data changes into Audit Log tables. Are there any pitfalls with this approach? The motivation for going with this approach is to decouple data changes logging from the main transactions.
    To be honest, I have not implemented this level of detail for audit logs, so I can only talk about this technique from a theoretical standpoint. You might try reposting the question as a new thread, so that it gets more attention. Or, better, yet, try posting it to the forums for your database server, since this is really a database questions and not a J2EE questions (and the answer will probably be database-specific anyway).
  11. Having seperate audit tables, with triggers to perform writes to them, is certainly a proven approach for auditing data in a relational database. I know lots of big shops that use this approach.

    It will certainly be quite performant, much more so than your application performing the audit writes itself (there will be lots more network round-trips required if the application does it, unless you batch them). It will also decrease the amount of code in your app and effectively make the auditing invisible to the application developer (but of course you know it is there).
  12. What about where the USERID should be derived from. If you have a business component (say EJB JAR with facades) and you want to audit changes should you:

    1) include USER_ID as an attribute which is passed from the client application, or
    2) derive this from the components security context. This may not be quite as flexible but may be the best way to do it from a security point of view?

    Greg
  13. Hi Ramesh,
        We also want to implement audit logging to the same scenario you described to audit log changes in the business object attributes.

    Just wondering how you did in your project so that I can reuse the same in my project.

    Thanks,
    Raghu

    We are using EJB Session beans and DAOs in our application and there is requirement for Audit Logging. Basically we want to do a Audit Log of changes in the business object attibutes. What is the best way to implement this.Thanks-Ramesh