Discussions

News: Article: A stepped approach to J2EE testing with Simulated DAO

  1. Many J2EE developers use the DAO (data access object) pattern. This article describes Simulated DAO, "the best of DAO, mock objects, and layered testing; letting you simultaneously improve both your testing results and your overall development method".

    The author defines some of the concrete benefits of layered testing with SDAO:

    It's cheap: Using a simulated database for testing and debugging saves you the cost of putting DB2 (for example) on every developer's desktop.

    It's easy: Building an application with servlets, JSP files, and EJB components is complex enough without having to deal with database errors, too. Layered testing allows you to work out your presentation and business logic without simultaneously worrying about the back-end database.

    It's fast: Separation by layer lets you isolate problems as they occur, which results in faster debug cycles. Some errors (such as TransactionRollbackException) are difficult to locate; removing the database layer from the equation lets you hone in on the actual problem more quickly.

    It's flexible: SDAOs can come in handy for performance profiling and testing. While some classes of performance problem (such as database deadlock) require an actual database to resolve, SDAOs let you obtain measurements of only domain and GUI performance, which you can then use to resolve problems at those layers.

    Read the Simulated DAO Article

    How many people are jumping on the mock objects, layered testing world? Many people talk about it, how many real applications use it?

    Threaded Messages (15)

  2. Kyle sez..[ Go to top ]

    Kyle sez: "It's cheap."

    No, it isn't. When you get into an application of any real complexity, this Mock Object Framework becomes a significant subsystem, that has to be developed, maintained, and debugged alongside the actual application. The developer hours needed to sustain this subsystem on a large project are not cheap.

    And of course, it will not replace a real testing framework. 95% of the unexpected performance problems I encounter occur from interacting with the back-end database. There are a whole host of errors (such as in transactional semantics) that will not be discovered in mock object testing. So if you go with mock objects, you have to maintain this alongside a real testing subsystem.

    If it is prohibitively expensive to install DB2 on your desktop, then don't do it. Connect to a remote server, or select an appropriate free environment( Postgres, MySQL, Hypersonic ) to install locally.

    Kyle sez: "It's easy."

    No, it isn't. The complexity of this mock object framework scales linearly with the complexity of the application. And bugs in the framework will mask bugs in the application.

    I am really surprised that someone who cites Kent Beck and Martin Fowler among his sources claims that this solution addresses the complexity of testing JSP's, EJB's, and persistence components at once. The way to manage this complexity is to continuously test, develop, and integrate bite-sized feature sets one at a time, not to add an additional complex subsystem that has to be maintained alongside the application.

    Kyle sez: "It's fast."

    Well, it runs fast. It is not fast to develop and maintain once you get to systems of reasonable complexity.

    Kyle indicates he already has a working DAO with Crud capabilities that runs against a persistence system. Why not use that to create your test objects (and rollback when you are done)? THAT is fast to develop.

    Kyle sez: "It's flexible... handy for performance profiling and testing"

    My experience is that it is almost completely useless for performance profiling (see above). I would say that it is completely useless for testing, because you are going to have to build a real test framework that operates alongside your mock object framework anyway.
  3. Maybe Not totally useless[ Go to top ]

    I find that mock objects (using a factory pattern) works quite well for allowing the frontend people to do UNIT testing. I find that frontend business logic "boundary condition" testing can be more easily done without having a database or whatever the datasource in the mix.

    Also, I find that during development, the backend typically lags the frontend. Convicing/negotiating with the DBA's has typically been a long process. The Local/Remote data location decisions also tends to slow things down as well. Finally at my current job, It takes (weeks) to get an MQ queue setup for me to receive messages (data) from my "clients". Finally, creating the "data layer" (every previous place I've worked has had one) takes sometime as well.


    All test cases using the mock objects must be recreated when the "real" data source becomes avaiable. I think mock objects are reasonable for Unit testing. Integration testing becomes a hassle with mock objects since the mock object framework becomes more complex.


    I agree that for profiling, I don't find mock objects that usefull, but mock objects does allow ISOLATION of the JSP/SERVLETS that you may be benchmarking. This takes the data access time out of the timing and let's you focus on timing your business logic (note, I have never done this) .
  4. huh?[ Go to top ]

    the front end ALWAYS lags behind the back end. Many tools exist to generate the tables and javabeans, the need to mock these almost never comes up, and when it does, it will take MORE time to create the mock objects then just go to the db.

    If this is not the case for you, clearly you are using the wrong tools for the job.
  5. huh?[ Go to top ]

    "If this is not the case for you, clearly you are using the wrong tools for the job."


    I don't know WHAT tool you are using but

    class Payment
    {
       public String getPaymentType()
       {
        return "DDA";
       }
    }

    seems pretty hard to beat. I can create a payment mock object WITH WSAD in less than a minute. I don't know any tool/db combo that can beat that.

    For unit testing I have found this technique very useful. And I would agree that this work becomes "throw away work" when the data layer comes on line. I don't spend a lot of time making or supporting mock objects, so maybe the model I am dealing with is simpler than what most people work with ? I have used this on jobs and it works well.
  6. Maybe Not totally useless[ Go to top ]

    <Fletcher>
    I find that mock objects (using a factory pattern) works quite well for allowing the frontend people to do UNIT testing. I find that frontend business logic "boundary condition" testing can be more easily done without having a database or whatever the datasource in the mix.
    /Fletcher

    Um, yes. But in my experience, as your application complexity grows you end up doing alot of double testing - testing once with the mock object system, and then doing integration testing with the real database. The killer here is that the integration testing with a real database can be very difficult if an error occurs - is the error in the DB reference data, the database schema, bad SQL that slipped through the mock object system, or other kinds of bugs that only show when using the real thing?

    In addition - at integration time the developer has often moved onto other things and doesn't have the code and its environment fresh in their minds. When something breaks at integration, this means the developer is going to be moving slowly trying to recollection how that particular piece works. If he had been using the real database at the time (well, hopefully a developer DB instance), he'd be tackling database problems as they occur with the whole problem set firmly fixed in his mind.

    <Fletcher>
    Also, I find that during development, the backend typically lags the frontend. Convicing/negotiating with the DBA's has typically been a long process. The Local/Remote data location decisions also tends to slow things down as well. Finally at my current job, It takes (weeks) to get an MQ queue setup for me to receive messages (data) from my "clients". Finally, creating the "data layer" (every previous place I've worked has had one) takes sometime as well.
    /Fletcher

    This is why your entire developer environment should be mostly under the control of developers, not third parties. Certainly DBAs and domain experts should be assisting the developers, sometimes extensively, while they're fiddling outside of code, but these people shouldn't be a bottleneck. Ideally, each developer should have their own complete environment that they can fiddle with at will. When this isn't possible (e.g. the resource is expensive), at the least developers should share a common resource that is theirs, and not QAs or staging or production.

    <Fletcher>
    I agree that for profiling, I don't find mock objects that usefull, but mock objects does allow ISOLATION of the JSP/SERVLETS that you may be benchmarking. This takes the data access time out of the timing and let's you focus on timing your business logic (note, I have never done this)
    /Fletcher

    What is the purpose of this isolation? It seems like a waste of time to me, since you have to profile the whole integrated system anyway. I think it would be far more productive for all involved to profile the true production code path, and then send the results for the appropriate party for tuning (e.g. if your screen is running slow, but profiling shows you're spending all your time for a business object to run, forward the result to the guy who wrote the business object; conversely, if you're into the XP thing fix the biz object yourself :-).

    In a perfect world with lots of time to implement every niggling detail perfectly, I can see the sense of massive isolation of all layers and testing everything that way (even testing all possible component combinations, even if they're not realized in the real app code). But this is the real world where both timeliness and quality both need to be addressed. In an abstract sense complete isolation everywhere is quite appealling. As a coder I like to see all the pieces tuned to perfection and everything done as close to "perfect" as I can achieve. But in a business environment, I have to prove that the production code paths work as advertised and also meets performance requirements, and also do this in a reasonable time frame. In that context, inserting an artificial mock object sub-system is going to slow me down, increase my work load, and reduce quality.

    Don't get me wrong - mock objects have their place. In particular, if the other side doesn't exist at all yet (e.g. the database guys haven't yet pulled their thumbs out and delivered the initial rev of the database schema), then mock objects can allow you to make progress. But this sort of case is making do when a resource is missing. It shouldn't, IMHO, be a normal modus operandi.

        -Mike
  7. Maybe Not totally useless[ Go to top ]

    <mike>
    Don't get me wrong - mock objects have their place. In particular, if the other side doesn't exist at all yet (e.g. the database guys haven't yet pulled their thumbs out and delivered the initial rev of the database schema), then mock objects can allow you to make progress. But this sort of case is making do when a resource is missing. It shouldn't, IMHO, be a normal modus operandi.
    <mike>

    Not to Mock you :) You've hit the nail on the head, I use this when the real resource does not exist. I have NEVER worked at a project where the backend people have delivered the initial rev of the database before the frontend people are deep in coding.

    As for redudant testing, this is not a problem since much of the testing is done with scripts. Each build, all scripts are run, so this is not a biggie.
  8. Right on Coby[ Go to top ]

    Yeah, this article is very misleading, short on details, and just plain sucks. Shame on the server side for showing it the light of day.
  9. Kyle sez..[ Go to top ]

    <Corby Page>
    Kyle indicates he already has a working DAO with Crud capabilities that runs against a persistence system. Why not use that to create your test objects (and rollback when you are done)? THAT is fast to develop.
    Page>

    We have a system that uses that approach, but as our system grew it tool longer and longer to run test suites. First minutes ... than hours !!! It is a big system and rollback is not the fastest operation in the world, especially that first you need to pretty much wipe out half of the data to ensure your tests are isolated from other changes to DB.
    The first solution was to use an always empty DB instance. It improved things a little bit, but it was till minutes. Not good for "continues integration" ... so we started to use mock DAO in a similar way to what Kyle descibes (he de facto described them already in his Webshere book). It was a bit of work to apply them (and we didn't apply them to all tests), but now we can run our tests much faster and more often.
  10. Kyle sez..[ Go to top ]

    <Swietlinskio>
    The first solution was to use an always empty DB instance. It improved things a little bit, but it was till minutes. Not good for "continues integration" ... so we started to use mock DAO in a similar way to what Kyle descibes (he de facto described them already in his Webshere book). It was a bit of work to apply them (and we didn't apply them to all tests), but now we can run our tests much faster and more often.
    /Swietlinskio

    How can you describe what you're doing as improving continuous integration? It sounds like you're able to unit test certain components more often because you've stubbed out the database with mock objects, but that's _not_ helping you towards continuous integration. In fact, the method you describe quite purposely delays integration.

    This is one of the classic tradeoffs in modern development - speed of development vs. completeness of testing during development. Personally, I prefer not to introduce artificial layers to speed up development. Instead, I look for the root cause that's slowing things down. In this case - maybe your DB schema or SQL is slow. Or perhaps your unit tests are too coarse grained. Or something similar. In any case, when developers are forced to "eat their own dog food" and develop with the real system w/out stubbing out major portions, performance problems have a way of being rapidly fixed. For the simple reason that the developers are forced to live with their own system, and have a powerful incentive to optimize it.

    Of course you can't always optimize everything (and even if you could it may be a waste of time), but even then readjusting the granularity of your unit tests so that you have more tests of narrower scope can still address the problem without introducing anything artifical into the equation.

    In case no one's guessed (:-) I'm a strong opponent of stubbing out existing systems during development work. When you do that you're delaying integration - and everyone knows that integration bugs are harder to track down and fix than bugs that pop up during straight development. You can pay now, or you can pay 5 times more a few days later....

        -Mike
  11. Kyle sez..[ Go to top ]

    Hi Mike,

    By 'continous Integration' we mean: 'integrating every day' and every time we integrate, we have to run all unit tests - at least once.

    Unit tests should be at all levels, and tests at high levels often will generate a lot DB accesses, if you don't 'stub out' the DB. No matter how efficient you make your SQL, there will be (well-tested) projects, where unit tests take a lot of time and so slow down the development process, if you don't 'stub out' the DB in most of the unit test. Of course there should remain some low-level unit tests, where you don't 'stub out' the DB.

    The development resources saved by decreasing unit test time may be invested in making better unit tests and more integration, which again will make you find bugs quicker - just the opposite of your conclusion.

    In my project we are in the nearly same situation as Krysztof, and I am right now looking for a way to 'stub out' time consuming elements, and read Kyles article with interest.

    regards
    Hardy
  12. Kyle sez..[ Go to top ]

    \Henneberg\

    By 'continous Integration' we mean: 'integrating every day' and every time we integrate, we have to run all unit tests - at least once.

    [and]

    The development resources saved by decreasing unit test time may be invested in making better unit tests and more integration, which again will make you find bugs quicker - just the opposite of your conclusion.
    \Henneberg\

    I'm afraid we'll have to agree to disagree on this one. If a significant number of your unit tests are stubbing out database access (or any other resource), then you're achieving code integration across developers, but not really verifying system integration.

    You may find certain classes of bugs more quickly in this method - but at the same time, you're hiding others because your tests are _not_ testing the production system.

    In short - I think running tests in an artificial environment can often give a false sense of confidence and progress, and mask serious integration bugs. You say you run unit tests to test integration once a day - is the non-stubbed out code so slow that it's painful to run once a day? If so, this goes back to my comment on eating your own dog food and the motivating factor it brings to developers....

         -Mike
  13. Useful ?[ Go to top ]

    <mike>
    How can you describe what you're doing as improving continuous integration? It sounds like you're able to unit test certain components more often because you've stubbed out the database with mock objects, but that's _not_ helping you towards continuous integration.


    ....


    In case no one's guessed (:-) I'm a strong opponent of stubbing out existing systems during development work. When you do that you're delaying integration - and everyone knows that integration bugs are harder to track down and fix than bugs that pop up during straight development. You can pay now, or you can pay 5 times more a few days later....

    <mike>

    It sounds like your NOT a strong proponent of UNIT testing. Mock objects allow you to thoroughly UNIT test a piece of code. The goal of UNIT testing is to carefully isolate the component and test it in a controlled enviornment. Mock objects does delay integration testing ( only by a small amount since the data group is working in parallel ) but provides BETTER unit testing. Better unit tested objects tends to have smaller bugs during integration testing.


    Also, working with a data layer where you rollback transactions is a fine way to test, but as systems get complicated, this may not be possible. If the performance of the data layer is inherently slow, it may be useful to use mock objects to test the reliablity of OTHER PARTS of the system in a controlled fashion. For example, I frequently leave my PC running a test when I leave work. Mock objects allow me to run 400k iterations of my system v.s. maybe 3000 iterations when the real data layer is used.

    Plese Note, I do run the same test WITH THE DATA LAYER. I do this AFTER I have run the test with the mock objects. If the version with the data layer has problems, typically, the problem is in the datalayer. Also, the database people get pissed when we run these test because we can't roll back everything so they frequently have to drop a bunch of tables and 'reset' the database after we run one of these tests over the weekend.
  14. Useful ?[ Go to top ]

    \Fletcher\
    It sounds like your NOT a strong proponent of UNIT testing. Mock objects allow you to thoroughly UNIT test a piece of code. The goal of UNIT testing is to carefully isolate the component and test it in a controlled enviornment. Mock objects does delay integration testing ( only by a small amount since the data group is working in parallel ) but provides BETTER unit testing. Better unit tested objects tends to have smaller bugs during integration testing.
    \Fletcher\

    I find for complex systems that unit testing is of limited value. There is value there, but the recent XP surge has over valued it.

    For example, imagine a system with 15 major sub-systems and a thousand classes and maybe 40 database tables. There is value in unit testing each of those classes and each of the sub-systems in isolation. However - the true complexity of the system, and the true structure of the application, is captured in the _interactions_ between the classes and the sub-systems and the database (and any other resources). It's important that the lowest level units operate as advertised - that's where unit testing comes in. But those units are rather trivial compared to the application as a whole, and finding and fixing bugs that reside wholly in a unit isn't all that difficult.

    Where the difficulty comes in is finding and fixing bugs that arise from the interactions of multiple classes/multiple subsystems. As such, while unit tests give me some guarantees, they are rather small guarantees. The much better verification is from the integration tests, and as such I run integration tests as often as possible, often several times a day. I may not always run the entire test, but I also ensure that when I test a piece of functionality it's a full-through test - from the front end all the way out to all required resources and back. To do anything less is to emphasize the easy bugs and delay even being aware of the more difficult bugs that arise at integration time.

    \Fletcher\
    Also, working with a data layer where you rollback transactions is a fine way to test, but as systems get complicated, this may not be possible. If the performance of the data layer is inherently slow, it may be useful to use mock objects to test the reliablity of OTHER PARTS of the system in a controlled fashion. For example, I frequently leave my PC running a test when I leave work. Mock objects allow me to run 400k iterations of my system v.s. maybe 3000 iterations when the real data layer is used.
    \Fletcher\

    From my point of view, all that matters to the end user is the reliability of the entire system. A perfect UI and perfect business layer won't mean anything if the system doesn't work as a whole.

    I've seen the sort of testing paradigm that you've talked about, and my response has always been the same: "What have you proved at the end of the test?". And the answer has always been "I've proved this one piece works in my testbed environment". Is there some value in that? Yes. Is there alot of value in it? No. To me, it's equivalent to saying "the code compiles". No one particularly cares if your code runs flawlessly in isolation. What matters is that it runs with the code it's intended to run with, full through.

    \Fletcher\
    Plese Note, I do run the same test WITH THE DATA LAYER. I do this AFTER I have run the test with the mock objects. If the version with the data layer has problems, typically, the problem is in the datalayer. Also, the database people get pissed when we run these test because we can't roll back everything so they frequently have to drop a bunch of tables and 'reset' the database after we run one of these tests over the weekend.
    \Fletcher\

    I don't entirely understand. What's this "they" piece? Where I work, developers don't need "database people" to clean up the development database. The developers do it. It sounds like your real problem is that database knowledge and/or access is concentrated outside of the development group, and it's bottlenecking development and testing. So why not let all developers have several development instances of the database that they can muck with at will?

        -Mike
  15. It doesn't HAVE to be that hard...[ Go to top ]

    "When you get into an application of any real complexity, this Mock Object Framework becomes a significant subsystem, that has to be developed, maintained, and debugged alongside the actual application. The developer hours needed to sustain this subsystem on a large project are not cheap."

    If you use a product like Polygenix AgileTest (http://www.polygenix.com), which generates mocks dynamically at the bytecode level, then you don't end up with a ton of additional artefacts that need to be maintained and kept in synch with the rest of the development effort.

    "And of course, it will not replace a real testing framework. 95% of the unexpected performance problems I encounter occur from interacting with the back-end database. There are a whole host of errors (such as in transactional semantics) that will not be discovered in mock object testing. So if you go with mock objects, you have to maintain this alongside a real testing subsystem."

    True enough, but I would suggest you're confusing functional testing with unit testing here. Nothing exists in isolation, but you can't begin to seriously test or debug a complex system until you have the confidence in the component objects and their immediate collaborations.

    "If it is prohibitively expensive to install DB2 on your desktop, then don't do it. Connect to a remote server, or select an appropriate free environment( Postgres, MySQL, Hypersonic ) to install locally"

    Same again. You may want to test communications and data store implementation at a functonal level but, if you incorporate these factors into your unit tests, not only do you compromise the integrity of those tests, you also introduce a mountain of uncertainty into the debugging process.

    "The complexity of this mock object framework scales linearly with the complexity of the application. And bugs in the framework will mask bugs in the application."

    Depends entirely on the manner in which you generate and maintain your mock objects, and the flexibility and ease with which the "framework" allows you to manage the behaviour model for the mock objects you use. Again, a product like Polygenix AgileTest puts paid to this common myth.
     
    "The way to manage this complexity is to continuously test, develop, and integrate bite-sized feature sets one at a time, not to add an additional complex subsystem that has to be maintained alongside the application."

    Couldn't agree more ;-)

    "It is not fast to develop and maintain once you get to systems of reasonable complexity."

    Doesn't HAVE to be...

    "Kyle indicates he already has a working DAO with Crud capabilities that runs against a persistence system. Why not use that to create your test objects (and rollback when you are done)? THAT is fast to develop."

    Fast to develop, but not cleanly isolated. Again, this comes down to the difference between unit and functional testing. Both are requirements of any good development, but neither should be used in place of each other. Nor should the boundaries between the two be blurred. At risk of waving the flag one too many times, AgileTest provides dynamic object substituion at the bytecode level (no additional artefacts or source code) that allows you to introduce mocks at ANY point in a class under test, so even a JNDI lookup to a pooled datasource can be mocked with VERY little intrusion into the test code. Now THAT is fast...;-)

    "My experience is that it is almost completely useless for performance profiling (see above). I would say that it is completely useless for testing, because you are going to have to build a real test framework that operates alongside your mock object framework anyway."

    Er.. Useless? ...see above ;-)
  16. real inherits mock = evolution[ Go to top ]

    I've used a similar approach before to provide mock objects for the session beans to handle the dependencies between front-end developers and an EJB tier.

    The main difference is that the real session bean implementations descend from the mock implementations - this provides a means of gradually evolving away from the mock implementations over time as functionality is developed.
    Front end developers can work without an application server using purely mock implementations and back-end developers can fully compile functional session beans that have a progressively changing mix of real and mock implementation.
    EJB developers override the base classes' mock methods (this is where it would be handy if Java had an "overrides" declaration in the language to state your intent and make the compiler check you had the method signature right!)

    To see what is left to develop in a system, remove the "extends Mock..." declaration from your session beans and let the compiler tell you whats missing. When you're ready to ship this relationship can be removed entirely.

    I wont pretend this is without drawbacks ie extra maintenance and inability to simulate all functionality but this can still be a useful tool.