News: Test Driven Scaffolding: a new agile method

  1. Test Driven Scaffolding: a new agile method (12 messages)

    The next evolution of Test Driven Development. Test Driven Scaffolding (TDS) is an idea I propose to improve development productivity by declaratively generating source code from unit tests by holistically merging the benefits of Test Driven Development with Code Generation. Let me capture your imagination: wouldn't it be really cool if we could write a unit test and then when we run the test, it writes the source (production) code for you so you don't have to write the code? You write the test and it declaratively creates the source code under the hood. Well this is the principle of TDS; making your test cases go the extra mile. Let's get more specific. I am interested using TDS for generating web-based user interfaces with component-based web frameworks like Wicket and Google Web Toolkit. I am not not alone as other people have been looking at ways to devise a way on how to create declarative user interfaces where one does not need to write the UI code. The Google Web Toolkit (GWT) engineers have been experimenting with a way to declaratively create GWT web applications but its declarative model is XML - given it is isomorphic to an Abstract Syntax Tree (AST). However wouldn't it be more agile if it was declarative created from a test case rather than XML? Let's look on how we can achieve this. I will now show you a simple demonstration of Test Driven Scaffolding to scaffold a GWT web application (user interface) from a test case. Oh, if you are unfamiliar with Google Web Toolkit then going to http://code.google.com/webtoolkit will provide a good primer. I will show you the test case, see its output and then look under the hood. Ok, lets start with the test case: import com.google.gwt.user.client.ui.Label; import com.google.gwt.user.client.ui.TextBox; import com.google.gwt.user.client.ui.Button; public class HelloWorldGWTApplicationTestCase { public static void main(String[] args) { final File input = new File("/path/to/template/folder/"); output = new File("/path/to/src/folder/"+applicationName+".java"); final GWTScaffoldTestCase gwtScaffoldTestCase = new GWTScaffoldTestCase(input, output); gwtScaffoldTestCase.assertApplicationName(applicationName); gwtScaffoldTestCase.assertGWTWidget(DecoratorPanel.class); gwtScaffoldTestCase.assertGWTWidget(Label.class); gwtScaffoldTestCase.assertGWTWidget(TextBox.class); gwtScaffoldTestCase.assertGWTWidget(Button.class); gwtScaffoldTestCase.run(); } } If we run this test case then it outputs the following file: import com.google.gwt.core.client.EntryPoint; import com.google.gwt.user.client.ui.RootPanel; import com.google.gwt.user.client.ui.Label; import com.google.gwt.user.client.ui.TextBox; import com.google.gwt.user.client.ui.Button; public class HelloWorld implements EntryPoint { private Label label = new Label(); private TextBox textbox = new TextBox(); private Button button = new Button(); public void onModuleLoad() { RootPanel.get().add(label); RootPanel.get().add(textbox); RootPanel.get().add(button); } } Lets go over what has been written in the test case and its output. After we have talked about what's been written in the test, I will show what's underneath the hood. The HelloWorldGWTApplicationTestCase class instantiates a GWTScaffoldTestCase object and makes several calls to its assertGWTWidget() method passing several Widget (Component) classes, followed by run(). Specifically after the object is instantiated it calls its assertApplicationName passing a String argument - the GWT module name. The next three lines call the assertGWTWidget() method which passes in three GWT widgets. We now are asserting a GWT widget hierarchy which looks like the following. 1.0 RootPanel 1.1 Label 1.2 TextBox 1.3 Button From writing the test case we have killed two birds with one stone as we have written our test case first and when we run it, it writes the foundational structure of our Hello World GWT application. This really promotes developers, more than ever before, to write tests before writing production code as they get a treat for doing it ;-) Your development team will be classically conditioned that if they write tests then they know they get the treat - the foundation structure of their code. Underneath the hood it uses an ontological model to create the output - a template engine. The template engine I used is FreeMarker. The code for GWTScaffoldTestCase is show below: import java.util.*; import java.io.*; import com.google.gwt.user.client.ui.Widget; import freemarker.template.Configuration; import freemarker.template.DefaultObjectWrapper; import freemarker.template.Template; public final class GWTScaffoldTestCase { private final Map root = new HashMap(); private final List widgets = new ArrayList(); private final List imports = new ArrayList(); private final Configuration cfg = new Configuration(); private final File input, output; private Template template; public GWTScaffoldTestCase(File input, File output) { this.input = input; this.output = output; root.put("widgets", widgets); root.put("imports", imports); try { cfg.setDirectoryForTemplateLoading(this.input); template = cfg.getTemplate("test.ftl"); } catch (IOException e) { //Do something with the exception } cfg.setObjectWrapper(new DefaultObjectWrapper()); } public final void assertApplicationName(String name) { root.put("name", name); } public final void assertGWTWidget(Class<!--? extends Widget--> clazz) { widgets.add(new GWTComponent(clazz.getSimpleName())); imports.add(clazz.getName()); return this; } public final void run() { try { Writer out = new OutputStreamWriter(new FileOutputStream(output)); template.process(root, out); out.flush(); } catch (Exception e) { //Do something with the exception } } } The FreeMarker template looks like: import com.google.gwt.core.client.EntryPoint; import com.google.gwt.user.client.ui.RootPanel; <#list imports as import> import ${import}; <!--#list--> public class ${name} implements EntryPoint { <#list widgets as widget> private ${widget.className} ${widget.variableName} = new ${widget.className}(); <!--#list--> public void onModuleLoad() { <#list widgets as widget> RootPanel.get().add(${widget.variableName}); <!--#list--> } } I argue that Test Driven Scaffolding is superior to the scaffolding features of, for example, the Ruby on Rails (RoR) web application framework. RoR's scaffold features are not driven from tests but from a script which generates the necessary files where TDS is driven purely from a test case and provides a richer declarative model in a more agile manner. However, this example is just a basic, primitive example as is not able to cope with component composition (where components are nested), asserting event handling, multiple instances of the same component class nor provides post scaffold preservation (ensuring the asserted component hierarchy is preserved if the hierarchy has been modified). I will leave how one could get around this primitive example's limitation as a thought experiment. James Perry is a software developer special specializing in Nature Inspired Computational Intelligence using Google Web Toolkit, Wicket and deploying applications to Google App Engine. To find out more about myself visit my newly created site: http://www.natureinspiredcode.com

    Threaded Messages (12)

  2. Interesting. Not sure however if this can be done with reasonable effort.
  3. Please feel free to elaborate. IMO it's achievable as the ontological model (template engine) is deterministic as you program the relevant logic to produce the test driven scaffold. However I am not sure if it is achievable in a non deterministic approach using only a heuristic; which would provide great agility as it wouldn't need maintenance when your favourite web framework's API changes.
  4. I don't really see how this would reduce the amount of code you're actually writing. Either you write test cases that are sufficient to handle literally all possible inputs and scenarios, plus sufficiently intelligent templates to process all of the test cases, or it won't work. And so you basically end up with two complete implementations of your code - the tests that were used to generate, plus the code itself. And heaven forbid you need to change your test cases and regenerate the code 6 months into a project. All in all the effort is going to far outweigh any benefits.
  5. Let us take the classic TDD use case where developer writes test case first. At the end of the exercise the test case will be a non compiling class. For example : class FooTest{ public void testCreateFoo(){ Foo myFoo = new Foo("simpleFoo"); assertNotNull(myFoo); } public void testMakeFooSayHello(){ Foo myFoo = new Foo("simpleFoo"); assertNotNull(myFoo); String greeting = null; assertNotNull(greeting = myFoo.getGreeting()); assertTrue("Hello".equals(greeting)); } } Once this code is written to the eclipse ide, eclipse helps me create the perfect scaffold for Foo by giving intuitive options for creating Foo in the appropriate package, adding a constructor that accepts a string, and adding a getGreeting() method. This is a simpler example but the code helper feature has lot more powerful helpers to complete an outline. Now, I am perfectly sure that I am not the only one who knows this and there are thousands of developers write code this way. And I am sure James Perry knows this feature and much more. So can you please help me understand what am I missing here?, Why would we reinvent a wheel?
  6. Good question[ Go to top ]

    B_G, A great question and I hope my answer provides clarification. You are totally correct that the Eclipse IDE provides code generation features which are similar (as they use the ontological model) but there are subtle differences. Let me explain in more detail. In your code, you're correct that Eclipse will assist in generating the code with the needed constructor and with the getGreeting method but without an implementation of the method. It's fundamentally like an eclipse helper but with more sophistication. AFAIK, specifically most of the Eclipse helper will only generate enough code so it can compile your test case where Test Driven Scaffolding-based helper does more. The Eclipse IDE helpers use the same ontological model so I could create a Eclipse helper to provide the features of TDS; so your argument that I am re-invented the wheel is fundamentally sound but I would say I am evolving the code generation wheel. For example, can Eclipse create me a GWT or Wicket application from my test case? No it wouldn't as it wouldn't know how to create the widget (component) hierarchy, what level a widget is should be in nor assert if my widget hierarchy has been accidentally changed (post verification) after it generated the source code. Another example: if i wanted to write a test case for a servlet to ensure in my doGet() method that I retrieve a request parameter as a long, say using something like assertGetRequestParamAsLong("id"), Eclipse doesn't have a helper to create such code to retrieve the request parameter and to treat it as a long. So to summarize, yes it is exactly like the Eclipse IDE helpers, but with more sophistication. If I get the time I might write my TDS code as an eclipse helper. Cheers, James.
  7. writing only tests == writing only code[ Go to top ]

    If you only write the tests, then there's no cross-control on your code and the generated code will only be as good as your "tests are". The key in TDD is that you write a test, then you write the code, and the two need to agree. In your idea, the tests ARE the code, and would then require separate testing. What you've done is no more than change the programming paradigm, but you haven't gotten a more reliable solution. You've only changed the representation. This has already been attempted in languages such as Prolog. Where you write the rules for the program, and the language "writes the program for you". However, though the code in Prolog is often more succint than the Java equivalent it isn't necessarily easier, and it isn't necessarily better. Representing UI code in a declarative manner like HTML or as in old style Windows programming.. and yes even like the newer (uglier) XML representations has its advantages as long as you get to write less code. The main benefit is that the declared UI is simpler to reason about that lines and lines of imperative code. Your testing idea doesn't really reduce the code size, and it doesn't seem to increase reliability. So I don't see the point. (But maybe I just don't get it)
  8. Hi James, 1. I do believe, it should be possible to create a source code from a set of test cases, provided the test case is complete and has all the necessary scenarios captured. 2. But in a complicated application development,one would have so much of scripting to do which could highly complicate the generation process and need to have developer interventions to correct it. 3. Also, TDS would work better for GWT based application, may not work for other frameworks(which uses no java class for UI representation). Your views. Thanks, Senthil Balakrishnan
  9. Deduction vs Induction[ Go to top ]

    My view is that, when programming, one goes from specific examples to general concepts. It is an inductive, synthesis task. In testing, on the other hand, one goes from generic concept to specific examples. It is a deductive, analysis task. There is no sense in fully defining a concept by testing it, as much as there is no sense in testing a concept by defining it. These are complementary tasks.
  10. Re: Deduction vs Induction[ Go to top ]

    And that said, I don't see how could your tests generated program be capable of handling other (more general) situations than that given by the written tests.
  11. Not enough information[ Go to top ]

    I would love to make it easier to keep tests and code in sync, but I think your idea misses an important point: there is not enough information in a test case to be able to provide an implementation. The test class given by Boni Gopalan is a perfect example. getGreeting() returns "Hello", but where does it get it from? Does it load it from a database? Compute it from an encrypted string? Get it as input from the user? Concatenate it from some other strings? Who knows? In order to provide enough detail in the test case to have that information, you will basically be writing an implementation anyway. Even your example with the GWT is flawed. You are providing some boilerplate implementation, but is that the whole implementation of the method? Probably not. Is the boilerplate even correct? Perhaps there are customized subclasses of the Google widgets that are being used. There won't be an assertMyCustomLabelWidget() method in the test framework, so I'd have to stick to the test for a plain Label even though that is not what I will use in the implementation. I could see an argument being made about going the other way but I think that would be a bad idea as well, except perhaps for bootstrapping unit tests on legacy code. Tests and implementations have fundamentally different focuses, and I think trying to mix the two just makes a mess.
  12. The idea is definitely creative, but seems amateur.
  13. It won't work for the general case. The way I see tests in TDD is that they serve as some sort of requirements for the program. That's why you write them first: I want the program to do this and that. Then you write the implementation that would fulfill your requirements. And then the cycle repeats. What you are advocating is basically to let the developer write only the requirements and the computer somehow will come up with the implementation. Well, I don't think there's such technology today (this is definitely going in the realm of Artificial Intelligence), that's why I fundamentally think that it will not work. Now supposedly that we do find a domain where this might be useful, or where this approach will work. In this case I don't think you care about the implementation, or that you should be changing it. Because the moment the requirements change, or new ones go in effect you will have to write new test cases that will (re)generate the implementation. How will it know how to reconcile the new implementation with your changes? So in essence then your testcases become the new source code. And boy, they surely will need to be tested. What, test the testcases? Yes, because the way I see there are multiple logical what/how levels. The first level is probably at the business level: the business person will state "what" he wants to do, some architect will come up with the high-level solution (the "how"). Then the architect's output will be the "what" input to a development team that will come up with the more detailed design - the "how". In the end an individual developer will write the specific "what" using TDD and then the resulting implementation will be the "how". So you see that TDD is helping only at the lowest level, and to help with the other levels - that's where all these proliferated ALM solutions come in place.