Retroweaver: Compile 1.5 source to 1.4 bytecode

Discussions

News: Retroweaver: Compile 1.5 source to 1.4 bytecode

  1. Retroweaver is an open source tool that allows you to compile 1.5 source code down to 1.4 bytecode (and can then run in earlier VMs). Pre-beta 1.5 javac compilers support "-source 1.5 -target 1.4", but this will not be supported in the final release. How much will tools like this enable us to jump on the 1.5 bandwagon earlier?

    From Retroweaver home page

    Retroweaver is a bytecode weaver that enables you to take advantage of the new Java 1.5 language features, while still retaining total binary compatibility with 1.4 virtual machines. Retroweaver operates by transforming Java class files compiled by a 1.5 compiler into version 1.4 class files which can then be run on any 1.4 virtual machine.

    Visit the Retroweaver home page

    Threaded Messages (26)

  2. My first reaction was "Wow, I wonder how he handles annotations and enums, which use new bits in the 1.5 JVM".

    Answer: he doesn't.

    Not sure the tool has much point without those...

    --
    Cedric
  3. Retroweaver benefits[ Go to top ]

    I think there is still value in Retroweaver even though enums and annotations aren't supported. Just having Generics is huge for me and prevents a lot of runtime errors. Autoboxing, static imports, and extended for-loops are also nice things to have that you don't get with the 1.4 compiler. Retroweaver seems to support all of those.

    For me, this is great because companies are slow to embrace the new JDKs and you get mandates like "thou shalt use 1.4". Third party products that I depend on, eg WebSphere, may be slow to support 1.5 but I'd like to start coding to the JDK 1.5 compiler immediately, even if I can't use enums.
  4. My first reaction was "Wow, I wonder how he handles annotations and enums, which use new bits in the 1.5 JVM".

    Answer: he doesn't.

    Not sure the tool has much point without those...
    Cedric,

    1) Retroweaver supports generics, autoboxing/unboxing, static import, extended for loops, and varargs - I find that being able to use those features is a significant selling point.

    2) Enums - My hope is to add support for these in the next release. It's a little trickier than the other stuff, because you're dealing with something that requires support from java.lang.

    3) Annotations - I probably shouldn't have said that Retroweaver has "no support" for annotations. The way I use annotations, they all get read prior to run time. For example, the @overrides annotation is used at compile time. Annotations that I have developed myself are applied at an enhancement step prior to runtime. My intuition is that JDO vendors will read JDO 2.0 annotations in an enhancement step prior to runtime. This is all compatible with Retroweaver. Annotations at runtime aren't currently supported by Retroweaver, but it's possible that they will be supported in a later version... with some generation of synthetic members, some replacement of JDK1.5 runtime classes, and some other magic.

    God bless,
    -Toby Reyelts
  5. I agree, I don't care about enums, they're not a big deal. I can live without very easily. However, being able to use generics and being able to support JDK 1.4 with generics is great! Generics actually do help with the quality and robustness of the code so it will be great that they're "backported" to 1.4. Enums, annotations... who cares?
  6. "Generics actually do help with the quality and robustness of the code so it will be great that they're "backported" to 1.4. Enums, annotations... who cares?"

    Is that just another way to day "I can live with taking from C++, but not from C#" ? :)
  7. New bits in the JVM[ Go to top ]

    "...which use new bits in the 1.5 JVM"

    Oh, I forgot to mention that Retroweaver already handles cases where "new bits are in the 1.5 JVM". For example, the JVM spec has changed so that LDC/LDC_W can load CONSTANT_Class constants - those instructions are issued by the 1.5 compiler now, when you use a class literal. Retroweaver finds those uses and replaces them with boilerplate code similar to that generated by 1.4 compilers. Retroweaver also replaces uses of java.lang.StringBuilder, which is a new class that is only present in the 1.5 runtime. The work is not trivial.

    Check it out - I'd love to have your feedback.

    God bless,
    -Toby Reyelts
  8. New bits in the JVM[ Go to top ]

    ...The work is not trivial.

    Check it out - I'd love to have your feedback.

    Perhaps you should not use BCEL to make it trivial. :-)
  9. Not use BCEL?[ Go to top ]

    Perhaps you should not use BCEL to make it trivial. :-)

    Sorry, I don't understand what you mean. Are you saying that I should use some different bytecode engineering library than BCEL, an entirely alternative approach (i.e. implementing an entire compiler from scratch), or what?

    Implementing a compiler would have required much more effort than my current approach, and BCEL was a fine library for the bytecode manipulation.

    God bless,
    -Toby Reyelts
  10. Not use BCEL?[ Go to top ]

    BCEL is quite ugly and heavyweight. You should really consider to take a look at ASM framework.
  11. Use ASM?[ Go to top ]

    Hi Eugene,

    I took a look at ASM. The API looks nice and elegant - very reminiscent if an AST you'd build with a compiler. The problem is that it doesn't seem very suitable for the kind of job Retroweaver is doing. It's more geared towards the kind of work that you've been performing: read-only analysis of a class file.

    BCEL is geared more towards advanced transformation of existing bytecode. For example, being able to drop a regex into an InstructionFinder is invaluable. BCEL's automatic detection of lost targets on instruction deletion is also a life-saver.

    God bless,
    -Toby Reyelts
  12. Use ASM?[ Go to top ]

    I took a look at ASM. The API looks nice and elegant - very reminiscent if an AST you'd build with a compiler. The problem is that it doesn't seem very suitable for the kind of job Retroweaver is doing. It's more geared towards the kind of work that you've been performing: read-only analysis of a class file.

    BTW, there are tree package, which provides similar abstraction to BCEL for in-memory bytecode representation, e.g. in case if you need to iterate trough it several times.

    However even without tree you can do very advanced code generation. CGLIB2 project is a good exemple (actually it was converted from BCEL).

    BCEL is geared more towards advanced transformation of existing bytecode. For example, being able to drop a regex into an InstructionFinder is invaluable. BCEL's automatic detection of lost targets on instruction deletion is also a life-saver.

    As far as I remember BCEL, ASM it is even more easy. All targets are Label's, and because you access it by reference you just can't loose it. :-)
  13. Use ASM?[ Go to top ]

    CGLIB was "converted" for performance reasons and BCEL had some multithreading problems (I hope it is fixed). ASM is better for libraries at runtime, but I think BCEL is good for build tools too, BCEL verifier is one of good tools.
  14. Use ASM?[ Go to top ]

    ...ASM is better for libraries at runtime, but I think BCEL is good for build tools too, BCEL verifier is one of good tools.

    BCEL is quite overcomplicated. Sure it has number of the good ideas implemented, but everything colud be done on ASM too.

    I really wonder if someone want to port BCEL's bytecode verifier on ASM. :-)
  15. Use ASM?[ Go to top ]

    It possible to use both without problems, ASM for production and BCEL verifier for tests. It can be meaningfull to implement some better verifier, but I see no meaning to implement the same verifier with ASM just for better API design.
  16. Use ASM?[ Go to top ]

    It possible to use both without problems, ASM for production and BCEL verifier for tests. It can be meaningfull to implement some better verifier, but I see no meaning to implement the same verifier with ASM just for better API design.

    You said that. ASM is for production, and the only missing piece is control flow analyzer that can resolve stack states for given instruction. The use for such CFA/verifier is for deep analysis and transformation of the execution flow.

    BTW, trace and dump visitors are good enough for debugging purposes.
  17. .[ Go to top ]

    Thank you Toby. Great work.
  18. Believe it or not, I couldn't find that info on the net.

    Thanks
  19. It's not publicly available information. If you are part of the CAP program, Sun gives an estimate of the FCS release.

    I would imagine that, if you took a look at how long it historically takes Sun to go from beta1 to FCS, you'd have a rough idea of how long it would probably take them to do that for this release.

    God bless,
    -Toby Reyelts
  20. Generics at runtime[ Go to top ]

    Hi Toby

    Is there support for discovering the actual type arguments for generics at runtime? We would need this to support class fields processed with your tool in JDO Genie.

    Cheers
    David
    www.jdogenie.com - JDO Genie - High performance JDO O/R maping
  21. Generics at runtime[ Go to top ]

    Is there support for discovering the actual type arguments for generics at runtime? We would need this to support class fields processed with your tool in JDO Genie.

    Hi David,

    The Signature attributes should still be in the class file. I don't know if the 1.5 reflection API will return type parameters for a version 1.4 class file that has Signature attributes. If it does, you'll see the type signatures in the 1.4 class just as if it were a 1.5 class. If it doesn't, you can read the Signature attributes out of the class file yourself - same as the code I sent you earlier. That code works on any version class file.

    People could also just run Retroweaver as the very last step before deployment. This works ok as long as your enhancer isn't adding any calls into the 1.5 runtime beyond what a compiler would do.

    God bless,
    -Toby Reyelts
  22. 1.3.1 support?[ Go to top ]

    Hi Toby

    Any chance of this working with 1.3.1 VMs any time soon?

    Cheers
    David
    www.jdogenie.com - JDO Genie - High performance JDO O/R maping
  23. 1.3.1 support?[ Go to top ]

    Any chance of this working with 1.3.1 VMs any time soon?
    Hi David,

    That seems to be the most common question I get. I haven't tested it out yet, but since (AFAIK) the class file format and compiler dependencies on the runtime haven't changed since 1.2, Retroweaver should work fine for any VM between 1.2 and 1.4. You can give it a quick test by using a hexeditor to change the version of a class file that's been transformed by Retroweaver.

    If all goes well, I'll add a new target-vm-version option to Retroweaver in the next release.

    God bless,
    -Toby Reyelts
  24. Very interesting, and I'd be interested to hear people's experiences with this. I haven't been dieing to get the 1.5 features.. but then.. I haven't spent too much time on it since 1.5 isn't for our environment for a while. I guess now I have to go look a bit more closely..

    Good job!
  25. If this is both possible (without bugs) and desirable (seems to be), why would Sun disable that support in the final release?
  26. If this is both possible (without bugs) and desirable (seems to be), why would Sun disable that support in the final release?
    There are VM changes that take place in 1.5. For example, special support was added for serialization of enums, class literals, annotations, generics-based reflection, etc...

    I think that Sun wanted to motivate people to move up to the entire 1.5 platform. Unfortunately, that's just not viable for a large number of people.

    God bless,
    -Toby Reyelts
  27. If this is both possible (without bugs) and desirable (seems to be), why would Sun disable that support in the final release?
    Just thought I'd say that Neal Gafter (Sun's Java compiler writer) has explained the reasoning for this decision over at Sun's forums: http://forum.java.sun.com/thread.jsp?forum=316&thread=503547

    To sum it up, Sun doesn't have any plans to support a -1.4 target, primarily for legal reasons. They would have to draw up an entire specification, TCK, etc... for the 1.5->1.4 language. Neal additionally gave a word of support for Retroweaver.

    God bless,
    -Toby Reyelts