Article: Using Javassist to transform methods in bytecode

Discussions

News: Article: Using Javassist to transform methods in bytecode

  1. Bytecode tweaking is getting into more and more projects. Javassist is a tool that enables bytecode manipulation, and this article walks you through some of the basics, and then an example of using Javassist to add interceptors via the bytecode.

    Javassist implements much looser compile-time checks on the code than required by the Java language specification, and an example is shown where you have to watch out for this.

    Read Using Javassist to transform methods in bytecode

    Threaded Messages (24)

  2. Question: why exactly would I dive into such a low level weaver when I can define interceptors easily with one of the existing AOP frameworks like Nanning, aspectwerkz, jboss etc.? That seems like building your own String class ...

    What is the reason for using javassist over those existing aop frameworks? Less dependencies? Smaller signature? Hmm - confused.
  3. Isn't Javassist exactly how some AOP frameworks are implemented?
  4. Yes but more precisely its how JBoss does its AOP.
  5. How can I able to implement this timing mechanism when I have existing object calling the method that I wanted to time afterall ?
  6. One of the concerns I have is what happens when the bytecode is weaved by more than one transforming product, I could easily see this creating a potential nightmare. If I only use say, AspectJ to weave byte code, that's one thing. But what happens when I have not only AspectJ modifying my byte code, but then on top of that, I have JDO, and then some other third-party product that also modifies my bytecode? Will it create potential bugs that will be virtually impossible to debug?
  7. isn't bytecode manipulation, just an alternative (in some situations, not all) to dynamic proxies?
  8. Javassist vs. Dynamic Proxies[ Go to top ]

    I've found creating objects at runtime using Javassist to be superior to using dynamic proxies.

    http://www.freeroller.net/page/hlship/20030708

    It's not an overwhelming difference, but it is appreciable.

    Since then, I've done some more interesting things. By doing the work at runtime, you can often code against classes (i.e. the invokevirtual bytecode) rather than coding against interfaces (i.e., the invokeinterface bytecode).

    http://javatapestry.blogspot.com/2003_09_07_javatapestry_archive.html#106329208189926255
  9. Bytecode manipulation is evil[ Go to top ]

    a
  10. Bytecode manipulation is evil[ Go to top ]

    I am convinced that bytecode manipulation is about the most stupid
    idea in the history of the JVM (or the Java Language). Considering that
    the whitebox re-use created by java code generating AOP frameworks like
    AspectJ is a potential nightmare in larger projects, bytecode manipulation
    will create even more clutter and interdependencies. Besides I wonder how
    bytecode manipulation in projects will ever survive a code review :-).
     
    One of my main concerns, apart from lack of readibility, maintainability etc. is that there is potentially a strong dependency between tool versions and jvm
    versions that is far higher than with the java language alone. As an example, download HyperJ (originally engineered for jdk1.1 and 1.2 I believe) and run it using jdk1.3 or even better run it on a couple of classes that were created with _different_ jdks and for _different_ jvm versions.

    Next thing, we will be told, that it might be a good idea to directly change some of the object code in the jvm itself. If you want to do peek and poke, go fetch your old Commodore 64 from the attic!

    No offense, Karl
  11. Bytecode manipulation is evil[ Go to top ]

    Hi Karl,

    I have to strongly disagree with your stance.

    The first flaw in your argument is that there is some large gap between the Java language and Java bytecode. If you study the JVM spec, you will see that the mapping between the two is almost one for one.

    The second flaw is that you assume that Java bytecode is much more prone to changes then the Java language. If you look, the JVM spec hasn't changed in many years. Our latest and greatest JVMs can still run ancient Java bytecode. Neal Gafter (Sun's compiler writer) recently demonstrated Sun's committment to an unchanging VM interface when he stated that upgrading invokevirtual to perform covariant dispatch was not an option. If HyperJ had problems with different JVM versions, then that is a problem of that particular aspecting system, not bytecode enhancement in general.

    The third flaw is that you assume that arbitrary transformations are written in bytecode manipulating programs. Rather, people choose to use aspects when OOP techniques don't provide satisfactory solutions. As an example, consider JDO. It implements a persistence aspect using bytecode manipulation. General OOP techniques just don't cut it for persistence - and we're learning as a community how, in general, aspects are much more applicable for applying cross-cutting services.

    You might want to gain a thorough understanding of Java bytecode and the JVM. It will help you understand the implementation of aspecting systems and their usefulness and applicability in general.

    God bless,
    -Toby Reyelts
  12. Bytecode manipulation is evil[ Go to top ]

    I feel that bytecode manipulation is evil.

    I don't know anything of AOP. But if you want AOP, shouldn't Java support it directly instead of using some frameworks and bytecode manipulation instead?

    I want that the source and the bytecode basically contain the same information.
  13. Bytecode manipulation is evil[ Go to top ]

    I don't know anything of AOP. But if you want AOP, shouldn't Java support it directly instead of using some frameworks and bytecode manipulation instead?

    For every problem to be solved, there is probably an optimal language out there waiting to be designed to solve that problem. Unfortunately, we live in reality (well, some of us anyway), and reality tells us that language design is not simple and is the results of lots of tradeoffs. Eventually, yes, I'd like to see some form of first-class, language-level typing support that provides for aspecting capabilities. I doubt that we will see a mainstream language gain that kind of capability for a long time to come.

    I want that the source and the bytecode basically contain the same information.

    I guess you hate compiled languages then?

    God bless,
    -Toby Reyelts
  14. Bytecode manipulation is evil[ Go to top ]

    I want that the source and the bytecode basically contain the same information.


    It is not bytecode manipulation problem, javac uses this evil way too. It is possible to generate/transform source code and use javac to compile garbage, but I see nothing better in this way.
  15. Bytecode manipulation is evil[ Go to top ]

    Hi Karl,

    >
    > I have to strongly disagree with your stance.
    >
    > The first flaw in your argument is that there is some large gap between the
    > Java language and Java bytecode. If you study the JVM spec, you will see that > the mapping between the two is almost one for one.

    I don't think that I assumed that somewhere. What I assumed is that I want to be able to perform code audits. I want to understand from a given set of artifacts what is going on. The more and the more diverse artifacts the harder is auditing.

    >
    > The second flaw is that you assume that Java bytecode is much more prone to
    > changes then the Java language. If you look, the JVM spec hasn't changed in
    > many years. Our latest and greatest JVMs can still run ancient Java bytecode.
    >

    Now that is nice. The problem here is: Will the old bytecode enhancement systems and the new java compiler work together to produce output runnable by the new runtime environment. As mentioned, this was clearly not the case with a tool like hyperj and I am convinced it will be a problem with other tools as well, because they use shortcuts and "holes" in the JVM specifications that may or may not work in a later version.

    >
    > The third flaw is that you assume that arbitrary transformations are written > in bytecode manipulating programs. Rather, people choose to use aspects when > OOP techniques don't provide satisfactory solutions. As an example, consider > JDO. It implements a persistence aspect using bytecode manipulation. General > OOP techniques just don't cut it for persistence -
    >

    One might argue with that claim. I find it striking, the still by far best AOP framework available, AspectJ, does not use bytecode manipulation but produces nice and clean intermediary java code. In my opionion that is the way to go. One of the reason, of course, being trust: I trust Sun or IBM to provide a JVM implementation that does and only does what it is intended to do. I don't necessarily trust company xyz to do that - although I may use their code.

    > You might want to gain a thorough understanding of Java bytecode and the JVM.

    As I said, aspect systems are interesting but bytecode manipulation is by no means necessary for that. And of course whether aspecting systems should ever be used without getting hold of the original sourcecode is a matter of serious debate. After all, clean and predictable AOP usage requires full white box knowledge of the internal of the objects the aspects act on.


    Peace, Karl
  16. Bytecode manipulation is evil[ Go to top ]

    Hi Karl,

    > >
    > > I have to strongly disagree with your stance.
    > >
    > > The first flaw in your argument is that there is some large gap between the
    > > Java language and Java bytecode. If you study the JVM spec, you will see that > the mapping between the two is almost one for one.
    >
    > I don't think that I assumed that somewhere. What I assumed is that I want to be able to perform code audits. I want to understand from a given set of artifacts what is going on. The more and the more diverse artifacts the harder is auditing.

    Byte code manipulation helps for this task, you do not need to read generated code to understand what is going on. And it is much more easy to audit "aspect", not dublicated or generated code.


    >
    > >
    > > The second flaw is that you assume that Java bytecode is much more prone to
    > > changes then the Java language. If you look, the JVM spec hasn't changed in
    > > many years. Our latest and greatest JVMs can still run ancient Java bytecode.
    > >
    >
    > Now that is nice. The problem here is: Will the old bytecode enhancement systems and the new java compiler work together to produce output runnable by the new runtime environment. As mentioned, this was clearly not the case with a tool like hyperj and I am convinced it will be a problem with other tools as well, because they use shortcuts and "holes" in the JVM specifications that may or may not work in a later version.
    >


     Byte code is verified by JVM before to execute, it can be produced by javac or any "evil" tool, but it will be verified and executed the same way.



    > >
    > > The third flaw is that you assume that arbitrary transformations are written > in bytecode manipulating programs. Rather, people choose to use aspects when > OOP techniques don't provide satisfactory solutions. As an example, consider > JDO. It implements a persistence aspect using bytecode manipulation. General > OOP techniques just don't cut it for persistence -
    > >
    >
    > One might argue with that claim. I find it striking, the still by far best AOP framework available, AspectJ, does not use bytecode manipulation but produces nice and clean intermediary java code. In my opionion that is the way to go. One of the reason, of course, being trust: I trust Sun or IBM to provide a JVM implementation that does and only does what it is intended to do. I don't necessarily trust company xyz to do that - although I may use their code.
    >

    Looks like AspectJ compiles code too.

    > > You might want to gain a thorough understanding of Java bytecode and the JVM.
    >
    > As I said, aspect systems are interesting but bytecode manipulation is by no means necessary for that. And of course whether aspecting systems should ever be used without getting hold of the original sourcecode is a matter of serious debate. After all, clean and predictable AOP usage requires full white box knowledge of the internal of the objects the aspects act on.
    >
    I agree, bytecode manipulation not necessary for code generation, but it is a better way to implement some of tools.

    >
    > Peace, Karl
  17. Bytecode manipulation is evil[ Go to top ]

    Byte code manipulation helps for this task, you do not need to read generated >code to understand what is going on. And it is much more easy to >audit "aspect", not dublicated or generated code.


    Yeah, but sooner or later you will hit the wall, where you desparately need to know what your system is actually doing under the hood. The classic example is a bug that you need to solve because the vendor keeps you on hold or just an all to common documentation bug (or simply missing documentation). Lucky you are, if you browse through the java code, the jsp/ejb/aop compiler have generated and do not need to read bytecode :-).

    BTW: How does manipulated bytecode work with decompilers? Is there a clean possibility to de-aspectize your code again?
  18. Bytecode manipulation is evil[ Go to top ]

    Byte code manipulation helps for this task, you do not need to read generated >code to understand what is going on. And it is much more easy to >audit "aspect", not dublicated or generated code.

    >
    > Yeah, but sooner or later you will hit the wall, where you desparately need to know what your system is actually doing under the hood. The classic example is a bug that you need to solve because the vendor keeps you on hold or just an all to common documentation bug (or simply missing documentation). Lucky you are, if you browse through the java code, the jsp/ejb/aop compiler have generated and do not need to read bytecode :-).
    >

    Yes, It is true for any tool, but this problem doe's not exist in opensource tools. It doe's not mean you need to learn tool implementation, it takes a few hours in open source project to fix this kind of problems, but I have never saw
    "ClassFormatError" in production, byte code is very trivial and it is very trivial to test and to verify it (it is trivial for machine)



    > BTW: How does manipulated bytecode work with decompilers? Is there a clean possibility to de-aspectize your code again?

    It must be possible, but I do not think decompilers can decompile code compiled by "unknown" compiler. I use javap for debuging, but there are a lot of tools.
  19. Bytecode manipulation is NOT evil[ Go to top ]

    What I assumed is that I want to be able to perform code audits.

    I don't understand this. You perform audits on the aspecting code, just as if you were to perform audits on any of your other code.

    The problem here is: Will the old bytecode enhancement systems and the new java compiler work together to produce output runnable by the new runtime environment.

    Yes. Any aspecting system using "shortcuts or holes in the JVM spec" is plainly a bad aspecting system.

    I find it striking, the still by far best AOP framework available, AspectJ, does not use bytecode manipulation

    I find it striking that I haven't been able to use AspectJ to solve any of the aspecting problems I need to solve, because I have always had to work on class files and not on source code.

    I don't necessarily trust company xyz to do that

    Company XYZ isn't providing a JVM implementation. A closer analogy is that they are providing a kind of compiler/code generator.

    bytecode manipulation is by no means necessary for that

    It is for the kind of aspects that I have written up to this point in time. Apparently, it's also quite convenient for any transparent aspects like persistence, transactions, and security.

    And of course whether aspecting systems should ever be used without getting hold of the original sourcecode is a matter of serious debate. After all, clean and predictable AOP usage requires full white box knowledge of the internal of the objects the aspects act on.

    I think you should avoid making such strong declarations contrary to existing evidence. Your statement is comparable to saying that all of these companies that are writing bytecode-enhancing JDO implementations can't possibly be doing it "cleanly and predictably" unless they have access to all of their clients' source code.

    Yeah, but sooner or later you will hit the wall, where you desparately need to know what your system is actually doing under the hood.

    You can always examine the bytecode if absolutely necessary - even pass it through a decompiler if you prefer gazing at source code. In any case, this is a red herring. For example, if I were to use a vendor's persistence engine that uses reflection to read/write the private data members of my class, it would be just as difficult for me to track down a problem (even moreso because of copyright/eula agreements) than if that engine enhanced my own classes. At least if my own classes are enhanced, I have the legal right to look through the bytecode and decompile them.

    BTW: How does manipulated bytecode work with decompilers? Is there a clean possibility to de-aspectize your code again?

    You don't need to "de-aspectize" code - you just stop aspecting it. Any half decent decompiler will be able to reverse-engineer code which has been enhanced or generated by an aspecting system - even when the bytecode couldn't have possibly come from Java source code.

    God bless,
    -Toby Reyelts
  20. Bytecode manipulation is NOT evil[ Go to top ]

    I think you should avoid making such strong declarations contrary to existing evidence. Your statement is comparable to saying that all of these companies that are writing bytecode-enhancing JDO implementations can't possibly be doing it "cleanly and predictably" unless they have access to all of their clients' source code.

    Read my statement carefully. I nowhere claim that. What I claim is that you need full blown whitebox knowledge about the classes you apply the aspects on - at least if they are not absolutely trivial aspects. Please don't tell me you have done that and it works for you. Because if it does, it is out of luck, plain and simple. Or because you may use it for a trivial aspect, for example performance timings, logging and the like. Even when doing persistence you will run in serious problems once you apply it on classes you don't know the internals of.

    You don't need to "de-aspectize" code - you just stop aspecting it. Any half decent decompiler will be able to reverse-engineer code which has been enhanced or generated by an aspecting system - even when the bytecode couldn't have possibly come from Java source code.

    Interesting statement. But obviously it can't be true. How will any decompiler be able to deduce which changes to the class hierachy or control flow were due to any one of the aspecting systems used and which were due to original source code.
    What makes you think you were the only one using aspects in the first place?
    Especially if a library passed through a number of hands.
    Even more interesting question: Will you have the legal right to apply aspects on compiled classes you may have bought? I would at least assume that you will lose all vendor support once you do that. After all this is like putting in a number of cables into a piece of computer hardware, isn't it?
  21. Bytecode manipulation is NOT evil[ Go to top ]

    Read my statement carefully. I nowhere claim that. What I claim is that you need full blown whitebox knowledge about the classes you apply the aspects on - at least if they are not absolutely trivial aspects. Please don't tell me you have done that and it works for you. Because if it does, it is out of luck, plain and simple. Or because you may use it for a trivial aspect, for example performance timings, logging and the like. Even when doing persistence you will run in serious problems once you apply it on classes you don't know the internals of.

    >
    Looks like you are trieng to solve not existing problems.
    Have you ever used some evil tool ? Do you have any serious problems with it ?
  22. Non existing problems....[ Go to top ]


    > Looks like you are trieng to solve not existing problems.
    > Have you ever used some evil tool ?
    > Do you have any serious problems with it ?
    >
    The technology that can solve problems can also create them. Now the
    question is: Does it solve more problems than it creates? If it does,
    is useful, if it does not, it is evil. Of course the product does not
    create the problem "by itself". Most of the time it is how people use
    it. Just to give an example: You have some class hierachy in bytecode
    only, bought from some vendor. Now, you *could* use aspects and bytecode
    manipulation to make these objects support transparent persistence. But *should* you? Judging from my project experience the answer is

    (a) no, this will probably create problems you will not be
    able to foresee nor manage
    (b) it will be done anyway, because it can be done and will cut
    development time short

    You might guess that I am in favor of gun control ;-).
    Sure a gun never killed nobody but given to incompetent
    or evil or nutty(!) people it can - and usually will -
    do a lot of harm. Likewise I am in favour of tool control.
    Overhyping aspects and putting it on the cashier
    shelves in tool supermarkets will have about the same
    effect than selling guns at the Home Depot
    checkout: Bloodbath!
  23. Non existing problems....[ Go to top ]


    > > Looks like you are trieng to solve not existing problems.
    > > Have you ever used some evil tool ?
    > > Do you have any serious problems with it ?
    > >
    > The technology that can solve problems can also create them. Now the
    > question is: Does it solve more problems than it creates? If it does,
    > is useful, if it does not, it is evil. Of course the product does not
    > create the problem "by itself". Most of the time it is how people use
    > it. Just to give an example: You have some class hierachy in bytecode
    > only, bought from some vendor. Now, you *could* use aspects and bytecode
    > manipulation to make these objects support transparent persistence. But *should* you? Judging from my project experience the answer is
    >
    > (a) no, this will probably create problems you will not be
    > able to foresee nor manage
    > (b) it will be done anyway, because it can be done and will cut
    > development time short

    I have never used transformations for "secret" classes and I do not have any problems with this use case. Do you have this problem ?.
    Good usage example is decorator for JDBC driver, dynamicaly generated decorators doe's not depend on JDBC version (some popular pool implementation uses generated decorators without problems ). Standart JAVA libraries use this evil technology too. See java.lang.reflect.Proxy, does it means JDK is evil too ?

    >
    > You might guess that I am in favor of gun control ;-).
    > Sure a gun never killed nobody but given to incompetent
    > or evil or nutty(!) people it can - and usually will -
    > do a lot of harm. Likewise I am in favour of tool control.
    > Overhyping aspects and putting it on the cashier
    > shelves in tool supermarkets will have about the same
    > effect than selling guns at the Home Depot
    > checkout: Bloodbath!
  24. Bytecode manipulation is NOT evil[ Go to top ]

    Read my statement carefully. I nowhere claim that.

    There must be some misunderstanding in terms here, because "white-box" means that you essentially have access to the source code of a component - as opposed to "black-box", which means only the interface is available to you.

    Please don't tell me you have done that and it works for you.

    At this point, I don't know what to say, except that there is a major communication disconnect here, or you just plain don't know what you're talking about.

    Interesting statement. But obviously it can't be true.

    We're not talking apples and oranges here. If aspected classes are delivered to a customer as a library, that customer won't ever "de-aspect" those classes, and there's no reason for the customer to do so - ever. If, on the other hand, I provide an aspect to a customer, then the customer can apply that aspect to his own classes. If he wishes to see exactly how my aspect modifies his classes, he can always examine or decompile the modified bytecode. If he wishes to remove my aspect from his classes, he simply stops applying the aspect.

    What makes you think you were the only one using aspects in the first place?
    Especially if a library passed through a number of hands.


    I'm not sure I understand your point. If you're concerned about aspects clashing with each other, it isn't an issue. When I write my aspects, they always assume that the user code is arbitrary beyond what the aspect requires (i.e. public constructor or some such mess). I think if you wrote some aspects yourself, you'd understand this. Sometimes I even use aspects in a complementary fashion. For example, I use my vendor's JDO bytecode enhancer to enhance my classes, then I run an aspect against those classes that take advantage of the newly available persistence aspect.

    Will you have the legal right to apply aspects on compiled classes you may have bought?

    I can see how modifying the binary code of a vendor's library might have certain implications as far as support is concerned, but that's not the primary usecase for aspects.

    You have some class hierachy in bytecode only, bought from some vendor.

    I think you're way too focused on one particular usecase of aspects - applying them to code you don't own. That is one valid use case, and I've made use of it - but more for debugging and patching. For example, I've patched a JDO library so that it is compatible with JDK 1.5, I've patched Sun's javax.comm.* library when there was no other workaround, and I'm in the midst of writing an aspect that will detect leaks of any resource whatsoever (i.e. Oracle JDBC driver, java.nio.Channel, javax.swing.JFrame, etc...). These all require me to apply an aspect to bytecode that I don't own, but it's a valid application of the technique.

    On the other hand, the much more common use of aspects will be either where

    a) I develop an aspect myself and apply it to my own classes or

    b) a vendor develops and provides me with an aspect which I then apply myself to my own classes

    Again, I think if you actually sit down and use some aspects yourself, these things will become more obvious to you.

    God bless,
    -Toby Reyelts
  25. Bytecode manipulation is NOT evil[ Go to top ]

    use some aspects yourself

    That should read develop some aspects yourself

    God bless,
    -Toby Reyelts