Discussions

News: PIT 0.19 Mutation testing for Java released

  1. PIT 0.19 Mutation testing for Java released (7 messages)

     

    PIT is an Open Source mutation testing system for Java that makes it possible to test the effectiveness of your unit tests.

    You can thinkof mutation testing as either as an automated test of your tests, or as a much more in depth form of code coverage.

    Unlike traditional line and branch coverage tools PIT does not just confirm that your tests execute your code, it confirms that your tests are actually able to detect faults in it.

    Mutation testing is conceptually quite simple. Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived.

    The quality of your tests can be gauged from the percentage of mutations killed.

    Mutation testing is an computationally expensive process and has not been widely used outside of academia, but PIT aims to make it a practical technique for real codebases by concentrating on both performance and ease of use.

    Threaded Messages (7)

  2. Broken, please fix![ Go to top ]

    This program does not build under maven owing to unsatisfied dependencies:

    ????equalsverifier.jar 0.6.3

    Please inform the submitter/author as there is no contact info on the home page . . . HTH.

  3. Broken, please fix!, additional[ Go to top ]

    Hmm, seems to run & do nothing, shame there's no forum, mailing list, IRC . . . 

    $ java -cp /home/super/Downloads/pitest-0.19-withdeps.jar:/home/super/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/super/.m2/repository/cglib/cglib-nodep/2.2/cglib-nodep-2.2.jar:/home/super/.m2/repository/com/sun/jersey/jersey-client/1.2/jersey-client-1.2.jar:/home/super/.m2/repository/com/sun/jersey/jersey-core/1.2/jersey-core-1.2.jar:/home/super/.m2/repository/com/sun/jersey/jersey-server/1.2/jersey-server-1.2.jar:/home/super/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-core/1.2/jersey-test-framework-core-1.2.jar:/home/super/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-inmemory/1.2/jersey-test-framework-inmemory-1.2.jar:/home/super/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/home/super/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/super/.m2/repository/commons-io/commons-io/1.4/commons-io-1.4.jar:/home/super/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/home/super/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/home/super/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/home/super/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar:/home/super/.m2/repository/net/sourceforge/cssparser/cssparser/0.9.5/cssparser-0.9.5.jar:/home/super/.m2/repository/net/sourceforge/htmlunit/htmlunit/2.8/htmlunit-2.8.jar:/home/super/.m2/repository/net/sourceforge/htmlunit/htmlunit-core-js/2.8/htmlunit-core-js-2.8.jar:/home/super/.m2/repository/net/sourceforge/nekohtml/nekohtml/1.9.14/nekohtml-1.9.14.jar:/home/super/.m2/repository/org/apache/httpcomponents/httpclient/4.0.1/httpclient-4.0.1.jar:/home/super/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar:/home/super/.m2/repository/org/apache/httpcomponents/httpmime/4.0.1/httpmime-4.0.1.jar:/home/super/.m2/repository/org/apache/james/apache-mime4j/0.6/apache-mime4j-0.6.jar:/home/super/.m2/repository/org/easymock/easymock/3.0/easymock-3.0.jar:/home/super/.m2/repository/org/glassfish/javax.servlet/3.0-b66/javax.servlet-3.0-b66.jar:/home/super/.m2/repository/org/glassfish/extras/glassfish-embedded-all/3.1/glassfish-embedded-all-3.1.jar:/home/super/.m2/repository/org/objenesis/objenesis/1.2/objenesis-1.2.jar:/home/super/.m2/repository/org/w3c/css/sac/1.3/sac-1.3.jar:/home/super/.m2/repository/xalan/serializer/2.7.1/serializer-2.7.1.jar:/home/super/.m2/repository/xalan/xalan/2.7.1/xalan-2.7.1.jar:/home/super/.m2/repository/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar:/home/super/.m2/repository/xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar org.pitest.mutationtest.MutationCoverageReport --reportDir pitreport --inScopeClasses uk.me.doitto.* --targetClasses uk.me.doitto.* --sourceDirs /home/super/Documents/projects/webapp-git/src11:40:51 PIT >> INFO : Dependency analysis reduced number of potential tests by 0Installing PIT agent

    SLAVE :  : 11:40:52 PIT >> FINE : Input file is /tmp/1308912051752048734145586815763074512019505081304.data
    SLAVE :  : 11:40:54 PIT >> INFO : Running 0 units
    SLAVE :  : 11:40:54 PIT >> INFO : Finished
    11:40:54 PIT >> INFO : Calculated coverage in 3 seconds.11:40:54 PIT >> INFO : Created  0 mutation test units11:40:54 PIT >> INFO : Running 0 units11:40:55 PIT >> INFO : Finished11:40:55 PIT >> INFO : Completed in 4 seconds.  Tested 0 classes.

  4. Author notified[ Go to top ]

    Any mods around?  These messages are now superfluous and can be deleted

  5. Broken, please fix!, additional[ Go to top ]

    Hi Ian,

    It sounds like the equals verifier repository was down for some reason. I've just checked I can access via web browser so the issue may be resolved.

    Regarding your second issue - this sounds like a classpath problem. PIT scans the classpath it is launched with for tests to run and classes to mutate that match the supplied filters. Looking at the classpath you are using I can't see your actual code in there, just your dependencies, which may explain the problem. If you add in the paths to your code and tests it should work.

    As your project looks to be built with maven it would however be far easier to use the maven plugin instead of the command line tool. If you are still having issues building it from source you should be able to install it using the released binaries and poms.

    mvn install:install-file -Dfile=pitest-0.19.jar -DpomFile=pitest-0.19.pom.xml
    mvn install:install-file -Dfile=pitest-maven-0.19.jar -DpomFile=pitest-maven-0.19-pom.xml

     

     

  6. No available from maven central[ Go to top ]

    PIT is now available from maven central.

  7. Is it different from Jester?[ Go to top ]

    I remember a similar project on SF:

    http://jester.sourceforge.net/

    Is PIT trying to achieve the same goal?

  8. Is it different from Jester?[ Go to top ]

    PIT isn't the first mutation testing system for java, there are several others :

    Jester

    Jumble

    Mu Java

    But PIT aims to improve on these systems, mainly in terms of performance and ease of use.

    Jester and Mu Java use source based mutation, which makes them pretty slow as they have to recompile the code for each mutation. Mu Java is also unable to mutate code with generics in making it useless for most codebases.

    Jumble is a byte code mutation system like PIT, but is less sophisticated in the way it chooses tests to run against a mutated class (PIT uses line coverage information to pick individual test cases from an enitre suite, Jumble uses all the test cases in a test class that matches a naming convention). As a result PIT runs much faster than Jumble and is also able to measure the coverage of entire test suites rather than individual test classes.

    So yes, PIT is trying to achieve the same goal as Jester, but they are very different systems.