Why Jenkins does not detect Failures

Jenkins, the brilliant Continuous Integration build server, has a bit of a problem with the Maven surefire jUnit test plugin. Last sunday, I discovered that our Jenkins build server suddenly started ignoring test failures. While the logfile clearly states that the Unittests contain failures, Jenkins marks the builds as “stable”.

After some digging around, I found that even though Jenkins explicitly tells you in the logfile that it will fail the build, it will not do so if the Surefire XML reports are not generated. In our case, somebody in the team decided that the generation of the XML Surefire reports was taking too long and had disabled them in the Maven pom.xml.

In order to solve this, I re-enabled the XML reports and voila, Jenkins happily started reporting errors again. Here is the correct Surefire plugin configuration for you to use in your maven pom.xml file:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.8</version>
    <configuration>

      <!-- Please note that Jenkins needs Surefire
           XML reports in order for detection to work.
           Keep this property set to false. -->
      <disableXmlReport>false</disableXmlReport>
    </configuration>
</plugin>

4 Responses to Why Jenkins does not detect Failures

  1. Sybren says:

    Disabling the disableXmlReport option… some programmers just love double negatives. What’s wrong with enableXmlReport=true?

  2. Christoffer Hammarström says:

    You’re thinking too small, even better would be <report>xml</report>, or enabled(true/1/on/yes)

  3. Jonathan says:

    Thanks very much man! You made my day! I was having this problem but with Failsafe (a fork of Surefire) in Jenkins and I could make it work by adding a Post Build step to copy the reports from the acceptance-test module to the “root”/target directory. Thank you!!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s