JUnit XML Parser (Jenkins)

The Tunnel client includes a Python-based JUnit XML report (Jenkins) converter/publisher.

> simscope_junit.py

This enables converting test results from junit XML → Simscope Job JSON, and publishes the regression and jobs to the Tunnel (RabbitMQ).

• This script is based on Jenkins usage, but also works with other JUnit implementations.

Running Simscope JUnit

To run the Simscope JUnit script, you need:

• REGR.json — Regression metadata, in JSON format
• JUNIT.xml — JUnit XML result file
• REGR_NAME — name of the regression to store results into Simscope as

# Replace variables: REGR.json, JUNIT.xml, and REGR_NAME
> tunnel/rabbitmq/simscope_junit.py REGR.json JUNIT.xml --regression=REGR_NAME

# Here is an example with the variables replaced
> tunnel/rabbitmq/simscope_junit.py smoke_123.json jenkins_smoke_123.xml --regression=smoke_run/123

Debug mode

To help with debugging, you can run the script with --debug option, which will parse and convert the XML into JSON format, and print the JSON to stdout, instead of publishing to RabbitMQ.

> tunnel/rabbitmq/simscope_junit.py --debug smoke_123.json jenkins_smoke_123.xml --regression=smoke_run/123

Example JUnit XML

For reference, this is an example JUnit XML report file.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
   <testsuite name="JUnitXmlReporter.constructor" errors="0" skipped="1" tests="3" failures="1" time="0.006" timestamp="2020-05-24T10:23:58">
      <properties>
         <property name="java.vendor" value="Sun Microsystems Inc." />
         <property name="compiler.debug" value="on" />
         <property name="project.jdk.classpath" value="jdk.classpath.1.6" />
      </properties>
      <testcase classname="JUnitXmlReporter.constructor" name="should default path to an empty string" time="0.006">
         <failure message="test failure">Assertion failed</failure>
      </testcase>
      <testsuite name="JUnitXmlReporter.constructor" errors="0" skipped="1" tests="2" failures="0" time="0.000" timestamp="2020-05-24T10:23:59">
         <testcase classname="JUnitXmlReporter.constructor" name="should default consolidate to true" time="0">
            <skipped />
         </testcase>
         <testsuite name="JUnitXmlReporter.constructor" errors="0" skipped="0" tests="1" failures="0" time="0" timestamp="2020-05-24T10:23:57">
            <testcase classname="JUnitXmlReporter.constructor" name="should default useDotNotation to true" time="0" />
         </testsuite>
      </testsuite>
   </testsuite>
</testsuites>

Example Simscope Job JSON output

The simscope_junit.py script converts the above JUnit XML to Simscope JSON jobs.

Based on the XML above, here is an example JSON published into Simscope:

{
    "testgroup": "JUnitXmlReporter.constructor",
    "category": "sim",
    "config": "should default path to an empty string",
    "start_time": "2020-05-24T15:23:58+00:00",
    "compute_ms": 6,
    "build": null,
    "result": "fail",
    "fail_message": "Assertion failed: test failure"
}