Testplan Usage Instructions

Note: this feature is still alpha status.

This feature allows writing a Testplan inside Gsheets or Excel, containing a list of Features, and scoring Regressions in Simscope against it.

1. Get User Simscope API token

If you don't already have an API token, get an API token.

2. Create testplan spreadsheet in Gsheets

Your testplan must have at least two columns (but can have more):

• Feature column (first column)
• Testcase column (last column)

Testcase column format

The Testcase column can contain a pattern to match tests against, or a regular expression

• Simple Example
  • alpha
• Regex Example
• alpha|beta

3. Download testplan from Gsheets to a .tsv file.

Note: this enables offline mode, where the worksheet is saved locally.

Inside Gsheets click:

• File
• Download
• Tab Separated Values (.tsv)

Copy this file into a Linux machine that can connect to Simscope.

4. Run the testplan script

Run the simscope-gsheets-testplan.py script, with:

• Testplan downloaded above.
• One or more regressions to score.

Note: you can run with --dry-run option, to score but skip publish results back into Simscope.

Single regression example (offline mode):

> path/bin/simscope-gsheets-testplan.py --addr http://simscope-hostname:1234 \
--testplan-tsv FILENAME.tsv --name my_regression_daily/1234

Multi-regression example (7-days). This example uses a single testplan, and scores multiple regressions against it.

> path/bin/simscope-gsheets-testplan.py --addr http://simscope-hostname:1234 \
--testplan-tsv FILENAME.tsv --name my_regression_daily --days=7 --component=cpu_top

After a successful upload, you should be able to open the regression in Simscope and see testplan results.