Simscope Release History — v290 — 299

1.299

1. `feature` Test tagging, by build and testgroup

BETA

This has a first beta implementation of Job Tagging, which enables grouping of tests by tags, for statistics and filtering purposes.

→ See Job Tags for more details.

2. `feature` simscope-import-changelists

Added simscope-import-changelists script to manually import changelists.

1.298

1. `feature` Added bug autorefresh

Simscope can now automatically trigger bug auto-refresh, based on the interval set in your simscope.config file.

For example, this refreshes every two hours:

[bugs]
# If this is set, automatically refresh all bugs after this amount of time
# NOTE: this works by calling the 'refreshplugin' script
autorefresh = "2h"

→ See Simscope JIRA for more details.

2. `bugfix` Job metadata scrolling

The previous version had an issue where job metadata table would not scroll.

1.297

This release contains bugfixes for Internet Explorer.

1.296

1. `feature` Regression notification CC list

Regression Watch now supports notification CC emails (in addition to the owner of the watch). Users can be notified either:

Daily summary
Immediately, if any regression fails

2. `feature` JIRA Cloud support

The Simscope JIRA APIs have been rewritten, and now work with both JIRA standalone and JIRA Cloud (ie from atlassian.net).

→ See Simscope JIRA for instructions on installing/configuring JIRA with Simscope.

1.295

1. `feature` Gantt Chart

Simscope can now render basic Gantt Charts for a regression.

Example Gantt chart for a regression:

Gantt Charts can be grouped by:

Test config (default)
By Build
By Test group

And sorted by:

Start time (default)
End time
Duration

2. `feature` Simscope GitHub diff now supports multiple projects

If you are using multiple projects inside of GitHub, the plugin now redirects properly from Regression → GitHub project.

See the simscope-regr-github script.

1.294

1. `bugfix` Regression coverage charts filtering with NOT filters fixed

For example, filtering coverage for !Line is now working again.

The previous version would filter out all regressions containing Line, which is incorrect.

2. `feature` Error Log in Admin GUI

For convenience, the latest Simscope [WARN]ings and [ERROR]s are now shown in the Simscope Admin area. This lets administrators view errors without needing a console/log view.

1.293

When filtering regressions by model version (ie changelist), you can now disambiguate across projects with the format:

PROJECT--CHANGELIST

For example, if there is a changelist 1001 in both the GPU and CORE projects, you can specify either:

CORE--1001
GPU--1001

Note: searching by CHANGELIST alone still works.

1.292

(advanced) This contains bugfixes for ambiguous models with the same ID.

If a user filters regressions by a model ID, and it matches the same ID across two projects, the error message now indicates the project and branch.

Here is an example for model 12345, matching project GPU and Alpha:

"error": "ambiguous changelist ID [12345] matches multiple projects:
[GPU/master] and [Alpha/master]"

1.291

1. Filter regressions by 'aborted'

Regressions can now be filtered to show only aborted regressions.

Tip: you can save a Regression Search with this checked, and get a daily (or instant email) for aborted regressions.

2. Added regular expression (regex) docs

See Regex page.

1.290

1. New JSON message type for Rabbit and Tunnel: regr-update

The regr-update message is an optional JSON message, which allows regression metadata fields to be updated after a regression is started.

The regr-update message can change most of the Regression fields.

You only need to send the fields to be changed. Blank fields will remain the same. For example, this JSON updates both the Jobs Submitted or Jobs Running fields:

{
  "name": "myregr/36",
  "submitted": 1,
  "running": 41
}

You can also update the custom metadata fields like this:

{
  "name": "myregr/1000",
  "custom_metadata": [
    ["Testpath", "/foo/bar/baz.txt"],
    ["Final Cov", "38.2"]
  ]
}

Instructions to use this feature:

Upgrade simscope to version 1.290 or higher.
Upgrade simscope-tunnel to version 1.19 or higher.
Use the latest simscope_tunnel_api.py
Call simscope_tunnel_api.py with a regr-update JSON message.

2. Regression Metadata filter supports multiple BOOLEAN expressions

The Regression Metadata filter now supports AND expressions, each separated by , (comma).

For example, to search for regressions containing both coverage and testenv:

coverage,testenv

This also supports OR operations using the regex | (pipe operator). For example, this searches for (a OR b) AND c:

(a|b),c

→ See the Regex Docs page for full regular expression documentation.

3. Regression Metadata on the Job details page

Regression Metadata is now also displayed for convenience on the Job details pages.

Here is an example job:

4. New script: simscope-regr-search

Simscope now has an example Python script, which can search regression results from the command-line.

NOTE: to use this script, edit simscope_client.py with your Simscope configuration, and each user needs to have an API token requested.

$ path/to/simscope-regr-search --days=1 --component=simscope --name=dev
3 regressions matched:
2020-11-18 14:05:04  dev/9078x            pass            2   0 sigs | simscope     default      9078x
2020-11-17 19:36:34  dev/9071x            50.0%         1/2   1 sigs | simscope     default      9071x
2020-11-17 12:32:26  dev/9068x            pass            2   0 sigs | simscope     default      9068x

To view command-line help:

$ path/to/simscope-regr-search --help

This also supports executing saved regression searches:

$ path/to/simscope-regr-search --saved-search="pdq/Top fails"
67 regressions matched:
...

5. Email CC now shows up in the terminal log

Example message:

[INFO ] Chart limit failed chart=regr-lower-pass email=payton@verops.com email-cc=admin@simscope.com,bob@simscope.com

Simscope Administration Guide