Simscope Release History — v360 — 369

1.369

1. `feature` Test grouping sort

Added client-side sort capability for the Tests grouping tables (both ascending and descending):

Category
Build
Test Group

Click any column name to change sort order.

This is saved as a client browser cookie, so the sort will be automatically remembered in the future.

This example shows tables with different sort orders selected:

2. `feature` Test group expansion save

When using the "Expand" checkbox feature on the Tests section, the selected value is now remembered in your browser cookies.

1.368

Added Component Purge capability, to delete entire component history (Jobs, Tests, Charts, Statistics) from Simscope.

Warning: purging components is an non-reversible operation. Please make a backup of the database beforehand, unless you are comfortable deleting portions of the database.

1.367

1. `bugfix` model_version shortening disable (abbrev=0)

For groups who want a full Regression model_version to be shown in the GUI, you can now set abbrev = 0 (ie abbreviate) in your simscope.config file.

This will disable model ID (version) shortening.

2. `bugfix` STDERR now logged properly for command-line plugins

If Simscope command-line plugins fail (like the commitinfo plugin), both STDERR and STDOUT of the command will be logged to the terminal, to aid in debugging.

In previous Simscope releases, only the STDOUT was logged.

3. `feature` Test Group assignments table

The Test Metrics page now lists Test Group assignees inside the Test Group table.

This lists the Top 3 assignees for each Test Group, in ascending order.
You can click the Expand hyperlink to also list the top 3 Signatures for each Test Group.

Example:

1.366

`bugfix` JIRA Python update script bugfix

In previous versions, if the Python script simscope-jira-update.py was invoked and a bug was not found among a batch of bugs, it could erroneously abort the script, skipping all bug updates.

This version will ignore JIRA bug updates on unknown bugs, but still update the other bugs.

1.365

1. `feature` Added Regression "External URL" plugin

Added Regression External URL Plugin.

This can be used to link a dynamic URL, invoked from a Python script. For example, auto-link to Jenkins from any Regression.

2. `feature` JIRA optional Component field

JIRA bug plugins have been updated to allow opening bugs without a Component field.

For groups who do not use JIRA Components.
Set the component to to None in Simscope JIRA plugin, and Simscope will omit the component field when creating bugs.

3. `feature` Regression Job Diff by Compute Time

Simscope now allows comparing two Regressions job-by-job, and detecting if jobs have changed in compute-time by a percentage threshold.

Enables detecting jobs which have the same result, but are slowing down in performance.
Compute-time choices:
- Off, 10%, 25%, or 50%

Example screenshot, where a job has the same result, but has changed in compute-time by more than 10%:

4. `bugfix` Unused Bug IDs are pruned automatically on server startup

SIMSCOPE-32

In previous Simscope versions, if a user linked a Bug ID to a rule, and then deleted the Bug ID from the rule, Simscope would continue to query the Bug ID when refreshing.

This would trigger an extraneous console warning on a daily basis.

Unlinked Bug ID's are automatically pruned during server startup now.

5. Increased default chart series limit

The default series limit for charts has been increased from 100 to 150 series names.

For example, if you render a Heatmap chart, this limits the chart to a maximum of 150 series rows.

If you render a chart with more series, all additional series will be grouped under (all other series)
For debugging, you can change the series limit by editing your chart URL and appending &max-series=200 (or any other number)

1.364

`feature` Formal Coverage improvements

Multiple improvements to Formal Coverage:

Bounded proof coverage for undetermined properties, based on minimum cycle count threshold
- If Property Bound is less than minimum cycles, the property is a fail
- Otherwise the property is a pass
Filter properties by type field: assert, cover, etc.
Filter properties by result field: cex, undetermined, etc.
Sort properties by time, cycle bound, result
Change the minimum cycle bound dynamically: 1, 5, 30, etc.

1.363

`bugfix` Email CC with inactive accounts

Email CC's (carbon copy) notifications are now checked against the Simscope active user list. If a user's email has been disabled, the email CC will be removed from the email.

Scenarios:

Email CC to an active user → ✅ Sent.
Email CC to an inactive user → ❌ Discarded.
Email CC to an unknown user → ✅ Sent.
- This enables you to send a chart or regression email notification to external users, or distribution lists.

1.362

1. `feature` Regression Watch — Notify Always

There is a new schedule choice for Regression Watch: Notify Always

This enables emailing the results of every regression, immediately after it finishes.

This is the same as Notify on Fail, except this only notifies for failing regressions

Watch Schedule

Schedule	Pass	Fail	Daily Summary
Off (default)	.	.	.
Watch Daily	.	.	✅
Notify on Fail	✅	.	.
Notify Always	✅	✅	.

2. `feature` Regression Watch emails now include metadata

If a Regression Watch email is sent for a regression which contains metadata values, these are now sent in the bottom of the email.

Here is an example watch email:

3. `feature` Regression Job Diff

You can now compare two Regressions, on a job-by-job basis (ie comparing individual test results against each other).

This is useful for comparing smoke or golden regressions against each other, where the exact same tests are run in both regressions.

For example, if a single job changed from pass to fail, the Test Diff will show them side-by-side.

To Diff Two Regression by Job:

From Regression Search, ✅ (check) any two Regressions.
Click the Δ Compare button at the bottom of the page.
Then click the Δ Diff Jobs button at the bottom of the page.

Here is an example Job Diff, showing a job that changed from Pass to Fail across 2 regressions:

Notes:

This compares using the job testconfig field, so the seeds must match up exactly for Simscope to make a comparison.
- Comparing random regressions will likely produce noise, as the job seeds will be different.
This only works with 2 regressions at a time. You can use Regression Signature Diff for 3 or more regressions.

4. `feature` "Show More" button for Test List search

When searching Test List history, Simscope defaults to 500 results displayed. This helps for performance, so the browser does not hang during rendering of thousands of results.

If you hit this limit, Simscope now prints a messages, where you can either:

Display the first 5000 tests.
Or download all test matches as a CSV file (for viewing in Excel, text editor, grep, or command-line script.

5. `bugfix` Changed Linux LDAP library dependency

Simscope now depends on a newer LDAP shared object library, as the old library is deprecated:

Old → libldap-2.4.so.2
New → libldap_r-2.4.so.2

1.361

1. `feature` New JIRA Python plugin flow

The JIRA Python Plugin has been rewritten to support:

Custom authentication (optional SSL certs).
Optional multiple instances. If you want to point Simscope at 2 or more JIRA URLs, you can do this now. For example, you can map:
- SIM-xx to jira-host1
- MARS-xx to jira-host2

The old JIRA scripts still work, but if you want to upgrade, you will need to merge your:

~/simscope_jira_config.py with the new template at TEMPLATE_simscope_jira_config.py

2. `feature` Added Regression Series Group-by modelversion

You can now drill down in Regressions based on model version, to see all runs grouped by version.

You can also group by:

Component
Regression Series
Branch
User

Simscope Administration Guide