Simscope Release History — v350 — 359

1.359

1. bugfix Regression Duration charts pass/fail

When rendering average Regression Duration charts, you can now group results by Pass vs Fail.

• Note the Duration metric tracks the wallclock time, from start to end time of the Regression.
  • (As opposed to Compute time charts, which track cumulative job compute time within Regressions.)

This groups into two chart series:

• pass → Regressions with 100% pass.
• fail → Regressions with at least one fail.

Here is an example daily chart for 7 days:

2. feature Regression Duration filtered by pass or fail

When using the Regression Duration metric, you can now filter out results based on the result of the regression, by using entering either pass or fail into the Classifications box.

This example shows average Regression Duration for 100% passing regressions (regressions with any fail are omitted from the chart):

1.358

1. feature Regression Watch debug

From any Regression details page, click Watches, to view all Regression watches that match this Regression (if any).

In this example, two Watches match Regression dev/12751x:

2. feature User Watches Summary

Simscope now has a Watches summary page, to help with inspecting and debugging:

• Regression Watches
• Chart Limit Watches

To access, go to Misc → Watches.

Here is an example:

3. bugfix Tests assignee filter

The Assignee filter in the Tests section has been changed from a text box to a dropdown user selection box.

4. bugfix OIDC fixes

OIDC user authentication has a bugfix if internet is disconnected.

1.357

1. feature Okta OIDC with Directory API

This Simscope release has the first working Okta OIDC implementation, with optional Directory API support.

2. feature Coverage Merging

The Coverage DB now automatically merges databases if you publish two or more times into the same regression.

• Merge will be skipped if you double-publish identical coverage.
• Merging locks the database (to prevent simultaneous merge race conditions), so you may see performance issues if many merges occur in parallel.
• The only way to overwrite instead of merging is to first delete coverage, and then publish.

1.356

1. feature Filter Tests metrics by assignee

Test search can now filter by Signature assignee field.

2. bugfix Tests header renamed columns

If build or testgroup has been renamed, the Tests table now shows the correct, renamed header.

3. feature Test History link from job details page

Simscope now links from Job details pages to a 7-day Test History search, to find historical runs of the same testcase.

The link automatically filters jobs based on the same:

• Test Build
• Test Group
• Test Name (if used)
• Test Config
• Component
• Regression Series

Example job details page with Test History link:

1.355

1. feature Added Days=0 selector (ie TODAY)

When selecting dates, you can now choose Days = 0.

• Days = 0 means show results from today only (midnight until the current time).
• Days = 1 means yesterday + today (ie from 1 day ago).

2. bugfix Rule template check for all fails

Simscope can now check all regression failures against template rules, if you enable the following in your simscope.config file:

[sigs]
alwayschecktemplates = true

3. feature Test search and charts: filter by username

When viewing results from the Tests section, you can now filter by Regression User, if you only want to view statistics from a specific user.

Also, charts can be configured to filter by user as well.

4. bugfix Dashboard heatmap chart numbers

When viewing heatmap charts from the dashboard, the numbers for each cell now show up if you expand the chart.

Previously, the only way to see numbers on each cell was to open the chart in the chart editor.

5. feature Regression Gantt chart: sort by Average runtime

The Regression Gantt chart now allows sorting of results by shortest average runtime to longest.

6. feature Regression view: fail signatures only

When viewing a single Regression, you can now click the dropdown to show only fail signatures.

• By default, Regressions show both pass and fail signatures.

7. feature Regression tool version string

If you are storing raw strings inside the Regression Tool Version field, add the following to your simscope.config file, to prevent shortening/hyperlinking of the field.

[regr]
toolversionstring = true

8. feature OIDC authentication with Directory API support

This version has optional support for Okta OIDC support, for user authentication, with Directory API support. This lets you provision user accounts externally and Simscope will dynamically enable or disable users.

Other OIDC providers can be added in future versions.

1.354

1. feature Live job Log plugin

Live jobs can now execute job plugin scripts, enabling viewing of the Live job log file dynamically within the job details web page.

• Note: this still requires a manual page refresh to update the web page.

2. feature Gantt updates

The Gantt chart page, when grouped by Build or Test Group, now shows a separate Avg Runtime column for each grouping.

• [Avg Runtime] = [Total Runtime] / [Total Jobs]
• Note: the Duration column is wallclock duration of the entire grouping (different than total runtime).

3. feature Regression Text Report assignees

The Regression Text Report now has an Assignees column, grouped for each build.

1.353

1. feature Live job improvements

Live jobs now automatically refresh from the job details page, while the job is running.

• Users can see the runtime field progressing in real-time, and when the job either passes or fails, they will get redirected to see the results.

2. bugfix Rule Template bugfix

In previous versions, if you edit a Rule Template and put in an invalid regular expression, the page would not show the error.

→ Now template errors are shown properly.

3. feature Bug create skeleton plugin

Added a Bug Create Skeleton Plugin, which can be used to create a bug from Simscope in any external bug tracker.

• The skeleton plugin is written in Python, but can be ported to any language which can parse JSON input and write JSON output.

4. feature Added "Latest Regression" symbol

To find the latest regression in Simscope, replace the regression name suffix with a $ symbol.

• This lets you bookmark a single page in a browser, and it always shows the latest regression.
• You can also use this in a REST API script to grab the latest run, without knowing it's #.

Regression URL examples:

1.352

1. feature Added User Online field

SIMSCOPE-29

The Simscope Users table now shows the latest time each user has been Online, to indicate which accounts are active, and which are not.

2. feature User API Token indicator

The Simscope Users table shows a checkmark for any users who have a valid API Token issued.

1.351

1. feature Rule Template optimizations

Rule Templates now are only checked during:

• The first instance of each Signature, within each Regression
  • If the first instance matches an existing Rule, templates are skipped
  • Otherwise, if the first instance matches a Rule template, this creates a Rule
  • Otherwise, Rule templates are skipped until the next Signature or the next Regression

The purpose of this is to allow many global templates simultaneously, without affecting job import performance dramatically.

This means that Rule Templates are checked once per Signature per Regression.

• For example, if a regression has 20 signatures and 1000 fails, templates are checked up to 20 times, rather than 1000 times.

2. feature Rule Template increase to 250

Rule Templates are now increased to a maximum of 250 active templates.

• Previously, this was limited to 25 active templates.

1.350

1. feature LIVE job tracking

→ See Live Jobs for more details.

2. feature Rule templates require manager

Adding/editing Rule templates now requires Manager user access or higher.

• This is to prevent novice users from using advanced rule templates.