Simscope Release History

How to upgrade?

Note: to upgrade your Simscope or Tunnel version, please follow the Upgrade instructions.

1.515

1. `feature` Added livetimeoutjoblimit option

Added livetimeoutjoblimit config variable to simscope.config, if you want to time out more than 500 Live Jobs per regression.

2. `feature` Added pagination to Live

Live jobs are now paginated.

3. `feature` Added primaryscorecategory option

Added primaryscorecategory config option to simscope.config, to show only a single Job Category in the Score (pass rate) for regression details.

For example, setting this to sim will show only simulation score at the top, rather than combined regression score including build, generation, etc.

4. `bugfix` Fixed scatter plot bug with +

SIMSCOPE-130

Scatter plots with regex-based regression searches are now working.

5. `feature` Improved display of Lost/Missing jobs

SIMSCOPE-131

Simscope now shows a Lost jobs column if there are lost/missing live jobs in a regression.

1.514

1. `feature` Improved metric charts in large browsers

Metric charts now automatically scale horizontally and vertically, to fit into large client web browser windows.

Previously charts had a fixed maximum size, so they may look small on large monitors.

2. `feature` Copy SHA to Clipboard

Users can now click on SHA (model versions) in Simscope to copy to full value of Model Version into the user's clipboard (for easy copy/paste).

For example, if a user clicks an abbreviated SHA abcdef in Simscope, it will copy abcdef1234567890 into their clipboard.

3. `bugfix` Live job timeout limit increased

SIMSCOPE-128

If a regression finishes with outstanding Live Jobs, Simscope will now convert up to 500 jobs into timeout errors.

Any remaining jobs will be left as live.

Previously it would only convert the first 100.

SIMSCOPE-129

In previous releases, if Simscope is embedded into iframe containers, it would print an incorrect warning about cookies into the browser window.

This bug is fixed.

5. `feature` Run Groups

Simscope can now show a List of Run Groups, which can be clicked to drilldown to view a list of regressions within that group.

This example shows a list of Weekly run groups:

6. `feature` Improved model-version charts

Rendering line charts with Time Period set to Model Version now render as scatter charts.

1.513

1. `bugfix` Fixed scatter plot limit bug

In previous releases, scatter plots were limited to 1000 data points. This has been increased to 5000, and if beyond that this will print an error message.

2. `bugfix` Added regression plugin output limit

Regression plugins are now limited to 10k lines of stdout output when executing, to prevent client browsers from hanging.

1.512

1. `bugfix` Fixed isoweek bug in Signatures

There was a bug in previous releases when using "This Work Week" or "Last Work Week" in the Signature search page.

2. `bugfix` Fixed ambiguity bug in sub-model-coverage

SIMSCOPE-126

Fixed a bug related to ambiguous regression and coverage model names.

For example, alpha.first.smoke/beta/12.fpu now resolves to:

Field	Value
Regression series	`alpha.first.smoke/beta`
Regression ID	`12`
Coverage model	`fpu`

1.511

1. `bugfix` Fixed a negative Live Job runtime bug

SIMSCOPE-125

When publishing Live jobs, Simscope now checks that the job JSON start timestamp is no more than one minute into the future of the Simscope server machine clock.

For example, if a compute node's machine clock is 10 minutes ahead of Simscope machine clock, this could possibly introduce negative runtime bugs, due to timestamp skew.

Now, if a live job is published beyond 1 minute into the future, Simscope will reject the live job.

Normal job finishes (pass or fail jobs) in the future are still allowed, but they will automatically be clamped to Simscope machine clock (so they are not future).

2. `feature` Increased Metrics DB to 20 fields

Increased the Metrics DB to a 20-field limit.

1.510

1. `bugfix` Fixed bugs in Live job timeout

SIMSCOPE-125

The livetimeoutsignature feature did not work previously in cases where regression submitted was not set.

Now it should always check for stranded/lost live jobs at the regr-finish event.

2. `bugfix` Fixed Issue cache delete

Issue cache delete works for uppercase alphanumeric issue names (ie bug IDs).

3. `bugfix` Fixed invalid Bug hyperlinks

If using regex for Bug ID's, and an Issue does not match the regular expression, users will not get an invalid hyperlink anymore.

4. `feature` Bulk Join users to multiple groups

Administrators can now bulk Join or Leave users into (or out of) multiple groups, separating by commas.

For example, you can now bulk join a user to 3 groups with this expression:

alpha,bravo,charlie

1.509

`feature` Added "This Week" search selector

SIMSCOPE-118

When searching regressions, signatures, or tests, users can now filter with Days using two new selectors:

This Week
Last Week

(This uses the ISO Week definition, where a Work Week is a range from Monday through Sunday.)

This feature allows creating Saved Searches which automatically clamp to a 7-day Monday-Sunday range, where the week automatically floats, based on the current date.

As opposed to a normal 7-day search, where it is 7 calendar days ago.

Here is an example regression search showing a Last Week selection.

1.508

1. `feature` Added Chart snapshot button

Chart data can now be snapshotted into a destination static chart, so the data will be frozen.

2. `bugfix` REST API: added final-point into chart JSON

The chart API endpoint now has a &final-point=true URL option, to enable emitting the final_point field into the output JSON.

3. `bugfix` Fixed coverage repopulate locking

When repopulating the coverage DB index, this no longer uses a global DB lock.

1.507

`feature` Added support for storing and rendering static/snapshot charts

Simscope can now store static charts, via the REST API, where the dataset numbers are stored within Simscope.

As opposed to normal Simscope charts, where the dataset is re-computed every time the chart is refreshed, based on stored filter parameters.

This feature allows populating charts or even an entire dashboard from a script using external datasets.

For example, store a chart from user JIRA bug activity.

Simscope has a simscope-demo-random-dashboard.py script which shows examples of:

Creating random bar, line, and heatmap charts.
Combining multiple charts to create an aggregated chart.
Creating a snapshot (saving a Simscope chart's data).
Creating a dashboard containing the above charts.

Here is an example generated dashboard:

1.506

1. `bugfix` Fixed custom timezone logging bug

Simscope now prints log messages using the timezone specified in simscope.config.

Previously log messages would be printed relative to the server operating system timezone.

2. `bugfix` Live job timeout metadata

If using Live jobs, and they time out, Simscope will now store the job's metadata into the failing timeout job.

Previously this would get inadvertently dropped.

Simscope now detects if browsers are blocking cookies, and prints a warning message.

Note: if cookies are disabled, Simscope user login will not work.

4. `feature` Added Horizontal Bar chart

When creating charts, users can now select Horizontal Bar to render a chart with horizontal (instead of vertical) bars.

5. `feature` Added Charts grouped by Test name

Charts can now be grouped by Test name.

6. `feature` Gantt chart expansion to browser width

Gantt Charts now automatically expand based on the Client Browser window width.

1.505

1. `bugfix` Fixed sub-coverage DB bugs

SIMSCOPE-121
SIMSCOPE-122
SIMSCOPE-123

Fixed 3 bugs in sub-model coverage DB:

Filtering coverage models by component or branch could incorrectly match no results, if a regression did not have a sub-coverage model with the exact same name.
Merging regression sub-coverage could incorrectly match no models.
Publishing sub-coverage with a simple regression name would be incorrectly rejected.

2. `bugfix` Fixed running-total charts with NULL points

Running-total charts containing NULL data points now do not automatically coerce to zero.

3. `feature` Regression cherry picked charts

In the regression search view, users can click checkboxes beside one or more regressions and get a chart with results from just the selected regressions (aka cherry picking).

4. `feature` Regression floating point metadata rounding

Regression custom metadata containing floating point values automatically get rounded to 3 decimal places.

For example, if publishing func_coverage with a value of 97.214981, this will be automatically rounded to 97.215

5. `bugfix` Charts with >100 days

Daily Chart searches are currently limited to 100 days.

If a user selects a number beyond this range, Simscope will automatically reduce it to 100, to indicate to the user the value has been clamped.

1.504

1. `feature` Chart number abbreviations using financial scales

Added a configuration parameter, to enable abbreviating chart values using financial scales instead of metric scales.

For example, 1.24 B (billion) instead of 1.24 G (giga)

[chart]
# Financial number scale: when enabled, abbreviate large numbers in charts using a
# financial-scale, instead of metric-scale.
# Example abbreviation: '1.24 B' (billions) instead of '1.24 G' (giga).
# Note: financial scales are automatically ignored when rendering byte-based units (e.g. 'GB')
financialnumbers = true

2. `feature` Added configurable homepage default filters

The Simscope homepage charts and metrics can now be filtered by default.

For example, this simscope.config ensures regression names contain release pattern:

[chart]
# Optional: set default URL filters for the homepage (affecting the treemap and statistics)
# Example: only show homepage stats for 'release' regressions
homepagefilters = "regr=release"

3. `bugfix` Removed issue=X from rule search

Since issue is an output field from rules, they cannot search based on issue=X terms.

These terms are now automatically removed.

4. `feature` Improved drilldown from coverage heatmaps

When rendering covergroup-based heatmap charts, clicking on a cell now drills into the desired coverage model, and sets the cover-group filter automatically to the desired cell.

5. `bugfix` Heatmap chart scroll in dashboards

Fixed a bug where dashboards containing heatmap charts prevented scrolling.

1.503

1. `feature` Added chart links from the Signature Search page

Simscope now allows checking one or more signatures, and building a variety of charts from them:

Daily Hits chart
Cumulative Hits chart
Fail per Million Jobs (normalized)

2. `bugfix` Fixed Chart Y-axis problems

Two bugs have been fixed in the Chart Y-axis:

If the data being rendered is integer-based, the Y-axis will always have a scale without decimals. For example, if rendering the metric #Fails, the Y-axis will not have a Y-division at 10.5.
When setting Y-axis bottom/top values, these should set as the true bottom/top.

Previously, this was a "soft" bottom/top, so you would often get different chart bottom/top than requested.

3. `bugfix` Saved chart parse errors

If charts are saved, but no longer load correctly (e.g. query too long), the values will now be populated into the HTML form, so it can be debugged.

Previously it was not possible to determine which chart setting is incorrect, to debug the problem.

4. `bugfix` Normalized charts

Normalized charts (e.g. fail-per-million-jobs) were not calculating properly. This bug is now fixed.

5. `feature` Added --page option for simscope-regr-search

The simscope-regr-search script now accepts both a --page and --per-page option, if you want to query paginated results.

6. `feature` Simscope apitoken allows 640 permissions

The simscope- scripts now allow a --apitoken=PATH.apitoken, and allow permissions to be 0640 (ie group-read privileges).

This allows sharing an api token among a user group.

7. `bugfix` Fixed triaged/resolved percentage calculation in charts

The Triage% and Resolved% charts were incorrectly computing, if signatures contain IGNORE jobs.

1.502

1. `feature` Multiple coverage models per regression (sub-model coverage)

SIMSCOPE-97

Simscope now supports multiple coverage models published into a single regression.

This allows publishing separate coverage types (e.g. line, formal, functional), or separate coverage models for sub-units

To do this, your coverage model name needs a separator string, under the configuration key submodelseparator field.

[coverage]
enabled = true

# Optional: Submodel separator: allow multiple coverage models in the same regression (suffix).
# Example using "." as separator: "myblock_smoke/123.top" and "myblock_smoke/123.fpu"
submodelseparator = "."

For example, if you set the separator to "." and upload two coverage models named as:

"alpha_cpu_cov/1024.top"
"alpha_cpu_cov/1024.ddr3"

Then Simscope will store 2 separate coverage models under a single parent regression:

alpha_cpu_cov/1024
- top
- ddr3

Here is an example showing 2 coverage models in a single regression:

2. `feature` Added regression sort by Score

Regression search results can now be sorted by Score (ie pass rate, highest-to-lowest).

1.501

1. `feature` Added JSON support for simscope-issue-update

The simscope-issue-update script now allows a JSON issue object to be published, using the --json-file option.

This allows both updating and resolving issues from the CLI (or API).

Example:

# Update via JSON file
> bin/simscope-issue-update BUG-123 --json-file foo.json

# Alternative: via --stdin option
> cat file.json | bin/simscope-issue-update BUG-123 --stdin

2. `bugfix` Chart signature drilldown

When clicking on signature counts from charts, the query page now searches based on regression timestamps, not job timestamps.

This fixes an inconsistency problem in previous releases where multi-day regressions will drill down into incorrect job counts, due to charts using regression timestamps, vs tables using job timestamps.

3. `bugfix` Moved warning tag to left side of regression table

If a regression contains a warning, it will now show up next to the name, rather than overriding the Duration column.

SIMSCOPE-119

The Simscope CSS <button> and <select> form elements now support tab focus within the user interface, so users can tab through form elements using their keyboard.

1.500

1. `feature` Added custom Regression JSON warning field

Regression JSON: Added optional custom warning field, to indicate a regression-specific warning in the GUI.

This can be published during regr-update or regr-finish events.

Here is an example regression with a custom Run Terminated warning:

warning

2. `feature` New signature bucketizer

Note: this leaves the existing Simscope bucketizer as v1 (or "" blank), so if you want to keep the existing behavior, it will remain.

Added an optional v2 signature bucketizer, configurable via simscope.config file:

This bucketizer has a customizable hierarchy collapse algorithm, where you can specify the hierarchy tokens to keep.

For example, this configuration keeps the second- and third- hierarchy tokens from left, and the first- and second- hierarchy tokens from the right:

[sigs]
# Custom hierarchy bucketizer config
# Omit left#0, keep left#1, left#2, keep right#2, keep right#1
bucketizer = "v2/1,2,-2,-1"

3. `bugfix` Dashboard Edit fixed

In the previous release, the Dashboard Edit page was inadvertently broken. This is now fixed.

Simscope Administration Guide