Simscope Release History

How to upgrade?

Note: to upgrade your Simscope or Tunnel version, please follow the Upgrade instructions.

→ See also Tunnel Release History

1.507

`feature` Added support for storing and rendering static/snapshot charts

Simscope can now store static charts, via the REST API, where the dataset numbers are stored within Simscope.

As opposed to normal Simscope charts, where the dataset is re-computed every time the chart is refreshed, based on stored filter parameters.

This feature allows populating charts or even an entire dashboard from a script using external datasets.

For example, store a chart from user JIRA bug activity.

Simscope has a simscope-demo-random-dashboard.py script which shows examples of:

Creating random bar, line, and heatmap charts.
Combining multiple charts to create an aggregated chart.
Creating a snapshot (saving a Simscope chart's data).
Creating a dashboard containing the above charts.

Here is an example generated dashboard:

1.506

1. `bugfix` Fixed custom timezone logging bug

Simscope now prints log messages using the timezone specified in simscope.config.

Previously log messages would be printed relative to the server operating system timezone.

2. `bugfix` Live job timeout metadata

If using Live jobs, and they time out, Simscope will now store the job's metadata into the failing timeout job.

Previously this would get inadvertently dropped.

Simscope now detects if browsers are blocking cookies, and prints a warning message.

Note: if cookies are disabled, Simscope user login will not work.

4. `feature` Added Horizontal Bar chart

When creating charts, users can now select Horizontal Bar to render a chart with horizontal (instead of vertical) bars.

5. `feature` Added Charts grouped by Test name

Charts can now be grouped by Test name.

6. `feature` Gantt chart expansion to browser width

Gantt Charts now automatically expand based on the Client Browser window width.

1.505

1. `bugfix` Fixed sub-coverage DB bugs

SIMSCOPE-121
SIMSCOPE-122
SIMSCOPE-123

Fixed 3 bugs in sub-model coverage DB:

Filtering coverage models by component or branch could incorrectly match no results, if a regression did not have a sub-coverage model with the exact same name.
Merging regression sub-coverage could incorrectly match no models.
Publishing sub-coverage with a simple regression name would be incorrectly rejected.

2. `bugfix` Fixed running-total charts with NULL points

Running-total charts containing NULL data points now do not automatically coerce to zero.

3. `feature` Regression cherry picked charts

In the regression search view, users can click checkboxes beside one or more regressions and get a chart with results from just the selected regressions (aka cherry picking).

4. `feature` Regression floating point metadata rounding

Regression custom metadata containing floating point values automatically get rounded to 3 decimal places.

For example, if publishing func_coverage with a value of 97.214981, this will be automatically rounded to 97.215

5. `bugfix` Charts with >100 days

Daily Chart searches are currently limited to 100 days.

If a user selects a number beyond this range, Simscope will automatically reduce it to 100, to indicate to the user the value has been clamped.

1.504

1. `feature` Chart number abbreviations using financial scales

Added a configuration parameter, to enable abbreviating chart values using financial scales instead of metric scales.

For example, 1.24 B (billion) instead of 1.24 G (giga)

[chart]
# Financial number scale: when enabled, abbreviate large numbers in charts using a
# financial-scale, instead of metric-scale.
# Example abbreviation: '1.24 B' (billions) instead of '1.24 G' (giga).
# Note: financial scales are automatically ignored when rendering byte-based units (e.g. 'GB')
financialnumbers = true

2. `feature` Added configurable homepage default filters

The Simscope homepage charts and metrics can now be filtered by default.

For example, this simscope.config ensures regression names contain release pattern:

[chart]
# Optional: set default URL filters for the homepage (affecting the treemap and statistics)
# Example: only show homepage stats for 'release' regressions
homepagefilters = "regr=release"

3. `bugfix` Removed issue=X from rule search

Since issue is an output field from rules, they cannot search based on issue=X terms.

These terms are now automatically removed.

4. `feature` Improved drilldown from coverage heatmaps

When rendering covergroup-based heatmap charts, clicking on a cell now drills into the desired coverage model, and sets the cover-group filter automatically to the desired cell.

5. `bugfix` Heatmap chart scroll in dashboards

Fixed a bug where dashboards containing heatmap charts prevented scrolling.

1.503

1. `feature` Added chart links from the Signature Search page

Simscope now allows checking one or more signatures, and building a variety of charts from them:

Daily Hits chart
Cumulative Hits chart
Fail per Million Jobs (normalized)

2. `bugfix` Fixed Chart Y-axis problems

Two bugs have been fixed in the Chart Y-axis:

If the data being rendered is integer-based, the Y-axis will always have a scale without decimals. For example, if rendering the metric #Fails, the Y-axis will not have a Y-division at 10.5.
When setting Y-axis bottom/top values, these should set as the true bottom/top.

Previously, this was a "soft" bottom/top, so you would often get different chart bottom/top than requested.

3. `bugfix` Saved chart parse errors

If charts are saved, but no longer load correctly (e.g. query too long), the values will now be populated into the HTML form, so it can be debugged.

Previously it was not possible to determine which chart setting is incorrect, to debug the problem.

4. `bugfix` Normalized charts

Normalized charts (e.g. fail-per-million-jobs) were not calculating properly. This bug is now fixed.

5. `feature` Added --page option for simscope-regr-search

The simscope-regr-search script now accepts both a --page and --per-page option, if you want to query paginated results.

6. `feature` Simscope apitoken allows 640 permissions

The simscope- scripts now allow a --apitoken=PATH.apitoken, and allow permissions to be 0640 (ie group-read privileges).

This allows sharing an api token among a user group.

7. `bugfix` Fixed triaged/resolved percentage calculation in charts

The Triage% and Resolved% charts were incorrectly computing, if signatures contain IGNORE jobs.

1.502

1. `feature` Multiple coverage models per regression (sub-model coverage)

SIMSCOPE-97

Simscope now supports multiple coverage models published into a single regression.

This allows publishing separate coverage types (e.g. line, formal, functional), or separate coverage models for sub-units

To do this, your coverage model name needs a separator string, under the configuration key submodelseparator field.

[coverage]
enabled = true

# Optional: Submodel separator: allow multiple coverage models in the same regression (suffix).
# Example using "." as separator: "myblock_smoke/123.top" and "myblock_smoke/123.fpu"
submodelseparator = "."

For example, if you set the separator to "." and upload two coverage models named as:

"alpha_cpu_cov/1024.top"
"alpha_cpu_cov/1024.ddr3"

Then Simscope will store 2 separate coverage models under a single parent regression:

alpha_cpu_cov/1024
- top
- ddr3

Here is an example showing 2 coverage models in a single regression:

2. `feature` Added regression sort by Score

Regression search results can now be sorted by Score (ie pass rate, highest-to-lowest).

1.501

1. `feature` Added JSON support for simscope-issue-update

The simscope-issue-update script now allows a JSON issue object to be published, using the --json-file option.

This allows both updating and resolving issues from the CLI (or API).

Example:

# Update via JSON file
> bin/simscope-issue-update BUG-123 --json-file foo.json

# Alternative: via --stdin option
> cat file.json | bin/simscope-issue-update BUG-123 --stdin

2. `bugfix` Chart signature drilldown

When clicking on signature counts from charts, the query page now searches based on regression timestamps, not job timestamps.

This fixes an inconsistency problem in previous releases where multi-day regressions will drill down into incorrect job counts, due to charts using regression timestamps, vs tables using job timestamps.

3. `bugfix` Moved warning tag to left side of regression table

If a regression contains a warning, it will now show up next to the name, rather than overriding the Duration column.

SIMSCOPE-119

The Simscope CSS <button> and <select> form elements now support tab focus within the user interface, so users can tab through form elements using their keyboard.

1.500

1. `feature` Added custom Regression JSON warning field

Regression JSON: Added optional custom warning field, to indicate a regression-specific warning in the GUI.

This can be published during regr-update or regr-finish events.

Here is an example regression with a custom Run Terminated warning:

warning

2. `feature` New signature bucketizer

Note: this leaves the existing Simscope bucketizer as v1 (or "" blank), so if you want to keep the existing behavior, it will remain.

Added an optional v2 signature bucketizer, configurable via simscope.config file:

This bucketizer has a customizable hierarchy collapse algorithm, where you can specify the hierarchy tokens to keep.

For example, this configuration keeps the second- and third- hierarchy tokens from left, and the first- and second- hierarchy tokens from the right:

[sigs]
# Custom hierarchy bucketizer config
# Omit left#0, keep left#1, left#2, keep right#2, keep right#1
bucketizer = "v2/1,2,-2,-1"

3. `bugfix` Dashboard Edit fixed

In the previous release, the Dashboard Edit page was inadvertently broken. This is now fixed.

1.499

1. `bugfix` Regression name regex not working properly

SIMSCOPE-117

When filtering by regression name field, it now supports regex operators properly.

For example, you can now enter smoke,!integrate_smoke and see a list of regressions with a name:

Contains smoke
Excluding those that contain integrate_smoke

2. `feature` Larger Regression pagination

There is now a dropdown box, enabling Regression searches to be run with a larger page size of 200, rather than the default of 50.

3. `feature` Added simscope-test-args script

Added a new simscope-test-args script, to summarize regression results by test arguments (ie +plusargs).

1.498

1. `feature` Improved regression details page

The regression details page has an improved interface:

Simplified command box
New Analysis... button to link to: Test List, Histogram, Gantt, etc.

2. `bugfix` Fixed Admin Reload notification

In previous releases, the Admin Reload button would popup duplicate notification messages upon reload.

3. `feature` Added regression metadata units

See the metaunit section in Regression metadata.

4. `bugfix` Improvements to Coverage Search/Merge

Two improvements to Coverage Merge:

Merge view now shows proper page links (previously could be buried under a scroll bar).
Added dropdown to toggle between 50 and 200 coverage results per page.

5. `feature` New chart X-axis summarizer

New Chart X-axis choice: regression tag.

6. `bugfix` Changed chart 'billions' unit suffix

Chart values in billions are now abbreviated by a G (giga) suffix instead of B (billions).

This caused confusion between bytes and billions.
Example value: 4.2 G

7. `feature` New /api/tagged/dump-csv parameter

The Dump API now supports both min and max timestamp parameters.

Example: &timestamp-min=2024-02-03T12:05:03-08:00&timestamp-max=2024-02-04T12:00:00-08:00

1.497

1. `bugfix` Fixed duplicate rule warnings during server load

Fixed these extraneous warnings during server load:

Error: duplicate Rule: an identical rule already exists: see Rule 447.1

2. `feature` Increased Quick Searches to 18

Increased quick saved search count from 12 to 18.

3. `feature` Dynamic logging change

For debugging purposes, Adminitrators can change the Simscope log level, to enable or disable trace logging messages.

4. `feature` CSV dump API updates

The job CSV dump API now dumps additional job fields.

1.496

1. `bugfix` Coverage merge selection incorrect

SIMSCOPE-113

When merging coverage, if specific coverage models are selected, the regression filters are ignored (for example days filter).

2. `bugfix` Blank passwords allowed

Blank user passwords are allowed again for Local user accounts, if LDAP is disabled.

1.495

1. `feature` Improvements to API: /dump-csv

The REST API endpoint /api/tagged/dump-csv has a batch of improvements:

Manager users can now access the endpoint (previously only administrators could access).
It now emits componentid, regrid, and finish_time fields.
New option to emit a packed job metadata array field: &job-metadata=true
New option to change delimiter from commas.
- For tab delimiter: &delimiter=%09
New optional timestamp-min URL parameter, to filter jobs by their finish_time value after a timestamp:
- Example: &timestamp-min=2024-02-03T12:05:03-08:00

2. `feature` Added job metadata unit to the Job Details page

The job details page now shows a unit column, if metadata is configured with units (e.g. kHz)

1.494

1. `feature` LDAP Referral change for ActiveDirectory

Changed LDAP library to turn off Referrals. This was causing long delays for some ActiveDirectory LDAP user searches.

2. `feature` Added LDAP allowemaillogin option

Added a new config option allowemaillogin.

By default, LDAP logins that look like email addresses will be rejected (to prevent users from attempting bogus user IDs), and users will be given an error message.

If your LDAP uses email addresses as logins, you can email allowemaillogin = true.

3. `feature` Saved coverage searches

Coverage searches can now be saved and restored, including coverage charts.

4. `feature` Job metadata units

Job metadata units are now show in the Job details page.

5. `bugfix` Regression group update

SIMSCOPE-108

The Regression field group value is now kept during regr-update messages, if left blank.

In previous releases, an update omitted group field would incorrectly clear the group field.

1.493

1. `bugfix` Shared search duplicate save

If shared searches are enabled, and a user saves into another user's existing search name, it will now be replaced, rather than creating a duplicate search name with the current user.

For example, if there is a shared search called user1/Smoke Runs, and user2 saves a search called Smoke Runs, it will now be saved as user1/Smoke Runs.

2. `feature` Coverage saved searches

SIMSCOPE-93

Coverage model searches can now be saved and restored.

3. `feature` Filter coverage by user

Coverage models can now be filtered by user name.

1.492

1. `bugfix` Fixed regression submitted JSON

During Regression JSON publish, if the submitted field is omitted (or if the value is sent and is a negative value), then the previous submitted value is kept for the Regression.

In previous releases, if submitted were set to a positive value, and then an update or finish occurred with submitted omitted, then the regression submitted would incorrectly get cleared to zero. Now, under this scenario, the previous value is kept.

2. `feature` Improved LDAP authentication error messages

If LDAP is enabled and users fail to authenticate correctly, Simscope now displays an error with the Administrator email address from the simscope.config file, to give the end-user a better contact point to debug authentication errors.

3. `feature` Coverage merge sorting

When performing coverage merges from the GUI, the source coverage table now allows sorting by columns (e.g. by score, coverpoints).

1.491

`bugfix` Improved SMTP server support

SMTP emails have improved SMTP server compatibility:

PLAIN and LOGIN authentication options.
STARTTLS and TLS options.
10-second timeout for connections (if starttls = true).
Added DEBUG messages to help debug SMTP configuration errors.
Added pool-based message publishing, improving performance by re-using network connections.

1.490

1. `bugfix` LDAP Session support

LDAP can now be optionally configured with Session support, which enables a more secure authentication when using LDAP as the provider.

Advantages if Session mode is enabled:

No storage of hashed credentials.
Sessions expire after 7 days, or any time Simscope is restarted. Users have to re-authenticate via LDAP when their sessions expire.

To enable:

[ldap]
# LDAP session mode is a more secure LDAP authentication scheme, which does not store hashed authentication credentials
enablesessions = true

2. `bugfix` Job Sort by Runtime

Fixed a bug where sorting jobs by the runtime field was inadvertently sorting by the compute field instead.

3. `feature` simscope-test-summary improvements

Added optional sub-grouping capability to the simscope-test-summary script:

# Example: Group by test group, and subgroup by milestone
config.SummaryGroupBy = ['testgroup/milestone']

# Add zero records for sibling group-by tables with missing keys
config.AddMissingSiblingRecords = True

4. `feature` Coverage model check-all

SIMSCOPE-109

The coverage model search page now has a check-all box, to check (or uncheck) all models on the page.

Simscope Administration Guide