Live Regression Progress

If you post a Regression with the optional JSON field submitted, Simscope will render a progress bar, displaying how many jobs have been finished vs total on the regression.

Note: Simscope does not yet track details of running jobs. It only tracks counts of how many jobs are submitted/running.

Here is an example regression at 50% completion progress (based on submitted=2):

Progress Updates (dynamic)

To change a live regression's submitted or running field (after the regression has started), post a regr-update JSON message.

→ See the Tunnel API docs for more details

Live jobs

Simscope also supports tracking individual Live job progress.

This lets you see the configuration, hostname, and runtime of each job within a regression.

If a Regression JSON was published with a submitted count, and then the regression finishes before submitted jobs are received, Simscope will mark the regression as ABORTED, to indicate that either:

The regression was killed (ie by a user or script).
Or one or more jobs were lost or vanished from the regression (they were submitted but results were not published to Simscope).
- Note: this can also occur if you publish jobs within the same regression that Simscope detects as duplicate

Regression Abort detection algorithm:

Regression submitted field is > 0
Regression is marked as finished (pending field is false)
Jobs published for the regression less than submitted
- (#pass + #fail + #skip) < #submitted

Debugging Aborted regressions

You can debug Aborted regressions by using the Regression Job Diff feature.

This view will show which tests are Missing between the regressions.

Example Aborted regression

Here is an example Aborted regression (where submitted=2 but only 1 job received by Simscope):

Live Regression Auto Timeout

Global Regression Default Timeout (autotimeouthours)

Simscope has a configuration flag called autotimeouthours, which is used to automatically mark LIVE regressions as finished, if no job updates for a regression are received in X-hours.

Note: this setting does not mean the regression must complete within X-hours. It simply means Simscope has received no job messages within an X-hour window. Each update resets the timer.

This is a timeout mechanism, to prevent runaway/killed regressions from showing a LIVE flag to users indefinitely.

Example setting in simscope.config for a 12-hour regression timeout:

[regr]
# Global Regression LIVE automatic timeout:
# If no update received within X hours, Simscope marks as 'finished' automatically.
# > https://admin-docs.simscope.com/regression/live.html
autotimeouthours = 12

Per-Regression JSON Live time limit (time_limit_hours)

The optional Regression JSON Field time_limit_hours is a per-Regression automatic timeout:

If time_limit_hours is set in the JSON, this regression will get a specific timeout.
- For example, you can have a single regression set to 4, 48, etc.
All regressions with time_limit_hours that are blank or 0 value will fall back to the global timeout of autotimeouthours.

Example Regression JSON with a 4.5 hour time limit:

{
    "name": "cpu_smoke/129",
    "component": "alpha_cpu_core",
    "time_limit_hours": 4.5
}

Manual Regression Finish

Administrators can manually mark a regression as finished from the GUI, to turn off the LIVE flag manually.

Click the LIVE tag next to a regression, and it will be marked as finished.

If the regression has pending jobs, it will automatically be marked as ABORTED.
If there are no pending jobs, it will be marked as normal regression finish (ie pass or fail).
If there are any Regression Watches that match, email notifications will be sent out now.

Example Manual Regression Finish

For example, if the LIVE tag next to myregr/66 is clicked:

Simscope will mark this regression myregr/66 as ABORTED, since it finished with 1 outstanding job:

Note: in this example, you there is asterisk next to score 100.0% *, indicating the completed score for the regression is 100%, but the raw score (including aborted/missing jobs) is 50%.

Simscope Administration Guide