Signature Bucketizing

Note: if you are getting signature overflow errors, see Signature Bucket Overflow.

Simscope automatically groups jobs into Signatures, by bucketizing error messages.

Signatures are an automatic grouping mechanism, to triage or debug similar jobs.
This includes passing jobs, which are bucketized into Pass Signatures.

Example Signature

Here is an example Fail Message and its automatic Signature.

Note the hex numbers have automatically been stripped.

Bucketing Algorithms

There are two methods available:

Builtin error bucketizer.
Custom bucketizer. Customers can choose to use their own bucketing algorithm, using the bucket JSON field.

Builtin Error Bucketizer

The Simscope bucketizer automatically buckets raw error messages from log files into Signatures.

For example, the following raw error message:

UVM_ERROR @ 109408: uvm_test_top.XbuEnv.m_xbu_scoreboard0 [uvm_test_top.XbuEnv.m_xbu_scoreboard]
[BMI ID:   {{2}}] Device function 0023 does not map to any valid device addr /projects/alpha/ver/block.sv:1242

Automatically gets bucketed error into the Signature:

UVM_ERROR ... [HIER].m_xbu_scoreboard0 [HIER].m_xbu_scoreboard BMI ID {{2}}
Device function ... does not map to any valid device addr ...

Notes on the above example:

The non-alpha characters (for example @) were detected and removed.
The timestamp was detected and removed.
The hierarchies were detected and removed.
The decimal number 0023 was detected and removed.
The ID {{2}} number was kept (due to keep region).
The path/line number were detected and removed.

Multiline fail messages

Simscope allows multiline fail messages (ie with \n characters) to be stored into the fail_message field of the Job JSON.

However, it bucketizes Signatures based on the first line of the message.

Subsequent lines are stored and viewable for debug, but ignored for bucketing purposes.

Signature `{{keep}}` sections

Simscope's bucketing can be disabled in regions, if there are specific numbers or symbols that need to be kept to distinguish between signatures.

This prevents Simscope from over-collapsing signatures.

To keep, surround the region of text with {{double curly braces}} and Simscope will leave that region of the message in the Signature unchanged.

a. Example without {{keep}}

For example, if you have this input error message:

Terminal error #305 died after 672.5s

Simscope would normally bucketize into a Signature as:

Terminal error ... died after ...

Notice both the #305 and the 672.5s were stripped in the output Signature bucket.

b. Example with {{keep}}

If you want to keep the error #305 in the output Signature bucket, surround the region in the error message that you want to keep with {{double braces}}, like this:

Terminal error {{#305}} died after 672.5s

Then Simscope will bucketize as this signature:

Terminal error {{#305}} died after ...

Notice the #305 was kept in the Signature bucket, but the 672.5s was stripped.

Signature `<<ignore>>` sections

Simscope's default bucketizer looks for special ignore (aka "don't care") fields in any error message and automatically strips them out. Simscope will automatically replace all characters these sections with <<ignore>>.

To flag any part of an error message as ignore, surround it with <<double angle brackets>>.

(input) Example error containing ignore sections:

UVM_ERROR my_checker.svh:1904 @ 4208.0ns: uvm_test_top.monitor_base [CHECKER] <<FIFO>>
data mismatched for Instruction : <<idiv	DWORD PTR _n$[ebp] $L477>>

(output) Stripped and bucketed signature:

UVM_ERROR my_checker.svh ... uvm_test_top.monitor_base CHECKER <<ignore>> data mismatched
for Instruction <<ignore>>

Details: Bucketing algorithm

Simscope's builtin Signature bucketizer follows this algorithm to bucket messages into Signatures:

Strip all extraneous whitespace (multiple spaces, tabs, etc).
Save off {{keep sections}}.
Strip out <<ignore sections>>. Replace with <<ignore>>.
Automatically strip out detected hierarchies (mydesign.top.chip2.mem3.assertEmpty). Replace with [HIER].
Automatically strip out detected paths and line numbers (/home/user/myworkspace/chip/run/787/design.v). Replace with ...
Strip out non-alphanumeric characters: !@+-[]()=.
Strip out decimal words (12492) and hexadecimal words (ab38f9), including X/Z (fXX01z). Replace with ... symbols.
Restore {{keep sections}} back into the message (from step #2).

Admin: componentizer

The Tunnel can optionally prefix component name into error messages, using the Componentize feature.

This allows splitting of generic error messages (like cycle timeout) into separate signatures, per component.

Admin: custom Bucketizer

Simscope's bucketizer can be overridden by administrators to perform an algorithm desired by a customer.

This can be useful, for example, to have a bucketizer which does NOT strip registers from an error message. For example, reg-30 may be an important distinguishing field in the error.

Note: switching to a custom bucketizer disables the builtin bucketizer, so the custom algorithm should be at least as intelligent as the simple bucketizer above, to handle the many bucketing cases.

To override the bucketizer, when a Job JSON is published to Simscope, set the "bucket" field to the desired string representation of the bucket to group the job into.

If you leave the "bucket" field blank (or omit it), Simscope will use its own bucketizer instead.

Custom bucket example

If you have your own fail bucketizer, you can bypass Simscope's internal bucketizer by setting the bucket field of the Job JSON.

For example, with this Job JSON:

{
    ...

    "fail_message": "t.cpu.core0.fs7 data mismatch filling store buffer: expected=24'h16a7 actual=24'h76a7",

    "bucket": "core0 store buffer"`

    ...
}

This forces the job to be grouped into the bucket:

core0 store buffer

For reference, Simscope's internal bucketizer would have grouped this job into the bucket:

[HIER].fs_ data mismatch filling store buffer expected ... actual ...

Simscope Administration Guide