Custom Job Metadata

Simscope can be customized to store custom metadata for each Job.

Note: Simscope also supports Regression-level Metadata (ie shared across the entire regression).

Job Metadata Use Cases

Job Metadata allows storing arbitrary key/value data for a single Job, for use in many scenarios, especially related to performance, stimulus, or coverage tracking.

Here are some common use cases:

1. Tracking average or peak memory usage from a simulation.
2. Tracking job disk usage (ie log files) per simulation.
3. Tracking machine compute load (ie uptime).
4. Tracking transaction-level performance:
   • transactions
   • bandwidth
   • latency
5. Monitoring RTL (design) performance, event, flag, or coverage counters.
   • For example, store cache hit vs miss counts across a simulation.

Job Metadata schema definitions

To enable job metadata, add a line for each metadata field into the [server] section of your simscope.config file, to describe the metadata schema.

The schema format for each metadata field is ID:TYPE:FIELDNAME

• ID — a unique integer that you can choose. This is only used internally within Simscope
• TYPECODE — the data type of the field value (see table below)
• FIELDKEY — a unique metadata field key
• (optional) UNIT — field unit (used for metadata charts)

Metadata Types

This table describes the data types allowed for metadata:

Note that x (hex) data format allows decimal and hexadecimal data value inputs, but will render data within the HTML as hexadecimal base, prefixed by 0x.

• For example, 1024 and 0x400 are both allowed as job metadata values, but will be rendered as 0x400 in the HTML.

Metadata Keys

The metadata keys are alphanumeric keys.

• It may not contain whitespace, quotes, or HTML-type characters: <>&#%?@*\:()~=

Characters allowed in the key:

Example metadata configuration

This example declares five job metadata fields:

Example simscope.config:

[server]

# Job metadata fields
metafields = "10:i:transcount"
metafields = "11:i:cpu.ops"
metafields = "12:f:machine_load"
metafields = "13:s:last_opcode"
metafields = "14:x:perf_reg1"

# kilobyte-based unit
metafields = "15:i:disk_usage:KB"

(optional) Metadata Units (scaled)

Metadata Units are optional an optional suffix on the metadata specification, which allow charts to both render a unit label, and scale values automatically.

Unit examples:

Unit Scales

Simscope can auto-scale values within charts, if the unit is prefixed with one of these scales:

Metadata Unit example

For example, if the metadata disk_used is specified as:

122:i:disk_used:KB

Then Simscope will automatically render a chart with byte-based values, and scale them accordingly by 1000 (due to KB).

• As an example, if the raw job metadata value is 5000, the chart will be rendered with a scaled value of 5MB (since the field unit was specified as kilobytes).

Here is a sample metadata chart without metadata unit specified:

Here is the same chart showing a kilobyte-based (KB) metadata specification:

• Notice that data values were automatically scaled to megabytes in the chart.

Publishing Jobs containing Metadata

Note: publishing job metadata is not required. Some jobs may have them and some may not.

The Job JSON contains an optional metadata field, which is a two-dimensional array.

The format is:

"metadata": [
    ["KEY1", "VALUE1"],
    ["KEY2", "VALUE2"]
]

Note that even though internally the field may be configured as an integer or float, this format requires the value to be escaped into a string variable. For example, the integer value 12345 should be posted as "12345". Internally, Simscope will convert to an integer.

Case insensitive

When publishing jobs, the metadata keys are case-insensitive.

• The metadata key cpu_trans is interpreted the same as CPU_TRANS

For example, this JSON:

"metadata": [
    ["cpu_trans", "10"]
]

Is interpreted the same as this JSON:

"metadata": [
    ["CPU_trans", "10"]
]

Example Job JSON Metadata

Here is an example JSON using the above configuration to post metadata in a job:

{
    "jobdir": "/home/runs/myregr/2/job-2",
    "category": "sim",
    "config": "sim +rand +tbuf +seed=1794 LOC=4",
    "build": "mybuild",
    "testgroup": "mytestgroup",
    "seed": "1794",
    "start_time": "2019-05-07T02:00:10.000111-05:00",
    "finish_time": "0001-01-01T00:00:00Z",
    "compute_ms": 1234,
    "host": "compute-12",
    "result": "fail",
    "exit_code": 100,
    "fail_message": "|t.cpu.core0.fs7| data mismatch filling store buffer: expected=24'h16a7 actual=24'h76a7",
    "cycles": 17,
    "metadata": [
        [
            "transcount",
            "4097"
        ],
        [
            "machine_load",
            "15.98"
        ],
        [
            "perf_reg1",
            "0x12487"
        ]
    ]
}

After import, Simscope shows the Job metadata inside the Metadata box on the Job Details page.

Here is an example:

Searching Jobs by Job metadata

Here is an example of searching for jobs in Signature 1273 containing gc.heap_objects > 0:

Customizing Signature Job Table: Add Metadata Columns (jobtablecolumns)

The Signature Job Table view can be customized to show columns in any desired order, using the jobtablecolumns config variable.

• You can add or remove any job field to/from the table.

For example, to add metadata disk_used_kb to the table, Edit your simscope.config file:

[sigs]
# Optional: change the order/layout of the Signature job table columns (job fields)
# NOTE: you can also add 'metadata' columns if you want to display them
jobtablecolumns = "time,message,component,branch,disk_used_kb,model,build,testgroup,attributes,runtime,cycles,state,assignee,issue,config,regr,cps,seed,host,user"

Example table showing disk_used_kb:

Metadata Storage

Metadata is not free. The Simscope database increases linearly with the number of metadata stored inside each jobs, so it is recommended to only store prudent fields alongside each job.

However, if metadata is declared in the Simscope Configuration, but not used in any job, there is no space used.

Also, Metadata for old jobs can be pruned, with the auto-pruning flow.

Metadata Limitations

Tunnel has the following default limits for job metadata:

• Maximum of 50 bytes per metadata key.
• Maximum of 300 bytes per metadata value.
• Maximum of 50 metadata entries per job (ie # of unique keys per job).

Simscope will automatically discard job metadata beyond these limitations.

Note: you can edit the tunnel config to increase these limits.