Job Details Plugin (ie Job Log)

Simscope can execute a shell script plugin before rendering any Job details HTML page, and then will insert the result inline for the user to see.

The most common use of this is to display a partial log file of the job, without requiring the user to open the log file directly.

The plugin can be any Unix executable:

Python, Perl, Ruby, bash shell script, C binary, etc.

Plugin input

The plugin will be sent a JSON representation of the job and its Regression, which can be used to inspect and render any data back out.

Sample Plugin output

Job details plugin

This page shows the first lines matching a job log inside the browser.

The part outlined in red was the STDOUT from the plugin script.

Configuration

To enable a job details plugin, add the following to your simscope.config file, and restart Simscope server:

[sigs]
# Job details plugin
jobdetailsplugin = "/PATH/TO/your-job-plugin"

# Enable job details plugin on both passing and failing jobs (vs fail jobs only)
# jobdetailspluginpass = true

This script will be executed automatically by the Simscope server process before each job details page is rendered.

The plugin will be executed as a child process of Simscope, as the same user ID as Simscope process, on the same machine (hostname) where Simscope is running.

Note: after enabling the plugin, you can modify the script source code at any time, without needing to restart the Simscope server. Every time a job details page is refreshed, the latest plugin is executed.

Caveats

For browser safety, the plugin currently only allows text rendering.

If you are interested in html or javascript rendering, please contact VerOps for a feature request.

Example Plugin: dump Job info (python)

Simscope comes with a job debug plugin called simscope-job-details . This plugin will dump the job fields and the session user (aka client user), for debugging purposes.

#!/usr/bin/python -S

# Simscope Job details example plugin

import json
import sys

if len(sys.argv) == 2:
    sig_file = open(sys.argv[1], 'r')
else:
    sig_file = sys.stdin

job_json = sig_file.read()
plugin_input = json.loads(job_json)

session_user = plugin_input['session_user']
job = plugin_input['job']

print(json.dumps(job, indent=4))

Example output

The above script will render a response similar to the following in the browser:

{
    "bucket": "make golint Error ...",
    "jobnum": "3",
    "assignee": "",
    "model_version": "8876",
    "result": "fail",
    "message": "make: *** [golint] Error 1",
    "category": "lint",
    "jobns": 1602895045661141000,
    "rulenum": "",
    "state": "NEW",
    "build": "release",
    "testargs": "",
    "config": "sh -c \"cd env && make TAGS lint vet\"",
    "model_branch": "default",
    "regrname": "release/8876",
    "sigid": 1204,
    "stateid": 0,
    "component": "alpha_simscope",
    "testname": "",
    "model_ts": 1602894914000000000,
    "host": "quackbook-air.local",
    "compute_ms": 2621,
    "active": true,
    "path": "/home/pdq/v/simscope/run/release/8876/3",
    "componentid": 29,
    "testgroup": "selftest",
    "issue": "",
    "userid": "pdq",
    "simcycles": 0,
    "jobid": "c6eqveeuze2w",
    "seed": "",
    "exitcode": 2
}

Example Plugin: Job Log (python)

Simscope comes with a sample Python script, called simscope-job-log . This script will automatically look in the job directory for either a job.stdout or job.stderr file and will return it back to the browser.

You can replace this to look for any files which contain logs

#!/usr/bin/python -S

# Simscope Job log plugin example

JOB_LOGFILES = set(['job.stdout', 'job.stderr'])

import json
import os
import sys

def print_logs(jobdir):
    if not jobdir or not os.path.isdir(jobdir):
        return

    jobdir = os.path.abspath(jobdir)
    dirfiles = sorted(os.listdir(jobdir))
    for filename in dirfiles:
        if filename in JOB_LOGFILES:
            # Found a legit log file; dump it
            filepath = os.path.join(jobdir, filename)

            print('-' * 60)
            print(filepath)
            print('-' * 60)

            # Dump the entire file
            # NOTE: you should use 'grep' or head/tail instead, so only parts of the log file are dumped to the browser!
            f = open(filepath, 'r')
            sys.stdout.write(f.read())
            f.close()

def main():
    if len(sys.argv) == 2:
        # Command-line testing
        jobdir = sys.argv[1]

    else:
        # Simscope live web
        job_json_file = sys.stdin
        job_json = job_json_file.read()
        plugin_input = json.loads(job_json)

        session_user = plugin_input['session_user']
        job = plugin_input['job']
        jobdir = job.get("path")

    print_logs(jobdir)

if __name__ == '__main__':
    main()

Simscope Administration Guide