Old: Publishing into the Tunnel from Python

Deprecation note

This Python publish flow is deprecated, unless you have a direct Python regression flow.

The only advantage of this Python-based flow is when performing a bulk publish multiple JSON files from a single session, and not needing to shell out to publish.

This flow will still work and is still supported, but I recommended to migrate over to:

> simscope-tunnel --publish

→ This new workflow is faster than simscope_tunnel_api.py, and has zero Python dependencies.

Intro

To publish JSON into the Tunnel via python requires the pika library.

The Tunnel comes with a Python example script, which can publish JSON messages into the tunnel, called simscope_tunnel_api.py .

This script can be called directly via shell/CLI (from any language).
If using a Python-based flow, you can import simscope_tunnel_api and call from functions.

Installing pika library

The pika library is an open-source library to communicate with RabbitMQ from Python.

To install the Python pika Library, run pip install pika

Note: you should use Python3, if available (although this works under Python2 as well).

Edit simscope_tunnel_api.py

Please see the Quickstart RabbitMQ for instructions on creating a RabbitMQ shared account.

Edit the following lines in simscope_tunnel_api.py, corresponding to your RabbitMQ configuration:

# ============================================================
# RabbitMQ Authentication info
RABBIT_HOST     = 'localhost'
RABBIT_PORT     = 5672
RABBIT_USERNAME = 'compute'
RABBIT_PASSWORD = 'hello_world987'
# ============================================================

Using simscope_tunnel_api.py

The simscope_tunnel_api.py can be run from Python2 or Python3, via the shell. You can also copy the code into your Python codebase, if you are using a Python simulation flow.

This can publish the following types of messages:

1. Regression Start

This message signals the start of a regression.

# Command usage
$ rabbitmq/simscope_tunnel_api.py regr-start --regression=REGRNAME --body=REGR.json

# Example
$ rabbitmq/simscope_tunnel_api.py regr-start --regression=release/7347 --body=rabbitmq/example-regr.json

2. Job Finish

This message signals the completion of a single job within a regression.

# Command usage
$ rabbitmq/simscope_tunnel_api.py job-finish --regression=REGRNAME --body=JOB.json

# Example
$ rabbitmq/simscope_tunnel_api.py job-finish --regression=release/7347 --body=rabbitmq/example-job-fail.json

3. (optional) Regression Update

This JSON message allows changing of regression metadata, while the regression is running.

# Command usage
$ rabbitmq/simscope_tunnel_api.py regr-update --regression=REGRNAME --body=regr.json

4. Regression Finish

This message signals the completion of a regression. All jobs should already be finished before this event is published.

Note that the "Regression Finish" messages do not need a body field.

This event signals to Simscope that no future messages should arrive on this regression.

# Command usage
$ rabbitmq/simscope_tunnel_api.py regr-finish --regression=REGRNAME

# Example
$ rabbitmq/simscope_tunnel_api.py regr-finish --regression=release/7347

Publishing multiple JSON messages simultaneously

You can publish multiple job-finish JSON messages back-to-back using multiple --body arguments.

$ rabbitmq/simscope_tunnel_api.py job-finish --regression=release/7347 \
--body=1.json --body=2.json --body=3.json

Advanced: CSV jobs

Note: this is an advanced feature, that most flows do not need to use.

NOTE: this message goes to the queue simscope-in, not tunnel-in (see the --routing-key argument).

# Command usage
$ rabbitmq/simscope_tunnel_api.py simscope-jobs --regression=REGRNAME --routing-key=simscope-in --body=JOBS.csv

# Example
$ rabbitmq/simscope_tunnel_api.py simscope-jobs --regression=release/7347 --routing-key=simscope-in --body=release_7347.csv

Simscope Administration Guide