Ewoks Macros

Bliss provides the possibility to load user scripts. This allows user defined python functions to be executed in the Bliss terminal.

Ewoks macros provide the same functionality with the difference that the functions are executed in another python process/environment, possibly on another machine or on the cluster.

DEMO_SESSION [1]: from blissoda.wrappers.ewoks_macros import EwoksMacroHandler
DEMO_SESSION [2]: ewoks_macros = EwoksMacroHandler()
DEMO_SESSION [3]: ewoks_macros
        Out [3]:
                Workflows:
                trigger_at   'PREPARED'
                queue        'user'

                Status:
                Enabled   False

                Remote scripts:
                Location    ESRFPath('/tmp/demo_resources_pfp9mgh/inhouse/id002510/id00/20251001/SCRIPTS')
                #scripts    0

To avoid importing and instantiating blissoda.wrappers.ewoks_macros.EwoksMacroHandler() explicitly, configure an ewoks_macros Bliss object in Beacon

- name: ewoks_macros
  plugin: generic
  class: EwoksMacroHandler
  package: blissoda.wrappers.ewoks_macros
  default_parameters:
    queue: user
  shared: false
  singleton: true

and add it to the Bliss session objects

- class: Session
  name: eh1
  setup-file: ./eh1_setup.py
  config-objects:
    - ewoks_macros

User Macros

Define a function in a file located in the SCRIPTS directory like any other Bliss user script. For example:

# File: /tmp/demo_resources_pfp9mgh/inhouse/id002510/id00/20251001/SCRIPTS/my_macros.py

def my_function(n=10):
    return list(range(n))

List Available Ewoks Macros

After loading one or more scripts you can list the ones that are loaded

DEMO_SESSION [4]: ewoks_macros.user_script_list()
List of python files for remote execution in [/tmp/demo_resources_pfp9mgh/inhouse/id002510/id00/20251001/SCRIPTS]:
- my_macros.py

Load and Execute Synchronously

Wait for the function to be executed remotely

DEMO_SESSION [5]: ewoks_macros.user_script_load("my_macros", blocking=True)
Merged [ewoks] namespace in session.
DEMO_SESSION [6]: ewoks.my_function()
        Out [6]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Load and Execute Asynchronously

Do not wait for the function to be executed remotely

DEMO_SESSION [7]: ewoks_macros.user_script_load("my_macros")
Exported [ewoks] namespace in session.
DEMO_SESSION [8]: future = ewoks.my_function()

Wait for the result whenever you need it

DEMO_SESSION [9]: future.result(timeout=None)
        Out [9]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Ewoks Worker

An Ewoks worker environment can be any python environment with ewoksjob[blissworker] installed

python -m venv myenv
source myenv/bin/activate
pip install ewoksjob[blissworker]

An Ewoks worker can be started from the command-line

export BEACON_HOST=id00:25000
ewoksjob worker --queues=user --loglevel=info --max-tasks-per-child=1

The --max-tasks-per-child=1 argument is required if modifications to the user scripts are to be taken into account without the need to restart the worker. Every job will be executed in a fresh subprocess of the worker.

The worker above is listening to the "user" queue which needs to be selected in the Bliss session as follows

DEMO_SESSION [10]: ewoks_macros.queue = "user"

Ewoks Slurm Worker

The python environment used on Slurm must at least have ewoks installed

cd .../SCRIPTS

python -m venv myenv
source myenv/bin/activate

pip install ewoks

The Ewoks worker can be started with the slurm pool type:

ewoksjob worker --queues=user --loglevel=info \
    --pool=slurm --concurrency=100 \
    --slurm-user=... --slurm-token=... --slurm-url=... \
    --slurm-pre-script="source .../SCRIPTS/myenv/bin/activate" \
    -sp partition="gpu" \
    -sp cpus_per_task=8 \
    -sp memory_per_cpu="8GB" \
    -sp time_limit="02:00:00" \
    -sp tres_per_node="gres/gpu=1"

Parameters for authentication (url, user, token) and for activating the python environment are required. Default slurm job parameters can be provided with -sp.