Skip to content

Pyxis

Pyxis is a SPANK plugin for the Slurm Workload Manager. It allows unprivileged cluster users to run containerized tasks through the srun command.
Pyxis requires Slurm and Enroot to work. Pyxis is only available in our accelerated nodes (gnx[501-533]).

Benefits

  • Execute the user's task in an unprivileged container.
  • Fast Docker image download.
  • Simple command-line interface.
  • Supports multi-node MPI jobs through PMI2 or PMIx (requires Slurm support).
  • Allows users to install packages inside the container.

Usage

srun

Run a command on a gnx node

srun -A <slurm_account> -p normal-a100-40  -t 00:30:00 cat /etc/os-release
…
PLATFORM_ID="platform:el8"

Run the same command, but now inside of a container

srun --container-image=/share/apps-x86/containers/enroot/ubuntu-rolling.sqsh --container-name=ubuntu -p normal-a100-40 -A <slurm_account> -t 00:30:00 cat /etc/os-release
…
PRETTY_NAME="Ubuntu 22.04.4 LTS"

Mount a file from the host and run the command on it, from inside the container

srun --container-image=/share/apps-x86/containers/enroot/ubuntu-rolling.sqsh  --container-mounts=/etc/os-release:/etc/os-release --container-name=ubuntu -p all-a100 -A <slurm_account> -t 00:30:00 cat /etc/os-release
…
PLATFORM_ID="platform:el8"

To see more options

srun --help  | grep container

sbatch Execute the sbatch script inside a container image:

#!/bin/bash
#SBATCH -p all-a100 -t 30:00 
#SBATCH --container-mounts /path/to/code
#SBATCH --container-workdir=/path/to/code
# SBATCH --container-image  /path/to/container.sqsh
#SBATCH --container-name <container_name>


srun code input