Explain failed jobs
job_explain runs fixed rules over the accounting record and the redacted log tail: out of memory, timeouts, missing modules, Python import errors, CUDA problems, disk full and more. Each finding carries its evidence and a fix.
Let any AI assistant work a Slurm cluster, safely.
One Go binary that is both an MCP server and a CLI. It gives Claude, Codex, Gemini, Hermes, OpenCode and your own scripts typed, audited, read-mostly access to the Ursa Major HPC cluster at UC Riverside: job status, failure diagnosis, costs, software modules, batch-script checks, results, and (opt-in, two-step) submit and cancel.
You: Why did job 236 fail?
Claude: [job_explain 236]
It failed after one minute on gpul4: the log says
"ModuleNotFoundError: No module named 'pandas'". The script calls python
without an environment. Load python-ml (PyTorch/GPU) or python-sci (CPU)
before running it, or create a uv/Pixi environment inside the job.
Everything the MCP tools do is also a command, and every command takes --json. Every result is an envelope with the data, the evidence, and the scheduler commands that produced it, so an answer can be checked rather than trusted.
job_explain runs fixed rules over the accounting record and the redacted log tail: out of memory, timeouts, missing modules, Python import errors, CUDA problems, disk full and more. Each finding carries its evidence and a fix.
cluster_status and partitions: nodes powered up or down, queue depth, limits, GPUs, prices per node-hour, and the dollars per hour being spent right now.
script_check statically checks a script against the real cluster: partition, limits, modules and their MPI prerequisite, GPU request, a core request on shared partitions, and worst-case cost. No submission.
modules_search, module_show and recipes: module versions, MPI prerequisites, GPU builds, prebuilt containers, known-good recipes and the shared reference databases.
my_usage and waste_report: node-hours, core-hours, CPU efficiency and estimated cost, plus avoidable spend from idle cores, idle nodes and repeated failures.
Paged, redacted log windows; listings and chunked reads of your own home, scratch and job folders; signed download links for job outputs when a staging bucket is configured.
interactive_help returns the exact srun/salloc command, its hourly and worst-case cost, and how to reach the login node. It runs no job.
Opt-in tier A1, granted per person: plan, show worst-case cost, then confirm with a single-use token. Caps on nodes, hours and dollars are enforced before anything reaches the scheduler.
Everything the MCP tools do is also a bifrost command. Two examples from the README:
$ bifrost check train.sbatch
PROBLEMS FOUND partition standard worst-case cost $0.13 (1 cores, 6% of a node)
error line 4: module "python3" not found in the cluster catalog; closest: pythia8, python-ml
error standard shares nodes between jobs: this script asks for no cores, so Slurm gives
it 1 core and about 8 GB on standard. Ask for the cores it needs ...
warning calls python without loading python-sci/python-ml or an environment
$ bifrost submit hello.sh
Plan (nothing submitted yet):
partition standard, 1 node(s), time limit 0 h 05 min
scheduler would start at ... on ... (standard)
worst case $0.12 (today so far $0.00 of $50.00 day cap)
Submit this job? [y/N] y
Submitted job 265 ... folder on the cluster: ~/bifrost-jobs/<stamp>-bifrost-hello
Submitting needs tier A1. Each submitted job runs in its own ~/bifrost-jobs/<stamp>-<name> folder, so its results are exactly that folder. Default caps: 4 nodes, 24 h, $25 per job, $50 per day; a plan over a cap is refused before anything reaches the scheduler.
The assistant sits on one side and the scheduler on the other. bifrost is the bridge: tier checks, rate limits, a short-TTL cache, redaction and audit, and a backend that can only build allow-listed commands.
squeue sacct sinfo scontrol --json module sbatch
Nothing to install. Recommended at UCR.
bifrost serve on Cloud Run, Streamable HTTP at /mcpusers.yaml (re-read on every request)Your own gcloud login, no server.
bifrost mcp over stdio, plus the full CLIbackend: iap: bifrost opens the IAP tunnel itself with an OS Login key (8 h expiry) and pinned host keysA caller only sees, and can only call, the tiers granted to them (tiers in the local config, users.yaml on the hosted server). Read tools carry readOnlyHint: true, act tools false.
Everyone
Opt-in per person
Research Computing
ticket_draft writes a reply draft for a "my job failed" ticket. It is never sent.
* when a staging bucket is configured. Also: resources hpc://catalog and hpc://policies; prompts diagnose_job, write_batch_script, monthly_usage_summary, triage_ticket. Full catalog: SPEC section 7.
bifrost assumes the model on the other end will sometimes be wrong, and will occasionally be manipulated by text it reads.
job_submit runs script_check, enforces caps and sbatch --test-only, and returns the plan, estimated start and worst-case cost. Nothing is submitted.
The exact action is stored under a random token: 10-minute expiry, bound to the plan hash, so a model cannot submit something different from what was shown.
job_submit_confirm accepts nothing but the token. The day cap is checked again; cancel, hold and release re-check ownership and state.
Remote commands can only be built by constructors that validate their arguments; caller values arrive only as positional parameters. A test pins the allow-list, and a mutation check disables each guard in turn and requires a test to fail.
State changes exist only in tier A1, and acting takes two steps. Other users' jobs are denied unless the caller has tier R2.
Reads are limited to your own home, scratch and job folders. Hidden and credential-like files are refused; symlinks are resolved and checked again. Log paths come from the job's accounting record, never from the caller.
Log lines, scripts and submit lines come back only in untrusted blocks with a note telling the model it is data, not instructions.
Tokens, keys, passwords, JWTs, private keys and credentials in URLs are removed from every output and from the audit log.
Every call, CLI or MCP, appends a JSON line (mode 600): caller, client, tool, redacted args, decision, commands, size, duration.
Short-TTL cache, per-user rate limits, at most 8 concurrent SSH sessions per connection (the login node allows 10), and no background polling.
Uploads and downloads use short-lived signed Cloud Storage links in a private 7-day staging bucket. Jobs fetch their own inputs, so compute nodes hold no credentials.
Shared partitions: a script must ask for its cores or script_check and job_submit refuse it, and costs are the share of the node the job holds. Programs such as a dashboard can use a pre-registered client with a tier ceiling and their own call budget. Details: SPEC section 9.
The hosted server needs a UCR account that Research Computing has added. The local binary uses your own gcloud login and SSH.
Point your client at this URL and sign in with Google when it asks:
https://bifrost-mcp-125853442225.us-central1.run.app/mcp
For example, in Claude Code:
claude mcp add --transport http ursa https://bifrost-mcp-125853442225.us-central1.run.app/mcp
Then run /mcp, pick ursa and choose Authenticate. Other clients are listed below.
Linux/macOS release binary to ~/.local/bin, checksum-verified:
curl -fsSL https://raw.githubusercontent.com/UCR-Research-Computing/ursa-bifrost/main/scripts/get.sh | sh
Or with Go:
go install github.com/UCR-Research-Computing/ursa-bifrost/cmd/bifrost@latest
Then set up and check:
bifrost config init # writes ~/.config/ursa-bifrost/config.yaml
bifrost doctor # checks gcloud, SSH to the login node, the catalog
bifrost status
Add it to a client as a stdio server, in any JSON config:
{ "mcpServers": { "ursa": { "command": "bifrost", "args": ["mcp"] } } }
The hosted server does OAuth 2.1 with dynamic client registration, PKCE and discovery, so any client that supports remote MCP servers signs you in by itself. Below, URL means the hosted /mcp URL above.
hermes mcp add ursa --url URL --auth oauth
Local: hermes mcp add ursa --command bifrost --args mcp
claude mcp add --transport http ursa URL
Then /mcp → Authenticate. Local: claude mcp add ursa -- bifrost mcp
Settings → Connectors → Add custom connector, paste the URL. On a Team or Enterprise plan an owner may need to allow custom connectors first.
codex mcp add ursa --url URL
Signs in right away (Codex 0.160+). Local: codex mcp add ursa -- bifrost mcp
opencode mcp add ursa --url URL
opencode mcp auth ursa
gemini mcp add -s user -t http ursa URL
Then /mcp auth ursa. Local: gemini mcp add -s user ursa bifrost mcp
A {"type": "mcp", "server_url": ...} tool in the Responses API. Keep allowed_tools to read tools when approval is off.
python3 examples/mcp_client.py login
python3 examples/mcp_client.py call job_explain '{"job_id": "236"}'
A complete client in one file, standard library only.
Config-file versions, local (stdio) variants and troubleshooting for every client: docs/CLIENTS.md.
Nothing in bifrost is specific to UCR except configuration: the login node, the catalog (partitions, prices, modules, recipes), log roots and caps. Start with bifrost config init, point ssh or iap at your login node, and write a catalog.
The hosted Cloud Run deployment (Secret Manager, sealed token store, users file) is documented step by step in docs/DEPLOY.md; deploy/deploy.sh does it idempotently.
deploy/deploy.sh plan # read-only: shows what is missing
deploy/deploy.sh apply # asks before each billable or IAM change
make check # gofmt, go vet, go test -race, build (before every commit)
scripts/mutation_check.sh # every safety guard must be caught by a test
scripts/mcp_smoke.py # drives `bifrost mcp` over stdio like a real client
Tests run against anonymized recordings of real Ursa Major output in testdata/ (Slurm 25.11.4). scripts/make_fixtures.py regenerates them from a read-only probe and refuses to write if a real username survives; CI checks again.
Also in the repo: agent/, a separate read-only web dashboard (ursa-agent) whose panels call bifrost tools directly as the signed-in person, with an assistant drawer that plans jobs that only run after you press Approve.