NAIRR Pilot readiness

NAIRR Pilot Readiness CI

NAIRR Pilot Workflow Readiness Dashboard

Monitors and displays the readiness of NAIRR pilot compute resources via automated CI/CD checks.

API

How readiness is measured

Probe types

Readiness is evaluated at two levels. Unit tests perform basic reachability checks, verifying that the login node is accessible, the job scheduler is responsive, and outbound network connectivity is available. Integration tests go further: they submit and execute a real end-to-end Pegasus WMS workflow directly on the resource, confirming it can plan, schedule, and complete a scientific job, not just respond to a ping.

For AI-focused resources, additional probes validate AI readiness by running representative training and inference workflows using real machine learning workloads. These tests confirm the resource can ingest data, execute model training or inference pipelines, and return results end-to-end, going well beyond basic connectivity checks.

What is PegasusWMS?

PegasusWMS is an open-source workflow management system developed at USC Information Sciences Institute. It maps abstract, resource-independent scientific workflows onto distributed HPC, cloud, and grid resources, automating scheduling, data movement, monitoring, and failure recovery.

Resource readiness explorer

Select a NAIRR resource card to open its recent check history, latest checks, links, timestamps, and statistics.

The small color strip on each resource summarizes the six checks. Click to show/hide descriptions.
Success

The probe completed successfully.

Failed

The probe failed.

Timeout

The probe did not finish within the time limit.

No Data

The probe does not apply to this resource, or cannot be run on this resource due to technical limitations of the probe itself.

Probe run trend

Daily probe pass rate over the selected window.

history API

Each period card groups nearby CI checks into a readable block. The stacked bar separates passed checks from failed, timed-out, or missing checks.

Resource reliability

Success rate by resource across all probes in the current window.

resources API

Higher bars mean more successful checks. The count beside each resource is the number of checks in the current window.

Probe reliability

Which readiness checks are stable and which checks need attention.

probes API

Readiness evaluation uses unit tests for basic reachability checks and integration tests executing actual Pegasus workflows. For AI-focused resources, additional tests cover AI readiness through representative training and inference workflows.

Readiness timeline

Readable 6-day blocks per resource, with daily dots underneath for exact pattern recognition.

history API derived rollup

Probe wait time

Scheduler queue wait before a probe job starts, per resource.

derived from stats

How long the CI probe job waits in the scheduler queue before execution begins. Longer waits may indicate queue pressure or resource unavailability.

Pegasus workflow wall time

Average end-to-end workflow duration per resource and probe over the current window.

derived from stats

Total elapsed time from workflow submission to completion, including scheduler wait, job execution, and data movement.

Probe failure reasons

Top classified causes from failed or timed-out probes in the selected window.

derived from logs

Recent failures and timeouts

Latest failed and timed-out checks with reason, timestamp, pipeline, and job link where available.