Most labs use more than one kind of storage, and move data between them as a project goes on. The right place depends on what the data is, how often you use it, who needs it, and its protection level. ITS also publishes a campus-wide comparison on its Storage at UCR page.
Storage by stage
| Stage | What it is for | Where |
|---|---|---|
| Working (hot) | Data being computed on right now | HPCC storage (GPFS) |
| Project (warm) | Lab data that must stay online and shared | CephRDS (pilot) |
| Collaboration | Documents, manuscripts, small shared files | Google Drive and OneDrive |
| Archive (cold) | Data you must keep but rarely read | Cloud archive |
| Cloud object | Data used by cloud workloads | Cloud accounts, Ursa Major |
| Published | Data shared openly with a DOI | Dryad and other repositories (see below) |
HPCC storage (GPFS)
Fast parallel storage attached to the cluster, for data being actively computed on.
CephRDS
On-campus project storage for lab data, instruments and sequencing libraries, with S3 access.
Google Drive and OneDrive
Campus collaboration storage for documents, manuscripts and small files. Managed by ITS.
Cloud archive (Ursa Major)
Long-term cloud archive for data you must keep but rarely read. Eligibility and limits apply.
Backups are not archives
A backup is a recent copy kept so you can recover from a mistake or a failure. An archive is the long-term record you are required to keep. They solve different problems:
- Know which of your storage is backed up, how often, and for how long. Each service page and its terms say so; when in doubt, ask.
- Keep at least one copy of irreplaceable raw data somewhere other than where you compute on it.
- Before you archive, check how long you are required to keep the data. See records retention.
Publishing data
Funders and journals increasingly ask for data to be shared. UC researchers can deposit data in Dryad, a general-purpose repository that issues a DOI for each dataset. The UCR Library can advise on repositories and data management plans.
Moving data
For large transfers between campus systems, national systems and collaborators, see Globus data transfer. For CephRDS, see the rclone guide.