1. What OSN is
The Open Storage Network is national S3 object storage funded by the NSF. It is allocated through NSF ACCESS and reached with ordinary S3 tools, from a laptop, the HPCC, Nautilus, an OSPool job or the cloud.
Object storage is not a mounted drive. You put and get whole files ("objects") in a bucket over HTTPS. In exchange, any number of people and machines can read the same data from anywhere.
2. When to use it
| You want to | Use |
|---|---|
| Share a dataset with collaborators at other universities | OSN |
| Make a dataset publicly downloadable for a grant or paper | OSN (anonymous read bucket), or a repository such as Dryad for a DOI |
| Feed the same data to jobs on several platforms | OSN, optionally connected to the OSDF |
| Keep lab data on campus, shared within UCR | CephRDS |
| Compute on data at the HPCC | HPCC storage |
| Keep records you are required to retain | Records retention and the cloud archive |
OSN is for open science data only. Do not put protected data there.
3. Buckets, keys and endpoints
Each OSN bucket has:
- an endpoint, the address of the pod that holds it (for example
https://sdsc.osn.xsede.org); - a name and a quota in terabytes;
- a read-write key pair for you and your writers;
- a read-only key pair you can hand to readers;
- an anonymous access setting: when on, anyone can read without keys (writing always needs keys).
You find all of these on the bucket's page in the OSN portal. Treat keys like passwords: never commit them to Git or paste them in email.
4. The path from nothing to data in a bucket
- Getting OSN storage: an ACCESS project, exchanging credits for OSN, signing in to the OSN portal, requesting a bucket, adding people.
- Moving data in and out of OSN: rclone, the AWS CLI, Cyberduck and Python; uploading, downloading, syncing and sharing.
- Using OSN from compute systems: reading and writing OSN from the HPCC, Nautilus, OSPool jobs and notebooks, and keeping your project in good standing.
5. Getting help
- OSN: help@osn.mghpcc.org and the OSN documentation.
- ACCESS allocations: ACCESS support.
- Research Computing can look over your ACCESS request and help you choose between OSN and campus storage. Contact us.