Accessing CephRDS with Python (boto3)

CephRDS is fully compatible with the Amazon S3 API, meaning you can interact with it programmatically using standard S3 libraries like Python’s boto3.

Prerequisites

Python 3.x installed
The boto3 library installed (pip install boto3)
Your CephRDS Access Key and Secret Key

Code Example

Below is a standard Python snippet to connect to your CephRDS bucket, list its contents, and upload a file. Because CephRDS is a private, on-premise cloud, it handles routing differently than standard AWS.

The crucial steps are:

Path-Style Addressing: You must explicitly set addressing_style='path' in the configuration.
No Region Needed: You can completely omit the region_name parameter.

import boto3
from botocore.client import Config

# Configuration
ENDPOINT_URL = 'https://rds.ucr.edu'
ACCESS_KEY = 'your_access_key'
SECRET_KEY = 'your_secret_key'
BUCKET_NAME = 'your_bucket_name'

# Initialize the S3 client
# Note: config=Config(s3={'addressing_style': 'path'}) is crucial for Ceph server
s3_client = boto3.client('s3',
    endpoint_url=ENDPOINT_URL,
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
    config=Config(s3={'addressing_style': 'path'})
)

# List objects in the bucket
response = s3_client.list_objects_v2(Bucket=BUCKET_NAME)
if 'Contents' in response:
    for obj in response['Contents']:
        print(obj['Key'])

# Upload a file
file_to_upload = 'local_data.csv'
s3_client.upload_file(file_to_upload, BUCKET_NAME, 'cloud_data.csv')
print("Upload complete!")

Security Note

Never hardcode your credentials directly into your scripts. It is highly recommended to load ACCESS_KEY and SECRET_KEY from environment variables or a .env file to prevent accidentally committing them to version control systems like GitHub.