python

boto3

The AWS SDK for Python
verified in CIv1.36+boto3 documentation ↗

Install

shell
boto3>=1.36

Configure

Configure the endpoint, region, credential and bucket in this tool’s vocabulary. Path-style addressing is recommended for dotted bucket names and clients without nested wildcard TLS, but virtual-hosted addressing is also supported.

In boto3Set it to
endpoint_urlhttps://s3.canada.popcloud.ca
region_namecanada
aws_access_key_idPCAK00EXAMPLEKEYID00
aws_secret_access_key<your secret access key>
Config(s3={'addressing_style': 'path'})path
Environment
example values
export AWS_ACCESS_KEY_ID=PCAK00EXAMPLEKEYID00
export AWS_SECRET_ACCESS_KEY=<your secret access key>
export AWS_ENDPOINT_URL=https://s3.canada.popcloud.ca
export AWS_REGION=canada
export POPCLOUD_BUCKET=pc-your-org-media

These are example values. Sign in and every snippet on this site fills in with your own endpoint, key and bucket.

Python
import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url=os.environ["AWS_ENDPOINT_URL"],   # https://s3.<region>.popcloud.ca
    region_name=os.environ["AWS_REGION"],          # canada
    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
    # Recommended for predictable custom-endpoint behavior. Virtual-hosted
    # requests are supported too for ordinary single-label bucket names.
    # Force SigV4 for presigned links too. Botocore may otherwise select the
    # legacy S3 signer for a custom endpoint even though normal calls use v4.
    config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
)

Use it

Upload an object

Python
s3.put_object(Bucket=bucket, Key=key, Body=body, ContentType="text/plain")

Upload a large object

Python
# upload_file switches to multipart on its own, with retries and
# concurrency — use it whenever you do not control the size.
s3.upload_file(path, bucket, key)

Download an object

Python
response = s3.get_object(Bucket=bucket, Key=key)
data = response["Body"].read()

Download to disk

Python
# The mirror of upload_file: multipart-aware, straight to disk.
s3.download_file(bucket, key, path)

List a prefix

Python
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket=bucket, Prefix=prefix):
    for obj in page.get("Contents", []):
        keys.append(obj["Key"])

Presign a URL

Python
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": bucket, "Key": key},
    ExpiresIn=3600,  # seconds; 7 days is the hard maximum
)
# Anyone holding this URL can read the object until it expires, with no
# credentials at all.

Delete an object

Python
s3.delete_object(Bucket=bucket, Key=key)

What to watch for

These apply to boto3 specifically. The full list covers the platform.

Blocker

Keys containing . or .. path segments are rejected

You’ll see
403 SignatureDoesNotMatch for a key with a dot segment in it, such as `./relative/../path.bin`, while every other key in the same run works.
Why
The URL is normalised somewhere between signing and verification, so the path that gets verified is not the path that was signed. This is a defect.
Instead
Normalise keys before uploading — collapse `.` and `..` segments yourself. This matters most when keys are derived from filesystem paths during a migration; a bulk copy will otherwise fail on a small, unpredictable subset of objects.
Friction

Buckets are not created over the S3 API

You’ll see
405 MethodNotAllowed from CreateBucket or DeleteBucket — `aws s3 mb`, `mc mb`, or a tool provisioning its own bucket on first run.
Why
Bucket lifecycle belongs to the control plane, which also allocates the storage account, the data-plane key and the CORS rules that come with it. Letting the edge create buckets would create half of one.
Instead
Create buckets in the dashboard, or with `POST /v1/buckets` on the control-plane API. Everything object-level then works normally against that bucket.

How this page is kept true

Every snippet above was extracted from examples/python/boto3, a program that uploads, downloads, compares bytes, lists, presigns and cleans up after itself. It runs in CI against a live sandbox organisation via make docs-verify-python. If it stops passing, this page is wrong and we treat that as a bug in the product.

Not yet run against the live sandbox — the runner is wired, the first verified run stamps this line.