S3 Viewer
Open app

How to upload large files to S3 (multipart upload)

TL;DR

S3 Viewer streams large files through its server into multipart uploads without bucket CORS setup, on AWS S3, Cloudflare R2, Backblaze B2, and other S3-compatible endpoints alike. Uploads protect existing files: AWS S3 and R2 enforce the condition directly, other endpoints get a HEAD check performed by S3 Viewer immediately before the write. Replacements require confirmation and readable original permissions and settings. New files may retry network failures; uncertain replacements require fresh review. The default app upload limit is 5 GiB and can be configured when self-hosting.

Steps

Step-by-step.

  1. 01

    In S3 Viewer: Upload → Upload files, or drag files onto the list

    Open the Upload menu and choose Upload files (or Upload folder), or drag files straight onto the file list. Your browser sends each file to the S3 Viewer server, which streams it into the bucket as a multipart upload without writing it to disk. This works on AWS S3, R2, B2, and other S3-compatible endpoints. No bucket CORS policy is needed. Existing files require confirmation and readable settings before replacement; some MinIO setups cannot provide those settings, in which case replacement is refused rather than guessed at.
  2. 02

    Review uncertain results before replacing a file

    New-file uploads can retry a lost browser connection while still protecting existing objects. Replacements are not automatically retried after an uncertain result: refresh and review first. Three files upload at once, and you can cancel them. The server attempts to remove unfinished parts when an upload fails or is cancelled.
  3. 03

    AWS CLI: aws s3 cp

    The CLI uses multipart automatically above 8 MB. No flags needed for normal files; tune multipart_chunksize in your AWS config for very large ones.
    aws s3 cp big.zip s3://bucket/big.zip
  4. 04

    AWS SDK: Upload from @aws-sdk/lib-storage

    Handles part splitting, parallelism, and retries for you. The right pattern for streaming uploads from a backend.
    import { Upload } from '@aws-sdk/lib-storage';

const u = new Upload({
  client: s3,
  params: { Bucket: 'b', Key: 'big.zip', Body: stream },
  queueSize: 4,    // parts in flight
  partSize: 8 * 1024 * 1024,
});
await u.done();
  5. 05

    Tune part size for very large files

    Default partSize in the SDK is 5 MB; raise to 64–100 MB for files over 50 GB so you don't hit the 10,000-parts cap. Math: max object size = partSize × 10,000.
  6. 06

    Verify with HeadObject

    After the upload completes, confirm the size and ETag match what you intended to upload.
    aws s3api head-object --bucket b --key big.zip
  7. 07

    Set a lifecycle rule to abort failed uploads

    Incomplete multipart uploads sit in the bucket and you pay for them until aborted. Add an S3 lifecycle rule to auto-abort uploads older than a few days — AWS recommends this in the well-architected framework.

Under the hood

What's actually happening.

S3's PutObject caps at 5 GB and is non-resumable — a network blip on a 4 GB upload restarts from byte zero. Multipart upload (CreateMultipartUpload → many UploadPart → CompleteMultipartUpload) splits the file into parts (5 MB minimum, 10,000 parts max), uploads them in parallel, and lets you retry only the failed parts. S3 Viewer uses multipart upload for large files and a single upload for small files. Both travel through the server without writing file contents to disk or requiring bucket CORS setup, on every connected provider. AWS S3 and R2 enforce the app's overwrite protections directly; B2 and other endpoints get the same check performed by S3 Viewer with a HEAD request immediately before the write. Replacement requires readable original permissions and settings, so unsupported encryption or unavailable settings stop it before the existing file is changed.

FAQ

Common questions.

What's the max file size I can upload to S3?

S3 Viewer's default upload limit is 5 GiB per file. A self-hosted instance can change MAX_UPLOAD_BYTES, subject to the app's multipart and provider limits. Large files stream in parts on every connected provider, including B2.

What is multipart upload in S3?

A three-step S3 API flow — CreateMultipartUpload, UploadPart (in parallel, 5 MB minimum each, up to 10,000 parts), and CompleteMultipartUpload. It splits a large file into pieces, uploads them in parallel, and lets you retry only the failed parts. Required for anything over 5 GB; recommended for anything over 100 MB.

What happens if my upload fails halfway?

S3 Viewer may resend a new-file upload after a browser network error, while keeping protection against overwriting an existing file. An uncertain replacement requires fresh review instead of automatic retry. Failed uploads attempt to clean up unfinished parts; a bucket cleanup policy helps cover interruptions where cleanup could not be confirmed.

Do I get charged for failed multipart uploads?

Yes — incomplete multipart uploads are stored and billed at standard storage rates until you abort them. Set an S3 lifecycle rule to auto-abort incomplete uploads after a few days.

Does Cloudflare R2 support multipart upload?

Yes. S3 Viewer supports multipart uploads to R2 and protects against overwriting an unreviewed file. Replacements require readable original settings, and R2 object rename and deletion both work the same way they do on AWS S3.

How do I upload large files to S3 from a browser safely?

Keep the credential on a server. Either the browser sends the file to your server, which streams it into the bucket with lib-storage's Upload (what S3 Viewer does — the bucket needs no CORS policy and the key never reaches the client), or your backend issues presigned URLs per part and the browser PUTs to them directly, which needs a CORS rule on the bucket allowing your origin.

Use S3 Viewer for this

Skip the CLI. Try it in the browser.

S3 Viewer turns the steps above into a single click. Open source, self-hostable, free.

Open appStar on GitHub