Moving gigabytes through a browser: uploads and downloads

The heaviest data in the system

A single shoot can produce hundreds of gigabytes: camera originals, drone footage, 4K raw clips and the finished films. All of it has to get from the studio to our storage, then from storage to a customer who may be on a phone on mobile data. This post covers the two directions that data moves:

• In: staff uploading multi-gigabyte videos from a browser.
• Out: customers downloading dozens of files, or just a few seconds of one.

The principle behind both: the bytes should take the shortest path, and never pass through our API.

Uploads: why the server never sees the file

The obvious way to upload is a form that posts the file to the API, which saves it to storage. That fails in three ways at our sizes:

• Cloud Run rejects request bodies over about 32 MB. A single 4K clip is often several gigabytes.
• The API would hold the whole file while it forwards it, tying up memory and a container.
• We'd pay to move every byte twice.

So uploads go straight from the browser to Cloud Storage. The API's only job is to say yes:

1. The browser asks the API for an upload ticket: "I want to upload ceremony.mp4, video/mp4, to booking 812."
2. The API checks the staff member's permissions and returns a signed upload URL for one specific object path.
3. The browser uploads the file directly to that URL.
4. The browser tells the API "done", and the API records the file in the database.

The content type is part of the signature, so the browser has to send exactly the type it asked for. That's a feature: a ticket for a video can't be used to upload something else.

Resumable uploads: surviving a bad connection

Our first version uploaded each file with one request. If the connection dropped at 65% of a 1 GB video, the upload failed and started again from 0%. On mobile data that could mean never finishing.

Cloud Storage supports resumable uploads. The browser opens an upload session, then sends the file in chunks. If a chunk fails, it asks storage how much actually arrived and carries on from there:

const GCS_CHUNK_SIZE = 16 * 1024 * 1024;   // must be a multiple of 256 KiB

while (offset < total) {
  const end = Math.min(offset + GCS_CHUNK_SIZE, total);
  const chunk = file.slice(offset, end);            // no copy: just a view of the file
  try {
    const result = await putChunk(sessionUri, chunk, `bytes ${offset}-${end - 1}/${total}`);
    if (result.done) return;
    if (result.committed > offset) consecutiveFailures = 0;
    offset = result.committed;
  } catch (err) {
    if (++consecutiveFailures > MAX_FAILURES) throw err;
    await sleep(Math.min(15_000, 1000 * 2 ** (consecutiveFailures - 1)));  // back off
    const status = await queryStatus(sessionUri, total);  // "how much did you get?"
    offset = Math.max(offset, status.committed);
  }
}

The details that matter:

• 16 MB chunks. A 1 GB video is about 64 requests, and a dropped connection loses at most 16 MB.
• Ask, don't assume. A "failed" chunk often partly arrived. Asking storage how many bytes it committed is what turns a drop at 65% into a resume from 65%.
• Exponential backoff, doubling the wait up to 15 seconds, so a flaky connection isn't hammered.
• The failure counter resets whenever progress is made, so a long upload on a patchy network doesn't give up because of failures spread over an hour.
• Survive a page reload. The session URL stays valid for days, and it's saved. If the tab closes, picking the same file again continues from where storage says it stopped.

Cleaning up after abandoned uploads

There's a gap in step 4. If the tab closes after the file reaches storage but before the browser says "done", the file sits in storage with no database record. Nobody can see it, and it costs money forever.

Every upload ticket leaves a small breadcrumb row in the database. At 3:30 every morning, a job looks at breadcrumbs older than the longest time an upload URL can be valid:

• If any table refers to that file, the upload finished normally: delete the breadcrumb.
• If nothing refers to it, the upload was abandoned: delete the file and the breadcrumb.

The safety check is deliberately broad. It looks in every table that stores file paths (media, raw footage, message attachments, payment proofs and more). A clean-up job that deletes the wrong thing is data loss, so the rule is: when in doubt, keep it.

Streaming: a story of its own

Customers also watch their films in the browser, including a watermarked preview before they've paid. That turned into our own video player: adaptive HLS streaming, ffmpeg encoding jobs, one file per quality to keep signing costs down, and a watermark burned into the pixels so developer tools can't remove it. It's big enough to deserve its own post, so it's Part 6.

Downloads: why not just zip it?

"Download all" seems to call for a zip file. For photos, a zip would be fine. For raw footage, which routinely runs to tens of gigabytes, a zip means every byte travels from storage, through our server, and back to the customer. We'd pay to move it twice, hold a server busy for the whole transfer, and hit the request time limit long before a big set finished.

Instead, the browser gets a signed link for each file and downloads them one at a time, 800 ms apart:

function next() {
  if (cancelled || index >= ready.length) return onDone?.(index, skipped);
  const file = ready[index++];
  const link = document.createElement("a");
  link.href = file.downloadUrl;   // signed, with Content-Disposition: attachment
  link.download = file.fileName;
  document.body.appendChild(link);
  link.click();
  link.remove();
  onProgress?.(index, ready.length, file.fileName);
  setTimeout(next, GAP_MS);
}

Why sequential? If you start 30 downloads at once, browsers silently drop most of them, and the survivors compete for bandwidth. One at a time, slightly spaced, is more reliable and finishes sooner overall.

The code comments are honest about the limits, and so is the interface: once the browser owns a download, a web page can't see its progress or restart it. "Cancel" stops the next file from starting; it can't recall files already handed over. The interface says exactly that rather than pretending otherwise.

Just the moment you want

Often a customer wants five seconds from a four-minute 4K clip, to send to family. Downloading 3 GB on a phone for that isn't realistic. So customers can mark an in and out point on a raw clip and download only that section.

The server runs ffmpeg against a read-only signed link to the original, so it reads only the part it needs and has no way to change the source. The result is saved as a new file under a separate clips/ folder, which expires automatically, because raw footage is already our largest storage cost. The original is never touched, and the full download still works exactly as before.

Takeaways

• Let the browser and storage talk directly; the API only issues permission.
• Use resumable, chunked uploads for anything large, and ask the server how much arrived rather than guessing.
• Record every upload you start, and clean up the ones that never finish, carefully.
• For big downloads, one signed link per file beats a zip.

Part 6 builds the video player on top of all this: HLS, ffmpeg, burned-in watermarks and a player of our own.