Building our own video player: HLS, ffmpeg and a watermark you can't delete

Why build a video player at all?

Every browser already plays video. You drop a <video> tag on the page, point it at an MP4, and you're done. YouTube and Vimeo will host and stream it for you. So why did a photography studio write its own player?

Because our videos aren't public content. They're a couple's pre-wedding film, delivered privately, and often not paid for yet. We needed four things no off-the-shelf option gave us together:

• Privacy. Films live in our private storage, and every link must be signed and expire. No public hosting, no "unlisted" links that get forwarded.
• A preview you can't strip. Before the balance is paid, customers watch a watermarked version. That watermark has to survive developer tools, screen recording and cropping.
• Phones on bad networks. A 4 GB original on a phone stalls. The film has to adapt to the connection.
• Our own look and controls. A quality menu, a switch to the full-quality original, hover previews on the timeline, and the studio's charcoal and gold.

This post walks through how it works, from ffmpeg on the server to the play button in the browser. If you haven't read Part 5 on uploads and downloads, it's a good warm-up.

How adaptive streaming works

We use HLS (HTTP Live Streaming). The idea is simple:

1. Encode the film at several qualities. This set is called a ladder.
2. Cut each quality into short pieces, a few seconds each.
3. Write a master playlist that lists the qualities, and a playlist per quality that lists its pieces.
4. The player downloads the master, measures how fast pieces arrive, and picks the best quality the connection can keep up with, switching as conditions change.

A master playlist is just a text file. Ours look roughly like this:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=6500000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
1080p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3300000,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2"
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1400000,RESOLUTION=854x480,CODECS="avc1.64001e,mp4a.40.2"
480p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=850000,RESOLUTION=640x360,CODECS="avc1.64001e,mp4a.40.2"
360p.m3u8

On a train, the player settles on 360p and keeps playing. On home Wi-Fi it climbs to 1080p. The customer doesn't touch anything.

Our ladders

Not every video needs every quality. We use three ladders:

Each quality has a bitrate ceiling, so a quiet scene doesn't waste data and a busy one can't blow the budget: about 6 Mbps for 1080p, 3 for 720p, 1.2 for 480p and 0.7 for 360p.

The ffmpeg command

Every quality is made by one ffmpeg run. Here's the heart of it, trimmed:

List<String> cmd = List.of(
    "ffmpeg", "-nostdin", "-loglevel", "error",
    "-i", sourceUrl,
    "-vf", scale + "," + watermarkFilter,        // resize, then burn in the watermark (previews only)
    "-c:v", "libx264", "-preset", "veryfast", "-crf", "23",
    "-maxrate", "3M", "-bufsize", "6M",          // the 720p ceiling
    "-pix_fmt", "yuv420p",
    "-force_key_frames", "expr:gte(t,n_forced*2)",
    "-sc_threshold", "0",
    "-c:a", "aac", "-b:a", "128k", "-ac", "2",
    "-f", "hls", "-hls_time", "4", "-hls_playlist_type", "vod",
    "-hls_segment_type", "fmp4",
    "-hls_flags", "single_file+independent_segments",
    "720p.m3u8");

Three flags do most of the clever work.

-crf 23 with -maxrate. CRF means "constant quality": ffmpeg spends as many bits as each scene needs to look the same. The max rate caps it, so a confetti shot can't spike the bitrate and stall the player.

Keyframes every 2 seconds, in every quality. A keyframe is a complete picture; the frames between are just changes. The player can only switch quality at a keyframe. If 1080p had keyframes at 2.0 s and 720p at 2.3 s, switching would stutter. -force_key_frames puts them on the same 2-second grid in every quality, and -sc_threshold 0 stops ffmpeg adding extra ones at scene cuts. The result: quality switches you can't see.

single_file. This one needs its own section.

One file per quality

Normally HLS writes each quality as hundreds of small files: a 20-minute film in 4-second pieces is 300 files per quality, 1,200 for the ladder.

For us that was a trap. Our storage is private, so every file the player fetches needs a signed URL, and on Cloud Run each signature is a network call of about 65 ms (Part 3 tells that story). 1,200 signatures for one film is over a minute of work.

With -hls_flags single_file, each quality becomes one fragmented MP4 file, and the playlist points at byte ranges inside it, something like:

#EXT-X-MAP:URI="720p.mp4",BYTERANGE="1245@0"
#EXTINF:4.000,
#EXT-X-BYTERANGE:1893412@1245
720p.mp4
#EXTINF:4.000,
#EXT-X-BYTERANGE:1776033@1894657
720p.mp4

One signed link per quality, however long the film. Browsers fetch the ranges with ordinary HTTP range requests, which Cloud Storage supports natively. As a bonus, each file also plays as a normal MP4 on its own.

Proving you're allowed in, without headers

The player downloads the master playlist, then follows the links inside it by itself. Safari's built-in player won't add custom headers to those requests, so the usual "send a token in the Authorization header" doesn't work.

So the credential goes in the URL path:

/api/v1/stream/{token}/master.m3u8
/api/v1/stream/{token}/720p.m3u8      ← a relative link, so it inherits the token

The token is short-lived, and the server decides at creation time which version it unlocks: preview or clean. Nothing the browser does can turn one into the other.

A watermark burned into the pixels

The obvious way to mark a preview is a "PREVIEW" label in CSS on top of the video. Anyone can delete it in developer tools in five seconds, and the video underneath is the finished film.

So previews are a separate encoding with the watermark drawn into every frame by ffmpeg's drawtext filter:

• the studio mark, large, in the centre;
• two fainter copies at 18% and 80% of the height, so cropping out the middle still leaves one;
• the customer's id, drifting slowly around the frame, so a leaked screen recording can be traced back.

The drift is the fun part. Its position is plain maths, evaluated by ffmpeg for every frame:

x = (w - text_w) × (0.05 + 0.9 × |((t / 9) mod 2) − 1|)
y = (h - text_h) × (0.08 + 0.84 × |((t / 13) mod 2) − 1|)

|(t mod 2) − 1| is a triangle wave: it slides from 1 to 0 and back, forever. The horizontal one takes 18 seconds for a round trip and the vertical one 26. Because 9 and 13 share no common factor, the combined path only repeats every 234 seconds, so the id wanders over the whole frame instead of tracing one short line you could blur out. Try it:

The customer id is written to a small text file and passed to ffmpeg with textfile=, after being reduced to a safe set of characters. So nothing in it ever has to be escaped inside the filter, and nothing a customer types can break the command.

Photos get the same treatment in a still version: five bands of the mark across the frame, offset left and right so no single crop clears them, and the id twice.

Encoding as background jobs

Encoding 1080p takes minutes, sometimes more than ten for a longer film on a single CPU. It can't happen while someone waits.

• One job per quality. Each goes onto a Cloud Tasks queue, which calls our encoder service over HTTP. Cloud Tasks gives a job at most 30 minutes, so four smaller jobs are safer than one big one. (Part 4 explains why jobs must arrive as HTTP requests on Cloud Run.)
• Whoever finishes last writes the master. Each job saves a small .inf file with its playlist line. After each job, the service checks whether every quality's .inf exists; if so, it writes master.m3u8. Writing it is idempotent, so two jobs finishing together can't produce a broken master.
• Only one caller queues the work. A conditional database update claims the job (Part 4 again), so a customer and a staff member opening the same film at once don't encode it twice.
• Memory is precious. On Cloud Run, /tmp lives in RAM. Each job encodes one quality, uploads it, deletes it and only then finishes.
• Failure is visible. After the last retry, the error is stored on the video's row, and staff see "Failed" with a Rebuild button.

Hover previews on the timeline

Hover over a YouTube progress bar and you see a tiny frame from that moment. We wanted the same, without the player downloading video to find it.

After a stream is ready, a job grabs one small frame every few seconds (every 5 seconds, or less often for long films, so there are never more than about 200), and tiles them 10 × 10 into JPEG sprite sheets, plus a thumbs.json describing the grid. It's written last, so a half-finished set is never visible.

Finding the right frame is a little arithmetic:

const frame = Math.floor(hoverSeconds / thumbs.interval);
const perSheet = thumbs.columns * thumbs.rows;        // 100 frames per image
const sheet = thumbs.sheets[Math.floor(frame / perSheet)];
const inSheet = frame % perSheet;

style = {
  backgroundImage: `url("${sheet}")`,
  backgroundPosition: `-${(inSheet % thumbs.columns) * w}px -${Math.floor(inSheet / thumbs.columns) * h}px`,
};

One image download gives you 100 hover frames. And the frames are cut from the same flavour's smallest quality, so hovering over a preview shows watermarked frames. A hover card must never leak what the paywall protects.

The player itself

The browser side is a React component, TheRarePlayer, used everywhere a video appears: customer galleries, shared galleries, raw footage, the admin panel and our public films.

• hls.js, loaded on demand. Most pages have no video, so the library is fetched with a dynamic import() only when a player starts. Where the browser supports Media Source Extensions (including newer iPhones), hls.js runs the show and we get a manual quality menu. On older iPhones we fall back to Safari's native HLS, with "Auto" only.
• Don't fetch 1080p for a thumbnail-sized player. capLevelToPlayerSize: true stops hls.js choosing a quality bigger than the player is on screen.
• A tidy quality menu. A small source can encode to the same size at several rungs, and "480p, 480p, 480p" is noise, so duplicate labels are merged.
• "Original", at the same moment. Paid customers can switch from the stream to the full-quality original file. The player remembers the position and resumes there. Camera formats the browser can't decode fall back to the stream, with a note pointing to Download.
• Keyboard shortcuts. Space or k to play, ← → to jump 5 seconds, f for fullscreen, m to mute.
• Honest states. "Preparing your video" while encoding is still running, a clear error with a Retry, never a blank black box.
• Preview mode. On top of the burned-in watermark, it disables picture-in-picture and the right-click menu, shows a "Preview" badge and an unlock button, and floats a drifting label with the viewer's own email. That matters for guests on a shared gallery, whose identity isn't in the pixels. These are deterrents; the pixels are the protection.

What's still rough

• When a stream finishes encoding, an open player doesn't notice. The customer has to reload the page.
• Each quality must encode within 30 minutes. Very long films on a single CPU get close.

Writing these down is part of the job. The next person to touch the player knows exactly where to start.

Takeaways

• Adaptive streaming is a ladder of qualities plus a playlist; the player does the rest.
• Align keyframes across every quality, or quality switches will stutter.
• Match your file layout to your costs: one file per quality saved us over a thousand signatures per film.
• Put credentials where the player will carry them: in the path.
• Protection lives in the pixels the server sends. Overlays are decoration.
• A little maths (two triangle waves) makes a watermark far harder to remove.
• Load heavy libraries only when they're needed.

Part 7 zooms out to security across the whole platform.