MP4 can encode superwhite and superblack, thanks to analog TV
I was making a video with Premiere Pro. It was supposed to be a screen recording, with a semi-transparent watermark on top of it. Only, when I set the watermark opacity to ~20% or lower, it didn’t appear at all! Or rather, where the screen recording was white, the watermark didn’t appear at all. It was as if the screen recording was even whiter than ordinary white, burning through the watermark.
Testing the watermark on top of other other videos, it worked just fine. I made a white reference video from a white image, and the overlay worked just fine. It was just this damned screen recording - how was it so white?
Closely watching back the screen recording, it didn’t look quite right. It was as if a “high contrast” filter had been applied. Light greys from the original screen became white, and dark greys became black. This happened in any video players (e.g. VLC, or Chrome). So the screen recorder (Windows Game Bar 🤔) had done a bad job, but in doing so, had somehow managed to encode super-white and super-black colors that burned through my watermark! How?
ffprobe tool shows video file metadata;
here’s what it says about my mysterious recording:
$ ffprobe screen_recording.mp4 ... Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv), 2560x1416 [SAR 1:1 DAR 320:177], 18207 kb/s, 30.10 fps, 59.94 tbr, 30k tbn, 59.94 tbc (default)
The important thing here is
This is the “pixel format” that the file metadata claims the data is using.
yuv part means it’s using the YUV color space.
420p part means it’s using 4:2:0 chroma subsampling.
But the most important part is the
This means it encodes the Y component in the range
16..235, instead of the normal
0..255 range of a single byte.
Wait ... what the fuck?
Yes, you read that right:
black is encoded as the value
and white is encoded as the value
But why would anyone do that?
The justification seems lost to history.
But it dates back at least to 1982, when BT.601 defined a method for encoding analog TV video in digital form.
Alas, that document does not justify the strange
so presumably it dates even further back to analog TV standards.
the fact that black-to-white is squashed into the range
means that the values
236..255 are ... open to interpretation.
The most obvious interpretation is that
0..16 are “even blacker than standard black”,
236..255 are “even whiter than standard white”,
similar to a high-dynamic-range video,
This would explain my mysterious screen recording:
perhaps it encodes its luminance values using the sensible
but for some reason advertises itself as using the
16..235 “TV” range.
To test, we can get
ffmpeg to scale the color range
jpeg range (another name for
mpeg range (another name for
ffmpeg -i screen_recording.mp4 -vf scale=in_range=jpeg:out_range=mpeg screen_recording_reencoded.mp4
And hey presto, this video looks perfect! Just like the original screen!
The above reencodes the whole video, but another option should be to change the metadata
to specify that the video is encoded with
In theory, the following command does this;
but sadly it seems to keep the metadata the same:
$ ffmpeg -i screen_recording.mp4 -pix_fmt yuv420p -c copy -color_range 2 -pix_fmt yuv420p screen_recording_metadata_changed.mp4 ... $ ffprobe screen_recording_metadata_changed.mp4 ... !!!! Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv), 2560x1416 [SAR 1:1 DAR 320:177], 18223 kb/s, 30.13 fps, 59.94 tbr, 30k tbn, 59.94 tbc (default)
More by Jim
- Your syntax highlighter is wrong
- Granddad died today
- The Three Ts of Time, Thought and Typing: measuring cost on the web
- I hate telephones
- The sorry state of OpenSSL usability
- The dots do matter: how to scam a Gmail user
- My parents are Flat-Earthers
- How Hacker News stays interesting
- Project C-43: the lost origins of asymmetric crypto
- The hacker hype cycle
- The inception bar: a new phishing method
- Time is running out to catch COVID-19
- A probabilistic pub quiz for nerds
- Smear phishing: a new Android vulnerability