What is the “floating dust” seen in the Starliner test landing colour footage?

Question

The first Starliner landing was in the very early morning. The live coverage was only able to show infrared pictures of the descent, however NASA's documentation of the event includes a colour video of the landing:

Something weird about this video is that there seems to be a lot of stuff floating around in the air around the capsule and parachutes, and in particular moved around by their motion.

It looks a bit like dust particles, or like video static – however it can't really be static noise, because that would not be affected by the spacecraft, and uncorrelated with the actual image information.

What's going on there?

I have a guess:

It is actually static, created by the image sensor because it was still to dark to shoot colour video. In fact the raw video contained a lot more severe noise, but they tried to reduce it in post-processing. Specifically, they used a (probably CNN-based) machine-learned denoiser. Such denoisers work essentially by pattern recognition: they have been trained on all kinds of video, including noisy and noiseless ones, and try to pick out parts that could be real instead of noise. In this case, the neural net “hallucinated” seeing floating dust particles in the static (maybe learned from underwater footage of jellyfish).
Again: this is just a guess, I don't know how they really processed the video.

Answer 1

I sincerely doubt this is "dust" of any kind. It looks and behaves like the classic sensor noise that is visible any time an image sensor is used at high gain. Noise in this case is typically described as random fluctuations in the voltage levels recorded by the image sensing system caused by factors such as local heat and variations between sensor pixels. In "normal" use, these things don't matter: if we record that something is 99% black for 99.1% black, you probably won't care... usually. But if there's very little light and so almost all the pixels are within 1% of 99% black, we can multiply the values supplied by the sensing system.

The odd bit out is that the noise appears to sort of flow with the parachute lines, but that is easily explained by the fact that many video compression algorithms work by recording the changes between frames instead of the the full content of a given frame. This is often done in comparison to "key frames", which is when there's a sudden, large-scale change in the image content; recording the delta between the previous frame and the current frame will end up taking more space than simply a compressed full-size frame.

After doing some research for this answer, I discovered there's a name for this type of compression artifact: motion compensation artifacts. In a general sense, the Wikipedia page for motion compensation and how it relates to compression is extensive, but the very small text on motion compensation block boundary artifacts in the article on compression artifacts gives a perfect description of what we see in this video:

Block boundary discontinuities can occur at edges of motion compensation prediction blocks. In motion compensated video compression, the current picture is predicted by shifting blocks (macroblocks, partitions, or prediction units) of pixels from previously decoded frames. If two neighboring blocks use different motion vectors, there will be a discontinuity at the edge between the blocks.

If you watch the gif at high magnification and watch the boundary area between the parachute lines and the clear area to the left of the parachute, you'll see that the "drift" occurs in visibly discrete blocks that look kind of like the picture below. Note that this is almost invisible in a static picture, even though it's pretty clear in motion.

The end result is that the noise (as recorded by the compression algorithm) gets tied to things around it that have more detail/contrast (in this case, the parachute) such that movement of the parachute and its lines is effectively transferred to the low-contrast areas around it.

leftroundabout raises a good point in their comment that you'd not expect to many compression artifacts on an earthbound stream, but the difference between raw video and high-quality video compressed with a modern algorithm can easily be two to three orders of magnitude. Video compression can be significantly more efficient in many cases because of things like motion compensation, so storing completely uncompressed video is extremely rare except if someone is actively editing or working with said video.

Here are two examples:

• An uncompressed screenshot of my desktop right now takes ~24 megabytes, while the same thing as as JPEG consumes merely 1MB of storage.

• Wikipedia says that uncompressed HD video at 24 FPS takes 1.4Gbit/s - that's 175 megabytes per second of video, which works out to 420GB for a single 5-minute video. By comparison, HD video on DVD can take no more than 29.4Mbit/s or 3.675 megabytes per second. Even Blu-Ray discs only bother to encode at 40Mbit/s.

Pure speculation on my part: I suspect that the video in question was recorded, compressed lightly, and that the already compressed video was bumped up a couple stops to make it easier to see the darker regions, then compressed again. That first compression step would not have prioritized the noise in the dark regions because it would be nearly invisible, but it became much more visible after the re-exposure / enhancement.

Answer 2

I agree with uhoh's comment -- this is photon shot noise and/or detector readout shot noise raised to a visible level by way of high gain applied to dark image areas. You can see similar effects most dramatically in footage (now there's an obsolete term!) from night-vision cameras, where the front-end gain makes detected photon events quite visible to the viewer.