17
$\begingroup$

Scientific spacecraft can generate very large amounts of data. However, both the storage available onboard the spacecraft and the bandwidth available to transmit the data to ground stations are limited. Thus, data compression is needed. What algorithms have been used successfully on spacecraft?

$\endgroup$
  • $\begingroup$ For Voyager's later life (section "Voyarer's Future"): "Voyager Project officials decided to use the Flight Data System in dual processor mode for the first time for the Uranus encounter to provide image data compression. Thus, the information content remained high even though the transmission rate was grossly reduced." "Recent Advances in Coding Theory for Near Error-Free Communications"(Cheung et al.): "technique pioneered by R. F. Rice ... algorithm is essentially a universal source code on the differences between successive pixels". $\endgroup$ – Paul A. Clayton Jul 17 '13 at 22:53
  • 1
    $\begingroup$ I've edited the tagging, removing the flight-computer tag and adding the data-systems tag. A flight computer is a specific piece of hardware. There can be other processors onboard a spacecraft. For a modern scientific mission, each instrument usually has its own processor; furthermore, data compression is often done by these processors so that the spacecraft's main processor doesn't have do this extra work. $\endgroup$ – GreenMatt Aug 16 '13 at 14:26
  • $\begingroup$ One of the things that's always bothered me is that people argue about how little info is lost in various lossy compression schemes ... and not how it affects detection of features. To the best of my knowledge, though, there's no archive of uncompressed data + the automated detection routines that need to run on it. $\endgroup$ – Joe Aug 17 '13 at 0:07
  • $\begingroup$ This is a broad, open-ended question with almost as many answers as there have been, and will be, spacecraft. No one is going to be able to write a "complete" answer, but several of us can describe what we know. (This is not the ideal kind of question for StackExchange style sites, though a good and interesting question anyway.) $\endgroup$ – DarenW Aug 19 '13 at 7:32
  • $\begingroup$ @DarenW: Poor choice of words on my part, thanks for pointing it out. That may explain the less than satisfactory answers I have received. I've edited to replace "techniques" with "algorithms". $\endgroup$ – GreenMatt Aug 19 '13 at 14:28
15
$\begingroup$

Image data is typically compressed using standard image compression algorithms. For example, the National Imagery Transmission Format, MIL-STD-2500C, defines a standard, extensible format for the transmission of imagery data within the military. The standard calls out several types of image compression compliant with the standard (such as JPEG).

Telemetry data is often not compressed at all. One rationale is that it is often several orders of magnitude less data than what is being generated by the payload (be it imagery, communications, or data from another type of instrument or sensor), and thus in the broad scheme of things not worth compressing. Telemetry is also often sent as a stream of data (rather than a file), which limits the types of compression schemes that could be used.

In fact, radio frequency signals are often intentionally increased in size through techniques like forward error correction (FEC) and convolution coding, which are means of adding redundant data to a signal to make the transmission more robust to transmission errors endemic to space communication links.

That said, there are standards for the lossless and lossy compression of spacecraft data. The Consultative Committee for Space Data Systems (CCSDS) has published a set of standards for Space communication links. There are currently four released standards that discuss lossless data compression and image data compression.

$\endgroup$
  • $\begingroup$ Thank you for the amazing sources, the lossless compression article was specifically interesting to me. $\endgroup$ – Magic Octopus Urn Jul 11 '18 at 21:21
13
$\begingroup$

I will focus on meteorological satellites. They're a prime example of satellites that measure very large quantities of data (and one I have personal experience with). The bottom line is: they don't really compress the data, but they might degrade/limit it. A specific property of meteorological satellites is that the users want the data fast, so that could make a difference compared to other kinds of satellites.

The Advanced Very High Resolution Radiometer (AVHRR) is a meteorological imager that has flown on meteorological satellites from NOAA and EUMETSAT since 1979. It images the entire Earth at a resolution of 1 km in five simultaneously operated channels. This resolution is so high, that the relatively short time during which it has a downlink connection is not enough for the NOAA satellites to downlink all the data. Therefore, they downlink global data only in a special format known as Global Area Coverage (GAC): 4 adjacent pixels are averaged, and 2 out of 3 scanlines are ignored. Arguably, this is a kind of lossy compression. Users that need the full-resolution data for their own region can download it directly from the satellite (and national meteorological agencies do), or pre-order full-resolution data for specific regions (but not globally).

Three properties are limited:

  1. bandwidth
  2. storage
  3. computational power

With enough computational power, the NOAA satellites could compress all the data and downlink all of it. Alas, the NOAA KLM series dates from 1999 and they don't have the computational power. Since then, all the three aforementioned properties have grown, and as far as meteorological satellites are concerned, it appears bandwidth and storage have grown more than computational power. Hence, I don't think there's any advanced compression going on for the vast amounts of data measured by meteorological satellites.

Of course, there are other satellites that also collect large quantities of data. I have no expertise on those, but maybe other people can contribute with relevant answers.

$\endgroup$
  • $\begingroup$ Can you clarify that no compression is going on? Or are they compressing it, e.g. with PNG, and that any further gains would take too much CPU/complexity/newer equipment? I find it hard to imagine that they are sending RAW data! $\endgroup$ – NPSF3000 Dec 10 '14 at 22:32
  • $\begingroup$ @NPSF3000 Meteorological satellites might continuously transmit raw data, line by line. This allows users to read out the images directly, as the satellite passes overhead. Compression makes sense if you dump the data once per orbit, but that would cause more delay than the application of nowcasting tolerates. $\endgroup$ – gerrit Dec 10 '14 at 22:57
  • $\begingroup$ even if you are streaming data there are some very simple compression techniques that can save a buttload of data. I have no knowledge of what they do, hence why I'm asking. $\endgroup$ – NPSF3000 Dec 10 '14 at 23:02
  • 1
    $\begingroup$ @NPSF3000 It is my understanding that they don't compress, I am 99% sure of that, but not 100%. $\endgroup$ – gerrit Dec 10 '14 at 23:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.