7
$\begingroup$

I was wondering; if all Starlink satellites had high-end cellphone quality cameras modules on simple telescopes, could the resulting images be combined and processed in such a way that a super-resolution imaging network be formed?

Could something like this have some competitive advantages, at least in some cases?


Relevant:

Super-resolution imaging

https://en.wikipedia.org/wiki/Event_Horizon_Telescope - Admittedly a global network of radio telescopes.

$\endgroup$
11

2 Answers 2

5
$\begingroup$

So, restricting myself to space observations:

No, for several reasons.

I. From an astronomical standpoint, cellphone cameras are terrible imagers. By themselves, they have tiny apertures -- typically 1 or 2 mm in diameter. Larger apertures do two things: improve the maximum possible angular resolution, and gather more light. The resolution scales with the diameter of the aperture; the Hubble Space Telescope (HST), with its 2.4-meter-diameter main mirror, has a resolution about two thousand times better than a cell-phone camera. The light-gathering power scales with the area of the aperture; this means that HST has the light-gathering power of about 5 million 1-mm-aperture cell-phone cameras.

Now, you did say "on simple telescopes", which implies that you're using the cell-phone camera just for its imager (the image sensor and the accompanying optics). But now you have to spend extra money on the actual telescope, including the special optics that send its light into the camera module; this telescope will probably be at least a meter in size and mass hundreds of kilograms, which means it will be at least as large as the Starlink spacecraft itself.

There are other problems, such as the lack of user-selectable filters. The WFC3-UVIS (UV + optical) camera on HST has about 60 different filters, for use in answering different kinds of scientific questions; a cell-phone camera sensor has no filters except for the per-pixel R, G, and B filters that consumer-use camera image sensors have. Even if you did add a filter module in front of the camera module, the fixed per-pixel RGB filters would mean that only 1/3 of the pixels would actually be usable at a given time (e.g., if you selected a "reddish" filter, only the pixels with R per-pixel filters would see any light).

Cell-phone cameras also have noisy electronics, resulting in noisy images. This is because typical cell-phone camera use involves scenes absolutely flooded with light (from an astronomer's perspective, anyway). The extra noise from the electronics is generally not noticeable in such cases. But if you're trying to observe faint astronomical objects, it actually matters. Astronomical visible-light imagers are both higher quality and cooled to liquid-nitrogen temperatures to reduce the electronic noise.

II. There's more to making a working astronomical space telescope than just sticking a camera module on a satellite. You need to be able to point the whole thing very precisely at your target, and keep it pointed in the correct direction while taking an image -- even though the satellite is moving rapidly through space. To do this, you need auxiliary ("guide") cameras and sensors, and computers to analyze the images of stars seen by the guide cameras and compute the necessary adjustments, and some means of rotating the satellite to keep it pointed properly, via gyroscopes, reaction wheels, or small thrusters.

III. "Super-resolution imaging networks" are not a thing -- except in the case of interferometric arrays (of which the Event Horizon Telescope is an example). But these work by preserving and combining the phase information of the incoming light from multiple telescopes. In the case of radio telescopes, the phase changes slowly enough that you can record it and combine it all later on a (super)computer. In the case of the EHT, the recorded data from a few days' worth of observations was so voluminous it was loaded onto hard drives that were flown to a central processing center.

Optical light changes phase much too fast to be feasibly recorded (and if you could, how would you transmit the information?), so the combination has to be done in real time by sending the light from different telescopes to a central instrument. So you don't want a "camera" recording images on each satellite; instead you want some means of redirecting the incoming light to a special central satellite where the light beams are combined. The combining has to be done with exquisite precision. This is possible on the ground, where none of the telescopes are moving; in orbit, with all the satellites constantly moving relative to each other, this would nightmarishly difficult.

(Note that I haven't mentioned anything about using "neural networks" or other forms of machine learning. That's because those would be useless, since they're meant to produce plausible-looking invented data, and what you want is real data -- what's actually out there in space right now.)

$\endgroup$
7
  • $\begingroup$ Thanks! Ia) Would the periscopes mentioned by @uhoh be worthwhile to any appreciable degree as an alternative to large cumbersome telescopes? Or would the best implementation still be very meager? Ib) That sounds very problematic. The only solution would be multiple cameras/periscopes with different filters, either on different satellites or multiple per? Complicated either way. Ic) I truly understand very little of how super-resolution imaging works and its limitations, but I think noise reduction from comparing thousands of noisy electronic images should be possible at least, right? $\endgroup$ Oct 7, 2021 at 21:44
  • $\begingroup$ IIa) If the aforementioned periscope is feasible, sized to a reasonable compact flat package for the satellite (even if the zoom still sucks at its size), the module will be very discrete compared to the satellite itself. I'm already relying on a hypothetical that may not be the case. But if said periscope is usable to an appreciable degree, then couldn't it be rotated independently of the satellite? IIb) Requiring more processing and sensors on the satellites themselves sounds like a serious limitation if the processing has to be done with very minimal latency. What's the latency rqrmnt? $\endgroup$ Oct 7, 2021 at 22:04
  • $\begingroup$ IIIa) This is I have no idea about anything territory, even more so than I and II. Optical super-resolution imaging arrays would probably have been better wording and they may not be comparable. Admittedly, the examples I've found of arrays of cameras achieving super-resolution imaging have cameras directly stacked adjacently to each other. The distance from each satellite makes things more complicated, and I don't know if any further information or inferences can be gleaned from this complexity - or if it's only a hurdle in processing images of satellites some distance from each other. $\endgroup$ Oct 7, 2021 at 22:24
  • 1
    $\begingroup$ @mancelpage The periscope camera design doesn't help at all. It allows for a smaller field of view (a telephoto effect) but doesn't change the aperture size, so you have exactly the same problems when it comes to limiting angular resolution and light-gathering power. $\endgroup$ Oct 7, 2021 at 22:25
  • 1
    $\begingroup$ This is an interesting example of combining multiple "small" cameras, where the images from the individual cameras are combined after the fact. This boosts light-gathering (less noise in final product), but the angular resolution remains the same as that of an individual camera. Note that the telephoto lenses = relatively large aperture, and the cameras are carefully aligned and oriented as a unit. astro.yale.edu/dragonfly/index.html $\endgroup$ Oct 7, 2021 at 23:07
3
$\begingroup$

This question seems to specifically be about ground observation, but the underlying limitations are the same as those for astronomical imaging.

By combining multiple images, you can do things like:

  • Reduce sensor noise or interference from ionizing particles hitting the sensor.
  • Similarly, reject transients such as flashes of reflected sunlight, passing aircraft, etc.
  • Combine imagery from different angles or times to compensate for lighting conditions, generate higher contrast images or highlight changes, cut out obstructions, or make 3D models.
  • Compensate for a low-resolution sensor.

What you can't do is increase resolution beyond the diffraction limit of the imaging optics. At best, you can produce a better image of the diffraction-blurred product of the optics.

Machine learning techniques can be used to guess at what features might produce the blurred image, but you're not truly imaging those features. If you know a vehicle is a particular make and model, you can synthesize a sharper image of it, but this is adding information to the image, not extracting more from it. A neural network will look for what it's been trained to find, rather than what's really there...in the end, you've just automated pareidolia. Look at DeepDream for extreme examples of this in action.

$\endgroup$
5
  • $\begingroup$ I'm personally more interested in space observation than ground. 'In optical Super-Resolution [imaging], the diffraction limit of systems is transcended.' Isn't the point of super-resolution imaging to create higher resolution images than what the original optics can produce? The larger the data pool, the fewer artifacts from neural networks. Related: Bayesian induction beyond traditional diffraction limit $\endgroup$ Oct 7, 2021 at 19:47
  • 2
    $\begingroup$ The optical techniques mentioned require things like structured illumination or direct probing of the near field which make them only suitable for microscopy. And neural networks, as I said, will produce what they've been trained on, not what's actually there. They don't extract additional details, they invent them. $\endgroup$ Oct 7, 2021 at 21:50
  • $\begingroup$ There are also arrays of cameras directly adjacent to each other that seem to achieve super-resolution imaging. I think the question is is this a good use case for optical super-resolution imaging rather than can optical super-resolution imaging overcome optical diffraction limits, and your above comment saying it isn't may very well be right. $\endgroup$ Oct 7, 2021 at 23:00
  • $\begingroup$ and yet space.stackexchange.com/a/24422/12102 The idea in the question is that you can have multiple images from each of many directions. In that case some people do indeed claim the final product after combining everything can be better than the simple diffraction limit of a single image. $\endgroup$
    – uhoh
    Oct 7, 2021 at 23:36
  • 1
    $\begingroup$ That "deblurring" of the Hubble image was to compensate for the improperly ground primary mirror. As the text notes, it falls far short of the diffraction limit. With correcting optics, Hubble gets images like this: esahubble.org/images/opo0322a. $\endgroup$ Oct 8, 2021 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.