How to transform from the spacecraft coordinate system to image coordinates?

Question

Assume e.g. that in the spacecraft coordinate system, I have a target vector

$ \vec{p_\text{t}} = \begin{pmatrix} -11.980\\ -68.051\\ 55.031 \end{pmatrix}\text{km}$,

which is related to the Earth Mean Equator (EME) J2000 reference frame. I also have the spacecraft offset vector relative to EMEJ2000

$ \vec{p_\text{sc}} = \begin{pmatrix} -262041633.729\\ 115960324.885\\ 89603436.304 \end{pmatrix}\text{km}$,

and its rotation quaternion (in the form $[\text{cos}(a/2), n_x \text{sin}(a/2)$, $n_y \text{sin}(a/2), n_z \text{sin}(a/2)]$)

$ q_\text{sc} = \left( 0.86476171, -0.43595577, -0.05902956, 0.24216783 \right)$,

as well as the offset vector of the spacecraft origin relative to the camera

$ \vec{p_\text{cam}} = \begin{pmatrix} -0.001234\\ -0.000157\\ 0.001390 \end{pmatrix}\text{km}$,

and its rotation quaternion (also in the form $[\text{cos}(a/2), n_x \text{sin}(a/2)$, $n_y \text{sin}(a/2), n_z \text{sin}(a/2)]$)

$ q_\text{cam} = \left( 0.99999815, 0.00009373, 0.00027332, 0.00190238 \right)$.

How can I use this information to calculate the position of the target in image (pixel) coordinates?

The camera has a FOV of $2.208° \times 2.208°$, $2048 \times 2048$ pixels, a resolution of $1.882 \cdot 10^{-5}$rad, a focal length of $0.7173\text{m}$, and a pixel size of $13.5 \cdot 10^{-6}\text{m}$.

Answer 1

I wrote a paper on a similar problem, long time ago. I'll try to provide the gist of it.

You need a model for the camera, so that you can map a point in 3D space to a point in the image. A simple model that is useful for this is the pinhole camera model. This model assumes an image plane at the focal depth of the camera.

In the following I use $p_\text{name}^{\Psi}$ to denote vector $p_\text{name}$ expressed in coordinate frame $\Psi$ and $H_{\Psi_1}^{\Psi_2}$ to denote a homogeneous coordinate transformation from $\Psi_1$ to $\Psi_2$.

Start with the 3D coordinates of the target and apply a coordinate transformation to a coordinate frame that has its origin in the camera's aperture:

$$ \begin{bmatrix} p_\text{target}^\text{camera} \\ 1 \end{bmatrix} = H_\text{EME}^\text{camera} \cdot \begin{bmatrix} p_\text{target}^\text{EME} \\ 1 \end{bmatrix} . $$

The transformation matrix $H_\text{EME}^\text{camera}$ can be computes as

$$ H_\text{EME}^\text{camera} = H_\text{sc}^\text{camera} \cdot H_\text{EME}^\text{sc}, $$ with $$ H_\text{EME}^\text{sc} = \begin{bmatrix} R_\text{EME}^\text{sc} & p_\text{sc}^\text{EME} \\ 0 & 1 \end{bmatrix} $$ and $$ H_\text{sc}^\text{camera} = \begin{bmatrix} R_\text{sc}^\text{camera} & p_\text{camera}^\text{sc} \\ 0 & 1 \end{bmatrix}. $$

$R_\text{EME}^{sc}$ and $R_{sc}^{camera}$ are rotation matrices that follow from your spacecraft and camera quaternions. See the Wikipedia page on quaternions to see how to convert from quaternions to rotation matrix.

With the target point converted to the appropriate camera coordinate frame, you can project $p_\text{target}^\text{camera} = (x, y, z)^\text{target}_\text{camera}$ onto the image plane as follows:

$$ \begin{aligned} a &= \frac{f}{z} x, \\ b &= \frac{f}{z} y, \end{aligned} $$

where $x,y,z$ are the coordinates of the target in the camera coordinate frame, $f$ is the focal length of the camera, and $a,b$ are the 2D coordinates of the target in the image plane.

From this, use the pixel size (pitch) to convert to image pixel coordinates.

The focal depth of the camera is typically known. This model does not take into account distortion by the lens, but I assume that the images you are provided with have been corrected already for visual distortions. If you have raw imagery that has not been corrected yet, you need to apply some correction model, which goes beyond this answer and my expertise.