I have been reading about outlier detection algorithm for improving determinations based on realistic data.

Outlier exclusion is an important tool in any analysis of realistic noisy data. There are many ways to detect/identify outliers, and some may be better than others for orbit determination.

I've been reading the book Advanced Kalman Filtering, Least-Squares and Modeling: A Practical Handbook by Bruce P. Gibbs. The author mentions outlier detection but it is discuss for only a few pages. He introduced two ways which are De-Weighting Large Residual and Data Editing. He said that data editing is used with much more success in satellite orbit determination.

I'd like to better understand if this is in fact true, and if so, why. Any other tips on the topic would be appreciated as well.

Three Algorithm

1) 𝑿^𝟐=γ€–(π’šβˆ’π’‰(𝒙))γ€—^𝑻 𝑺^(βˆ’πŸ) (π’šβˆ’π’‰(𝒙))

2) Residual(i) > 3√(HP 𝐻^𝑇 + R)

3) 3-Sigma

EDITED: I want to share what I learn from outlier detection if I am wrong, Please improve me! The RMS of the residuals(eq.7.4-4) is computed first, and then significance of the residual is checked under the n.RMS value(eq.7.4-3). When the significance of the residual is larger than the n.RMS(eq.7.4-3) value, the corresponding observation is marked. If there exists any outlier in the observations, the OD process is repeated in which the marked observations are not used.

Also, to find the significance of residual instead of (eq.7.4-3), only 3*RMS can be used but I don't know how much it is sensitive.

Therefore, If I am not wrong, OD process will run minimum two times if there is at least one outlier. The process: first-calculate residuals and RMS and Second-If there is an outlier, don't calculate and skip that measurement for next in OD performance.

I think the eq7.4-5 is not correct because For example, if you have 3 measurement data(range, azimuth, and elevation), you can obtain one significance of residual such as matrix (3x1)^T*(3x3)*(3x1)= (1,1) so that the result should be 3x1 because the significance of residuals should be calculated for each sensor type. Otherwise, limit of the significance of residuals will be same for all different sensors.

enter image description here

  • 1
    $\begingroup$ This is a great question! I've adjusted the wording of your question a bit to make it better suited to the way Stack Exchange works. "What's the best..." phrasing is discouraged as it leads to answers that are primarily opinion based. In this case it's clear you'd like to understand more the tradeoffs between various methods and how one might go about determining which method is objectively better for orbit determination problems. By the way, is the book's title Advanced Kalman Filtering, Least-Squares and Modeling: A Practical Handbook? $\endgroup$
    – uhoh
    Jun 28, 2018 at 23:25
  • 1
    $\begingroup$ @uhoh Thanks for the editing. Yes, that's book. you can find the subject end of chapter 7. $\endgroup$
    – Ugur
    Jun 29, 2018 at 19:16
  • 1
    $\begingroup$ Great, thanks! Next time just go ahead and edit your post. In Stack Exchange we use the comments to discuss and improve the posts, but since comments are considered temporary, it's always best to incorporate new information back into the main post where more people will see it. I've started to read the book now, and it's really interesting! Thanks for the referral. $\endgroup$
    – uhoh
    Jun 29, 2018 at 23:25
  • 1
    $\begingroup$ Also, there is Kalman filter in Chapter 8, I which is mentioned about data editing. I will also read again, because i looked at quickly before. i understand %80 the outlier but i need to figure out when you detect the outlier how can you remove/avoid from them. I can share what I understand and make an algorithm for that or small example according to data editing :) $\endgroup$
    – Ugur
    Jun 29, 2018 at 23:33
  • $\begingroup$ What kind of data do you have? And how many measurements? My guess, as a novice who has only written OD tools using simulated (and noisy) data (no real data), is that if you have enough measurements, your covariance might have a short blip, but your estimate error should recover quickly. Moreover, one of the techniques often used is to run several filters at the same time with different starting and ending measurements, eventually iterating on them if they are batch of CKF filters. Hence, I'm not sure you need to remove outliers. $\endgroup$
    – ChrisR
    Jun 30, 2018 at 1:30


Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.