6.4
Multiplying Multidimensional Gaussians
In the previous chapter we incorporated an uncertain measurement with an uncertain estimate by multiplying their Gaussians together. The result was another Gaussian with a smaller variance. If two piece of uncertain information corroborate each other we should be more certain in our conclusion. The graphs look like this:
In [14]: import stats
xs = np.arange(-5, 10, 0.1) mean1, var1 = 0, 5
mean2, var2 = 5, 1
mean, var = stats.mul(mean1, var1, mean2, var2) ys = [stats.gaussian(x, mean1, var1) for x in xs] plt.plot(xs, ys, label=’M1’)
ys = [stats.gaussian(x, mean2, var2) for x in xs] plt.plot(xs, ys, label=’M2’)
ys = [stats.gaussian(x, mean, var) for x in xs] plt.plot(xs, ys, label=’M1 x M2’)
plt.legend() plt.show()
The combination of measurement 1 and 2 yields more certainty, so the new Gaussian is taller and narrower - the variance became smaller. The same thing happens in multiple dimensions with multivariate Gaussians. Here are the equations for multiplying multivariate Gaussians. You will not need to remember these equations, as they are backed into the Kalman filter equations that will be presented shortly. This computa- tion is also available in FilterPy using the multivariate multiply() method, which you can import from filterpy.common.
µ = Σ2(Σ1+ Σ2)−1µ1+ Σ1(Σ1+ Σ2)−1µ2 Σ = Σ1(Σ1+ Σ2)−1Σ2
µ =σ 2 1µ2+ σ22µ1 σ2 1+ σ22 , σ2= 1 1 σ2 1 + 1 σ2 2
This looks similar to the equations for the multivariate equations. This will be more obvious if you recognize that matrix inversion, denoted by the -1 power, is like division since AA−1= I. I will rewrite the inversions as divisions - this is not a mathematically correct thing to do as the order of operations becomes ambiguous (matrix multiplication is not commutative), but it does help us see what is going on.
µ ≈Σ2µ1+ Σ1µ2 Σ1+ Σ2 Σ ≈ Σ1Σ2
(Σ1+ Σ2)
In this form we can see that these equations are just the linear algebra form of the univariate equations. Now Let’s explore multivariate Gaussians in terms of a concrete example. Suppose that we are tracking an aircraft with a couple of radar systems. I will ignore altitude as this is easier to graph in two dimensions. Radars give us the range and bearing to a target. We start out being uncertain about the position of the aircraft, so the covariance, which is just our uncertainty about the position, might look like this.
In [15]: P0 = np.array([[6, 0], [0, 6]])
stats.plot_covariance_ellipse((10, 10), P0, facecolor=’y’, alpha=0.6)
Now suppose that there is a radar to the lower right of the aircraft. Further suppose that the radar is very accurate in the bearing measurement, but not very accurate at the range. That covariance, which is just the uncertainty in the reading might look like this (plotted in blue):
In [16]: P1 = np.array([[2, 1.9], [1.9, 2]])
stats.plot_covariance_ellipse((10, 10), P0, facecolor=’y’, alpha=0.6) stats.plot_covariance_ellipse((10, 10), P1, facecolor=’b’, alpha=0.6)
6.4. MULTIPLYING MULTIDIMENSIONAL GAUSSIANS 151
To combine the information we multiply the two Gaussians together. Let’s see the result of that. I will use FilterPy’s multivariate multiply method to perform the multiplication.
In [17]: from stats import multivariate_multiply
P2 = multivariate_multiply((10,10), P0, (10,10), P1)[1]
stats.plot_covariance_ellipse((10, 10), P0, facecolor=’y’, alpha=0.2) stats.plot_covariance_ellipse((10, 10), P1, facecolor=’b’, alpha=0.6) stats.plot_covariance_ellipse((10, 10), P2, facecolor=’y’)
Here I have plotted the original estimate it a very transparent yellow, the radar reading in blue, and the estimate generate by multiplying the two Gaussians together in yellow.
It retained the same shape and position as the radar measurement, but is smaller. We’ve seen this with the one dimensional Gaussians - multiplying two Gaussians makes the variance smaller because we are incorporating more information, hence we are less uncertain. But the main point I want to make is that the covariance shape reflects the physical layout of the aircraft and the radar system.
Now lets say we get a measurement from a second radar bank, this one to the lower right, which I will plot in blue against the yellow covariance of our current belief.
In [18]: P3 = np.array([[2, -1.9], [-1.9, 2.2]])
stats.plot_covariance_ellipse((10, 10), P2, facecolor=’y’, alpha=0.6) stats.plot_covariance_ellipse((10, 10), P3, facecolor=’b’, alpha=0.6)
Again, to incorporate this new information we will multiply the Gaussians together.
In [19]: P4 = multivariate_multiply((10,10), P2, (10,10), P3)[1]
stats.plot_covariance_ellipse((10, 10), P2, facecolor=’y’, alpha=0.2) stats.plot_covariance_ellipse((10, 10), P3, facecolor=’b’, alpha=0.6) stats.plot_covariance_ellipse((10, 10), P4, facecolor=’y’)
6.4. MULTIPLYING MULTIDIMENSIONAL GAUSSIANS 153
You can see how the multivariate Gaussian’s shape reflects the geometry of the problem. The first radar system was at a 45 degree angle to the aircraft, and its error in the bearing measurement was much smaller than the error in the range. This resulted in a long and narrow covariance ellipse whose major axis was aligned with the angle to the radar system. The next radar system was also at a 45 degree angle, but to the right, so the two measurements were orthogonal to each other. This allowed us to triangulate on the aircraft, which resulted in a very accurate final estimate. We didn’t explicitly write any code to perform triangulation; it was a natural outcome of multiplying the Gaussians of each measurement together.
To make sure you understand this, what would the Gaussian look like if we only had one radar station, and we receive several measurements from it over a short period of time? Clearly the Gaussian would remain elongated in the axis of the bearing angle. Without a second radar station no information would be provided to reduce the error on that axis, so it would remain quite large. As the aircraft moves the bearing will typically change by a small amount, so over time some of the error will be reduced, but it will never be reduced as much as a second radar station would provide.
To round this out lets quickly redo this example but with the first radar system in a different position. I will position it directly to the left of the aircraft. The only change I need to make is to the Gaussian for the measurement from the radar. I used
Σ = 2 1.9 1.9 2
Why did this result in a 45 degree ellipse? Think about that before reading on. It was 45 degrees because the values in the diagonal were identical. So if x=10 then y=10, and so on. We can alter the angle by making the variance for x or y different, like so:
In [20]: P1 = np.array([[2, 1.9], [1.9, 8]])
The radar is to the left of the aircraft, so I can use a covariance of Σ = 2 1.9
1.9 2
to model the measurement. In the next graph I plot the original estimate in a very light yellow, the radar measurement in blue, and the new estimate based on multiplying the two Gaussians together in yellow.
In [21]: P1 = np.array([[2, 0], [0, .2]])
P2 = multivariate_multiply((10,10), P0, (10,10), P1)[1]
stats.plot_covariance_ellipse((10, 10), P0, facecolor=’y’, alpha=0.2) stats.plot_covariance_ellipse((10, 10), P1, facecolor=’b’, alpha=0.6) stats.plot_covariance_ellipse((10, 10), P2, facecolor=’y’)