Performance Metrics

A number of performance metrics have been developed to evaluate and compare the front sampling ability of both preplanned and adaptive AUV front tracking missions. e variables measured during the front tracking missions to calculate the performance metrics are given below, followed by the calculations for the metrics themselves.

Variables

tmission= Total mission time

vavg= Average AUV speed

vnav= Actual AUV speed at a given time (sample)

Nspd= Total number of AUV speed sample points

vavg = 1 Nspd N∑spd i=1 (vnav)i (5.16)

θ= Front intersect angle

Dtotal= Total distance traveled

Dtotal= N∑pos

i=1

√

(xi− xi−1)2+ (yi− yi−1)2 (5.17)

where Npos = Total number of AUV position locations

Df ront= Total possible distance AUV could have tracked along front, given tmission. Estimated by the best-

case calculation (AUV perfectly tracks the front, crossing the front at an angle of θ).

Don_f ront= Distance AUV tracked along the front line (≤ Df ront)

Ncross= Number of front crossing points, total, while tracking front

Df rom_f ront= Perpendicular distance (closest point of approach) from AUV position, (x, y), to front esti-

mate line

Df rom_f ront= −mx + y − b√

(m2_{+ 1)} (5.19)

where m and b are the slope and intercept, respectively, of the front line estimate in the local x-y grid

∂T /∂r= e temperature (T ) gradient in the across-front direction, relative to the front estimate line

∂T ∂r = ∂T ∂Df rom_f ront (5.20) Metrics

ρ= Crossing Density; i.e., how many front crossings were made by the AUV per unit length of the front line that was tracked (higher values equal better performance)

ρ = Ncross Don_f ront

(5.21)

Dcross = Distance between Crossings; i.e., the average distance the AUV traveled between front crossings

(higher values equal worse performance)

Dcross=

1

ρ (5.22)

ϵ= Front Sampling Efficiency; i.e., the percentage of Df ront that was tracked and sampled by the AUV

(higher values equal better performance)

ϵ = Don_f ront

Df ront × 100%

(5.23)

ER = Excess Ratio; i.e., how much of the AUV’s travel distance was in excess of the distance along the front

that the AUV captured the front (higher values equal worse performance)

ER = Dtotal Don_f ront

FEE = Front Estimate Error, which compares the|∂T /∂r|maxlocation to the local estimated front location,

as captured by the AUV (higher values equal worse performance)

FEE = Df rom_f ront@|∂T /∂r|maxon a zigzag leg (5.25)

TC = Tracking Confidence, which is an evaluation of the confidence level that the actual front was fol-

lowed/sampled by the AUV, expressed as a percentage (higher values equal better performance)

TC = 2 ( N+ Ntot ) × 100% (5.26)

where N+ _{is the number of above-average ∂T /∂r bins, N}

tot is the total number of ∂T /∂r bins, and the

scaling factor of 2 accounts for the fact that most ∂T /∂r sample bins have below-average values, and a minority of samples have above-average values due to sharp peaks in ∂T /∂r in the across-front direction, so at best it would be expected to see N+_/N

tot = 0.5.

In order to determine the distance each AUV traveled along the front, Don_f ront, each virtual experi-

ment run had to be replayed twice in post-processing using the Google Earth interface for Ocean Vehicles (GEOV) with auto-updating overlays of the temperature field showing highlighted isothermal lines at the AUV-calculated frontal temperature. For each replay, Google Earth’s Path tool was used to track the path of one AUV along the frontal isotherm, discontinuing and resuming the path segments when the AUV strayed from and reacquired the front (respectively). e sum of the path segment lengths from a single replay was recorded as Don_f rontfor the AUV whose path was mapped. All other performance metrics variables that did

not involve Don_f rontfor calculation were actively recorded and updated as the virtual experiments were run-

ning. e MOOS process pFrontTrackMetrics was written to keep track of the performance metrics variables’ values as the virtual experiments ran.

Once all of the performance metrics variables’ values were extracted from the data logs and Don_f ront

values were determined, the actual performance metrics were calculated. When performing the data extrac- tion, only data collected on both AUVs while the preplanned straight zigzag was being executed were used. In the virtual experiments where both AUVs were adaptively tracking the front, a timer was set to stop both AUVs’ missions simultaneously to maintain equal mission time. is allows us to keep the total mission time for both AUVs approximately equal to improve validity of comparison of the two front mapping techniques (preplanned vs. adaptive) and evaluation of adaptive front tracking configuration variables.

end of this chapter, higher Crossing Density (ρ) and Sampling Efficiency (ϵ) values indicate better performance tracking along the front, while higher Distance between Crossings (Dcross) and Excess Ratio (ER) values

indicate worse performance. It is often desirable to maintain some across-front motion of the AUVs as well, thus extremes of huge ρ and tiny Dcross and ER values are not necessarily optimal.

In summary, sailing along the front is good, but in many cases, crossing the front frequently is also good. e desirable performance metrics criteria for these two cases are summarized here:

• Sailing very closely along the front should maximize Crossing Density and Sampling Efficiency and minimize Distance between Crossings and Excess Ratio as much as possible.

• Maintaining crossing of the front over some length scale—i.e., the Rossby radius of deformation (e.g., 3–10 km)—while sailing along the front should maximize Sampling Efficiency and result in relatively high Crossing Density and low Distance between Crossings and Excess Ratio (but not the extreme values desirable for directly along-front motion).