Decoding cyclical data

For some tasks, such as the wind direction prediction problem which inspired the box and bullseye datasets, one or more of the outputs of the network may be real-valued cyclic data. In this case, it is necessary to code these output values in the same way that cyclic data must be encoded for input to a network. Any of the encoding schemes discussed previously can also be used for coding output values. However when using cyclic data as outputs, it is also necessary to perform a decoding process to convert the encoded output from the network into the cyclic value which is required. For the linear encoding this decoding is straightforward, requiring only the inverse of the original linear scaling used in encoding. However this simplicity is offset by the problems which the encoding discontinuity could be expected to cause during training. For the other encoding schemes the process of decoding is more complicated as it is necessary to combine two

output units together to calculate the cyclic value being output by the system.

At least three different decoding methods can be used for the trigonometric encoding. All involve the calculation of the angle represented by the output unit pair. This angle can then be scaled to convert back to the desired cyclic data.

The most obvious of these is to divide the value of the sine output unit by the value of the cosine output unit and to calculate the arctangent of the resulting value. By examining the signs of the sine and cosine units the quadrant in which the angle lies can be found and hence the output angle can be calculated. This will be referred to as the arc decoding.

A second possibility is to calculate the arcsine of the sine output unit and the arccos of the cosine output unit. Both of these operations will yield two possible angles. If the network's output was exactly correct two of these angles would be identical and hence would be the desired angle. In practice it is necessary to compare the four combinations of candidate angles, and select the pairing of arcsine angle and cosine angle with the least variation. These two angles can then be averaged to yield the final angle. This will be called the paired decoding.

A problem can arise with paired decoding if the output units do not use a squashing function, which is commonly the case when training a network to produce real-valued outputs. In this situation the absolute values of the output units may exceed 1, which will result in an error if they are passed to the arcsine and arccos functions. In this research this problem was addressed by truncating the output units values to a maximum magnitude of 1.

A third possible decoding scheme is to generate a table of templates, consisting of the output encoding for a range of possible output values. To decode a pair of output values they are compared to each template and a measure of the difference is calculated (in this research the mean squared error was used, but any reasonable distance measure could be used). The template with minimum distance from the actual output values is selected and its corresponding angle is used. The resolution of the values provided by this scheme is determined by the number of templates generated.

All of these trig decoding methods can also be used with a cartesian encoding. However it is necessary to eliminate the effects of the magnitude from the output units prior to decoding them. First both output values need to be multiplied by the maximum magnitude value to eliminate the scaling inherent in this encoding. The output magnitude can then be calculated as the root of the sum of the squares of these two values. The output values are then divided by this magnitude before applying the angular decoding scheme.

The trig decoding methods can also be adapted to work with the sawtooth encoding. The template method requires no modification, except that the sawtooth encoding is used to generate the templates. Similarly for the paired method, the basic technique remains the same except that inverse functions of the sawtooth encodings are used in place of the arcsin and arccos functions (such functions are easily generated due to the linear nature of the original sawtooth functions). Again the problem of output values with a magnitude greater than 1 can be addressed by truncation of the output values to a maximum magnitude of 1.

The arc decoding requires the sawtooth equivalent of the arctangent function, which is implemented by the following equation. 24

Let s represent the sawtooth equivalent of the sine, and c represent the sawtooth equivalent of the cosine. Let y=s/c (the sawtooth equivalent of the tangent). The arcsawtooth (AS) in degrees is given by:

if s >=0 then if c >=0 then AS=90(y/(1+y)) else AS=90((y-2)/(y-1)) else if c>=0 then AS=90((3y-4)/(y-1)) else AS=90((2+3y)/(y+1))