SSVEP Estimation Protocol

Let S (τ ) be the signal from a randomly-selected instance τ of an SSVEP experiment. We refer to τ as a “trial.” The signal may be the actual raw EEG time series, a processed EEG spectrum, or even an arbitrary index into a table of signal exemplars. What is important is that S (τ ) must be an observable random variable of the unobservable trial instance τ which ranges over the underlying probability space (T, Prob) of the experiment. That is, each trial τ ∈ T, whose occurrences governed by the probability measure [128] Prob [ · ], encapsulates all the uncertainties of the SSVEP experiment:

1

• Variations between subjects chosen at random.

• Variations within the same subject on different days, different times of the same day, or different moods.

• Impedence variations caused by non-uniform application of electrode gel. • Artifacts.

• Stray electromagnetic fields.

• The particular stimulus frequencies used for this trial.

• The stimulus frequency to which the subject attends as well as the time interval of attention. • The dice-rolls or coin-tosses we may use for randomized decision rules.

• The vast number of other unnamed lurking variables in any SSVEP/EEG experiment.

Let Fssvep(τ ) be the (random) set of stimulus frequencies we are trying to estimate from S (τ ) during trial τ . Note that Fssvep(τ ) should include all possible fundamental stimulus frequencies which were flashing when S (τ ) was observed. It may also include some harmonics and subharmonics [9] of these fundamental SSVEP frequencies.

Let Ftest?be an initial set of test frequencies. When given any single-trial signal S, our estimation protocol is to succesively test every f ∈ Ftest? for its presence or absence in Fssvep(τ ). Thus we are performing frequency estimation by m-ary testing [127], where m is the size of Ftest?. A given testing procedure may or may not have access to Fssvep(τ ); that is, it may or may not be blind.

An essential aspect of SSVEP estimation, especially for BCI applications, is that every trial partitions Ftest? into three subsets:

2. Those f ∈ Ftest?\ Fssvep(τ ) which thus were not present during this trial but might have been. These could be, for example, frequencies of non-selected fields during a BCI test.

3. All other f ∈ Ftest?.

In typical blind SSVEP frequency estimation, case (3) vastly outnumbers (1) and (2). For example, the data used in Chap. 7 contained 614 initial test frequencies while there were only 19 possible distinct stimulus fre- quencies and harmonics. In such a situation, simple m-ary control procedures such as Bonferroni correction (see [129] for a survey) are out of the question because they yield impractically small corrected significance levels. Even less conservative procedures such as those used to control the false discovery rate (FDR) [130] yield overall significance levels which often produce no discoveries at all. This because the signal-to-noise ratio of SSVEP signals is usually too low for estimation statistics to yield sufficiently small P-values even at fundamental stimulus frequencies.

Moreover, non-stimulus frequencies that we particularly want to reject may slip into the signal, the most important example being the frequencies of non-selected but still visible fields during BCI experiments. The assessment of any algorithm must test its ability to positively exclude these potential contaminants even when blind.

It is therefore essential to develop estimation and validation procedures which can distinguish the combined situation (1) ∪ (2) from (3).

A solution to all the problems of the preceeding paragraphs is to assume a second randomly-varying set Fnull(τ ) of null frequencies such that Fnull(τ ) ∩ Fssvep(τ ) = ∅. Once the signal S (τ ) is observed, the hypotheses we decide at each test frequency f ∈ Ftest?are

H0(f ): f ∈ Fnull(τ )

Halt(f ): f ∈ Fssvep(τ )

is, we must allow undetermined as a possibility.

We require a ground truth frequency estimation algorithm for SSVEP experiments as a baseline. This may be:

• A look-up table from a laboratory notebook. • An electronic spectral analyzer or oscilloscope. • Gold-standard spectral estimation software.

• A synthetic ground truth algorithm which simulates real SSVEP results. This can be used for validat- ing a new algorithm. It is the ground truth used to produce the tables and figures of Chap. 7.

The ground truth may or may not be blind. We want to measure the performance of a comparison algorithm (which is usually assumed to be blind) against this ground truth.

Both the ground truth and comparison algorithms may involve randomized decision rules [127]. However we make the assumption that the randomization procedures of the two algorithms are independent given S. That is, once the signal outcome S = S (τ ) is known, whatever τ -dependent dice-rolls or coin-tosses each uses to make its decisions about H0(f ) and Halt(f ) are statistically independent of one another. We also assume that any randomized rules of each individual algorithm are independent at distinct test frequencies, given S. (This latter assumption actually excludes some interesting potential algorithms but is required for the calculation of the parametrized contingency tables discussed below. However it is sufficiently general to hold for most practical algorithms including those from Chap. 7.)

6.2 Example: Synthetic SSVEP Algorithms      Pssvep[f | S] = pssvep(f ) · s (f ) Pnull[f | S] = pnull(f ) · s (f ) , where, for all f ,

                     pssvep(f ) > 0 pnull(f ) > 0 pssvep(f ) + pnull(f ) 6 1 s (f )def= S (f ) / max g S (g) .

6.3 The Contingency Table Statistic

Since we do not assume that the algorithm always determines the truth values of the two hypotheses, there are 9 possible outcomes for a test at f ∈ Ftest?. The full 3 × 3 contingency table at f is defined by Table 6.1:

Table 6.1

The 3 × 3 contingency table at each test frequency f showing the primary indicator terms and the null hypothesis bias b0. ( c 2016 IEEE)

Ground Truth

H?(undetermined)

Halt True H0True 1 − b0 b0

Comparison Algorithm Accept Halt (Positive) TP?(f ) FP?(f ) tP (f ) fP (f ) P?(f ) Accept H0 (Negative) FN?(f ) TN?(f ) fN (f ) tN (f ) N?(f ) Accept ? (Neither) Fn (f ) Fp (f ) malt?(f ) m0?(f ) m (f )