Chapter 3: Conceptual Framework and Research Methodology 56
3.8 Performance Evaluation 86
3.8.2 Trading Metrics 87
This thesis explores several trading metrics that are widely used by real life traders and investment firms. To provide systematic analysis, we classify the trading metrics into two parts: (1) general trading metrics and (2) key trading metrics. Given the breadth of performance measures examined in this thesis, this distinction allows us to emphasise the key and relevant metrics when comparing multiple trading systems. This distinction can be supported by prior research.73 The latter, in particular, lists the salient metrics for
73 For example, with respect to the general metrics, although net profit is important, it must be referred to risk to make sense (Vanstone 2006). In a similar vein, the number of trades merely indicates how many trades are triggered by the trading system, and by itself provides little information. In comparison, for the key performance metrics, drawdown is said to be a more important gauge of performance compared to net profit (Rotella 1992). Likewise, it is widely acknowledged that Sharpe and Sortino ratios, for instance, are the most essential performance measures. In any event, since the trading systems are ultimately sorted based on their Sharpe ratios (which is established in both academic research and practice as the leading measure) following Eakins and Stansell (2003), the classification does not affect the final outcome of our trading systems ranking.
88
trading performance. Indeed, the Fidelity Investments Wealth-Lab Developer similarly indicates these key metrics as among the ‘bottom line’ of performance measures.74 Altogether, we examine a total of 20 trading metrics ranging from profitability to risk, trades, and their risk-return tradeoffs. A detailed discussion of these metrics goes outside the scope of this thesis. Interested readers should refer to Chande (1997), Grossman and Zhou (1993), Kaufman (1998), Link (2003), Pardo (2008), Rotella (1992), Ruggiero (1997), Sharpe (1966, 1994), Sortino and Satchell (2001), Tharp (1998), Vanstone and Finnie (2009) and Vanstone and Hahn (2010).
3.8.2.1 General Trading Metrics
The first category deals with 13 general metrics, which comprise the following: (1) net profit; (2) net profit %; (3) annualised gain %; (4) number of trades; (5) average profit; (6) average profit %. (7) exposure; (8) winners: winning trades; (9) winners: win rate; (10) winners: average profit %; (11) losers: losing trades; (12) losers: loss rate and (13) losers: average loss %. These metrics can be briefly described as follows.
Net profit refers to the total dollar profit generated after deducting trading costs. Net profit % indicates total net profit in terms of its percentage of initial budget (starting capital). Annualised gain % refers to the compounded annual growth rate (CAGR), which is also known as the annual percentage return (APR). It shows the smoothed average rate of return on the basis of compounding the starting capital annually. Number of trades simply shows the total number of round-trip trades and open positions. Average profit (profit %) is the average dollar (percentage) return per trade after deducting trading costs. Exposure refers to the total area of portfolio equity exposed to the market during the period.
Winners: winning trades (win rate) refers to the number (ratio) of winning trades produced by the trading systems, while winners: average profit % indicates the average percentage profit of the winners. Similarly, losers: losing trades (loss rate) refers to the number (ratio) of losing trades generated by the trading systems, while losers: average
89
loss % indicates the average percentage loss of the losing trades. It is generally desirable for a trading system to generate more winners than losers, or similarly, better directional forecast. Nonetheless, this also depends on the system average profits (losses) when the trade is a winner (loser).
3.8.2.2 Key Trading Metrics
More important for our study, the second group of metrics details the principal performance measures of a trading system. This thesis explores seven key metrics: (1) profit factor; (2) payoff ratio; (3) maximum drawdown %, (4) recovery factor; (5) ulcer index; (6) Sharpe ratio and (7) Sortino ratio. Briefly stated, these performance measures can be given as follows.
Profit factor is a measure of profitability and is calculated by dividing gross profit with gross loss. Chande (1997) advises the factor to exceed one, although Vanstone and Finnie (2009) assign a stricter threshold of two. Payoff ratio indicates the efficacy of a trading system in acquiring returns relative to losses, measured by dividing the average percentage profit over the average percentage loss. It is desirable for the ratio to exceed two (Vanstone & Finnie 2009). Maximum drawdown % is the percentage decline of the largest peak to valley in the equity curve. According to Pardo (2008), drawdown is one of the best measures to evaluate the overall risk of a system. Recovery factor is computed by dividing the absolute value of net profit by the maximum drawdown. It shows how effective the trading system is in overcoming the effects of drawdown. Chande (1997) proposes a minimum factor of two to be exceeded for the trading system to be desirable. Ulcer index measures volatility in terms of drawdown by square rooting the quotient of sum squared drawdowns divided by the period.
Ultimately, the superiority of a trading system is gauged by its risk-return tradeoffs. The most popular approach, the Sharpe ratio (Sharpe 1966, 1994), measures the return to its variability (volatility). In this study, the ratio is computed by dividing the annualised average return with its annualised standard deviation. In general, the value above one indicates the system has good risk-return tradeoffs. Varga (2006) contends that a ratio above 0.7 is also acceptable. Accordingly, when contesting different strategies, the best trading system is the one with the highest Sharpe ratio.
90
One of the main limitations with the Sharpe ratio, however, is that it penalises both upside and downside volatility as equally risky. In reality, investors are more concerned with the downside risk (the risk of loss) whereas the ‘risk’ of upside (profit) is actually preferred. This limitation is addressed in the Sortino ratio (see Sortino & Satchell 2001). The ratio only considers ‘bad’ volatility (i.e., downside deviation instead of standard deviation) in the denominator, and therefore, provides robustness to the results on return to variability. Like the Sharpe criterion, the higher ratio indicates the better system.
Despite the limitations of the Sharpe measure, it remains the most popular approach in measuring investment risk-return tradeoffs (Feibel 2003). This is also supported by Lo (2002, p.36), who notes that it is ‘one of the most commonly cited statistics in financial analysis’. Accordingly, its use as the primary performance measure in this thesis allows for comparability with other studies. As discussed in Section 3.2.2, the Sharpe ratio is also used as the primary metric in order to rank the trading systems (i.e., for Propositions 2 and 4). This approach is consistent with Eakins and Stansell (2003). Further, to provide a numerical assessment of the economic significance of the fusion trading systems, the percentage increase in the Sharpe (and Sortino) ratio by combining the trading strategies will also be noted.75 In the table that follows, we provide the mathematical formula for the key trading metrics.
75 As measured by the differences in the Sharpe (Sortino) ratios between the fusion trading system and the base strategy, divided by the Sharpe (Sortino) ratio produced by the base strategy. The base strategy refers to the constituent trading systems, and in the case of FUSION-NNTS, it also refers to the CFUS-NNTS.
91 Table 3.5
Key Trading Metrics
Acronym Metric Operationalisation
PF Profit Factor GP PF GL PR Payoff Ratio AW PR AL MD% Maximum Drawdown %
% V P MD P RF Recovery Factor NP RF MDD UI Ulcer Index 2 1 N i i D UI N
SR Sharpe Ratio p f p p p R R R SR ST Sortino Ratio p f p pd pd R R R ST The table shows the mathematical formula for the key metrics. PF is a measure of profitability, where GP (GL) is the gross profit (gross loss). PR indicates the efficacy of a trading system in acquiring returns relative to losses, where AW (AL) indicates the mean percentage profit (loss) for winning (losing) trades. MD% refers to the largest market peak (P) to valley (V) percentage decline. RF shows how effective the trading system is in overcoming the effects of drawdown, where NP (MDD) denotes net profit (maximum dollar drawdown). UI measures volatility in terms of drawdown (D) during the period (N). SR conveys the risk-adjusted return for the trading systems, where Rp
denotes the annualised mean return, Rf refers to the risk-free rate and σp refers to the
annualised standard deviation. The calculation for ST is similar to the SR, but utilises downside deviation of portfolio return (σpd). This study assumes Rf = 0; therefore, SR
(ST) equals Rp divided by σp (σpd).
It is important to highlight that, typically, the calculation of the Sharpe (and Sortino) ratio uses excess returns (Sharpe 1994). Nonetheless, since our trading systems do not involve removing cash from the portfolio, and hence there are no other costs incurred or streams of income acquired, the calculation of the Sharpe (and Sortino) ratio assumes a zero risk-free rate of return (see Vanstone 2006). Kaufman (1998) also argues that for practical purposes, the risk-free rate is often ignored from the calculation of the ratio. Briefly stated, since there is no risk-free rate used, the excess return effectively equals the mean profit (see Table 3.5). Note that the use of average return (rather than excess return) is also applicable for a zero-investment strategy (Capaul, Rowley & Sharpe 1993). In any event, the approach adopted in this thesis is consistent with other studies, for example Thawornwong, Enke and Dagli (2003) and Vanstone and Hahn (2010).
92