Calculates the percentage of cases where forecast error is greater than the benchmark model.
Syntax
PB(X, F, M, Mode)
X is the eventual outcome time series sample data (a onedimensional array of cells e.g. row or column).
F is the forecast time series data (a onedimensional array of cells e.g. row or column).
M is the seasonal period (for nonseasonal time series, set M=1 (default) or leave it blank).
Mode is a switch to select the basis of the calculated PB (0=regular error (default), 1=MAE, 2=MSE)
Mode  Description 

0  Relative Absolute Error (default) 
1  Mean Absolute Error (MAE) 
2  Mean Squared Error (MSE) 
Remarks
 The time series is homogeneous or equally spaced.
 The time series X and F must be of identical size
 The time series X or F may include observations with missing values (e.g. #N/A or blank).
 Observations with missing values in Y or F are excluded from the MASE calculation.
 The regular Percentage Better (PB) is calculated by following formula:
$$ \mathrm{PB}=\left\{\begin{array}{l} {\frac{\sum_{t=2}^N I\{\left  e_t \right  \lt \left  e_t^* \right  \}}{N1}} \\ {\frac{\sum_{t=M+1}^N I\{\left  e_t \right  \lt \left  e_t^* \right  \}}{NM}} \end{array}\right. \begin{array}{l} \{y_i\}\,\mathrm{is\,nonseasonal} \\ \{y_i\}\,\mathrm{is\,seasonal} \end{array} $$
Where:
 $\{e_t^*\}$ is the forecast error of the naïve benchmark model calculated insample.
 $\{e_i\}$ is the forecast error of the actual forecast.
 $I\{.\}$ is an operator that yields the values of zero or one, in accordance with the expression:
$$ I(e_t)=\left\{\begin{array}{l} 1\\ 0 \end{array}\right. \begin{array}{l} \left  e_t \right  \gt \left  e_t^* \right  \\ \left  e_t \right  \leq \left  e_t^* \right  \end{array} $$
 In short, PB demonstrates the average number of times that forecasting method overcomes the naïve forecasting method
 The Percentage Better measure counts the number of cases when the forecasting model is superior to the naïve benchmark, but it does not evaluate the degree in which they are different
 PB does not indicate the amount of possible improvement. Thus, it is possible to have one method that performs slightly better than the benchmark method for 99 series but much worse on 1 series, giving that method a PB score of 99 even though the benchmark method is preferable.
 The close variant of PB is based on measures other than absolute forecast error. NumXL PB function supports two measures: MAE and MSE. The Percentage Better of MAE is calculated by the following formula:
$$ \mathrm{PB(MAE)}=100\%\, \times\,\mathrm{mean(}I\{\mathrm{MAE}\lt \mathrm{MAE^*}\}\mathrm{)} $$
$$ \mathrm{PB(MAE)}=100\%\, \times\,\mathrm{mean(}I\{\mathrm{MSE}\lt \mathrm{MSE^*}\}\mathrm{)} $$  The PB function is available starting with version 1.65 HAMMOCK.
Examples
Example 1:


Formula  Description (Result)  

=PB($B$3:$B$21,$C$3:$C$21,1,0)  MASE (88.89%)  
=PB($B$3:$B$21,$C$3:$C$21,1,1)  MASE (94.44%)  
=PB($B$3:$B$21,$C$3:$C$21,1,2)  MASE (94.44%) 
Files Examples
References
 R.J. Hyndman, A.B. Koehler, "Another look at measures of forecast accuracy", International Journal of Forecasting, 22 (2006), pp. 679688
 James Douglas Hamilton; Time Series Analysis; Princeton University Press; 1st edition(Jan 11, 1994), ISBN: 691042896
 Tsay, Ruey S.; Analysis of Financial Time Series; John Wiley & SONS; 2nd edition(Aug 30, 2005), ISBN: 0471690740
 D. S.G. Pollock; Handbook of Time Series Analysis, Signal Processing, and Dynamics; Academic Press; Har/Cdr edition(Nov 17, 1999), ISBN: 125609906
0 Comments