\section{The Benchmark Program}\label{sec:BenchMark}
The benchmark program may be used to compare the execution
speed and accuracy of the proposed hybrid method with an
alternative implementation of the \nrm routines.

By default the elements of the data vectors used are random
in the range $(0,1)$.  The program then times the proposed
and comparison methods and calculates the relative errors in
the computed results. For each value of $n$, a number of data
vectors are generated and both the average execution times and
the maximum relative errors are recorded.  Details of the other
types of data vectors that may be used along with information
on how to change the extent of the benchmarks may be found in
Section~\ref{sec:optfile}. A number of these input data vector
options were used for testing the \FaithfulNorm implementation
described in~\cite{Graillat:2015:ECF}

The values of $n$ used in the current release are $10^p$,
for $p=1, 2, \ldots, 5$ and the number of vectors generated
is $50*10^m, m= 5,4, \ldots,1$ respectively. These values
may easily be altered by changing the statements as described
in the declaration section of program, \fvarn{sdBenchmark},
in the file \filename{sdBenchmark.f90}.

The output generated by the program provides a synopsis of
the execution times and the accuracy as well as two sets of
ratios; one of the proposed hybrid method execution times
over the comparison method execution times and the other,
the reciprocal of the first. Values of the first set,
labelled \textit{Remark/Other}, that are greater than one
indicate the factor by which the comparison method is faster
than the proposed hybrid method. For the column labelled
\textit{Other/Remark}, values greater than one indicate
the factor by which the hybrid method is faster than the
comparison method.

\subsection{Optional Benchmark Data File}\label{sec:optfile}
If a file with the name \filename{BenchData.dat} exists
in the directory where a benchmark executable is run then
this file is read and the data used to alter a number of
the benchmark parameters.

The format of this data file is as follows:

\begin{description}
\item[Line 1:] \textit{precision} (integer) 

Choose whether to run either the single or double precision
benchmark or both:

  \begin{description}
  \item[0:] run both single and double precision benchmarks,
            \textit{(Default)}

  \item[1:] run single precision only,

  \item[2:] run double precision only.

  \end{description}

\item[Line 2:] \textit{types} (integer) 
Choose whether to run either the real or complex routines
or both:

  \begin{description}
  \item[0:] run both real and complex routines,
            \textit{(Default)}

  \item[1:] run real routines only,

  \item[2:] run complex routines only.

  \end{description}

\item[Line 3:] \textit{timeAccuracy} (integer) 

Choose whether to perform either timing or accuracy measurements
or both:

  \begin{description}
  \item[0:] both time and accuracy,  \textit{(Default)}

  \item[1:] time only,

  \item[2:] accuracy only.

  \end{description}

\item[Line 4:] \textit{signElts} (integer) 

Sets the sign pattern to be applied to the elements of the data vector. All data vector options determined by \textit{dataDesc} (see below) generate only positive elements. The selected sign pattern is then applied.

The currently available options are:

  \begin{description}
  \item[0:] signs of elements are left unchanged,  \textit{(Default)}

  \item[1:] the sign of each element is assigned randonly,

  \item[2:] all elements are set positive,

  \item[3:] all elements are set negative.

  \end{description}

At the moment options 0 and 2 have exactly the same effect; they have both been made available for possible future extensions to the package.

\item[Line 5:] \textit{dataDesc} (integer) 

Choose the type of input vector required. If the value chosen is
one of the user defined ranges (\textit{userDefRandom,
userDefAscending, userDefDescending}) then it is necessary
to provide further information. See details after the list
of options:

  \begin{description}
  \item[1:] \textit{zeroOneRandom} -- random values in the range $(0,1)$  \textit{(Default)}.

  \item[2:] \textit{zeroOneAscending} -- random values in the range $(0,1)$ sorted into ascending order. This is the worst case for the current LAPACK routines in that it forces a change of scaling factor for every element in the vector.

  \item[3:] \textit{zeroOneDescending} -- random values in the range $(0,1)$ sorted into descending order. 

  \item[4:] \textit{userDefRandom} -- random values in a user defined range. Details of how to define the intervals to be used are given at the end of this section.

  \item[5:] \textit{userDefAscending} -- as 4. above with the data sorted into ascending order.

  \item[6:] \textit{userDefDescending} -- as 4. above with the data sorted into descending order.

  \item[7:] \textit{simpleAscending} -- a data vector containing a simple, monotone increasing, sequence of elements, s.t. $x_i = 1 + (i-1)/n, i = 1, \ldots, n$. Range: $ [1, 2)$.

  \item[8:] \textit{simpleDescending} -- a data vector containing a simple, monotone decreasing, sequence of elements, s.t. $x_i = 1 + (n-i)/n, i = 1, \ldots, n$. Range: $ [1, 2)$.

  \item[9:] \textit{spuriousUflow} -- a vector containing data
  that will cause harmless underflows (i.e., setting to zero
  will not affect the result). The vector length requested,
  $n$, must be greater than 2. Then $x_1 = 2^{e_{max}-1}$,
  $x_2 = 2^{e_{max}-t}$ where $t=24$ (single precision) and
  $t=53$ (double precision). $\left \{ x_i \right \}_{i=1}^n$
  are chosen to be in the exponent range $[ eu, -1 ]$.

  \item[10:] \textit{noOflow} -- given the value of $n$,
  random elements are chosen so as to guarantee that the
  value of the resultant $l_2$ norm is representable as a
  valid floating point number in the chosen precision. The
  range used is dependent on $n$ and is almost the whole
  range available except that the maximum exponent 
  used is reduced. Note: this data could still cause an
  overflow to occur if no scaling is applied, for example,
  using a simple square and add loop. Exponent range: $[ eu,
  e_{max}-t ]$ where $t = \lceil \log_2(n) \rceil + 2$.

  \item[11:] \textit{aroundOne} -- all the randomly generated
  values are around one in magnitude. Exponent range: $[-5, 5]$.

  \item[12:] \textit{reallySmall} -- all the randomly generated
  values are very small in magnitude although the interval
  chosen does span the low and middle ranges of both Blue and
  Kahan's algorithms. Exponent range: $[eu, est]$.

  \item[13:] \textit{normGradUflow} -- all randomly generated
  values are denormalized numbers. Exponent range: $[eu,
  e_{min}-t ]$ where $t = \lceil \log_2(n) \rceil + 2$.

  \item[14:] \textit{fullExpRange} -- the randomly generated
  values are chosen from the whole range of floating point
  numbers available. Note that it is technically possible that
  the resultant Euclidean norm value is not representable
  in the floating point format being used. To guarantee the
  result will be representable, see the \textit{noOflow} option
  above. Exponent range: $[ eu, e_{max} ]$ 

  \item[15:] \textit{lowHybridScale} -- the randomly generated
  values are chosen to cause the underflow flag to
  signal when using the simple loop in the hybrid method. This
  will result in a call to Kahan's routine, a second pass
  through the data and scaling. The first value in the data
  vector has been chosen to cause an underflow and the rest
  of the values all lie in the low range defined for Kahan's
  method.

  \item[16:] \textit{highHybridScale} -- the randomly generated
  values are chosen to cause the overflow flag to
  signal when using the simple loop in the hybrid method. This
  will result in a call to Kahan's routine, a second pass
  through the data and scaling. The first value in the data
  vector has been chosen to cause an overflow and the rest
  of the values all lie in the high range defined for Kahan's
  method.

  \item[17:] \textit{bothHybridScale} -- the randomly generated
  values are chosen to cause both the overflow and
  underflow flags to signal when using the simple loop in the
  hybrid method. This will result in a call to Kahan's routine,
  a second pass through the data and scaling. The first two
  value in the data vector has been chosen to cause an underflow
  and overflow respectively and the rest of the values all
  lie in the low and high ranges defined for Kahan's method.

  \item[18:] \textit{KahanComp} -- this is as
  \textit{highHybridScale} above but the element causing the
  overflow is the last element rather than the first. This
  causes a worst case input vector for the original version
  of Kahan's method.

\end{description}

If \textit{dataDesc} options \textit{userDefRandom},
\textit{userDefAscending}, or \textit{userDefDescending} are
chosen, then more input data is required to define the data
ranges to be used when generating the element values. A data
range is specified by a pair of integers $(minExp, maxExp)$
which defines the range of element values to be $(2^{minExp},
(2-\epsilon)2^{maxExp}]$, i.e., any valid floating point number,
except $2^{minExp}$,
where the exponent is in the range $[minExp, maxExp]$. To
make the data generation as flexible as possible, multiple
intervals are allowed; no checks are performed on the given
interval values so the defined intervals need not be disjoint.

The form of the additional data is:

\item[Line 6:] \textit{numIntervals} (integer)

Provide the number of range definitions (i.e., pairs of exponent
values) required.

\item[Line 7--6+\textit{numIntervals}:] (pairs of integers)

Defines the minimum and maximum exponent values that are
used to define each interval. The order of these values is
actually immaterial.

If more than one interval is defined the number of elements
generated in each interval is $n_{int} = \lfloor n/numIntervals
\rfloor$ except the last interval which contains $n -
(n-1)n_{int}$. The elements are generated a block at a time
contiguously in the vector.

\end{description}

If the file, \filename{BenchData.dat}, contained the following
lines then just the routine \fvarn{SNRM2} (single precision,
real) would be used for accuracy testing only. The input
vector would contain elements whose magnitudes were all in
the range $(1024, 16384)=(2^{10}, 2^{14})$ and whose signs
were randomly chosen.

\begin{verbatim}
1  : single precision only (i.e., restrict to snrm2 and scnrm2)
1  : real routine only (i.e., restrict further to snrm2)
2  : only perform the accuracy tests
1  : set the sign of each element randomly
4  : select user defined data; random values in a specified range
1  : number of ranges to be defined
10 13  : magnitude of elements in the range (2**10, 2**14)
\end{verbatim}

If a data file is not present then the benchmark is run
with the defaults shown in the data description above; i.e.,
all four routines
(single and double precision, real and complex) are run for both
timing and accuracy on data vectors where all the elements are randomly generated in the range $(0,1)$. This is equivalent to a \filename{BenchData.dat} file containing

\begin{verbatim}
0  : run single and double precision routines
0  : run real and complex routines
0  : perform both timing and accuracy tests
0  : the sign of each element is left unchanged (equivalent to 2 in this case)
1  : random data in the range (0,1)
\end{verbatim}

The subdirectory \filename{Datafiles} of \filename{Benchmarks}
contains a number of example data files along with a
\filename{ReadMe.txt} file providing details of their settings. To
use any of these files as input to the benchmark executables,
it must first be renamed \filename{BenchData.dat} in the same
directory as the benchmark is being executed.

