THE STATISTICAL THEORY BEHIND AND A WORKING EXAMPLE OF
THE ESTIMATION OF ANALYTICAL PRECISIONS FROM PAIRED RCPA QAP RESULTS

Dr Tom Hartley, Quality Manager & Senior Biochemist, Pathology Department, Royal Hobart Hospital.

January 2003

The RCPA Quality Assurance Programme for General Serum Chemistries and Therapeutic Drugs is designed in such a way that a pooled 'high value' serum is mixed with a pooled 'low value' serum to give a linearly related set of eight concentrations. These are then distributed to participants in random order and in such a way that participants receive each of the eight concentrations twice within the cycle. In other words each participating laboratory submits eight pairs of data.

The designers of the Programme have elected to estimate a laboratories analytical precision from the standard error of the estimate derived from their linear regression analysis of their target values, on the x axis, versus the participating laboratories on the y axis.

In the opinion of this author there is a more correct approach to estimating a laboratories precision by examining the degree of agreement between their paired data. The statistical theory behind this is as follows :

Members of a pair are signified by : xi and xi'

The number of pairs = N

Estimates of real errors on a pair are :

xi - (xi + xi')/2 = (xi - xi')/2

xi' - (xi + xi')/2 = (xi' - xi)/2

Sum of Squares of the estimates of the errors, SSQ :

SSQ = 1Nå{ ((xi - xi')/2)2 + ((xi' - xi)/2)2 )}

= 2 * 1Nå{ (xi - xi')2/4}

= 1Nå{ (xi - xi')2/2}

Degrees of Freedom = DF = Number of individual measurements

= 2 * N = 2N

Standard Deviation of an Error Distribution = Ö (SSQ / DF)

= Ö ( (1Nå{ (xi - xi')2/2} ) / 2N )

= Ö ( (1Nå{ (xi - xi')2} ) / 4N )

In the context of the RCPA QAP General Serum Chemistry Programme the SD (Precision) calculation reduces to :

Ö ( (18å{ (xi - xi')2} ) / 32 )

TABLE 1
WORKED EXAMPLE

Pair xi xi' (xi - xi')/2 (xi' - xi)/2 Deviations 2 (xi - xi') (xi - xi')2
1 107 109 -1 +1 1 & 1 2 4
2 245 244 +0.5 -0.5 0.25 & 0.25 1 1
3 390 414 -12 +12 144 & 144 24 576
4 513 505 +4 -4 16 & 16 8 64
5 646 618 +14 -14 196 & 196 28 784
6 776 780 -2 +2 4 & 4 4 16
7 898 965 -33.5 +33.5 1122.25 & 1122.25 67 4489
8 1083 993 +45 -45 2025 & 2025 90 8100

SUMS 7017
14034

DIVISOR 16
32

Standard Deviation 20.94
20.94

The data in Table 1 were taken from a data set submitted to the RCPA QAP, who estimated the participating laboratory's precision for this assay (Valproate) to be 27.2. Clearly this reassessment had come in with a better precison value than the QAP. Extending this approach to all 42 chemistries enrolled by that laboratory we have established that their precision were always numerically smaller than the figure quoted by the QAP and generally more in line with their internal assessments of analytical precision from their internal daily QC data. A complete tabulation of the QAP and the laboratory's re-assessment of precisions for the RCPA QAP General Serum Chemistry Programme's Cycles 56 and 59 are shown in Table 2. Their current estimates of their analytical precison from their internal QC data are also shown. All assays were performed on a Vitros 950 analyzer apart from Fructosamine and Troponin I.

TABLE 2
QAP ASSESSMENT AND LABORATORY RE-ASSESSMENT OF PRECISION ESTIMATE FOR
RCPA QAP GENERAL SERUM CHEMISTRY - Cycles 56 and 59


CYCLE 56 CYCLE 59 INTERNAL QC
CHEM
QAP PRECISION
ASSESSMENT
RECALCULATED
PRECISION
QAP PRECISION
ASSESSMENT
RECALCULATED
PRECISION
PRECISION
MEAN
ALB 1.16 0.637 1.21 0.354 0.3 19
ALP 10.1 6.142 19.5 15.104 2.3 109
ALT 3.5 2.424 5.2 2.475 1.6 46
AMS 12.4 2.069 11.1 6.819 3.3 119
AST 6.3 4.202 8.9 6.072 1.2 27
BICARB 0.75 0.559 1.35 0.919 0.9 9.4
CA 0.06 0.026 0.059 0.034 0.03 1.86
CARB 2.4 2.027 2 0.906 1.1 9.9
CBILI 2.5 1.403 4.9 3.725 2.1 21.0
CHOL 0.06 0.031 0.089 0.045 0.05 3.06
CK 41.4 9.984 32.3 14.257 4.8 174
CKMB 2.8 1.899 2.3 1.561 5.6 242
CL 1.1 0.729 0.7 0.559 0.6 91
CREAT 6.7 5.181 5.5 3.631 1.3 105
DIG 0.2 0.082 0.12 0.085 0.23 1.42
FE 0.64 0.426 0.64 0.504 0.7 21
FRUCT 330.1 188.538 53.8 33.282 40 547
GGT 4.6 1.299 4.5 1.846 1 26
GLUC 0.31 0.238 0.32 0.14 0.16 3.8
HDL 0.033 0.018 0.063 0.043 0.16 1.82
K 0.06 0.043 0.06 0.031 0.02 2.98
LACT 0.22 0.079 0.24 0.043 0.10 1.35
LDH 16.5 10.201 35.2 18.426 15.8 459
LI 0.09 0.053 0.085 0.043 0.02 0.78
LIPASE 57.6 16.33 20.8 5.223 5 264
MG 0.025 0.016 0.034 0.01 0.01 0.61
NA 1.2 0.919 1.2 0.771 1.1 124
OSMO 3.2 2.721 6.2 3.221 1 290
PARACET 219.7 160.285 21.6 11.774 1.2 198
PHENOBARB 7.9 5.136 5.3 4.175 3.6 42
PHENY 4.7 3.549 7.3 5.48 3.9 25
PHOS 0.033 0.017 0.033 0.013 0.02 0.71
SALI 0.046 0.03 0.055 0.029 0.01 0.72
TBILI 1.8 1 5.5 4.822 1.6 14.6
THEO 3.4 1.741 5.3 3.446 7.2 47.1
TPROT 1.4 0.968 1.9 1.381 0.48 44.5
TRANSFERRIN 0.212 0.179 0.061 0.032 0.06 1.73
TRIGS 0.035 0.023 0.023 0.017 0.01 1.80
TROPI 0.253 0.187 0.998 0.899 0.3 2.7
URATE 6.1 2.568 6.6 3.921 1.9 172
UREA 0.31 0.226 0.31 0.253 0.06 4.56
VALP 27.2 20.942 36.6 32.005 5.7 170

Further examination of the laboratory's reassessment results in Table 2 also revealed two interesting points :

Part of the explanation of these differences may have laid in a possible quantitative differences between the mean concentrations in the QAP samples and the mean concentrations in the Internal QC material. The former could have been assigned to the mid point between the lowest and highest target concentrations and this used to calculate a coefficient of variation but this would be tenuous from a statistical point of view.

The author has posted an online resource which can be used by readers to reassess their end of cycle data along the lines described in this report. In addition it provides the opportunity for users to reassess their fixed and proporational biases by means of comparisons with their reagent group's target low and high concentrations, as an alternative to the QAP's assigned low and high concentration values.