Uncertainty, Calibration and Probability: The Statistics of Scientific and Industrial Measurement [2nd ed.] 0750300604, 9780750300605

Citation preview

Uncertainty, Calibration and Probability The Statistics of Scientific and Industrial Measurement

Second Edition

The Adam Hilger Series on Measurement Science and Technology

Uncertainty, Calibration and Probability The Statistics of Scientific and Industrial Measurement Second Edition

C F Dietrich BSc, PhD, FlnstP, ChP

Published in 1991 by Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742

Published in Great Britain by Taylor & Francis Group 2 Park Square Milton Park, Abingdon Oxon OX14 4RN

© 1973, 1991 by C. F. Dietrich First published, 1973, Second edition, 1991

No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 I International Standard Book Number-10: 0-7503-0060-4 (Hardcover) International Standard Book Number-13: 978-7503-0060-5 (Hardcover) Library of Congress Card Number 90-38472 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

FTrademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data

Dietrich, C. F. (Comelius Frank) Uncertainty, calibration, and probability: the statistics of scientific and industrial measurement/C.F. Dietrich.-2nd ed. p. cm. Includes bibliographical references and index. ISBN 0-7503-0060-4 (hbk.) I. Distribution (Probability theory). 2.Mensuration-Statistical methods. I. Title. Il. Series. 1991 QA273.6.D53 90-38472 519.2-dc20

informa

Taylor & Francis Group is the Academic Division ofT&F Informa plc.

Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com

To Drusilla

Contents Paragraph numbers Preface

1

Uncertainties and Frequency Distributions

Introduction: calibration Uncertainty of measurement Sources of uncertainties Uncertainties as distributions Histograms Frequency distributions Probability Asymmetry Gaussian or normal distribution Uncertainty of measurement: definition 2

1.01 1.10 1.11 1.12 1.15 1.16 1.17 1.18 1.20 1.21

The Gaussian Distribution

Simple derivation of the Gaussian distribution Determination of constants A and A. Standard deviation of Gaussian distribution Alternative derivation of Gaussian function Generalization of proof Approximation to the mean J.l of a distribution Approximation to the standard deviation a Mean absolute deviation 11 Combination of two Gaussian distributions Standard deviation of the mean or standard error Standard deviation of the standard deviation Tolerance intervals and tolerance probabilities vu

2.01 2.08 2.11 2.12 2.20 2.22 2.23 2.25 2.26 2.30 2.31 2.35

viii

Contents

Confidence intervals and confidence probabilities with reference to the mean value Weighted mean Standard deviation of the weighted mean Estimation of the standard deviation of the weighted mean from observations made Standard error of the mean by external consistency Standard error of the mean by internal consistency Examples on Chapter 2

3

4

General Distributions A double integral theorem Expressions for the double integral theorem Alternative forms of theorem Integral functions General case: combination of uncertainty populations or frequency distributions Standard deviation of combined or convoluted distribution Rule for compounding variances of uncertainty distributions Standard deviation of L~ xr Standard deviation of arithmetic mean or standard error Combination of a stochastic distribution with a Gaussian distribution Standard deviation of the combination of a stochastic distribution with a Gaussian distribution (for alternative see paragraphs 3.56 and 3.57) Relation between Br and ar of a stochastic distribution Alternative formula for standard deviation of the combination of a stochastic and a Gaussian distribution Example Skewness Coefficient of Kurtosis or peakedness of a distribution Distribution of difference between two distributions The fitting together of two sets of parts with standard deviations of rr 1 and rr 2 and means of x1 and x2 Examples on Chapter 3 Rectangular Distributions Rectangular distribution Compounding or convoluting two rectangular distributions Probability distribution formed by combining two rectangular distributions whose semi-ranges are ± h and ± k, k ;:,: h Standard deviation of above combination

2.47 2.54 2.59 2.61 2.65 2.69

3.02 3.22 3.24 3.27 3.30 3.37 3.40 3.41 3.42 3.43 3.51 3.53 3.55 3.57 3.62 3.65 3.66 3.67

4.05 4.07 4.10 4.11

Contents

ix

Combination of two rectangular distributions with semi-ranges of ±a and ±2a 4.15 Combination of two rectangular distributions each with a semi-range of ±a 4.16 Compounding of three rectangular distributions with semi-ranges of ±j, ± k, ±h. General case j :::;· h + k, j ~ k ~ h 4.17 Combination of three rectangular distributions of equal semi-range ±a 4.28 Combination ofthree rectangular distributions with semi-ranges of ±a, ±2a and ±3a 4.31 Compounding of three rectangular distributions. General case j~h+k,j~k~h 4.33 Combination ofthree rectangular distributions with semi-ranges of ±a, ± l.5a and ± 3a 4.39 Compounding of four rectangular distributions each with a semi-range of ±a 4.41 Combination of a Gaussian and a rectangular distribution 4.49 Approximate rectangular density function 4.55 Combination of a rectangular and a Gaussian distribution 4.57 Numerical computation 4.62 Combination of a rectangular distribution with a Gaussian distribution: probability of an uncertainty in the range - 20" to 20" 4.65 Calculation of approximate probabilities for the combination of rectangular distributions with themselves and with a Gaussian distribution 4.68 Examples on Chapter 4 5

6

Applications Sizing machine with two measuring heads Worked example Sizing machine with one measuring head with known uncertainty of measurement Parts with Gaussian distribution Worked example Parts with rectangular distribution Worked example Interpolation procedure Examples on Chapter 5 Distributions Ancillary to the Gaussian The Student or t distribution Probability of the mean Jl lying within a given range The chi-square (x 2 ) distribution

5.02 5.20 5.27 5.29 5.42 5.46 5.63 5.66

6.02 6.07 6.13

X

Contents Parameters of the x2 distribution Use of x2 distribution to find a maximum and a minimum value of CJ, for a given probability, where the given probability is that of the maximum value of CJ being exceeded and also that of CJ being less than the minimum value of CJ F distribution Parameters of the F distribution Tolerance limits, factors, probabilities, and confidence limits, factors and probabilities The uncertainty of a single reading and the probability of it occurring over a given range: without confidence probabilities. Known parameters Jl and CJ Uncertainty of a single reading: without confidence probabilities. Known parameters x and CJ Uncertainty of a single reading: without confidence probabilities. Known parameters x and s Uncertainty of a single reading: without confidence probabilities. Known parameters Jl and s Uncertainty of a single reading: with confidence probabilities. Known parameters x and CJ Uncertainty of a single reading: with confidence probabilities. Known parameters x and s Uncertainty of a single reading: with confidence probability. Known parameters Jl and s Probability of an uncertainty in the mean: confidence probability and confidence limits Probability of an uncertainty in CJ Index to uncertainties (Gaussian) Derivation of standard deviation CJ from extreme range Derivation of standard error CJx (standard deviation of mean) from extreme range References Examples on Chapter 6

7

A General Theory of Uncertainty In trod ucti on Random distributions Systematic uncertainties Central limit theorem Conclusion

8

The Estimation of Calibration Uncertainties

Standard deviation of a function of n variables Standard deviation of the mean of a function of n variables

6.15

6.16 6.17 6.19 6.21 6.23 6.26 6.29 6.31 6.32 6.34 6.36 6.39 6.41 6.43 6.44 6.47

7.01 7.03 7.06 7.11 7.23

8.02 8.06

Contents Generalized standard deviation in the mean Estimation of component uncertainties Measurable uncertainties Guidance on which tables to use Selection of tolerance probability Estimated uncertainties Rectangular uncertainties Maximum value only uncertainties Gaussian estimated uncertainties Combination of uncertainties to give total uncertainty Summary of terms Examples on Chapter 8 9

8.07 8.08 8.09 8.11 8.13 8.14 8.15 8.17 8.18 8.20 8.27

Consistency and Significance Tests

t test. Comparison of two means when both sets of observations belong to the same population, and so (J 1 = (J 2 , that is both sets have equal variances Combined mean and standard deviation of two consistent means t test. Comparison of two means which do not come from the same population, and so (J 1 #- (J 2 , that is their variances are not equal (i.e. inconsistent means) F test. Comparison of two standard deviations when both samples come from the same population Relationship of F distribution to other distributions F test. Internal and external consistency tests Weights Results consistent Results inconsistent x2 test to check if a hypothetical distribution F(x) fits an empirically obtained distribution Kolmogorov-Smirnov statistic Examples on Chapter 9 10

XI

9.01 9.07 9.08 9.09 9.10 9.12 9.18 9.28 9.30 9.31 9.36

Method of Least Squares

Determination of a mean plane Readings along diagonals Parallel sides Improved coverage of plate Calculation of mean plane Change of axes Standard deviation from mean plane Curve fitting: fitting a straight line Weighted mean line

10.02 10.05 10.08 10.12 10.15 10.21 10.25 10.26 10.30

xii

Contents

Standard deviation and standard error in the mean of y; from 10.31 the mean line r = mt: 10.32 Standard deviation am of the gradient m of the mean line Alternative derivation for standard deviation am of the 10.33 gradient m of the mean line Standard deviation of intercept c 10.34 Alternative derivation for standard deviation of intercept c 10.35 Standard deviation of the intercept x 0 10.36 10.37 A simple treatment of the straight line y = mx + c 10.44 Correlation (linear) 10.58 Significance of r Determination of the constants p, q and sin a normal frequency distribution with two variables (determination from measured values of the variables u and v) 10.65 Derivation of line of regression from 10.69( 1) 10.70 Note on the value of r 10.78 Non-linear correlation and best fit for a parabola 10.80 Alternative derivation of best fit for a parabola 10.82 Least squares fit for general function 10.83 Criterion for choice of best curve or functional relation to fit given data points 10.84 The use of orthogonal polynomials in curve fitting 10.85 10.91 Fitting of observed points to a polynomial Derivatives 10.94 10.96 Worked examples Worked example to illustrate the method of curve fitting 10.100 described and to assess its efficiency Weights 10.101 Selection of data points 10.102 References Examples on Chapter 10 11

Theorems of Bernoulli and Stirling and the Binomial, Poisson and Hypergeometric Distributions

Bernoulli's theorem Average and most probable value of r for Bernoulli's expansion Standard deviation of Bernoulli distribution Bernoulli's limit theorem Stirling's theorem Bernoulli's theorem: approximation using Stirling's theorem Alternative derivation of an approximation to binomial probabilities for moderate values of p, the probability of a single event (i.e. p ~ 1/2) Central limit theorem Poisson distribution

11.01 11.10 11.12 11.13 11.15 11.18 11.21 11.21 11.26

xiii

Contents

Note on Poisson's law of distribution Average value of r Standard deviation of Poisson's distribution The hypergeometric function Standard deviation and mean value of the hypergeometric distribution Approximation to the hypergeometric probability function Approximation to the sum of the hypergeometric probabilities Examples on Chapter 11 Appendix I

11.30 11.31 11.32 11.33 11.37 11.40 11.41

Tables

Choice of table Table 1: Values of the density function of the normal distribution f(c)

= e-c'12 /J(2n)

Table 11: Values of the integral

1

fkl

J(2n)

-c'/2

-k 1 e

d c=

r

p -k1tok1

Table Ill: Values of k1 corresponding to

-k~okl J(~n) =

1 2 kle-c'!

de

Table IV: Student t distribution: J1 and s known Table V: Confidence limits for the mean J1 Table VI: Chi-square

x2 distribution

Table VII: Confidence intervals for

(J

the standard deviation

Table VIII: F test. Upper limits for F. Probability ofF exceeding these values = 0.05 Table IX: F test. Upper limits for F. Probability of F exceeding these values = 0.01 Table X: x and (J known. Tolerance limits for two tolerance probabilities Table XI: x and s known. Tolerance limits for two tolerance probabilities Table XII: x and (J known. Tolerance limits for three tolerance probabilities and two confidence probabilities Table XIII: x and s known. Tolerance limits for four tolerance probabilities and three confidence probabilities Table XIV: J1 and s known. Tolerance limits for three tolerance probabilities and two confidence probabilities

Contents

XIV

Table XV: Probabilities associated with Student Table XVI:

z

q=l

2.21 Now it will be shown later (see paragraph 2.29) that the required density function for z is given by (see 2.29(3)) e- J1 3 (see 3.13(6) and note also that xd > xJ and

that the limits for dfl have been inverted for this reason: the sum implied this anyway since on substituting for ea a minus sign was introduced. 3.16 Finally let us consider the case when the terms of the top row of the array are multiplied by terms in the second row to the left of those shown by the full lines of the array. It is seen that as the set of terms used in the second row starts farther and farther to the left then more terms at the beginning of the first row are left out of the sum. 3.17 Let us consider the sum 3l1g

=

Lf(

xc

+ re

)1/1{(!1 2 + gea)- a(xc + b

re)}eab

3.17( 1)

where f1 = flz

+ gea

3.17(2)

and 3.17(3) g = 0 gives the series 1 11 9 = r since r = ( flz - f1 d /ea (see 3.09 ( 1)). As g increases, the minimum value of r in 3 ry 9 increases as more terms to the left of the top row are left out of the sum. The minimum value of r for given g is obtained

by putting

Therefore !1 2

+ gea - axe - rea b

= Zr

3.17(4)

Thus substituting for flz from 3.07(2) we obtain rmin

=

f1 - flz

g = ---

ea

from 3.17(2)

3.17(5) 3.17(6)

Uncertainty, Calibration and Probability

60

on substituting for flz from 3.17 ( 3 ). The maximum value of r which is always attained in the summation is given by xd =XC+ re or rmax = (xd- xJ/e. The maximum value of g is given when rmin is equal to (xd- xJ/e. The maximum value of g is thus given when r min attains its maximum value and thus

3.17(7) Thus the maximum value of fl is given by substituting gmax in 3.17(2) giving

flmax = flz

+ a(xd - xJ = axd + bzr = fl4

3.17(8)

and 3 1J 9 can be written as

3.17(9) ea

valid for (xd- xc)/e ~ g ~ 1. As before, when r ~ oo, g ~ oo and bx e ~ 0 then the sum represented by 3 1J 9 tends to the integral

fx,

ax) dx f(x)ljf (fl -- b 11-hz, b

a 3ijl' = -

~

bre

=

3.17(10)

a

valid for fl 4

~ fl ~ flz.

The limits for x are obtained from X= Xc

Upper limit Xmax = xc limit of r in 3.17(9).

+re

3.05(5)

+ ((xd- xJ/e)e = xd from r = (xd

xc)/c, the upper

Lower limit xmin = xc + ((fl- axe- bzr)/ca)£ = (fl- bzr)/a from r = (Jl- axe- bzr)/ca, the lower limit of r in 3.17(9). The limits for fl are obtained from fl = flz

+ gca

3.17(2)

When

3.17(11)

upper limit When g

= 1,

fl

=

flz

+ w~flz

as

e~O

lower limit

3.17(12)

General Distributions Let us now consider the sum

3.18

a

61

I

o=l

-b I

g=l

As before, bp = bg;;a = ;;a since bg = 1. Thus in the limit when r--+ oo, g--+ oo and ;; --+ 0, the sums in 3.18 ( 1) tend to the integrals 1 a

-

[/1-

J1'4 3 ~ 11 dp = -1 J1'4 dp Jxd b

/1,

ax] dx f(x)t/1 - b {1- bz,

{1,

3.18(2)

a

where the limits are obtained as at the end of paragraph 3.17, that is, the limits for x are obtained from 3.05(5) whilst those of 11 are obtained from 3.17(2). 3.19 It is to be noted that when

11

=

ax

+ bz

3.05(1)

and

b(zr- zc)?: a(xd - xc)

3.07(3)

with xd > xc, Zr > ze, and a and b are positive, that

+ bze axe + bzr axe + bze axd + bzr

p 1 = axd

3.07(1)

112

3.07(2)

=

113 = p4 =

3.13(5) 3.17(8)

giving 3.19(1) and that 3.10(1)

f

!l-bz,

_

2YJI'

=

ba

x,

a

(11-

ax) f(x)t/1 - b - dx

3.14(2)

Uncertainty, Calibration and Probability

62 valid for 11 1

~ J1 ~

113

_

afx•

f(x)ljJ (11-ax) - - dx

3'7 11 ""'-

b

b

!1-bz,

3.17(10)

a

valid for 114 ~ 11 ~ Jlz. 3.20 If we now consider x and z related by 3.20( 1)

11""' ax- bz

where once again b(zc- ze) > a(xd- xJ, xd > Xe, zc > ze, a and b are positive, and we repeat the arguments of paragraphs 3.05 to 3.18, we obtain an analogous set of integrals, namely 3.20(2) valid for 11~ ~ 11 ~ 11~ 3.20(3) valid for 11~ ~ 11 ~ 11~

a 3~~ = b

Ix, !!+bz,

11 ) dx f(x)ljJ (ax-b

3.20(4)

a

valid for 11'1

~

11

~ 11~

where 3.20( 5)

and =

axd - bze

3.20(6)

Jl~ =

axe - bzc

3.20(7)

Jl~ =

axe - bze

3.20(8)

11~ =

axd - bzc

3.20(9)

11'1

As before 1 J~~~, 1 ~~ d11 a ~

-

1 =-

Jl'~ d11 Jx• f(x)ljJ (ax11) dx --

b ~

~

b

3.20( 10)

11+bz,

3.20( 11) 3.20(12)

General Distributions

63

It is to be especially noted that when ax - bz is used instead of ax + bz, that the corresponding integrals for ax - bz are not obtained by changing the sign of b in the set for ax + bz. Note that the integral expressions for 1ij 11 and 1ij~ are both integrated between xc and xd and are valid between the second largest and third largest value of J.-1, where the J.-1' expressions are obtained from the J.1. expressions by writing - bz for bz in 3.07 ( 1), 3.07 (2 ), 3.13(5) and 3.17(8). In the expressions for 2 ij 11 and 2 ij~ the lower limit of each integral is x"' but the upper limit of 2 ij11 is J.1.- bze whilst that of 2 ij~ is p + bzf. Each expression is valid between the third and fourth largest values of p for each expression, and the upper limit of each integral contains a z term, common to the two values of J.1. between which each integral is valid, but with its sign reversed. Finally 3 ij 11 and 3 ij~ each have xd as their upper limit, but the lower limit of 3 ij 11 is (p- bzd/a whilst that of 3 ij~ is (p + bze)fa. Each expression is valid between the largest and second largest values of p for each expression, and the lower limit of each integral contains a z term common to the two values of p between which each integral is valid, once again with its sign reversed, i.e. bzf for 3 ij 11 and - bze for 3 ij~. 3.21 If we again consider the relationship J.1. = ax

+ bz

3.05(1)

but this time for 3.21 ( 1) with zf > zc, xd > xc, with a and b positive as before, then 1/J(z) should be placed at the top of the array and there there will be more terms in f(x) than in 1/J(z). In the expressions for the IJ 11

should be replaced by

and the other constants replaced as follows: b by a, a by b, ze by xc, xc by ze, zf by xd and xd by zf, yielding 1ij11

valid for

J.-L 1

=

~

f'

1/!(z)f(J.l ~ bz) dz

3.21(2)

~ J.l ~ J.lz

3.21(3)

64 valid for JJ 2

Uncertainty, Calibration and Probability ~

fJ

~ fJJ 3i11'

valid for JJ 4

~

fJ

~

=

~ fz, a

!l-ax. b

t/J(z).f(fJ- bz) dz a

3.21(4)

JJ 1 . Note that

3.21(5) The values of JJ 1 , JJ 2 , JJ 3 and JJ 4 are given by 3.07(1), 3.07(2), 3.13(5) and 3.17(8). The new fJ are obtained by interchanging the constants as prescribed above in the previous fJ and using 3.21 ( 1). If we consider the relationship

= -ax + bz

1.1

3.21(6)

with 3.21(1) then using the rules given at the end of paragraph 3.20 and operating on the ij 11 s of 3.21(1), 3.21(2) and 3.21(3) we have the new ry 11 s and limits as follows

= -axd + bze = -axe + bzr

3.21 (8)

JJ 3 = -axe+ bze

3.21(9)

+ bzr

3.21(10)

> fJJ > /J4 > fJl

3.21 ( 11)

t/l(z).f( bz:

3.21(12)

JJ 1 JJ 2

JJ 4 = -axd

3.21 (7)

These four expressions yield fl2

using 3.21 ( 1), and thus

i7 =

1 11

valid for JJ 3 ~ fJ

~ /J4

2i1~' = valid for JJ 4

~

fJ

~

f' ~t ~

~

a

~

fJ

~

t/l(z).f( bz: fJ) dz

3.21(13)

t/J(z).f(bz- fJ) dz

3.21(14)

JJ 1

3i71' = valid for fJ 2

b

fJ) dz

fz, l'+ax, h

a

fJ 3 . Let us now consider the relationship

fJ = ax- bz

3.20( 1)

General Distributions

65

with 3.21 ( 1) then we obtain the ij~< and their limits by considering the expressions 3.20(2) to 3.20( 4) and 3.20( 6) to 3.20(9) and by using the rules given at the beginning of paragraph 3.21. Thus we have 11'1 = axd - bze

3.20(5)

11~ =

axe - bzr

3.20(6)

11~ =

axe - bze

3.20(7)

11~ =

axct - bzr

3.20(8)

f'

1 ij~ = ~ ~(z)f( bz: 11 ) dz

3.21(15)

valid for 11~ ?: 11 ?: 11~ 3.21(16) valid for 11~ ?: 11 ?: 11~

_,

3Yf 11

valid for 11'1 ?: 11?:

b

=-

11~·

a

fz,

~(z)f

(bz11) dz --

!l+ax, b

a

3.21(17)

Finally if we consider the relationship

11= -ax+bz

3.21(6)

with 3.07 ( 3) we obtain the ry 11 s and their limits by considering the expressions 3.21(7) to 3.21 ( 10) and 3.21 ( 12) to 3.21 ( 14) and by using the rules given at the beginning of paragraph 3.21, whence

+ bze

3.21 (7)

-axe+ bzr

3.21(8)

113 = -axe+ bze

3.21 (9)

+ bzr

3.21(10)

11 1 = -axd

11 2

=

114 = -axct

66

Uncertainty, Calibration and Probability

Thesefourexpressionsyield,u 2

_

t'l11

valid for ,u 4 ~ ,u ~

~

,u 4

~

,u 3

~.Ut

using3.07(3). Thus we have

(ax+,u) bafx, x, f(x)t/1 - b - dx

=

,U3

L

3.21(18)

11+bz,

2ii 11

~

=

a

f(x)t/1( ax: ,u) dx

3.21(19)

f(x)t/1 (ax+,u) - - dx

3.21 (20)

valid for ,u 3 ~ ,u ~ .Ut

_

3'1 11

afx,

=-

b

b

!l+bz, a

valid for ,u 2 ~ ,u ~ ,u 4 . One final word, it is permissible to change the sign of ,u; thus if p = ax- bz, and b(zc - Z 0 ) ~ a(xd - xc), then if we change the sign of ,u, we set ,u = -ax + bz, again for Zr- zc ~ ajb(xd- xc). The size sequence of the various ,u will also be reversed. To see this, compare 3.20(2) to 3.20(4) with 3.21(18) to 3.21(20), where the only difference is the sign of ,u in the variable t/1 and the interchange of the ,u in the limits, since for ax- bz, .Ut ~ ,u 3 ~ ,u 4 ~ ,u 2 whilst for - ax + bz, ,u 2 > ,u 4 > ,u 3 > .Ut, and the limits for tiJ 11 , 2 1] 11 and 3 1]11 are thus reversed, that is respectively to ,u 3 ~ ,u 4 , ,u 3 ~.Ut and ,u 2 ~ ,u 4 .

Expressions for the Double Integral Theorem 3.22

Let us now consider the sum

=I tiJge +I 21Jge +I 31Jge

sl

3.22( 1)

where for the sake of clarity the limits have been omitted. A little consideration of the array and the relevant sums involved, that is equations 3.11 ( 1 ), 3.15( 1) and 3.18(1), shows that the sum S 1 is made up of the sum of the products of each term of the top row with each term of the bottom row. This can be also written Xd -Xc

k = (z, -

r=--

e

I

f(r)s·

r=O

I

z,)!!_ ea

ea

1/J(r)- =

k=O

b

s2

3.22(2)

where

z=Z 0

+ kea/b

3.22(3)

+re

3.05( 5)

and

x=

Xc

67

General Distributions Now (Jz = bkeajb = eajb since [Jk = 1 and and e ~ 0, s2 tends to

Zr

> z•. Thus when k and r ~ oo 3.22(4)

Now we know that when g and r tend to infinity and e tends to zero the sum S 1 tends to ~1 [JJ1. 2~J1dp+ IJ12 l~Jldp+ JJ1· 3~J1dp

J

3.22(5)

1'2

f11

f1J

Also, since all the series involved are absolutely convergent, S 1 = S2 , since S 1 is the same as S2 with the terms rearranged. Thus since f(x) and r/J(z) are bounded and continuous except for a finite number of discontinuities, the limit S 1 =limit S 2 and, finally,

112

ax) fxd f(x)r/1 (/1-_Ihz,+ax, 11 • dx dp ,...._A-.,

1

+-b

r----A-.. hze + axd

Xc

b

1'4

~ dp Jxd + b1 i~ hz, +

ax,

I'- hz,

(/1

ax) dx f(x)r/J - b 3.22(6)

This identity is valid for

b(zr- z.) ;;:,: a(xd- xJ

3.07(3)

and J1

= ax + bz

3.05( 1)

where p1 , p2 , p3 and p4 are given by 3.07(1), 3.07(2), 3.13(5) and 3.17(8), and /14 ;;:,: flz ;;:,: /11 ;;:,: /13 · 3.23 If b(zr- z.) ~ a(xd- xc), then r/J(z) should be placed on the top line of the array, and

Uncertainty, Calibration and Probability

68

should be replaced by

(f.l- bz) dz

~(z)f ----a~

the other constants being interchanged as follows: b by a, a by b, ze by x"' xc by ze, Zr by xd, and xd by Zr, as described at the beginning of paragraph 3.21 yielding

_bz) dz fz, ~(z)f (f.l----~

+-1 lhz.+ax, df.l bzr +axe

Q

fz,

+-1 lbz,+ax, df.l a

Q

Ze

bze + axd

~(z)f

(f.l- bz) dz ----~

a

J.1 - axd

h

3.23(1)

valid for 3.21(1) with f.1 = ax

+ bz

3.05( 1)

and where f.1 1, 11 2 '· f.1 3 and f.1 4 are given by 3.07( 1 ), 3.07(2 ), 3.13( 5) and 3.17(8), and /14 ~ /11 ~ f.12 ~ f.13· Similar identities exist for the other functions of f.l, that is for f.1 = ax - bz, etc., which have been discussed in the previous paragraphs.

Alternative Forms of Theorem 3.24 The identity 3.22( 6) can be written in alternative form by using equation 3.05( 1), solving for x = (f.l - bz )/a and substituting for x in terms of z. This gives

Ix, x,

f(x) dx

I

(f.l

b 1 fbz,+ax, fz, z, ~(z) dz =; Jbz.+ax, df.l z, +-1 ibz,+ax, df.l a

1

+a

bze + axd

Ib

bzr +axe

~(z)f

h

~(z)f

j l - axd

b

(f.l _bz) dz ----~

a

f1 - axd

fbz,+ax, df.l fz,

bz)

~(z)f - a - dz

(f.l-_-bz) dz a

3.24( 1)

69

General Distributions

valid for 3.07(3) Similarly, the identity 3.23( 1) can be written in alternative form by using equation 3.05(2), that is z = (Jl- ax)/b and substituting for z in terms of x. This gives

3.25

(Jl _ ax)

x. Iz, 1 rbz,+ax, I a Ix, f(x) dx z, lj;(z) dz = b Jbz,+ax, djl x,

f(x)lj; - b - dx

I

I'-

+-1lhz,+ax. dJ1 b ~+~

bz, + ax,

a

JL-~

+-1 lbz,+ax, d}l Jx• b

bz,

I' -

(jl-

b

a

bz,

ax)

f(x)lj; - - dx

(Jl _ ax)

f(x)lj; - - dx

b

a

3.25( 1)

valid for b(zr- ze) ~ a(xd - xJ. It should be noted that the two forms in terms of x, that is 3.22(6) and 3.25( 1) have different limits, the former applies to the case b(zr- ze) ·~ a(xd - xJ and the latter to the case b(zr- ze) ~ a(xd - xJ. The other two forms which are expressed in terms of z, that is 3.23 ( 1) and 3.24( 1), also have different limits and it is to be noted that 3.23(1) covers the case when b(zr- ze) ~ a(xd- xJwhilst3.24(1)coversthecaseb(zr- ze) ~ a(xd- xc). 3.26

Integral Functions 3.27

Let

+ bz

3.05( 1)

a(xd - xc)

3.07(3)

Jl = ax

and b(zr - ze)

~

The integral function of Jl made up of JL-bz,

+

Ix. x,

valid for

(Jl-

ax) dx f(x)lj; - b

valid for Jlz

~

J1

~

Jlt

+

}1

1~ Jl ~ Jl3

(Jl-

fx.

ax) dx } f(x)lj; - JL-bz, b a

valid for

Jl

4

~

J1

~

Jlz

3.27 ( 1)

Uncertainty, Calibration and Probability

70 where

= axd + bze

3.27(2a)

Jlz = axe + bzc

3.27(2b)

J13 = axe + bze

3.27(2c)

axct + bzc

3.27(2d)

}1 1

}1 4 =

and J14 ~ Jlz ~ J11 ~ J13

is of interest since, as we shall see later, it represents the combined or convoluted probability density function of f(x) and lj;(z), where these are themselves probability density functions, and x and z are related by the equation J1 = ax + bz (3.05(1)). It is to be noted that if x and z are the mean values of x and z, then the mean value of J1 is given by

fi

=

ax + bz

3.27(2e)

J1

=

ax

+ bz

3.05(1)

If

but b(zc- ze)

~

a(xd- xJ (3.21(1)), then the appropriate integral is

P(Jl)=l(zi1!i+liil'+3ii 1J=~{L +

f'

lj;(z)f(Jl~bz)dz

a

valid for

}1 2 ~

J1

~

J13

1/J(z)f( J1 ~ bz) dz + I:~ax. lj;(z)f(Jl ~ bz) dz} b

valid for

}1 1 ~

J1

~

Jlz

valid for

}1 4 ~

J1

~ }1 1

3.27(3)

where J1 4 ~ J1 1 ~ Jlz ~ J1 3 and the J1 are given by 3.27(2a), (2b), (2c) and ( 2d ). The integral is obtained by operating on 3.27 (1 ), using the rules given at the beginning of paragraph 3.21. 3.28 Integral 3.27 (I) can be expressed in terms of z, and integral 3.27 ( 3) in terms of x, by using J1 = ax + bz (3.05(1)) and recalculating the limits for x or z for the integral signs. Note that the limits for the J1 remain unaltered

71

General Distributions

when x and z are interchanged. P(JL) ( 3.27 ( 1)) as a function of z is P(/1) = -1 a

{f-b z,

(JL

+

bz) dz 1/J(z)f - a

fb 1'-ax.

(JL

bz) dz 1/J(z)f - a

b

valid for !1 2 ? 11 ? /11

valid for 111 ? 11 ? /13

+ r~ax, 1/J(z)f(/1 ~ bz) dz}

3.28(1)

b

valid for /1 4 ? 11 ? !12 where /14 ? llz ? /11 ? 113 · P(/1) (3.27(3)) as a function of xis P(/1) =

1{f

b

I' -bz,

11-bz,

x,

a

- dx - bax) f(x)I/J (11

1'-hz, + fa

- dx - bax) f(x)I/J (11

a

+

I

valid for !1 2 ? 11 ? 113

x,

11-bz,

valid for /11 ? 11 ? llz

ax) dx } -f(x)I/J (/1b

3.28(2)

valid for 114 ? 11 ? 11 1 where /14 ? /11 ? !lz ? 113· Similar functions exist for the other functions of 11· For 11 = ax- bz

3.20( 1)

and 3.07(3) then

l{ L f(x)I/J( J-l+hz,

P(/1) =

~(ziil' + 1ii 11 +

3i7 11 )

=

a

ax; 11 ) dx

valid for 114 ? 11 ? llz 3.28(3) a

valid for /1 3 ? 11 ? /14 where /11 ? /13 ? /14 ? llz ·

valid for 11 1 ? 11 ? 113

Uncertainty, Calibration and Probability

72

For 3.26( 1)

1-l = ax- bz but with

3.21(1) then

P({l)=t(ziil'+tii~t+3iitt)=~{f

h t/l(z)f(bz:{l)dz

valid for {l 3 ;?: 1-l ;?: 1-lz 3.28(4) h

valid for where

{l 1 ;?: {l 4 ;?:

{l 4 ;?:

valid for

fl ;?: {l 3

{l 1 ;?:

fl ;?:

{l 4

J1 3 ;?: Jlz and where !1 1 = axd - bze

3.20(5)

axe - bzr

3.20(6)

fl3 = axe - bze

3.20(7)

= axct - bzr

3.20(8)

+ bz

3.21(6)

Jlz

{l 4

=

For 1-l

=

-ax

and 3.07 ( 3) then

P({l)

=

~(ziitt + tiitt + 3iitt) =

t{ L f(x)t/1( a

valid for

f

ax: J1) dx

{l 3 ;?:

fl ;?:

{l 1

+ 1-l) Jxd (ax + 1-l) } + x,xd f(x)t/1 (ax - b - dx + Jl+hz/(x)t/1 - b - dx

3.28( 5)

a

valid for J1 4 ;?: J1 ;?: !13 where Jlz;?: !14 ;?: {l 3 ;?: 3.21(9) and 3.21(10).

{l 1

and

{l 1 ,

valid for Jlz ;?: {l ;?: /-l4 flz, J1 3 and {l 4 are given by 3.21(7), 3.21(8),

73

General Distributions

For p

=

-ax

+ bz (3.21(6)), but with 3.21(1)

b(zr- ze) ~ (xd- xJ then

P(p)=l(2ij 1 +1ij 1 +3ij 1 )=~{L +

f'

1/f(z)f(bz:p)dz

h

valid for p 4

1/J(z)f( bz:

p)

dz

~ p ~ P1

+ f:~ax, 1/f(z).f( bz:

P)

dz}

3.28(6)

b

valid for p 3 ~ p where p 2 ~ p 3

~

p4

~

~

p4

p 1 and where

valid for p 2 ~ p

~

p3

+ bze

3.21 (7)

p 2 = -axe+ bzr

3.21 (8)

p 3 = -axe+ bze

3.21(9)

+ bzr

3.21(10)

p1 =

-axd

p 4 = -axd

Range of P(p) 3.29 It is interesting to note that in all six cases of P(p) considered, namely p = ax + bz, p = ax- bz, and p = -ax + bz, with b(zr- zc) ~ a(xd- X 0 ) or b(zr- ze) ~ a(xd- X 0 ), for each value of p the total range of the convolution function P(p) is equal to b(zr - ze) + a(xd - X 0 ). Likewise the range of 2 ij 11 or of 3 ij 11 for b(zr- ze) ~ a(xd - X 0 ) is always a(xd - xJ whilst for b(zr- ze) ~ a(xd- xJ it is always b(zr- ze). The range of 1ij 11 is always b(zr- ze)- a(xd - xJ when b(zr- ze) ~ a(xd - xc) and is always a(xd- xJ- b(zr- ze) when b(zr - ze) ~ a(xd - xJ.

General Case: Combination of Uncertainty Populations or Frequency Distributions 3.30 Let IX( x) be a bounded function between - c and d, and since it is to be considered as a probability density function it must also be positive between these two limits. If a function is to be a density function between the limits - c to d its integral between these limits must be equal to unity. Manifestly 1X(x) 3.30( 1) px = sd -c IX( X) dx

is such a function (see Figure 3.30).

Uncertainty, Calibration and Probability

74

-c

-r-

Figure 3.30

3.31 We first assume, as a matter of conveniencet, that x mean value of a(x) between the limits -c to d. Now X

fc

a(x) dx =

fc

=

0 gives the

3.31(1)

xa(x) dx

and Jd_ c a( x) dx > 0, thus for x to be zero

I:c

3.31 (2)

xa(x) dx = 0

The standard deviation (Jx of Px, which is defined as the square root of the mean second moment about the mean value of x, is thus given by a2X

=Id x2P =Jd_cx a(x)dx =Id x a(x)dx X

dx

-c

2

2

Jd_ca(x)dx

-c

A

3.31(3)

where A=

I:c a(x)

dx

3.31(4)

3.32 Similarly, if {J(z) is another function, bounded and positive between - e to f, with its mean at zero, then, as before, 3.32( 1)

and the associated density function is given by pz

t This

{J(z)

=se-efJ(z)dz

is merely a question of choice of co-ordinates.

3.32(2)

75

General Distributions whilst its standard deviations a z is given by the expression a2 = Ir z2p dz = Jr_ez2{3(z)dz =If z2{3(z)dz z -e z JC-ef3(z) dz -e B

3.32(3)

B = f/(z)dz

3.32( 4)

where

Let us now consider the compound probability of an uncertainty ax + bz, where the density functions for x and z are given by equations 3.30(1) and 3.32(2). Let 3.33( 1) J1 + bJl )!: ax + bz )!: J1

3.33

Then the probability of an uncertainty lying between the limits specified by the above inequality is P xp z dx dz, integrated over the range specified by the inequality and by the limits of the functions P x and P z, that is - c to d, and - e to f respectively. 3.34 Let us first specify that a

3.34( 1)

f+e)!:-(d+c)

b

and integrate Pz over the range J1

+ bJl - ax

J1 - ax

- - - - - )!: z )!: - - b b

3.34(2)

Thus 11 +b11 -ax

{_a:

p dz z

=

If3(z) dz

=

{3((}1- ax)/b) bJl

B

B

b

3.34(3)

h

from the mean value theorem and where {Jz required probability P11 d}1 is given by

=

bJ1/b from 3.34(2). Thus the

Jla(x)f3 ( bax) - dx dJ1/bAB integrated over the range specified by 3.33( 1) and the limits of the functions a(x) and b(z). Now the limits of J1 are given by using the expression J1 = ax

+ bz

3.34(4)

Thus the minimum value of J1 is given by x = -c and z = -e, and so Jlmin = - (ac + be). The maximum value of J1 is given by x = + d, and z = f, giving f.lmax = ad + bf. But we cannot just integrate this expression between these limits, because a(x) is defined only between -c to d and f3(z) is defined only between -e to f; outside these ranges they are to be considered zero but, since neither will in fact be zero outside these limits, means must be

Uncertainty, Calibration and Probability

76

found of circumventing this problem. The functions P(x) and P(z) may be specified by algebraic expressions such that at -c and d, P(x) is zero and at - e and f, P( z) is zero and, in between these values, the algebraic expressions give a close enough fit to P(x) and P(z). However, outside these limits the expressions will have finite values and thus integrating the expression for P(11) between the limits l1min = - (ac + be) and l1max = ad + bf will lead to an incorrect value for the integral P(11 ). 3.35 If we now consider the array in paragraph 3.06 we see that the integrals 1 ~ 11 , 2 ~ 11 and 3 ~ 11 which we obtained from the array satisfy the conditions we have stated in the previous paragraph. The two functions f(x) and t/J(z) used in conjunction with the array were defined between set finite limits, were zero outside these limits and, further, were related by precisely the same relation as we are now considering, that is 11 = ax + bz. In the case of the array we obtained three values for the function P(11) of 11, each valid over a certain range of J1. Thus the function P(11) we require is given by considering the function P(11) given by 3.27 ( 1). 3.36 If we compare the two functions of x and z we see that f( x) of 3.27 ( 1) is equivalent to Px a(x)/A and that t/J(z) of 3.27(1) is equivalent to Pz fJ(z)/ B. The relationship between z and xis the same, that is 11 = ax + bz. Thus all we have to do is to compare limits. This leads to

=

=

XC=

-c,

xd = d,

ze

= -e, and Zr = f

Thus, inserting the expressions for Px and Pz in 3.27(1) we have 1P(I1) , r~----------~---------fJ+bc

valid for 11 1 ?: 11 ?: 11 3 zP(11) ~~

+ ( -1bAB

fd

-c

~~ dx a(x)f3 (11-ax) b

valid for 11 2 ?: 11 ?: 111

1 fd a(x)/3( 11 - ax) dx)} d11 b

+ (bAB

11 -br a

valid for 11 4 ?: 11 ?: 11z

3.36( 1)

77

General Distributions

where 11 2 = -ac + bf} 114 =ad+ bf

11 1 =ad- be, 11 3 = -ac- be,

3.36(2)

Thus the integral 1 P(J1) gives the value of P(Jl), the required density function over the range 11 1 ): 11 ): 11 3 . Similarly 2 P(J1) and 3 P(J1) give P(/1) over the ranges given in 3.36( 1). The probability of an uncertainty in a given range of 11 is obtained by integrating the appropriate functions, 1 P(J1), 2 P(J1) and 3 P(J1), over the required range of Jl. If desired, of course, P(/1) can be written in the form I'

P(/1) d11 == A1B {

+ be

f_ca

J1

et.( X) dx

+ bJl- ax

{-axb

f3(z) dz

h

+re

J1

a(x) dx

+

bi

er

I

-er

-3er -5·5a

-4a

-2a

'' " 3er

er

0 -a

\

a

2a

5·5a

-uncer-tainty-

Figure 4.40 Combination of three rectangular distributions with semi-ranges of 3a, 1.5a and a, with a normal distribution of equal standard deviation superposed

starting point the functions P 1 , P 4 and P 7 of paragraph 4.28 which represent the composite function for the combination of three rectangular distributions each having a semi-range of ±a. 4.42 Starting first with 4.28( 1) we combine this with a rectangular distribution of semi-range ±a. As we are again considering the probability of occurrence of J.l = x + z, we put a = b = 1 in 4.07 ( 2 ). Since the range of function 4.28 ( 1) is 2a, equal to that of the rectangular distribution we are combining with it, it does not matter which function we choose to substitute for 1/J or f in 3.27 ( 1), since 4.07 ( 3) is satisfied either way round in this special case of equality. Let us substitute 4.28( 1) for f(x) and the rectangular distribution of semi-range a for 1/J. 4.43 Thus we put 4.28( 1) and

1/J(z) = 1/2a

4.43( 1)

By inspection, the limits are xc = - 3a, Ze

= -a,

xd Zr

= -a (see 4.28 ( 1)) =+a

Thus substituting in 3.27 ( 1) we obtain the following components

f.!+a

= {(3a + x) 3 /96a 4 } _ 3 a = (4a + J.l) 3 /96a 4

4.43(2)

Uncertainty, Calibration and Probability

130 valid for - 4a

J1

~

- 2a, range

~

P 2(J1) = valid for - 2a is zero.

~

J1

P3(J1) =

= 2a.

r-:a{(3a + x)2/32a4} dx

4.43(3)

- 2a, range = 0. P z(J1) is thus omitted because its range

~

I~aa {(3a + x) 2/32a4} dx

= {(3a

+ x) 3 /96a 4 };~a =

{8a 3 - (2a

+ J1) 3}/96a4

4.43(4)

valid for - 2a ~ J1 ~ 0, range = 2a. 4.44 Dealing now with 4.28(2), we proceed to combine this with the rectangular distribution of semi-range ±a. Proceeding as in paragraph 4.42, since the range of 4.28(2) is also equal to 2a, we can again choose 4.28(2) or the a rectangular distribution to substitute in place of t/J or f in 3.27 ( 1). Let us in this case substitute 4.28(2) for t/J(z) and the a rectangular distribution for f(x). 4.45 Thus we put f(x) = 1/2a

4.45( 1)

and 4.28(2) By inspection, the limits are

ze = -a,

Zr

= +a (see 4.28(2))

Substituting in 3.27 ( 1) we obtain the next three components of the composite probability distribution as

using J1 = x

+ z,

that is, 4.07(2) with a = b = 1

= {3a 2 x + (Jl-

x?/3}~-;;a/16a 4

= {-(Jl + a) 3 /3 + 3a 2 J1 + 17a 3 /3}/16a 4 valid for - 2a

~

J1

~

4.45(2)

0.

P 5 (J1)

=fa {3a

2 -

(Jl- x) 2 }/16a 4

4.45(3)

Rectangular Distributions

valid for 0

~

J.1

~

131

0. P 5 (J.1) is thus omitted because its range is zero.

P 6 (J.1) = =

J:_a {3a

(}.1- x) 2 }/16a4

2 -

[{3a 2 x- (}.1- x) 3/3}/16a 4J:-a 4.45( 4)

valid for 0 ~ J.1 ~ 2a. Finally we combine the rectangular distribution of semi-range ±a with distribution 4.28(3). As previously, since the range of 4.28(3) is also 2a it can be substituted for either f(x) or !/J(z) in 3.27(1). Let us substitute 4.28(3) for f(x) and the a rectangular distribution for !/J(z). Thus we put 4.46

f(x)

= (3a-

xf/16a 3

4.28(3)

and 4.46(1)

!/J(z) = 1/2a

By inspection, the limits are xc =a,

xd = 3a

-a,

Z0 =

(see 4.28(3))

=a

Zc

Substituting in 3.27 (1) we obtain the final three terms of the composite distribution we require, giving P 7 (J.1) =

f

Jl+a

a

dx

{(3a- x) 2 /16a 3 } 2a

= - - 1

96a 4

valid for 0

~

J.1 ~ 2a.

f f

3

P 8 (J.1) = ~ J.1 ~

4.46(2)

1- (3a- x) 2 dx 32a 4

4.46(3)

a -

a

valid for 2a

{( 2a - J.1) 3 - Sa 3}

2a. P 8 is thus omitted as its range is zero. P 9 (J.1) =

3

1- (3a- x) 2 dx 32a 4

-

a

1'-a

= {- ~ (3a96a

= ( 4a-

valid for 2a

~

J.1

~

4a.

J.1) 3 /96a 4

x)3}3a Jl-a

4.46(4)

Uncertainty, Calibration and Probability

132 4.47

Where overlapping of functions occurs it is complete, and so makes the task of sorting into ascending intervals easier. Doing this, and adding we get the final components of the combination of these rectangles as follows:

+ /1) 3j96a 4

P'1 = ( 4a valid for - 4a

- 2a, range

~

11

~

p~

=

_1_ {-(2a + /1)316a 2 6a 2

~

11

valid for - 2a

~

2a

=

~~ + /1)3 + 1a + 3!1} 3a 2

, __1_{-(2a-11? _(a-!1) 3 2 2 2 16a 6a 3a

~

11

~

2a, range

+ 1a _ 3/1 }

~

11

~

4a, range

4.47(2)

2a

=

P~ = ( 4a- /1) 3 j96a 4

valid for 2a

4.47(1)

0, range = 2a

P3valid for 0

4.43(2)

=

4.46( 4)

2a.

4.48 The standard deviation of the combination is given by 3.41 ( 1) (J

=

J{4( a I 3)} = 2

1.154a

Figure 4.48 shows a graph of this combination, with a Gaussian distribution of equal standard deviation superposed. Since the range of the combination is ± 4a, its range exceeds ± 3(J = 3.462a, and so besides the probability of an uncertainty exceeding ± 2(J, the probability of an uncertainty exceeding ±3(J is given, and compared with the corresponding probabilities for a Gaussian distribution. Note the very close correspondence between the combination distribution and the Gaussian distribution. Standard deviation of four combined rectangles et= 1·154a

Probability of an uncertainty greater than 20' =0·0420 Four rectangles combined =0·0455 Normal distribution 30' =0·0007 Four rec I angles combined =0·0027 Normal distribution

1 - 0·385

Four rectangular "" "---di51ributions

>-

3a =-et-

u

c

"

"'c:r

::J

U:"'

"

Normal distribution ~

~

I

-3cr

-cri

0

30"

-a -uncertainty~

Figure 4.48 Combination of four rectangular distributions each with a semi-range of a, with a normal distribution of equal standard deviation superposed

133

Rectangular Distributions

Combination of a Gaussian and a Rectangular Distributiont 4.49 Before leaving the discussion of the combination of rectangular distributions, we investigate the convolution of a rectangular distribution with a Gaussian one. Let the Gaussian distribution be

e- z2j2o-~

4.49( 1)

pk = ----,-erk~(2n)

where erk is the standard deviation. If the standard deviation of the rectangular distribution is erh, then its range is ±~3 · erh and it will be represented by 1 ph=--,---2~3-erh

4.49(2)

that is the reciprocal of twice the range. Referring to paragraph 4.07, a= b = 1 since we require the density function of 11 = x + z (see 4.07(2)). Thus, in order to satisfy 4.07(3), we substitute Pk in place of tf;(z) and Ph in place of f(x). By inspection xc = -~3erh, xd = ~3erh, ze = - oo, Zr = oo. Thus substituting in 4.07(1) we have 4.50

/11 =

-eo,

/13 = - 00,

f.12 =

oo

/14 = CO

Referring now to the validity limits of the three integrals of 3.27 ( 1), we see that only P if.l) survives since 11 1 = 11 3 and 11 2 = f.1 4 . Thus substituting in P 2 (f.1) of 3.27(1) for tf;(z) and f(x) we have as the density function of our combination .j3 a, and that the standard deviation of the parts is (JT, with a mean of .Xr. We assume also that each limit point has a probability function associated with it, such that the probability ofthe part being accepted varies from 0 to 1 over a small range of sizes. We will also need to assume a suitable form for this function and to derive expressions for the fraction of parts the machine will accept which are ( 1) the correct size, (2) undersize and ( 3) oversize. Also derive expressions for (4) the fraction of correct parts rejected, ( 5) the fraction of oversize parts rejected and (6) the fraction of undersize parts rejected. 5.04 Let Pp be the density function of the manufactured parts, which we will assume to be Gaussian, that is e- (x- xT) 2/2o5.04( 1) Pp= (Jrj(2n) 1

(See Figure 5.04.) 150

151

Applications

p

I ---- e,-~-- e, ___...._

0 x,-e,=a

i,

-Size x -

x,•e,=b

Figure 5.04

Let the sizing machine be set to accept parts between the sizes a and b, b > a. If the machine were perfect then the fraction of undersize parts rejected would be Ja_ oo Pp dx whilst the fraction of oversize parts rejected would be Jh" Pp dx. 5.05 However, since no sizing machine is perfect, some oversize parts will be accepted as will some undersize parts, and correspondingly some correct parts will be rejected either as oversize or undersize. Now with each limit point a or b at which the sizing machine has been set, there will be an associated distribution function. Let this function be denoted by f(x). At the lower limit point a we write f(x) as f 1 (x- a), whilst at b we write it as f 2 ( x - b). The meaning of this function is as follows. Consider f 1 ( x - a); if we put x = a, then f(O) is the probability that a part of size a would be accepted. As xis increased we should expect the probability of a part being accepted to increase, that is .f1 (x-a) > f(O) for x >a. Correspondingly if x b we should expect .f2 ( x - b) < f (0) and vice versa. The probability of rejection at the point x for f 1 is {1- f 1 (x- a)} whilst the probability of rejection by .f2 is { 1- f 2 (x- b)}. The probability to be associated with x = a or b will be discussed later in paragraph 5.07 et seq. In order to size components, parts must first pass through the upper sizing limit band then proceed to the lower sizing limit a. The probability of a part of size x being accepted is given as follows. The probability of acceptance at limit b, which means passing through the limit point, is .f2 (x- b), whilst the probability of acceptance at limit a, which means not passing through the limit point, is given by .f1 (x- a), giving the total probability of acceptance as the product of the probabilities of these two separate events, that is .f2 ( x - b ).f1 ( x - a). The probability of rejection is found as follows. There

152

Uncertainty, Calibration and Probability

are now two possibilities. (i) The part is rejected at limit point b, that is, it will not pass through the limit point, and the probability of this happening is [1- f 2 (x- b)]. (ii) The part is accepted by limit point b, and passes through to limit point a. The probability of acceptance at limit point b is / 2 ( x - b). The part is now rejected at limit point a, that is it passes through, and the probability of this happening is [ 1 - / 1 ( x - a)]. The compound probability of these two successive events, leading to a rejection, is thus / 2 (x- b)[l- / 1 (x- a)], that is the product of the two probabilities. The total probability of rejection is given as the sum of the probabilities of(i) and (ii), that is [l- / 1 (x- a)f2 (x- b)]. This value also follows from the relationship that the probability of acceptance plus the probability of rejection is equal to unity. A fraction

5.06

e-(x-xT)'/2ai dx

5.06( 1)

of the manufactured parts lies between x and x + dx. Thus the fraction dP A of these accepted by the sizing machine will be 5.06(2)

Correspondingly the fraction dPR rejected will be 5.06(3)

Thus the fraction of parts whose sizes lie between a and b, that is correct size parts which are accepted, will be CA=

r

dPA

5.06(4)

The fraction of oversize parts accepted will be 5.06(5)

whilst the fraction of undersize parts accepted will be 5.06(6)

153

Applications Similarly the fraction of correct size parts rejected will be CR =

f

5.06(7)

dPR

and the fraction of oversize parts rejected will be OR=

I"

5.06(8)

dPR

whilst the fraction of undersize parts rejected will be

UR

=

J:oo dPR

5.06(9)

Note that 5.06(10) The form ofJ( x) 5.07 In order to proceed we must now make some assumptions about the form of f(x). If we consider the lower limit of a sizing machine, we require a function which is small when x is much less than a and which approaches 1 when x is much larger than a. Also when x = a there is a case for assuming that f(x) is 0.5, that is that acceptance or rejection is equi-probable. Further there is some case for considering f(x) to be anti-symmetrical about the limit point, that is, the probability of acceptance is increased when apart is a given amount larger than the value a, by the same amount as its probability of acceptance is decreased, when its size is the same amount smaller than the value a. Also the slope of the function should be linear in the region of the set position of the limit point a, where the probability of acceptance is 0.5. A simple function fulfilling these requirements is e-(x- a- p) 2 /2p 2

f 1 (x- a ) = - - - - 1:

valid for - oo

~

x

~

a and e-(x- a+ p) 2 /2p 2

f 1 (x- a)= 1 - - - - - 1:

valid for

a~

x

~

5.07(1)

5.07(2)

oo where 1:

= 1.213 0613

5.07(3)

(See Figure 5.07(1).) When x =a this function has a value 0.50, when x =a+ p its value is 0.8885, and when x = a- p its value is 0.1115. These latter two values are

154

Uncertainty, Calibration and Probability expj-(r-a-p.J'/2p: f.=

T

Valid for - oo ~ x

~

l

f.::: 1 _ ,-

a

expj-(r-a+p,l'/2p,'j T

Valid for

a

~:x: ~

A

A

oo

1-0T-----------------------~-----------------------

t

p 0·5

0·8885

0·1115

a Appearance of f 1(x- a) distribution function

Figure 5.07(1)

thus the probabilities of acceptance at plus and minus one p from the limit point. Note in this instance pis not a true standard deviation in the defined sense of the word, and it is for this reason that the symbol pt has been used rather than (J. For the upper limit, the two functions will be / 2 (x-

b)=

{

e-(x-b-p) 2 /2p 2 }

1 -----------

5.07(4)

r

t The distribution function f 1 ( x - a) has a density function given by d {! 1 ( x - a)}/ dx, which has a standard deviation given by a = 0.8298p. A similar relation holds for f 2 (x- b). The distribution functions fdx- a) and f 2 (x- b) are reasonable approximations to the normal distribution as the following table shows: p A..

r

1-P

'

-"---

r

'

ka

Normal

f(x)

Normal

f(x)

a 2a 3a

0.6827 0.9545 0.9973

0.6909 0.9517 0.9963

0.3173 0.0455 0.0027

0.3090 0.0483 0.0037

where P = Probability that ka ;::, x - a ;::, - ka 1 - P =Probability that x- a;::, ka and x- a::::; -ka where f(x) = ! 2 {(b- ka)- b}- ! 2 {(b + ka)- b} = f 2 ( - ka)- f 2 (ka) = P, ka;::, x- b;::, -ka and f(x) = ! 1 {(a-ka)- a}- ! 1 {(a-ka)- a}= f 1 (ka)- ! 1 ( - ka) = P, ka;::, x- a;::, -ka

155

Applications

f,: 1-

exp)-lr-b- p.J'/2p~J

f,:

T

Valid

exp[-lr-b+ p.J'/2p~J

Valid for b

for-co.sx~b

T

~r ~eo

t

p 0·5

0·8885

I

0·1115 Ox~

Appearance of f 2(x- b) distribution function

Figure 5.07(2)

valid for - oo :::::; x :::::; b and e-(x -- b

+ p) 2 /2p 2

fz(X- b ) = - - - - -

'

5.07(5)

valid for b:::::; x:::::; oo. (See Figure 5.07(2).) 5.08 The derivation of the standard deviation of f 1 is obtained as follows: e-(x- a- p) 2 /2p 2

f1(x- a ) = - - - - -

'

valid for - oo :::::; x :::::; a and

e-(x-

a+

5.07(1)

p) 2 /2p 2

f 1 (x- a)= 1 - - - - - -

'

5.07(2)

valid for a :::::; x :::::; oo. Since f 1 ( x - a) is a distribution function, its density function is found by differentiation. Whence dfl

dx

- e - ( x - a- p) 2 /2p 2

------;;--. (x- a- p) -rp2

5.08( 1)

valid for - oo :::::; x :::::; a and dfl

dx valid for a :::::; x :::::; oo.

e-(x

-a+

p) 2 /2p 2

-----o---.

-rp2

(x- a+ p)

5.08(2)

156

Uncertainty, Calibration and Probability

Thus the standard deviation of the complete function is given by

CD -e-(x- a- pf/2p 2

-----;:;--. (x- a) 2 (x- a- p)dx r:p2

f

ao e-(x- a+ p) 2 /2p 2

+ In integral

CD put

2

r:p

a

(x- a- p) p

I

-1

.(x-a) 2(x-a+p)dx

= y whence

CD becomes

e-y';z

- -ao -,-(Y + l)2yp2dy . (x-a+p) In mtegral (2) put p

5.08(3)

5.08(4)

= z whence (2) becomes 5.08(5)

The total integral is thus -2 r:

Jao e-V'I2(V2- l)Vp2 dV =I= az 1

Integrating by parts we have I= -2p2 r:

Joo (V2- l)d(e-V'/2) 1

l 0

+ 4p2 r:

e-1/2

5.08(7)

Applications

157

Considering the last integral we have

since e- V'/ 2 is a symmetric function and

-1 J+1 Joo Joo J-oo + -1 + +1 = -oo whence

Thus Jr' e-V'I 2dV can be found from Table II of Appendix I. Evaluating I gives (J

=

0.8298p

5.08(8)

or p = 1.205 08(J

Using

f 2 gives a similar value for

5.08(9)

(J.

t5 1 and t5 2 positive. Lower limit measuring system 5.09 We now have the problem of finding p for each limit point, whilst a and b are the sizes between which the machine is required to accept parts. p is found as follows. Take a measured part of known size, say a, and set the sizing machine to reject parts less than a using only the lower limit measuring system. But suppose that the setting dial of the lower limit measuring system reads in error, and that when it is set to the size a it is really set to reject parts less than a'= a+ and

k 2 = (b 2 - c 1 + p)jp k1 = (b 2 - h 1 + p)jp c 1 and -f +a,.::; b 1 then k 2 = (b 2 +f-a+ p)jp k 1 = (b 2 - h 1 + p)jp b 1 then k 2 = (b 2 +f-a+ p)jp k 1 = ( b2 + f - a + p ), i.e. integral = 0 (equal limits).

If -

f + a ~ c 1 and - f + a ,.::; b 1 then

CA 3 = (b 2

-

bd/2!

if -f +a> b1 and f CA 3 = (b 2

+a~

b2 then

5.51(3)

+ f- a)/2!

if -f +a> b 1 and f +a< b2 then CA 3 = (f +a+ f - a)/2! = 1

n)[

CA4 = pj( 2

4.fr

P -k 2 tok 2

-k,:'ndk

J 1

5.51(4)

Uncertainty, Calibration and Probability

196 where if -

f + a :( c 1

and k 1 if - f +a> c 1 and k 1 if -f +a> b 1 and k 1 if - f +a> b 1 and k 1

then k 2 = (b2 - b 1 + p) I p = (b 2 - b 2 + p)IP = 1 and - f +a:( b 1 then k 2 = (b 2 - b 1 + p)lp = (b 2 - b 2 + p)lp = 1 and f +a~ b 2 then k 2 = (b 2 +f-a+ p )I p = (b 2 - b 2 + p)lp = 1 and f +a< b 2 then k 2 = (b 2 +f-a+ p)lp = (b 2 - f-a+ p)IP CAs=

p~~nl-k~ok 2 -k~okJ

5.51(5)

where

f + a :( c 1

then k 2 = (b 1 - b 1 - p) I p = - 1 and k 1 = (b 1 - b 2 - p)lp if - f + a > c 1 and - f + a :( b 1 then k 2 = (b 1 - b 1 - p )I p = - 1 and k 1 = (b 1 - b 2 - p)lp if -f +a> b 1 and f +a~ b 2 then k 2 = (b 1 + f - a - p)lp and k 1 = (b 1 - b 2 - p)IP if - f +a> b1 and f +a< b 2 then k 2 = (b 1 + f - a - p)IP and k 1 = (b 1 - f - a - p)lp if -

CA6

= pj( 2n)[ P

4fr

-k 2 tok 2

-k~ok 1

J

5.51(6)

where if f

+ a ~ c2

then k 2 = (b 2 - b2 - p )I p = - 1 and k 1 = (b 2 - c 2 - p)lp if f + a ~ b 2 and f + a < c 2 then k 2 = (b 2 - b 2 - p )I p = -1 andk 1 =(b 2 -f-a-p)IP if f + a < b 2 then k 2 = (b 2 - f - a - p )I p and k 1 = (b 2 - f - a - p)lp, i.e. integral is zero. CA7

= pj(2n)[ 4/r

p -k 2 tok 2

-k~ok,

J

5.51(7)

where if f +a~ c 2 then k 2 = (b 1 - b 2 - p)lp and k 1 = (b 1 - c 2 if.f +a~ b 2 and f +a< c 2 then k 2 = (b 1 - b 2 - p)lp and k 1 = (b 1 - f - a - p )I p if f +a< b 2 then k 2 = (b 1 -f--a- p )I p and k 1 = (b 1 - f - a - p)lp, i.e. integral is zero.

-

p)lp

197

Applications

Finally CA= CA1- CA2 + CA3- CA4- CAS+ CA6- CA7

5.52

eR

The required expression for

- fb, dx {

eR-

c,

-

e-(h,-

1-

2f

after substitution is

x+ p)'f2p'} + le, -dx {e-(b,- x- p)'/Zp'} r

h,

+ fh'dx{e-(h,-x-t;p)'/2p'} + c,

2f

5.51 (8)

r

2f

r

f"'dx{1- e-(h,-x-p)'/2p'} Jh, 2f r

5.52(1)

As before each integral reduces to the form p l(b-y,+p)/p

I=-

2fr

(h- y,

+ p)/p

e-z'/2

5.52(2)

where y 1 is the lower limit and y 2 is the upper limit in the appropriate term in 5.52( 1). Thus CR

=[

b 1

2f

C

1

J

l(b,-c, +p)fp

-

(b,- h,

e-z'/2

dz

+ p)/p

~

1

-1 ~

p l(h,- h,- p)/p -z'/2 d +e z 2/r (h,- c,- p)fp p l(b,-c,+p)/p 2 +e-z 12 dz

2fr

(h,- h,

+ p)/p

~

1

+ 5.53

[Cz-2f bz] -2fr-p

-1 ~

l(b,-h,-p)/p (h,- c,- p)/p

e

-z'/2d

z

5.52(3)

Proceeding as before the components of CR can now be written as if -

f +a

CR 1

c 1 then

~

= (b 1

-

cd/2!

f + a > c 1 and - f + a CR1 = (b 1 + f- a)/2! if- f + a > b 1 then

if -

CR1

~

b 1 then

=0

CRz

= pJ( 2

4fr

n)[

P

P

-k 2 to k2

-k 1 to k 1

J

5.53( 1)

5.53(2)

Uncertainty, Calibration and Probability

198

where if- f

+a~

and if - f + a > and if- f +a> and in this case.

c1 k1 c1 k1 b1 k1

then k 2 = (b 1 - c 1 + p)jp (b 1 - b1 + p )/ p = 1 and - f + a ~ b 1 then k 2 = ( b1 + f - a + p )/ p = (b 1 - b1 + p)jp = 1 then k 2 = (b 1 +f-a+ p)jp = (b 1 +f-a+ p)jp, i.e. limits are equal and so CRz

=

=

0

5.53(3)

where if-f+a ~c 1 thenk 2 =(b 1 -b 1 -p)/p= -1 and k 1 = (b 1 - c 2 - p)jp if- f +a> c 1 and --!+a~ b 1 and f +a> c2 then k 2 = (b 1 - b1 - p)jp = --1 and k 1 = (b 1 - c2 - p)jp if - f +a> c 1 and - f +a~ b1 and f +a~ c2 then k 2 = (b 1 - b 1 - p )/ p = -1 and k1 = (b 1 - f - a - p )/ p if- f +a> b1 and f +a~ c 2 then k 2 = (b 1 +f-a- p)jp and k1 = (b 1 - f - a - p )/ p if- f +a> b 1 and f +a> c2 then k 2 = (b 1 +f-a- p)jp and k 1 = (b 1 - c 2 - p)jp CR 4 = p_j( 2

4fr

where if- f

nl[

P

P

-k 2 tok 2

-k 1 tok 1

J

5.53(4)

c 1 then k 2 = (b 2 - c 1 + p)jp and k1 = (b 2 - b 2 + p )/ p = 1 if- f +a> c 1 and - f +a~ b 1 and f +a> b1 then k 2 = (b 1 +f-a+ p )/ p and k1 = (b 1 - b2 + p )/ p = 1 if - f +a> b 1 and f +a~ b2 then k 2 = (b 2 +f-a+ p )/ p and k 1 = (b 2 - b2 + p)jp = 1 if - f +a> b1 and f + a < b2 then k 1 = (b 2 +f-a+ p)jp and k 1 = (b 2 - f-a+ p)jp +a~

If f

+ a > c2 then

CR 5 = (c 2

iff

+a~

-

b 2 )/2f

b2 andf

CR 5 = (f

+a~

c2 then

5.53(5)

+ a- b1 )/2f

iff +a< b2 then CR5 = 0 CR6 = pJ( 2

4fr

n)[

P -k 2 to k 2

-k~o k

J 1

5.53( 6)

Applications

199

where if f

+a>

c 2 then k 2 = ( b 2 - b 2 - p )/ p = -1 and k 1 = (b 2 - c 2 - p)/p if f + a ~ b2 and f + a ::::; c2 then k 2 = (b 2 - b 2 - p)jp = -1 and k 1 = (b 2 - f - a - p)/p if f +a< b2 then k 2 = (b 2 - f - a - p )/ p and k 1 = ( b1 - f - a - p )/ p, i.e. in this case of equal limits CR 6 = 0. Finally eR = cRl - cRz

_I

+

cR3

+

cR4

+

5.53(7)

cRs - cR6

The required expression for 0 A after substitution is

5.54

OA-

oo

dx { e-(b,- x- pJ'/2p'

-

~ ~

r

e-(b,- x- pJ'/2p'}

------

5.54( 1)

r

As before each integral can be transformed to the form p i(b- y, + p)/ p

I=2fr

2

e-z fl

dz

5.54(2)

(b-y,+ pJ/p

If 5.54( 1) is now transferred as above, we have that p OA=2fr

i(b2-c,-p)jp

2

e-z/ 2

(b,-f-a-p)/p

p dz--2fr

i(b,-c,-p)jp

2

e-zl 2

dz

(b,-f-a-p)/p

The components of 0 A can now be written as

5.55

OAt= pJ( 2n) [

4fr

p

p

-k 2 to k 2

-k 1 to k 1

J

5.55( 1)

where if f

+ a ~ c2

then k 1 = ( b1 - c 2 - p )/ p and k 1 = (b 2 - f - a - p)/p if f +a< c 2 then k 2 = (b 2 - f - a - p)/p and k 1 = ( b2 - f - a - p )/ p, i.e. in this case of equallimits 0 At = 0. OA 2

=

n)[

pJ( 2

4fr

P -k 2 tok 2

-

P -k 1 tok 1

J

5.55(2)

where if f

+ a ~ c2

then k 2 = and k 1 = (b 1 if f +a< c 2 then k 2 = andk 1 = (b 1 Finally

( b1 - c 2 - p )/ p f - a - p)/p (b 1 - f - a - p)jp f - a - p)/p,i.e.inthiscaseofequallimits0A 2 = 0. 5.55(3)

Uncertainty, Calibration and Probability

200

5.56 The expression for U A after substitution of the rectangular distribution is given by

5.56(1) Transforming each integral to the form p 1=2fr

l(b-y,+p)/(1 (b- Yz

2

e-zl 2 dz

5.56(2)

+ p)/p

as before we have

5.56(3) The components of U A can now be written as

5.57

UAl

= p.j( 2n)[ P -k to k 4fr 2

-k~o

2

k1

J

5.57(1)

where if - f if -

f

c 1 then k 2 = (b 1 +f-a+ p )/ p and k 1 = (b 1 - c1 + p)/p + a > c 1 then k 2 = (b 1 + f - a + p) / p and k 1 = (b 1 + f - a + p )/ p, i.e. in this case of equal limits U Al = 0. +a~

UA2

= p.j( 2 4f1;

n)[

P -k 2 to k 2

-

P -k 1 to k 1

J

5.57(2)

where c 1 then k 2 = (b 2 +f-a+ p)/p and k 1 = (b 2 - C 1 + p)/p if - f +a> c 1 then k 2 = (b 2 +f-a+ p)/p and k 1 = (b 2 +f-a+ p)/p, i.e. in this case UA 2 Finally if- f

+a~

=

0. 5.57(3)

5.58 The expression for OR after substitutiong of the rectangular distribution is given by

OR=

I

CY)

dx {e-(b,-x-p) 2 /2p 2 -

~ ~

r

e-(b,-x-p) 2 /2p'}

+ 1 ·---r

5.58(1)

201

Applications Transforming each integral, as before, to the form

p I=-

f(b-yl+p)/p

2fr

2

e-z 12 dz

5.58(2)

(b-y,+p)/p

we have

OR=

1 -f-a-p)/p

-

f

-

dz +[!+a- Cz]

2J

(b,- c,- p)/ p

(b 2

5.59

e-z'/2

r(bl-c,-p)/p

J(b

e-z'/l

dz

5.58(3)

f-a- p)/ p

The components of OR are given as

ORl = p,j(2n)[ p 4/r -1< tok 2

2

p -k 1 tok 1

J

5.59( 1)

where if f +a> c 2 then k 2 = (b 1 - c 2 - p )/ p and k 1 = ( b1 - f - a - P) I P if f +a~ c 2 then k2 = (b 1 - f - a - p)/p and k 1 = (b 1 - f - a - p )/ p, i.e. in this case of equal limits OR 1 = 0.

If f

+a>

OR 2 = (.f if f

c 2 then

+a-

c 2 )/2f

5.59(2)

+ a ~ c 2 then

OR2 = 0

n)[

OR3 = p,j( 2 4Jr

P

P

-k 2 tok 2

-k 1 tok 1

J

5.59(3)

where if f +a> c 2 then k 2 = (b 2 - c 2 - p)/p and k 1 = (b 2 - f-a- p )/ p if f +a~ c 2 then k 2 = (b 2 - f - a - p )/ p and k 1 = ( b 2 - f - a - p )/ p, i.e. in this case of equallimits OR 3 = 0. Finally 5.59( 4)

5.60 Lastly the expression for U R after substitution of the rectangular distribution is given by

UR

I

Cl

=

-OC;

dx {

-

2f

1-

e -(bl -X+ p) 2 /2p 2 r

e -(b,- X+ p) 2 /2p 2 }

+ ----r

5.60( 1)

Uncertainty, Calibration and Probability

202

Transforming each integral, as before, to the form p

I=2/r

i(b-y,+p)fp

2

e-z 12 dz

5.60(2)

+ p)/p

(h _ y 2

we have UR= [

c+f-a]

-

2/ p

+2/r

P ---

i(b,+f-a+p)fp

2/r

i(b2 +f-a+ p)/ p

2

e--z/2

(b,-c,+p)fp 2

e-z /l

dz

5.60(3)

(b2-c,+p)fp

The components of U R are given as

5.61

f + a ~ c 1 then URl = (c 1 + f - a)l2f if - f + a > c 1 then if -

URl

5.61(1)

=0

n)[

URz = p)( 2

4j1:

P

-k~ok 1

-k 2 tok 2

J

5.61 (2)

where if- f if -

f

c 1 then k 2 = (b 1 +.f-a+ p)lp and k 1 = (b 1 - c 1 + p)lp + a > c 1 then k 2 = (b1 + .f - a + p) I p and k 1 = (b 1 + f - a + p )I p, i.e. in this case of equal limits U Rz = 0. +a~

UR3

= p)( 2

4fr

n)[

P -k 2 to k 2

-

P -k 1 to k 1

J

5.61(3)

where c 1 then k 2 = (b 2 +f-a+ p)lp and k 1 = (b 2 - C 1 + p)IP if - .f + a > c 1 then k 2 = (b2 + f - a + p) I p and k 1 = (b 2 + f - a + p )I p, i.e. in this case of equal limits U R 3 = 0. Finally if- f

+a~

5.61(4)

The formulae of application two have been used to prepare a set of tables so that the effect of varying the standard deviation of the manufactured parts aP and that of the measuring machine am, together with the offset of the mean a of the manufactured parts and the inset of the tolerance limits

5.62

203

Applications

± TP/2 by i =km am can be seen. The table concerned is numbered XXII, dealing with the Gaussian distribution of manufactured parts. Worked Example The following worked example will show how Table XXII may be used. A manufacturer is required to make parts which have a diameter which must be maintained within a tolerance range of -0.0003 to +0.0003 in for a given size. He has manufacturing equipment which can produce parts such that their standard deviation from the mean varies by ± 0.000 15 in. His measuring equipment has a standard deviation of 0.000030 in and the manufacturing equipment can produce parts such that the mean of the distribution of the manufactured parts lies within 0.0001 in of the required value for the parts. It is required to find the measurement conditions necessary for the proportion of incorrect parts accepted not to exceed one per cent of the total parts made and for the proportion of correct parts rejected to be kept to a minimum. 5.64 Firstly we convert all the required variables to a percentage of the total tolerance range, i.e. 0.0006 in. 'a' the offset of the mean as a percentage is 0.0001/0.0006 X 100%, i.e. 16.67%, (JP= 0.00015/0.0006 X 100 = 25% and am= 0.00003/0.0006 x 100 = 5.0%. kP the Gaussian tolerance factor for the manufactured parts is thus given by 2kPa P = TP, i.e. 2kP x 25 = 100 or kP = 2. Consulting Table XXII we look at the section having a= 15, a P = 25 and a m = 5. There are three lines which meet these conditions, namely with km = 0, 1 and 2. Because of the offset of the mean of the manufactured parts far more oversize parts are made than undersize parts. (See Figure 5.64.)

5.63

Correct parts Shaded areas show wrong parts

-2ap

-rip

0

0. 67cr

ap

2ap

Area under graph gives percentage of parts

Figure 5.64

Area under graph gives percentage of parts

Uncertainty, Calibration and Probability

204

Looking up the three lines mentioned in the tables, we find on tabulating the results, the following values a

km

sm

kp

sP

c

CA

eR

u

UA

UR

5 5 5

2 2 2

25 25 25

91.46 91.46 91.46

89.90 86.87 82.08

1.56 4.59 9.38

0.47 0.47 0.47

0.08 0.02 0.00

0.39 0.45 0.46

15 15 15

0 5 10

0 1 2

0

OA

OR

w

WA

WR

8.08 8.08 8.08

0.99 0.22 0.03

7.08 7.86 8.05

8.54 8.54 8.54

1.07 0.24 0.03

7.47 8.3 8.51

We now look at the tables again, but with 'a' changed to 20, and with i, km, sm, kP and sP the same. This gives the following results a

km

Sm

kp

Sp

c

CA

eR

u

UA

UR

5 5 5

2 2 2

25 25 25

88.24 88.24 88.24

86.37 82.82 77.40

1.86 5.42 10.84

0.26 0.26 0.26

0.05 0.01 0.00

0.21 0.25 0.25

20 20 20

0 5 10

0 1 2

0

OA

OR

w

WA

WR

11.51 11.51 11.51

1.3

0.29 0.03

10.19 11.22 11.47

11.76 11.76 11.76

1.36 0.30 0.04

10.40 11.46 11.73

We now interpolate (linear interpolation will be sufficient) for 'a' = 16.67 between corresponding lines of the three groups of three lines, giving a

km

Sm

kp

sP

c

CA

eR

u

UA

UR

5 5 5

2 2 2

25 25 25

90.38 90.38 90.38

88.72 88.52 80.52

1.66 4.87 9.87

0.40 0.40 0.40

0.07 0.02 0.00

0.33 0.38 0.39

16.67 16.67 16.67

0 5 10

0 1 2

0

OA

OR

w

WA

WR

9.23 9.23 9.23

1.10 0.24 0.03

8.12 8.98 9.19

9.62 9.62 9.62

1.17 0.26 0.03

8.45 9.36 9.55

5.65 The parameters in the last three lines are now the same as those of the problem, i.e. a= 16.67%, sm = 5.0% and sP = 25% with kP = 2. Looking at the results we see that WA = 1.17 is just larger than the required 1 %, so we interpolate between lines 1 and 2 to find the value of km which will give 1 %.

Applications

205

The final set of results is as follows: a

km

Sm

kp

sP

c

CA

eR

u

UA

UR

5

2

25

90.38

88.12

2.27

0.40

0.06

0.34

16.67

0.95

0.19

0

OA

OR

w

WA

WR

9.23

0.94

8.28

9.62

1.00

8.62

We see that the number of correct parts rejected is 2.27%. If WA is reduced below 1 %, then the percentage of correct parts rejected rises rapidly. It is interesting to note that our criterion of 1 % of wrong parts accepted has been attained with an inset of only 0.19 x 5 = 0.95% of the tolerance range.

Interpolation Procedure If the values of kv, sP, sm and a do not occur in the tables, the required result can be obtained by means of interpolation. kv will of course be determined by the value of sP and so will not be part of the interpolation. This leaves three variables sP, sm and a. Suppose that sP were 17.8 %, sm were 6.2% and 'a' were 11.6% and it is required that the percentage of wrong parts accepted should not be more than 1 %. Firstly we look for two lines where tabulated values of sP straddle the value of sP required and where the values of sm and a are the nearest to the required values both either above or below these values and where the value of km is the lowest consistent with satisfying the WA requirement. Since the WA required is 1 % we can choose the two lines 5.66

a

km

Sm

kp

sP

eR

WA

10

0

0

5

3

16.67

0.45

0.18

5.66( 1)

10

0

0

5

2.5

20.00

0.81

0.42

5.66(2)

We also choose the two lines 10

5

5

3

16.67

1.5

0.04

5.66(3)

10

5

5

2.5

20.00

2.5

0.10

5.66( 4)

From lines 5.66( 1) and 5.66( 2) we get 10

0

0

5

2.81

17.8

5.66(5)

and from lines 5.66(3) and 5.66(4) we get 10

5

5

2.81

17.8

5.66( 6)

206

Uncertainty, Calibration and Probability

Now choose pairs of lines where sm > 6.2, i.e. 7.5 say, and we get a

km

Sm

kp

sv

eR

WA

10

0

0

7.5

3

16.67

5.66(7)

10

0

0

7.5

2.5

20.00

5.66(8)

10

7.5

7.5

3

16.67

5.66(9)

10

7.5

7.5

2.5

20.00

5.66(10)

2.81

17.8

5.66( 11)

17.8

5.66( 12)

and

and

and

Lines 5.66(7) and 5.66(8) yield 10

0

0

7.5

and lines 5.66(9) and 5.66(10) give 10

7.5

7.5

2.81

From 5.66( 5) and 5.66( 11) interpolating between 5 and 7.5 for sm 10

0

0

6.2

2.81

17.8

=

6.2 we get 5.66(13)

Interpolating between 5.66( 6) and 5.66(12) for sm again equal to 6.2 we get 10

6.2

6.2

2.81

17.8

5.66(14)

We now repeat the above process for a= 15 with the other variables the same. This will yield 15

0

15

6.2

0

6.2

2.81

17.8

5.66(15)

6.2

2.81

17.8

5.66(16)

Interpolating now to obtain a= 11.6% we get using 5.66(13) and 5.66(15) 11.6

0

0

6.2

2.81

17.8

5.66( 17)

and from 5.66(14) and 5.66(16) we get 11.6

6.2

6.2

2.81

17.8

5.66(18)

207

Applications

We can now interpolate between 5.66( 17) and 5.66( 18) to obtain the required value for WA using km as the variable. Other problems may need the pairing oflines containing km = 1 and 2, 2 and 3, or intermediate values as required.

Examples on Chapter 5 Parts are manufactured on a machine, and these are acceptable provided they lie within the range ± T 12 on either side of the mean. The standard deviation of the manufactured parts is equal to a P T I 6. If the standard deviation of the equipment used to measure the manufactured parts is a m T I 10, find the percentage WA of wrong parts accepted and the percentage CR of correct parts wrongly rejected where the acceptance limits are taken rrm inside the tolerance limits ± T 12 on either side, and the mean of the manufactured parts is in the middle of the tolerance range. Examples 5.1

=

=

= 0.023%, eR=

Answer: WA

3.768%.

=

=

In the previous problem, if rrP T14 and rrm Tl20 and the acceptance limits are set in from the tolerance limits by 2rr m T I 10, find WA and CR. Answer: WA = 0.018%; eR= 7.134%.

Example 5.2

=

=

Tl6 and rrm = Tl20 and the acceptance limits are the If rrP same as the tolerance limits, find WA and CR. The mean of the manufactured parts is in the middle of the tolerance range. Answer: WA = 0.064%; eR= 0.202%.

Example 5.3

Example 5.4

Show that approximately 1 (e"',-1/1,- e"',-1/1, WA = rrrPJ(2n)

+ e"',-1/1,- e"'·-.P•)

where cp 1 = [B 2 I4A- CJoAl• cp 3 = [B 2 I4A- CJuA 1 ,

cp 2 = [B 2 I4A- C]oAz cp 4 = [B 2 I4A- C]uAz

~ 1 = [ { )(2A)( c

2 -

2: ) LAl

+p

~ 2 = [ { )(2A)(

C2 -

2: ) } UAZ

+p

C1 -

2:)} UAl- p

~ 3 = [ { )(2A)(

~ 4 = [ { J(2A)( c

1 -

2: ) LA 2

+

J J

/2p 2 /2p 2

J pJ/2p

/2p 2 2

and where the A, B, and C terms are those given in paragraphs 5.37 and 5.38.

208

Uncertainty, Calibration and Probability

Example 5.5 A firm is asked to make parts within ± 0.0002 in of a nominal size. It can produce parts with a distribution whose standard deviation is 0.0001 in. The centre of this distribution can be maintained within ±0.0001 in of the nominal value. The measuring equipment has a standard deviation of 0.000 05 in. Find the percentage of correct parts rejected (as a percentage of the total parts made) if the percentage of wrong parts accepted must not exceed 1 %. State also the required inset i to produce the required result, i.e. km, i as a percentage of TP and i in inches, and also the percentage of correct parts produced. Answer: eR = 17.9 %,

km

= 0.92, i = 11.5 %, i = 0.000 046 in,

c=

84%.

Example 5.6

A manufacturer is asked to produce parts within a tolerance TP/2 of ± 0.0002 in of a nominal size. He is told that he must not produce more than 5% scrap because of the value of the work already done on the parts. Further, not more than 1% of incorrect parts may be accepted. If the firm can produce parts with a standard deviation of 16.67% of the total tolerance TP with a mean lying within ± 15% of the centre of TP find the maximum value of the standard deviation of the measuring equipment that will meet the specification, together with the required inset and the percentage of correct parts rejected. (Jm = 16.2%, km= 0.54, i = -8.37%, i.e. (Jm = 0.000065 in, i = -0.000035 in, WA = 1.00%, (CR + WR) = 5.00%, eR= 4.26%.

Answer:

Note: that i the inset is negative, i.e. is an offset, and also that most of the 5% scrap is made up of correct parts rejected, i.e. 4.26%. This value could of course be reduced by making (J m smaller. Example 5.7

A customer requires parts to be made to lie between the tolerance limits ±0.0001 in of a nominal size. He specifies that no incorrect parts will be accepted. The manufacturer has production equipment which can produce parts with a spread of ± 3(J = ± 0.0001 in and with the mean of this spread lying within 0.000 03 in of the centre of the tolerance limits. His measuring equipment has a standard deviation of 0.00001 in. Find the necessary inset of the measuring limits in order to achieve the necessary 'no incorrect parts to be accepted' proviso. Find also the percentage of incorrect parts produced W, the percentage of correct parts rejected CR, and thus the total of parts rejected. Answer: Inset= 15% == 0.00003 in. Percentage of incorrect parts produced=

1.79. Percentage of correct parts rejected= 10.93. Total percentage of parts rejected = 12. 72. Note: the high percentage of correct parts rejected. This is mainly because of the non-centrality of the mean of the manufactured parts with the centre of the tolerance limits. Some improvement can of course be attained by reducing (Jm, the standard deviation of the measuring equipment.

6 Distributions Ancillary to the Gaussian

6.01 In our consideration of frequency or probability distributions hitherto, we have assumed that the constants of these distributions, that is the standard deviation (J and the mean value Jl, have been known exactly. In practice these constants are derived from a finite number of readings, and so their values will have a degree of uncertainty associated with them which will diminish with the increase in the number of readings taken. In this chapter we shall deal briefly with methods and ancillary distributions which allow the uncertainty in the basic constants to be allowed for.

The Student or t Distribution 1 6.02 The first of these special distributions is known as the Student (pseudonym for W S Gosset) or t distribution. It will be recalled that Table II of Appendix I gives values of

-/(~n)f~k~e-c'l2dc= -k~okl

6.02(1)

where X-Jl

C=--

and

6.02(2)

(J

Thus

P -k 1 to k 1

gives the probability for x lying between the limits 6.02(3)

where (J is the standard deviation of the distribution and 11 is the mean value. 209

210

Uncertainty, Calibration and Probability

Let us replace (J in 6.02(2) by the estimates. of the standard deviation, derived from n readings and given by

6.03

s.

=

j{f 1

.X)z}

(x,n- 1

2.24(9)

where v = the number of degrees of freedom. In the usual case, where a single variable is considered, v = n- 1. Thus 6.02(2) becomes 6.03( 1) on making the substitution stated above, and replacing c by t •. 6.04 If we now wish to calculate the probability of - k 2 ::::; t ::::; k 2 or its alternative expression, the probability of 6.04( 1) then the required probability is no longer expressed by 6.02( 1), but by the integral of the Student distrubution, that is

f k,

fk, ( 1 + -t;)-·(v+l)/2 dtv - r((v + 1)/2) /

f(tv) dtv -

-k,

r(v/2)'-" (vn)

v

-k,

6.04(2)

where r( a) is the gamma function. When a is even r(a/2) = (a/2- 1)(a/2- 2) ... 3.2.1 valid for a ?c 4. If a When a is odd r(a/2)

= 2,

6.04(3)

r( 1) = 1.

= (a/2- 1)(a/2- 2) ... 5/2. 3/2.1/2.)n

6.04(4)

valid for a ?c 3. If a = 1, r(!) = )n. Figure 6.05 shows a plot of f(t.), for v = n- 1 = 4, with a plot of a normal distribution of equal standard deviation superposed. It is to be noted

6.05

0·4 I " ' ' \ \;---Normal distribution Student distribution

-er

0

(]'

2 F 0 A property of the F distribution which is sometimes useful interchanging v1 and v2 , we have the relationship

IS

that on 6.20( 1)

where p in the left-hand term is the probability ofF being either larger or smaller than F 1 , whilst 1 - p in the right-hand term is the probability ofF being either larger or smaller than F 2 • In the first term 2

2/

2

2/

V1 F 1-~-X1 s~ xUv2

whilst in the second term

F2 = s2 = X2 v2

si xi/v1

Paragraph 9.10 gives the relationship between the F distribution and the t distribution. The use of the F function will be described in Chapter 9.

Tolerance Limits, Factors, Probabilities, and Confidence Limits, Factors and Probabilities We have dealt with this subject to a limited degree in Chapter 2, paragraphs 2.35 to 2.53, but in doing so we assumed that the constants of

6.21

t See Appendix II for proof.

Distributions Ancillary to the Gaussian

217

the distributions considered, that is the standard deviation a and the mean f.l, were known. In many if not most practical cases, a and f.1 are derived from

a fairly limited number of observations, usually of the order of ten or twenty, and so the values of a and f.1 obtained are themselves random variables. 6.22 Before proceeding with how to deal with this dilemma we shall define some terms. la

Tolerance Limits or Interval

This is a range or interval defined by limit points between which a given fraction or percentage of observations or readings is expected to lie.

1b

Tolerance Factor

This is a quality which when multiplied by the estimated standard deviation or standard deviation gives the semi-range over which a fraction of observations (readings) is expected to occur. 1c

Tolerance Probability

f3tp

A tolerance probability is the fraction or percentage of observations expected to be within a tolerance range or interval. 2a

Confidence Limits or Interval

This is the range or interval defined by limit points between which the mean value f.1 of a quantity is expected to lie with a probability expressed

by the associated confidence probability. 2b

Confidence Factor

This is a quantity which when multiplied by the estimated standard deviation or standard deviation gives the semi-range over which the quantity is expected to occur with a given probability. 2c

Confidence Probability f3cv

This is the probability associated with a given confidence range, when the range is associated with the occurrence of the mean value of a quantity. Alternatively, if a confidence probability is associated with the occurrence of a single reading or observation, then it is defined as follows. If a large number of groups of readings have been taken, each containing the same number of readings, then the confidence probability f3cv is equal to the proportion of groups in which at least a certain proportion of readings (tolerance probability) lies between specified limits (tolerance limits). In practice the confidence probability to be associated with a single group or sample can be found without taking any further readings (see paragraphs

218

Uncertainty, Calibration and Probability

6.31 to 6.32). A confidence probability and a tolerance probability taken together thus deal with the occurrence of a single reading, whilst a confidence probability by itself deals with the occurrence of a mean value. As the number of observations n tends to infinity, a tolerance limit tends to a constant, whilst under the same circumstances a confidence limit tends to zero.

The Uncertainty of a Single Reading, and the Probability of it Occurring over a Given Range: Without Confidence Probabilities. t Known Parameters p and n 6.23 This case is one that we have already dealt with in paragraphs 2.35 to 2.46, but we will restate very briefly how the probability of an error lying in a given range is found or alternatively how to find the range associated with a given probability. Given the range of uncertainty x 1 to x 2 , to find the probability of occurrence of an uncertainty in this range

6.24 Calculate IJ.l- x 1 1and IJ.l- x 2 l, where the straight lines round each difference mean the modulus or positive value of the difference, and divide each modulus by rJ the standard deviation. Let

ifl-X1i_ k (J

-

1

1

6.24( 1)

1

6.24(2)

and

lfl-X21_ k (J

-

2

Consult Table 11, Appendix I, and look up the probability associated with each of the values of k; let these be P 1 and P 2 respectively. There are four possible answers, depending on the signs of J1- x 1 and J1- x 2 , thus: (Jt- x 1 ) and (Jt- x 2 ) both positive

Take the larger probability, subtract the smaller and divide by two, then this is the required probability, that is .. . P2- P1 probability reqmred = P = - - 2

t This means that in fact the confidence probability is 0.50 for this case.

Distributions Ancillary to the Gaussian

219

2 (f.l- xd negative, (f.l- x 2 ) positive Add P 1 to P 2 and divide by two, that is

P=P1+P2 2

3 (/-l - xd and (f.l - x 2 ) both negative Subtract smaller probability from larger and divide by two, that is p2- p1

P=--2 4 Usual case, when (f.l- xd = -(f.l- x 2 ), that is f.1 = (x 1 + x 2 )/2 or as it is more usually written, the probability of an uncertainty in the range f.1 - J to f.1 + J Divide b by a and look up this value under k 1 of Table 11. The corresponding probability gives the required probability for the range. Given a probability for a range, to find the associated range

6.25 We will deal here only with ranges symmetrically distributed about the mean. Suppose the probability of a range is given asP, then Table Ill of Appendix I, should be consulted and the appropriate probability located from the columns carrying the integral sign. Let k 1 be the corresponding value from the k column. Then the range required is from f.1 - k 1 a to f.1 + k 1 a.

Uncertainty of a Single Reading: Without Confidence Probabilities. Known Parameters x and u 6.26 This is a case that occurs fairly frequently. For instance, if an instrument such as a measuring microscope is used to make repeat readings of the position of a line, then a very large number of readings can be made and a reasonably accurate figure found for a, the standard deviation of the spread of readings. If the microscope is now used to locate the position of a workpiece from a comparatively small number of readings, then the range of a single uncertainty for a chosen probability can be found as follows. Calculation of the range of a single reading for a selected probability er known

6.27 Let n readings be taken and the mean i found, then the tolerance range for a given tolerance probability for a single reading is given by

Uncertainty, Calibration and Probability

220

consulting Table X of Appendix I and choosing the value of k4 from the appropriate probability column and from the row corresponding to n the number of readings. The required range for x is then given by .X- k 4 (J to .X + k 4 (J. It is to be noticed that if the estimate of (J had been found from n readings, then the range would have been much larger (see known parameters .X and s, paragraph 6.29). The value of the coefficient k4 of (J is given by

where

c~

is the value of c which satisfies the equation

f"'

1 2 .j(2n) c, e-c fldc =a

Alternatively 1- 2a

= -1- Jc, .j(2n)

e-c fl de 2

-c,

(see Table 11), and thus 6.27( 1)

Derivation of k 4 If .X were the true mean then the uncertainty in x would be given by but .X itself is a variable with a standard deviation of (J / and so the uncertainty in x will be obtained by combining the distribution for x with that of .X, to give a distribution centred on .X, where the latter is the mean of x found from n readings. The standard deviation of the combination is given by 6.28

Jn,

k(J,

(J

( +-1) 1

1/2

n

(see 3.41 ( 1)) and since the distribution of x and .X are normal, so is that of the combination, whose variable we will call z. Thus z may be looked on as the total uncertainty due to the variation of .X and x. An uncertainty of given probability is thus given by 1 )1/2 k1 (J ( 1 + ~ '

giving

6.28(1)

221

Distributions Ancillary to the Gaussian

Uncertainty of a Single Reading: Without Confidence Probabilities. Known Parameters x and s 6.29 This is a very common case, and occurs when i and s are found from n readings. n

.X= Ix,/n

2.22( 1)

1

and

s; =

Ln ( x,- x-)2 1

n- 1

2.24(8)

where v = n - 1 the number of degrees of freedom of a single variable. Since i and s. are both approximations to p and (J', respectively, obtained from a

limited number of readings, then the possibility of the limiting values of i and s. being different from those found must be allowed for. 6.30 Table XI of Appendix I gives two values of the tolerance probability {31P, that is 0.95 and 0.99, and for either of these probabilities enables the range associated with n readings to be found in terms of i and s•. The range of x is given by to where k 5 is chosen from the appropriate probability column k 5 is given by the expression

/31p and row n. 6.30( 1)

where kn _ 1 , 2 a is derived from the equation

Ptp =

1 - 2cx =

f~~~·2, f( t.) dt.

6.30(2)

(see Table IV). As t •. 2 a = k2 ; v = n- 1, thus 1 k 5 = k2 ( 1 + ~

)1/2

6.30(3)

Derivation of k 5

An indication of how k 5 is obtained is as follows. Both x and i are obtained from n values of x and thus both have a t. distribution. As in the previous case, since x and i are both variables, in order to find the total uncertainty, we combine the two distributions, which together have a combined estimated standard deviation of s.( 1 + 1/n) 1' 2 • If we call the variable ofthe combination

222

Uncertainty, Calibration and Probability

z, then we can write Z-jl

-------,--,-:- = t

Sv(1

+ 1/n)112

6.30(4)

v

Whence it follows that the uncertainty of z is given by 1 )1/2

x- k 2 ( 1 + ~

( 1)

and thus

x + k2

to

sv

k5 = k2 1 + ~

( + 1) 1

~

1/2

1/2

6.30(3)

Uncertainty of a Single Reading: Without Confidence Probabilities. Known Parameters Jl and s 6.31 We have already mentioned this case, as it represents the Student 't' distribution; sv is found from 2.24(8), that is

s;

=I (x,1

x)z

n- 1

2.24(8)

The range for a selected probability is obtained as Table IV, Appendix I, gives k 2 for four probabilities.

where (X=

f"' J(tv)dtv t,

and See 6.04(2) for f( tv).

Uncertainty of a Single Reading: With Confidence Probabilities. Known Parameters x and a Let x be derived from n readings and let (J be known. Now x is not the limiting mean and is itself a variable. Since (J is known, the standard

6.32

Distributions Ancillary to the Gaussian

223

deviation of x is aI Jn. Let us decide that the tolerance probability required is f3 1P and since xis a variable, the confidence probability will be proportional to the position of x. Let us choose a confidence probability for x, say f3cp• then f3cp =

Ik,

1 f(c) de=-;-k, v (2n)

Ik,

,

e-c 12 de

6.32( 1)

-k,

Since the standard deviation of x is a I J n the range of x is 6.32(2) where x0 is the value of the mean from n readings. Consider Figure 6.32. The normal distribution curve shown has a standard deviation of unity, and its mean is displaced by k 1 / Jn from the zero position. This value is obtained by putting a = 1 in 6.32(2). The value of k 6 , such that the probability of an error is equal to /31P, is given by f3t = -1P

J(2n)

Ik.

e-(c-kdJn) 2 /2 de

6.32(3)

-k.

Writing

z

=

kdJn- c

6.32(4)

6.32(3) becomes 6.32(5)

Figure 6.32

224

Uncertainty, Calibration and Probability

6.33 It is at once obvious that as the mean of the curve in Figure 6.32 moves to the left the proportion of readings between the limits - k 6 to k 6 increases until the mean reaches the position 0 and then decreases again until the position - kd -.)non the other side of the ordinate axis is reached, when the proportion is the same as for z = kd -.)n. Thus in a proportion f3cp of sample groups of n readings, each group will contain at least a proportion /31P of readings lying between the limits - k6 to k6 • Table XII of Appendix I gives two values of tolerance probability for each of the two values of the confidence probability given. Having selected a tolerance probability and a confidence probability, k 6 is chosen from the appropriate column and the row corresponding to the number of readings. The required range is then given by x- k 6 a to x + k 6 a.

Uncertainty of a Single Reading: With Confidence Probabilities. Known Parameters i and s 6.34 This is the most usual case, that is when readings. As before

x and s are obtained from n

r~n Xr

-

2.22( 1)

X=£...r= 1

n

and 2 s =

r~n L.

r=l

V

The tolerance probability

(x,- x)2

2.24(8)

n-1

/3 1P is derived from 1

-+r

_1_ -.}(2n)

f.jn e-c'/2 de= {3 .jn- r 1

tp

6.34( 1)

and k7 , the coefficient of the tolerance limit, is given by k 7 = r-.)(n/x~.fJ)

6.34(2)

where n is the number of readings, v = n- 1, and x;,fJ,, is defined by P( X2

>

xzv,{i,, ) -- f3 cp

6.34(3)

where f3cp is the confidence probability, and is the probability of X2 being greater than x;,fJ,,- These formulae for k 6 were developed by A Wald and J W olfowitz. 6.35 Table XIII of Appendix I gives three values of the confidence probability f3cp• 0.90, 0.95, 0.99, and for each of these values gives four values of the tolerance probability {3 1P, that is 0.90, 0.95, 0.99 and 0.999. The tolerance limits are given as x- k 7 s to x + k 7 s.

Distributions Ancillary to the Gaussian

225

Uncertainty of a Single Reading: With Confidence Probability. Known Parameters p. and s The required tolerance limits are obtained by consulting Table XIV of Appendix I and the required tolerance limits are J1- k 8 s to J1 + k 8 s, where s is found from n readings using 2.24(8). Two values of the confidence probability f3cp are given, and for each of these, two values of the tolerance probability {31P are given. 6.37 The coefficient k 8 is derived as follows. The value of z;.(l _ fJ,.J is obtained from tables of chi-square, that is

6.36

Prob(x 2 > x;,(l- /l,,J) where

=

1-

f3cp

6.37(1)

f3cp is the confidence probability. Alternatively Prob(x 2 < x;,fl)

=

f3cp

6.37(2)

Since from 6.16(4) V 2 Xv,(l

6.37(3)

-fl,,)

the maximum value of (J is given by 6.37(4) Now consider Figure 6.37. If the tolerance probability is {31P and we consider the curve of (Jmax shown in the figure, it is at once apparent that k1 is given by 6.37(5)

~k,

(":)

...... ... ';::

c::: ;:

~ ~

N

~ 20

10 3.00 1.04

2.81 0.89

3.16 1.00

1.10

0.76

0.99 3.15

3.28

2.81

3.69

1.36

4.43

2.65

10.16

3.66

2.81

0.20

2.81

0.50

5.60

14.09

0.99

5.52

0.98

13.83

2.89

2.99

3.25

3.83

0.97

2.81

0.94

2.81

0.86

2.81

0.73

2.81

1.00

2.82

1.01

2.84

1.05

2.96

1.39

3.90

2.81

2.81

2.81

2.81

1.00

2.81

1.00

2.81

1.00

2.81

1.00

2.81

1.00

2.81

1.00

2.82

1.01

2.85

1.11

3.13

2.81

2.81

2.81

2.81

1.00

2.81

1.00

2.81

1.00

2.81

1.00

2.81

for each set of measurements. us/(sR/,jq) of the random uncertainty. =IX

is the ratio of the standard deviation of the systematic uncertainties to the standard deviation of the mean

t' is a pseudo value of the tolerance coefficient when the approximation 7.15(1) is used, t,rr is the tolerance coefficient when the Welch approximation for combining 't' distributions is used and 'k' is the Gaussian distribution coefficient. t,rr is the preferred coefficient, q is the number of observations made

p = 0.995

5

3

::t:...

""'Vl"

N

~

s· ....

~

~

;:::

c:: '......."'

~

~

...c

(I)

;:s-

'""-3

e.

(I)

;:::

...

~

C)

246

Uncertainty, Calibration and Probability

Assuming a Gaussian distribution, and taking k = 2.0, giving a minimum probability of 0.954, we have an uncertainty of ± 3.652 units. Adding the semi-ranges leads to an uncertainty of 10 units. Dividing this figure by the standard deviation of 1.826 we obtain k = 5.476, leading to a probability of unity. As most commercial laboratories doing calibration work use a tolerance probability of 0.95, we see that adding the systematic uncertainties leads to a result approximately three times the value given for the accepted 0.95 probability. The other model of assessing systematic uncertainties, which is sometimes used, sums the uncertainties as [L:;::~ a;,] 1 12 . Comparing this with the method described here we see that the ratio of the two methods gives the value R

= [J~

a;,

J1k[:~~ T 12

a;,/3

12

= J3jk

If k = 2 this gives R = 0.866. If k = 3 the result is even worse, i.e. R = 0.577.

Thus this method gives too small a value of the uncertainty and like the adding method of paragraph 7.19 gives no accompanying probability. 7.21 One final point on the suggested method, the only time that assuming a Gaussian distribution will lead to a significant difference between the probability for the convoluted systematics and an assumed Gaussian distribution is when one of the systematic uncertainties has a range much larger than the others. The best way to deal with this is to calculate the total standard deviation comprising the random component and all the systematics excluding the large one, i.e. [

s~ + m:t-11 a;,/3 J/2

If a. is the large systematic, then form the ratio

[

s~ + mJ~ 1a;,/3

J1 12

a./ J3

=11

If the value of the above ratio is less than about 0.8, then the true probability

of the combined systematics and random component is likely to be at least 1% higher than the 95.4% for k = 2. For k = 3 the difference falls to about 0.18 %. If the ratio is less than 0.5, the corresponding percentages are about 2.2 and 0.26. 7.22 In order to deal with this special case, Table XXIII has been produced, which gives values of k for given values of the probability against the ratio 17 and also values of the probability P for given values of k against the ratio. 1J is defined as the ratio of a a! a R where a a = Gaussian standard deviation and aR = standard deviation of a rectangular distribution. The Table gives the method for dealing with the case of a large rectangular distribution to be combined with a Gaussian distribution or a Gaussian distribution and smaller rectangular distributions.

A General Theory of Uncertainty

247

Conclusion Overall the method for the combination of all the uncertainties is simple and has the merit of giving a minimum uncertainty to be associated with a given tolerance range. The assumption of a rectangular distribution for a given semi-range also maximizes the associated standard deviation. The method also has the merit of associating a probability with a given tolerance range. 7.23

8 The Estimation of Calibration Uncertainties

8.01

In this chapter we shall deal with the application of the methods for calculating the various types of uncertainties which have been discussed in previous chapters. Before this is done, however, it is necessary to consider functional relationships, and the magnitude of the uncertainties that they produce.

Standard Deviation of a Function of n Variables 8.02

If F is a function of x 1 , x 2 , ... , xn, and in a particular measurement situation, suppose we are interested in the value of F for which the values of the independent variables are ;X 1 , ;X 2 , ;X 3 , ;X 4 , ... , ;Xn- Thus, providing F is continuous and has continuous derivatives in the region near ;X 1 , ;X 2 , ... , ;Xn, we can expand Fin a Taylor series. Thus 8.02( 1)

If we neglect all terms above the first order, this reduces to

F

aFl

aFl

aFl

axl

ax2

axn

= F 1 + - bx 1 + - bx 2 + ··· + - bxn

8.02(2)

where 8.02(3) and 8.02( 4)

with 248

The Estimation of Calibration Uncertainties

249

and similarly with to We can neglect the terms above the first order because we have assumed that the 8'F ax~

are continuous and finite, and since the c:h; are small, terms involving powers of the c:h; above one with coefficients involving the 8'F ax~

can be neglected. If we write

F- F1 = oF then we can write 8.02(2) as i=n 8F 8.02(5) oF= I ox;axi i=1

8.03 This expression gives the error in F from F 1 caused by a change or error in the variables 1 x 1 , 1 x 2 , ... , 1 x" to 1 x 1 + Ox 1 , 1x 2 + bx 2,... , 1 xn + bx". The bxs can of course be plus or minus. If we now square the expression for bF, we have

c5F2 = i'fbxf(aF)z + pf qf (aF)(aF) oxpbxq 8.03(1) axp axq p#q p=l q=l axi i=l Suppose many measurements are made of bF, each leading to a new set of bx;. We will distinguish a particular set of measurements by adding the suffix r to the variables ox and oF. Thus the rth set of measurements, using 8.03( 1), becomes

(c5F)? =

L L -axp -axq 0xP,bxqr - + p=nq=n(aF)(aF) _L bx;, (aF)z axi

i=n

8.03(2)

p=l q=1

r=l

If t sets of measurements are taken, and the sum of 8.03(2) is found for all

r from I to t we have the sum r==t

L (c5F,)2 r= 1 If we divide this by t we have the square of the standard deviation ofF, that is

(J;.. =

= 'f ;I," oxt(aF)z/t 'f (oF,)z 8x; t

r=1

+

r=1 i=1

(aF) r=t - bxp,bxq,/t I (aF) I q=n I p=n axp p#q axq r=1 p=1 q=1

8.03( 3)

Uncertainty, Calibration and Probability

250 8.04

Now for any p and q the sum

will tend to zero as t gets large, since any product t5xP.t5xqr is as likely to be negative as positive. Thus in the limit when t ~ oo

r=t

ui = L

(oF)2 /t

i=n

It5xf., -

8.04(1)

OX;

r=1 i=1

Now

t5x?lr r=t ~

L...- =

r= 1

t

2

8.04(2)

(Jx;

the variance or standard deviation squared, of the ith variable of x. Thus

0p)2 ui = L u;, -OX; (

i=n

8.04(3)

i=1

If the us are derived from m readings where m is not very large, then the us should be replaced by ss where s is the best estimation of the standard deviation for a small number of readings. Therefore i=n

2- ~ 2 SF- L...sx,

(

i=1

and U 2 =S 2

0

-

p)2

OX;

(m-1) --

m

8.04(4)

2.24(5)

Writing

(oF)= A;

8.04(5)

OX;

a constant, we have i=n

2A2 i

8.04(6)

~ 2A2 2 SF= L..., Sx, i

8.04(7)

2 UF

~ = L..., Ux,

i= 1

or i=n i= 1

251

The Estimation of Calibration Uncertainties

This formula can be derived alternatively as follows:

8.05

F

(oF)

=

F1

i=n + ;~ 1 ox; bx; + (neglected higher terms)

=

F1

+

i=n

I

8.05( 1)

A;bX;

i= 1

Comparing with 3.40( 4) we see that since 8.05( 1) is a linear function of X; in the region of X;= 1 x;, that is eh;= (x;- 1 x;), then the standard deviation ofF, by comparison with 3.40(5), is given by i=n

A2i

8.05(2)

i=n 2 '\' 2 A2 SF= !..... Sx, i i= 1

8.05(3)

2

(J F

= '\' L.,

2

(J Xi

i= 1

or

Standard Deviation of the Mean of a Function of n Variables If each of the variables X; has been measured t times, then from 3.42(3) the standard deviation of i; is equal to

8.06

(J X;

Jt =

-

8.06( 1)

(J-

x,

Now

z/ t

(JF

=

i~n CJ;,Af

L., - -

i= 1

t

8.06(2)

Thus writing 8.06(3) where CJp is defined as the standard error in the mean ofF, we have, using 8.06( 1), that 2 _

(Jp-

i=n '\'

!.....

CJ;c,2 A2i

8.06(4)

s~,Ar

8.06(5)

i= 1

or

s} =

i=n

I

i= 1

252

Uncertainty, Calibration and Probability

Generalized Standard Deviation in the Mean If the standard deviation CJ; for each variable X; is found from a different number of readings mi, then the standard deviation in the mean for each variable is 8.07

8.07(1) Now if we consider the variation in the mean of each variable, that is of xi, then we can write 8.02(5) as

15F =

i=n

oF

i=l

oxi

L ().Xi-

8.07(2)

where bi; =X;- 1 x;

8.07(3)

Now 8.07(2) is a linear function in X; and thus the standard deviation of 8.07(2) is given by i=n

CJ}c

=

2>iAf

8.06(4)

i= 1

where CJpc; is the generalized standard deviation of F when the standard errors for each variable are derived from different numbers of observations m;. When the m;s are all equal, then of course 8.07( 4) or Sp(;

= Sp

8.07(5)

Estimation of Component Uncertainties When an instrument or piece of equipment is calibrated it is important to state the uncertainty of the measurements, and in order to do this, it is necessary to take into account every possible factor likely to cause a variation in the value of the parameter being measured.

8.08

Measurable Uncertainties 8.09 The easiest uncertainties to calculate are those whose distribution constants can be determined from repeated readings. These uncertainties will be designated measurable uncertainties, since the associated constants of the

The Estimation of Calibration Uncertainties

253

distributions involved can be obtained with increasing accuracy as the number of readings is increased. The larger the number of readings the smaller will be the uncertainty in the calculated value of the standard deviation and of the mean. In the great majority of cases the distributions obtained will be of the Gaussian form. An important point to be considered when measurable uncertainties are to be combined with non-measurable uncertainties is to be clear whether it is the uncertainty in the mean, or the uncertainty of a single reading for the measurable uncertainty, that is to be combined. If a measuring instrument is being used to find the magnitude of a parameter or of a workpiece, then it is the uncertainty in the mean that should be used. If, on the other hand, one is expressing the uncertainty to be expected from a single reading, then it is the deviation of a single reading from the mean that should be combined. 8.10 When Gaussian uncertainties are calculated it is necessary to make up one's mind whether confidence probabilities are required. In general the distribution constants are found from a limited number of readings, usually well under a hundred and often under twenty, and so only the estimates for f1 and a are known, that is .X and s respectively. Under these circumstances the calculated tolerance range for a given tolerance probability is likely to be too small, whilst correspondingly the calculated tolerance probability for a selected tolerance range is likely to be too large, when the calculations are made assuming a Gaussian distribution and using Tables 11 and Ill of Appendix I. A better approximation to the correct tolerance probability or tolerance range is obtained by using one or other of the tables allied to the Gaussian distribution.

Guidance on Which Tables to Use The information given under the heading 'Choice of Table' in the section devoted to tables (see Appendix I) sets out in tabular form the range that can be derived from each table, together with the table number and appropriate constant that each table gives. The first column of each set of information gives the parameters that have been found or are known, and it is these in conjunction with column 2 that determine which table should be used. 8.12 Section (i) enables the uncertainty range associated with a given probability to be found (a) without confidence probabilities and (b) with confidence probabilities. Section (ii) enables the probability associated with a given uncertainty range to be found. This section is however restricted to large values of n, the number of readings, although its results are approximately true for smaller values of n. Section (iii) enables the uncertainty range for the mean to be found when the associated probability is given. 8.11

254

V ncertainty, Calibration and Probability

Selection of Tolerance Probability 8.13 It is important to note that whatever value is selected for the total tolerance probability, the same value must be selected for each component uncertainty.

Estimated Uncertainties 8.14 Under this heading is included all uncertainties which cannot be directly measured, and since we have to rely on previous experience or on calculation, we will designate this class of uncertainties as estimated uncertainties.

Rectangular Uncertainties 8.15 We now come to that class of uncertainties which are very difficult to calculate. In many cases all that can be said is that a particular variable can lie between two limit points, usually plus and minus some value on either side of the mean. It is necessary of course to find the functional relationship between the variable in question and the parameter which is the subject of measurement. This should be done when finding the appropriate constant of proportionality between the measured variable and the variable whose variation we are considering. Often there is a known mathematical relationship and, if so, then aF I ax is the required coefficient, where F is the measured parameter and x the variable whose plus and minus range we are considering. If aF I ax is not known, then it should be found by measuring the values of F for two discrete values of x, whence 8F 8x

F1 - F2 - ---

8.15(1)

X1- Xz

8.16 Since nothing is known about the distribution of the type of variables we are considering they should be assumed to be rectangular. Thus, if the assumed range is ±a, about the mean, the standard deviation is, by paragraph 4.06, equal to ± al ..)3.

Maximum Value Only Uncertainties 8.17 There are some cases where the uncertainty distribution for a particular variable consists of just two values, usually equally distributed about the mean. If these two values are ± b, then the standard deviation is given by b (see paragraphs 3.54 to 3.57).

The Estimation of Calibration Uncertainties

255

Gaussian Estimated Uncertainties 8.18 This is a very difficult case to estimate, since it is usually much easier to say that a given variable always lies within some tolerance range than to say that the probability of a variable lying within some stated range is such-and-such a figure. One procedure commonly adopted is to choose for the uncertainty range selected the same tolerance probability as that selected for the total uncertainty. For instance, if the estimated tolerance range is ±j and the tolerance coefficient found from Table Ill for a chosen probability P is k 1 , then the equivalent standard deviation is jlk 1 . When the total uncertainty is found and the combined standard deviation is multiplied by k 1 , this effectively means that the contribution to the root mean square uncertainty for this particular uncertainty isj. Thus the doubt in this instance is really centred on the probability to be associated with j; it might be too low or too high. 8.19 If estimated uncertainties of the type we are considering are known to be Gaussian it is usually not very difficult to estimate a maximum value, but it is extremely difficult to estimate the associated probability. In general it is much safer to estimate the probability on the low side, that is to 0.954 or even 0.917, corresponding to 2.00 and 1.732 standard deviations respectively. In this case if the uncertainty range is ±j as before, the equivalent standard deviation is j I k1 where k 1 is found from Table HI for the chosen value of probability. If now the tolerance probability for the total uncertainty is greater than the probability selected for the uncertainty j, and the total tolerance coefficient is k'1 , then the effective contribution to the total uncertainty by the Gaussian-estimated uncertainties is represented by the term k'j I k 1 which is larger than j and thus gives some margin of safety. It is not recommended that this type of uncertainty be used. It is better to use a rectangular distribution.

Combination of Uncertainties to Give Total Uncertainty Gaussian-type measurable uncertainties

UGM

(random uncertainties)

8.20 Take each Gaussian-type standard deviation (J or estimated standard deviation sv and multiply it by its appropriate tolerance factor (see paragraphs 8.09 to 8.13 and particularly paragraphs 8.11 to 8.13 ). Each of these products should now be squared and the results added together. 8.21 If each Gaussian-type measurable standard deviation is s, and n ~ r ~ 1, where n is equal to the number of uncertainties of this type, and each Gaussian-type tolerance coefficient is ,ki, each corresponding to the probability for which the total uncertainty is required, then the effective uncertainty

256

Uncertainty, Calibration and Probability

contribution from the Gaussian-type errors U GM is given by UGM

= tt:(,k;sYr

12

8.21(1)

,k; is the 'r' component uncertainty and i is the particular number chosen, designating which tolerance coefficient has been chosen, i.e. kb k 2 , k 3 , k4 , k 5 , k 6 and k7 • The values of the ,k;s will depend on the tables used and of course on the number of readings taken. If a contribution ,k;s, represents the uncertainty in the mean, a little care must be exercised over the relation between the ,k; and the s,. If the s, is the standard deviation of a variable, and Table Ill is used the ,k; is k 1 where k 1 is the k selected from Table Ill and n is the number of readings. If Table IV is used the ,k; is equal to k2/ If, however, Table V is used the ,k; is equal to k 3 . If the standard error is found, that is s,/ and this is used in place of s" then ,k; when Table Ill is used is equal to k 1 , and when Table IV is used ,k; is equal to k 2 . If the standard error is used for s, then Table V should not be used, but Table IV used instead. 8.22 Some caution needs to be observed here in the use of equation 8.21 ( 1). If any of the tolerance coefficients chosen are k 2 , k 3 or k 5 , from Tables IV, V and XI respectively, then it is necessary to find Veffas described in paragraph 7.15 when, of course, all standard deviations contributing to the uncertainty must be used to find Verr· U GM should now be written as

Jn

Jn.

Jn,

r=n

U'GM =k2,.,rr [ f._, "

{

_r=l

k

~ k

}2-Jl/2

8.22( 1)

1or2

r

If i = 2, 3 or 5 then the k in the denominator for each term r should be k2 for the probability chosen for the number of readings made to obtain s,. Correspondingly if i is not 2, 3 or 5, then the k in the denominator for each term r should be k 1 from Table Ill for the chosen probability. 8.23 If k 2 , k 3 or k 5 have not been used in preparing U 0 M, then UoM remains unaltered as in 8.21 ( 1).

Combination of estimated systematic rectangular system uncertainties (see paragraphs 8.15 and 8.16) URE

8.24 If the range of each rectangular error is a,, then the standard deviation of each is a,/ )3 and the total standard deviation is (

r=n

I

r=l

a;)l/2

~

3

=

(J"RE

8.24( 1)

for n such distributions. The probability for which the total uncertainty is required is now used to find the tolerance coefficient k 1 which is found from Table Ill. The effective uncertainty contribution from the rectangular errors

The Estimation of Calibration Uncertainties

257

URE to the total uncertainty is thus given by URE

=

k{ ~

8.24(2)

a;y/Z

This procedure is justified by the results obtained in Chapter 4 and summarized in paragraphs 4.67 and 4.68 and Table VI. The true probability of an error occurring in the range 8.24(3) is always greater than that given by the procedure of this paragraph, but in general the value found is very close to the correct value. Alternatively it means that the probability of an uncertainty occurring outside the range 8.24( 3) is slightly less than that calculated. If UGM has been calculated, i.e. k 2 , k 3 and k 5 have been used, then URE should be replaced by 8.24( 4) i.e. the k 2 corresponding to

Veff

obtained from the

t

distribution.

Combination ofmaximum value only systematic uncertainties (see paragraph 8.17) UMu 8.25 If the value of each of these uncertainties is b,, the total standard deviation should be calculated as

8.25( 1) The effective uncertainty contribution U MU from the maximum value only of systematic uncertainties to the total uncertainty is thus given by 8.25(2) or 8.25(3) where k 1 is the tolerance coefficient obtained from Table Ill corresponding to the required probability

P

for the total uncertainty. The justification

-kiiDki

for the mode of calculation of this contribution to the total uncertainty is given in the section beginning at paragraph 3.48 where it is shown that the

258

Uncertainty, Calibration and Probability

probability associated with 8.25(2) is slightly less than the correct value. UMu is used if Veff has been used. Combination of Gaussian systematic estimated uncertainties

UGE

8.26 If jr is the assumed magnitude of a Gaussian estimated uncertainty of this type for a given probability, then the effective standard deviation isjr/rk 1, where rk 1 is the tolerance coefficient obtained from Table III for the assumed probability of occurrence of the uncertainty. The resulting effective standard deviation is thus 8.26( 1)

and if k 1 is the tolerance coefficient for the total uncertainty, the effective contribution to the total uncertainty from the systematic Gaussian estimated uncertainties U ou is given by UoE = k1

{

r=n

r~l CMrk1) 2

}1/2

8.26(2)

If k 2 , k 3 or k 5 have been used in U GM, then as before U GE should be replaced

by

8.26( 3)

Total uncertainty of measurement

8.27 The total uncertainty of measurement UT is given by the square root of the sum of the squares of the effective component uncertainties given in paragraphs 8.09 to 8.26 and thus 8.27( 1)

a

UoM =total of Gaussian-type measurable uncertainties for which

b

U RE

c

U MU

d

U GE

observations have been made (see paragraphs 8.09 to 8.13 and 8.20 to 8.25). = total of rectangular-type systematic uncertainties (see paragraphs 8.15, 8.16 and 8.26). = total of maximum value only systematic uncertainties (see paragraphs 8.17 and 8.27). = total of Gaussian-type estimated uncertainties (see paragraphs 8.18, 8.19 and 8.28).

It is to be noted that the appropriate tolerance coefficient by which the total standard deviation is multiplied is either the Gaussian k 1 or the effective 't' coefficient k 2 derived from equations 7.15(2), 7.15(3) or 7.15(5).

259

The Estimation of Calibration Uncertainties Summary of Terms

8.21 ( 1)

a

or "f..., _ k 2,~ { r=n V'GM-

{

r

k.s r 1

--

}2}1/2

8.22( 1)

,k!or2

r=1

where: n = number of Gaussian-type distributions. ,k; = the appropriate tolerance coefficient of the rth variable, selected from an appropriate table (see paragraph 8.11 et seq.) for the tolerance probability chosen for the total uncertainty. The value of i will depend on the table used and corresponds to the suffix given to k in a particular table. s, = the estimated standard deviation of the rth variable, and is given by

s,=

{

X)2}1/2

(X; _

i=n

L --n- 1

i= 1

2.24(8)

where n = number of observations made of the rth variable and X=

i=n X; " f...,-

i= 1

2.22( 1)

n

8.24(2)

b

or

URE = k2,rrtt>a,/3) 2

r 12

if

Vetr

is used

8.24( 3)

where: n = number of rectangular distributions. k 1 = the tolerance coefficient corresponding to the chosen tolerance probability for the total uncertainty P . k 1 is selected from -k,tok,

Table Ill from the value of

P

. k2,ff is found from

Vetr

using

-k,tok,

Table IV.

a, = semi-range of rth rectangular uncertainty. c

8.25(2)

where: k 1 =the same as that for URE· b, = the magnitude of the rth maximum value only systematic uncertainty. d

UGE = k! tt~ Urlrk!) 2

r

12

8.26(2)

260

Uncertainty, Calibration and Probability

or V(m = kz,ff

tt: r (M,kd

12

8.26(3)

where: k 1 =the same as that for URE and UME· j, = estimated magnitude of rth Gaussian estimated uncertainty for estimate probability P,. ,k 1 = tolerance coefficient for the above variable corresponding to the estimated probability P,. ,k 1 is obtained from Table Ill, k 1 ~ ,k 1 . k 2,ff = tolerance coefficient if Veff has been used. Consideration of the contributions from the component uncertainties may reveal what appear to be anomalies. For instance, suppose that the rectangular contribution is made up of two uncertainty distributions of equal standard deviation aj vf3, then the combined standard deviation is vfta. Thus if k 1 is the tolerance coefficient corresponding to a tolerance probability of 0.997, then k1 = 3 and the rectangular contribution is k1 a = 3 x vfta = 2.4489. But the total range of the combined rectangular distributions is only 2a, and they thus appear to be contributing an amount greater than their combined range. 8.28

Examples on Chapter 8 Example 8.1 The area A of a triangle is given by tab sin() where a and b are adjacent sides and () is the included angle. If sa is the estimated standard deviation of the side a, sb that of side b, and s8 the estimated standard deviation of the included angle e, show that the estimated standard deviation sA of the area of the triangle is given by

Answer: Now ab sin() A=-2

and so log A = log a + log b + log(sin ()) +log Differentiating we have dA

-

A

da

= -

a

db + - + cot () b

d()

t

The Estimation of Calibration Uncertainties

261

and thus, using 8.04( 4 ), we have

Ohm's law states that current, voltage and resistance are related by the expression E/ R = I, where E is the voltage across the resistance R, and I is the resulting current. If the standard deviations of these quantitites are respectively aE, a Rand a 1, find an expression for a 1 in terms of a E and aR. Example 8.2

Answer: 2

(JI

(ai +-ai)

=I 2 E2

R2

where /, E and R are the nominal or mean values of these quantities. Example 8.3

Charles' Law states that pressure P, volume V, and absolute temperature Tare related by the expression PV = RT where R is the gas constant for one gramme molecule of gas. Find an expression for the standard deviation a P of the pressure in terms of the standard deviations of the volume av and temperature ar. Answer:

where P, V and Tare the nominal or mean values of these quantities in the case considered. The wattage W of an alternating electric current is given by the expression W = El cos t 0 ) for v2. In shortened form this is written

9.10

Prob.(F > F 0 jl:v 2) = Prob. (t 2 > t~lv 2 ) 9.10(1) Table XV of Appendix I gives some tabulated values of t 2 for v = 1 to 200 and for four probabilities of t 2 being exceeded. The proof of 9.1 0( 1) is as follows. If we take the expression for the density function for F, that is

F - r((vl + Vz)/2)v~'/2.v~'/2pv,;z-1 .f( ) - r(vd2)r(v 2/2)(v 2 + v1F)(v,+v,)/Z where 0 :s; F :s; oo and put v1 = 1 and v2 = v, we obtain r((l + v)/2)vvi2p-1/2 .f(Fj1·v)= r(v/2)r(1/2)(v + F)fl+vJ/Z ·

9.10(2)

Now

r(:t)

= ~n

Thus with a little rearrangement and manipulation r((l .f(FI1: v)

=

+ v)/2)(1 + F/v)-(v+l)/Zp-l/Z r(v/2)~(vn)

9.10(3)

and

I

00

.f(F 11: v)

=

Pro b. F > F 0

for

and

Vz = V

Fo

If we write

9.10(4) in 9.10(3) we obtain .f(tzlv)

=

r((l + v)/2)(1 + t2/v)-(v+l)/2t-1 r(v/2)

9.10(5)

Consistency and Significance Tests

273

and differentiating 9.1 0( 4) we get dF = 2t dt. Thus

f

00

f(Fj1:v)dF = fcof(t 2 jv)2t dt to

Fo

-- 2Joo r((l

+ v)/2)(1 + tz/v)-(v+l)/Zdt r(v/2)

to

9.10(6)

which is twice the integral of the Student t distribution between t 0 and oo, where t 0 = 0 (from 9.10(4)). Thus

JF

f

oof(Fj1:v)dF = Prob. F > Fo

Fo

= 2 Prob. t > t 0 = 2J .f(tlv) dt 00

to

Finally, if we have a symmetrical density distribution for the variable x as in Figure 9.10, then let the probability of 9.10(7)

x > x 0 be given by P 0

Since the function is symmetrical the probability of x < -x 0 is also given by P 0

9.10(8)

If 9.1 0(7) is squared, then it is clear that the probability of x 2 > x6 is also given by P 0 . But if9.10(8) is squared, then the equality is reversed, and we have x 2 > x6, but the probability is the same as that for x < -x 0 , that is P 0 . Thus the probability of x 2 > x6 is the sum of 9.10(7) and 9.10(8) and we have for a symmetrical distribution that

Prob. x 2 >

x6 = 2 Prob. x

> x0

Since the Student distribution is symmetrical it follows that Prob. t 2 > t6 = 2 Prob. t > t 0 thus completing the proof of the proposition 9.1 0( 1).

Figure 9.10

274

Uncertainty, Calibration and Probability Relationship to "/ distribution

Let FP(v 1 :v 2 ) indicate a value ofF for v1 and v2 , where p Is the probability of this value being exceeded. Now

9.11

6.17( 3) Now x2 I v = s2 / a 2 (see 6.16( 4)) for a Gaussian distribution, and so as v---+ oo and s2 ---+ a 2 , x2 /v---+ 1. Thus if v2 ---+ oo in 6.17(3) we have 9.11(1) where x;v, is the value of x2 for vl degrees of freedom, whose probability of being exceeded is p. Similarly 9.11(2) from 6.20(1). It follows from 9.11(1) and 9.11(2) that F(v 1 :oo)-+ 1 when v1 ---+ oo, and F( oo :v 2 )---+ 1 when v2 ---+ oo.

F Test. Internal and External Consistency Tests Besides being used to decide on the consistency of two sets of data as in paragraph 9.09, the F test can be used to decide whether a number of sets of observations are part of a larger normal population. If individual sets are affected by systematic errors then the complete set of observations will not be homogeneous, and the F test may be used to reveal this inhomogeneity. 9.13 Suppose that there are a total of N observations and that there are m separate sets, each containing nq observations, where L~~T nq = N, and where the nq are generally unequal, that is not equal to N /m. Let the mean of each set be xq where 9.12

9.13(1) where w,q is the weight of each observation x,q· Let f1 denote the mean value of the population and a the standard deviation. Then

9.14

where xis the mean of all the observations and is given by 9.17(2) (see also paragraphs 2.62 and 2.66 for the derivation). Using the properties of the

275

Consistency and Significance Tests means it can be shown that qf"I'"Wrq(X,\- /1)2 = qtmri'" Wrq(Xrq; Xq)2 q=lr=l~

x6

q=lr=l

+

~

xi

qimriq Wrq(Xq;

q=lr=l

x)2

~

X~

+

qimri'"Wrq(X;J1)2t

q=lr=l

9.14(1)

~

X~ Now each term of 9.14( 1) is distributed as x2 independently of the others. The degrees of freedom of the four terms of this expression are respectively N, L~:::T(nq- 1) = N- m (=v 1 ), m- 1 (=v 2 ), and 1. If the whole set is homogeneous, then '\'q=m )2( m- 1) X!2/ vl L...,q=l '\'r=n L....r=l4 Wrq ( Xrq- Xq sl2 F=--= =X22/ v2

'\'q=m'\'r=n L...,q=l L...r=l4 Wrq (XqX-)2(N

-m )

s22

9.14(2)

will be distributed as F with ( N - m, m - 1) degrees of freedom. 9.15 The F test is used in this instance to compare two values of the standard deviation of the mean of the observations, one obtained by using the means of the m sets, and the other by using the mean of all the readings. The former is known as the standard error by internal consistency, whilst the latter is known as the standard error by external consistency. 9.16 The square of the standard error in the mean, by internal consistency, is given by 9.16(1) since there are N - m degrees of freedom, that is there are a total of N observations and m values of the xq, giving N -m independent variables. (See paragraphs 2.69 to 2.71 for the derivation of sf.) 9.17 The square of the standard error in the mean by external consistency is given by 2

'\'q=m'\'r=n (-)2 L...,q= 1 L...r= 1" Wrq Xq- X

s - =.o--=-=---=:--'----2- (m- 1)I~:TI~:::~"wrq

t See Appendix Ill for proof.

9.17(1)

276

V ncertainty, Calibration and Probability

(See paragraphs 2.67 to 2.68 for the derivation of s~ .) The weighted mean is given by -

X

"'q=m~r=nq

=

L.,q=l L.,r=1 W,qXrq =:=::-=--=.:::::-"--'-"----'"'"'q-m "r-nq W L.,q= 1 L.,r= 1

9.17(2)

rq

(See paragraphs 2.62 and 2.66 for the derivation of .X.)

Weights 9.18 Weights are usually chosen so that the largest weight is given to readings which have the smallest standard deviation. It is usually not possible to give an independent weight to each reading, but it is possible to give a weight to each set of nq readings. We have seen from paragraph 2.58 that the appropriate weight for a set of readings is proportional to the reciprocal of their variance, and so we may put 9.18(1)

w,q = 1js 2 (xq)

where s 2 (xq) is independent of r for given q, and is given by r=n,1

s 2 (xq) =

I

9.18(2)

(x,q- .xq) 2 /(nq- 1)

r= 1

(See paragraph 2.65 for the derivation of s 2 (xq).) In general there will thus be m different weights, each corresponding to a set of readings nq. 9.19 Alternatively, all the readings can be given equal weights, in which case w,q can be put equal to unity. This case may be assumed if the separate s 2 (xq) given by 9.18(2) do not differ significantly from one another. Consideration will now be given to the deriving of the sets of equations relevant to the two cases mentioned above.

Case I. Readings given equal weights, that is w,q

=I

9.20 The variance of each set of readings is given by 9.18(2). The mean of each set is given by r =nq

xq

=

L x,q/nq

9.20( 1)

r= 1

from 9.12(1 ). The variance of the mean by internal consistency by putting w,q = 1 in 9.16( 1) and leads to q=m r=nq

si=

L L (x,q- xq)

q=l r=1

2 /N(N-

m)

si

is given

9.20(2)

277

Consistency and Significance Tests

where sf has N - m degrees of freedom. The variance of the mean by external consistency s~ is given by substituting w,q = 1 in 9.17( 1). Thus 2

s2

=

q~m nq(.xq L. q=1

x)2

N(m-1)

9.20(3)

where s~ has m - 1 degrees of freedom. The weighted mean is given by

x= =

q::;:m

r=n~1

L L x,q/N

q=1 r=1 q=m

I

nqxqjN

9.20( 4)

q=1

where in 9.17(2) w,q is put equal to unity and 9.20(1) is used to replace 2:~:::~'1 x,q by nqxq. Thus

- )2 S2 F = _J_ = (m - 1)'\'q=m'\'r=nq( £....q=1 L...,r=1 x,q- Xq ( N -m ) '\'q( - )2 s 22 f....q= m 1 nq xq - x

Case 2.

9.20(5)

Each set of observations given a weight inversely proportional to its variance, that is w,q = l/s2 (xq)

9.21 The variance of each set of readings is again given by 9.18(2). The mean of each set is given by using 9.13( 1) and putting 9.21(1) giving 9.21(2) 9.22 The variance of the mean by internal consistency is given by substituting the value of w,q given by 9.21 ( 1) in 9.16( 1). This gives

Now by 9.18(2) r =n,1

I

r= 1

(x,q- xq) 2 /s 2 (xq) = nq- t

Uncertainty, Calibration and Probability

278

Thus, substituting in

si for this expression we have 2

s 1-

D~7'(nq- 1) ---==-==-~--::---(N- m)Ir::inqls 2(xq)

N-m (N-

m) I::7' nql s 2(xq) 1

9.22( 1)

where 9.22(2) 9.23 The variance of the mean by external consistency is given by substituting the value ofwrq given by 9.21(1) in 9.17(1). Thus

2 I:~7' L~~'i· (xq - x)2 I s2(xq) . s2 = (m - 1) I::7' I~:i•1 I s 2 (xq) - I:~inq(Xq- x?ls 2 (xq)

1)I::inqls2(xq) I:~7'(xq- x) 2 ls 2 (xq)

- (m_

-(m- 1)I:~i1ls 2 (xq)

9.23(1)

using 9.22 ( 2). 9.24 The weighted mean is given by substituting for wrq in 9.17(2). Thus -

X

I:~7'L~~'i·x,qls2(xq) I:~7' nql s 2(xq)

= =-=-=---:'----' -

_ I:~7'nq.Xqls 2 (xq) - I::inqls 2(xq)

since xq

=

r=nq

L x,qlnq

9.20( 1)

r= 1

Thus finally 9.24( 1) using 9.22(2). Hence F

where

si has N

si

(m- 1)

si

I::i(xq- xfls (xq)

= - = =----'------::----::2

- m degrees of freedom and si has m - 1.

9.24(2)

Consistency and Significance Tests

279

9.25 If the results being compared cover a number of workers, the number of degrees of freedom of s 1 may not be known. If the number is thought to be large it is sometimes assumed to be infinite. Thus we have the case of s~/si ~ 1

Now

9.11(2) F( I- p),(l',: X. I

Thus 9.25(1) where sV si ~ 1, and so z

X(l-pi.•·, = Vz

s~

2

9.25(2)

SI

where v2 = m -- 1 is the number of degrees of freedom for s 2 , p is the probability of the calculated value of x2 being exceeded, and so 1 -pis the probability of x2 being less than the calculated value. (See Table VI, Appendix I. The last four columns of this table should be consulted in order to fix the probability for the value x2 given by 9.25(2) being equal to or less than a tabulated value.)

sVsf

~ 1

This is the case when the value ofF found from si Is~ is less than unity, and has to be inverted, to provide a value ofF larger than unity in order that Tables VIII and IX, Appendix I, may be used. Now 6.17 ( 3) If F is inverted, then

and so if v1

-->

oo, we have 9.25(3)

and thus 9.25(4)

280

Uncertainty, Calibration and Probability

where p is the probability of the calculated value of x2 being exceeded. In this case the first four columns of Table VI, Appendix I, should be consulted in order to fix the probability for the value of x2 given by 9.25( 4) being equal to or greater than a tabulated value. If x2 is less than a tabulated value, then the readings are homogeneous. 9.26 When the number of degrees of freedom for both s 1 and s2 are known, and F has been found either by 9.20(5) Case 1 and/or 9.24(2) Case 2 then Tables VIII and IX should be consulted to see if the values of F given for either case is reasonable. If the value of F obtained from the readings is greater than that given by Table VIII, then the probability of occurrence of the value of F obtained from the observations is less than 0.05. If the value from the observations is also greater than that obtained from Table IX, then the probability of occurrence of the value ofF obtained is less than 0.01. If the probability of occurrence ofF is found to be greater than 0.05 the readings can be assumed to be consistent, if the probability ofF lies between 0.05 and 0.01 the consistency of the readings should be treated as suspect, whilst if the probability ofF is less than 0.01 the observations should be regarded as inconsistent. 9.27 It should be noted that when Tables VIII and IX are used, the value of F should always be greater than unity, and in order to achieve this condition si j s~ = F may have to be inverted. A further point to note is that the v 1 of the tables always refers to the numerator of the fraction for F, whether F has been inverted or not.

Results Consistent

Weighted mean 9.28 If the results are found to be consistent, then the weight of each mean xq is given by equation 9.18 ( 1). If Case 1 is used the mean is given by qo=:m

.X=

I

nq.Xq/N

9.20( 4)

q= 1

whilst if Case 2 is used the weighted mean is given by 9.24( 4) Case 1 of course gives equal weight to each reading, whilst Case 2 gives each reading of group q a weight of ljs 2 (xq), that is inversely proportional to the variance of each group.

281

Consistency and Significance Tests

Standard error of the mean

9.29 This is given generally by 9.29(1) (See paragraphs 2.61 to 2.64 for the derivation of s 2 (x) and .X.) Now

2 = ( Xrq

+ (x;

2XrqXq

-

2xqx

= (xrq- .Xq) 2

+ Xq-2) + 2XrqXq+

.x; + 2xqx- 2xrqx

x2 ) -

+ (.Xq-

-2 - Xq

x) 2

+ 2.Xq(xrq- xq)-

2x(xrq- xq) 9.29(2)

Now

L WrqXq(Xrq- Xq) =

r=l

"

r=nq

r=nq

r=n 4

Xq L WrqXrq- x; L Wrq r=l r=l

9.29(3)

=0 since .X q =

'r=nqw

L...r= 1

rq X rq

9.13(1)

=--==----''-----"'"

LWrq

Similarly

=0

9.29(4)

again by 9.13(1). Thus q=m r=nq

L L

q=m r=nq

Wrq(Xrq- i) 2 = L L wrq(xrq- Xq) 2 q=l r=l q=l r=l

+

q=m r=nq

L L wrq(xq- x) 2 q=l r=l

9.29(5)

Uncertainty, Calibration and Probability

282 Thus sz(.X)

=

-)2 ( '\'q=m'\'r=n,1 L.,q= 1 L..r= 1 w,q x,q- X '\'q-m'\'r--nq (N - 1) L.,q= 1 L..,r= 1 w,q

- )2 ( '\'q=m'\'r=nq L.,q = 1 L.,r = 1 Wrq Xrq - Xq '\'q-m'\'r--nq (N - 1 ) L.,q= 1 L..r= 1 w,q

+

-)2 ('\'q=m'\'r=nq i..Jq= 1 L.,r= 1 Wrq Xq - X (N

(N-

-

1) '\'q-m '\'r=nq L..,r= 1 w,q L,q= l

(m- 1)

m) si + - = ---N-1

N-1

s~ 9.29(6)

si

and s~ are given in the general case by equations 9.16( 1) and 9.17 ( 1), respectively, and by equations 9.20(2) and 9.20(3), respectively, for Case 1 and by equations 9.22( 1) and 9.23( 1), respectively, for Case 2.

Results Inconsistent 9.30

If the results of the analysis of the grouped observations lead to the conclusion that the results are inconsistent, the best mean is given by taking the mean of the sum of the m means, in each case giving these means a weight of unity. Thus q=m X x=L:-~ q=1 m

9.30( 1)

The standard error of this mean is given by s 2 (x)

=

1

q=m

I

m(m-1)q=1

(xq - x) 2

9.30(2)

It is to be noted that Case 1 leads to the same formulae if each set of measurements contain equal numbers of observations. (See equation 9.20(3 ).)

·i Test to Check if a Hypothetical Distribution F(x) Fits an Empirically Obtained Distribution

9.31

In order to carry out this test the abscissa (x axis) should be divided up into n intervals, each containing at least four values of the empirically observed frequency distribution. If, however, n - 1 is greater than about eight and the sample size greater than about forty, then it is permissible for the number of observations in an interval, in isolated cases, to be as low as one. Let each interval q contain j~ observations. If a sample observation should lie at a common boundary point of two intervals the value 0.5 should be added to each of the adjoining k From the hypothetical distribution

Consistency and Significance Tests

283

F(x) the probability pq of an observation occurring in each interval should now be found. The total number of observations is I~=~fq = N and the

expected number of observations in each interval is Npq. 9.32 The expression

I (f',·

q=n

q

q= 1

- Np )2 q

Npq

____,

x;

9.32( 1)

when v----> oo, and where v is the number of degrees of freedom, equal to

n - 1 if no parameters have to be calculated to find the pq of the formula

given by 9.32( 1 ). The relationship given by 9.32( 1) was discovered by K Pears on. 9.33 The left-hand side of expression 9.32( 1) is now evaluated using the empirical data, and the hypothetical distribution F(x). If k parameters have to be estimated to find the fitted frequencies, then v = n - 1 - k. If m groups of samples, each containing n classes, are submitted to a x2 test based on 9.32( 1), then the number of degrees of freedom is v = (n- 1 )(m- 1). If each sample requires k parameters to be found in order to calculate the pq, then v = (n - 1 - k )(m - 1). Interpretation of test from Table VI, Appendix I

9.34 Having found the value of x2 using 9.32( 1), Table VI is consulted and the appropriate row for v selected. If the value of using 9.32( 1) is less than the numbers in columns 1 to 4 of the tabulated values of x2 , and greater than those in columns 5 to 8, then the function f can be assumed to fit the empirical data, since the probability of the value of x2 (9.32( 1)) being exceeded is greater than 0.05, whilst the probability of a smaller value is also greater than 0.05. is greater than any or all of the values given in columns 9.35 If, however, 1 to 4, then the probability of its being exceeded is less than the probability corresponding to the column which immediately exceeds it and greater than the probability of the column it immediately precedes. For example, if x~ lay between the values given by columns 1 and 2, then the probability of its being exceeded is less than 0.05 but greater than 0.025. In this particular case the fit would be regarded with some suspicion and further investigation made. If x~ lay between columns 2 and 3 or 3 and 4 then the fit given by F should be rejected.

x;

x;

Kolmogorov-Smirnov Statistic 9.36 The Kolmogorov--Smirnov statistic is used as a basis for two tests: ( 1) for the comparison of a known distribution with one defined by measured

284

Uncertainty, Calibration and Probability

data, and (2) for the comparison of two distributions defined by measured data. For details, see Appendix I, Tables XX and XXI.

Examples on Chapter 9 Two means for the value of an electrical resistance, obtained by using two different examples of the same type of equipment, are 10.103 n and 10.11 n. The first value has an estimated standard deviation for the resistance of 0.03 n, whilst the second has an estimated standard deviation of 0.06 n. The first determination was the result of fifteen readings, whilst the second was the result of ten readings. Find between what probability limits the t test places the tvd calculated, and express an opinion on the consistency of the two means. Example 9.1

Answer: Using 9.03(1a)

s = (

0.03 2

V

Using 9.03(2) to find Svd

svd

X

14 + 0.06 2 23

X

9) 112

= 0.04423

we have

1 1 = 0.044 23 ( -- + 10 25

)1/2 = 0.016 550

Therefore

i\-

x2

tvd = - - Svd

0.007 = 0.423 0.016 55

vd= 10+ 15-2=23 From Table IV, we find that the tv, for v = 20 or 25, are much greater than the value found, and thus the two means are consistent. Example 9.2 Two laboratories have determined the mean value of a standard resistance. The first laboratory finds the mean value to be 1.000 14 n, with an estimated standard deviation of 0.000 32 n from twelve readings; the second laboratory finds the mean value to be 0.999 28 n with an estimated standard deviation of 0.000 41 n from ten readings. use the t test to determine whether the two means are consistent and state the value of tvd· Answer: tvd = 3.107 and vd = 20. The two means are inconsistent because the probability of tvd ;:;:, 3.107 for vd = 20 lies between 0.005 and 0.01 which is too small a probability for consistency.

Consistency and Significance Tests

285

Example 9.3 The mean value of the temperature found using an optical pyrometer is 125l.Ooc with a standard deviation of 3.1 oc from ten readings. A second optical pyrometer makes the mean temperature 1241.1oC with a standard deviation of 2.6°C from seven readings. Determine if the two means are consistent, and state tv" ahd the range of probabilities between which the probability of it being exceeded lies.

Answer: tvd = 3.4 and vd = 15. The probability of tvd being exceeded lies between 0.005 and 0.001 and therefore the two means must be considered inconsistent. Example 9.4 Two values of a resistance have been found using different types of equipment. If the value found, using the first type of equipment, gave a mean value of 1000.014 n with a standard deviation of 0.032 n from eleven readings, whilst the second type of equipment gave a mean value of 1000.054 n with a standard deviation of 0.074 n from sixteen readings, state if the two mean values are consistent. Assume that the two standard deviations are not equal.

Answer: The two means are consistent. tvd = 1.917 and vd = 21.85 (round to 21 ). Example 9.5 Two standard deviations have been found, one the result of sixteen readings has a value of 0.0002 V, whilst the other one the result of eleven readings has a standard deviation of 0.0005 V. Check to find if the two standard deviations are consistent.

Answer: The standard deviations are not consistent as they yield a value for

F of 6.25, which has a probability of occurrence of less than 0.01.

Example 9.6 Check the two standard deviations given in Example 9.1 to see if they are consistent.

Answer: The probability ofF being exceeded is approximately 0.01, and so one of the standard deviations should be regarded with considerable suspicion. As Example 9.1 suggests that the two means are consistent, the reason for the wide difference in the standard deviations could be the result of friction in one of the pieces of equipment. Example 9.7 A standard resistance has been measured a number of times by different laboratories, and there is some difference in the mean values obtained and also in the standard errors in the means obtained. Use the F test to decide if the results are homogeneous.

Hint: Assume weights proportional to 1/s 2 (xq). Case 2 of test.

286

Uncertainty, Calibration and Probability Mean value of resistance (Q)

v, = n- 1 ( n = number of readings)

s, Estimated standard deviation of mean

1.000152 1.000 174 1.000182 1.000 168 1.000152 1.000194 1.000 185 1.000143

10 10 8 8 8 10 9 8

0.000008 0.000015 0.000007 0.000012 0.000009 0.000016 0.000014 0.000010

L

Vr

= 71

Answer: F

=

s~

2 = 89.51 sl

Note that the value of si/si has had to be inverted to obtain a number larger than 1 for F. The results are not compatible as the value ofF obtained has a very small probability of occurrence, that is much less than 0.01. The result implies either that some laboratories have inaccurate measuring equipment or that the resistance measured is unstable. Example 9.8 The mean values of a number of measurements of the resistance of a particular transfer standard have been made, but the number of individual measurements for each of the means is not known. Use the x2 test (see paragraph 9.25) to ascertain if the group of measurements are homogeneous or compatible.

s

Mean value of resistance ( n)

Estimated standard deviation of mean

100.1256 100.1281 100.1272 100.1295 100.1262 100.1274 100.1251 100.1252 100.1262

0.0005 0.0005 0.0011 0.0009 0.0006 0.0007 0.0009 0.0008 0.0012

Consistency and Significance Tests

287

Answer:

sz

_2

si

= 6.874

x2 =

6.874 x 8 = 54.992

The results are not homogeneous as the value of x2 is large and the probability of its occurrence is low, that is less than 0.005. The standard may be unstable or the measurements of some of the laboratories may be suspect, perhaps because of unaccounted-for uncertainties.

10 Method of Least Squares 10.01 In this chapter we shall apply the method of least squares 1 to a number of problems including curve fitting and the determination of a mean plane. The method is best illustrated by application to a particular problem.

Determination of a Mean Plane 10.02 This is a problem which occurs in metrology when a surface plate is calibrated, and it is required to find the plane that will best fit the empirically measured points. When this plane has been determined it is possible to determine the departures from flatness of individual points on the surface plate and to determine the overall error. 10.03 We will first derive the formula for the determination of the separate points which contribute to the mean plane. The surface to be measured is usually surveyed by measuring along certain lines. (See Figure 10.03.) The dots represent points at which the height of the test surface above some arbitrary line is calculated. The number of points chosen will depend on the size of the surface plate but the number along a diagonal may be between

A

G

8

t

t

F

D

H

Figure 10.03

288

c

289

M et hod of Least Squares

ten and twenty, the latter figure being suitable for a fairly large table of the order of 6 ft by 4 ft. 10.04 The readings are obtained by taking successive readings, with an auto-collimator, of the light reflected from a mirror mounted on a three-point suspension table. At each move the mirror mount is moved through a distance equal to the spacing of the third foot from the other two. The number of readings along any line should, for computational reasons, always be an even number. Alternatively the gradient can be read directly by using an instrument such as a Talyvel, which is also mounted on a three-point suspension table.

Readings along Diagonals 10.05 Let rx 1 be the auto-collimator reading for the first position, rx 2 for the second, etc., up to rx" where the rxs are the measured angles converted to radians, and where n is the number of readings for a diagonal. If the spacing between the third foot and the other two is dd, the height of the measuring points about the arbitrary line 00' (see Figure 10.05) is given by r=p

zP

=

L ddrxr

10.05(1)

r=O

(it should be noted that cx 0 = 0 by definition) where p is the pth position relative to zero (xp = pdd) and zP is the height of the pth point above 00'. 10.06 If we now join the point 0 (x = 0) with the last point on the diagonal P (x", z") then z" is given by I~:;:~ ddcx,. The height above 00' at the pth position (xp) on the line OP is thus r=n

(pjn)

L ddcxr = z~

r=O

:X:p = d.,:x:,: 0 where d. = :X:p-

p Xp. 1

0

Figure 10.05

290

Uncertainty, Calibration and Probability

The difference between

z~

and z P is given by r=n

I

z~- zP = (p/n)

r=p

darx,-

r=O

I

10.06( 1)

darx,

r=O

Let us now draw a line RS parallel to OP through the point Q(xn 12 , z" 12 ), and calculate the height of the points on the surface from this new line RQS (see Figure 10.06). The spacing between the straight lines OP and RS is seen to be z~ 12 - zn 12 and thus the required height z~ is seen to be given by z~ = z~ 12 -

Zn;z - (z~- zp) r=nj2

r=n

=

±L darx, -- L

darx, - (p / n)

r=O

r=n

r=p

r=O

r=O

L darx, + I

darx,

or 10.06(2) This gives a series of points such that r=n

z "0

=

z n"

= i1 '""' L.,

rx, ·-

r= 0

and z~12 = 0. 10.07 If the two diagonals AC and BD of Figure 10.03 are calculated as above, then to complete the survey, the formula for the height of the parallel sides must be found. Let the value of z~ and z~ for the diagonal AC be b and for the diagonal BD let the value be a, where a and b and the heights of the measured points on the two diagonals are all measured from the reference plane xy defined by the two reference lines RQS and R'QS', where R'QS' is the reference line for one diagonal and RQS is the reference line for the other, and Q is their point of intersection.

o'

0

Figure 10.06

291

M et hod of Least Squares

Parallel Sides 10.08 Let us consider the side BC of Figure 10.03. If we proceed as before the distances of points on the curve obtained below the line joining the starting and end points are given as before by r=n

z~- zP = (p/n)

I

r=p

I

ddcxr-

r~o

ddar

10.06( 1)

r~o

(see Figure 10.08) where dd is the spacing between the third foot and the other two feet of the mirror mount. The height of the curve RQS, given by the measured points above the x' y' reference plane, is given by the difference between the height of the lineRS above the reference plane x' y' and (z~- z.n). Thus z~'

=a+ (b- a)(pjn) --

=

r=p

I

r~o

=

(z~-

zp)

r=n

ddar- (p/n)

L ddcxr + (p/n)(b- a)+ a

r~o

ddc~ a,- (p/n) :t~ a,)+ (p/n)(b- a)+ a

10.08(1)

10.09 Equation 10.08( 1) is used to calculate the height of the points, making up the parallel sides, above or below the reference plane xy. If desired, the magnitude of the negative point having the greatest magnitude can be added to all the readings, making all readings greater or equal to zero. This of course is only relevant if negative values exist. It is then a very simple matter to judge the departure of the plate from the flat, that is the xy reference

a

~~~~--------------~~--~~ 0 X, X, Xp Xn rj Figure 10.08

292

Uncertainty, Calibration and Probability

z

z;

z;'

plane, since the readings give this departure, that is and (see equations 10.06(2) and 10.08(1) respectively). 10.10 Referring to Figure 10.03, if the lines GH and FE are measured and are made to join up to the lines AB and DC, and AD and BC respectively, then in general the two lines GH and FE will not intersect where they cross, and neither will either of them intersect the two diagonals AC and BD at the centre Q. If an adjustment is required for this non-intersection of the lens GH and FE at the centre then this may be made as follows. 10.11 If the intersection height of the two diagonals at the centre Q above the chosen reference plane is z 1 , and if the heights of the lines GH and FE at the centre are z 2 and z 3 respectively, then the mean intersection point can be taken as (2z 1 + z 2 + z3 )/4 = z. The correction amount (z- z 1 )( 1 - 2sdw) should now be added to points on the two diagonals AQC and BQD, where s 1 is the distance from the centre Q along any diagonal to the corresponding corner, and w is the length of a diagonal. The corresponding correction for the paths QGA, QGB, QHD and QHC is (z- z 2 ){ 1 - 2s 2 /(u + v)} where s2 is the distance from Q along either of the above paths, u is the length of AB BC and v is the length of AD BC. Similarly the correction for the paths QFA, QFD, QEB and QEC is (Z- z 3 ){ 1- 2s 3 /(u + v)} where s 3 is the distance from Q along the paths just quoted. The s are of course always positive. For the calculation of the uncertainty of the difference in height between any two points on a surface, see Appendix IV.

=

=

Improved Coverage of Plate 10.12 The surface plate in Figure 10.03 was divided into 49 datum points obtained from 46 measurements using a Union Jack configuration. A better coverage of the surface is obtained by using the layout scheme shown in Figure 10.12. This shows 25 datum points obtained from 48 measurements.

A

N

+

~----~~----~------~~-----iM

R

0

H

Figure 10.12

-

293

M et hod of Least Squares

The arrows show the directions in which measurements should be taken starting with the position given by the arrow. The procedure for working up the results is initially the same as for Figure 10.03, i.e. the Union Jack is treated first, the diagonals made to cross in the middle with identical heights, and the heights of the ends of each diagonal made equal. The lines AB, CD, AD and BC are then hung from the ends of the diagonals at A, B, C and D. The lines GH and EF are hung from the centres of the outside lines of sight. This leads in general to three different values of the heights at the centre Q, say z 1 , z 2 and z 3 , where z 1 is the height of the crossing point of the diagonals and z2 and z 3 are the heights of EF and GH at their centres at Q. The mean height at the centre is taken as before as (2z 1 + z 2 + z 3 )/4 and the correction for each measurement point extended back along the sight lines to the corners using the expressions given in paragraph 10.11. The rest of the sight lines should be hung from the outside generators after the above corrections have been made, and any non-agreement at crossing points averaged. 10.13 For larger surface plates the number of measurement points may be increased by increasing the number of rectangles. The working up procedure remains the same. For a large 6ft by 4ft table, each sight line might have 13 datum points leading to 169 datum points in all with 336 measurements. The formula for working out the number of datum points and number of measurements for a layout like Figure 10.12 is datum points

=

n2

measurements= 2(n- 1)(n

+ 1)

where n =number of datum points on a side. The number of measurements may be somewhat reduced for large tables by adopting the layout scheme shown in Figure 10.13. The accuracy obtained will be slightly reduced. The basic difference is that the number of datum points along the long sides only is doubled. The working up procedure is the same as for the previous layout. Corrections to the added points apart

Figure 10.13

294

V ncertainty, Calibration and Probability

from the outside ones covered by the Union Jack procedure are optional. If required, the correction to the non-intersecting points excluding those on the outside sight lines is given by ((z~- z,) + (z~+ 1 - zr+ d)/2, where z~ and z~+ 1 are the corrected values for two successive intersecting points on either side of a point to be corrected and z, and zr+ 1 are the corresponding points on the appropriate long sight line. This value should be added, if positive, or subtracted, if negative, from the point to be corrected. The number of points for such a scheme is .

n(n+l)

datumpomts = - - 2

measurements= (n -- 1)(3n

+ 7)/4

where n is the number of datum points along a long side, (n + 1)/2 is the number of datum points along a short side and along a diagonal. If n = 13 say, for a 6ft x 4ft table then the number of datum points is 91 and the number of measurement points is 138, a considerable reduction on n = 13 for the full rectangle scheme of Figure 10.12 of 336. 10.14 A different approach to surface plate measurement using a least squares method has been developed at the National Physical Laboratory at Teddingtont. For a rectangular table the layout of Figure 10.12 is advocated. This does lead to a large number of observations. For a large 6ft x 4ft table the number of datum points is 13 per side and for each diagonal the number of datum points is 169 with 364 observations. If n =number of datum points per side, number of datum points

=

n2

number of observations

=

2n(n

+ 1)

The method requires considerable computation being performed on the measurement data, using a specialized computer program. The method would seem to give reliable results.

Calculation of Mean Plane 10.15 We now have a map of the surface where the height of each measured point is given either above or below the xy reference plane. Now the equation of a plane is given by a' x + b' y + c' z = f or alternatively

z = a"x + b"y

+!'

t See NPL reports MOM 5 1973 and MOM 9 1974.

10.15(1)

295

Method of Least Squares

The deviation of each point on the surface plate from this mean plane will be obtained by subtracting from each zj, (co-ordinates xi, Yi) representing the height of each measured point above the reference plane xy, the value of z given by 10.15( 1) corresponding to the height of the mean plane above the reference plane xy for the same co-ordinates xi, Yi· 10.16 Thus the deviation of each point from the mean plane is given by zj- zi

zj- a" xi- b" Yi-

=

f'

10.16(1)

Now if each of these deviations from the mean plane is a random deviation, and all these random deviations belong to the same random population, then the probability of all the deviations found occurring together is given by p

1

=

-

ff {

e-(zj-z)2/2"2}

10.16(2)

0""(2nt12 i=l

where IT implies the product of the term in the bracket with like terms, where j varies between 1 and n, n being the number of terms. This may be written

alternatively as P =

1 0""(2nt12

{ i=•(z'.-z/} 1 1 ·exp -

L

i=l

20"2

10.16(3)

Now the expression covered by the summation sign sigma is a function of a", b" and f', and in order that the set of deviations obtained should have

occurred, P must be made a maximum by choosing the right values of a", b" and f'. It is easily seen that for P to be a maximum j=n

W

=

L (zj- zY

10.16(4)

j= 1

must be a minimum, and thus we require the values of a", b", and f' corresponding to the three conditions

aw

aw =O

-=0 aa" '

8b"

aw =O

and

ar

10.16(5)

Principle of least squares 10.17 It is seen that in applying the condition that P should be a maximum, known as the 'principle of maximum likelihood', that this in turn has led to the requirement that the sum of the squares of the deviations should be a minimum. This latter condition is known as the principle of least squares 2 and is often applied directly without using the principle of maximum likelihood. 10.18 Substituting for zi in 10.16(4) we have, using 10.16(1), that j=n

W

=

L (zj- a" xi- b"yi- f')

j=l

2

10.18(1)

296

Uncertainty, Calibration and Probability

Using equation 10.18(1) we have

aw

j~n

oa"

j~ 1

-- = -

aw =

-

ob"

-

aw "'if'

u

I: x.(z'.- a"x.- b"y.- f') =

I

j=n

j~ 1

J

J

I

J

f1

/1

y ·( z · - a x · - b y. -1

1

1

0

10.18(2)

f )= 0

10.18(3)

J

1

1

j~n

=-

.L (zj- a" xi- b"yi- f')

J~

1

=

0

10.18(4)

Rewritten these become 10.18(5) 10.18(6) and 10.18(7) where L stands for I:1~1. Eliminating f' between 10.18(5) and 10.18(7) by multiplying the former by nand the latter by L:xi and subtracting 10.18(7) from 10.18(5) gives

a"{nL:xJ- (l::xY}

+ b"{nL:xiyi- L:xiLYi}

=

nL:xizj- LXiLzJ 10.18(8)

Similarly, eliminating f' between equations 10.18( 6) and 10.18(7) by multiplying the former by nand the latter by LYi and subtracting 10.18(7) from 10.18(6) we have

a"{nL:xiyi- LXiLYi} + b"{nL:yJ- (LYY}

=

nL:yizj- LYiLz) 10.18(9)

10.19

10.18(8) and 10.18(9) can be rewritten as 10.19( 1)

and 10.19(2) where 10.19(3) Thus 10.19(4)

297

M et hod of Least Squares

and 10.19(5) Now from 10.18(7)

f

'

LZ} n

a"Lxi n

b"LYi n

10.19(6)

=-------

and thus substituting for a" and b" from 10.19(4) and 10.19(5) gives

J'

= LZ} _ (Yxz'9yy-

n

9yz~9xy)LXi n

Yxx9yy- Yxy

_ (9yz'Yxx- Yxy;xz·)LYi Yxx9yy- Yxy n

10.19(7) The equation of the mean plane is thus

10.19(8) 10.20

Now the co-ordinates of the centroid of all the points are

n

n

n

and if the co-ordinates of this point are substituted for x, y and z in 10. 19(8) it is clearly seen that the expression is identically equal to zero, and thus the centroid lies in the mean plane.

Change of Axes 10.21

Let the centroid co-ordinates be given by

x=

"X· _L... ___,,

-

and

n

LZ}

10.21(1)

z=--

n

Now let us change the centre of our co-ordinate system to x, ji and z. Thus if the new co-ordinates are represented by capital Xi, Yj and Zj, we have Xi= xi- xi,

Yj

= Yi-

Yi

and

Zj

=

zj- zj 10.21(2)

298

Uncertainty, Calibration and Probability

10.22

Expressed in these co-ordinates, the true mean plane becomes 10.22( 1)

Z =a" X+ b"Y

from 10.19(8) and W=I(Z)-ZY

=

j=n

I

(Zj- a" xj- b" 1}) 2

10.22(2)

j= 1

The conditions for a minimum in this case reduce to two conditions, namely

ow =O

and

oa"

o~=O ob"

10.22(3)

from 10.16(4). Thus

, "x.- b" Y. ) o = I x. (z.-a ow oa"

10.22(4)

="~ Y.(Z'·- a" X.- b" Y.)

10.22(5)

-=

J

J

J

J

and oW = 0 ob"

J

J

1

1

Thus 10.22(6) and X.Y. a"" ~}}

Y2 + b"" ~}

=" Y.Z'.

~}}

10.22(7)

Eliminating b" between 10.22(6) and 10.22(7) gives 2 yS Y.Z'. "X. J J~ J J ~ J a" = "~ Y J "~ X J.z'.IXJI YJ- (IXjl}f

10.22(8)

Similarly 2 Y." X J.Z'.J J J~ ~ b" = "~ X J "~ X J.Z'..I - "X.

IXJIYJ- (IXjlj)

2

10.22(9)

10.23 Either set of equations, that is 10.19(4), 10.19(5), 10.19(6)and 10.15(1) or 10.22(8), 10.22(9) and 10.22(1) can be used to find the mean plane, but the set using the centroid co-ordinates is decidedly the easier to calculate. Once x, y and z' have been found and the new variables X, Y and Z' found using 10.21(2), a" and b" are easily found using 10.22(8) and 10.22(9). 10.24 Usually the overall departure from the mean plane is not much different from the overall difference of height obtained by using the analysis given in paragraphs 10.02 to 10.11. The mean plane is, however, useful if the

M et hod of Least Squares

299

standard deviation of the departure of the surface plate from the mean plane is required.

Standard Deviation from Mean Plane 10.25 The standard deviation of the points from the mean plane will give some indication of the statistical quality of the plate, and the value of this is found as follows. The equation of the mean plane may be written a" X+ b"Y- Z = 0 (a"z + b"2 + 1)112

---:---c----cc

10.25(1)

The distance of any point on the plate X j• l:j, Zj from the mean plane is given by a" X.J + b" Y.J - Z'..I (a"2 + b"2 + 1)112

10.25(2)

The standard deviation is thus given by s-

{

j=n

"

(a"X-+b"Y.-Z'-)2 }1/2 J

.T

J

j~1 (n- 2)(a" 2 + b" 2 + 1)

10.25(3)

n - 2 appears in the denominator because two constants have had to be found, namely a" and b".

Curve Fitting: Fitting a Straight Line 10.26 There are many cases in which it is known that two variables, say x and y, are connected by a straight line relationship. Owing to measuring inaccuracies, if a graph is plotted of y against x, we get a series of points which do not lie on a straight line. The problem is to find the best straight line through the points obtained. In this analysis the values of the x will be assumed to be correct, whilst only the y values will be assumed to have uncertainties. 10.27 Let the straight line be

y=mx+c

10.27(1)

then if y; corresponds to the measured ys the sum of the squares of the deviations is given by i=n

W=

L (y;- YY

i= 1

10.27(2)

300

Uncertainty, Calibration and Probability

where there are n points. Substituting for y from 10.27(1) we have i=n

I

i~

(y;- rnxi- c) 2

=

10.27(3)

W

1

and thus, using the principle of least squares, we have, in order that W is a m1mmum

aw

aw

oc

10.27( 4)

IxJy;- rnxi- c)= 0

10.27(5)

and

-=0

am

-=0

Thus and 10.27(6)

that is 10.27(7)

and 10.27(8)

Eliminating c between 10.27(7) and 10.27(8) by taking 10.27(7) times n minus 10.27(8) times I xi, we have

or 10.27(9)

We obtain c by the operation 10.27(7) times gtvmg

I

xi minus 10.27(8) time~

Ixf

or 10.27( 10)

10.28 An alternative expression is obtained as follows. Let x be the mean of all the xi and .V the mean of all the y;. Thus -

i~n xi

X=L_,i~ 1 n

10.28( 1)

Method of Least Squares

301

and

_, "y;

10.28(2)

y=L..-n

Now let e; be the deviation of X; from x and let r; be the deviation of y; from ji', thus X;=

x + e;

10.28(3)

+ T;

10.28(4)

and

y;

= ji'

and 10.28( 5) since 2xie; = 0 because if we substitute for X; from 10.28(3) in 10.28(1) we get nx = I (x + e;) = nx + Le; and therefore

L e; =

0

10.28 ( 6)

Also

L x;y; =I (x + e;)(ji' + r;) =

Ixf

+ Le;r; + Y'Le; + xir; 10.28(7)

smce 10.28(6) Further we have

L'; =

0, because on substituting for ji'n =

y;

from 10.28(4) in 10.28(2)

L(Y' + r;) = nji' + L';

and therefore 10.28(1) 10.29 Thus, substituting in 10.27(9) for we have

Ix;y;, Ix;, LY;, Ixf and (IxY

10.29( 1)

302

Uncertainty, Calibration and Probability

whilst the same substitution in 10.27(10) gives

10.29(2)

The required straight line is thus y-y_,

"'a.r.

' ( - X-) =L.. - -' X

10.29(3)

Isf

or r

10.29(4)

=m~:

This is sometimes called the line of regression of y on x. A point to note is that the line of regression passes through x, f. The two equations of condition 10.27(7) and (8) can be written as 10.29(5)

and

mx + c

=

f

10.29(6)

(L sl}jn is sometimes written as 0'~ where 0'xis the standard deviation of xi from its mean

x. Now

10.29(5) can be written as I}

1

x(mx + c- f)+ m I--'-=- I siri n n which in view of 10.29(6) reduces to

g1vmg 10.29( 1)

(L rf)jn is sometimes written as 0'; where O'y is the standard deviation of Yi from its mean y'.

M et hod of Least Squares

303

Weighted Mean line 10.30 If each point x 1, y1 has a weight w1, then the gradient mw is easily shown to be 10.30(1)

Similarly

Y~L W;efw-

XwL W;B;w!iw

L W;efw

10.30(2)

The correct value for the weight w1 is 1/ u;Li• where u yLi is the standard deviation of y; from the mean line. For single points uyLi is not generally known, but if each point is the mean of a number of observations, n1, then each point is given by x 1 and

y; = ( ~ Y7)

I

n1

in which each w1 = nJu;Li• where uyLi is the standard deviation of the from the mean line at the point x 1, y1• Note that

y7

10.30(3)

and 10.30(4)

where w1 is the value stated above. Also note that in the weighted equations eiw = W;(X;- xw), riw = W;(Y;- y~) and weighted Ie?w = W;(X;- xw) 2 .

Standard Deviation and Standard Error in the Mean of yj From the Mean Line -r = me 10.31

The sum of the squares of the residuals is equal to i=n

L (y;-mx

i= 1

1

-c) 2

=

i=n

L (r

1

-meY

10.31 ( 1)

i= 1

Since there are two equations of condition, and m and .X are fixed, there are only n - 2 degrees of freedom. Thus the best approximation to the square of the standard deviation u;L of the y; about the mean line is given by 2

SyL =

I:~~ (r;- meY

n-

2

10.31(2)

304

Uncertainty, Calibration and Probability

The weighted variance 10.31(3) (See also paragraphs 10.71 to 10.76.) The square of the standard error in the mean of y; from the mean line is thus given by 2

S-L

Y

=

me; )2 n(n - 2)

'\'i=n ( L.i=l r ; -

10.31 (4)

or alternatively

L(Y;- mx;- c)

2

10.31 (5)

n(n- 2)

The weighted variance is given by 10.31(6)

Standard Deviation um of the Gradient m of the Mean Line 10.32

The value of m for each line through that is

x, y is given by 10.32( 1)

X;-X

Now the mean value xw of a set of values X;, each of weight W;, is given by '\'nl W·X· ~ l l X==-w '\'n L...l W;

Thus the mean value of the gradient for n lines through

10.32(2)

x, y is given by 10.32(3)

Now we know that 10.29( 1) and in order that these two equations should be identical 10.32( 4)

305

M et hod of Least Squares

Now the estimate of the standard deviation in the mean a, of a set of quantities zi is given by 10.32(5) where q =number of dependent variables, that is m and 2

srn =

x. Thus

I,~ af{'r:jei- m) 2 (n-2)'I,~a?

10.32(6) since the mean gradient is equal to m= (z)w and zi may be written

s2 m

L~(y;-mxi-c) 2

=-=-------'---

(n-2)I~(xi-x) 2

= Tjei.

Alternatively a~

10.32(7)

The weighted estimate of the standard deviation of the weighted mean gradient mw is given by 10.32(8)

Alternative Derivation for Standard Deviation a m of the Gradient m of the Mean Line 10.33

10.29(1) Since the ei are accurately known, and the errors are confined to the 'i• we may write the standard deviation of m in terms of the standard deviation of the Ti. Thus 10.33(1) from n

(j)p)2

s~ =Is? 1

iJxi

8.04(4)

Uncertainty, Calibration and Probability

306 Thus

10.33(2) since 10.33(3) Now

since ally deviations are assumed to have the same standard deviation. Thus 8z m

as in 8.32(6).

= s;LLt ef _ s;L _ L'l (t;- meY t Cl:'le?) 2 -L'lef-(n-2)L'lef

10.33(4)

s;L is given by 10.31(2). Standard Deviation of Intercept c

10.34

Now

.Y=mx+c

10.29(5)

since x and y are on the mean line. Now y, m and c can be considered as variables, but since xis considered to be known accurately it can be considered to be a constant. From 8.04( 4) szF

i=n

= "' L.,

(oF)2

szX· -o:,

8.04(4)

c=y-mx

10.34(1)

i~ 1

·

uxi

Now we write and putting F

=c we have 2

Se

(ac)

= oy

2

2

Syf

(ac)

+ om

2

2

Sml

10.34(2)

t For confidence and tolerance limits of y for given x, that is for ylx for a mean line, see Appendix V. tIt should be noted that sp 1 and sm are uncorrelated. Formula 8.04(4) can only be used if the sx, are uncorrelated.

307

Method of Least Squares Now

ac = 1

ac am

and

ap

-=

-

-x

10.34(3)

therefore s c2

= =

+i

s?yl

10.34( 4)

2s m 2

'\'i=n ( )2 ~i= 1 1 + rl I. rr - 2("'I c:;rY !'I c:?}

--'--='-------'=--'-----=----=-----'-

(n- 2)

(~:>r

+ r 2 I. rf (n-

2r 2 I. rf)

2)

using 10.44( 3)

I.rf(l-r 2 ) n-2 =

from 10.75(3)

s;(l - r 2 )

10.76(3)

from 10.75(3). Similarly 10.76(4)

s~L = s~(l - r 2 )

The weighted standard deviations s;Lw and s~Lw are given by n

(n- 2)

10.76(5)

and 10.76(6) Similarly s~L = s~dn, etc.

10.76(7)

The weighted values of s~w and s;w are given by s

2

xw

=

L~ wic:f

L wi

n

·--

(n- 2)

10.76(8)

Uncertainty, Calibration and Probability

330 and

s

"n

2

yw

n = L..l W(t'; · - 2

LW;

10.76(9)

(n- 2)

Analogously to 10.76(3) and 10.76(4) 10.76(10) and 10.76(11) Note that 10.76(12) and 10.76(13)

n 10.77

It should be noted that if

ax1

= 0, then

ay1

=

ayL

and the line of

regression is

y-

a .X) y = r____l'(x-

10.44( 4)

(]"X

If 0" yl = 0 then

0" xl

=

0" xL

and the line Of regression is 1 O"y

y- y =- -(x- x) r ax

10.45( 1)

If ay1 =I 0 and ax1 =I 0 then the mean line is given by 10.56(1), (2), (3) and (4) or by solving 10.52( 1), which yields two solutions, a best line and worst one.

Note on the Value of r 10.78

Now 10.44(3)

but nothing has so far been said about the values that r can assume. From a perusal of 10.69( 1) it would seem likely that r could reasonably go from 0 to ± 1, but the limits of ± 1 are not obvious. The proof that the limits of r are plus or minus unity is as follows.

331

Method of Least Squares

10.79 Consider

(a 1 b2 - a2 bd 2 = afb~ + a~bf- 2a 1 a2 b1 b2 whence s=n r=n

L L (a,b.- a.bY = 2LLa;b;- 2LLa,b,a.b.

(r =I= s)

s=1 r=1

therefore n

n

n

n

n

n

1

1

1

1

1

1

L La;b; = L La,b,a.h. + L L(a,b.- a.b,) 2 /2 r =1= s

10.79(1)

in any term, and thus 10.79(2)

Now n

n

n

n

n

1

1

1

1

1

La; Lb; = La;b; + L La;h;

(r=l=s)

10.79(3)

Also

(La,b,)

n

2

=

La;h; + LLa,b,a.b.

10.79(4)

1

Thus using 10.79(2) we see that

n ( n fa;Lb;;:: fa,b, )2

10.79(5)

or 10.79(6) Thus, comparing with 10.44(3) we see that the right-hand side of 10.79(6) is equivalent to r 2 • Therefore 1 ;:: r 2 or 1 ;:: r ;:: - 1.

Non-linear Correlation and Best Fit for a Parabola 10.80 The method used to find the coefficient of linear correlation given in paragraphs 10.27 to 10.29 and 10.44 can be immediately extended to the case of parabolic correlation. If

, x-x

X=-(Jx

and

y ' =y-y -(Jy

10.80( 1)

332

Uncertainty, Calibration and Probability

where n

x = IxJn,

10.80(2)

and n

(}~

n

= I(x;- x) 2 /n,

O";

1

= I(Y;- yfjn

10.80(3)

1

then the coefficient of parabolic correlation is given by

)0

in the equation 10.80( 4)

where this equation represents the best fit of a parabola to the points (x'1, y'1 ), (x~, y~), ... , (x~, y~). 10.81 A is thus found using the method of least squares, that is it is that value of Awhich makes I (y; - A.x? )2 a minimum, and therefore differentiating with respect to A we have 10.81 ( 1) Therefore

zi(x;- x?(Y;- y) O"yi(x;-x)4

10.81(2)

=O"x

I(x;- x?I(x;- x)z(Y;- y) {ni(Y;- y)2}1/2I(x;- x)4

10.81(3)

If the readings are weighted, then the sum of the squares of the deviations from the required parabola that are to be weighted is given by I w;(y; - Ax? f Differentiating as before leads to a value for A given by

O"~w

L w;(x; -

Xw) 2 (Y; - Yw)

O"ywiw;(x;- xw)4

10.81 (4)

where now '

xi~

Xw

X;w=--O"xw

and

1

Yi- Yw

Yiw=---

O"yw

10.81(5)

M et hod of Least Squares

333

and

"w·X· L. I l

-

10.30(3)

Xw=~ L.. W;

"w-y.

L. l l y= --

w

10.30(4)

LW;

L W;(X; -

Xw) 2

LW; 2

(Jyw

L W;(Y; ~ Yw) 2 LW;

10.81(6) 10.81 (7)

Usually estimates of Sxw and Syw will have to be used in place of axw and and these are given, respectively, by

s

2

xw

s

=

2

yw

LW;(X;- xw) 2 . n t ·-LW; (n-3) =

L W;(Y; -

Yw)2

LW;

n n- 3

·--

ayw

10.81(8)

10.81(9)

The estimated standard deviation in the mean of Y; from the mean parabola is given by

10.81(10) Unweighted Values

If the readings are unweighted, that is belong to the same population, then 2

S -L y

=

La;(y;- Ax?)2

=---'------'---

n(n-3)

L {Y;- y- A(ay/a;)(x;- .X) 2 } 2§ n(n- 3)

10.81(11)

t The factor n - 3 is used because three constants are involved for a parabola. tSee 10.81(13). §See 10.81(14).

Uncertainty, Calibration and Probability

334

and the standard deviation of a reading from the line is given by

L u;(y; -

A.xi)2 (n- 3)

2

s -

=--'-----

yL-

L {Y;- y- A.(uy/O"~)(x- x) 2 } 2 t (n- 3)

10.81(12)

In this case y;, x; and A. are given respectively by 10.80(1), 10.81(2), and ux and aY are replaced by their estimates

n n- 3

8 2 =--o-2

and

Y

Y

in 10.81(12) and in A. (10.81(2)). It is to be noted that A O"yw =A.' Syw = w2 w2 O"xw Sxw

L W;(X;- Xw) 2(Y;- Yw) " ( -)4 L... W; X;- Xw

10.81 ( 13)

where Aw contains O"xw and O"yw and A.~ contains sxw and syw· Similarly A. uy

= A.' sy =I (x;- x) 2(y;- y)

.,.~

s~

L: (x; - x) 4

10.81(14)

where A. contains ux and aY and A' contains sx and sY. The interpretation of A. is quite different from r in the case of linear correlation.

Alternative Derivation of Best Fit for a Parabola 10.82 The general equation for a parabola with vertical axis is

y =a+ bx

+ cx 2

10.82( 1)

If n pairs of measurements are taken of x and y, where n is greater than three, then the method of least squares gives the best fit for a parabola. Let then pairs of measurements be (x 1 , y 1 ); (x 2 , y 2 ); ... ; (xn, Yn) and let us assume that all the errors lie in the y values. Thus the deviation of the y value from the least squares parabola value of y for any point is given by

by;

=

Y; - a - bx; - cxf

10.82(2)

The least squares parabola is thus obtained by solving the three equations

o oa tSee 10.81(14).

=

o ob

= o(L: byf> = 0 oc

10.82(3)

Method of Least Squares

335

Least Squares Fit for General Function 10.83

A general function of x can be written as 10.83( 1)

where a 1 , a 2 , ••• , a. are constants. If the values are assumed correct the error or deviation of any point x,, y, from the best least squares fit curve is given by Dyi = Yi- f(xi,a 1 ,a 2 , ... ,a.). Thus the best least squares fit curve is obtained by solving the equations

aci yf) = aci by?) = ... = aci by?) = 0 oal

oa2

oa.

10.83(2)

for a 1 , a 2 , ••• , a•. The determination of the precision of the constants so obtained is found in a similar way to those obtained for the straight line (see paragraphs 10.31 to 10.36 ).

Criterion for Choice of Best Curve or Functional Relation to Fit Given Data Points 10.84 Let the two relations to be compared be cp(x, a 1, a 2, ... , ap) and .f(x, a 1 , a 2 , .•• , aq) where cp hasp constants and f has q constants. As before, let all the errors be assumed to be in the y values. Thus the error

10.84( 1) and the error 10.84(2)

DJYs = Ys- f(x.)

The variance of the Yi from the best fit line cp(xJ is thus given by sJ..

=

ri. r =I

(D"'yy = nq> - p

ri• r =I

{Yi- cp(xJY n