3 <meta http-equiv=
"Content-Type" content=
"text/html; charset=US-ASCII">
4 <title>Testing a sample mean for difference from a
"true" mean
</title>
5 <link rel=
"stylesheet" href=
"../../../../math.css" type=
"text/css">
6 <meta name=
"generator" content=
"DocBook XSL Stylesheets V1.77.1">
7 <link rel=
"home" href=
"../../../../index.html" title=
"Math Toolkit 2.5.1">
8 <link rel=
"up" href=
"../st_eg.html" title=
"Student's t Distribution Examples">
9 <link rel=
"prev" href=
"tut_mean_intervals.html" title=
"Calculating confidence intervals on the mean with the Students-t distribution">
10 <link rel=
"next" href=
"tut_mean_size.html" title=
"Estimating how large a sample size would have to become in order to give a significant Students-t test result with a single sample test">
12 <body bgcolor=
"white" text=
"black" link=
"#0000FF" vlink=
"#840084" alink=
"#0000FF">
13 <table cellpadding=
"2" width=
"100%"><tr>
14 <td valign=
"top"><img alt=
"Boost C++ Libraries" width=
"277" height=
"86" src=
"../../../../../../../../boost.png"></td>
15 <td align=
"center"><a href=
"../../../../../../../../index.html">Home
</a></td>
16 <td align=
"center"><a href=
"../../../../../../../../libs/libraries.htm">Libraries
</a></td>
17 <td align=
"center"><a href=
"http://www.boost.org/users/people.html">People
</a></td>
18 <td align=
"center"><a href=
"http://www.boost.org/users/faq.html">FAQ
</a></td>
19 <td align=
"center"><a href=
"../../../../../../../../more/index.htm">More
</a></td>
22 <div class=
"spirit-nav">
23 <a accesskey=
"p" href=
"tut_mean_intervals.html"><img src=
"../../../../../../../../doc/src/images/prev.png" alt=
"Prev"></a><a accesskey=
"u" href=
"../st_eg.html"><img src=
"../../../../../../../../doc/src/images/up.png" alt=
"Up"></a><a accesskey=
"h" href=
"../../../../index.html"><img src=
"../../../../../../../../doc/src/images/home.png" alt=
"Home"></a><a accesskey=
"n" href=
"tut_mean_size.html"><img src=
"../../../../../../../../doc/src/images/next.png" alt=
"Next"></a>
26 <div class=
"titlepage"><div><div><h5 class=
"title">
27 <a name=
"math_toolkit.stat_tut.weg.st_eg.tut_mean_test"></a><a class=
"link" href=
"tut_mean_test.html" title='Testing a sample mean for difference from a
"true" mean'
>Testing
28 a sample mean for difference from a
"true" mean
</a>
29 </h5></div></div></div>
31 When calibrating or comparing a scientific instrument or measurement
32 method of some kind, we want to be answer the question
"Does an
33 observed sample mean differ from the "true
" mean in any significant
34 way?". If it does, then we have evidence of a systematic difference.
35 This question can be answered with a Students-t test: more information
36 can be found
<a href=
"http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target=
"_top">on
40 Of course, the assignment of
"true" to one mean may be quite
41 arbitrary, often this is simply a
"traditional" method of measurement.
44 The following example code is taken from the example program
<a href=
"../../../../../../example/students_t_single_sample.cpp" target=
"_top">students_t_single_sample.cpp
</a>.
47 We'll begin by defining a procedure to determine which of the possible
48 hypothesis are rejected or not-rejected at a given significance level:
50 <div class=
"note"><table border=
"0" summary=
"Note">
52 <td rowspan=
"2" align=
"center" valign=
"top" width=
"25"><img alt=
"[Note]" src=
"../../../../../../../../doc/src/images/note.png"></td>
53 <th align=
"left">Note
</th>
55 <tr><td align=
"left" valign=
"top"><p>
56 Non-statisticians might say 'not-rejected' means 'accepted', (often
57 of the null-hypothesis) implying, wrongly, that there really
<span class=
"bold"><strong>IS
</strong></span> no difference, but statisticans eschew this
58 to avoid implying that there is positive evidence of 'no difference'.
59 'Not-rejected' here means there is
<span class=
"bold"><strong>no evidence
</strong></span>
60 of difference, but there still might well be a difference. For example,
61 see
<a href=
"http://en.wikipedia.org/wiki/Argument_from_ignorance" target=
"_top">argument
62 from ignorance
</a> and
<a href=
"http://www.bmj.com/cgi/content/full/311/7003/485" target=
"_top">Absence
63 of evidence does not constitute evidence of absence.
</a>
66 <pre class=
"programlisting"><span class=
"comment">// Needed includes:
</span>
67 <span class=
"preprocessor">#include
</span> <span class=
"special"><</span><span class=
"identifier">boost
</span><span class=
"special">/
</span><span class=
"identifier">math
</span><span class=
"special">/
</span><span class=
"identifier">distributions
</span><span class=
"special">/
</span><span class=
"identifier">students_t
</span><span class=
"special">.
</span><span class=
"identifier">hpp
</span><span class=
"special">></span>
68 <span class=
"preprocessor">#include
</span> <span class=
"special"><</span><span class=
"identifier">iostream
</span><span class=
"special">></span>
69 <span class=
"preprocessor">#include
</span> <span class=
"special"><</span><span class=
"identifier">iomanip
</span><span class=
"special">></span>
70 <span class=
"comment">// Bring everything into global namespace for ease of use:
</span>
71 <span class=
"keyword">using
</span> <span class=
"keyword">namespace
</span> <span class=
"identifier">boost
</span><span class=
"special">::
</span><span class=
"identifier">math
</span><span class=
"special">;
</span>
72 <span class=
"keyword">using
</span> <span class=
"keyword">namespace
</span> <span class=
"identifier">std
</span><span class=
"special">;
</span>
74 <span class=
"keyword">void
</span> <span class=
"identifier">single_sample_t_test
</span><span class=
"special">(
</span><span class=
"keyword">double
</span> <span class=
"identifier">M
</span><span class=
"special">,
</span> <span class=
"keyword">double
</span> <span class=
"identifier">Sm
</span><span class=
"special">,
</span> <span class=
"keyword">double
</span> <span class=
"identifier">Sd
</span><span class=
"special">,
</span> <span class=
"keyword">unsigned
</span> <span class=
"identifier">Sn
</span><span class=
"special">,
</span> <span class=
"keyword">double
</span> <span class=
"identifier">alpha
</span><span class=
"special">)
</span>
75 <span class=
"special">{
</span>
76 <span class=
"comment">//
</span>
77 <span class=
"comment">// M = true mean.
</span>
78 <span class=
"comment">// Sm = Sample Mean.
</span>
79 <span class=
"comment">// Sd = Sample Standard Deviation.
</span>
80 <span class=
"comment">// Sn = Sample Size.
</span>
81 <span class=
"comment">// alpha = Significance Level.
</span>
84 Most of the procedure is pretty-printing, so let's just focus on the
85 calculation, we begin by calculating the t-statistic:
87 <pre class=
"programlisting"><span class=
"comment">// Difference in means:
</span>
88 <span class=
"keyword">double
</span> <span class=
"identifier">diff
</span> <span class=
"special">=
</span> <span class=
"identifier">Sm
</span> <span class=
"special">-
</span> <span class=
"identifier">M
</span><span class=
"special">;
</span>
89 <span class=
"comment">// Degrees of freedom:
</span>
90 <span class=
"keyword">unsigned
</span> <span class=
"identifier">v
</span> <span class=
"special">=
</span> <span class=
"identifier">Sn
</span> <span class=
"special">-
</span> <span class=
"number">1</span><span class=
"special">;
</span>
91 <span class=
"comment">// t-statistic:
</span>
92 <span class=
"keyword">double
</span> <span class=
"identifier">t_stat
</span> <span class=
"special">=
</span> <span class=
"identifier">diff
</span> <span class=
"special">*
</span> <span class=
"identifier">sqrt
</span><span class=
"special">(
</span><span class=
"keyword">double
</span><span class=
"special">(
</span><span class=
"identifier">Sn
</span><span class=
"special">))
</span> <span class=
"special">/
</span> <span class=
"identifier">Sd
</span><span class=
"special">;
</span>
95 Finally calculate the probability from the t-statistic. If we're interested
96 in simply whether there is a difference (either less or greater) or not,
97 we don't care about the sign of the t-statistic, and we take the complement
98 of the probability for comparison to the significance level:
100 <pre class=
"programlisting"><span class=
"identifier">students_t
</span> <span class=
"identifier">dist
</span><span class=
"special">(
</span><span class=
"identifier">v
</span><span class=
"special">);
</span>
101 <span class=
"keyword">double
</span> <span class=
"identifier">q
</span> <span class=
"special">=
</span> <span class=
"identifier">cdf
</span><span class=
"special">(
</span><span class=
"identifier">complement
</span><span class=
"special">(
</span><span class=
"identifier">dist
</span><span class=
"special">,
</span> <span class=
"identifier">fabs
</span><span class=
"special">(
</span><span class=
"identifier">t_stat
</span><span class=
"special">)));
</span>
104 The procedure then prints out the results of the various tests that can
105 be done, these can be summarised in the following table:
107 <div class=
"informaltable"><table class=
"table">
128 The Null-hypothesis: there is
<span class=
"bold"><strong>no difference
</strong></span>
134 Reject if complement of CDF for |t|
< significance level
138 <code class=
"computeroutput"><span class=
"identifier">cdf
</span><span class=
"special">(
</span><span class=
"identifier">complement
</span><span class=
"special">(
</span><span class=
"identifier">dist
</span><span class=
"special">,
</span>
139 <span class=
"identifier">fabs
</span><span class=
"special">(
</span><span class=
"identifier">t
</span><span class=
"special">)))
</span>
140 <span class=
"special"><</span> <span class=
"identifier">alpha
</span>
141 <span class=
"special">/
</span> <span class=
"number">2</span></code>
148 The Alternative-hypothesis: there
<span class=
"bold"><strong>is
149 difference
</strong></span> in means
154 Reject if complement of CDF for |t|
> significance level
158 <code class=
"computeroutput"><span class=
"identifier">cdf
</span><span class=
"special">(
</span><span class=
"identifier">complement
</span><span class=
"special">(
</span><span class=
"identifier">dist
</span><span class=
"special">,
</span>
159 <span class=
"identifier">fabs
</span><span class=
"special">(
</span><span class=
"identifier">t
</span><span class=
"special">)))
</span>
160 <span class=
"special">></span> <span class=
"identifier">alpha
</span>
161 <span class=
"special">/
</span> <span class=
"number">2</span></code>
168 The Alternative-hypothesis: the sample mean
<span class=
"bold"><strong>is
169 less
</strong></span> than the true mean.
174 Reject if CDF of t
> 1 - significance level:
177 <code class=
"computeroutput"><span class=
"identifier">cdf
</span><span class=
"special">(
</span><span class=
"identifier">complement
</span><span class=
"special">(
</span><span class=
"identifier">dist
</span><span class=
"special">,
</span>
178 <span class=
"identifier">t
</span><span class=
"special">))
</span>
179 <span class=
"special"><</span> <span class=
"identifier">alpha
</span></code>
186 The Alternative-hypothesis: the sample mean
<span class=
"bold"><strong>is
187 greater
</strong></span> than the true mean.
192 Reject if complement of CDF of t
< significance level:
195 <code class=
"computeroutput"><span class=
"identifier">cdf
</span><span class=
"special">(
</span><span class=
"identifier">dist
</span><span class=
"special">,
</span>
196 <span class=
"identifier">t
</span><span class=
"special">)
</span>
197 <span class=
"special"><</span> <span class=
"identifier">alpha
</span></code>
203 <div class=
"note"><table border=
"0" summary=
"Note">
205 <td rowspan=
"2" align=
"center" valign=
"top" width=
"25"><img alt=
"[Note]" src=
"../../../../../../../../doc/src/images/note.png"></td>
206 <th align=
"left">Note
</th>
208 <tr><td align=
"left" valign=
"top"><p>
209 Notice that the comparisons are against
<code class=
"computeroutput"><span class=
"identifier">alpha
</span>
210 <span class=
"special">/
</span> <span class=
"number">2</span></code>
211 for a two-sided test and against
<code class=
"computeroutput"><span class=
"identifier">alpha
</span></code>
216 Now that we have all the parts in place, let's take a look at some sample
217 output, first using the
<a href=
"http://www.itl.nist.gov/div898/handbook/eda/section4/eda428.htm" target=
"_top">Heat
218 flow data
</a> from the NIST site. The data set was collected by Bob
219 Zarr of NIST in January,
1990 from a heat flow meter calibration and
220 stability analysis. The corresponding dataplot output for this test can
221 be found in
<a href=
"http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target=
"_top">section
222 3.5.2</a> of the
<a href=
"http://www.itl.nist.gov/div898/handbook/" target=
"_top">NIST/SEMATECH
223 e-Handbook of Statistical Methods.
</a>.
225 <pre class=
"programlisting">__________________________________
226 Student t test for a single sample
227 __________________________________
229 Number of Observations =
195
230 Sample Mean =
9.26146
231 Sample Standard Deviation =
0.02279
232 Expected True Mean =
5.00000
234 Sample Mean - Expected Test Mean =
4.26146
235 Degrees of Freedom =
194
236 T Statistic =
2611.28380
237 Probability that difference is due to chance =
0.000e+000
239 Results for Alternative Hypothesis and alpha =
0.0500
241 Alternative Hypothesis Conclusion
242 Mean !=
5.000 NOT REJECTED
243 Mean
< 5.000 REJECTED
244 Mean
> 5.000 NOT REJECTED
247 You will note the line that says the probability that the difference
248 is due to chance is zero. From a philosophical point of view, of course,
249 the probability can never reach zero. However, in this case the calculated
250 probability is smaller than the smallest representable double precision
251 number, hence the appearance of a zero here. Whatever its
"true"
252 value is, we know it must be extraordinarily small, so the alternative
253 hypothesis - that there is a difference in means - is not rejected.
256 For comparison the next example data output is taken from
<span class=
"emphasis"><em>P.K.Hou,
257 O. W. Lau
& M.C. Wong, Analyst (
1983) vol.
108, p
64. and from Statistics
258 for Analytical Chemistry,
3rd ed. (
1994), pp
54-
55 J. C. Miller and J.
259 N. Miller, Ellis Horwood ISBN
0 13 0309907.
</em></span> The values result
260 from the determination of mercury by cold-vapour atomic absorption.
262 <pre class=
"programlisting">__________________________________
263 Student t test for a single sample
264 __________________________________
266 Number of Observations =
3
267 Sample Mean =
37.80000
268 Sample Standard Deviation =
0.96437
269 Expected True Mean =
38.90000
271 Sample Mean - Expected Test Mean = -
1.10000
272 Degrees of Freedom =
2
273 T Statistic = -
1.97566
274 Probability that difference is due to chance =
1.869e-001
276 Results for Alternative Hypothesis and alpha =
0.0500
278 Alternative Hypothesis Conclusion
279 Mean !=
38.900 REJECTED
280 Mean
< 38.900 NOT REJECTED
281 Mean
> 38.900 NOT REJECTED
284 As you can see the small number of measurements (
3) has led to a large
285 uncertainty in the location of the true mean. So even though there appears
286 to be a difference between the sample mean and the expected true mean,
287 we conclude that there is no significant difference, and are unable to
288 reject the null hypothesis. However, if we were to lower the bar for
289 acceptance down to alpha =
0.1 (a
90% confidence level) we see a different
292 <pre class=
"programlisting">__________________________________
293 Student t test for a single sample
294 __________________________________
296 Number of Observations =
3
297 Sample Mean =
37.80000
298 Sample Standard Deviation =
0.96437
299 Expected True Mean =
38.90000
301 Sample Mean - Expected Test Mean = -
1.10000
302 Degrees of Freedom =
2
303 T Statistic = -
1.97566
304 Probability that difference is due to chance =
1.869e-001
306 Results for Alternative Hypothesis and alpha =
0.1000
308 Alternative Hypothesis Conclusion
309 Mean !=
38.900 REJECTED
310 Mean
< 38.900 NOT REJECTED
311 Mean
> 38.900 REJECTED
314 In this case, we really have a borderline result, and more data (and/or
315 more accurate data), is needed for a more convincing conclusion.
318 <table xmlns:
rev=
"http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width=
"100%"><tr>
319 <td align=
"left"></td>
320 <td align=
"right"><div class=
"copyright-footer">Copyright
© 2006-
2010,
2012-
2014 Nikhar Agrawal,
321 Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos, Hubert
322 Holin, Bruno Lalande, John Maddock, Jeremy Murphy, Johan R
åde, Gautam Sewani,
323 Benjamin Sobotta, Thijs van den Berg, Daryle Walker and Xiaogang Zhang
<p>
324 Distributed under the Boost Software License, Version
1.0. (See accompanying
325 file LICENSE_1_0.txt or copy at
<a href=
"http://www.boost.org/LICENSE_1_0.txt" target=
"_top">http://www.boost.org/LICENSE_1_0.txt
</a>)
330 <div class=
"spirit-nav">
331 <a accesskey=
"p" href=
"tut_mean_intervals.html"><img src=
"../../../../../../../../doc/src/images/prev.png" alt=
"Prev"></a><a accesskey=
"u" href=
"../st_eg.html"><img src=
"../../../../../../../../doc/src/images/up.png" alt=
"Up"></a><a accesskey=
"h" href=
"../../../../index.html"><img src=
"../../../../../../../../doc/src/images/home.png" alt=
"Home"></a><a accesskey=
"n" href=
"tut_mean_size.html"><img src=
"../../../../../../../../doc/src/images/next.png" alt=
"Next"></a>