]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | <html> |
2 | <head> | |
3 | <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII"> | |
4 | <title>Testing a sample mean for difference from a "true" mean</title> | |
5 | <link rel="stylesheet" href="../../../../math.css" type="text/css"> | |
6 | <meta name="generator" content="DocBook XSL Stylesheets V1.77.1"> | |
7 | <link rel="home" href="../../../../index.html" title="Math Toolkit 2.5.1"> | |
8 | <link rel="up" href="../st_eg.html" title="Student's t Distribution Examples"> | |
9 | <link rel="prev" href="tut_mean_intervals.html" title="Calculating confidence intervals on the mean with the Students-t distribution"> | |
10 | <link rel="next" href="tut_mean_size.html" title="Estimating how large a sample size would have to become in order to give a significant Students-t test result with a single sample test"> | |
11 | </head> | |
12 | <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> | |
13 | <table cellpadding="2" width="100%"><tr> | |
14 | <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td> | |
15 | <td align="center"><a href="../../../../../../../../index.html">Home</a></td> | |
16 | <td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td> | |
17 | <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> | |
18 | <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> | |
19 | <td align="center"><a href="../../../../../../../../more/index.htm">More</a></td> | |
20 | </tr></table> | |
21 | <hr> | |
22 | <div class="spirit-nav"> | |
23 | <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> | |
24 | </div> | |
25 | <div class="section"> | |
26 | <div class="titlepage"><div><div><h5 class="title"> | |
27 | <a name="math_toolkit.stat_tut.weg.st_eg.tut_mean_test"></a><a class="link" href="tut_mean_test.html" title='Testing a sample mean for difference from a "true" mean'>Testing | |
28 | a sample mean for difference from a "true" mean</a> | |
29 | </h5></div></div></div> | |
30 | <p> | |
31 | When calibrating or comparing a scientific instrument or measurement | |
32 | method of some kind, we want to be answer the question "Does an | |
33 | observed sample mean differ from the "true" mean in any significant | |
34 | way?". If it does, then we have evidence of a systematic difference. | |
35 | This question can be answered with a Students-t test: more information | |
36 | can be found <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">on | |
37 | the NIST site</a>. | |
38 | </p> | |
39 | <p> | |
40 | Of course, the assignment of "true" to one mean may be quite | |
41 | arbitrary, often this is simply a "traditional" method of measurement. | |
42 | </p> | |
43 | <p> | |
44 | The following example code is taken from the example program <a href="../../../../../../example/students_t_single_sample.cpp" target="_top">students_t_single_sample.cpp</a>. | |
45 | </p> | |
46 | <p> | |
47 | We'll begin by defining a procedure to determine which of the possible | |
48 | hypothesis are rejected or not-rejected at a given significance level: | |
49 | </p> | |
50 | <div class="note"><table border="0" summary="Note"> | |
51 | <tr> | |
52 | <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td> | |
53 | <th align="left">Note</th> | |
54 | </tr> | |
55 | <tr><td align="left" valign="top"><p> | |
56 | Non-statisticians might say 'not-rejected' means 'accepted', (often | |
57 | of the null-hypothesis) implying, wrongly, that there really <span class="bold"><strong>IS</strong></span> no difference, but statisticans eschew this | |
58 | to avoid implying that there is positive evidence of 'no difference'. | |
59 | 'Not-rejected' here means there is <span class="bold"><strong>no evidence</strong></span> | |
60 | of difference, but there still might well be a difference. For example, | |
61 | see <a href="http://en.wikipedia.org/wiki/Argument_from_ignorance" target="_top">argument | |
62 | from ignorance</a> and <a href="http://www.bmj.com/cgi/content/full/311/7003/485" target="_top">Absence | |
63 | of evidence does not constitute evidence of absence.</a> | |
64 | </p></td></tr> | |
65 | </table></div> | |
66 | <pre class="programlisting"><span class="comment">// Needed includes:</span> | |
67 | <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">math</span><span class="special">/</span><span class="identifier">distributions</span><span class="special">/</span><span class="identifier">students_t</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> | |
68 | <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> | |
69 | <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iomanip</span><span class="special">></span> | |
70 | <span class="comment">// Bring everything into global namespace for ease of use:</span> | |
71 | <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">math</span><span class="special">;</span> | |
72 | <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">std</span><span class="special">;</span> | |
73 | ||
74 | <span class="keyword">void</span> <span class="identifier">single_sample_t_test</span><span class="special">(</span><span class="keyword">double</span> <span class="identifier">M</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sm</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sd</span><span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">Sn</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">alpha</span><span class="special">)</span> | |
75 | <span class="special">{</span> | |
76 | <span class="comment">//</span> | |
77 | <span class="comment">// M = true mean.</span> | |
78 | <span class="comment">// Sm = Sample Mean.</span> | |
79 | <span class="comment">// Sd = Sample Standard Deviation.</span> | |
80 | <span class="comment">// Sn = Sample Size.</span> | |
81 | <span class="comment">// alpha = Significance Level.</span> | |
82 | </pre> | |
83 | <p> | |
84 | Most of the procedure is pretty-printing, so let's just focus on the | |
85 | calculation, we begin by calculating the t-statistic: | |
86 | </p> | |
87 | <pre class="programlisting"><span class="comment">// Difference in means:</span> | |
88 | <span class="keyword">double</span> <span class="identifier">diff</span> <span class="special">=</span> <span class="identifier">Sm</span> <span class="special">-</span> <span class="identifier">M</span><span class="special">;</span> | |
89 | <span class="comment">// Degrees of freedom:</span> | |
90 | <span class="keyword">unsigned</span> <span class="identifier">v</span> <span class="special">=</span> <span class="identifier">Sn</span> <span class="special">-</span> <span class="number">1</span><span class="special">;</span> | |
91 | <span class="comment">// t-statistic:</span> | |
92 | <span class="keyword">double</span> <span class="identifier">t_stat</span> <span class="special">=</span> <span class="identifier">diff</span> <span class="special">*</span> <span class="identifier">sqrt</span><span class="special">(</span><span class="keyword">double</span><span class="special">(</span><span class="identifier">Sn</span><span class="special">))</span> <span class="special">/</span> <span class="identifier">Sd</span><span class="special">;</span> | |
93 | </pre> | |
94 | <p> | |
95 | Finally calculate the probability from the t-statistic. If we're interested | |
96 | in simply whether there is a difference (either less or greater) or not, | |
97 | we don't care about the sign of the t-statistic, and we take the complement | |
98 | of the probability for comparison to the significance level: | |
99 | </p> | |
100 | <pre class="programlisting"><span class="identifier">students_t</span> <span class="identifier">dist</span><span class="special">(</span><span class="identifier">v</span><span class="special">);</span> | |
101 | <span class="keyword">double</span> <span class="identifier">q</span> <span class="special">=</span> <span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t_stat</span><span class="special">)));</span> | |
102 | </pre> | |
103 | <p> | |
104 | The procedure then prints out the results of the various tests that can | |
105 | be done, these can be summarised in the following table: | |
106 | </p> | |
107 | <div class="informaltable"><table class="table"> | |
108 | <colgroup> | |
109 | <col> | |
110 | <col> | |
111 | </colgroup> | |
112 | <thead><tr> | |
113 | <th> | |
114 | <p> | |
115 | Hypothesis | |
116 | </p> | |
117 | </th> | |
118 | <th> | |
119 | <p> | |
120 | Test | |
121 | </p> | |
122 | </th> | |
123 | </tr></thead> | |
124 | <tbody> | |
125 | <tr> | |
126 | <td> | |
127 | <p> | |
128 | The Null-hypothesis: there is <span class="bold"><strong>no difference</strong></span> | |
129 | in means | |
130 | </p> | |
131 | </td> | |
132 | <td> | |
133 | <p> | |
134 | Reject if complement of CDF for |t| < significance level | |
135 | / 2: | |
136 | </p> | |
137 | <p> | |
138 | <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> | |
139 | <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span> | |
140 | <span class="special"><</span> <span class="identifier">alpha</span> | |
141 | <span class="special">/</span> <span class="number">2</span></code> | |
142 | </p> | |
143 | </td> | |
144 | </tr> | |
145 | <tr> | |
146 | <td> | |
147 | <p> | |
148 | The Alternative-hypothesis: there <span class="bold"><strong>is | |
149 | difference</strong></span> in means | |
150 | </p> | |
151 | </td> | |
152 | <td> | |
153 | <p> | |
154 | Reject if complement of CDF for |t| > significance level | |
155 | / 2: | |
156 | </p> | |
157 | <p> | |
158 | <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> | |
159 | <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span> | |
160 | <span class="special">></span> <span class="identifier">alpha</span> | |
161 | <span class="special">/</span> <span class="number">2</span></code> | |
162 | </p> | |
163 | </td> | |
164 | </tr> | |
165 | <tr> | |
166 | <td> | |
167 | <p> | |
168 | The Alternative-hypothesis: the sample mean <span class="bold"><strong>is | |
169 | less</strong></span> than the true mean. | |
170 | </p> | |
171 | </td> | |
172 | <td> | |
173 | <p> | |
174 | Reject if CDF of t > 1 - significance level: | |
175 | </p> | |
176 | <p> | |
177 | <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> | |
178 | <span class="identifier">t</span><span class="special">))</span> | |
179 | <span class="special"><</span> <span class="identifier">alpha</span></code> | |
180 | </p> | |
181 | </td> | |
182 | </tr> | |
183 | <tr> | |
184 | <td> | |
185 | <p> | |
186 | The Alternative-hypothesis: the sample mean <span class="bold"><strong>is | |
187 | greater</strong></span> than the true mean. | |
188 | </p> | |
189 | </td> | |
190 | <td> | |
191 | <p> | |
192 | Reject if complement of CDF of t < significance level: | |
193 | </p> | |
194 | <p> | |
195 | <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> | |
196 | <span class="identifier">t</span><span class="special">)</span> | |
197 | <span class="special"><</span> <span class="identifier">alpha</span></code> | |
198 | </p> | |
199 | </td> | |
200 | </tr> | |
201 | </tbody> | |
202 | </table></div> | |
203 | <div class="note"><table border="0" summary="Note"> | |
204 | <tr> | |
205 | <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td> | |
206 | <th align="left">Note</th> | |
207 | </tr> | |
208 | <tr><td align="left" valign="top"><p> | |
209 | Notice that the comparisons are against <code class="computeroutput"><span class="identifier">alpha</span> | |
210 | <span class="special">/</span> <span class="number">2</span></code> | |
211 | for a two-sided test and against <code class="computeroutput"><span class="identifier">alpha</span></code> | |
212 | for a one-sided test | |
213 | </p></td></tr> | |
214 | </table></div> | |
215 | <p> | |
216 | Now that we have all the parts in place, let's take a look at some sample | |
217 | output, first using the <a href="http://www.itl.nist.gov/div898/handbook/eda/section4/eda428.htm" target="_top">Heat | |
218 | flow data</a> from the NIST site. The data set was collected by Bob | |
219 | Zarr of NIST in January, 1990 from a heat flow meter calibration and | |
220 | stability analysis. The corresponding dataplot output for this test can | |
221 | be found in <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">section | |
222 | 3.5.2</a> of the <a href="http://www.itl.nist.gov/div898/handbook/" target="_top">NIST/SEMATECH | |
223 | e-Handbook of Statistical Methods.</a>. | |
224 | </p> | |
225 | <pre class="programlisting">__________________________________ | |
226 | Student t test for a single sample | |
227 | __________________________________ | |
228 | ||
229 | Number of Observations = 195 | |
230 | Sample Mean = 9.26146 | |
231 | Sample Standard Deviation = 0.02279 | |
232 | Expected True Mean = 5.00000 | |
233 | ||
234 | Sample Mean - Expected Test Mean = 4.26146 | |
235 | Degrees of Freedom = 194 | |
236 | T Statistic = 2611.28380 | |
237 | Probability that difference is due to chance = 0.000e+000 | |
238 | ||
239 | Results for Alternative Hypothesis and alpha = 0.0500 | |
240 | ||
241 | Alternative Hypothesis Conclusion | |
242 | Mean != 5.000 NOT REJECTED | |
243 | Mean < 5.000 REJECTED | |
244 | Mean > 5.000 NOT REJECTED | |
245 | </pre> | |
246 | <p> | |
247 | You will note the line that says the probability that the difference | |
248 | is due to chance is zero. From a philosophical point of view, of course, | |
249 | the probability can never reach zero. However, in this case the calculated | |
250 | probability is smaller than the smallest representable double precision | |
251 | number, hence the appearance of a zero here. Whatever its "true" | |
252 | value is, we know it must be extraordinarily small, so the alternative | |
253 | hypothesis - that there is a difference in means - is not rejected. | |
254 | </p> | |
255 | <p> | |
256 | For comparison the next example data output is taken from <span class="emphasis"><em>P.K.Hou, | |
257 | O. W. Lau & M.C. Wong, Analyst (1983) vol. 108, p 64. and from Statistics | |
258 | for Analytical Chemistry, 3rd ed. (1994), pp 54-55 J. C. Miller and J. | |
259 | N. Miller, Ellis Horwood ISBN 0 13 0309907.</em></span> The values result | |
260 | from the determination of mercury by cold-vapour atomic absorption. | |
261 | </p> | |
262 | <pre class="programlisting">__________________________________ | |
263 | Student t test for a single sample | |
264 | __________________________________ | |
265 | ||
266 | Number of Observations = 3 | |
267 | Sample Mean = 37.80000 | |
268 | Sample Standard Deviation = 0.96437 | |
269 | Expected True Mean = 38.90000 | |
270 | ||
271 | Sample Mean - Expected Test Mean = -1.10000 | |
272 | Degrees of Freedom = 2 | |
273 | T Statistic = -1.97566 | |
274 | Probability that difference is due to chance = 1.869e-001 | |
275 | ||
276 | Results for Alternative Hypothesis and alpha = 0.0500 | |
277 | ||
278 | Alternative Hypothesis Conclusion | |
279 | Mean != 38.900 REJECTED | |
280 | Mean < 38.900 NOT REJECTED | |
281 | Mean > 38.900 NOT REJECTED | |
282 | </pre> | |
283 | <p> | |
284 | As you can see the small number of measurements (3) has led to a large | |
285 | uncertainty in the location of the true mean. So even though there appears | |
286 | to be a difference between the sample mean and the expected true mean, | |
287 | we conclude that there is no significant difference, and are unable to | |
288 | reject the null hypothesis. However, if we were to lower the bar for | |
289 | acceptance down to alpha = 0.1 (a 90% confidence level) we see a different | |
290 | output: | |
291 | </p> | |
292 | <pre class="programlisting">__________________________________ | |
293 | Student t test for a single sample | |
294 | __________________________________ | |
295 | ||
296 | Number of Observations = 3 | |
297 | Sample Mean = 37.80000 | |
298 | Sample Standard Deviation = 0.96437 | |
299 | Expected True Mean = 38.90000 | |
300 | ||
301 | Sample Mean - Expected Test Mean = -1.10000 | |
302 | Degrees of Freedom = 2 | |
303 | T Statistic = -1.97566 | |
304 | Probability that difference is due to chance = 1.869e-001 | |
305 | ||
306 | Results for Alternative Hypothesis and alpha = 0.1000 | |
307 | ||
308 | Alternative Hypothesis Conclusion | |
309 | Mean != 38.900 REJECTED | |
310 | Mean < 38.900 NOT REJECTED | |
311 | Mean > 38.900 REJECTED | |
312 | </pre> | |
313 | <p> | |
314 | In this case, we really have a borderline result, and more data (and/or | |
315 | more accurate data), is needed for a more convincing conclusion. | |
316 | </p> | |
317 | </div> | |
318 | <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> | |
319 | <td align="left"></td> | |
320 | <td align="right"><div class="copyright-footer">Copyright © 2006-2010, 2012-2014 Nikhar Agrawal, | |
321 | Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos, Hubert | |
322 | Holin, Bruno Lalande, John Maddock, Jeremy Murphy, Johan Råde, Gautam Sewani, | |
323 | Benjamin Sobotta, Thijs van den Berg, Daryle Walker and Xiaogang Zhang<p> | |
324 | Distributed under the Boost Software License, Version 1.0. (See accompanying | |
325 | file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) | |
326 | </p> | |
327 | </div></td> | |
328 | </tr></table> | |
329 | <hr> | |
330 | <div class="spirit-nav"> | |
331 | <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> | |
332 | </div> | |
333 | </body> | |
334 | </html> |