[ceph.git] / ceph / src / boost / libs / math / doc / distributions / f_dist_example.qbk

[section:f_eg F Distribution Examples]

Imagine that you want to compare the standard deviations of two
sample to determine if they differ in any significant way, in this
situation you use the F distribution and perform an F-test.  This
situation commonly occurs when conducting a process change comparison:
"is a new process more consistent that the old one?".

In this example we'll be using the data for ceramic strength from
[@http://www.itl.nist.gov/div898/handbook/eda/section4/eda42a1.htm
http://www.itl.nist.gov/div898/handbook/eda/section4/eda42a1.htm].
The data for this case study were collected by Said Jahanmir of the 
NIST Ceramics Division in 1996 in connection with a NIST/industry 
ceramics consortium for strength optimization of ceramic strength.

The example program is [@../../example/f_test.cpp f_test.cpp], 
program output has been deliberately made as similar as possible
to the DATAPLOT output in the corresponding 
[@http://www.itl.nist.gov/div898/handbook/eda/section3/eda359.htm
NIST EngineeringStatistics Handbook example].

We'll begin by defining the procedure to conduct the test:

   void f_test(
       double sd1,     // Sample 1 std deviation
       double sd2,     // Sample 2 std deviation
       double N1,      // Sample 1 size
       double N2,      // Sample 2 size
       double alpha)  // Significance level
   {

The procedure begins by printing out a summary of our input data:

   using namespace std;
   using namespace boost::math;

   // Print header:
   cout <<
      "____________________________________\n"
      "F test for equal standard deviations\n"
      "____________________________________\n\n";
   cout << setprecision(5);
   cout << "Sample 1:\n";
   cout << setw(55) << left << "Number of Observations" << "=  " << N1 << "\n";
   cout << setw(55) << left << "Sample Standard Deviation" << "=  " << sd1 << "\n\n";
   cout << "Sample 2:\n";
   cout << setw(55) << left << "Number of Observations" << "=  " << N2 << "\n";
   cout << setw(55) << left << "Sample Standard Deviation" << "=  " << sd2 << "\n\n";

The test statistic for an F-test is simply the ratio of the square of
the two standard deviations:

F = s[sub 1][super 2] / s[sub 2][super 2]

where s[sub 1] is the standard deviation of the first sample and s[sub 2]
is the standard deviation of the second sample.  Or in code:

   double F = (sd1 / sd2);
   F *= F;
   cout << setw(55) << left << "Test Statistic" << "=  " << F << "\n\n";

At this point a word of caution: the F distribution is asymmetric,
so we have to be careful how we compute the tests, the following table
summarises the options available:

[table
[[Hypothesis][Test]]
[[The null-hypothesis: there is no difference in standard deviations (two sided test)]
      [Reject if F <= F[sub (1-alpha/2; N1-1, N2-1)] or F >= F[sub (alpha/2; N1-1, N2-1)] ]]
[[The alternative hypothesis: there is a difference in means (two sided test)]
      [Reject if F[sub (1-alpha/2; N1-1, N2-1)] <= F <= F[sub (alpha/2; N1-1, N2-1)] ]]
[[The alternative hypothesis: Standard deviation of sample 1 is greater
than that of sample 2]
      [Reject if F < F[sub (alpha; N1-1, N2-1)] ]]
[[The alternative hypothesis: Standard deviation of sample 1 is less
than that of sample 2]
      [Reject if F > F[sub (1-alpha; N1-1, N2-1)] ]]
]

Where F[sub (1-alpha; N1-1, N2-1)] is the lower critical value of the F distribution
with degrees of freedom N1-1 and N2-1, and F[sub (alpha; N1-1, N2-1)] is the upper
critical value of the F distribution with degrees of freedom N1-1 and N2-1.

The upper and lower critical values can be computed using the quantile function:

F[sub (1-alpha; N1-1, N2-1)] = `quantile(fisher_f(N1-1, N2-1), alpha)`

F[sub (alpha; N1-1, N2-1)] = `quantile(complement(fisher_f(N1-1, N2-1), alpha))`

In our example program we need both upper and lower critical values for alpha
and for alpha/2:

   double ucv = quantile(complement(dist, alpha));
   double ucv2 = quantile(complement(dist, alpha / 2));
   double lcv = quantile(dist, alpha);
   double lcv2 = quantile(dist, alpha / 2);
   cout << setw(55) << left << "Upper Critical Value at alpha: " << "=  "
      << setprecision(3) << scientific << ucv << "\n";
   cout << setw(55) << left << "Upper Critical Value at alpha/2: " << "=  "
      << setprecision(3) << scientific << ucv2 << "\n";
   cout << setw(55) << left << "Lower Critical Value at alpha: " << "=  "
      << setprecision(3) << scientific << lcv << "\n";
   cout << setw(55) << left << "Lower Critical Value at alpha/2: " << "=  "
      << setprecision(3) << scientific << lcv2 << "\n\n";

The final step is to perform the comparisons given above, and print
out whether the hypothesis is rejected or not:

   cout << setw(55) << left <<
      "Results for Alternative Hypothesis and alpha" << "=  "
      << setprecision(4) << fixed << alpha << "\n\n";
   cout << "Alternative Hypothesis                                    Conclusion\n";
   
   cout << "Standard deviations are unequal (two sided test)          ";
   if((ucv2 < F) || (lcv2 > F))
      cout << "ACCEPTED\n";
   else
      cout << "REJECTED\n";
   
   cout << "Standard deviation 1 is less than standard deviation 2    ";
   if(lcv > F)
      cout << "ACCEPTED\n";
   else
      cout << "REJECTED\n";
   
   cout << "Standard deviation 1 is greater than standard deviation 2 ";
   if(ucv < F)
      cout << "ACCEPTED\n";
   else
      cout << "REJECTED\n";
   cout << endl << endl;

Using the ceramic strength data as an example we get the following
output:

[pre
'''F test for equal standard deviations
____________________________________

Sample 1:
Number of Observations                                 =  240
Sample Standard Deviation                              =  65.549

Sample 2:
Number of Observations                                 =  240
Sample Standard Deviation                              =  61.854

Test Statistic                                         =  1.123

CDF of test statistic:                                 =  8.148e-001
Upper Critical Value at alpha:                         =  1.238e+000
Upper Critical Value at alpha/2:                       =  1.289e+000
Lower Critical Value at alpha:                         =  8.080e-001
Lower Critical Value at alpha/2:                       =  7.756e-001

Results for Alternative Hypothesis and alpha           =  0.0500

Alternative Hypothesis                                    Conclusion
Standard deviations are unequal (two sided test)          REJECTED
Standard deviation 1 is less than standard deviation 2    REJECTED
Standard deviation 1 is greater than standard deviation 2 REJECTED'''
]

In this case we are unable to reject the null-hypothesis, and must instead
reject the alternative hypothesis.

By contrast let's see what happens when we use some different
[@http://www.itl.nist.gov/div898/handbook/prc/section3/prc32.htm 
sample data]:, once again from the NIST Engineering Statistics Handbook:
A new procedure to assemble a device is introduced and tested for
possible improvement in time of assembly. The question being addressed
is whether the standard deviation of the new assembly process (sample 2) is
better (i.e., smaller) than the standard deviation for the old assembly
process (sample 1).

[pre
'''____________________________________
F test for equal standard deviations
____________________________________

Sample 1:
Number of Observations                                 =  11.00000
Sample Standard Deviation                              =  4.90820

Sample 2:
Number of Observations                                 =  9.00000
Sample Standard Deviation                              =  2.58740

Test Statistic                                         =  3.59847

CDF of test statistic:                                 =  9.589e-001
Upper Critical Value at alpha:                         =  3.347e+000
Upper Critical Value at alpha/2:                       =  4.295e+000
Lower Critical Value at alpha:                         =  3.256e-001
Lower Critical Value at alpha/2:                       =  2.594e-001

Results for Alternative Hypothesis and alpha           =  0.0500

Alternative Hypothesis                                    Conclusion
Standard deviations are unequal (two sided test)          REJECTED
Standard deviation 1 is less than standard deviation 2    REJECTED
Standard deviation 1 is greater than standard deviation 2 ACCEPTED'''
]

In this case we take our null hypothesis as "standard deviation 1 is 
less than or equal to standard deviation 2", since this represents the "no change"
situation.  So we want to compare the upper critical value at /alpha/
(a one sided test) with the test statistic, and since 3.35 < 3.6 this
hypothesis must be rejected.  We therefore conclude that there is a change
for the better in our standard deviation.

[endsect][/section:f_eg F Distribution]

[/ 
  Copyright 2006 John Maddock and Paul A. Bristow.
  Distributed under the Boost Software License, Version 1.0.
  (See accompanying file LICENSE_1_0.txt or copy at
  http://www.boost.org/LICENSE_1_0.txt).
]
Commit	Line	Data
7c673cae FG	1	[section:f_eg F Distribution Examples]
	2
	3	Imagine that you want to compare the standard deviations of two
	4	sample to determine if they differ in any significant way, in this
	5	situation you use the F distribution and perform an F-test. This
	6	situation commonly occurs when conducting a process change comparison:
	7	"is a new process more consistent that the old one?".
	8
	9	In this example we'll be using the data for ceramic strength from
	10	[@http://www.itl.nist.gov/div898/handbook/eda/section4/eda42a1.htm
	11	http://www.itl.nist.gov/div898/handbook/eda/section4/eda42a1.htm].
	12	The data for this case study were collected by Said Jahanmir of the
	13	NIST Ceramics Division in 1996 in connection with a NIST/industry
	14	ceramics consortium for strength optimization of ceramic strength.
	15
	16	The example program is [@../../example/f_test.cpp f_test.cpp],
	17	program output has been deliberately made as similar as possible
	18	to the DATAPLOT output in the corresponding
	19	[@http://www.itl.nist.gov/div898/handbook/eda/section3/eda359.htm
	20	NIST EngineeringStatistics Handbook example].
	21
	22	We'll begin by defining the procedure to conduct the test:
	23
	24	void f_test(
	25	double sd1, // Sample 1 std deviation
	26	double sd2, // Sample 2 std deviation
	27	double N1, // Sample 1 size
	28	double N2, // Sample 2 size
	29	double alpha) // Significance level
	30	{
	31
	32	The procedure begins by printing out a summary of our input data:
	33
	34	using namespace std;
	35	using namespace boost::math;
	36
	37	// Print header:
	38	cout <<
	39	"____________________________________\n"
	40	"F test for equal standard deviations\n"
	41	"____________________________________\n\n";
	42	cout << setprecision(5);
	43	cout << "Sample 1:\n";
	44	cout << setw(55) << left << "Number of Observations" << "= " << N1 << "\n";
	45	cout << setw(55) << left << "Sample Standard Deviation" << "= " << sd1 << "\n\n";
	46	cout << "Sample 2:\n";
	47	cout << setw(55) << left << "Number of Observations" << "= " << N2 << "\n";
	48	cout << setw(55) << left << "Sample Standard Deviation" << "= " << sd2 << "\n\n";
	49
	50	The test statistic for an F-test is simply the ratio of the square of
	51	the two standard deviations:
	52
	53	F = s[sub 1][super 2] / s[sub 2][super 2]
	54
	55	where s[sub 1] is the standard deviation of the first sample and s[sub 2]
	56	is the standard deviation of the second sample. Or in code:
	57
	58	double F = (sd1 / sd2);
	59	F *= F;
	60	cout << setw(55) << left << "Test Statistic" << "= " << F << "\n\n";
	61
	62	At this point a word of caution: the F distribution is asymmetric,
	63	so we have to be careful how we compute the tests, the following table
	64	summarises the options available:
65
66	[table
67	[[Hypothesis][Test]]
68	[[The null-hypothesis: there is no difference in standard deviations (two sided test)]
69	[Reject if F <= F[sub (1-alpha/2; N1-1, N2-1)] or F >= F[sub (alpha/2; N1-1, N2-1)] ]]
70	[[The alternative hypothesis: there is a difference in means (two sided test)]
71	[Reject if F[sub (1-alpha/2; N1-1, N2-1)] <= F <= F[sub (alpha/2; N1-1, N2-1)] ]]
72	[[The alternative hypothesis: Standard deviation of sample 1 is greater
73	than that of sample 2]
74	[Reject if F < F[sub (alpha; N1-1, N2-1)] ]]
75	[[The alternative hypothesis: Standard deviation of sample 1 is less
76	than that of sample 2]
77	[Reject if F > F[sub (1-alpha; N1-1, N2-1)] ]]
78	]
79
80	Where F[sub (1-alpha; N1-1, N2-1)] is the lower critical value of the F distribution
81	with degrees of freedom N1-1 and N2-1, and F[sub (alpha; N1-1, N2-1)] is the upper
82	critical value of the F distribution with degrees of freedom N1-1 and N2-1.
83
84	The upper and lower critical values can be computed using the quantile function:
85
86	F[sub (1-alpha; N1-1, N2-1)] = `quantile(fisher_f(N1-1, N2-1), alpha)`
87
88	F[sub (alpha; N1-1, N2-1)] = `quantile(complement(fisher_f(N1-1, N2-1), alpha))`
89
90	In our example program we need both upper and lower critical values for alpha
91	and for alpha/2:
92
93	double ucv = quantile(complement(dist, alpha));
94	double ucv2 = quantile(complement(dist, alpha / 2));
95	double lcv = quantile(dist, alpha);
96	double lcv2 = quantile(dist, alpha / 2);
97	cout << setw(55) << left << "Upper Critical Value at alpha: " << "= "
98	<< setprecision(3) << scientific << ucv << "\n";
99	cout << setw(55) << left << "Upper Critical Value at alpha/2: " << "= "
100	<< setprecision(3) << scientific << ucv2 << "\n";
101	cout << setw(55) << left << "Lower Critical Value at alpha: " << "= "
102	<< setprecision(3) << scientific << lcv << "\n";
103	cout << setw(55) << left << "Lower Critical Value at alpha/2: " << "= "
104	<< setprecision(3) << scientific << lcv2 << "\n\n";
105
106	The final step is to perform the comparisons given above, and print
107	out whether the hypothesis is rejected or not:
108
109	cout << setw(55) << left <<
110	"Results for Alternative Hypothesis and alpha" << "= "
111	<< setprecision(4) << fixed << alpha << "\n\n";
112	cout << "Alternative Hypothesis Conclusion\n";
113
114	cout << "Standard deviations are unequal (two sided test) ";
115	if((ucv2 < F) \|\| (lcv2 > F))
116	cout << "ACCEPTED\n";
117	else
118	cout << "REJECTED\n";
119
120	cout << "Standard deviation 1 is less than standard deviation 2 ";
121	if(lcv > F)
122	cout << "ACCEPTED\n";
123	else
124	cout << "REJECTED\n";
125
126	cout << "Standard deviation 1 is greater than standard deviation 2 ";
127	if(ucv < F)
128	cout << "ACCEPTED\n";
129	else
130	cout << "REJECTED\n";
131	cout << endl << endl;
132
133	Using the ceramic strength data as an example we get the following
134	output:
135
136	[pre
137	'''F test for equal standard deviations
138	____________________________________
139
140	Sample 1:
141	Number of Observations = 240
142	Sample Standard Deviation = 65.549
143
144	Sample 2:
145	Number of Observations = 240
146	Sample Standard Deviation = 61.854
147
148	Test Statistic = 1.123
149
150	CDF of test statistic: = 8.148e-001
151	Upper Critical Value at alpha: = 1.238e+000
152	Upper Critical Value at alpha/2: = 1.289e+000
153	Lower Critical Value at alpha: = 8.080e-001
154	Lower Critical Value at alpha/2: = 7.756e-001
155
156	Results for Alternative Hypothesis and alpha = 0.0500
157
158	Alternative Hypothesis Conclusion
159	Standard deviations are unequal (two sided test) REJECTED
160	Standard deviation 1 is less than standard deviation 2 REJECTED
161	Standard deviation 1 is greater than standard deviation 2 REJECTED'''
162	]
163
164	In this case we are unable to reject the null-hypothesis, and must instead
165	reject the alternative hypothesis.
166
167	By contrast let's see what happens when we use some different
168	[@http://www.itl.nist.gov/div898/handbook/prc/section3/prc32.htm
169	sample data]:, once again from the NIST Engineering Statistics Handbook:
170	A new procedure to assemble a device is introduced and tested for
171	possible improvement in time of assembly. The question being addressed
172	is whether the standard deviation of the new assembly process (sample 2) is
173	better (i.e., smaller) than the standard deviation for the old assembly
174	process (sample 1).
175
176	[pre
177	'''____________________________________
178	F test for equal standard deviations
179	____________________________________
180
181	Sample 1:
182	Number of Observations = 11.00000
183	Sample Standard Deviation = 4.90820
184
185	Sample 2:
186	Number of Observations = 9.00000
187	Sample Standard Deviation = 2.58740
188
189	Test Statistic = 3.59847
190
191	CDF of test statistic: = 9.589e-001
192	Upper Critical Value at alpha: = 3.347e+000
193	Upper Critical Value at alpha/2: = 4.295e+000
194	Lower Critical Value at alpha: = 3.256e-001
195	Lower Critical Value at alpha/2: = 2.594e-001
196
197	Results for Alternative Hypothesis and alpha = 0.0500
198
199	Alternative Hypothesis Conclusion
200	Standard deviations are unequal (two sided test) REJECTED
201	Standard deviation 1 is less than standard deviation 2 REJECTED
202	Standard deviation 1 is greater than standard deviation 2 ACCEPTED'''
203	]
204
205	In this case we take our null hypothesis as "standard deviation 1 is
206	less than or equal to standard deviation 2", since this represents the "no change"
207	situation. So we want to compare the upper critical value at /alpha/
208	(a one sided test) with the test statistic, and since 3.35 < 3.6 this
209	hypothesis must be rejected. We therefore conclude that there is a change
210	for the better in our standard deviation.
211
212	[endsect][/section:f_eg F Distribution]
213
214	[/
215	Copyright 2006 John Maddock and Paul A. Bristow.
216	Distributed under the Boost Software License, Version 1.0.
217	(See accompanying file LICENSE_1_0.txt or copy at
218	http://www.boost.org/LICENSE_1_0.txt).
219	]
220