Chapter 5: One Factor Designs
As explained in Simple Linear Regression Analysis and Multiple Linear Regression Analysis, the analysis of observational studies involves the use of regression models. The analysis of experimental studies involves the use of analysis of variance (ANOVA) models. For a comparison of the two models see Fitting ANOVA Models. In single factor experiments, ANOVA models are used to compare the mean response values at different levels of the factor. Each level of the factor is investigated to see if the response is significantly different from the response at other levels of the factor. The analysis of single factor experiments is often referred to as one-way ANOVA.
To illustrate the use of ANOVA models in the analysis of experiments, consider a single factor experiment where the analyst wants to see if the surface finish of certain parts is affected by the speed of a lathe machine. Data is collected for three speeds (or three treatments). Each treatment is replicated four times. Therefore, this experiment design is balanced. Surface finish values recorded using randomization are shown in the following table.
The ANOVA model for this experiment can be stated as follows:
The ANOVA model assumes that the response at each factor level, , is the sum of the mean response at the th level, , and a random error term, . The subscript denotes the factor level while the subscript denotes the replicate. If there are levels of the factor and replicates at each level then and . The random error terms, , are assumed to be normally and independently distributed with a mean of zero and variance of . Therefore, the response at each level can be thought of as a normally distributed population with a mean of and constant variance of . The equation given above is referred to as the means model.
The ANOVA model of the means model can also be written using , where represents the overall mean and represents the effect due to the th treatment.
Such an ANOVA model is called the effects model. In the effects models the treatment effects, , represent the deviations from the overall mean, . Therefore, the following constraint exists on the s:
Fitting ANOVA Models
To fit ANOVA models and carry out hypothesis testing in single factor experiments, it is convenient to express the effects model of the effects model in the form (that was used for multiple linear regression models in Multiple Linear Regression Analysis). This can be done as shown next. Using the effects model, the ANOVA model for the single factor experiment in the first table can be expressed as:
where represents the overall mean and represents the th treatment effect. There are three treatments in the first table (500, 600 and 700). Therefore, there are three treatment effects, , and . The following constraint exists for these effects:
For the first treatment, the ANOVA model for the single factor experiment in the above table can be written as:
Using , the model for the first treatment is:
Models for the second and third treatments can be obtained in a similar way. The models for the three treatments are:
The coefficients of the treatment effects and can be expressed using two indicator variables, and , as follows:
Using the indicator variables and , the ANOVA model for the data in the first table now becomes:
The equation can be rewritten by including subscripts (for the level of the factor) and (for the replicate number) as:
The equation given above represents the "regression version" of the ANOVA model.
Treat Numerical Factors as Qualitative or Quantitative?
It can be seen from the equation given above that in an ANOVA model each factor is treated as a qualitative factor. In the present example the factor, lathe speed, is a quantitative factor with three levels. But the ANOVA model treats this factor as a qualitative factor with three levels. Therefore, two indicator variables, and , are required to represent this factor.
Note that in a regression model a variable can either be treated as a quantitative or a qualitative variable. The factor, lathe speed, would be used as a quantitative factor and represented with a single predictor variable in a regression model. For example, if a first order model were to be fitted to the data in the first table, then the regression model would take the form . If a second order regression model were to be fitted, the regression model would be . Notice that unlike these regression models, the regression version of the ANOVA model does not make any assumption about the nature of relationship between the response and the factor being investigated.
The choice of treating a particular factor as a quantitative or qualitative variable depends on the objective of the experimenter. In the case of the data of the first table, the objective of the experimenter is to compare the levels of the factor to see if change in the levels leads to a significant change in the response. The objective is not to make predictions on the response for a given level of the factor. Therefore, the factor is treated as a qualitative factor in this case. If the objective of the experimenter were prediction or optimization, the experimenter would focus on aspects such as the nature of relationship between the factor, lathe speed, and the response, surface finish, so that the factor should be modeled as a quantitative factor to make accurate predictions.
Expression of the ANOVA Model as Y = XΒ + ε
The regression version of the ANOVA model can be expanded for the three treatments and four replicates of the data in the first table as follows:
The corresponding matrix notation is:
The matrices , and are used in the calculation of the sum of squares in the next section. The data in the first table can be entered into the DOE folio as shown in the figure below.
Hypothesis Test in Single Factor Experiments
The hypothesis test in single factor experiments examines the ANOVA model to see if the response at any level of the investigated factor is significantly different from that at the other levels. If this is not the case and the response at all levels is not significantly different, then it can be concluded that the investigated factor does not affect the response. The test on the ANOVA model is carried out by checking to see if any of the treatment effects, , are non-zero. The test is similar to the test of significance of regression mentioned in Simple Linear Regression Analysis and Multiple Linear Regression Analysis in the context of regression models. The hypotheses statements for this test are:
The test for is carried out using the following statistic:
where represents the mean square for the ANOVA model and is the error mean square. Note that in the case of ANOVA models we use the notation (treatment mean square) for the model mean square and (treatment sum of squares) for the model sum of squares (instead of , regression mean square, and , regression sum of squares, used in Simple Linear Regression Analysis and Multiple Linear Regression Analysis). This is done to indicate that the model under consideration is the ANOVA model and not the regression model. The calculations to obtain and are identical to the calculations to obtain and explained in Multiple Linear Regression Analysis.
Calculation of the Statistic
The sum of squares to obtain the statistic can be calculated as explained in Multiple Linear Regression Analysis. Using the data in the first table, the model sum of squares, , can be calculated as:
In the previous equation, represents the number of levels of the factor, represents the replicates at each level, represents the vector of the response values, represents the hat matrix and represents the matrix of ones. (For details on each of these terms, refer to Multiple Linear Regression Analysis.)
Since two effect terms, and , are used in the regression version of the ANOVA model, the degrees of freedom associated with the model sum of squares, , is two.
The total sum of squares, , can be obtained as follows:
In the previous equation, is the identity matrix. Since there are 12 data points in all, the number of degrees of freedom associated with is 11.
Knowing and , the error sum of squares is:
The number of degrees of freedom associated with is:
The test statistic can now be calculated using the equation given in Hypothesis Test in Single Factor Experiments as:
The value for the statistic based on the distribution with 2 degrees of freedom in the numerator and 9 degrees of freedom in the denominator is:
Assuming that the desired significance level is 0.1, since value < 0.1, is rejected and it is concluded that change in the lathe speed has a significant effect on the surface finish. The Weibull++ DOE folio displays these results in the ANOVA table, as shown in the figure below. The values of S and R-sq are the standard error and the coefficient of determination for the model, respectively. These values are explained in Multiple Linear Regression Analysis and indicate how well the model fits the data. The values in the figure below indicate that the fit of the ANOVA model is fair.
Confidence Interval on the ith Treatment Mean
The response at each treatment of a single factor experiment can be assumed to be a normal population with a mean of and variance of provided that the error terms can be assumed to be normally distributed. A point estimator of is the average response at each treatment, . Since this is a sample average, the associated variance is , where is the number of replicates at the th treatment. Therefore, the confidence interval on is based on the distribution. Recall from Statistical Background on DOE (inference on population mean when variance is unknown) that:
has a distribution with degrees of freedom . Therefore, a 100 () percent confidence interval on the th treatment mean, , is:
For example, for the first treatment of the lathe speed we have:
In the DOE folio, this value is displayed as the Estimated Mean for the first level, as shown in the Data Summary table in the figure below. The value displayed as the standard deviation for this level is simply the sample standard deviation calculated using the observations corresponding to this level. The 90% confidence interval for this treatment is:
The 90% limits on are 5.9 and 11.1, respectively.
Confidence Interval on the Difference in Two Treatment Means
The confidence interval on the difference in two treatment means, , is used to compare two levels of the factor at a given significance. If the confidence interval does not include the value of zero, it is concluded that the two levels of the factor are significantly different. The point estimator of is . The variance for is:
For balanced designs all . Therefore:
The standard deviation for can be obtained by taking the square root of and is referred to as the pooled standard error:
The statistic for the difference is:
Then a 100 (1- ) percent confidence interval on the difference in two treatment means, , is:
For example, an estimate of the difference in the first and second treatment means of the lathe speed, , is:
The pooled standard error for this difference is:
To test , the statistic is:
In the DOE folio, the value of the statistic is displayed in the Mean Comparisons table under the column T Value as shown in the figure below. The 90% confidence interval on the difference is:
Hence the 90% limits on are and , respectively. These values are displayed under the Low CI and High CI columns in the following figure. Since the confidence interval for this pair of means does not included zero, it can be concluded that these means are significantly different at 90% confidence. This conclusion can also be arrived at using the value noting that the hypothesis is two-sided. The value corresponding to the statistic , based on the distribution with 9 degrees of freedom is:
Since value < 0.1, the means are significantly different at 90% confidence. Bounds on the difference between other treatment pairs can be obtained in a similar manner and it is concluded that all treatments are significantly different.
Plots of residuals, , similar to the ones discussed in the previous chapters on regression, are used to ensure that the assumptions associated with the ANOVA model are not violated. The ANOVA model assumes that the random error terms, , are normally and independently distributed with the same variance for each treatment. The normality assumption can be checked by obtaining a normal probability plot of the residuals.
Equality of variance is checked by plotting residuals against the treatments and the treatment averages, (also referred to as fitted values), and inspecting the spread in the residuals. If a pattern is seen in these plots, then this indicates the need to use a suitable transformation on the response that will ensure variance equality. Box-Cox transformations are discussed in the next section. To check for independence of the random error terms residuals are plotted against time or run-order to ensure that a pattern does not exist in these plots. Residual plots for the given example are shown in the following two figures. The plots show that the assumptions associated with the ANOVA model are not violated.
Transformations on the response may be used when residual plots for an experiment show a pattern. This indicates that the equality of variance does not hold for the residuals of the given model. The Box-Cox method can be used to automatically identify a suitable power transformation for the data based on the following relationship:
is determined using the given data such that is minimized. The values of are not used as is because of issues related to calculation or comparison of values for different values of . For example, for all response values will become 1. Therefore, the following relationship is used to obtain :
Once all values are obtained for a value of , the corresponding for these values is obtained using . The process is repeated for a number of values to obtain a plot of against . Then the value of corresponding to the minimum is selected as the required transformation for the given data. The DOE folio plots values against values because the range of values is large and if this is not done, all values cannot be displayed on the same plot. The range of search for the best value in the software is from to , because larger values of of are usually not meaningful. The DOE folio also displays a recommended transformation based on the best value obtained as shown in the following table.
Confidence intervals on the selected values are also available. Let be the value of corresponding to the selected value of . Then, to calculate the 100 (1- ) percent confidence intervals on , we need to calculate as shown next:
The required limits for are the two values of corresponding to the value (on the plot of against ). If the limits for do not include the value of one, then the transformation is applicable for the given data.
Note that the power transformations are not defined for response values that are negative or zero. The DOE folio deals with negative and zero response values using the following equations (that involve addition of a suitable quantity to all of the response values if a zero or negative response value is encountered).
Here represents the minimum response value and represents the absolute value of the minimum response.
To illustrate the Box-Cox method, consider the experiment given in the first table. Transformed response values for various values of can be calculated using the equation for given in Box-Cox Method. Knowing the hat matrix, , values corresponding to each of these values can easily be obtained using . values calculated for values between and for the given data are shown below:
A plot of for various values, as obtained from the DOE folio, is shown in the following figure. The value of that gives the minimum is identified as 0.7841. The value corresponding to this value of is 73.74. A 90% confidence interval on this value is calculated as follows. can be obtained as shown next:
Therefore, . The values corresponding to this value from the following figure are and . Therefore, the 90% confidence limits on are and . Since the confidence limits include the value of 1, this indicates that a transformation is not required for the data in the first table.