So previously, we looked at the normal probability plot or half-normal probability plot as an analysis method for an unreplicated two to the k designs. However, the subjectiveness of that technique has been problematic for some people. They would prefer to have a more formal analytical method. There have been several of those proposed in the literature. There's a 1998 article by Hamada and Balakrishnan that compared some of these methods, and they very much liked a method proposed by Lenth in 1989. I like this method myself. I'm very familiar with it and I've used it quite a bit. It turns out it's quite easy to implement and it is available in some software packages. Here's a general description of how the Lenth method works. Suppose you have, lets say m contrast C_1, C_2 on up to C_m, and those are the contrast associated with your unreplicated two to the k. Lenth's method basically uses the smallest and absolute value contrasts to get an estimate of error. It's done by basically using the smallest 50th percentile of your contrast, and has this smallest and absolute value. The basis of this idea is that you really don't expect more than about half of the contrast to actually be significant. So he defines what he called the pseudo standard error in terms of a multiple of the median of the absolute value of this contrast. Then the psuedo standard error could be used to judge the significance of effects or contrasts. The margin of error that he uses is a 2.5 percent point of t, times the pseudo standard error with an adjustment to the degrees of freedom. The degrees of freedom he took as m the number of contrast divided by three. Then if you have a group of contrasts, then a simultaneous margin of error could be defined as again, another t multiple times the pseudo standard error where there is an adjustment made on the percentage point to account for the fact that there are m contrasts that are being tested. For the recent planned example, we can show with a little bit of calculation that the margin of error would be 6.75 and the simultaneous margin of error would be 13.7. If we go back and compare those to the effect estimates that we had in Table 612, this is the outcome table for the recent plan experiment. The simultaneous margin of error would show that the four largest effects are significant. The main effect of C, which is the fifth largest effect, is insignificant according to the margin of error criteria, but not with respect to the simultaneous margin of error. But remember, the AC interaction was important. So I would definitely want to put C in the list of significant effects to preserve the hierarchy of the model. So in this case, the Lenth method produces exactly the same result that we would get from examining the normal probability plot. Now, a number of authors have studied Lenth's method and have felt that the values of the margin of error and simultaneous margin of error are probably too conservative and they have little power to detect significant effects. So these authors have suggested replacing the ME and SME multipliers with adjusted multipliers, and they've provided some recommendations for those. Those are what you see in the table at the bottom of the slide. The results that these authors all get, are similar. So this is a reasonable set of adjusted multipliers if you're going to use this approach. Now, on the other hand, I think a better approach to this is to use a real-time on the fly simulation of this method, the Lenth Method, to actually judge the significance of effects. That is exactly what the JMP software does. The JMP screening platform gives you a set of parameter estimates that are tested using a simulation-based method. In effect the way this simulation-based method works, is the null model is simulated 10,000 times. By the way using the length estimate of error, the pseudo standard error. It then creates a reference distribution for each one of the effects using this simulation output, these 10,000 trials. So now we have a reference distribution against which we can compare the actual observed value of the length statistic, when we actually analyze the data from our experiment. That's what you're seeing here. What you're observing in the table here is the estimate of each of the effects. Now these are actually the regression coefficients, not the effect estimates. Then the relative standard error. That relative standard error is based on a residual standard error of one, and then there's a pseudo, what Lenth would call a pseudo t-ratio, which compares the estimate to the length-statistic that you generate for that effect to the reference distribution of those values that were obtained by simulation. You can then look at the probability that your pseudo t-value is in the tail of that simulated reference distribution. For example, for the temperature effect, the pseudo t-ratio is 8.24, and the pseudo t-value, that is the proportion of the simulated reference distribution that is greater than that is 0.0004. So that would be a highly significant value because it's much less than say a reference value of 0.05. So if you look at this pseudo t-value or these pseudo p-values that you see in the far right-hand column, you notice that this procedure suggests that temperature, then temperature times concentration, temperature times stirring rate, stirring rate and then concentration are all significant effects and that everything else has pseudo p-values that are in excess of 0.05. So this method produces again, exactly the same answer that we would get from looking at the normal probability plot. You see the normal plot in the figure on the right. This particular implementation of Lenth's Method is very good and I highly recommend it for unreplicated designs. But I think I would always along with these psuedo t-ratios and pseudo p-values obtained by simulation, I think I would also look at the normal plot or the half-normal plot as a way to just reinforce what you are seeing in the simulation analysis. So that's a little overview of Lenth's Method and how it's implemented in one very nice software package. I hope you'll find this useful.
