Power Analysis for One-Way ANOVA

Jessie Morrill Author

11/07/2024 Added

7 Plays

Description

Dr. Jessie Morrill describes how to conduct a power analysis for a One-Way ANOVA using R Studio.

Searchable Transcript

Search:

[00:00:00.000]All right, so in today's video, I'm going to be discussing how to do a power analysis for a one-way ANOVA using RStudio.
[00:00:13.860]So to do that, you're going to need to make sure that you have the package called PWR installed onto your computer.
[00:00:23.000]And then every time, as per usual, that you are going to start running code using a particular package,
[00:00:29.820]you need to call that package to your library.
[00:00:32.640]And you do that by using the line of code library and then in parentheses the name of the package, which in this case is PWR.
[00:00:40.620]Select that line of code and click run.
[00:00:43.620]So you can see in the bottom left-hand corner of your screen that that line of code now shows up in the console
[00:00:51.520]as it has been running.
[00:00:52.980]So next I want to show you a really helpful trick when learning about any new particular function in R.
[00:01:00.920]So to be able to understand how that code works, you can type in the line or you can type in the word,
[00:01:08.920]excuse me, help and then in parentheses put the name of the function that you're working on.
[00:01:13.920]So in this case, we're doing a power analysis for a one-way ANOVA and the function for that is PWR.ANOVA.
[00:01:22.920].
[00:01:22.960]Test. So I'm going to select that line of code, click run.
[00:01:26.960]And what ends up happening is all of the background information that you need for that particular function shows up in the lower right hand corner of your screen.
[00:01:37.960]So in this case, you can see that this function does power calculations for balanced one-way analysis of variance tests.
[00:01:47.960]And so you can see a description here. And then what's the most useful is to
[00:01:52.940]be able to see the different arguments that are found within that function.
[00:01:57.940]And so you're going to see the first argument in this case is something called K,
[00:02:03.940]which is representative of the number of groups or the number of treatments that you have in your experiment that you're thinking of planning.
[00:02:12.940]The next argument is going to be N, which is representative of the number of observations per group.
[00:02:20.940]So most of the time,
[00:02:22.920]you are not going to know what this value is.
[00:02:26.920]This is what you're going to be solving for.
[00:02:29.920]So oftentimes you will omit the N portion of this function from the code when you run it because this is what you're solving for.
[00:02:37.920]So N again is going to be equal to the number of observations per group.
[00:02:42.920]And in this case, I want to make sure that you're aware that this is the number of experimental units per group.
[00:02:48.920]So if you're doing a study where pin is the experimental unit,
[00:02:52.900]this would give you the answer for the number of pins that are required to be able to detect a significant difference in your trial.
[00:03:02.900]So the next argument that I want to point to is something called F.
[00:03:07.900]F is representative of the effect size.
[00:03:11.900]So if you look over here on my screen, I have in the notes for you some details about the effect size.
[00:03:18.900]So in this particular example, you're going to enter a value.
[00:03:22.880]That's representative of the size of the effect that you expect to see.
[00:03:28.880]So if you expect to see something that's a really small, minor effect, you'd enter the value 0.1.
[00:03:35.880]If you expect to have a medium sized effect, you would enter something around 0.2 to 0.3 or 0.25.
[00:03:45.880]If you expect to see a really large effect, on the other hand, you'd enter something close to 0.4.
[00:03:52.860]It's really important that whenever you're looking at various functions for this type of power analysis that you understand what you're entering for f.
[00:04:02.860]There's different ways that the effect size can get represented.
[00:04:06.860]And so really understanding what value you're adding to the equation or function is really important.
[00:04:13.860]So all in all, what is trying to be represented here is something that is related to the ratio of the sums of squares.
[00:04:22.840]For the effect and the total sums of squares that you have for your particular experiment.
[00:04:29.840]So if you were to take the ratio of those two sums of squares values and calculate that out,
[00:04:36.840]what you'd end up finding is that a small effect would have something around zero point zero one,
[00:04:43.840]whereas a medium effect might come out to be somewhere around zero point zero six and a large effect could be somewhere around zero point one.
[00:04:52.820]So if you actually calculated out that particular value, then you would just relate these back and forth.
[00:04:59.820]So if you calculated this ratio out to be somewhere around zero point one four, that would be a pretty large effect.
[00:05:07.800]And so that would be where you'd come up to this effect size here and you'd say, well, I'm going to enter zero point four into my calculation.
[00:05:16.860]So the next argument in this piece of code is going to be the significance level.
[00:05:24.660]And so the significance level, more often than not, is going to be 0.05.
[00:05:30.320]So this represents alpha or the type 1 error rate for your experiment.
[00:05:35.680]The next value is going to be power.
[00:05:39.080]So power is equal to 1 minus the type 2 error rate.
[00:05:44.120]Or you might hear this referred to as 1 minus beta.
[00:05:48.620]So once you know those values, you have everything you need for conducting your power analysis.
[00:05:56.800]And so I failed to mention that the typical value that you would enter for power will more often than not be 0.8.
[00:06:05.720]So if we come over to our script, what I'm going to show you is an example power analysis.
[00:06:14.000]That I'm going to run here where I've set K equal to three treatment groups.
[00:06:18.960]We're looking at a small effect size, which is 0.1 with a significance level of 0.05.
[00:06:26.560]And we've set power equal to 0.8.
[00:06:30.120]If I run that line of code, what you end up seeing in the bottom left hand corner of your screen or in the RStudio console is the result of this power analysis.
[00:06:40.780]And what you can see is going down now.
[00:06:43.780]Now to the value that results for n is equal to 322.
[00:06:49.440]So you can see on the bottom note here, n is equal to the number of experimental units per treatment group.
[00:06:57.120]So in this case, we would round up 322.
[00:07:00.740]We would round that up to be 323 total units per group because we can't have a portion of an experimental unit, right?
[00:07:10.660]So we take 323.
[00:07:13.740]And we multiply that by our three treatment groups.
[00:07:17.500]What we end up getting is we need a total of 969 experimental units divided among those three treatment groups in our trial.
[00:07:28.200]So in this case, if the experimental unit was the animal,
[00:07:32.180]you would have just shy of 1,000 animals in your experiment
[00:07:37.540]to be able to confidently detect a significant difference
[00:07:42.380]for something that had a very small effect size.
[00:07:45.080]So now what I want to show you is if we change this effect size up to 0.2,
[00:07:50.900]where we're thinking this is closer to a medium effect,
[00:07:54.060]and we run that line of code, what you end up seeing as the result
[00:07:58.060]is somewhere around 82 experimental units per treatment group.
[00:08:03.880]So that's a pretty big difference.
[00:08:05.340]So I hope you can appreciate that depending upon the effect size
[00:08:10.140]that you expect to see,
[00:08:11.900]that really has an impact on the number of experimental units
[00:08:15.420]that are required when designing an experiment with sufficient power.
[00:08:19.820]So again, now I want to show you if we change that effect size up to 0.3
[00:08:25.800]and we run that line of code,
[00:08:27.840]the number of experimental units per treatment group,
[00:08:30.920]again, decreases pretty substantially down now
[00:08:35.480]to about 37 experimental units per treatment group.
[00:08:41.420]If we move this up to 0.4, where that's kind of a more large effect size,
[00:08:45.960]and we run that, the number of experimental units required per treatment group
[00:08:50.680]now becomes somewhere close to 22 experimental units per treatment group.
[00:08:55.960]So I hope you can appreciate again the impact that the effect size has
[00:09:02.640]on the number of experimental units that are required.
[00:09:05.220]Now, I want to show you an example of how you can use
[00:09:10.940]this code if you don't know what the effect size is,
[00:09:15.420]but you do know the number of observations per group.
[00:09:18.640]So we can run this here where we've got k equal to 3,
[00:09:22.620]n is equal to, say, 6 experimental units per treatment group.
[00:09:27.000]We've got the significance level at 0.05 and power set to 0.8.
[00:09:32.580]So if we select that line of code and we run it,
[00:09:36.020]what you can see is now the value that we get
[00:09:40.460]for f, where we solved for f.
[00:09:43.340]And the f value here comes out to be 0.8.
[00:09:47.480]So if we said previously that 0.4 was equivalent
[00:09:51.380]to a large effect size, you can see that to detect
[00:09:55.340]a very significant difference, when you only have
[00:09:58.280]6 experimental units per treatment group,
[00:10:00.800]you need to have a very, very large effect size
[00:10:04.900]that results from the difference that you see
[00:10:07.980]between your treatment groups.
[00:10:09.980]I hope this was insightful.
[00:10:11.680]I encourage you to spend some time playing with this,
[00:10:15.220]entering values, and seeing how the number of treatment groups
[00:10:19.200]and the effect size changes the number of experimental units
[00:10:24.040]that are required to be able to detect a significant difference
[00:10:27.560]in whatever type of experiments you're trying to run.

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Power Analysis for One-Way ANOVA

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments