Test of Independent Proportions
Instructions for using the Test of Independent Proportions
If you know the numerators of two groups (i.e., the top part of fractions) and their respective denominators (i.e., the bottom part of fractions), then you can compare the two groups using the Test of Independent Proportions (TIP) to determine if the difference between the two groups is statistically significant. If the p-value is less than 0.05, then we can say that we are 95% confident that difference between the groups is meaningful (i.e., statistically significant) and not just accidental. There are many ways that the TIP can be used for educational purposes. For instance, you can use the TIP to compare your school to your school on the Fall and Winter administrations of a Diagnostic Test (e.g., the percent of students scoring Level 3+ in the Fall and Winter). You can also use a the TIP to compare your school to another school for the same administration of a Diagnostic Test (i.e., the percent of students in your school scoring Level 3+ on the Winter Diagnostic and the percent of students in another school within the same grades as yours scoring Level 3+). You can also use the TIP to compare results of different groups of students on the same Diagnostic Test within your own school (e.g., different ethnic/racial groups and grade levels). The following two examples explain in more detail how to enter the numerators and denominators. They also show how statistical significance is largely dependent upon the size of the groups being compared. Example 1 If 150 students in a school scored Level 3 or higher on the fall Reading Diagnostic Test out of 300 students at the school who took the test, the fraction would be 150/300, which equals the percent of students getting Level 3 or higher on the Fall Reading Diagnostic Test (i.e., 50%). If 165 students at the same school scored Level 3 or higher on the Winter Reading Diagnostic Test out of 305 students at the school who took the test, the fraction would be 165/305, which equals the percent of students getting Level 3 or higher on the Winter Reading Diagnostic test (i.e., 54.5%). That is more than four percentage points different, which looks like a big difference. However, if you enter the two numerators and denominators into the TIP calculator (i.e., numerator 1 = 150 and denominator 1 = 300; numerator 2 = 165 and denominator 2 = 305) and click on the Calculate button, you will find that the difference between the two groups is not statistically significant. Example 2 Suppose you had the same percent correct on the Fall and Winter Reading Diagnostic Tests as in example 2, but now the numerators/denominators were as follows:Fall =1500/3000 (50%) Winter = 1650/3050 (54.5%).
Explanation of Educational Effect Size
Statistical significance helps us to understand whether the difference between two groups is due to chance. Even when there is statistically significant difference between two groups, however, we do know the extent of that difference as it pertains to the actual impact the difference has on student achievement. That is the job of the Educational Effect Size. Effect Sizes are reported only when there is a statistically significant difference between two groups. They range from inconsequential (i.e., even though there is a statistically significant difference between two groups, there is virtually no difference in the impact on student performance) to Exceptional (i.e., that there is an especially large difference in impact of student achievement, a difference that is rarely seen). Effect Sizes can be positive or negative.
Educational Effect Size |
||
VERY LARGE (+) |
||
LARGE (+) |
||
SUBSTANTIAL (+) |
||
MODERATE (+) |
||
SLIGHT (+) |
||
INCONSEQUENTIAL (+) |
||
INCONSEQUENTIAL (-) |
||
SLIGHT (-) |
||
MODERATE (-) |
||
SUBSTANTIAL (-) |
||
LARGE (-) |
||
VERY LARGE (-) |
Reverse Test of Independent Proportions
Instructions for using the
Reverse Test of Independent Proportions
(also know as the Significant Target Calculation Tool)
The Reverse Test of Independent Proportions (Reverse TIP) reports what percentage change would be needed to have a statistically significant difference between two sets of comparable data. Some examples of comparable scenarios to use the Reverse TIP: computing the necessary amount of difference between the Fall and Winter Diagnostic assessment, comparing one demographic group to another on the same assessment, comparing your school to another school on the same assessment. The reverse TIP is useful for setting objectives, and is appropriate to be used when setting your Indicator Targets for your School Improvement Process (SIP).
To perform the Reverse TIP, enter the proficiency percentage from the previous year in the "Last Year's Percentage" box. Then enter the number of students that will be tested in the current year and in the prior year. When you click "Calculate", the difference in percentage required to show statistical significance will appear along with the student count necessary to meet that target, as well as the total Indicator Target percentage.