«Research Findings from the Tennessee Value-Added Assessment System (TVAAS) Database: Implications for Educational Evaluation and Research Abstract The ...»
However, there is evidence that, aggregated statewide, students at the highest levels of achievement show somewhat less academic growth from year to year than their lower-achieving peers. In a 1997 study, Wright, Horn, and Sanders (1997, p. 63) investigated simultaneously the effects of teachers, intraclassroom homogeneity, and class size on achievement gain. The analyses revealed that the two most important factors impacting student gain are differences in classroom teacher effectiveness and the prior achievement level of the student. The teacher effect is highly significant in every analysis and has a larger effect size than any other factor in twenty of the thirty analyses. A notably nonsignificant factor was class size.... The main effect for heterogeneity was statistically significant in only two of the thirty analyses, approximately the number that would be expected to occur by chance.
Regarding the factor of prior achievement level of the student, Wright et al. (1997, pp. 65-66) note that
no universally applicable pattern emerges, but it is worth noting that out of the twenty-six analyses in which achievement level was significant, the largest gains occurred in the lowest achievement group twelve times, in one of the two middle groups eight times, and in the highest group six times. Similarly, the smallest gains occurred in the highest achievement group fifteen times, in one of the two middle groups six times, and in the lowest group five times. In other words, there is a disturbingly common but not universal pattern for the best students to make the lowest gains...
. Disproportionately, high-scoring students were found to make somewhat lower gains than average and lower-scoring students.
In a discussion of their findings, Wright et al. (1997) consider explanations for this shed pattern," so called because academic gains drop off as achievement level rises, creating a downward slope like a shed roof. They conclude (p. 66) that possible explanations include lack of opportunity for high-scoring students to proceed at their own pace, lack of challenging materials, lack of accelerated course offerings, and concentration of instruction on the average or below-average student. This finding indicates that it cannot be assumed that higher-achieving students will "make it on their own."
While this study discovered that "student academic level was found to be significantly related to academic progress" (p. 66), one factor was shown to be far more significant in predicting student academic growth. The "shed" patterns, while predictable to some degree by the previous academic standing of the student, were much more significantly related to the effectiveness of the teacher (Wright et al., 1997, p. 66).
Differences in teacher effectiveness were found to be the dominant factor affecting student academic gain. The importance of the effects of certain classroom contextual variables (e.g., class size, classroom heterogeneity) appears to be rather minor and should not be viewed as inhibiting to the appropriate use of student outcome data in teacher assessment. These results indicate that any realistic teacher evaluation process should include as a major component a reliable, valid measure of a teacher's effect on student academic growth. If the ultimate goal is the improvement in academic growth of student populations, one must conclude that improvement of student learning must begin with the improvement of relatively ineffective teachers regardless of the student placement strategies deployed within a school.
Finally, it is worth noting that results from the ACT and annual writing assessment data from all students in grades four, eight, and eleven are also linked to the TVAAS database. In another TVAAS study, which considered only the top quartile of Tennessee eighth graders, huge differences in mean ACT scores, obtained from these students four years later, were observed among school districts, indicating the enormous difference in effectiveness that exists among districts beyond elementary school (Graphical Summary, 1995).
Residual Effects of Teachers on the Academic Gains of Students (summarized from Sanders & Rivers, 1996) In a 1996 study entitled Cumulative and Residual Effects of Teachers on Future Student Academic Achievement, Sanders and Rivers examined the question, "does the influence of a teacher's effectiveness in facilitating academic growth for his/her students continue when these students advance to future grades?" (Sanders & Rivers, 1996, p. 1). The question was explored using data from two large Tennessee metropolitan school systems over a period of four years. For this study, only data pertaining to mathematics were considered.
Sanders and Rivers estimated teacher effects for teachers who taught mathematics in grades three, four, and five. Subsequently, for the purpose of the study, the teachers' effects were divided into five quintiles, with the least effective teachers comprising the first quintile and the most effective teachers the fifth. Student records were linked to those of their teachers, rendering it possible to trace the progress of the students through sequences of teachers identified by their effectiveness.
Sanders and Rivers found that "by looking at sequences in which the fifth-grade teachers were comparable in terms of effectiveness, it is possible to see the residual effects of prior year teachers" (Sanders & Rivers, 1996, p. 4). The findings demonstrate that students assigned to ineffective teachers continue to show the effects of such teachers even when these students are assigned to very effective teachers in subsequent years. Although an effective teacher can facilitate excellent academic gain in students during the years in which they are assigned to them, the study found that "the residual effects of relatively ineffective teachers from prior years can be measured in subsequent student achievement scores" (Sanders & Rivers, 1996, p. 4; see also Jordan, Mendro & Weersinghe, 1997).
Furthermore, when the data were aggregated by student achievement level, it was found that ineffective teachers were ineffective with all students, regardless of the prior level of achievement. As the level of teacher effectiveness increased, students of lower achievement were the first to benefit, and only teachers of the highest effectiveness were generally effective with all students. Only the teachers in the fifth quintile produced adequate gains in the highest-achieving students. Because of this, lower-achieving students were more likely than higher- achieving students to make adequate gains, year to year.
The implications of this finding are that only the most effective teachers-the top 20 per cent-are providing instruction that produces adequate gain in high-achieving students, while students in the lower achievement levels profit from all but the least effective teachers. Therefore, the majority of the brightest students fail to achieve to their potential year after year and, in the long run, attain a level of achievement far below that of their more fortunate peers who have benefited from the most effective teachers. This effect is observable in school systems that vary extensively in socioeconomic level, racial composition, and location. Sanders and Rivers (1996, p. 6) state that the teacher effects are both additive and cumulative with little evidence of compensatory effect of more effective teachers in later grades. The residual effects of both very effective and ineffective teachers were measurable two years later, regardless of the effectiveness of teachers in later grades.
Another finding emerged from the Sanders and Rivers study. On examination of the racial composition of the classes of teachers by effectiveness quintiles, they (p. 6) found that more black students than would be expected, based on the ethnic makeup of the system, were assigned to the least effective teachers.... However, the achievement within the two ethnic groups is comparable across the five levels of teacher effectiveness. These analyses suggest that students of the same prior levels of achievement tend to respond similarly to teacher effectiveness levels.
In other words, African American students and white students with the same level of prior achievement make comparable academic progress when they are assigned to teachers of comparable effectiveness.
However, at least in the system studied, black students were disproportionately assigned to the least effective teachers. Regardless of race, students who are assigned disproportionately to ineffective teachers will be severely academically handicapped relative to students with other teacher assignment patterns.
From the research findings summarized in this report and from other studies, as well (see, for example, Jordan et al., 1997), it is clear that teacher effectiveness is the major factor influencing student academic gain. This finding holds major implications for the field of educational evaluation, the most obvious of which is that educational assessment that does not address teacher effectiveness is, at the very least, seriously limited in its ability to serve its primary purpose-to provide a basis in fact for educational improvement. It is only when educational practitioners-teachers as well as school and school system administrators-have a clear understanding of how they affect their students in the classroom that they can make informed decisions about what to change and what to maintain. Linking student outcomes to school, school system, and teacher effects can provide this invaluable information.
With the advent of TVAAS methodology, obstacles that previously hindered the effective and appropriate use of standardized test data for the purpose of educational evaluation have been removed.
These data can now provide best linear unbiased predictors of the effectiveness of individual teachers, as well as schools and school systems, in regard to student academic gain. Even though standardized test data cannot provide all the information necessary to determine the effectiveness of a teacher or a school, they can be invaluable and indispensable in linking student outcomes to specific teachers and schools.
If an assessment system is to provide data at the teacher level, students must be tested annually with fresh, equivalent, nonredundant tests that exhibit a high level of reliability and validity, regardless of whether they are standardized tests or are some alternative type of outcomes assessment. This testing should continue throughout the scholastic career of each child. Currently, many states routinely test children in the lower grades, but only recently have significant numbers of states and districts begun constructing, piloting, and administering end-of-course tests in the high schools. These new tests on the secondary level are vital if academic progress is to be maximized for every student. The previously mentioned research on the variation among ACT scores of students who were in the top quartile according to their eighth-grade TCAP scores is a particularly good example of why this is so. The fact that there is great variation in academic attainment of students of similar ability depending on where they attend high school is valuable information, to be sure, but it is impossible to know from one omnibus test what went wrong in which class under which teacher that would account for the fact that some very bright students end up scoring considerably lower as seniors than their academic peers in the eighth grade do.
To provide that level of diagnostic information, end-of-course data are essential.
As more and more states codify academic standards that all students are expected to meet, the question of responsibility becomes paramount. If students are responsible for attaining the standards, then teachers are responsible for teaching them. If students have differing abilities to learn, then somehow all must still be presented with the opportunity to learn. Responsible assessment is a necessary component of responsible teaching and learning.
Any educational assessment system is limited if it does not provide measures of the effects of schools, school systems, and teachers on the academic growth of students.
Without this information, educational improvement efforts cannot address the real factors that have been proven to have the greatest effect on student learning. Assessment systems of appropriate tests and methodologies for analyzing them are now available. It is through these new assessment systems that take into account the impact of educational experiences in the school and in the individual classroom that the promise of effective education for all students can come closer to realization.
References Graphical Summary of Educational Findings from the Tennessee Value-Added Assessment System (TVAAS), 1995. (1995). Knoxville: University of Tennessee Value-Added Research and Assessment Center.
Jordan, H.R., Mendro, R.L., & Weersinghe, D. (1997). Teacher effects on longitudinal student achievement: A preliminary report on research on teacher effectiveness. Paper presented at the National Evaluation Institute, Indianapolis, IN.
McLean, R.A., & Sanders, W.L. (1984). Objective component of teacher evaluation: A feasibility study.
Working Paper No. 199. Knoxville: University of Tennessee, College of Business Administration.
Sanders, W.L., & Rivers, J.C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Research Progress Report. Knoxville: University of Tennessee Value-Added Research and Assessment Center.
Sanders, W.L., Saxton, A.M., & Horn, S.P. (1997). The Tennessee Value-Added Assessment System (TVAAS): A quantitative, outcomes-based approach to educational assessment. In Millman, J. (ed.), Grading Teachers, Grading Schools, Thousand Oaks, CA: Corwin Press.
Sanders, W.L., Saxton, A.M., Schneider, J.F, Dearden, B.L., Wright, S. Paul, & Horn, S.P. (1994).
Effects of building change on indicators of student academic growth. Evaluation Perspectives, 4(l) pp 3 and 7.
Wright, S.P., Horn, S.P., & Sanders, W.L. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 1](1), 57-67.