WWW.SA.I-PDF.INFO
FREE ELECTRONIC LIBRARY - Abstracts, books, theses
 
<< HOME
CONTACTS



Pages:   || 2 |

«Research Findings from the Tennessee Value-Added Assessment System (TVAAS) Database: Implications for Educational Evaluation and Research Abstract The ...»

-- [ Page 1 ] --

WILLIAM L. SANDERS AND SANDRA P. HORN

University of Tennessee Value-Added Research and Assessment Center

Research Findings from the Tennessee Value-Added

Assessment System (TVAAS) Database: Implications for Educational

Evaluation and Research

Abstract

The Tennessee Value-Added Assessment System determines the effectiveness of school systems,

schools, and teachers based on student academic growth over time. An integral part of TVAAS is a

massive, longitudinally merged database linking students and student outcomes to the schools and systems in which they are enrolled and to the teachers to whom they are assigned as they transition from grade to grade. Research conducted utilizing data from the TVAAS database has shown that race, socioeconomic level, class size, and classroom heterogeneity are poor predictors of student academic growth. Rather, the effectiveness of the teacher is the major determinant of student academic progress.

Teacher effects on student achievement have been found to be both additive and cumulative with little evidence that subsequent effective teachers can offset the effects of ineffective ones. For these reasons, a component linking teacher effectiveness to student outcomes is a necessary part of any effective educational evaluation system.

Introduction In the mid-1980s, the level of concern for the state of education in the United States rose across the nation in the wake of the publication of A Nation at Risk. In 1989, President George Bush convened an Education Summit with the governors of the fifty states to consider the problems facing education in America and to develop a plan of action for addressing these problems. To this end, the governors, with input from the White House and from experts in the field of education, developed six National Education Goals. How these goals were to be accomplished was left up to the individual states. In those states in which such legislation had not already been adopted, governors returning from the Education Summit began challenging their legislators to develop legislation that would enable the achievement of the National Goals for Education. The legislation enacted as a result of this impetus varied radically from state to state, but despite the differences, reform legislation held in common the call for higher academic standards and greater accountability linked to assessment of educational outcomes.

In Tennessee, a major attempt to improve educational opportunities for students predated the 1989 educational summit. The Comprehensive Education Reform Act (CERA) was enacted under Governor Lamar Alexander in 1984. CERA included a major increase in educational spending and a Career Ladder system for teachers. The teacher evaluation system developed to assess candidates for the upper levels of the Career Ladder was performance based and included an extensive portfolio in addition to three intensive days of on-site observation and dialogue with state-trained evaluators. The portfolio was dropped after the first year due to extremely negative teacher reaction and was replaced by an enhanced dialogue. Many teachers believed that the Career Ladder was bogus because the observations and dialogues could be "performances" rather than illustrations of actual teaching expertise, even though two of the three observations were supposedly unannounced. Eventually, in 1997, the Career Ladder was frozen. Teachers who had already achieved Career Ladder status would retain it, but no new evaluations would take place.

Discussions were initiated in 1989 that would lead Tennessee to its second major education legislative initiative in less than a decade. A law suit filed by a group of the smaller school districts in the state (was the impetus for the second legislative initiative.) It contended that it was the state's responsibility under the state constitution to provide equal funding across districts to ensure an equal educational opportunity for all students. On March 11, 1992, after two years of discussion and debate, the Education Improvement Act (EIA) was signed into law by Governor Ned McWherter. This legislation included another major increase in funding for education in the state, which, in turn, required a second increase in the state's sales tax in less than ten years. Legislators in both parties demanded a strong accountability provision be included in the act to ensure that the new monies would be spent to improve student academic achievement. The Tennessee Value-Added Assessment System (TVAAS), along with

–  –  –

measures including promotion, attendance, and dropout rates of individual schools, would provide information to form the base for the state's new educational accountability system.

TVAAS, referred to in the EIA as the Sanders model, was the methodology designated to ascertain the effectiveness of school systems, schools, and teachers in producing academic growth in Tennessee students, thereby linking student academic outcomes to educational evaluation for the first time. TVAAS required the convergence of a statewide testing program, which tests each student each year in several academic subjects, and an unprecedented application of a statistical approach that enables a massive multivariate longitudinal analysis even with fractured student records, which are always present in realworld student achievement data.





Initially, much skepticism was expressed concerning both the use of student achievement data in this manner and the application of statistical mixed-model theory and methodology to yield fair, objective, and unbiased estimates of the system, school, and teacher effects on the rate of academic progress of populations of students. However, detailed external reviews from both the statistical and educational evaluation communities have confirmed that the properties of the TVAAS results are as claimed by Sanders and his associates.

Beginning in 1993, reports have been issued to the educators and public of the State of Tennessee on the effectiveness of every school system and school that serves elementary students in any of the third through eighth grades and for selected high school math courses. Even though teacher reports are not a public record, TVAAS has provided teachers and their administrators with estimates of teacher effectiveness, as well. The EIA states that TVAAS data will be a part of the evaluation of those teachers for whom such data are available. However, how large a part it will be is left to the discretion of the evaluative body. It cannot be the only source of data in a teacher's evaluation.

Recently, a new State Framework for Local Evaluation of teachers has been adopted. Under the new Framework, teachers and their principals devise a professional development plan that must correlate with the plans developed for their schools, just as the school plans must correlate with the plans for the system. The connection between professional growth of the teacher and school improvement must be explicit. Thus, student academic growth, an integral part of virtually every school improvement plan, becomes a part of successful teacher evaluation for the first time on a statewide basis. TVAAS teacher reports are providing the data from which individual professional development plans are drawn.

Successful teacher evaluations include an assessment as to how well the teacher has accomplished the goals and professional growth incorporated in the professional development plan. Subsequent TVAAS teacher reports may also furnish evidence that improvement has or has not been achieved.

Beyond the use of the TVAAS database for its assessment and evaluation purposes, it has become a launching pad for research to investigate both accelerators and inhibitors of academic growth of student populations. Important educational questions are now being addressed with a research efficiency that only millions of longitudinal merged records allow. This article is an attempt to summarize the most important findings to date.

A Brief Description of TVAAS

The Tennessee Value-Added Assessment System is a statistical method of determining the effectiveness of school systems, schools, and teachers. TVAAS uses statistical mixedmodel theory and methodology to enable a multivariate, longitudinal analysis of student achievement data. (For a detailed description of TVAAS methodology, see Sanders, Saxton & Horn, 1997.) These data include student scores on (1) the Tennessee Comprehensive Assessment Program (TCAP), a group of tests in five subject areas (math, science, social studies, reading, and language arts) administered annually to all Tennessee students in grades three through eight, and (2) end-of-course tests in high school subjects, currently in five mathematics subjects. The statistical models used in TVAAS are not restrictive as to the indicator variables that can be employed in the process. Rather, any variables linear in their metrics, highly correlated with curricular objectives, and possessing appropriate measurement sensitivities could be used.

–  –  –

Each student's test data are accumulated over time and are linked to that student's teacher(s), school(s), and school system(s). TVAAS utilizes the scaled scores students make over time to model their learning patterns. By taking advantage of the longitudinal aspect of the data, it is possible to note when the normal pace of academic growth deviates. By following growth over time, the child serves as his or her own "control." This enables the partitioning of school system, school, and teacher effects free of the exogenous factors that influence academic achievement and that are consistently present with each child over time.

The massive database comprising these data, currently in excess of 5 million records, makes it possible to utilize many years of information in determining the effectiveness of educational institutions and personnel. In addition to the robustness of the statistical process, further provisions are made within the EIA to ensure fairness. These provisions include basing assessment on at least three years of data and requiring that schools, school systems, and teachers cannot be assessed solely on the basis of TVAAS.

The primary purpose TVAAS serves in the EIA is to provide information for summative evaluation regarding how effective a school, system, or teacher has been in leading students to achieve normal academic gain over a three-year period. TVAAS reports, issued annually, include information on student gains for each subject and grade for the three most recent years as well as the three-year average gains.

The cumulative average gain is the primary indicator by which success is measured. According to the EIA, the standards to which school districts and schools within each district are held accountable are expressed in terms of academic gains instead of an expectation set in terms of absolute scores as has been the case in other states. The increasingly important emphasis on student outcomes in the assessment of educational effectiveness is clear from this legislative provision.

If the purpose of educational evaluation is to improve the educational process, and if such improvement is characterized by improved academic growth of students, then the inclusion of measures of the effectiveness of schools, schools systems, and teachers in facilitating such growth is essential if the purpose is to be realized. Of these three, determining the effectiveness of individual teachers holds the most promise because, again and again, findings from TVAAS research show teacher effectiveness to be the most important factor in the academic growth of students.

Even though the driving force for the creation of TVAAS was for summative evaluation, the real power of the process lies in its ability to serve as a data source for formative evaluation and for educational research. Schools, systems, and teachers receive reports detailing their effectiveness with students of different achievement levels so that they may more effectively plan their curricula, pedagogy, and special programs. This information has been found to be invaluable by many teachers and school administrators involved in curricular planning, program evaluation, and developing strategies to meet the needs of students with differing academic attributes and abilities. The reports allow school systems to pinpoint grade and subject problems and successes and to direct efforts and resources accordingly. School reports inform principals not only about how effective the fourth-grade math program is, for example, in regard to enhancing student academic gain but also whether it is equally effective in encouraging such growth in its high achievers as well as in its low-achieving students. Similar information is provided to each classroom teacher along with the formal teacher reports, which have been issued annually since 1996.

In addition to the analysis and reporting of the effects of educational entities for both summative and formative purposes, research initiatives are a major priority. Already many significant findings have resulted from analysis of TVAAS data. Research to date has focused on factors impacting the academic growth of students.

Research Findings Race, Socioeconomic Factors, and Academic Gain

–  –  –

For grades three through eight, the cumulative gains for schools across the entire state have been found to be unrelated to the racial composition of schools, the percentage of students receiving free and reduced-price lunches, or the mean achievement level of the school. These consistent findings have verified the contention that by allowing each student to serve as his or her own control (the longitudinal component of TVAAS), the inclusion of exogenous covariables to ensure fairness in the estimates of system, school and teacher effects is not necessary. Schools, systems, and teachers who do best under TVAAS are those who provide academic growth opportunities for students of all levels of prior academic attainment.

The Building Change Effect (excerpted from Sanders, Saxton, Schneider, Dearden, Wright, Paul & Horn, 1994) When the first TVAAS reports of school system effects on student academic growth were being developed in late 1992, it was observed that certain systems had a noticeable drop in gain for all subjects at certain grades. However, the point of retarded gains varied from system to system. After further examination at the system level, it was hypothesized that the entry point into the receiving school could be causing the retardation in growth.

Subsequently, an analysis to test this hypothesis using the records of all Tennessee students was initiated. Since students' records are matched and merged over all systems in the state, school change patterns can be ascertained. School configurations across the state vary enormously, representing nearly all possible combinations of grades two through eight.

The population of Tennessee students relative to transfer status were presented as three groups-those who attended the same school in two contiguous years, those who changed schools and entered the new school at the lowest grade level offered in the receiving school, and those who changed schools but entered the new school at any grade other than the lowest offered. The study found that the negative effect of student transfer to any grade other than the lowest grade offered in the new school is negligible for transfers prior to the seventh grade. However, the effects of transfer are entirely different when students transfer to the lowest grade in the new school building. When populations of students change buildings, regardless of the grade level, the loss in expected academic gain is dramatic. The mean gain for students who transfer to the lowest grade of their new school was measurably lower in all twenty-five subject-grade combinations. Severe retardation in gains was most pronounced in grades six and seven, the grades at which many school systems routinely transfer students en masse to middle school or junior high.

Factors Impacting Academic Gain Among High-Achieving Students

TVAAS data have shown that some schools and, indeed, some school systems have been successful in addressing the needs of students at all levels of achievement as evidenced by their ability to consistently show normal-and sometimes, exceptional-academic progress for students of all academic abilities.



Pages:   || 2 |


Similar works:

«Language Testing in Asia Volume one, Issue two July 2011   An Examination of the No Fail Policy in Thailand and the Effect on Community Relations KRISTIN M. HALLIGAN Assumption College, Thailand Bio Data: Kristin Halligan is currently finishing a Master's Degree in Multidisciplinary Science with a focus on International Education from the State University of New York. She is a second grade teacher at Assumption College Primary section in Bangkok, Thailand. Abstract No fail or social promotion...»

«Multiculturalism’s Five Dimensions Dr. James A. Banks on Multicultural Education Dr. James A. Banks, author of Educating Citizens in a Multicultural Society (Teachers College Press, $22.95), spoke recently with NEA Today’s Michelle Tucker about the concept he’s developed called “the five dimensions of multicultural education.” Especially for NEA Today Online readers, here is the complete interview. A shorter version ran in the September 1998 issue. Could you briefly describe the five...»

«European Journal of Science and Theology, September 2012, Vol.8, Supplement 2, 227-238 _ THE PEDAGOGIC AND THERAPEUTIC CHARACTER OF SAINT JOHN THE FASTER’S CANONS FROM ACRIBIA TO CONDESCENSION Ioan Cozma* ‘1 Decembrie 1918’ University of Alba Iulia, 5 Gabriel Bethlen Str., 510009, Alba Iulia, Romania (Received 20 May 2012, revised 11 July 2012) Abstract The penitential canons of Saint John the Faster are part of the canons called supplementary canons. The following three penitential...»

«Editorial: Guru Purnima by Durga Ahlund and M. G. Satchidananda Volume 11, number 2 Summer 2004 From the unreal lead us to the Real Form darkness lead us to the Light From death lead us to Immortality Gurupurnima is the festival where devotees celebrate The Guru. This year we celebrated it on July, 2004, at the Quebec Ashram. It was a magical night, with a full moon, pale yellow in color, on a balmy night. Later that night a terrific lightning storm came up and held us in awe. Purnima means...»

«The Fifty-first Dragon Heywood Broun Author : Essays Category : globalv November 2011 Submit by : Read this on Full Online Books Link : www.fullonlinebooks.com Source : Of all the pupils at the knight school Gawaine le Coeur-Hardy was among the least promising. He was tall and sturdy, but his instructors soon discovered that he lacked spirit. He would hide in the woods when the jousting class was called, although his companions and members of the faculty sought to appeal to his better nature by...»

«Journal of Thought, Fall-Winter 2011 79 Christine E. Sherretz Mindfulness in Education: Case Studies of Mindful Teachers and Their Teaching Practices Christine E. Sherretz University of Louisville Introduction For generations, educational philosophers, parents, business people, and practitioners have argued that public schools promote mindless standardization that stifles creativity, curiosity, and enthusiasm for learning. Dewey (1933) argued that schools try to instill uniformity and...»

«APPENDIX 111: TO SING HIS GLORY Revised and Expanded 2002 Contents of Appendix I11 by Sant Ajaib Singh Naam Ki Mehima Aparampar / 247 Guru Kirpal Mere Ghar Aana / 248 Sawan Chan Varga / 249 Chalo Man Satguru Ke Darbar / 250 Likh Chitthiya Sawan Nu / 251 Sawan Sawan Duniya Kehendi / 252 Mujhe Apna Bana Lo Kirpal / 253 Sawan Dayalu Ne Rim Jhim Lae / 254 Guru Guru Guru Guru / 256 Man Mandir Me Aao / 257 Sab Par Dya Karo Guru Pal / 258 Tum Se Mera Janam Janam / 259 Dekh Laya Asi Dekh Laya / 260...»

«2015 BUSINESS EDUCATION RESEARCH CONFERENCE PROCEEDINGS Sponsored by Association for Research in Business Education National Association for Business Teacher Education National Business Education Association P a g e | 2 Table of Contents Page Full Papers A Comparative Review of Mississippi Business Teacher Certification Methods.7 Pamela Scott Bracey, Mississippi State University, Mississippi State, MS Michelle Taylor. Mississippi State University, Mississippi State, MS An Emerging Trend: Cloud...»

«Available online at www.sciencedirect.com Educational Research Review 2 (2007) 130–144 Research review The use of scoring rubrics: Reliability, validity and educational consequences Anders Jonsson ∗, Gunilla Svingby School of Teacher Education, Malmo University, SE-205 06 Malmo, Sweden Received 3 August 2006; received in revised form 3 May 2007; accepted 4 May 2007 Abstract Several benefits of using scoring rubrics in performance assessments have been proposed, such as increased...»

«Writing for Change RAISING AWARENESS OF DIFFERENCE, POWER, AND DISCRIMINATION 3.1 Understanding Pejorative Terms Objectives Identify and analyze pejorative terms. Explore attitudes surrounding the terms. Describe how the terms contribute to perpetuation of stereotypes and ineffective communication. Explore how and by whom meaning is determined. Materials needed Copies of the handout Possibly, reference materials Time needed An hour or more Instructor directions Have your students read the given...»

«Medicus 5 Simple Keys® (5SK®) Student Guide Find a 5SK® instructor near you: http://5sk.com/
 Version 1.1 (2014-10-29) • (© 2012-2014, All Rights Reserved) Introduction Commonalities Among the Game’s Greatest Players During the last 100 years, the game’s best players have exhibited a few similarities, which most golfers aren’t aware of. Here at Medicus Golf, we came to this conclusion by spending countless hours studying hundreds of the game’s greatest golfers. Looking at these...»

«Document code: CLT/CPD/DIA/2008/RP/83 Title: Final Report of the Roundtable : Challenges of Education for Peace, Intercultural and Inter-religious Dialogue in Georgia, Armenia and Azerbaijan Date of publication: 2007 Sector of origin: CLT/CPD Roundtable Meeting CHALLENGES OF EDUCATION FOR PEACE, INTERCULTURAL AND INTER-RELIGIOUS DIALOGUE IN GEORGIA, ARMENIA AND AZERBAIJAN 27-28 June, 2007 Was organized in Tbilisi in June, (24 -27) 2007. The meeting gathered participants of the ToT course in all...»





 
<<  HOME   |    CONTACTS
2017 www.sa.i-pdf.info - Abstracts, books, theses

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.