FREE ELECTRONIC LIBRARY - Abstracts, books, theses

Pages:     | 1 || 3 | 4 |   ...   | 5 |

«Draft, April 21, 2008 Teacher Effects: What Do We Know? Helen F. Ladd Edgar Thompson Professor of Public Policy Studies and professor of economics Duke ...»

-- [ Page 2 ] --

Models of this form (but with additional explanatory variables as discussed below) are typically referred to as value-added models and are commonly used in the literature to estimate β, namely the effect of current teachers on current achievement. Their popularity comes largely from their simplicity and intuitive appeal. Logically, it makes sense that one would want to control statistically for the achievement, or knowledge, that the student brings to the classroom at the beginning of the year when estimating the effect of her current teacher. In addition, the valueadded model is flexible in that it does not impose a specific assumption about the rate at which knowledge persists over time; instead it allows that rate to be estimated. Nonetheless, the model is valid only if the underlying assumptions about the constancy of effects are valid. Further such models raise statistical concerns because of the inclusion on the right hand side of the equation of the lagged achievement term, which in the presence of serial correlation would be correlated with the error term.

Gains model This last statistical problem can be avoided by assuming there is no decay in knowledge so that the persistent parameter, α, equals 1 and moving the lagged achievement term to the left

hand side of the equation. This procedure generates the gains model:

Ait –Ait-1 = β Tit + εit. (5) In this case, the parameter, β, refers to the effect of teacher quality on the gains in achievement.

If the assumptions underlying the initial value-added model are correct, however, and the decay rate is not zero, the gains model is mispecified. The reason is that the term (α-1)Ai,t-1 is now missing from the right hand side of the equation. To the extent that prior year achievement is positively correlated with teacher effects, the teacher effects would be biased downward. Thus, within the framework of education as a cumulative process, the shift to the gains model solves one statistical problem but introduces a new one.

Full value added (or gains) model with student fixed effects In fact, most researchers estimate a richer form of the simple model in equation 4, one that includes time varying student characteristics, classroom or school characteristics, and student fixed effects. This full model can only be estimated with longidudinal data on individual students and multiple cohorts of students. If data are available for only a single cohort of

Ladd, Teacher Effects, Draft April 21, 2008

students, no classroom characteristics such as class size or the composition of the students can be included in the equation because teachers and their classrooms are indistinguishable.

Ait = αAit-1 + β T it + γ Xit + δCit + θi + ηit (6) Where Ait, A i, t-1 and T are as defined above and Xit are time varying student variables 3 Cit are classroom and school characteristics in year t θi are student fixed effects ηit is an error term For this model to be consistent with the cumulative model of the education process, the same assumptions that were needed to derive the simple value added model in equation 4 are needed.

In particular, each of the variables must exert a constant linear effect on student achievement in each year and their effects on student achievement must all decay at the same rate (1-α).

The student fixed effects are a crucial part of this enriched model. They control for the time-invariant characteristics of students – both those that are measurable and those that are not and under certain assumptions address the fundamental problem highlighted above, namely that the teachers are not randomly assigned to students. The inclusion of student fixed effects means that the teacher effects are derived from the within-student variation in student achievement. The key assumption needed for student fixed effects to address fully the concern about nonrandom sorting is that students are assigned to teachers based on their permanent or average characteristics rather than on any time-varying unmeasurable characteristics. Most value added studies of teacher effects either implicitly or explicitly make this assumption. I return below to Jesse Rothstein’s recent test of the validity of this assumption.

In the context of these models, the teacher variables are typically entered as 0-1 indicator variables, either for each teacher or for each teacher by year. Thus teacher effects are estimated by the method of teacher fixed effects (in contrast to the method of random effects), an approach that seems reasonable given the goal of determining the effectiveness of a specific group of actual teachers.

Two issues arise in the estimation and interpretation of such models. One is the technical challenge of using a program such as STATA to generate teacher effects in a model that also includes student fixed effects. Though STATA can easily handle one set of fixed effects through the process of demeaning – e.g by subtracting the mean value for each student from all the variables in the model – it cannot use that procedure simultaneously for a second set of fixed effects. A natural solution to that technical problem is to create a new set of indicator variables that combine the student and teacher indicator variables into a single set of student-teacher indicator variables. Though that process works well for some purposes, it has the disadvantage of making it difficult to capture the individual teacher effects. New programs are becoming available to address this technical program (Corneliβen. 2006). An alternative solution to this technical problem is to replace the student fixed effects with a vector of student characteristics.

Often included as time varying student variable are indicators for whether a student has changed schools, either independent of other students or as part of a move with others from one level of schooling to another.

Ladd, Teacher Effects, Draft April 21, 2008 That approach, however, misses all the unmeasurable characteristics of students that could well be correlated with teacher quality.

The second issue relates to measurement error. The coefficients of the teacher indicator variables are estimated with different degrees of precision. Had they been estimated by random effects rather than by fixed effects, estimates for individual teachers would have been shrunken toward the mean, with the amount of shrinkage greater for the teacher effects that are estimated with less precision. Letting βt* represent the predicted teacher effect for teacher t that emerges from a fixed effects specification, βt the true value and ε a random error, we can express the predicted teacher effect that emerges from a fixed effect specification as a function of the true

effect plus an error term as follows:

β t* = β t + ε (7) One might then calculate an adjusted teacher effect for any given teacher as a weighted average of the estimated teacher effect for that teacher and the mean teacher effect for the sample as a


λ βt* +(1-λ) mean Bt* (8) where λ = Varβt /(Var βt + Var ε).

Thus, the larger is the random error of the estimate, the smaller is λ and the greater the weight placed on the mean teacher effect. Though such an adjustment is conceptually straightforward, it can be difficult to implement in practice because it requires the standard errors for each of the estimated teacher effects, which can be difficult to estimate (Lockwood, McCaffrey and Sass, 2008). One implication of this shrinkage procedure is that teachers who teach small numbers of students are unlikely to be identified as either particularly effective or particularly ineffective teachers. Although the outcome on the low side may be appropriate since it would protect decent teachers with small classes from being unjustly sanctioned, the shrinkage procedure could also keep some very effective teachers from being recognized.

Additional considerations Though much more could be said about this standard value added (or gains model), I add here only two additional considerations. The first refers to the role of parents. As pointed out by Todd and Wolpin ( 2003) compensating behavior by parents could potentially mute the estimated differences in teacher effectiveness. That outcome would occur if parents spend more productive time working on school work with their children when their children have ineffective teachers than when they have effective teachers.

Another is whether to include school fixed effects in the model. Often they are not included, particularly if student fixed effects are in the model, as in equation 6. In the absence of student fixed effects, the addition of school fixed effects can help mitigate the problem caused by the non-random assignment of teachers to students. Their inclusion in the model means that teacher effects are identified solely by differences in teacher quality within schools. As a result, the estimates of teacher effects are not contaminated by the fact that the more effective teachers are more likely to end up in schools with the more able and more motivated students. Including school, rather than student fixed effects, however, does not account for the possibility that the

Ladd, Teacher Effects, Draft April 21, 2008

more able students within a school may be assigned to the higher quality teachers. 4 At the same time, their inclusion means that a teacher’s effectiveness is measured relative to other teachers in the school rather than to a broader set of teachers. As shown in Table 1 above, the overall estimated variation in teacher effectiveness will be smaller when school fixed effects are included than when they are excluded.

How stable are teacher effects?

In most cases one would expect that a teacher who is very effective (or ineffective) in one year would be similarly effective (or ineffective) in the following year. Hence, one way to evaluate the validity of the teacher effects that emerge from value added models is to examine their stability from one year to the next. The more unstable they are they less useful they are likely to be for making high stakes decisions about teachers.

Only a few studies have explored the stability of teacher effects (Ballou, 2005, Aaronson et al (2007) and Koedel and Betts (2007). Such studies find that teacher effects are quite unstable. For example, consider the findings of Koedel and Betts(2007) for teachers in San Diego. After ranking the teachers by their estimated fixed effects for two years in a row, the researchers find that among those who are ranked in the lowest quintile in the first year, only 30 percent stay in that quintile in the next year and another 31 percent move up to one of the top two quintiles. A similar pattern emerges at the top of the distribution. While 35 percent of teachers who are initially ranked in the top quintile remain in that quintile in the second year, 30 percent of them fall to the first or second quintile (cited in Lockwood, McCaffrey and Sass, 2008, p. 3).

The most complete study of the stability of teacher effects is by Lockwood, McCaffrey and Sass (2008). This study is based on middle school math teachers in six large Florida districts from 2000/01 to 2004/05. The authors focus on middle school teachers because the fact that they teach multiple sections of students means the teacher effects estimated for them are likely to be more stable than those for elementary school teachers, and on math teachers because teacher effects are generally larger for math than for reading. The authors start with a very simple gains model -- one with student fixed effects and teacher-by-year fixed effects – and then examine how modifying the model changes the results. They estimate all models as the district level and do not include school fixed effects. Thus, the teacher effects are measured relative to the average of all teachers in the district in the relevant subject and grade range, not relative to the average teacher at a given school.

The findings are quite clear. The correlations of teacher fixed effects across adjacent years in each district are moderately low, typically in the range of 0.3 to 0.5 and do not change much as the model is modified with additional covariates or modified. In addition, like Koedel and Betts (2007), the authors find substantial movement of teachers from one part of the effectiveness distribution to another in successive years. For example, among teachers who were in the top quintile (not quartiles as in Koedel and Betts) in 2003/04,the percentages of teachers who remained in that quintile in the next year ranged from a high of 46 in Broward County to a low of 23 percent in Palm Beach (Lockwood, McCaffrey and Sass, 2008, Table 3).

At the elementary level, the nonrandom matching of students to teachers appears to be a far larger problem than the nonrandom matching of students to teachers across classrooms within schools in North Carolina (Clotfelter, Ladd and Vigdor, 2006).

Ladd, Teacher Effects, Draft April 21, 2008 The authors tried to determine the causes of the instability by examining the effects of class size, whether or not the test scores are normalized, the extent to which teachers have some students in common, and the addition of covariates to the value-added model. With a few minor exceptions the instability of the effectiveness rankings was not very sensitive to the various changes. 5 The authors conclude that their findings suggest the need for caution in using valueadded estimates of individual teacher productivity for high-stakes personnel decisions.

The Rothstein challenge Another challenge to the validity of the value-added approach to estimating teacher effects appears in a recent paper by Jesse Rothstein (2007). As emphasized above, one of the advantages of longitudinal data sets for estimating teacher effects is that they permit the researcher to use student fixed effects to control for the time-invariant student-level characteristics – both measured and unmeasured – that may be correlated with the teacher measures. The inclusion of fixed effects for students solves the problem of the non-random matching of students to teachers, however, only when such matching is based on the time invariant characteristics of the students, such as their basic ability or motivation. Rothstein refers to such matching as “static tracking” and contrasts it to the “dynamic tracking ” that occurs when school administrators sort students into classrooms and teachers in a non-random way that is based in part on the student’s current performance.

He correctly emphasizes the importance of testing the assumption of static tracking and does so by introducing a placebo. In particular, using data for one cohort of elementary school students in North Carolina, he estimates a value added model that includes not only the student’s current teacher (e.g. her fourth grade teacher ) but also the student’s subsequent teacher in the following grade (e.g. her fifth grade teacher). If the basic value-added model is correct, the fifth grade teacher should have no impact on the student’s four grade test scores (or more precisely in the context of a model with student fixed effects, on the extent to which the student’s fourth grade test score deviates from the her average test scores). In fact, however, he finds that the student’s fifth grade teacher has almost as big an impact on her fourth grade scores (in reading) as does her fourth grade teacher. He argues that that this outcome occurs because the student’s fourth grade test score is used to determine her fifth grade teacher.

Pages:     | 1 || 3 | 4 |   ...   | 5 |

Similar works:

«MWBE MINORITYAND WOMEN-OWNED BUSINESS ENTERPRISES ASSET MANAGEMENT AND FINANCIAL INSTITUTION STRATEGY REPORT 2014-2015 Fiscal Year Submitted: September 1, 2015 Thomas K. Lee Updated: November 16, 2015 Executive Director & Chief Investment Officer New York State Teachers’ Retirement System (800) 348-7298 NYSTRS.ORG This page intentionally left blank. Table of Contents Page Financial Highlights Introduction – System Overview New York’s MWBE Asset Management and Financial Institution...»

«Course Descriptions AAST 2300 INTRODUCTION TO AFRICAN AMERICAN STUDIES (3-0) This course introduces students to the African American experience in the United States, including an interdisciplinary analysis of the African American experience in politics, the arts, folklore, religion, economics, sociology, psychology, and community development; and an examination of local history, contemporary issues, and recent events in the African American community. AAST 2337 ECONOMICS OF SOCIAL ISSUES (3-0)...»

«Program on Education Policy and Governance Working Papers Series The Value of Smarter Teachers: International Evidence on Teacher Cognitive Skills and Student Performance Eric A. Hanushek, Marc Piopiunik, Simon Wiederhold§ PEPG 14-06 Harvard Kennedy School 79 JFK Street, Taubman 304 Cambridge, MA 02138 Tel: 617-495-7976 Fax: 617-496-4428 www.hks.harvard.edu/pepg/ § Hanushek: Hoover Institution, Stanford University, CESifo, and NBER, hanushek@stanford.edu; Piopiunik: Ifo Institute at the...»

«A TEACHER’S GUIDE TO THE SIGNET CLASSIC EDITION OF GEORGE ORWELL’S By LISA SESSIONS, Asheville Junior High, Asheville, NC SERIES EDITORS: W. GEIGER ELLIS, ED.D., UNIVERSITY OF GEORGIA, EMERITUS and ARTHEA J. S. REED, PH.D., UNIVERSITY OF NORTH CAROLINA, RETIRED A Teacher’s Guide to the Signet Classic Edition of George Orwell’s 1984 2 INTRODUCTION George Orwell’s 1984 offers a thought-provoking learning experience for high schools students. It provides challenging reading, stimulating...»

«Teaching Contingent Valuation and Promoting Civic Mindedness in the Process Roland Cheo Department of Economics, Monash University Abstract Economics majors are often assumed to lack civic mindedness.The purpose of this paper then is to demonstrate how by engaging students in the proper understanding of contingent valuation (CV) methodology and by evaluating a social service, we can improve student outcomes in two areas: increasing their competence in research design as well as in the process...»

«Table of Contents How Teachers Use the Textbook: Lessons from Three Secondary School Classrooms Kathleen A. Hinchman Syracuse University When the science room was nearly filled with tenth grade students, a bell rang. Chris Raymond walked to the front of the room and said, “Take out your homework.” Students rustled through notebooks and binders, each producing a printed 8 l/2 by 11 inch sheet of paper. Chris looked around at the students, checking to see who did and did not produce this...»

«Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles Andrew Bacher-Hicks Harvard University Thomas J. Kane Harvard University and NBER and Douglas O. Staiger Dartmouth College and NBER Author Note Bacher-Hicks: John F. Kennedy School of Government, Harvard University, 79 John F. Kennedy Street, Cambridge, MA 02138 (e-mail: abacherhicks@g.harvard.edu). Kane: Harvard Graduate School of Education, 50 Church St., 4th Floor Cambridge, MA 02138 (email:...»

«What Every Teacher Needs to Know about CHILD ABUSE A resource guide for educators and school personnel Brought to you by: Family Nurturing Center 8275 Ewing Blvd. Florence, KY 41042 859-525-3200 www.familynurture.org Dear Educator: We are pleased to provide you with this resource packet to help you address the difficult issue of child abuse and neglect. Millions of children are reported each year as victims of physical abuse, emotional maltreatment, neglect and sexual abuse. As a teacher or...»

«130 International Conference on e-Learning’14 Augmented Reality Textbook for Future Blended Education Galina Ivanova, Yuksel Aliev, Aleksandar Ivanov Abstract: The report presents an augmented reality textbook for future blended education. The efficiency of blended learning is discussed. The characteristics of augmented reality technology are described. An experimental augmented reality textbook for mechanical engineering students’ blended training in the University of Ruse is presented....»

«Learner’s Strategy Use to Guess word Meanings during Interactive Read-Aloud: A Case Study Author Name: Xiaoya Zhou University of Stirling British Council ELT Master’s Dissertation Awards: Commendation MSc in Teaching English to Speakers of Other Languages Learner's Strategy Use to Guess word Meanings during Interactive Read-Aloud: A Case Study August 2014 Abstract Story read-aloud has been advocated to be effective in promoting children's vocabulary acquisition (Robbins & Ehri, 1994; Elley,...»

«THE BOOT CAMP SURVIVAL GUIDE DEVELOPED FROM MIKE VOLKIN'S THE ULTIMATE BASIC TRAINING GUIDEBOOK Preface Prepare Yourself Early Drill Instructors: The Mental Game Interview with a Drill Instructor Day 1 Tips for Success Frequently Asked Questions Conclusion © 2007 Military Advantage, Inc. 0 Preface I knew absolutely nothing about the military when I entered basic training. I had no military family history and no prior desire to ever join the military. It was on the tragic morning of September...»

«Summer reading loss is the basis of almost all the rich/poor reading gap. Richard L. Allington Anne McGill-Franzen rallingt@utk.edu amcgillf@utk.edu University of Tennessee College of Education, Health, and Human Sciences.To appear in: R. Horowitz and S. J. Samuels (eds.).The achievement gap in reading: Complex causes, persistent issues, and possible solutions. New York: Routledge. Summer reading loss is the basis of almost all of the rich/poor reading gap. Although recent federal policy has...»

<<  HOME   |    CONTACTS
2017 www.sa.i-pdf.info - Abstracts, books, theses

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.