FREE ELECTRONIC LIBRARY - Abstracts, books, theses

Pages:     | 1 |   ...   | 5 | 6 ||

«Consortium for Educational Research and Evaluation– North Carolina Comparing Value-Added Models for Estimating Individual Teacher Effects on a Statewide ...»

-- [ Page 7 ] --

Consortium for Educational Research and Evaluation–North Carolina 34 Comparing Value-Added Models August 2012 This study showed that the most common VAMs produce teacher rankings having markedly high correlations with each other, particularly in models with no violations or only modest violations of SUTVA. With few exceptions, the Spearman rank correlation did not highly discriminate between models. Further, many of the observed rank correlations that we deemed inadequate in comparison to those of the best performing models would have been viewed as high in any other research setting. Much the same could be said for the percentage of agreement in being categorized in the bottom 5%, which had both very high levels of agreement and little in the way of discriminating power. On the other hand, when categorization into the bottom 5% was framed as a question of the rate and number of teachers falsely identified as ineffective—false positives—severe differences between some models emerged, particularly under the scenario of non-ignorable assignment. Further, the findings as presented actually understate the risk to education agencies from choosing the wrong model. For a negative assignment scenario, under the best model (the HLM3), 221 5th grade teachers would be misclassified, while under the worst model (the DOLS), 436 5th grade would be misclassified, a difference of 215 teachers. Recall, however, that it is equally true that the same number of teachers would be misclassified as highly effective, and that in both cases teachers who are misclassified as ineffective (or effective) are counterbalanced by an equally large group of teachers who should have been identified as ineffective (effective) but were not. This implies that the actual difference is four times the amount, for 860 teachers per grade level. And it further underscores the risks associated with using any of these models, as even the best model, which had a misclassification of 221 5th grade teachers as ineffective, produced an overall misclassification of almost 900 teachers per grade.

This points to the second implication that this study raises: the risks to these misclassified teachers and their students will depend upon the uses of these evaluations, particularly the stakes assigned. Based on our findings, we believe that four of the tested VAMs, the HLM3+, HLM3, URM, and SFE, can provide objective information about the effectiveness of individual teachers in affecting the test score changes of their students for low stakes purposes. The evidence in this study suggests that the use of any VAMs for high stakes purposes is quite risky, even for the best performing models. The evidence also suggests that several of the VAM models are likely to be less accurate in estimating the actual effects of teachers, including the TFE, TFEIV, and DOLS.

Until additional research shows otherwise, these models should be considered risky even for low stakes purposes, although some of them performed well on some criteria even in this study.

Consortium for Educational Research and Evaluation–North Carolina 35 Comparing Value-Added Models August 2012 References Amrein-Beardsley, A. (2008). Methodological concerns about the Education Value-Added Assessment System. Educational Researcher, 37(2), 65–75.

Arellano, M., & Bond, S. (1991). Some tests of specification of panel data: Monte Carlo evidence and an application to employment equations. The Review of Economic Studies, 58, 277–298.

Ballou, D., Sanders, W., & Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29(1), 37–65.

Browne, W. J., Goldstein, H., & Rasbash, J. (2001). Multiple membership multiple classification (MMMC) models. Statistical Modeling 2001 (1), 103–124.

Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2006). Teacher-student matching and the assessment of teacher effectiveness. The Journal of Human Resources, 41(4), 778–820.

Goldhaber D., & Hansen, M. (2008). Assessing the potential of using value-added estimates of teacher job performance for making tenure decisions. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.

Gordon, R., Kane, T. J., and Staiger, D. O. (2006). Identifying effective teachers using

performance on the job (Hamilton Project Discussion Paper). Washington, DC:

Brookings Institution.

Guarino, C. M., Reckase, M. D., & Wooldridge, J. M. (2012). Can value-added measures of teacher performance be trusted? (Working Paper #18). East Lansing, MI: The Education Policy Center at Michigan State University.

Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature, 24, 1141–1177.

Harris, D. N. (2009). Teacher value-added: Don’t end the search before it starts. Journal of Policy Analysis and Management, 28(4), 693–699.

Henry, G. T., Kershaw, D. C., Zulli, R. A., & Smith, A. A. (in press). Incorporating teacher effectiveness into teacher preparation program evaluation. Journal of Teacher Education.

Hill, H. C. (2009). Evaluating value-added models: A validity argument approach. Journal of Policy Analysis and Management, 28(4), 700–712.

Holland, Paul W. (1986) Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.

Consortium for Educational Research and Evaluation–North Carolina 36 Comparing Value-Added Models August 2012 Koedel, C., &Betts, J. R. (2011). Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique. Education Finance and Policy, 6(1), 18–42.

McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T.A., & Hamilton, L. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67–101.

Morgan, S. L., & Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge, UK: Cambridge University Press.

Nye, B., Konstantopolous, S., & Hedges, L. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237–257.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

Raudenbush, S. W. & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20(4), 307–335.

Reardon, S. F., & Raudenbush, S. R. (2009). Assumptions of value-added models for estimating school effects. Education Finance and Policy, 4(4), 492–519.

Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252.

Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay and student achievement. The Quarterly Journal of Economics, 25(1), 175–214.

Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104(8), 1525–1567.

Rubin, D. B., Stuart, E. A., & Zanutto, E. L. (2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29(1), 103– 116.

Sass, T. (2008). The stability of value-added measures of teacher quality and implications for teacher compensation policy. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.

Schochet, P. Z., & Chiang, H. S. (2010). Error rates in measuring teacher and school performance based on student test score gains. Washington, DC: Institute for Education Sciences.

Consortium for Educational Research and Evaluation–North Carolina 37 Comparing Value-Added Models August 2012 Tekwe, C. D., Carter, R. L., Ma, C. X., Algina, J., Lucas, M. E., et al. (2004). An empirical comparison of statistical models for value-added assessment of school performance.

Journal of Educational and Behavioral Statistics, 29(1), 11–36.

Todd, P. E., & Wolpin, K. I. (2003). On the specification and estimation of the production function for cognitive achievement. The Economic Journal, 113, f3–f33.

Vale, C. D., & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions.

Psychometrika, 48, 465–471.

Wright, S. P., White, J. T., Sanders, W. L., & Rivers, J. C. (2010). SAS EVAAS statistical models. Cary, NC: The SAS Institute.

–  –  –

Pages:     | 1 |   ...   | 5 | 6 ||

Similar works:

«The Academic and Social Value of Ethnic Studies A Research Review Christine E. Sleeter National Education Association Research Department Ronald D. Henderson, Director The Academic and Social Value of Ethnic Studies A Research Review Christine E. Sleeter National Education Association Research Department Ronald D. Henderson, Director The National Education Association is the nation’s largest professional employee organization, representing 3.2 million elementary and secondary teachers, higher...»

«RTI Toolkit: A Practical Guide for Schools Six Reasons Why Students Are Unmotivated (and What Teachers Can Do) Jim Wright, Presenter 12 July 2011 National Association of School Psychologists Summer Critical Issues Conference/Indianapolis, IN Contents  Motivation Deficit 1: The student is unmotivated because he or she cannot do the assigned work...02  Motivation Deficit 2: The student is unmotivated because the ‘response effort’ needed to complete the assigned work seems too great....»

«No 5 – 29 May 2015 Welcome It is amazing how quickly time passes during a term at school. We are now half way through with three weeks to go until teachers complete their assessments before preparing end of semester reports. While we see high quality learning in every classroom every day, this newsletter tells a story about the many special events and highlights that have happened over the last few weeks to challenge, extend and enrich the learning opportunities available to Blackwood High...»

«Journal of Education and Practice www.iiste.org ISSN 2222-1735 (Paper) ISSN 2222-288X (Online) Vol.6, No.6, 2015 Guidelines on How to Read a Physics Textbook and the Assessment of the Readability of Recommended Physics Textbooks in Secondary Schools in Osun State of Nigeria Akinyemi Olufunminiyi Akinbobola Department of Special Education and Curriculum Studies, Adeyemi College of Education, Ondo, Nigeria akinbobola2006@yahoo.com Abstract This study assessed the readability of the four...»

«1 COMMENTARY BY KHENPO NAMDROL 4. Karma: the relationship of cause and effect First of all, Patrul Rinpoche pays homage to his root guru by describing his qualities: Abandoning nonvirtue and taking up virtuous actions, following the teachings on cause and effect, His conduct in accord with the ascent of the vehicles, His perfectly pure view with no trace of clinging, At the feet of my peerless guru, I bow down. Patrul Rinpoche’s root guru, Jigme Gyalwai Nyugu, had achieved the view of the...»

«4 HAMPSTEAD AVENUE HIGHGATE 5063 PHONE: 8271 9622 FAX: 8373 6075 EMAIL: dl.1063_info@schools.sa.edu.au INTERNET: http://web.highgates.sa.edu.au/argon Principal Mr Phil Johansen Term 2 No 1 Thursday 12 May 2016 PRINCIPAL’S MESSAGE DIARY DATES Welcome back to school for a busy term 2. We warmly welcome two new staff members Emily Saint-Bowman, PE Teacher for term 2, and Friday 13 May Emily White working with Amy in Room 30. Early Years Assembly The annual Greenhill South Partnership review was...»

«Social Work Education in Europe Franz Hamburger, Günther Sander & Manfred Wöbcke Translator’s note: Approximate translations of the terms ‘Fachschule’ and ‘Fachhochschule’ are, respectively ‘technical college’ and ‘college’. However, since there is no exact correspondence between German and English educational establishments, the German terms have been retained throughout this document. All European states with the exception of the small principalities offer training for the...»

«Commedia dell' arte (six workshops) ASSESSMENT CRITERIA Playing a comedy character PUPILS TARGET Understand and play a comedy character more successfully Gifted and Talented Consideration In order to cater for Gifted and Talented pupils the teacher must identify such pupils and adapt each lesson plan to suit the students' individual needs. Lesson plans should be adapted by the teacher in the classroom by adopting some of the strategies listed below for G&T pupils.  Open questions in...»

«MASSACHUSETTS DIVISION OF PROFESSIONAL LICENSURE POLICY REGARDING REVIEW OF CONVICTION AND PENDING CRIMINAL CASE DATA REGARDING APPLICANTS FOR PRIVATE OCCUPATIONAL SCHOOL LICENSURE Effective February 3, 2016 Pursuant to M.G.L. c. 112, § 263, the Division of Professional Licensure (DPL) must review criminal offender record information (CORI) of principals, administrators, employees, instructors, and other individuals related to private occupational schools for purposes of school licensure and...»

«Ida B... and Her Plans to Maximize Fun, Avoid Disaster, and (Possibly) Save the World by Katherine Hannigan NOVEL STUDY ©Arlene Manemann 2013 © Arlene Manemann: The English Page. The receiver of this document is granted permission to reproduce the pages in needed quantities for the purchaser's classroom only. Minor editing is allowed; the document remains under copyright even when edited. Duplication for other classes or by other teachers or for use in wide distribution as within a school...»

«MWBE MINORITYAND WOMEN-OWNED BUSINESS ENTERPRISES ASSET MANAGEMENT AND FINANCIAL INSTITUTION STRATEGY REPORT 2014-2015 Fiscal Year Submitted: September 1, 2015 Thomas K. Lee Updated: November 16, 2015 Executive Director & Chief Investment Officer New York State Teachers’ Retirement System (800) 348-7298 NYSTRS.ORG This page intentionally left blank. Table of Contents Page Financial Highlights Introduction – System Overview New York’s MWBE Asset Management and Financial Institution...»

«This case has been abridged by the instructor. Abridgements are not noted. COURT OF CRIMINAL APPEALS OF TEXAS November 22, 1989 RAY EDWIN BILLINGSLEA, APPELLANT v. THE STATE OF TEXAS, APPELLEE Petition for Discretionary Review from the Fifth Court of Appeals, Dallas County. En Banc. Duncan, III, Judge. McCormick, P. J., not participating. White, J., dissents. Author: Duncan Appellant was charged with the offense of injury to an elderly individual pursuant to V.T.C.A. Penal Code, § 22.04(a)(1)....»

<<  HOME   |    CONTACTS
2017 www.sa.i-pdf.info - Abstracts, books, theses

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.