WWW.SA.I-PDF.INFO
FREE ELECTRONIC LIBRARY - Abstracts, books, theses
 
<< HOME
CONTACTS



Pages:   || 2 | 3 | 4 | 5 |   ...   | 14 |

«University of Nebraska - Lincoln DigitalCommons of Nebraska - Lincoln Dissertations and Theses in Statistics Statistics, Department of 8-2010 ...»

-- [ Page 1 ] --

University of Nebraska - Lincoln

DigitalCommons@University of Nebraska - Lincoln

Dissertations and Theses in Statistics Statistics, Department of

8-2010

Estimating Teacher Effects Using Value-Added

Models

Jennifer L. Green

University of Nebraska at Lincoln, jennifer.green@huskers.unl.edu

Follow this and additional works at: http://digitalcommons.unl.edu/statisticsdiss

Part of the Statistical Models Commons Green, Jennifer L., "Estimating Teacher Effects Using Value-Added Models" (2010). Dissertations and Theses in Statistics. Paper 6.

http://digitalcommons.unl.edu/statisticsdiss/6 This Article is brought to you for free and open access by the Statistics, Department of at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Dissertations and Theses in Statistics by an authorized administrator of DigitalCommons@University of Nebraska Lincoln.

ESTIMATING TEACHER EFFECTS USING VALUE-ADDED MODELS

by Jennifer L. Green

A DISSERTATION

Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of Requirements For the Degree of Doctor of Philosophy Major: Statistics Under the Supervision of Professor Erin E. Blankenship Lincoln, Nebraska August, 2010

ESTIMATING TEACHER EFFECTS USING VALUE-ADDED MODELS

Jennifer L. Green, Ph.D.

University of Nebraska, 2010 Advisor: Erin E. Blankenship Value-added modeling is an alternative approach to test-based accountability systems based on the proportions of students scoring at or above pre-determined proficiency levels. Value-added modeling techniques provide opportunities to estimate an individual teacher’s effect on student learning, while allowing for the possibility to control for the effect of non-educational factors beyond a school system’s control, such as socioeconomic status. However, numerous considerations exist when using value-added models to estimate teacher effects and defining what the teacher effects really describe.

Chapter 2 provides an introduction to value-added methodology by describing several value-added models available for estimating teacher effects and their respective advantages and disadvantages. Modeling variations and their impact on estimated teacher effects are also discussed in addition to the various statistical and psychometric issues associated with estimating value-added teacher effects.

Because value-added analyses require high-quality longitudinal datathat are often not available, Chapters 3 and 4 propose methodology for analyzing less-than-ideal assessment data. Chapter 3 proposes value-added methodology for analyzing longitudinal student achievement data not on a single developmental scale and addresses issues arising when using a layered, longitudinal mixed model to analyze gains in standardized scores.

The chapter also discusses methods for estimating teacher effects on student learning before and after entering professional development programs and applies these methods of analysis to achievement data.

Chapter 4 describes the use of curve-of-factors methodology to analyze longitudinal achievement data collected from two differently scaled assessments in a single year and subject, such as mathematics. Assuming data come from a curve-offactors model structure, a simulation study evaluates the performance of the proposed curve-of-factors model in its ability to accurately rank teachers in the presence of either complete or missing test data and compares it to the performance of the Z-score

–  –  –

I am grateful for all of the guidance and encouragement my advisor, Dr. Erin Blankenship, has provided me throughout this whole process. Without her support and unending patience, this dissertation would not have reached its full potential. I also thank my other committee members, Dr. Walt Stroup, Dr. Steve Kachman, Dr. Ruth Heaton and Dr. Jim Bovaird, for their help and support throughout my academic program. I am eternally grateful for my committee’s willingness to spend countless hours answering questions and helping me solve the seemingly unsolvable. In addition, I thank Dr. Jim Lewis for all of the support and guidance he has provided over the past three years. I appreciate the faculty’s sincere enthusiasm for my research and their investment in my education and my future. I especially thank Joe, Brianna, my parents, my family and my friends for their patience, support, understanding and encouragement during this time.

Without you, this dissertation could not have been written. To all: thank you for believing

–  –  –

This research is supported by the Math in the Middle Institute Partnership. The Math in the Middle Institute Partnership is funded by a Math-Science Partnership grant from the National Science Foundation (NSF MSP Grant #EHR-0142502). Any opinions, findings and conclusions or recommendations expressed in this material are those of the

–  –  –

1 Introduction

1.1 Value-Added Models for Estimating Teacher Effects

1.2 Estimating the Impact of a Professional Development Program on Student Learning

1.3 Using Parallel Processing Methodology to Estimate Teacher Effects............... 3  2 Value-Added Models for Estimating Teacher Effects





2.1 Introduction

2.2 Value-Added Models

2.2.1 Covariate Adjustment Models

2.2.2 Gain Score Models

2.2.3 Multivariate Models

2.2.3.1 Cross-Classified Models

2.2.3.2 EVAAS Model

2.2.3.3 Variable Persistence Model

2.3 Estimating Teacher Effects

2.3.1 Layered Models

2.3.2 Best Linear Unbiased Predictors

2.3.3 Impact of Model Specification on Teacher Effect Estimates

2.4 Issues

2.5 Summary and Future Work

3 Estimating the Impact of a Professional Development Program on Student Learning

3.1 Introduction

3.2 Methods for Estimating the Impact of a Professional Development Program on Student Learning

3.2.1 Background

3.2.2 EVAAS Layered Teacher Model

3.2.3 Modified Layered Model

3.2.4 Z-scores

3.3 Example: Math in the Middle Institute Partnership

3.3.1 Modified Layered Model Implementation

3.3.2 Results

3.4 Summary and Future Work

4 Using Parallel Processing Methodology to Estimate Teacher Effects

viii

4.1 Introduction

4.2 Parallel Processing Methodology

4.2.1 Introduction to Parallel Processing

4.2.2 Parallel Processing in a Value-Added Context

4.3 Example: Student Achievement Simulation Study

4.3.1 Simulation Study Description

4.3.2 Results

4.4 Summary and Future Work

Conclusions

References

Appendix A

Appendix B

–  –  –

Figure12.1: Relationship among Models

Figure 2.2: Comparison of Multivariate Models for Modeling Longitudinal Student Achievement Data

Figure32.3: Student Scores and Teachers over Time

Figure43.1: Comparison of Before Participation Effects between Teachers Participating (n = 37) and Not Participating (n = 280) in M2

Figure53.2: Comparison of Differences between Before and After Participation Effects for MPS Teachers (n = 37) and a Subset of MPS Teachers (n = 22) Participating in M2

Figure64.1: Visual of Cross-Classified Data Structure

Figure74.2: Level One Curve-of-Factors Model

 mˆ Figure84.3: Comparison of RMSE iP for the Curve-of-Factors (solid), CA Z-score (dash) and MA Z-score (dotted) Models with Complete and Missing Tests, and MA/CA Z-score (dot-dash) Model with Missing Tests for Specific Percentiles and Years

–  –  –

and MA Z-score (dotted) Models with Complete and Missing Tests, and MA/CA Z-score (dot-dash) Model with Missing Tests for Specific Percentiles and Years

–  –  –

Table12.1: Comparison of Coefficient Matrices for Teacher Effects in Non-Layered and Layered Models

Table22.2: Teacher Effect Estimates from NLM(0), NLM(0.

7), LM(0) and LM(0.7)..... 31  Table33.1: Comparison of Z Matrix in Non-Layered and Layered Models

Table43.2: Zb and Za Matrices

Table53.3: Standardized Layered Z Matrix

Table63.4: Standardized Zb and Za Matrices

Table73.5: MPS Middle School Assessments between 2003-04 and 2007-08.

................ 52  Table84.1: Comparison of Z Matrix in Non-Layered and Layered Models

Table94.2: MPS Middle School Assessments between 2003-04 and 2007-08.

................ 70  Table 4.3: Assessments used as Responses for Simulation Analysis with Missing Tests 73  Chapter 1 Introduction Over the past several years, there has been a national effort to hold students to higher academic standards. This effort includes holding states accountable for assessing measurable student outcomes. Value-added modeling is an alternative approach to testbased accountability systems interested in the proportions of students scoring at or above pre-determined proficiency levels. Value-added modeling techniques estimate the contribution of educational factors, such as teachers, to growth in student achievement, while allowing for the possibility to control for the effect of non-educational factors beyond a school system’s control, such as socioeconomic status. Value-added modeling methods provide opportunities to estimate the proportion of variability in achievement or student growth attributable to teachers, as well as estimate an individual teacher’s effect on student learning.

School districts and policymakers desire to use teacher effect estimates for a variety of purposes, from informing educational systems how students are affected by current practices and conditions to making high-stakes decisions regarding teacher salary and/or employment. These estimates are also desired to evaluate the effectiveness of professional development programs. However, even though value-added modeling methods infer causal effects of teachers on student growth, the assessment data are not obtained from randomized, experimental studies. Consequently, several obstacles need to be addressed before value-added modeling should be used in these ways.

1.1 Value-Added Models for Estimating Teacher Effects Chapter 2 serves as a background and introduction to value-added methodology.

Several value-added models available for estimating teacher effects are described, as are the models’ respective advantages and disadvantages. Modeling variations, such as the use of layered versus non-layered design matrices and the specification of teacher effects as fixed or random, are also discussed, and the impact of such considerations on estimated teacher effects is explained in detail using an example provided by Wright and Sanders (2008). The various statistical and psychometric issues associated with estimating value-added teacher effects are highlighted, providing a summary of the current state of value-added modeling research and recommendations for future work.

1.2 Estimating the Impact of a Professional Development Program on Student Learning Professional development programs focus on preparing teachers to meet the recent initiatives on improving the quality of student instruction, but rigorous evaluations are needed to determine whether these programs are actually effective. Value-added modeling techniques provide opportunities to estimate the relationship between teacher development and student learning, but most require student achievement data to be on a single developmental scale over time (McCaffrey, Lockwood, Koretz, & Hamilton, 2003). Typically, available assessment data do not meet such requirements, limiting analyses that can be conducted. Chapter 3 proposes alternative value-added methodology, specifically the use of Z-scores, for analyzing less-than-ideal longitudinal student achievement data collected from a mixture of norm- and criterion-referenced assessments to estimate the impact of a professional development program on student learning. The chapter discusses methodology for estimating teacher effects on student learning before and after entering professional development and addresses issues arising when using a layered, longitudinal linear mixed model to analyze gains in standardized scores. The methodology is applied to data collected from a mathematics professional development program in mathematics education, the Math in the Middle Institute Partnership (M2), and the results are discussed.

1.3 Using Parallel Processing Methodology to Estimate Teacher Effects Few studies have addressed how to use value-added models to analyze achievement data not on a single developmental scale (Green, Smith, Heaton, Jiao, & Stroup, under review; Rivkin, Hanushek, & Kain, 2005), and even fewer, perhaps none, have discussed how to use information from multiple instruments in a single year that are on different scales, potentially both within and between instruments over time. When modeling multiple outcome measures, instead of a single measure across time, parallel process, or multivariate, growth curve models can estimate the relationship between the growth trajectories for each of the parallel measures and allow researchers to investigate changes in latent factors over time instead of changes in observed scores. Chapter 4 describes the use of parallel processing, specifically curve-of-factors, methodology to analyze longitudinal student achievement data collected from two different assessments in a single subject, such as mathematics, and estimate teachers’ effects on student learning. Assuming data come from a curve-of-factors model structure, a simulation study evaluates the performance of the proposed curve-of-factors model in its ability to accurately rank teachers in the presence of either complete or missing test data and compares it to the performance of the Z-score methodology proposed in Chapter 3.

Chapter 2 Value-Added Models for Estimating Teacher Effects

2.1 Introduction Since the enactment of No Child Left Behind (NCLB) (2001), education systems, in theory, have held students to higher academic standards, and states are accountable for assessing measurable student outcomes. States receiving Title I funds for improving the academic achievement of disadvantaged students must require schools to make adequate yearly progress (AYP). While states are given latitude with regards to what is meant by “adequate,” in general, this means the proportion of students achieving pre-determined proficiency levels on state assessments is expected to increase annually until all students in particular grades are deemed proficient or higher.

Value-added modeling is an alternative approach to test-based accountability systems interested in the proportions of students scoring at or above pre-determined proficiency levels. Value-added modeling techniques estimate the contribution of educational factors, such as teachers, to growth in student achievement, while allowing for the possibility to control for the effect of non-educational factors beyond a school system’s control, such as socioeconomic status. Value-added modeling methods provide opportunities to estimate the proportion of variability in achievement or student growth attributable to teachers, as well as estimate an individual teacher’s effect on student learning. When these methods identify large differences in teacher effectiveness, they also have the potential to help researchers identify what characteristics highly effective teachers possess and motivate informed improvements in education (McCaffrey, Lockwood, Koretz, & Hamilton, 2003).

Teacher effect estimates can be used for a variety of purposes, from informing educational systems how students are affected by current practices and conditions to making high-stakes decisions regarding teacher salary and/or employment. However, even though value-added modeling methods infer causal effects of teachers on student growth, the assessment data are not obtained from randomized, experimental studies.



Pages:   || 2 | 3 | 4 | 5 |   ...   | 14 |


Similar works:

«ONLINE HOMEWORK VERSUS PEN AND PENCIL HOMEWORK: DO THE BENEFITS OUTWEIGH THE COSTS? James Penner, PhD, CPA, Western Michigan University Elizabeth Kreuze, RN, PhD Candidate, Medical University of South Carolina Sheldon Langsam, PhD, CPA, Western Michigan University Jerry Kreuze, PhD, CPA, Western Michigan University This article discusses the advantages and disadvantages of online versus pen and pencil homework completions. While the use of online homework completion is rapidly growing, concerns...»

«„Wychowanie w Rodzinie” t. VII (1/2013) Petro SIREDZHUK Narodowy Przykarpacki Uniwersytet im. Wasyla Stefanyka, Ukraina Nationalistic Upbringing of a Ukrainian Child Based on “Woman’s Lot” Magazine Nacjonalistyczne wychowanie ukraińskiego dziecka na podstawie czasopisma „Kobiecy Los” Streszczenie Artykuł przedstawia najważniejsze składniki idei pedagogicznych nacjonalistycznego wychowania ukraińskich dzieci, wyznawanych przez redaktor czasopisma „Kobiecy Los” Olenę...»

«Ohm Shri Gurudevaaye namah Poedjaa Mantra’s Een geromaniseerde conversie Pravien Mahabier Met dank aan Shri Ashok Mahabier Inhoudsopgave INHOUDSOPGAVE 1.0 PAWIETRA KARAN (REINIGING VAN HET STOFFELIJKE EN GEESTELIJKE LICHAAM). 4 1.1 Zuiveren van de zitplaats (aasan pawitra karan) 1.2 Zuiveren van het lichaam (shaririek pawitra karan) 1.3 Aanbidden van moeder aarde (prithwie praarthanaa) 1.4 Reinigen van de handen 1.5 Zuiveren van de organen (aachman) 1.6 Reiniging van de hersenkanalen (shikha...»

«This is the author’s version of a work that was submitted/accepted for publication in the following source: Karuppiah, Nirmala & Berthelsen, Donna Catherine (2011) Multicultural education : the understandings of preschool teachers in Singapore. Australasian Journal of Early Childhood, 36(4), pp. 38-42. This file was downloaded from: http://eprints.qut.edu.au/45903/ c Copyright 2011 Early Childhood Australia Inc. Notice: Changes introduced as a result of publishing processes such as...»

«Available online at www.sciencedirect.com SYSTEM System 36 (2008) 214–226 www.elsevier.com/locate/system The classroom language use of a Korean elementary school EFL teacher: Another look at TETE Dae-Min Kang Center for Educational Research, Seoul National University, Sahn 56-1, Sillim 9-dong, Gwanak-gu, Seoul 151-748, Republic of Korea Received 9 March 2007; received in revised form 30 September 2007; accepted 17 October 2007 Abstract The continued interest in better EFL pedagogy for...»

«Building Motivation in the K-12 Art Classroom Page Andrews July 01, 2011 Abstract Student motivation is a universal challenge among teachers of every instructional level and content area. Motivation, particularly in the art classroom, is a challenge for many art educators. My study looks to unwind the intricate web of student motivation and to identify effective motivational strategies that art educators use on a daily basis within their classrooms. This qualitative study, conducted in a...»

«Robert Brooks, Ph.D. The following is a version of a chapter that appears in Understanding and Managing Children’s Classroom Behavior: Creating Sustainable, Resilient Schools (2007) by Sam Goldstein, Ph.D. and Robert Brooks, Ph.D. published by John Wiley & Sons. Developing the Mindset of Effective Teachers In Chapter Two we outlined the key characteristics of the mindset of effective educators. We noted that the differing mindsets or assumptions that educators possess about themselves and...»

«Michael Johnstone Material prepared by Karl Hickfang A Biographical Sketch of Michael Fuller Johnstone by Karl Hickfang It is my sincere hope that the reader will accept the use of first-person in writing of a man who influenced and inspired so many choral directors in the state of Texas. During a summer of 1949, I was finishing requirements for a bachelor of Music degree. I looked forward to my first teaching position as the band director at Bonham High School, my alma mater. Near the end of...»

«2012 HSC Notes from the Marking Centre – English Extension 1 Contents Introduction Module A: Genre Question 1 Questions 2, 3 and 4 Module B: Texts and Ways of Thinking Question 5 Questions 6, 7 and 8 Module C: Language and Values Question 9 Questions 10 and 11 Introduction This document has been produced for the teachers and candidates of the Stage 6 course in English Extension 1. It contains comments on candidate responses to the 2012 Higher School Certificate examination, indicating the...»

«THE AMERICAN BOMBING OF AN AFGHAN HOSPITAL – MISTAKE OR WAR CRIME? CLE Credit: 1.0 Wednesday, May 11, 2016 10:40 a.m. 11:40 a.m. Cascade Ballroom B Kentucky International Convention Center Louisville, Kentucky A NOTE CONCERNING THE PROGRAM MATERIALS The materials included in this Kentucky Bar Association Continuing Legal Education handbook are intended to provide current and accurate information about the subject matter covered. No representation or warranty is made concerning the application...»

«September 2004 | Volume 62 | Number 1 Teaching for Meaning Pages 42-45 Reading and Rewriting History Students learn to read critically as they plunge into primary and secondary sources looking for historical fact. Sam Wineburg and Daisy Martin Several years ago, we toured a swanky new middle school rising on the broken asphalt of an urban parking lot. This public school had a September 2004 mission statement that read like a recruiting poster for a high-tech start-up: Students would gain the...»

«Teaching Chemistry Through The Jigsaw Strategy Example 1 Topic Thalidomide: A Controversial Chiral Drug Subtopics 1. Why is drug chirality important?2. What caused the thalidomide tragedy?3. Why has thalidomide been approved for sale again? Level Secondary 6-7 Curriculum Links Stereoisomerism Enantiomerism Chiral carbon compounds Medium of instruction English Copyright © 2007 by Quality Education Fund, Hong Kong All rights reserved. Prepared by Professor Derek Cheung, The Department of...»





 
<<  HOME   |    CONTACTS
2017 www.sa.i-pdf.info - Abstracts, books, theses

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.