«Exploring teacher effects in outcomes of a growth mindset intervention in seventh grade science Shumow, L., & Schmidt, J. A. Abstract This ...»
Procedures Intervention. The mindset intervention consisted of the Brainology™ program – a sixweek, web-based tutorial that teaches students that the brain is like a muscle and can grow in intelligence. Students are provided with information about how the brain responds to learning and how they can improve their learning through effort, study strategies, and behavioral choices (sleep, diet, and stress management). The researchers met with student participants once per week and took primary responsibility for delivering the web content and for leading both anticipatory and follow-up activities to each lesson. Teachers were always present during these ‘Brainology days’. The program also included teacher education activities; teachers met on several occasions with the researchers for individualized sessions to discuss the mindset concept, how mindset develops, and practical details about implementation including discussion of extension activities to be done in class. The central role of the teacher in fostering mindset and in emphasizing the concepts during instruction and through statements made to students was emphasized. Teachers received a teacher’s manual containing numerous extension activities,
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESonline access to the Brainology program and to students’ work in the program, and two brief (3-5 page) supplementary readings on mindset.
Data collection. Students completed surveys that measured their beliefs about the malleability of intelligence, goal orientation, and interest in science before (pre), immediately after (post), and several months after (follow up) the intervention. Prior to the start of data collection in classrooms, participant teachers completed a survey in which they provided information about their demographic characteristics, professional training, and current teaching assignment. Also included in the survey was a series of questions used by Blackwell, Trzesniewski & Dweck (2007) to assess mindset. These items exactly mirrored those in the student survey.
Classrooms were observed on 11 different occasions before, during, and after the intervention which was significantly more than the sufficient number of observations suggested by some studies (e.g., Shih, 2013) therefore allowing us to effectively capture the qualities of these classrooms. On each of these 11 occasions, a team of two to three observers recorded instructional activities, and multiple dimensions of classroom context including event sampling of explicit and implied messages conveyed by teachers and students regarding goals, effort, evaluation, feedback, encouragement and study strategies. Reliability on the ratings among coders was very high, and notes from all coders present were used to compile a comprehensive set of field notes documenting mindset messages expressed by teachers and students in the classroom. These field notes were later coded (see description of coding below).
Teachers also participated in a formal semi-structured interview immediately following the intervention. Schools provided information from student records including demographic information and student grades.
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESStudent Measures Malleability of Intelligence. Four items were used to measure students’ beliefs about the malleability of intelligence or mindset. The items asked students to report on a six-point scale (disagree a lot = 1, agree a lot =6) whether they believed it was possible to change one’s intelligence in science (two items) or whether science intelligence is fixed (two items which were reverse scored to create this variable). A factor analysis provided evidence of the construct validity of this subscale. Cronbach’s alpha for these items was:.60 in the initial survey, and.74 in both the post intervention and follow-up surveys. Items were drawn from published studies (Aronson, Fried & Good, 2002; Blackwell et al., 2007), which reported test-retest reliabilities ranging from.77 to.82.
Mastery goal orientation. A mastery goals scale was created from four items on the student survey (I do science work to learn new things, I want to work on hard science work, hard assignments mean I’ll learn, and my goal in science is to learn as much as possible). Three of these items were measured on a six-point scale (1=disagree a lot, 6=agree a lot), and one item was measured on a five-point scale (1=strongly disagree, 5=strongly agree), therefore all scores were converted to z-scores to create a composite score. Cronbach’s alpha was.79,.79, and.82 on the pre, post, and follow-up surveys respectively.
Grades. School officials provided students quarterly grades in science from school records. First quarter grades served as the initial grade, second quarter grades served as the postintervention grade, and third quarter grades (which aligned with follow up surveys and observations) served as the follow-up.
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESTeacher and Classroom Measures.
Endorsement of motivational strategies. Teachers were asked to rate the efficacy of a number of different motivational strategies for males and for females on a scale of 1= not at all to 5 = very effective. Items on this rating list generally represented strategies that foster intrinsic or mastery goals on the one hand or curtail them in favor of extrinsic motivation on the other. For the purpose of this study, the teacher ratings of strategies that were effective for males and females were averaged.
Global ratings by class period. Immediately following the class period, observers rated three aspects of the overall classroom environment during the class period observed. Emotional climate of the class described overall interaction patterns between teachers and students in the class and was rated on a three-point scale as negative (indicating unpleasantness, anger, or hostility), neutral (generally flat, not emotionally charged), or positive (respectful, friendly, caring, helpful). Productivity/Organization indicated how well the class was organized and run in terms of routines, directions, and time management and was rated on a four point scale from 1= chaotic to 4 = highly efficient. Teacher enthusiasm described the interest and passion communicated by the teacher during the class period using a four point scale from 1 = projects boredom to 4 = passionate.
Classroom activities. The instructional method that the teacher was using and the type of work that students were doing were recorded by observers on an observational form. We adopted the criteria of Duke (2000) in classifying the instructional practices that the majority of students were doing in the classroom (p. 210). The time when the activity began and when the activity code changed was recorded. Activities were categorized and coded as: 1. teacher presentation,
2. individual seatwork, 3. group seatwork, 4. tests/quizzes, 5. whole-class discussion, 6. student
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESpresentations/demonstrations, 7. Video/movie, 8. lab work, 9. non instructional time, 10. off taskactivity, 11. activities related to study (completing brief surveys or writing tasks).
Global ratings by activity. For each instructional activity (excluding non-instructional and off task activities), observers made global ratings on four dimensions. On Task referred to the proportion of students who appear to be on task during the classroom activity (1= ¼ of students on task, 2 = ¼-1/2 of students on task, 3= more than ½ to ¾ of students on task, 4= More than ¾ of students on task). This global rating was dependent upon attention and participation.
Conceptual development indicated the degree to which teachers promoted higher order thinking, critical thinking, elaboration (why, how, compare), and problem solving, leading students to go beyond fact and recall to make inferences, hypothesize, analyze, interpret, reason on a four point scale from 1= almost none to 4 = extensive. Direct Instruction indicated the degree to which rote learning is emphasized and was rated on a scale from 1 = almost none to 4 = extensive.
Instructional Feedback described the extent to which teachers support and extend student learning through responses, scaffolding, promotion of student skills, and participation in activities on a scale from 1 = almost none to 4 = extensive.
Mindset messages. Observational event sampled field notes were coded for the purpose of coding teacher-provided messages related to mindset. For each teacher, we coded field notes from a total of 11 days per teacher: one day of regular instruction per week in each classroom for two weeks prior to the intervention, the six weeks in which the Brainology program was being implemented, and three weeks post intervention later in the school year. The day of the week we observed varied from week to week. Field notes were coded using the NVivo10 software program. Mindset messages were identified as any explicit statement or behavior that referred to Brainology program content, task difficulty/ease, effort, study strategies, ability, or performance
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMEScriteria, regardless of whether the reference explicitly mentioned mindset. Each mindset message was coded along multiple dimensions which recorded the nature of the messages as promoting or undermining a growth orientation. Messages that were coded as promoting a growth mindset specifically mentioned growth of intelligence, referenced Brainology content, emphasized effort, or suggested/modeled study strategies. Messages that were coded as undermining a growth mindset included those that clearly mentioned a fixed view of intelligence, valued low effort, and focused on task ease, difficulty, and ability without reference to effort. Once coding in NVivo10 was completed, data were analyzed using SPSS.
Outcomes by Teacher Malleability of intelligence. A mixed between-within subject analysis of variance was conducted to assess whether the Brainology program impacted students’ beliefs about the malleability of intelligence differently by teacher between the pre-intervention and follow-up period. There was a significant interaction between teacher and time, Wilks’ Lamda =.91, F (2, 134) = 6.87, p =.001, partial eta squared =.09, indicating a moderate effect size for the difference in increase in belief about the malleability of intelligence by teacher. Figure 1 displays the results of this analysis. Table 1 presents the pre-test, post-test and follow-up scores. As seen in the table, students in both classes developed stronger beliefs about the malleability of intelligence after participating in the intervention with Donna’s students growing more than Celia’s. During the follow-up period, Celia’s students regressed nearly to the point where they had been prior to the intervention whereas Donna’s fell slightly but nevertheless, maintained considerable gains.
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESMastery goal orientation. A mixed between-within subject analysis of variance was conducted to assess whether the Brainology program impacted students’ mastery goal orientation differently by teacher from pre-intervention to follow-up. There was a marginally significant interaction between teacher and time, Wilks’ Lambda =.96, F (2, 135) = 2.83, p =.06, partial eta squared =.04, indicating a small effect size for the difference in change in belief about mastery goals by teacher. As can be seen in Figure 2, students in Donna’s classroom increased in mastery goal orientation from pre to post-intervention and maintained those gains through follow-up, whereas, students in Celia’s classroom increased from pre to post-intervention, but decreased back to their pre-intervention levels at follow-up. Means and standard deviations can be seen in Table 1.
Grades. A mixed between-within subject analysis of variance was conducted to assess whether the grades of students in the mindset condition differed by teacher between the preintervention and follow-up period. There was a significant interaction between teacher and time Wilks’ Lambda =.79, F (2, 135) = 17.9, p =.000, partial eta squared =.21, indicating a large effect size for the change in students’ science grades by teacher from before intervention to follow-up. Figure 3 displays the results of this analysis. Table 1 presents the pre-intervention, post-intervention, and follow-up grades. Donna’s students’ grades improved during the intervention and were maintained across the course of the study whereas Celia’s did not.
Teacher Beliefs Celia expressed deep interest in the mindset intervention; it was new information for her and she saw it as exciting. Donna was also committed to the project because she was familiar with the importance of the content students would learn during the intervention. She was also
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESusing her participation in the study as part of the professional development plan that her district required of all post-tenured teachers.
Donna’s and Celia’s score on the mindset measure identified each of them as having a growth mindset in terms of science intelligence. As can be seen in Table 2, both teachers moderately endorsed practices associated with mastery goals (e.g. “teaching them strategies for learning”). In contrast to Donna’s weak endorsements of such strategies, Celia endorsed using strategies often associated with fostering a performance approach goal orientation (Patrick, Mantzicopoulos, & Sears, 2012). For example, she moderately endorsed comparing students to one another and strongly endorsed telling students that they were one of the best in the class as a means of motivating them. Neither teacher recommended strategies such as embarrassing students for poor performance as being motivational. Celia was more affirming of the motivational value of using consequences like rewards and contacting parents than Donna was.
Both teachers endorsed the motivational value of praise.
Teacher Instructional Practices.
General classroom climate. Observers rated Donna as more enthusiastic than Celia.
Donna and Celia’s classrooms did not differ in terms of either the emotional or organizational climate of the classroom (see Table 3). The emotional climate in both teachers’ classes fell between neutral and positive with both leaning toward the positive pole. Both also were rated more often as being well organized as opposed to inefficient in managing their classrooms.
Instructional activities. The teachers both covered the state and district curriculum for seventh grade science and the percent of time each teacher’s classes spent in various instructional activities was very similar. Although both spent similar amounts of time making teacher presentations, during Donna’s presentations, she more often questioned the students and
TEACHER EFFECTS IN MINDSET INTERVENTION OUTCOMESconducted demonstrations of phenomena they were studying. Ratings given by observers during the activities indicated that Donna was more likely than Celia to use direct instruction and to facilitate students’ conceptual development (see Table 3). These tendencies might explain why Donnas’ students were more likely to be on task than Celia’s.
Individual seatwork was assigned a similar amount of time by Celia and Donna.
Although Celia was more frequently seen circulating among the students looking at their work than Donna, there were no differences in ratings for instructional feedback provided to the students. Neither teacher spent much time doing labs. Our observations further indicated that the labs were not inquiry based.