The goal of the five-year effort was to address key challenges that teachers and students confront with college and career readiness standards, which require students to show advanced literacy proficiency not only in English classes, but also in other core subjects.
Collaboration and transformation pave the way to higher college and career/workplace readiness
A dominant feature of Reading Apprenticeship is its emphasis on creating a collaborative culture. Over five years, English, biology, and U.S. history teachers supported each other through professional development and in applying Reading Apprenticeship in their classrooms. Similarly, students learned how to work collaboratively in groups and to rely on their classmates for meaningful discussion and learning.
Collaboration brought transformation. Students attending more than one class using the Reading Apprenticeship framework were able to carry and apply skills from one class to another and more efficiently learn the material.
Praise from RAISE participants
High School Principal
Dramatic increase in student reading passage rates: from 29% to 54% from September to January, then to 61% several months later.
District Leader
District administrators see teachers make big impact through “true professional learning communities.”
Please note that the RAISE project has concluded, but educators looking for a professional learning experience with Reading Apprenticeship can find information about current opportunities on our services page.
Overview
RAISE was awarded a top rating and five years’ funding in the U.S. Department of Education i3 competition, based in part on findings from three previous randomized controlled studies of Reading Apprenticeship. In those studies, high school students who experienced Reading Apprenticeship in a single course — biology, U.S. history, or grade 9 academic literacy — made significant literacy and content knowledge gains, as well as gains related to motivation and academic identity.
RAISE was both a larger and more intensive version of the Reading Apprenticeship interventions already studied. It was designed to amplify a student’s experience with Reading Apprenticeship through a school-based model. At each site, teachers in multiple disciplines integrated Reading Apprenticeship practices into their teaching. Students, therefore, experienced Reading Apprenticeship in up to three successive years: English language arts in grade 9, biology in grade 10, and U.S. history in grade 11.
In a subset of the 300+ participating schools, a randomized controlled study assessed the impact of this “increased dosage” on students’ engagement and academic achievement. Research on implementation was included as part of this study.
A second focus of the research was a “scale-up” study, designed to illuminate issues of fidelity and capacity building when a complex professional development-based intervention is implemented on a large scale. For this study, evaluators will gather data from the large group of teachers in the 260+ schools that were not part of the study. In addition, evaluators conducted case studies in a subset of these schools.
The scale-up effort was a partnership between Reading Apprenticeship at WestEd and four local education agencies — one each in the states of Indiana, Michigan, Pennsylvania, and Utah plus several school districts in California — that collaborated with SLI to support, sustain, and further disseminate Reading Apprenticeship during the grant period.
Questions
Researchers from Empirical Education and IMPAQ set out to address the following research questions in the randomized controlled study:
- What is the impact of Reading Apprenticeship on teachers’ ability to integrate disciplinary literacy practices and explicit literacy instruction into high school biology, U.S. history, and English language arts classes?
- What is the impact of Reading Apprenticeship on students’ reading behaviors, attitudes, and strategies, including reading persistence and the ability to implement problem-solving and comprehension strategies?
- What is the impact of Reading Apprenticeship on students’ academic achievement in English language arts, biology, and U.S. history?
- What is the impact of Reading Apprenticeship on students’ academic attainment, course performance, and retention in high school?
Methodology
Intervention Plan
The intervention for the randomized controlled study and the scale-up study is the same: 10 days of discipline-specific Reading Apprenticeship teacher professional development (grade 9 English Language Arts, high school biology, and high school U.S. history), monthly on-site meetings, and online resources. Reading Apprenticeship-trained facilitators in collaboration with local education agencies will deliver the professional development. Leadership development for supporting the on-site monthly teacher meetings is explicitly designed into the model.
Design and Sample
The study will use a group randomized experimental design in 40-44 schools that serve a high proportion of students eligible for free and reduced-price lunch, African American and Latina/o students, and English Language Learners. Half the schools will be randomly assigned to the experimental group and half to a wait-listed control group. In each school six to nine teachers, two to three teachers of each target course, will be followed, along with their students.
Measures of school participation and implementation of Reading Apprenticeship in the professional development and classroom teaching are being developed, as are measures of the impact of this implementation on student learning experiences, engagement, and achievement. In addition, evaluators will document the scale-up of the intervention in five states to identify important features of the school and district contexts that may influence the success of the intervention.
Summary of impact and scale-up evaluation reports
Over five years, RAISE reached more than 1,700 high school teachers in 274 schools and nearly 700,000 students. Through the RAISE project Reading Apprenticeship helped these students make progress on one of the country’s most crucial challenges: improving academic literacy. Research shows that the RAISE project improved teachers’ ability to strengthen students’ reading of complex texts in science, history, and English classes — a key solution to removing barriers to college and career readiness. Two independent evaluations demonstrate Reading Apprenticeship’s effectiveness in improving secondary literacy instruction and achievement at scale.
One evaluation — a comprehensive impact study conducted by Impaq International — used a random assignment design, with 22 intervention schools and 20 control schools in Pennsylvania and California, to assess the impact of Reading Apprenticeship on teachers’ instructional practices, students’ classroom experiences, and students’ literacy skills. Evaluators used multiple data sources, including teacher and student surveys and an assessment that simulates subject area reading and learning tasks, developed by ETS.
A second study — conducted by Empirical Education — used surveys, focus groups, and case studies to identify lessons from managing the scale-up across four of the five states.
Key findings regarding impact
“A student who would walk into one classroom and not engage at all walked into this [RAISE] classroom and it was amazing to see the engagement. The level of conversation and critical thinking the students were doing was impressive.”
– Eric Turman, Principal, Reading High School, PA
Teaching Practice
- RAISE had statistically significant impacts on teachers’ use of core Reading Apprenticeship practices and on their confidence in delivering literacy instruction.
- Intervention teachers fostered greater student independence, engaged students more often in metacognitive problem-solving processes, and provided more opportunities for students to practice comprehension strategies and collaborate to comprehend subject-area texts. Intervention teachers also reported greater confidence in their literacy instruction.
“We noticed that students were engaging with the tests a lot more, writing on passages, circling key words in science, and ‘talking to the text.’ We also noticed they were using the skills we taught in English class in science class.”
– Heather Arena, Teacher Leader, Exeter High School, PA
Student Literacy Learning and Achievement
- Students in Reading Apprenticeship classrooms reported significantly greater opportunities to share reading processes and problem solving, and indicated that reading instruction was more integrated into their content-area learning. These outcomes parallel impact findings on teacher practice.
- Consistent with prior studies indicating that Reading Apprenticeship can improve students’ ability to comprehend and reason from subject-area texts, RAISE had a positive and statistically significant impact on student literacy in science classes, as measured by the ETS assessments. Although results for the other two subjects were not statistically significant, there was a meaningful, positive effect for ELA classrooms.
- For all students (across subject areas and grade levels) and for key subgroups (English Language Learners, low-income students, and BIPOC students), RAISE had positive but not statistically significant impacts on student literacy skills, as measured by the ETS assessment.
The results of this impact evaluation suggest the RAISE intervention had a significant impact on instructional practices that are the hallmark of Reading Apprenticeship — practices that support students in gaining experience and greater independence in doing the work required of them by today’s higher standards.
The strong results for science suggest that science teaching can be an especially productive area for improving student comprehension and reasoning from text, aligned with the higher standards for literacy and college and career readiness being adopted by many states. The RAISE project had greater impact with science students than with ELA or history students, and had greater impact in Pennsylvania schools than in California schools. These differential impacts provide important opportunities to improve future efforts to scale up Reading Apprenticeship — opportunities, for example, to examine the circumstances and supports that best enable the Reading Apprenticeship approach to be effective.
Key findings regarding scale-up
“The support and trust from our administrators was great. If we asked to try a new strategy, they would sit in to learn about it and then support our doing it.”
– Nick Burch, Teacher Leader, Quakertown Community High School, PA
Formative Feedback
- Teachers rated Reading Apprenticeship professional development highly, and other data indicated high levels of initial implementation and the promise of sustainability, including teachers and principals planning for implementation in the following year.
- Teachers and administrators demonstrated high levels of buy-in and commitment, reporting that Reading Apprenticeship was an appropriate literacy approach for their classrooms and a means of improving student achievement in their schools.
- Teachers and administrators identified competing priorities as the biggest challenge to implementation of Reading Apprenticeship.
“What I’ve learned from other teachers throughout the country has been incredible in terms of what it’s done for my classroom. Listening to others troubleshoot problems and come up with bright ideas to deepen student learning has been amazing.”
– Rob Cushman, Biology Teacher and Teacher Leader, Wyomissing Senior High School, PA
Scale-Up Processes
- Early establishment of team meetings proved to be an important predictor of which schools would grow in their implementation of Reading Apprenticeship and which would atrophy.
- Teachers identified collaboration with colleagues as the most effective activity in building their capacity to implement Reading Apprenticeship.
- Over the years studied, schools fell into differential patterns of scale, with some adding additional teachers over time, others “staying the course,” and still others losing teachers involved in the initiative.
The evaluation demonstrated that the Reading Apprenticeship approach can successfully reach and positively impact large numbers of teachers and students. The RAISE project’s investment in site-based team meetings and teacher leader development proved key to the successful implementation of Reading Apprenticeship at scale.
Download the full impact and scale-up reports.