2009年-世界发展银行全球_Measuring_Beginner_Reading_Skills___An_Empirical_Evaluation_of_Alternative_Instruments_and_their_Potential_Use_for_Policymaking_and_Accountability_in_Peru_91页_788kb

Summary of "Measuring Beginner Reading Skills: An Empirical Evaluation of Alternative Instruments and their Potential Use for Policymaking and Accountability in Peru"

Core Content

This policy research working paper evaluates alternative methods of measuring beginner reading skills in Peru, focusing on the use of oral fluency tests and written comprehension tests, and proposes a framework for their combined application in improving education quality and accountability.

Main Points

• Reading Skills Importance: Reading is the foundation of all school learning. Students who cannot read fluently are at a disadvantage in accessing knowledge and becoming autonomous learners.
• Early Reading Skills: The five main early reading skills are phonemic awareness, phonics, fluency, vocabulary, and text comprehension.
• Reading Fluency and Comprehension Relationship: There is a positive and significant correlation between reading fluency and comprehension. Fluent readers are more likely to understand what they read.
• Nature of the Relationship: The correlation is non-linear and discontinuous, with the strength of the relationship depending on the level of fluency, difficulty of the questions, and social characteristics such as the child’s first language and access to preschool education.
• Testing Approaches: The paper compares individual oral fluency tests and group-administered written comprehension tests (ECE). It concludes that using both tests together provides more comprehensive and reliable data than using either alone.
• Psychometric Analysis: Both tests adequately measure reading ability, though improvements are suggested. For the ECE test, adding more items with increasing difficulty is recommended. For oral tests, longer passages and varied question difficulty are suggested.
• Accountability and Policymaking: The study supports the use of reading assessments to establish baseline performance, set attainable goals, and track progress at different levels (individual, class, school, etc.). It proposes a model for reporting results that emphasizes reading standards, mobilizes the education community, and identifies students in need of support.
• Social Inequality: There is a significant performance gap between Ashaninka indigenous students and other groups, even when controlling for social factors like gender, location, and school type.

Key Recommendations

• Use improved versions of both oral and written reading tests to evaluate early grade reading skills.
• Present results in a way that supports accountability and facilitates action at all levels.
• Develop reporting models that allow for comparisons across schools and tracking of student progress over time.
• Ensure that fluency and comprehension are measured together, as they are interconnected and complementary.
• Consider social characteristics in interpreting test results, especially for disadvantaged groups.
• Avoid over-reliance on fluency tests alone for coaching, as they may not directly improve comprehension and could create pressure on students.

Key Instruments and Methods

• Oral Fluency Test: Administered individually, designed to measure reading speed and accuracy.
• Written Comprehension Test (ECE): Group-administered, standardized test designed by the Ministry of Education (MINEDU) for the 2006 census evaluation.
• Psychometric Analysis: Used to evaluate the discriminatory power of the tests, normality of scores, and item functioning.
• Rasch Analysis: Applied to assess the difficulty of items and the effectiveness of fluency as a measure.
• Differential Item Functioning (DIF): Analyzed to determine if items behave differently across social groups.

Context and Purpose

• The study is part of the Peru Recurso 3 Programmatic AAA initiative, which aims to strengthen accountability through results-based standards in education.
• It was conducted in the Junin region using a sample of 475 third-graders.
• The paper provides a basis for dialogue between the World Bank and the Ministry of Education (MINEDU) on improving reading assessments and standards.

Conclusion

The paper concludes that using improved versions of both oral and written reading tests is not only possible but also desirable for monitoring and improving early reading skills. It emphasizes the importance of combining these assessments to provide a more accurate and useful picture of student performance, which can inform policies and accountability mechanisms in education.

Key Figures and Tables

• Table 1: NAEP Oral Reading Fluency Scale for Grade 4 (2002)
• Table 2: Summary of best use for oral and standardized pencil-and-paper tests
• Table 3: Match between ECE test items and curricular standards
• Table 4: Distribution of third-grade students by school type
• Table 5: Descriptive statistics of fluency and comprehension variables
• Table 6: Normality of standardized scores (Shapiro-Wilk test)
• Table 7: Mean comparison between lowest and highest performers
• Table 8: Difference in relative position between oral and written test performances
• Table 9: Score distribution differences between Ashaninka and non-Ashaninka students
• Table 10: Correlation between fluency and comprehension (Spearman Rho)
• Table 11: Correlation between fluency and comprehension for Ashaninka and non-Ashaninka students
• Table 12: Regression models (adjusted)
• Table 13: Mean comparison using Rasch scores (U-Mann Whitney test)
• Table 15: Hypothetical comparative report card
• Table 16–20: Distribution of students and schools by UGEL, location, gender, and access to preschool education

Figures

• Figure 1: Map of item difficulty using Rasch scores
• Figure 2: Histograms of Z scores for Ashaninka students
• Figure 3: Relationship between oral fluency and written comprehension
• Figure 4: Estimated relationship between fluency and comprehension for Ashaninka and non-Ashaninka students
• Figure 5: Improvement in ECE scores with fluency
• Figure 6–22: Box plots of Z scores by fluency, language, location, school type, and gender

Conclusion

The paper underscores the importance of using both oral and written reading assessments to improve the accuracy and utility of reading skill measurement. It highlights the interconnectedness of fluency and comprehension and recommends policy changes that support accountability and improve learning outcomes in Peru.