"Learning Outcomes in Developing Countries: Four Hard Lessons from PISA-D"
Pritchett, L. and Viarengo, M. 2021. Learning Outcomes in Developing Countries: Four Hard Lessons from PISA-D. RISE Working Paper Series. 21/069. https://doi.org/10.35489/BSG-RISE-WP_2021/069
Insights about education systems from this study
The learning crisis reveals a massive gap between developing countries learning outcomes and other higher performing countries. Closing the gap between boys and girls, rich and poor, or advantaged and disadvantaged in developing countries will not raise learning outcomes nearly enough — because even the advantaged students are not gaining mastery of foundational skills. We must address global equity by "raising the floor" and targeting low learning at a systemic level.
For a summary list of education system insights from other studies, see the Policy brief builder.
Context and methods
This study explores the PISA-D results from seven countries and examines inequality in learning outcomes based on five dimensions: sex, rurality, home language, immigrant status, and socio-economic status. The seven countries studied are Cambodia, Ecuador, Guatemala, Honduras, Paraguay, Senegal, and Zambia.
The learning crisis is increasingly acknowledged in the global community: kids are in school but they are not learning. The UN's Sustainable Development Goals now include learning targets to address this problem.
The SDG learning goals raise an important question: how much of the gap to universal literacy and numeracy is due to a country's education system not being "inclusive and equitable" and how much is due to a country's education system not providing a "quality education"?
The Programme for International Student Assessment, also called PISA, is a global study by the OECD that is conducted every three years and measures 15-year-olds academic performance in math, science, and reading. Because the OECD consists of majority developed and democratic countries, the triannual PISA studies do not measure learning in developing nations around the globe.
Thus, the OECD has encouraged the participation of low- and middle-income countries in their new PISA for Development testing, also called PISA-D. Seven low- and middle-income countries participated in PISA-D and the results were explored in this study: Cambodia, Ecuador, Guatemala, Honduras, Paraguay, Senegal, and Zambia. The study further revealed the severity of the learning crisis.
PISA-D results from the seven countries were used to examine learning outcomes overall and based on five different dimensions of inequality. PISA-D tests the math, reading, and science skills of in-school 15-year-olds. The scores are normed so that the typical student in the OECD scores 500, or a Level 3. Level 2 includes basic skills such as making inferences, extracting relevant information, and making literal interpretations. SD4 4 measures Level 2 and above as minimum proficiency for learning goals. PISA Level 1c reflects a roughly rote level of understanding in which students can “only perform to the simplest of questions presented in a familiar format.”
The PISA-D assessment included 37,000 students from the seven countries.
To measure socioeconomic status, the authors utilized the PISA ESCS (Economic, Social, and Cultural Status) scale, which provides information on parent's highest level of education, occupation status, home possessions, and more. The other dimensions; sex, rurality, home language, and immigration status; were also measured by the PISA assessment.
Pritchett and Viarengo compiled the PISA-D results from the seven developing countries and then analyzed that data by estimating regressions using three different learning outcomes as the dependent variable and incorporating the five identified dimensions of inequality. Those outcomes included the estimated score of a given student, an indicator that the student is at or above PISA Level 2 proficiency, and an indicator that the student is at or above PISA Level 1c.
For more detailed information about the regression equation and methods, see Pritchett & Viarengo 2021.
Some general results found that:
- Only 12 percent of children who were tested met minimum proficiency levels for math, and just 23 percent for reading.
- In an OECD country like Denmark, only 13.6% of students are at Level 2 or below in math (the majority falling at Level 3 or higher).
- However, in the seven countries studied, percentages of students at Level 2 or below in math ranged from 70.9% in Ecuador to 97.7% in Zambia.
The authors identified four hard lessons from the results of the PISA-D analysis.
- “The Advantaged, SES Elite of PISA-D Countries Perform Badly by Global Standards”
- Across these countries, even the elite (male, urban, natives, and home language) are receiving an inadequate education when compared to the global average: Less than a quarter of advantaged students in Senegal, Cambodia, Zambia, Guatemala, and Paraguay reach PISA-D Level 2, or minimum proficiency in reading and math as indicated by SDG4.
- Less than a half of advantaged students in Ecuador reach the same level.
- With the exception of Ecuador, less than a third of the advantaged (male, urban, native, home speakers of the language of instruction, upper SES) children enrolled in public schools in the seven countries reached the minimum SDG target for PISA level 2 or higher in math, with similar low levels for science and reading.
- Even the high performing advantaged, ESCS elite child in Zambia still performs below the minimal SDG threshold for mathematics.
- “Even a very poor child in Vietnam (2 standard deviations below the Vietnamese mean) is, on average, predicted to perform better than the high performing, advantaged, ESCS elite, child in Zambia.”
- “Inequality by ESCS of those assessed is not higher in the PISA-D countries.”
- The inequality of learning between low SES and high SES students in these developing countries was generally smaller than in OECD countries and may not be as much of a determining factor on children’s learning outcomes within the current system.
- “If the PISA-D countries had the same relationship of learning to ESCS as Denmark (as an example of a typical OECD country) or Vietnam (a high-performing developing country) their enrolled ESCS disadvantaged children would do worse, not better, than they actually do.”
- “Social inequalities are large and important to eliminate,” but the magnitude of global inequality is larger.
- The disadvantages in learning outcomes along sex, rurality, home language, and immigration are "absolutely large, but still small compared to the enormous gap between the advantaged, SES students and the SDG minimums."
- In comparison to the massive global inequalities between countries, "remediating within-country inequalities in learning, while undoubtedly important for equity and justice, leads to only modest gains towards the SDG targets."
- “Too few students are doing really well.”
- There are few children in these countries achieving a high level of performance. In OECD countries, about 30% of children reach PISA level 4 or higher. In these PISA-D countries, less than a few thousand students, and sometimes only a few hundred, reach these high performance levels.
Download open-access slides
These open-access slides can be freely adapted for presentations or other forms of communication, as long as they are cited appropriately: File:Pritchett & Viarengo, 2021.pptx
For open-access slides from other studies, see the Policy brief builder.
- Crouch, Luis and Caine Rolleston. 2017. "Raising the Floor on Learning Levels: Equitable Improvement Starts with the Tail." RISE Programme Insight Note.
- Pritchett, L. and Viarengo, M. 2021. Learning Outcomes in Developing Countries: Four Hard Lessons from PISA-D. RISE Working Paper Series. 21/069. https://doi.org/10.35489/BSG-RISE-WP_2021/069