Education technologies are not a cure-all for improving reading outcomes
By Dr. Benjamin Piper, Senior Director, Education Research.
As the price of laptops, tablets, and smartphones has drastically decreased, international education practitioners and policymakers have increasingly turned to information and communications technology (ICT) interventions in low-resource classrooms in Sub-Saharan Africa to improve the quality of early grade reading outcomes. It is clear that the use of ICT can have many potential benefits in the classroom, including providing more opportunities for individualized learning and increased exposure to reading materials.[i]
However, many questions remain as to how and whether these benefits in learning experience and environment can translate into a substantial improvement in reading outcomes. Further, with little research on which to draw regarding the cost-effectiveness of different ICT interventions, education policymakers and governments are making multi-million dollar decisions to invest in ICT programs while lacking the complete picture on cost-effectiveness.
So can the use of ICT in low-resource classrooms lead to substantial shifts in reading ability, and is it cost-effective in comparison to literacy programs that don’t incorporate ICT techniques?
These are the complex questions that are explored in a recent paper that I co-authored, “Does Technology Improve Reading Outcomes: Comparing the Effectiveness and Cost-effectiveness of ICT Interventions for Early Grade Reading in Kenya.”
In this paper, my co-authors and I compare the effectiveness and cost of three different ICT interventions: a base literacy program (the Kenya Primary Math and Reading (PRIMR) Initiative) that provided instructional tutors with tablets, as well as two more intensive ICT interventions, one which provided teachers with tablets and the other of which incorporated e-readers for students. The conclusion we reached is multi-faceted, but nonetheless important for practitioners and policymakers to recognize:
When compared to traditional literacy programs, the more intensive ICT interventions did not produce large enough gains in learning outcomes to justify the cost.
This is not to say that each of the ICT interventions did not produce improvements in students’ reading ability. On the contrary, all three showed improvements in reading outcomes when compared to a control group of students not participating in an intensive literacy program. The tutor tablet group emerged as the most effective ICT intervention, producing an improvement in English oral reading fluency of 9.9 correct words per minute (cwpm) over the control group. The teachers with tablets came in second, with 8.5 cwpm, and students with e-readers in last, with only 6.1 cwpm. However, when compared to the non-ICT versions of the PRIMR literacy program, none of the ICT interventions showed significant advantages in improved reading fluency and comprehension over non-ICT-based literacy interventions.
This finding underscores the importance of evaluating the effectiveness of ICT interventions as compared to less expensive non-ICT literacy programs, rather than in isolation.[ii] Had we only evaluated each of these interventions against one another, the results would have indicated an impressive effect on learning outcomes that might have made them eligible for increased funding.
In addition, when deciding how and whether to use ICT in low-income classrooms, it is essential to compare the actual costs of various technologies to get a full picture of a program’s value-added. An examination of costs revealed that the e-readers were the least cost-effective of the ICT interventions, with the lowest improvements in English reading outcomes at 6.1 cwpm, and an added cost of a whopping forty dollars per student, even when a purchase of the cheapest e-reader on the market was analyzed. This is an added cost that is often unsustainable for schools in developing countries where resources are hard to come by. On the other hand, the instructional tutors with tablets group was not only the most effective in producing positive outcomes, but it was also the cheapest ICT option, costing only an additional ten cents per student. However, the cost-effectiveness of all of these programs might still be significantly lower than a clear investment in high quality literacy programs.
To be sure, there are many difficulties in designing an effective ICT intervention that could account for the lack of added value that we discovered in our particular study. In order for ICT to be effective in low-income environments, practitioners must be cognizant of context and end users, and have a specific structure and purpose.[iii] The minimal value added for e-readers could have been due to a need for more structured reading activities, a re-adjustment for age-appropriateness.
Further, in additional to monetary cost, an opportunity cost existed for all three interventions. Many of the teachers, tutors, and students lacked exposure to technology and the time and energy spent on learning how to use the technology reduced the amount of time for instructional improvement activities. Upon inspection, we also discovered that the tutors and teachers were infrequently using the more advanced features of the tablets that may have yielded further improvements in results.
While it seems there might be minimal value added for ICT interventions against their additional cost, there may still be promise for technology that is targeted at instructional improvement with the goal of easing the load of instructional supervisors. The positive results of the tutor tablet intervention group, which was built on the PRIMR literacy program, suggest that ICT has the most potential to be effective when aligned with a well-designed literacy program. These should be important considerations when designing an ICT intervention. However, when costs are considered, there are non-ICT interventions that could have larger impacts on learning outcomes with reduced costs; one such option could include assigning the best teachers to the first grade when children are learning how to read, rather than to the end of primary school as many schools do.
This highlights why it is important for policymakers to consider not only whether ICT programs are producing positive outcomes, but whether they are doing so in the most cost-effective manner.
We should resist leaning on ICT as a cure-all for poor literacy, but instead incorporate it into existing programs in ways that are actually beneficial to teachers and students.
In the context of Kenya, this is especially important considering the Kenyan government’s recent decision to provide tablets enough for each child in Grade 1, costing at least $200 per student. Beyond Kenya, the findings of this study should serve as important considerations in the policy and funding environments of other sub-Saharan countries.
By focusing on the nuances of ICT, namely what works best, what will be most cost-effective, and what the effect will be on end-users, we can harness the potential of ICT to make a valuable difference in classrooms in low-income countries.
For more studies like this from RTI’s international education team, please visit shared.rti.org.
If you like what you just read, please hit the green “Recommend” button below so that others might stumble upon this story. For more content like this, scroll down and follow RTI Intl Dev on Medium & Twitter. ____________________________________________________
[i] Grace, J., Kenny, C., 2003. A short review of information and communication technologies and basic education in LDCs-what is useful, what is sustainable? Int. J. Educ. Dev. 23 (6), 627–636. http://dx.doi.org/10.1016/s0738-0593(03)00062-2
[ii] See related study: http://shared.rti.org/content/mobiliteracy-uganda-mlit-uganda-results-controlled-trial-sms-based-literacy-support-program
[iii] Wagner, D., 2014. Mobiles for Reading: A Landscape Research Review. USAID, Washington, DC. http://pdf.usaid.gov/pdf_docs/pa00jtgt.pdf.
