This study investigated the relationship between readability of 73 TIMSS text-only multiple-choice questions from Trends in International Mathematics and Science Study (TIMSS) 2003 and performance of two groups of South African learners: those with limited English-language proficiency (learners attending African schools) and those with better English-language proficiency (learners attending non-African schools). The investigation hoped to identify how readability of questions affected science achievement and explain some of the difficulties experienced by South African second language learners. The investigation is complex, because both groups were exposed to the same intended curriculum, performance is affected by the level of knowledge and skills that learners possess, as well as language factors. Other factors that influence performance is the non-availability of resources, the level of functionality of the schools and teachers who lack the knowledge and confidence to teach science. This study investigates research questions relating to readability of TIMSS items and South African learners' performance in the 2003 study.