It started in 2009 and some surprises emerged, such as the gap of up to seven years between countries. Schleicher has argued that equity and performance are possible and that it is not a choice between high performance for the few or average performance for the many, only a matter of best practice. Countries such as Finland, for example, emerged as having equitable but outstanding results, although they seem to be waning. He also claims that funding is not the key factor for success. Many countries have consistently spent more and more on education but with marginal improvement. What is important is to look for the practices that give good value for money.
PISA tests
The Programme for International Student Assessment (PISA) takes place every three years and evaluates and tests the skills and knowledge of 15 year-old students worldwide. Over 70 countries (technically economies) have participated. To give some idea of the scale, in 2012, 510,000 learners were assessed in maths, reading and science. This is statistically representative of 28 million 15 year-olds in those economies. The schools are selected randomly and the test, which lasts two hours, is a mixture of open-ended and multiple-choice questions. The tests are not linked to a school curriculum but designed to assess the application of knowledge to real-life problems. Background information is collected through survey data and is then used to interpret the data. The aim is to allow countries to assess the data over time.
PISA and policy
Unfortunately, politicians and interest groups distort PISA to meet their own ends. They cherry pick results and recommendations, ignoring the detail. Finland is quoted as a high performing PISA country, yet it is a small, homogeneous country with no streaming, high levels of vocational education, no substantial class divisions and no private schools. Ministers of Education read them, quote them and use them as levers for change. An important factor, Schleicher thinks, is the way a country, its citizens and politicians, values its teachers. A culture of high expectation helps, as does autonomy for teachers. But politicians have tend to use PISA results to chase their own agendas – to attack state schooling, argue for more privatization, increase testing or increase funding.
Criticism
Like the real Leaning Tower of PISA, some argue that the PISA results are built on flimsy foundations and are seriously skewed. Nevertheless, they have become a major international attraction for educators, and regularly spark off annual educational ‘international arms’ races.
The problems in the data are extreme, as PISA compares apples and oranges. PISA is seriously flawed, some claim, because of the huge differences in demographics, socio-economic ranges and linguistic diversity within the tested nations. There are many skews in the data, including the selection of one flagship city (Shanghai) to compare against entire nations. External tuition is ignored. Immigration skews include numbers of immigrants as does the effect of selective immigration, migration towards English speaking nations and first-generation language issues. There is also the issue of taking longer to read irregular languages. Selectivity in the curriculum is also a problem as it is also possible to skew your curriculum towards PISA - dump sport, music, the arts and humanities and you can produce stellar results.
Sven de Kreiner Danish statistician says PISA is not reliable at all. In the reading tests 28 questions were supposed to be equally difficult in every country. PISA has failed here, he claims, as differential item functioning - items with different degrees of difficulty in different countries - are common. In fact he couldn't find any that worked without bias. Items are more difficult in some countries. He used his analysis to show that the UK can move up to 8th or down to 36th.
Interpretation
Once the test results have been published there is little recognition of the weaknesses. First, differences between countries’ performance are not that large, usually statistically insignificant. Second, whether or not a country has moved up or down the league tables is not that meaningful partly because the absolute differences in scores between countries are not that great. Third, the constituent group of comparators changes from study to study and from year to year.
Influence
PISA is a double-edged sword and Schleicher’s belief and optimism about its validity and its status as a force for good may be misplaced. League tables and results are misrepresented and an arms race has ensued where politicians apply partial policies that reflect their own political interests, rather than an objective reading of the results.
Bibliography
Schleicher, A. and Tamassia, C., 2000. Measuring Student Knowledge and Skills: The PISA 2000 Assessment of Reading, Mathematical and Scientific Literacy. Education and Skills.
Schleicher, A., 2006. Where immigrant students succeed: a comparative review of performance and engagement in PISA 2003: © OECD 2006. Intercultural Education, 17(5), pp.507-516.
Schleicher, A., 2009. Securing quality and equity in education: Lessons from PISA. Prospects, 39(3), pp.251-263.
Schleicher, A., 2007. Can competencies assessed by PISA be considered the fundamental school knowledge 15-year-olds should possess?. Journal of Educational Change, 8(4), pp.349-357.
Schleicher, A., 2017. Seeing education through the prism of PISA. European Journal of Education, 52(2), pp.124-130.
No comments:
Post a Comment