User:Vikipedia02/sandbox

{{Student editor|course=[ https://outreachdashboard.wmflabs.org/courses/University_of_Sussex/Education_for_Development Education for Development]}}

The Southern and Eastern Africa Consortium for Monitoring Educational Quality

Political map of Africa

Overview

The Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) is a non-governmental organisation, established in 1995 , which carries out large-scale cross-national research studies in the region of Southern and Eastern Africa . These studies aim to monitor and evaluate the quality of education of the participating countries , including the conditions of schooling and the performance levels of pupils and teachers . This information is generated from surveys which test participants on mathematics, reading and health knowledge . Pupils involved in the survey also provide information on their home environment . SACMEQ also aims to provide training to educational officials and researchers to improve their capacity to monitor, evaluate and compare schooling conditions and basic educational quality . The information generated by SACMEQ projects is used to review how educational policies are working and to inform the creation of new ones .

Origins

SACMEQ was established in 1995 following a research and training program, ran by the UNESCO International Institute for Educational Planning (IIEP), which took place in 1990 in the Ministry of Education in Zimbabwe . This program involved follow up workshops in which education planners and researchers from the Department of Education in Zimbabwe and surrounding countries expressed to the IIEP the need for better monitoring and assessment of their educational systems . This resulted in the official launching of SACMEQ in 1995 by 7 Ministries of Education (Kenya, Malawi, Mauritius, Namibia, Tanzania, Zambia, and Zimbabwe) , facilitated by the IIEP , with long-term support provided by the Government of the Netherlands . SACMEQ became officially registered as an international non-governmental organisation (NGO) by the government of Zimbabwe in 1997 . At this point SACMEQ had a membership of 15 Ministries of Education (Botswana, Kenya, Lesotho, Malawi, Mauritius, Mozambique, Namibia, Seychelles, South Africa, Eswatini, Tanzania (Mainland), Tanzania (Zanzibar), Uganda, Zambia, and Zimbabwe) .

UNESCO logo

The first two SACMEQ projects gathered overlapping data at two different points in time, and therefore the results of SACMEQ I provided a baseline for SACMEQ II . Between the first SACMEQ project (1995-1999) and the second (2000-2004) participation in the research increased from seven countries to fourteen . SACMEQ I involved around 20,000 pupils from 1,000 primary schools whilst SACMEQ II involved around 40,000, 5,300 teachers and 2,000 heads from 2000 primary schools . SACMEQ I collected data on educational inputs, general conditions of schooling, and literacy levels of students in Grade 6 . SACMEQ II focused on mathematics as well as reading/literacy .

In 2004, the IIEP published the findings of the research from SACMEQ I and II in a report which suggested policies for addressing issues relating to educational inputs, the general conditions of schooling, equity assessments for resource allocation and pupil literacy levels .

The third SACMEQ project (2005-2010) was participated in by all 15 SACMEQ Ministries of Education and included the assessment of pupils and teachers' knowledge on the health matters of HIV and AIDS . Data collection for this project took place in 2007 and included 61,396 pupils, 8,026 teachers, and 2,779 schools .

The fourth SACMEQ project focused on the conditions of schooling and educational quality of 15 school systems (Botswana, Kenya, Lesotho, Malawi, Mauritius, Mozambique, Namibia, Seychelles, South Africa, Eswatini, Tanzania (Mainland), Tanzania (Zanzibar), Uganda, Zambia, and Zimbabwe) .

Aims

The main aim of SACMEQ is to monitor and evaluate the quality of education across countries of Southern and Eastern Africa through the assessment of students and teachers' performance levels in the areas of reading, mathematics and health knowledge .

Another main aim of SACMEQ is to expand opportunities for educational planners and researchers in Ministries of Education. This is done through focusing on capacity building, involving training workshops in the areas of project design, sampling, data collection, instrument construction, data entry and data cleaning.

SACMEQ also aims to use the information gained from carrying out its research studies in the formulation of plans to improve educational quality, using data as the foundation of policy making and practice .

Framework

SACMEQ is a large-scale organisation and has many different departments and teams. The Assembly of Ministers, made up of the 16 Ministers of Education, meet biannually and are responsible for setting the policies and programs of SACMEQ .

The daily operating of SACMEQ is managed by the SACMEQ Coordinating Centre (SCC) . This includes the overseeing of operational activities to implement the policies and programs set by the Assembly of Ministers . The SCC is presided over by the SACMEQ Director, currently holding this position is Mrs Toziba Masalila . The SCC was transferred from the IIEP in Paris to the University of Gaborone in Botswana in 2014 to be closer to the National Research Coordinators (NRCs) .

An NRC is appointed for each country by their respective Ministers of Education, their role is to oversee the implementation of SACMEQ projects within their respective countries . SACMEQ encourages the collaboration of NRCs to “share and exchange skills and successful experiences” .

SACMEQ is also made up of different committees. The Managing Committee, elected by the Assembly of Ministers on a rotational basis every two years , oversees SACMEQ’s administration and finance, and the Scientific Committee provides training, technical support and advice to the SCC .

SACMEQ has a number of partner organisations, including the Ministry of Foreign Affairs of the Netherlands and the UNESCO International Institute for Educational Planning .

The Government of the Netherlands has been providing financial support to the consortium since its formation . Many have argued that the role of international organisations (IO) in the founding of international large-scale assessments (ILSA) has effects on the focuses of the studies. Cardoso demonstrates that the cofounding of SACMEQ by UNESCO is linked to the studies’ framework, which is focused on schools, grades and curricula . A similar framework is found in other ILSAs such as TIMSS and PIRLS, both launched by the UNESCO supported International Association for the Evaluation of Educational Achievement (IEA) . For Cardoso, this demonstrates how IOs' doctrines shape the ILSAs they launch, and how this then leads to the reinforcing of these doctrines by influencing countries’ policymaking .

Implementation and method of surveying

The foundation of SACMEQ studies is the training of planning officers within ministries of education . This includes the training of ministry officers in the areas of research methodology, with the aim of increasing the ability of staff to carry out large-scale, scientific surveys . This type of training is called capacity building, meaning that a countries capacity is built within their ministry of education to be able to evaluate educational quality .

The target population of the students taking part in SACMEQ studies are Grade 6 primary school pupils, a SACMEQ working paper highlights 3 reasons for this . Firstly, participation rates are high in Grade 6 therefore the performance of students at this Grade is demonstrative of the contribution of primary schooling to competency levels. Secondly, the assessment is administered in official languages, which school children are introduced to in Grades 3-4, therefore using a target population any lower than Grade 6 would be too close to the transition point between local and national languages. Finally, it was suggested by the NRCs that students in grade levels lower than 6 would be unable to provide valid home background information such as their socioeconomic environment .

The data collection process takes place over 2 days within schools to allow for the collection of missing or incomplete information on the second day after reviewing the data from the first day . Data is collected for SACMEQ by fieldworkers who follow an instruction manual throughout the entire data collection process . This manual contains detailed instructions on the random selection of participants and the fieldworkers undertake thorough training in following these instructions beforehand, in order to ensure the validity of the whole process . Data collected during projects includes background information on schools and the home environment of pupils . It also includes the assessment of students and teachers in the areas of mathematics and reading . Participants are also assessed on their knowledge of health .

Teachers complete booklets consisting of background information, health knowledge assessments, and either a mathematics or reading assessment depending on which subject they teach . Reading tests include the assessment of three domains: narrative prose (storytelling), expository prose (the conveying of factual information or opinion), and documents . Assessed domains in the mathematics tests are number, measurement and space-data (geometric shapes, charts and tables of data) . Teacher tests were designed to both suit the high competency levels expected of teachers, and to overlap enough with student tests for the performance of pupils and teachers to be measured on the same scale .

Data collected in cross-national studies is often challenged with regards to its validity, three dimensions to this challenge have been pointed out . The first is student achievement tests, of which questions are drawn as to whether they account for variations in culture and curricula . Concern has been highlighted by some researchers that the tests used in SACMEQ studies may align with some curriculums more than others and that not all pupils would have learnt the skills being assessed to the same extent .

Another dimension surrounds target populations and sampling, the question is whether target population definitions are consistent across countries and if sampling avoids bias . One criticism of the target populations in cross-national studies is that often countries differ in their policies regarding things like the age a child starts school and the length of enrolment . Researchers argue that this leads to problems with using populations based on age or grades . These problems include the impact that the length a child is enrolled in school and their age has on their achievement, this has been said to be problematic when performance is compared cross-nationally . Cross-national studies have addressed these issues using a ‘pure grade’ definition for their target populations, in SACMEQ Grade 6 students were the target population .

The reporting of results is the third main dimension of challenges to the validity of data in cross-national studies . Ranking countries in order of mean achievement has been heavily criticised for leading to ‘superficial... interpretations of the results’ which are then used to point out one variable as the reason for differences in the performance of countries .

Reporting

The performance of participants is reported as a mean score scaled using Rasch model of item response theory and as percentages in 1 of 8 competency levels identified for both reading and mathematics . Percentages are disaggregated by region, sex, school location (rural/urban) and socioeconomic status (low/high) .

The results from the SACMEQ assessments are collected and sent to the SACMEQ National Research Coordinating Centre in each country for analysis, a copy of results is also sent to the main SCC for cross-country comparisons . Each country then issues a policy brief and detailed country report which describes the background of their educational system, contextual information, the performance of students and teachers, and any policy recommendations derived from the findings .

Participating countries use the data collected by SACMEQ to compare their performance levels against expected norms, other countries, and across time . SACMEQ is able to compare outcomes internationally and against country specific standards of student proficiency due to the use of customised national indicators, with the decision of what is ‘acceptable’ student performance defined in each country by national experts . Countries use this to monitor and assess whether their performance is improving over time and in comparison to how other countries are performing, this data is then used to inform educational policy .

Outcomes of the assessments demonstrate countries performance, but input and process data are also collected by the SACMEQ projects . Input variables include the use of various learning resources, for example textbooks, and how successful they have been in reaching policy expectations . Process variables assessed include the frequency students receive homework and are aided at home with this homework . These data provide decision makers with in-depth ideas about problem areas in their education systems and has “shaped policies that focus on quality enhancement” .

The process of data collection within SACMEQ studies is focused on to a large extent, with high-quality data being one of the aims of the projects . SACMEQ does not use samples of data from an education system until it has been established as being up to date, reliable and accurate . Where data is missing or outdated, resources are put into “the preparation of an accurate statistical database” . In the second SACMEQ project, the data files were sent to the “Monitoring Educational Quality” unit at the IIEP in Paris, consistency checks were carried out for variables and files were checked and cross-checked for two years (2001-2003) before being confirmed to be clean .

Influence

Previous Director of SACMEQ Saul Murimba states that data from SACMEQ I and II allowed countries to review trends in quality and the impacts of new policies . Murimba also states that the focus of SACMEQ projects on obtaining clean, high-quality data which has gone through many rounds of checking, has led to a “culture of quality” throughout ministries, in some cases leading to more rigorous quality checking in the regular practice of data collection .

International studies such as SACMEQ lead to consistent, in-depth reviewing of a country's curriculum, many have seen this as a benefit of participating in these studies . Arguing that analysing a country’s curriculum and comparing it to other countries’ during the process of creating assessments to be used internationally, leads to countries thoroughly reviewing their curriculums, allowing for attention to be brought to aspects which need attention .

The international orgranisations behind many international large-scale assessments claim that the data collected provides a basis for the making of new educational policies and the reviewal of how policies are working .

It has been argued that the data from ILSAs are more often used in reforming education systems linked with political agendas and competition between countries . This argument draws on the reduction of data to average scores and rankings, which leads to the adoption of policy agendas under pressure to conform with high performance countries . This is argued to clearly demonstrate that ILSAs are used to reinforce the infrastructural global governance of education, by generating comparable evidence which shapes what valuable education policy is .

South Africa

SACMEQ reports are used to inform education ministers about what areas of the countries education system need to be prioritised. During SACMEQ III it was found that there were large performance disparities between pupils in South Africa, whereby pupils in rural schools, who generally have lower socio-economic status, “perform worse academically” in comparison to pupils in wealthier schools who have access to necessary resources . This has led education ministers, such as the Minister for Basic Education Angie Motshekga, to address these disparities by making decisions such as the prioritisation of “farm, rural and township schools” in ensuring they are “functional and are adequately equipped” . The Director-General of the Department of Basic Education Bobby Soobrayan also stated that SACMEQ III’s findings were to be used in the department’s “Action Plan 2014 Toward Schooling 2025 and its goals” which included aims such as the strengthening of teacher training and the provision of textbooks for every pupil for each of their subjects . A report published on SACMEQ III in South Africa stated that the country's participation in the project developed its capacity to evaluate educational quality at a local level and to compare its performance against other countries .

The SACMEQ III tests and questionnaires used to collect data in South Africa were available only in English and Afrikaans. Experts have pointed out that this put many students who do not speak these languages at a disadvantage, and that this is a race and class related issue as most students who speak English as a first language are white and affluent .

Summary

SACMEQ is an organisation which aims to monitor and evaluate educational quality using large-scale cross-national assessments . 16 Ministries of education across the Southern and Eastern Africa region partake in these assessments . Data is collected during these assessments through the use of surveys in which teachers and Grade 6 pupils are tested on mathematics, reading and health knowledge .