A Tale of Two Systems: Examining the Implications of School Evaluation for School Leadership

A Tale of Two Systems: Examining the Implications of School Evaluation for School Leadership

Download presentation

26 SES 09, Leadership in the Face of Accountability and Testing
Paper Session
Time:2016-08-25
13:30-15:00
Room:OB-H1.51
Chair:Gerry MacRuairc

A Tale of Two Systems: Examining the Implications of School Evaluation for School Leadership

This paper seeks to examine the impact school external evaluation has on school leadership practice through the use of two case studies based on the systems of evaluation in Norway and Ireland. These school systems have different approaches to school evaluation;  the Norwegian system has a narrow,  more limited focus on legal compliance while Ireland has a broader focus on all aspects of school practice and outcomes. In setting the context for this review of school evaluation three broad areas were identified to act as a framework within which to conduct this critique i.e. the intended, implemented and attained or achieved outcomes of inspection and evaluation (see Akker, J. van den (1998). This framework has been used in the past to examine situations where there are identified ‘major gaps between ideals and outcomes’ (ibid, p.6).  Because the literature identifies such gaps relating to evaluation and inspection, this model provides a very clear critical lens of enquiry to be applied to both national contexts. It also provides a strong framework within with to present and critically interrogate the increasing range of scholarship emerging in relation to inspection and evaluation.

The first part of the framework relates to what is intended by School evaluation in both national contexts. Specifically the model explores the stated/ espoused purposes of school evaluation by policymakers and in policy document and seek to identify the extent to which evaluation is intended to serve accountability, improvement or other outcomes.

The second area focuses on how the evaluation and inspection is actually conducted in schools. i.e.  how the policy intentions espoused by ministries and inspectorates are enacted in schools, how evaluation is  or how they are translated into practice by inspectors and inspection teams.

Finally and arguably the most important dimension of this model, is a critical examination of the actual outcomes of school evaluation on what school leaders are expected to do to mediate the outcomes of school evaluation in order to take account of recommendations arising from evaluation. 

A key focus in this report will be the extent to which each of these domains are aligned or the extent to which there are incongruities both within and between each domain of practice.

Two main theoretical perspectives will facilitate this comparative analysis:

Some recent work on the multiple, sometimes contradictory and incongruent purposes of inspection and evaluation. Key issues here will include work on neo- liberal imperatives driving the development of inspection and evaluation systems, the homogenisation of inspection process, the increasing comparative focus of inspection results.

The manner in which evaluation and inspection function as very substantial constituents of the pedagogical device - conceived by Bernstein (1990) and formed of the rules of Distribution, Recontextualisation and Evaluation. The Pedagogic Device also facilitates the relay of rules and procedures and transforms knowledge in to pedagogic communication (Singh, 2002) and in this way operates similarly to Bourdieu’s (1990) concept of habitus. The Pedagogic Device (Bernstein, 1990, 1996, 2000) goes further in demonstrating modalities of control with regard to evaluation development and implementation. The device appropriates other discourses, which are selected, transmitted, sometimes altered and influence the operation of professional fields. Bernstein (1996) explained that the device orders and reorders discourse through grammars in this case as they relate to evaluation,  within itself and ‘regulates the distribution of knowledge by either enhancing or constraining classroom communication’ (McFadden and Munns, 2010, p. 362).

Method

This study consists of two phases: the first phase is a discourse analysis of the main documents in each of the countries that frame inspection and evaluation activities in each of the countries. These documents provide considerable detail in relation to the intended focus of school evaluation. In this way they articulate what it is policy makers, ministries and municipalities hope to achieve by their evaluation technologies and how it is the process they use in conducting school evaluation

The second phase involves empirical research with school leaders and teachers in relation to the intended and achieved elements of evaluation. Interviews and discussion groups will be used to explore this aspect of the study and to examine in particular how evaluations impact teaching and learning in classrooms, the priorities of schools and school leaders and teacher and leader identity.

Expected Outcomes

It is expected that the study will reveal a number of unintended consequences of evaluation on classroom practice and school development activities. Initial analysis has revealed that the recent range of accountability measures in each of the countries represented in this analysis have shifted the impact of evaluation and inspection with very clear implications for school leadership. Leaders are often caught in the gaps created by the differences between the stated function of evaluation and the actual experience of evaluation in schools.

References

Allen, R. B. (2012). How should we treat underperforming schools? A regression discontinuity analysis of school inspections in England. Centre for Marker and Public Organisation (CMPO). Bristol: University of Bristol.
Altrichter, H. a. (2015). Does accountabilty pressure through school inspections improve school improvement? School Effectiveness and School Improvment , 26 (1), 32-56.
Bernstein, B. (1990). Class, codes and control, Vol. IV: The structuring of pedagogic discourse. London: Routledge. Bernstein, B. (2000). Pedagogy, symbolic control and identity: Theory, research, critique (rev. ed.). London:
Rowman & Littlefield.
Ehren, M. a. (2008). The relationsip between school inpsections, school characteristics, and school improvement. British Journal of Educational Studies , 56 (2), 205-227.
Ehren, M. P. (2015). Setting expectations ofr good education: how dutch school inspections drive improvement. School Effectiveness and School Improvement: An international journal of research policy and practice , 26 (2), 296-327.
Hopman, S. T. (2008). No child, no school, no state leaft behind: Schooling in the age of accountabiltyy . Journal of Curriuclum Studies , 40 (4), 417- 456.
Ozga, J. (2000). Education policy in the United Kingdom: the dialectic of globalisation and identity. Australian Educational Researcher , 27 (2), 87-95.
Ozga, J. (2014). Knowledge, Inspection and the Work of Governing. Sisyphus Journal of Education , 2 (1), 16-38.
Van Bruggen, J. C. (2010). Inspectorates of Educaiton in Europe: Some comparatvie remarks about their tasks and work . Standing International Conferecne on Inspections (SICI).
Wilcox, B. &. (1996). Inspecting Schools: Holding Schools to account and helping schools to improve. Buckingham: University Press .

Author Information

Gerry MacRuairc (presenting)
University College Dublin, Ireland

Eli Ottesen (presenting)
University of Oslo
Department of Teacher Education and School Research
Oslo