Quality in School Education, a "Socrates" Pilot Project

National Report from The Norwegian National Committee

Preface

In 1997, the European Commission in Brussels invited Norway into the pilot project "Quality in School Education". Norway participated with five schools: Alta videregående, Nord-Østerdal videregående, Riksfjord barne- og ungdomsskole, Sverresborg ungdomsskole and Vestmyra ungdomsskole. The pilot project lasted from August 1997 to July 1998. The National Committee wants to thank the schools for their enthusiasm and good work, and also thank the European commission for accepting Norway as a participant. The pilot project has given valuable experience about self-evaluation by schools.

Ola Moe

National Coordinator

Contents

Appendices

1. Members of The National Committee - participating schools with addresses
2. Map of Norway with the five schools marked
3. Schedule for the project in Norway

NATIONAL REPORT

Initial stage - organisation

National level

The National Education Office in Sør-Trøndelag, which is a regional office under the supervision of the Ministry of Education, Research and Church Affairs, was asked to be the national administrator in a letter from the Ministry dated June 16th 1997. A National Committee was organised with three members from the Ministry and the Director in Sør-Trøndelag as members ( Appendix 1). The Ministry informed the teacher unions´ national offices about the project and decided in agreement with the unions that further information should be given from the National Education Office in Sør-Trøndelag.

Selection of schools

Six upper-secondary and twelve lower-secondary schools were given the chance to apply for participation in the project. They were chosen by the National Committee. Officials in the Ministry and the regional education offices supplied the committee with names of relevant schools. Fifteen filed their applications in time. From the fifteen, five schools were recommended to the European Commission. The National Committee followed the criteria from the Commission. Three of the five applied to follow subproject 1, two applied for subproject 2. The schools are situated in different parts of Norway (map, Appendix 2). Two are upper-secondary schools, three lower-secondary. One is a combined school: It has pupils both at primary and lower secondary levels. The upper-secondary schools have classes for general education and different lines of vocational training. All five were accepted by the Commission (Appendix 1).

School level

The five schools organised local steering groups and contracted "critical friends" as the project guidelines suggested. They needed some time after the start of the school year around August 20th, but matters were sorted out during the first week of September. The schools were free to seek professional support from their regional education office or a university or college.

External evaluation

The National Committee made arrangements for an external evaluation of the project. The national evaluation is supervised by Programme for Educational Research at the Norwegian University of Science and Technology in Trondheim. The report will be written by Åse Streitlien, the Regional College in Telemark (Høgskolen i Telemark, Notodden).

A brief summary of the activities in Norway

Some of this information is also given in the chapter Monitoring at national level. The National Committee drew up a time schedule for the project (Appendix 3). Work was done according to the plan with some minor adjustments. The schools organised steering groups and contacted "critical friends" in early September and submitted Questionnaire no. 1 by September 15th, 1997. The questionnaires were completed by the head teachers of the schools with some help from local steering groups. Not too many teachers, pupils or parents were involved as the deadline came so early.

The self-evaluation profiles (in the Guidelines) were completed before the Luxembourg conference. The National Committee had a meeting with the schools in Oslo before leaving for Luxembourg.

The National Committee arranged a national conference (workshop) on December 2nd, 1997. Local steering groups and the relevant regional, educational offices were invited. Each school reported from their work so far and commented on factors with positive and negative influence on the project. All schools were challenged to explain their evaluation profiles: Who had taken part in the judgements and what kind of evidence that lay behind the conclusions.

The schools were put together in groups to discuss experiences and strategies. The national coordinator headed the conference, summarised and gave some advice for further work (Appendix 4). The schools were asked to complete an interim report by the first week of January.

The National Committee submitted its preliminary report to the Commission on January 19th, 1998.

The National Committee visited the five schools in February and March. The visits were meant to take place in January, but the committee decided to postpone them till after the second national conference (workshop), February 9th and 10th. The committee sent a list of questions and subjects for discussion in advance (Appendix 5) .

The second national conference was arranged on February 9th and 10th, 1998, for school steering groups and "critical friends". The National Committee had invited three speakers with professional background in school-based evaluation. Two of them were also "critical friends", Geir Halland (Sverresborg) and Gunnar Stangvik (Alta). The third one, Lars Monsen, had been doing research in the field and written several reports, some of them ordered by the Ministry. The national coordinator commented on the schools´ interim reports. A summary with some "reminders" were mailed the schools afterwards (Appendix 6). The schools were also asked to submit a second interim report by March 31st, 1998. During the February workshop the participating pupils were very active. They wished to spend some more time together and were allowed money to have a separate meeting for all the pupil representatives in April. It took place in Alta, April 20th - 21st, 1998, and there is a separate report from the meeting (Appendix 7).

The use of "critical friends" represented something new in school-based evaluation in Norway. The National Committee planned a meeting with the "critical friends" in March. Unfortunately the busy "friends" could not find a common date for a meeting. Instead the committee arranged for a written evaluation (Appendix 8). The five "critical friends" answered a short questionnaire about their activities and impressions.

The national coordinator visited all the schools during May and early June, 1998. The schools were asked to finish their final reports by the first week of July, 1998.

Remarks form the national committee

The remarks and considerations in the following paragraphs are based on:

The response form the participating schools to the Commission´s questionnaires and profile form.
Two interim and one final written report from each of the schools to the National Committee.
Background information from the schools following the interim and final reports.
Information from and discussions in the two workshops arranged in Trondheim.
Two visits to each school by members of the National Committee.
Information from the "critical friends" given to the National Committee.
General conclusions about school-based evaluation in Norway given in research reports.

Self-evaluation and external evaluation

In Norway there is a tradition with school-development processes that includes self-evaluation since the mid-seventies (new national curriculum in 1974). During this period there has been a distinction between "school-based evaluation", which is the school´s own responsibility and activity (self-evaluation) without any premises given by any external part and "school-evaluation" that is the more open concept which also contains "external evaluation". In the last three-four years this distinction has been challenged.

Since October 1997 self-evaluation has been mandatory for all schools in Norway. The municipal authority is given the responsibility to organize and monitor the activity. This implies that the schools are expected to do the evaluation themselves, but that the task to some extend, can be defined by the municipal authorities.

External evaluation has been more focused since 1992 as a consequence of new legislation (1992), reforms in upper secondary and vocational education (1994) and reforms in primary and lower secondary education (1997). With the new legislation from 1992 the municipal authorities were given extended responsibility. The legislation and national curriculum became more oriented towards principles, content and attainment goals, while local boards and schools were given more of the responsibility to decide how to realise the curriculum. As a consequence the national level will have less opportunity to manage the educational system by detailed regulations about how it is tooperate. Instead the Ministry has to focus more on how pupils actually benefit from the education they get, and under which conditions they are best able to accomplish their learning tasks.

At the moment the Ministry is working on a White Paper on a national system for evaluation. A "national system" in this context will include what the national level does, what the local authorities are expected to do and what the schools are to do. A national system must take into account the differences in the obligations of the State, the school boards and the schools, but at the same time the importance of cooperation.

The national level is responsible for legislation, the national curriculum, teacher education, finance (transferring money to local governments). Local governments are responsible for buildings, hiring teachers, local organisation of education, allocation of funds to schools, in-service training of personnel. Schools have the professional responsibility for the education.

Evaluation programmes must be related to the different responsibilities at the national, local and school levels. National evaluation programmes will search for knowledge about the system as a whole in order to improve for instance legislation and/or the national curriculum. But facts, figures and conclusions from evaluation programs may be essential input also for local governments and schools in their efforts to improve. How they are presented therefore will be of importance.

It has been known for a long time, that if the professional, teaching practice is to be improved, it has to be done by the schools and teachers themselves. That is why motivation and understanding of the self-evaluation processes are so important. And this is also the reason for the importance of taking care of the essence of the concept "school-based" evaluation - that the schools themselves must have some freedom in the process.

Analysis of the final questionnaire

General remarks

Primary schools in Norway are familiar with school-based developmernt projects, even if the practice has differed conciderably between schools. In this pilot project Vestmyra and Riksfjord were very experienced in the field. Sverresborg is a more traditional school with high standards, but less practice in self-evaluation. The two upper-secondary schools had little experience, which is more typical for this kind of schools, even if there are experienced upper secondary schools. Upper-secondary schools are more heterogeneous and than primary and lower-secondary schools.

The experienced schools are more aware of how processes of this kind must be planned and led. They know the importance of visible leadership, they understand how to work with the staff, pupils and parents as a whole and they have established a culture with ambitions of bcoming better. They are less afraid of putting difficult questions on the table. Usually they also are self-confident with a high-profiled image. Experienced schools have developed an understanding of themselves as learning organisations, while less trained schools often have a more individualised culture - the school is more a number of separate classrooms, than a coherent unit.

According to reports from researchers and others, Norwegian practice up to now has had some obvious insufficiencies, even when looked at in experienced schools.

Some have to do with method:

Problems and questions are often raised in a too general way.
Fact-finding and analysis have not been done in an exact manner (as a consequence).
Relevant theory and/or established knowledge is not brought to attention.
There has been little activity looking for evidence to decide whether practice actually has improved.
Most evaluation activities are based on subjective statements, impressions and feelings that can be important in some cases, but they often need to be supplemented by survey indicators or other means of documentation.

There are positive exceptions from this general picture.

The same observation can be made in the pilot-project so far. Nord-Østerdal is an exception by relating its work to theory. Sverresborg became aware of the methodological problems along the way and started an impressive gathering of information by the end of the school year, that was later analysed with statistical methods in an almost "scientific" way. Next year they will identify some specific indicators of good teaching and learning practice and work with improvements. Riksfjord and Vestmyra are preparing a more thourough analysis of gathered information.

Some reported insufficiences have to do with the process:

The projects have difficulties with engaging everybody on the staff.
The school management and/or project steering groups often report that the projects have been very useful and educational, but the questions dealt with are often not felt to be very important by the teachers.
The projects are often taken care of by project steering groups without involving headmasters or other school managers.
Even after several years it is hard to find any effects on the teachers and their practice.
Pupils, parents and others who are dependent on the schools´ efforts, are often not involved in the projects.

Here the participating schools also differ as the responses to the questionnaires show.

Researchers also report that very much of the evaluating activity is concentrated on how the school is organised. There is little attention to standards, to what pupils learn, and to teaching and learning processes in the classrooms. All five schools in the pilot project show interest in "school as a learning place" and especially the lower-secondary schools are trying to develop tools that can be of help in assessing the quality of teaching and learning.

The pilot project

How was the project received

All five schools were very enthusiastic. They expected to learn more about how to do self-evaluation. They were curious about what the international connections would lead to. Some schools also felt it an honour to be selected.

There were some uncertainties: Would it be too time-consuming? Was it possible to reach sensible conclusions in only one year? How would the project match the other activities the school was engaged in?

Did any part of the project work well?

The use of "critical friends" was a success. Different kinds of "consulting assistance" are known in other school-based projects. The concept "critical friend" gave certain connotations that were helpful and exciting. It is interesting that the five schools chose "critical friends" with different backgrounds: One professor of philosophy (Nord-Østerdal), one professor in education (Alta), one assistant professor in education (Sverresborg), one municipal, professional consultant in education (Vestmyra) and finally, one general manager of a shipyard (who also was a father with children in the school) (Riksfjord). The shipyard manager and the professor of philosophy got the best rating.

The "critical friends" were well received as discussion partners - as persons who could point to fundamental questions and necessary steps in the evaluation process. They were used in different ways. Some communicated primarily with the school project steering group (Sverresborg, Alta), some of them also met the rest of the staff, pupils and parents (Nord-Østerdal, Riksfjord).

The involvement of pupils worked well. It is a central element in the Norwegian, national, curriculum reforms, both in primary and secondary education, that pupils should be more involved in planning and evaluating their own learning activities. Reports from researchers show that this is not often the case, even if it is very often spoken of as important. The National Committee accepted to pay for a separate meeting in Alta for two pupil representatives from each school. A report from the meeting is enclosed (7).
Four of five schools found it very useful to include parents in their work. In upper secondary school the contact with parents usually is less frequent than in primary schools. But also the upper secondary schools in the project say that they will try to strengthen their communication with the parents.
The headmasters, school project steering groups and others that were very directly involved in the pilot project, report that they found it interesting, that it gave new insight in a complex problem and that they would like to continue the work next year. In respons to question 14 all schools say that self-evaluation improves management.
The documents from the Commission were generally well accepted. Some criticised the self-evaluation profile form as being too general. Otherwise the practical guide seemed to be useful.
The discussions around the self-evaluation profile are assessed positively (question 6.2). The assessment of the impact on the school is also more positive than negative (question 12). Sverresborg feels it is a little too early to respond. The attitude towards self-evaluation is more positive than negative (question 14) and four of five schools say that self-evaluation should have more importance in Norway (question 15).

Major obstacles

The schools were asked to comment on this point also in their interim reports.

A self-evaluation process is time-consuming. It has to find its place among other day-to-day businesses. This makes it difficult to involve all the teachers, the whole staff, pupils and parents in the way that is felt necessary (questions 8.4 and 9.4).

Schools that organised the self-evaluation as separate projects (Nord-Østerdal, Alta) seemed to have greater difficulties than schools that integrate self-evaluation in their regular schedules lead by the school management. This has to do with authority and legitimation if teachers or others are sceptical. During the last visit to the schools in June, this was discussed and the school management intended to take more direct responsibility next year.

All schools found it hard to make the project interesting and useful for all teachers. This can be explained in different ways: Partly it is a consequence in schools where the culture is very individual, partly it can be understood if the evaluation focuses on subjects with variable importance to the teachers.
In the general remarks (above) the National Committee points to the fact that the Norwegian practice has been a bit weak in method. The pilot project reflects this. The National Committee has on several occasions asked the schools to be more specific about why certain subjects were chosen for evaluation. To describe what kind of indicators, arguments or facts that lay behind. The common answer is that they, in a democratic way, looked for subjects that best covered the joint interests of the groups. Consequently it may be hard to find the right tools to assess if the situation is getting better. (See answers to questions 8.2 and 9.2). Only one school (Nord-Østerdal) brought theory and existing knowledge into the project in a systematic way. The positive part is that the democratic approach improves the climate for self-evaluation.
None of the schools completed the self-evaluation profile at the end of the project (question 6.1). This may be explained by the fact that the profile form was felt to be too general.
The European dimension (question 4) had very little importance in Norway except for the use of the documents. The European Internet home page was established rather late and offered only addresses and general information. The newsletter got a low rating. The National Committee could have pushed more on this aspect, but a couple of attempts, through e-mails to schools in other countries, gave no response. Three schools gave the Luxembourg conference a positive rating.
Maybe it is not a major obstacle, but the size and structure of the two upper-secondary schools represent specific challenges. The two schools have classes for general education and classes for different lines of vocational training within the same school. They both have units that were separate, individual schools until few years ago. They will probably need some time to develop the "we" culture that is needed as a basis for self-evaluation.
It must be noted that the local, municipal authorities, showed very little interest in the project. They were kept informed by the National Committee along the way. The committee is certain that involvement from local school authorities will strengthen the processes.

Who were involved

All groups were involved except non-teaching staff. It is commented above that pupils´ and parents´ involvement were felt to be especially valuable. All schools report problems with getting all the teachers interested.

Was the project a new experience?

It was not a new experience for any of the schools, but some of them had more experience than others (Riksfjord, Vestmyra). Nord-Østerdal had a parallel evaluation going on, led by another steering group. The National Committee is of the opinion that the two projects should have been integrated instead of being handled separately.

Change of teaching and learning?

Only two schools took part in subproject 2. None of them documented any change in teaching or learning. Four schools report a stronger commitment to teaching and learning (question 12) and four of five schools have the belief that self-evaluation can improve teaching (question 14). The fifth, Sverresborg, feels it to early to tell, at this stage in the process.

Some examples of good practice

Sverresborg is already mentioned as an interesting school. It had not much experience before the project. There was some hesitation in the school, especially among the teachers. The school management knew the school was strong in educational standards, but more uncertain about other areas. This is reflected in their self-evaluation profile in the beginning of the project. Sverresborg seemed to be the school with the lowest self-esteem in some areas even if they knew they produced strong results. During recent last months the school has completed an impressive analysis of teachers' and pupils' views on teaching and learning as a basis for planned actions in the coming years. They are reluctant to draw hasty conclusions (questionnaire no. 2). In the committee's opinion, Sverresborg is the school that shows the strongest commitment to documentation and verification of positions.

Riksfjord and Vestmyra are the schools with the broadest experience of such evaluations. Both schools have developed strong school cultures with a genuine interest in improving practice, and loyalty towards the assessment processes. They have a qualified school management that look upon school-evaluation and school development as very important matters. It is interesting that these two schools, that are geographically separated by more than 1 000 kilometres, "found each other " as a result of the pilot project.

Monitoring at national level

Information about how the project was organised in Norway is also given in the first part of this report.

National committee

The National Committee had four members, three from The Ministry of Education, Research and Church Affairs and the national coordinator who is the regional director of education in Sør-Trøndelag.

Teachers´ unions were not represented in the committee but received running information through the National Education Office in Sør-Trøndelag.

The National Committee arranged two national workshops for all the schools with their local steering groups. Minutes from the workshops accompanied by some "good advice", were sent the schools. Members of the committee visited the schools twice. The school visits followed a short manual that was distributed in advance of the first visit (Appendix 5).

The schools were asked to submit two interim reports, one by the end of December, 1997, and the other around April 1st, 1998, and a final report July 1st, 1998.

Local support

The five schools come from different counties in Norway. The National Education Offices in these counties were asked to assist in the project in the way the schools wanted. They also took part in the national workshops.

Critical friends

The "critical friends" were chosen by the schools. The National Committee gave no directions about who could be chosen as a "critical friend". The National Committee asked the "critical friends" to complete a short questionnaire about how they judged and conducted their task (Appendix 8).

Networks

There was little contact between the schools except when they met at the workshops. Riksfjord and Vestmyra will cooperate more systematically in the year to come. Riksfjord and Vestmyra have taken part in other local networks on school-based evaluation.

Inspectorates

Norway does not have any inspectorate at either local or national levels. The committee visited the schools, but the visits were not seen as inspections.

Recommendations for policy making

There is little evidence about how school-based evaluation actually improves teaching and learning. The recommendations of the National Committee are therefore given with some reservations.

As mentioned above, the Ministry of Education, Research and Church Affairs is presently working on a White-Paper on a national system for evaluation of school education that will include school-based evaluation. The Norwegian policy and principles will be affirmed after the White-Paper has been passed by parliament.

Nevertheless, both this pilot-project and research reports from other self-evaluation projects, do indicate some kind of actions.

At school level:

It is important to give support along the way. Relevant theory and examples of good practice should be brought to attention. The experience with "critical friends" was successful. If an external part follows the project, more pressure is put on the school to keep the pace.
External stakeholders, such as parents, local politicians, local businesses, other schools (for instance an upper-secondary school that get pupils from a lower-secondary school) must be partners in the process.
In Norway it seems necessary to be more aware of theory, method, preciseness and evidence.
If the school is inexperienced, the first task must be to work on motivation and understanding in order to develop an evaluative culture in the school.
The self-evaluation must be led or strongly backed by the school management. When it is led by project steering groups with limited authority in the school, there is a risk of marginalization. The evaluative actions should be part of a regular schedule. The teachers and pupils should be made more aware of the evaluative aspect of tasks already dealt with. In this sense there seems to be a paradox: School-based evaluation is most successful if it is not identified as such, but is an integrated part of day-to-day business.
Self-evaluation must concentrate more on pupils' attitudes towards learning and pupils' benefits (outcome) as indicators for what is good teaching.
Self-evaluation must also concentrate more on the social-psychological climate in the classroom, how teachers and pupils communicate and the interrelationship between pupils.

Local government (municipal) level:

Local authorities must be more specific about what is expected from schools.
Local authorities must develop routines for monitoring self-evaluation processes in schools (and other municipal activities).
Municipal boards (or administration) must stimulate networking between schools.
Municipal boards must pay more attention to self-evaluation as a subject in school leadership in-service training.
Municipal boards should establish some sort of education consultant/assistance programmes

National level

Establish a national programme for evaluating the educational system.
Pay more attention to self-evaluation in national teacher-training programmes.
Develop tools and guidelines (like the one in the pilot project) that can help schools.
Be more systematic in feeding back facts and conclusions from national evaluation programmes to local authorities and schools.
Monitor the municipal boards´ policy and actions.

APPENDIX 1.

The Norwegian National committee

Ida Drage, deputy head of department in The Ministry of Education and Church Affairs

Annbjørg Rimeslåtten, senior executive officer in The Ministry of Education and Church Affairs

Eva Schøyen, senior executive officer in The Ministry of Education and Church Affairs

Ola Moe, director of National Education Ofiice in Sør-Trøndelag

National coordinator

Ola Moe

Statens utdanningskontor i Sør-Trøndelag,

Statens hus

7005 Trondheim, Norway

tlf 47 73 94 93 00 fax: 47 73 94 93 53

Participating schools

Alta videregående skole,

N - 9500 Alta, Norway

Tlf: 47 78 44 11 00 fax: 47 78 44 11 90

Headmaster: Guri Lund

Nord-Østerdal videregående skole,

N - 2500 Tynset, Norway

Tlf: 47 62 48 13 77 fax: 47 62 48 07 88

Headmaster: Jan Inge Nordeng

Riksfjord skole,

6420 Aukra, Norway

Tlf: 47 71 17 41 88 fax: 47 71 17 42 34

Headmaster: Knut Rød

Sverresborg skole,

Framveien 20,

N- 7020 Trondheim, Norway

Tlf : 47 73 53 29 10 fax : 47 73 51 11 41

Headmaster : Sidsel Nyhagen

Vestmyra ungdomsskole

Postboks 154,

N - 8201 Fauske, Norway

Tlf: 47 75 64 46 99 fax: 47 75 64 33 75

Headmaster: Roland Rolandsen

APPENDIX 3

Socrates - pilotprosjekt: Skjematisk plan for gjennomføring av pilotprosjektet Revidert 14.05.98