Validität und schriftliche Sprachkompetenz: Eine Studie zur Bewertung schriftlicher Leistungen im Fach Deutsch an schwedischen Schulen

Validität und schriftliche Sprachkompetenz: Eine Studie zur Bewertung schriftlicher Leistungen im Fach Deutsch an schwedischen Schulen

Validity is a fundamental aspect of quality within the field of testing and assessment. Despite this fact, little research has been conducted on the validity of written assessment in a Swedish educational context and even less regarding assessment in an additional foreign language. In particular, there is little work on teachers’ scoring and the relation between ratings of students’ performances in upper secondary education and the external reference levels of the CEFR, the widely used framework from Council of Europe.

Against this background, the present study was designed with the aim of examining the validity of the assessment of students’ written language proficiency in German at different steps according to the Swedish school system. The focus is on (1) raters’ construct conceptualisation, (2) inter-rater consistency of the Swedish raters, and (3) the relationship between Swedish ratings and ratings at a B1 level of the CEFR. The student data comprise 60 texts written in L2 German by Swedish upper secondary school students in courses aiming for three different steps of the national curriculum. The essays were scored by (a) students’ own teachers, b) external Swedish raters using Swedish national performance standards and (c) external CEFR raters in terms of the B1 level of the CEFR. Analysis of scores and rater comments were based on qualitative and quantitative methods, for example qualitative data analysis (QDA) and descriptive, correlational and reliability statistics.

The results were analysed in relation to theoretical concepts of validity and central validation frameworks. The findings show that raters pay attention to a wide range of aspects in students’ written proficiency, although, to some extent, different interpretations of how student texts should be evaluated in relation to the national performance standards and a tendency to comment more on certain aspects could be observed. Analyses of inter-rater consistency indicate that the ablitity of Swedish raters to rank students’ performances is satisfactory, but that there are challenges in reaching general agreement, especially for intermediate and higher scores. Additionally, the results suggest that a pass level of Tyska 5 at upper secondary school is roughly equivalent to a B1-level of the CEFR. Finally, the thesis highlights the importance of rater training and discussions about assessment as part of strengthening teachers’ professional assessment competence. By exploring validity from different perspectives, the study provides a more complete picture of learner written assessment in an additional foreign language in Sweden and contributes to a deepened conceptual understanding of validity aspects within a Swedish educational context.