Assessment: reports from the ATN Conference (IV)
Panel: Assessing in the disciplines
The panel, two papers, both focus on what staff are thinking about assessment, especially in response to institutional change. How do they make decisions? What do they think about assessment in a lived way? Importance of disciplines emerges strongly here.
Assessment for learning, learning through assessment: perspectives from creative industries (Hong and Vaughn)
We assess all the time as we make our journey through life; perhaps we need assessment for better living. A key principle – quality – what and how to students know it? They need to be provided with evidence and examples of quality work to know what to aim for. This helps to share and negotiate goals, and have more transparent outcomes.
Cites Boud (2007) – system is inert, conservative and slow to change; fears of the effort involved in major assessment change and also what it would reveal about the system.
Emphasises the need for assessment FOR learning, using criterion-referenced assessment, to avoid assessment OF learning; focus on patterns of assessment, number, type and weight will be mandated.
The research project described in the paper is about the ‘lived reality’ of Creative Industries teachers; are they changing their practices – is it a myth that they are conservative? Research involved interviews with staff across all 11 disciplines in CI plus core units. Array of sizes and types of programs – from 1500+ student units in core; to 10 students 3rd year performance. Questions probed impact of assessment regime on their working lives, understandings, current practices, innovations, how do students engage (as perceived by staff). Data collected, being analysed currently. While emphasising assessment, the research uses literature on teaching and learning practices as well as assessment directly to code the data from interviews. [As always, I would question whether the language used can be coded reliably in this way; no comment on what coder reliability testing was done; perhaps the variety of disciplines makes the language of education irreducible to shared norms?]. Much of this research involves assessing whether teachers self-describe as transmitting K or facilitating learning; or who are interested in teaching content or enabling students to develop conceptions. [Ultimately, too, the coding of expressions may also turn them into something they are not!]
Findings:
The ‘chaotic patchwork’: assessment decision making in the disciplines (Readman and Allen)
Research reported here was 19 interviews with selected staff who would be able to talk meaningfully about assessment practice decisionmaking; data support other work done by the researchers to map what is in curriculum documents.
What were the factors – personal, institutional, disciplinary? How did they influence assessment? What were the perceptions OF these factors (Symbolic Interactionism, focusing on interplay of structure and agency). Used the typology of know, believe, and do. Was there consistency? Did people know do something but, but not do it? Did they believe in what they did?
Notes gender and race perspectives influence academics’ implementation of assessment; the creation of ‘typologies’ of students. Excellent finding – students as criteria blind / seeking / savvy – they either game the system or, worse, have no capacity to game the system. [This finding does tend to undo the educationists’ lust for criteria to solve all problems: it simply creates a new set of problems, especially when we consider Rust’s criticism of the pursuit of explicitness]
What influences: own experiences as a student; inheriting courses by others which challenge; mentors; collaboration; experience and industry; students’ reaction to assessment; evaluations; norm-referencing / bell-curve influences; policy not rated highly – because research dates from a time when policy was light touch. [Based in SI, this reporting lumps together a variety of things, without attempting to categorise].
Interviews showed staff theorising their own practice (Bruner (1999)); people were ‘developing’ through reading and inquiry, not just going to workshops, contexts of practices – hybridity, multiple interpretations of policy; Neumann 2001 – academic disciplines are very important; rule-bending to feel ok about policy and achieve things.
Critical Commentary
Throughout the conference there has been considerable discussion of and alarm at the lack of collaborative enterprise amongst students – e.g. Hong focuses us on the stressed student ‘alone’ doing her assignment. Yet there is considerable evidence that students cooperate a lot on assignments, either with individuals or collectives, in ways that are all about the informal learning context, the support offered by the pre-existing networks and communities of learning which are not visible to teachers (and probably work because they are invisible). Perhaps one of the challenges here is to start realising students are not the poor creatures, beset by dramas and not able to cope unless academics to their job ‘right’? Students actually respond to the challenges of it not being quite right, or somewhat difficult. Academics need to work with the ‘invisible partnerships’ – teacher and student groups, which are not represented in a manner accessible to academic; but cannot either rely on, nor attempt to actively mobilise those partnerships.Another key question that arises for me, as I listen to some discussion of Chris Rust’s earlier work (2002) and now. Rust appears to have moved on from a strong enthusiasm for close alignment of assessment with learning outcomes, to a more considered judgment that, at times, learning outcomes impede effective and creative assessment approaches (since learning outcomes are artificially extracted from the complexity of what is the goal). Outcomes really do pose a major question, as well, when one considers authenticity – perhaps there is a ‘translation’ needed from the specifics of an assessment back to the learning outcomes? That is: assignment x which attempts y and is assessed against z: we need to show how y and z ‘re-express’ the more formalistic and inflexible learning outcomes.
Readman’s paper brings up a very neat point regarding academic perspectives that students are “assessment driven”, as a negative. What they mean is: students are driven by the pursuit of marks (not necessarily high marks, but ‘personally satisfying marks’; such marks (in the minds of students) can be ‘too low’ for academics who want people to aspire to quality or that can be ‘too high’ for academics who want people to judge their own competence accurately. Of course this kind of pursuit of quality is a validation of the life choices of the academics themselves! Should we care if students are driven by marks? Should we re-educate them? Challenge them? Get them to realise it is not worthwhile? Will disputes with students over ‘you need to learn that learning isn’t what you thing’ overcome the capacity of academics and students to form coherent partnerships with differing perspectives?
