Summary of the Adaptable Assessment seminar

Annika Johnson, University of Bristol
Published October 2020

Of all the changes, fixes and adaptations the university teaching community has had to make in response to the Covid-19 pandemic, the changes to assessment might just be the ones which stay with us permanently. Economics teaching has traditionally relied on high-stakes, final, closed-book exams. There has been increasing diversity in assessment over recent years, with some course leaders opting to try presentations or coursework in their summative portfolios, but the pandemic has forced us to accelerate this process and develop alternatives at pace. Forced to incur the sunk cost of adapting our assessments, it is easy to believe that the post-Covid assessment equilibrium will see exams occupying a much smaller slice of the economics assessment pie.

The symposium sourced input, advice and resources from colleagues and institutions across the UK and internationally. It’s clear from the survey results that many adapted their exams for online delivery, but many others converted to coursework, modified questions or altered planned presentation formats. At some institutions, exams were cancelled entirely for those outside their final year, while others found their entire degree measured on assessment completed in the Covid-19 impacted period. The symposium brought these adaptations and experiences together, producing a substantial knowledge bank well worth exploring before embarking on new assessment design. Practical advice ranging from simple, but important tips such as ‘Google your questions!’ (Stefania Paredes Fuentes), to time-saving matrix MCQs (Carlos Cortinhas) and more radical overhauls to improve participation such as motivation and participation such as multi-stage assessments (Michael McCann).

There is much to learn from these early ventures into adaptable assessment but three elements emerged as the clear focal points of the three-month conversation:

• Format: The majority of our assessments were planned for 1-3 hours in an invigilated examination hall. With this option removed, exams moved online and were completed in a given time period. The time allotted ranged from a few hours to a few days in order to accommodate concerns over time zones, technology failure and the need to secure a suitable environment in which to complete important assessment. Some disliked the idea of exams and replaced their assessment with coursework, finding that short time limits can be substituted with restrictive word counts.
• Question Style: Sitting an assessment in a different environment and over a longer time period meant questions needed to be reworked. Asking students to replicate parts of the lecture notes is something many tried to avoid; it’s unclear what such questions would be assessing in an open-book environment. There was instead a shift towards use and application of the course content. Questions became centred around words such as ‘justify,’ ‘interpret,’ ‘explain’ and ‘use the data’. In doing so, assessments encouraged the use of the skills and abilities we want our students to take with them when they finish their studies and took a step closer towards the authentic assessment style advocated by some.
• Student Motivation: University study should be about more than assessments and grades but it still remains the focus for many. Removing assessments entirely (or swapping summative assessment for formative) reduced engagement, particularly among weaker students. As Kay Sambell explained in the symposium’s synchronous session, assessment in the open-book environment can be used as a carrot to encourage students to make progress in their learning. Moving away from a single, high-stakes final assessment to multiple smaller assessments allows for teaching to inform assessment, which in turn informs our teaching again. It also presents an opportunity to ask questions which students can take longer to contemplate and meaningfully relate to their own experiences.

The new assessment styles are inevitably accompanied by some concerns. Amongst the most discussed were poor academic practice and workload. Taking assessments in an unproctored environment opens up new opportunities for dishonest participation which needs careful consideration if the value of the qualification’s signal is to be maintained. Technology in the form of plagiarism detection provides some assistance in this, but combinations of other assessment types, such as presentations, mini-vivas or multi-stage assessments offered alternative means of assessing the student’s own work and contributions. Workload is more difficult to tackle without institutional change. So often teaching time is distributed according to the number of lectures and classes and not to time taken for developing quality assessment. If it is assessment which drives participation, then this needs equal consideration in the planning of a unit’s teaching.

One final concern which deserves attention is the students’ experience and perception of assessment. It is important we remember what students have done before encountering our own unit’s assessment. If students have been trained to take high-stakes, replication based exams throughout school and into their early years of university education, then a sudden change of direction is likely to cause some severe discomfort regardless of how much we can see the benefit it the new assessment style. There is a need to give students time to adapt and see the advantage in the new way of doing things, however the pandemic may have given them very good reason to do just that. Students finishing high-school have been let down by the high-stakes exam system and were denied the opportunity to prove their capabilities. Perhaps now, more than ever before, they will be open to new styles of assessment, particularly if they are well supported in the conversion and the benefits for their learning and achievement are made clear. With continuing national and global uncertainty, there is good opportunity to do this.

It’s unlikely the exam has disappeared forever, but its role in university economics assessments seem likely to be diminished (whether by choice or necessity) in at least the short run. For a variety of reasons, many indicated a wish to return to the exam format in the future. However, given the sunk cost involved in developing Covid-proof assessment for the forthcoming year and the opportunities this presents in terms of creating better output from students, our new assessments may very well be something which stay with us for some time to come.