Case study 1 – A marking exercise

Part of the Handbook chapter on Assessment and Feedback

Background

As previously discussed in Chapter 2, students may not respond to feedback because they do not understand the assessment criteria. One approach to this problem is to provide more detailed written guidance. However given the difficulty that many students find with the precise meaning of much of the language, this response on its own will probably meet with limited success. In order to complement the written guidance, sessions could be arranged that actively engage the students with the assessment criteria. One such approach is a marking exercise. The rationale behind this type of initiative is that students will gain a better understanding of terms such as ‘analysis’ and ‘evaluation’ when they see examples in the context of a specific economics assessment. Research by Rust et al. (2003) found that these sessions had a statistically significant impact on the achievement of business students in subsequent coursework. The following case outlines how this type of activity was introduced into a mandatory first year skills module entitled ‘Developing Academic and Professional Skills’. This is a 10 credit module on the economics single honours degree programme at Coventry University which runs in the autumn semester/term with approximately 150 students. It operates with a  one-hour lecture per week plus a one-hour workshop per week. There are approximately 25 students in each workshop. The module aims to provide students with a range of skills which will enable them to become effective learners in higher education and enhance their chances of obtaining graduate employment.

The activity:

• Initially the students have a lecture outlining academic writing skills with a focus on the format and conventions of essay writing. At the end of the lecture they are provided with copies of two different student answers to the same essay question. The examples are usually chosen from either the stage one Microeconomics or stage one Macroeconomics module. Where possible they are chosen from the previous year. The ‘clean’ copies are downloaded from ‘Turnitin’ with the students name blanked out.
• The students have to mark the essays before attending the next workshop. They are asked to identify the strengths and weaknesses of each essay and ways in which they could have been improved. When undertaking this task they are expected to make reference to some of the key issues raised in the previous lecture. They are also expected to identify which essay is the best and explain why.
• In the workshop they are put into groups of four or five and given 25 minutes to compare their notes before reporting back to the class on some of the strengths and weaknesses. They also have to come to an agreement on which of the essays they believe to be best and provide some justification for their choice. The reporting back takes about 15 minutes
• The tutor spends the last 10 minutes pulling some of the key points together and gives an indication of the actual mark and feedback provided. The tutor then provides each student with annotated and marked versions of the sample assignments. If time is available there is some discussion of how the tutor’s marked versions compare with the student evaluations.

Evaluation

It is difficult to assess the benefits quantitatively as this requires a control group which is prevented because of ethical concerns. In order to get a more intuitive sense of the effectiveness of the activity, a series of student focus groups were carried out. Comments about assessment criteria included:

‘We are often given a marks scheme but it is written in lecturer’s language.’

‘It is difficult to apply what was said in the assessment criteria as the language is very general and difficult to apply to a particular assignment.’

‘I was unsure of the precise meaning of comments such as – you need to go into more detail with your economics analysis.’

Comments about the marking exercise included:

‘I found reading examples of other students work on similar topics far more useful than being given any assessment criteria or marking sheets.’

‘Different lecturers weighted different aspects of the criteria they provided differently. Without the examples we can’t tell exactly what they want.’

Overall the feedback was very positive with students requesting that similar activities should be introduced into other economics modules on the course.

Alternative ways of running the activity

The marking exercise could be implemented in a number of different ways.

When do you get the students to read and mark the exemplars?

The decision to get the students to mark the work outside of class was taken in this case because the timetabled slot for the workshop was only 50 minutes long. It was thought that this would not provide enough time for the students both to mark a 1500 word piece of work and discuss the comments with their peers. However this does create a problem as there are always some students who attend but have not completed the work. One possibility is to group them together on the basis of those who have and those who have not completed the task. If a longer slot (minimum of 90 minutes) can be timetabled the activity might work more effectively if the marking is completed in class. The students could be given 25 minutes to mark each assignment before discussing their thoughts and reporting back their agreed grades and feedback. A drawback with carrying out the marking in class is that students will have different reading speeds and so will finish at different times making it difficult to start the group discussions. Also some students may struggle to understand the piece of writing in the time allocated in class.

How much guidance do you provide the students with?

The lecture on academic writing skills may not be an effective way of communicating assessment requirements. Student feedback from the focus groups suggested that they had not found it very useful. The comments tended to support the previously discussed point that students often find it difficult to understand assessment standards if they are communicated in a passive environment and outside of the context of a particular piece of writing. A more effective alternative may be to omit the lecture altogether and instead provide the students with copies of the marking sheets and assessment criteria that were used to mark the essays originally. They could then use these to mark the work themselves either before class or in-class. Another option would be to ask the students to mark the work without any guidance. They would have to decide what makes a good assignment and in the process develop their own assessment criteria. This could then be compared with the actual assessment criteria used by the tutor with some discussion around the similarities and differences. This may prove a useful way of overcoming any misconceptions students have about academic writing.

Some reservations about the use of marking activities

Some of the following concerns have been frequently expressed by tutors:

Providing exemplars is an example of spoon feeding!

Some colleagues have argued that providing exemplars is an example of spoon feeding and may contribute towards learned dependence i.e. over reliance on the tutor. There does appear to be some tension between the guidance demanded/expected by students and the onus placed on independent/self-directed learning by tutors. However the transition of students from school to university is probably more difficult than it has ever been. The learning environment they will have experienced at school is increasingly likely to have been one of detailed guidance and high dependency. League tables have generated strong incentives for teachers to ensure their pupils achieve the highest grades possible and a rational response has been to increase the level of coaching. Given this prior experience, departments may need to introduce activities such as the marking exercise in order to help their students become more independent learners. The focus should be on using these methods with the first years with the aim of developing their self-directed learning skills as the course progresses. The ultimate aim is to make the students independent learners by the end of the course and they may require some assistance towards this goal in the early stages.

Using exemplars will increase the risk of plagiarism

There may be worries that students will try to copy the exemplars when completing their own work. This makes the choice of examples very important. They need to be similar enough to current assignments for students to see the relevance, while not so similar that they believe they can simply copy them. It is also important to stress that the purpose of exemplars is to provide a guide and not a model answer. A related concern is that the exemplars may be interpreted in a very prescriptive manner. Rather than an example, they may be read as a blueprint for success. This may encourage the student to stick rigidly to the structure of the good exemplars and stifle originality. Perhaps to overcome this problem it would be helpful to use two very different essays which both received a very high mark.

Providing the exemplars without carrying out the marking activity would save the tutor valuable time and effort

It might be tempting to think that simply providing the students with the samples of work without carrying out the workshop could capture most of the learning gains but at a far lower cost. However evidence from Rust et al. (2003) suggests that being actively involved in the marking activity is an important determinant of the improved student performance. Simply providing examples of work may only capture a small fraction of the potential learning benefits as it remains a fairly passive way of transmitting the assessment information.

Some suggestions on how to make the activity as effective as possible

Make sure you spend time focusing on the more difficult aspects of the assessment criteria

Experience from running the workshops has shown that students are far more confident in judging some aspects of the work than others. They found it relatively straightforward to comment on items such as ‘referencing’, ‘labelling on diagrams’, ‘quality of English’ and ‘presentation’. They will be far less confident applying criteria such as analysis, evaluation and synthesis. It is very important that the discussion in the workshop focuses on these more difficult issues and the tutor is able to highlight sections in the exemplars to help illustrate what is meant by these terms in the context of an actual economics assignment.

Make the exemplars as recent as possible

Experience from running this workshop has shown that the economics students prefer the sample assignments to be as recent as possible. Ideally they would be from the previous academic year. This does mean having to change the examples every year.

Try to introduce the activity in more than one module

It may be optimistic to expect many of the students to appreciate fully the meaning of terms such as ‘analysis’ and ‘evaluation’ after just one marking exercise. The activity will probably need to be repeated across a number of modules before they develop an understanding similar to that of the academic staff. Ideally it would be used in both Microeconomics and Macroeconomics modules in the first year of an undergraduate degree programme.

Back to section 3.3.2 of the Handbook chapter

↑ Top