Assessment and Feedback

Jon Guest, Coventry University

Edited by Rebecca Taylor, The Open University
Published January 2013

Pedagogical topics: 

1. Introduction

1.1 Background

The aim of this handbook is to supplement the existing one currently available on the Economics Network website. This first handbook focused on some of the important fundamentals of writing assessments such as ensuring they are designed to develop and promote the learning outcomes of the module. It also included some general discussion of various types of assessments, for example group work, presentations, multiple choice, essays etc. This handbook concentrates on certain types of assessment in more detail, including some case studies of how they have been introduced into economics modules. In addition, given the poor scores consistently found on feedback in the National Student Survey, a much greater emphasis has been placed on the issue of assessment.

1.2 The importance of assessment and feedback

It is worth repeating a point made in the initial handbook about the influential impact of assessment and feedback on student learning. For the majority of students the assessment on a module determines (a) what topics they study (b) how much time they spend studying and (c) how they go about studying. It tends to be the assessments rather than the teaching which for many students determines whether they take a surface or deep approach to their learning. Educational researchers have also stressed the importance of good feedback. Some influential meta-analyses have found that high-quality feedback is one of the most important determinants of student learning (Hattie at al., 1996; Black and William, 1998; Hattie and Jaeger, 1998).

Given the importance attached to assessment and feedback in the learning process it is somewhat worrying that much of the evidence suggests that the HE sector does not do it very well. In particular the standard of feedback received by students has come in for much criticism. As Rust et al. (2003) comments:

‘And if the literature suggests we are bad at assessment generally, the evidence is that it is in the area of feedback that we are possibly worst of all.’

QAA reviewers have commented regularly on the lack of adequate feedback given to students. The questions on feedback consistently receive the lowest satisfaction scores on the National Student Survey with, for instance, only 59% of respondents agreeing that ‘Feedback on my work has been prompt’ whilst only 57% agreed that ‘Feedback on my work has helped me to clarify things I did not understand’. These scores were significantly lower than those for other sections of the survey including those on teaching and overall satisfaction. For example 89% of respondents agreed that ‘Staff were good at explaining things’ while 84% agreed that ‘Overall I am satisfied with my course’. There is variation between different disciplines and unfortunately evidence from the NSS suggests that economics students are less satisfied than the average across all subjects on the assessment and feedback questions. The average score on the five questions on assessment and feedback was 60.4%, 4% lower than the average across all subjects with the worst performance relating to Question 8 – ‘I have received detailed comments on my coursework’. Only 52% of economics students agreed with the statement – 10% lower than the average across all subjects. Details can be seen in Table 1.

Table 1: National Student Survey Data 2010

Question All subjects (%) Economics (%)
The teaching on my course
 
 
1 – Staff are good at explaining things.
89
88 (-1)
2 – Staff have made the subject interesting.
81
75 (-6)
3 – Staff are enthusiastic about what they are teaching.
85
81 (-4)
4 – The course is intellectually stimulating.
85
86 (1)
Assessment and feedback
 
 
5 – The criteria used in marking have been clear in advance.
70
65 (-5)
6 – Assessment arrangements and marking have been fair.
73
75 (2)
7 – Feedback on my work has been prompt.
59
57 (-2)
8 – I have received detailed comments on my work.
62
52 (-10)
9 – Feedback on my work has helped me clarify things I did not understand.
57
53 (-4)
Overall Satisfaction
 
 
22 – Overall, I am satisfied with the quality of the course.
84
85 (1)

Comments from student focus groups run by staff from the Economics Network in nine departments in the academic year 2010–11 were also not very favourable.

Student comments included:

‘You can get an essay with two completely different marks on like 51 and 84 and they can say the same thing ... we hardly actually get any constructive feedback.’

‘They just give me a mark that doesn’t help me in anyway.’

‘The feedback you get on a test is so bad, often you’ll just get a number and that’s it, or you will have a tick and good.’

This handbook will deal with feedback in some detail.

1.3 Why does our performance appear to be so disappointing?

It is probably a combination of resourcing issues and the use of traditional methods of feedback that are no longer very effective. Increasing student numbers and reduced unit costs have put pressures on the quality of the learning experience for undergraduates. Economies of scale can be found in the classroom by simply increasing the size of lectures, seminars and workshops. Evidence from Table 1 suggests that this has been achieved, whilst on the whole maintaining the quality of the teaching. However, finding economies of scale in traditional methods of assessment and feedback is more difficult. Producing written feedback on double the number of assignments usually means double the amount of work for the tutor. The rational response of many departments has been to:

  • reduce the amount of assessment, in particular formative assessment;
  • reduce the amount and quality of feedback provided on a particular piece of work;
  • make greater use of group assessments. This has also created problems with staff having to find ways of dealing with free riders and methods of allocating marks amongst the individual group members. Students have repeatedly expressed strong concerns about the use of summatively assessed group work.

There may also have been relatively modest innovation in assessment for two major reasons. Firstly staff may tend to overestimate the effectiveness of the feedback that they provide and therefore see no need to change. Survey evidence tends to support this proposition. For example Carless (2006) found that 38.4% of HE tutors thought students were ‘often given detailed feedback which helped them improve their next assignment’, whilst only 10.6% of students responded in the same way. A second reason for the lack of innovation may be because of the perceived risks involved. Tutors may be more willing to try different methods of teaching because if they are unproductive it only affects one class. However if they experiment with different types of assessment which prove to be ineffective, the impact could be much more significant and long lasting.

Even though there has been a reduction in the quantity of feedback and a relative lack of innovation in this area, many economics lecturers still devote a significant amount of time and effort to carefully written detailed comments in the expectation that they will be used. Unfortunately a substantial number of students fail to collect their assignments and a high proportion of those who do seem to ignore the feedback and repeat the same mistakes time after time. This suggests that tutors are spending significant amounts of time on an activity which is producing very limited learning benefits. The process appears to be both time consuming for the tutor and unproductive for the student. Given the educational evidence on the importance of feedback on the learning process this is a very puzzling finding. Why don’t students appear to engage with the feedback traditionally provided by tutors in the form of written comments on summatively assessed work? The next chapter will consider some possible explanations.

2. Why don’t students engage with the feedback provided?

There may be a number of reasons why students tend not to engage with and act upon the feedback provided by tutors at university. This chapter will outline some of these potential explanations.

2.1 Feedback needs to be dialogicnot just monologic

Many tutors perceive feedback as predominately a post-summative assignment event. It is usually provided via written comments and tends to be a passive activity for the students. Some authors have described this one-way process as ‘monologic’. At its core, feedback is a communication process and requires information to be sent and understood. However many educational researchers have concluded that this one-way process using written comments is ineffective.

2.2 It needs to be timely

Students are much more likely to engage with feedback if they receive it while the process of researching and writing the assessment is still fresh in their minds. This is the rationale behind the inclusion of Question 7 in the NSS. Unfortunately, one of the drawbacks of producing carefully crafted and detailed written comments is that it takes time. With the increase in student numbers it could take weeks, maybe even months, to complete the process. This causes problems because by the time the student has received the feedback they have moved on and started studying for subsequent assignments. The comments they do receive no longer seem that relevant to them. The quicker feedback is returned to the students the more likely it is to seem useful. Departments may have to accept a trade-off between the timing and quality of feedback – a tension that exists between scoring well on Question 7 and 9 in the NSS. In terms of the impact on student learning, imperfect/incomplete feedback provided instantly while the assignment is still fresh in the student’s mind may prove more effective than detailed and well crafted feedback made available much later.

2.3 They are distracted by the mark

There is evidence that some students simply look at the mark and ignore the feedback comments (Wojtas, 1998). Receiving marks that are either higher or lower than anticipated can have an emotional impact on students and can cause them to ignore the feedback comments. The mark seems to block the feedback comments from the student’s view! The problem has been made worse by the common practice of posting marks on a VLE. Students no longer have to collect their work in order to see their grades and many don’t.

2.4  They don’t understand the feedback

This may occur because students cannot read the tutor’s handwriting. It may also happen because students do not fully understand the whole assessment method and the standards required. If the feedback relates back to the assessment criteria and students do not fully understand that criteria then they will not understand the comments. As Sadler (1987) notes:

‘One of the conditions necessary for the intelligent use of feedback is that the learners know not only their own levels of performance but also the level or standard aspired to or expected.’

There has been a real drive in the past decade to improve the transparency and consistency of the assessment process so that students know and fully understand the basis on which their work will be marked. For example, Question 5 on the NSS asks students whether ‘The criteria used in marking have been made clear in advance’. Unfortunately, as shown in Table 1, the number of economics students who agreed with this statement is below the average across all subjects. The response of most institutions has been to make it standard practice for tutors to provide plenty of written guidance prior to deadline dates. This is normally expected to include the learning outcomes being assessed, the assessment criteria and marking guidelines. The QAA benchmarking document in economics suggests that when assessing students’ work some of the following criteria may be adopted:

  • How far have students focused on questions asked and/or identified key problems?
  • How well have students chosen the arguments, the relevant theory or model, to relate to the area specified or question asked?
  • How good is the quality of explanation?
  • How well have students demonstrated consistency, coherence and purposeful analysis?
  • How successfully have students used evidence?
  • How well have students collected, processed, analysed and interpreted relevant data?
  • How deep is the extent of critical evaluation?
  • How well have students demonstrated knowledge of relevant literature?

Do students fully understand what is meant by phrases such as ‘purposeful analysis’ and ‘critical evaluation’ when they see them in written criteria or guidelines? It might prove very difficult to communicate a shared understanding of these terms in a form of words that students would fully understand. If the feedback provided refers back to the assessment criteria (i.e. ‘not enough critical evaluation’ or ‘not enough analysis’) the students may not be able to engage with the comments because they simply do not understand what they mean. Students may also be unaware of the extent to which they do not fully understand the comments.

Difficulties with the understanding of assessment guidelines can be exacerbated by lecturers writing the comments predominantly for the benefit of external examiners or QAA assessors rather than the student. There is a danger that comments may read as if they are part of a formal piece of work for the academic community rather than for the average undergraduate!

Perhaps the only way to effectively communicate what we mean by terms such as ‘analysis’ and ‘critical evaluation’ would be to provide examples in the context of an actual piece of work. However it is important that the information is communicated in an active rather than a passive manner. A case study in the next chapter will outline some activities that could be used to engage the students with the assessment criteria.

2.5 They don’t know how to respond to the feedback

Sometimes a lecturer might provide a student with excellent feedback that is fully understood but is still not acted upon. We tend to assume that when they arrive at university students are ready to receive feedback in the manner provided and that they understand how to act upon it. However they might not fully appreciate how important feedback is in the learning process and hence the impact that it can have on their academic achievement. They may sometimes perceive that feedback is only relevant to the assignment they have just written and fail to see its relevance and usefulness for future assignments, i.e. the potential for ‘feedforward’. In some circumstances they may see the potential ‘feedforward’ but do not know how to engage with it. They may appreciate and fully understand their weaknesses but do not know how to improve.

The next chapter will outline and discuss a number of different assessment activities that could be used to overcome some of the problems identified in this chapter. These include:

  • providing feedback quickly;
  • overcoming the problems caused by marks;
  • alternatives to written feedback;
  • helping students to understand assessment criteria;
  • helping students to appreciate the usefulness of feedback.

The chapter includes some detailed case studies based on the experience of economics lecturers who have introduced some innovative assessment activities into their teaching. The challenge is to see if it is possible to introduce and maintain high quality methods of assessment and feedback given large class sizes and the heterogeneity of the student population.

3. Some innovative assessment and feedback activities

This chapter will outline some assessment and feedback activities that could be used to help overcome some of the problems identified in Chapter 2. It will include two detailed case studies of innovations that were introduced onto an economics degree programme.

3.1 Delivering quick feedback in modules with large student cohorts

Some possible methods that could be used to provide prompt feedback include:

Providing feedback after reading a sample of the assignments

Instead of reading all the assignments before providing any feedback, the lecturer could read through a sample of the assignments and report back on some of the common weaknesses as soon as possible. The weaknesses could be communicated to the students in a number of ways:

  • verbally in class time – maybe at the beginning of the next lecture. (This will be more effective the closer the lecture is to the hand-in date for the coursework.)
  • written form – i.e. via a written announcement on the VLE.

As previously discussed, imperfect feedback received quickly can often be more effective than detailed feedback provided much later.

Providing feedback before reading the assignments!

Ideally the hand-in date and time for the coursework should be at the start of a lecture. If the work has to be handed in to a central point then the hand-in date could be set on the same day as the lecture. If possible, the hand-in time should be set an hour before the lecture begins. Without the time constraint there is a danger that a number of students might miss the lecture because they are busy completing the coursework! At the start of class the students should be provided with a coloured sheet with some numbered points that have been previously prepared. These numbered points should include a brief explanation of the likely mistakes you anticipate the students will have made on their assignments and/or some of the key features or characteristics of a good answer. With only limited teaching experience, lecturers can soon predict the sort of mistakes the students will make. You could provide examples of useful diagrams or mathematics you would expect to see in an excellent answer. Typically about 15 numbered points have been included when this approach has been used on a level one microeconomics module. Some examples are shown below:

  1. The essay has no introduction! Instead it launches straight into the main body of the answer.
  2. Relevant diagrams have been included but not explained. Diagrams should form a central part of the explanation and application of relevant microeconomic theory to the question.
  3. Relevant diagrams have been included with some explanation but these were not developed in enough detail or applied to the question.
  4. General explanation of elasticity is included but not applied to the question, i.e. implications of inelastic demand and supply for the size of price changes in response to changes in demand/supply.
  5. A diagram should have been included to illustrate the impact of a per unit subsidy – see the textbook page 95.
  6. A diagram has been included to illustrate the impact of a per unit subsidy but the impact on producer/consumer surplus and economic efficiency has not been illustrated or explained in enough detail – see the textbook page 110.
  7. The essay does not have a conclusion and comes to an abrupt end. A conclusion should pull together the key ideas/arguments from the main body of the essay.
  8. Poor English – you should seek help and support from the Centre for Academic Writing.

Do not simply start the lecture by explaining each point on the sheet. Instead, let the students read through the list for five minutes and anticipate that the room will go very quiet as they realise the sort of issues they should have included and the mistakes they have made! After five minutes of quiet reading time, explain some of the points on the handout to the students. The amount of time and number of points you want to explain is up to the individual but it is probably best not to spend more than 10–15 minutes on this activity. When you take the work away and mark it you can include statements such as ‘please see the following points on the handout: 3, 7, and 8’ rather than having to write out the same comment over and over again on the assignment. The written comments that you do make can be more personalised to each piece of work.

One potential issue with this method of feedback is how you deal with students who hand in work late with an agreed extension. For this reason, when used in the microeconomics module the session had to be delayed for one week as a couple of students had an agreed extension. However when surveyed at the end of the academic year, 85% of the economics students still found this to be the most effective piece of feedback that they had received.

3.2 Overcoming the distraction caused by students seeing their marks

One possible method is to not release marks on the VLE until the students have collected their coursework from the seminar/workshop. The lecturer would take a marks list to the workshop and return the coursework without a mark. The students are then given some time to try and work out their mark from the feedback. They could also be encouraged to compare the feedback on their work with that of their peers in order to help them with the process. A grade incentive could also be used to encourage the students to take the activity seriously. For example they could be awarded the higher of the two marks if they self-assessed within 5% of the lecturer’s assessment.

3.3 Some methods/activities to help students engage with and understand the assessment criteria

What methods could be used to encourage students to engage with assessment criteria both before they start studying for the coursework and while they are researching/writing the coursework? Two possible activities will be outlined in some detail.

The first activity is a marking exercise which could help students to engage with the assessment criteria before they start their assignments. A detailed case study is included which outlines and evaluates how the exercise was implemented onto an economics degree programme. The second activity is the use of self-assessment sheets. These could be used as a way of encouraging students to engage with the assessment criteria while actually writing their assignments. Some discussion of the different ways of introducing these sheets is included as well as some discussion of the potential drawbacks and how these might be avoided.

3.3.1 Encouraging students to engage with the assessment criteria before they start studying for a piece of coursework

Case study 1 – A marking exercise

Background

As previously discussed in Chapter 2, students may not respond to feedback because they do not understand the assessment criteria. One approach to this problem is to provide more detailed written guidance. However given the difficulty that many students find with the precise meaning of much of the language, this response on its own will probably meet with limited success. In order to complement the written guidance, sessions could be arranged that actively engage the students with the assessment criteria. One such approach is a marking exercise. The rationale behind this type of initiative is that students will gain a better understanding of terms such as ‘analysis’ and ‘evaluation’ when they see examples in the context of a specific economics assessment. Research by Rust et al. (2003) found that these sessions had a statistically significant impact on the achievement of business students in subsequent coursework. The following case outlines how this type of activity was introduced into a mandatory first year skills module entitled ‘Developing Academic and Professional Skills’. This is a 10 credit module on the economics single honours degree programme at Coventry University which runs in the autumn semester/term with approximately 150 students. It operates with a  one-hour lecture per week plus a one-hour workshop per week. There are approximately 25 students in each workshop. The module aims to provide students with a range of skills which will enable them to become effective learners in higher education and enhance their chances of obtaining graduate employment.

The activity:

  • Initially the students have a lecture outlining academic writing skills with a focus on the format and conventions of essay writing. At the end of the lecture they are provided with copies of two different student answers to the same essay question. The examples are usually chosen from either the stage one Microeconomics or stage one Macroeconomics module. Where possible they are chosen from the previous year. The ‘clean’ copies are downloaded from ‘Turnitin’ with the students name blanked out.
  • The students have to mark the essays before attending the next workshop. They are asked to identify the strengths and weaknesses of each essay and ways in which they could have been improved. When undertaking this task they are expected to make reference to some of the key issues raised in the previous lecture. They are also expected to identify which essay is the best and explain why.
  • In the workshop they are put into groups of four or five and given 25 minutes to compare their notes before reporting back to the class on some of the strengths and weaknesses. They also have to come to an agreement on which of the essays they believe to be best and provide some justification for their choice. The reporting back takes about 15 minutes
  • The tutor spends the last 10 minutes pulling some of the key points together and gives an indication of the actual mark and feedback provided. The tutor then provides each student with annotated and marked versions of the sample assignments. If time is available there is some discussion of how the tutor’s marked versions compare with the student evaluations.

Evaluation

It is difficult to assess the benefits quantitatively as this requires a control group which is prevented because of ethical concerns. In order to get a more intuitive sense of the effectiveness of the activity, a series of student focus groups were carried out. Comments about assessment criteria included:

‘We are often given a marks scheme but it is written in lecturer’s language.’

‘It is difficult to apply what was said in the assessment criteria as the language is very general and difficult to apply to a particular assignment.’

‘I was unsure of the precise meaning of comments such as – you need to go into more detail with your economics analysis.’

Comments about the marking exercise included:

‘I found reading examples of other students work on similar topics far more useful than being given any assessment criteria or marking sheets.’

‘Different lecturers weighted different aspects of the criteria they provided differently. Without the examples we can’t tell exactly what they want.’

Overall the feedback was very positive with students requesting that similar activities should be introduced into other economics modules on the course.

Alternative ways of running the activity

The marking exercise could be implemented in a number of different ways.

When do you get the students to read and mark the exemplars?

The decision to get the students to mark the work outside of class was taken in this case because the timetabled slot for the workshop was only 50 minutes long. It was thought that this would not provide enough time for the students both to mark a 1500 word piece of work and discuss the comments with their peers. However this does create a problem as there are always some students who attend but have not completed the work. One possibility is to group them together on the basis of those who have and those who have not completed the task. If a longer slot (minimum of 90 minutes) can be timetabled the activity might work more effectively if the marking is completed in class. The students could be given 25 minutes to mark each assignment before discussing their thoughts and reporting back their agreed grades and feedback. A drawback with carrying out the marking in class is that students will have different reading speeds and so will finish at different times making it difficult to start the group discussions. Also some students may struggle to understand the piece of writing in the time allocated in class.

How much guidance do you provide the students with?

The lecture on academic writing skills may not be an effective way of communicating assessment requirements. Student feedback from the focus groups suggested that they had not found it very useful. The comments tended to support the previously discussed point that students often find it difficult to understand assessment standards if they are communicated in a passive environment and outside of the context of a particular piece of writing. A more effective alternative may be to omit the lecture altogether and instead provide the students with copies of the marking sheets and assessment criteria that were used to mark the essays originally. They could then use these to mark the work themselves either before class or in-class. Another option would be to ask the students to mark the work without any guidance. They would have to decide what makes a good assignment and in the process develop their own assessment criteria. This could then be compared with the actual assessment criteria used by the tutor with some discussion around the similarities and differences. This may prove a useful way of overcoming any misconceptions students have about academic writing.

Some reservations about the use of marking activities

Some of the following concerns have been frequently expressed by tutors:

Providing exemplars is an example of spoon feeding!

Some colleagues have argued that providing exemplars is an example of spoon feeding and may contribute towards learned dependence i.e. over reliance on the tutor. There does appear to be some tension between the guidance demanded/expected by students and the onus placed on independent/self-directed learning by tutors. However the transition of students from school to university is probably more difficult than it has ever been. The learning environment they will have experienced at school is increasingly likely to have been one of detailed guidance and high dependency. League tables have generated strong incentives for teachers to ensure their pupils achieve the highest grades possible and a rational response has been to increase the level of coaching. Given this prior experience, departments may need to introduce activities such as the marking exercise in order to help their students become more independent learners. The focus should be on using these methods with the first years with the aim of developing their self-directed learning skills as the course progresses. The ultimate aim is to make the students independent learners by the end of the course and they may require some assistance towards this goal in the early stages.

Using exemplars will increase the risk of plagiarism

There may be worries that students will try to copy the exemplars when completing their own work. This makes the choice of examples very important. They need to be similar enough to current assignments for students to see the relevance, while not so similar that they believe they can simply copy them. It is also important to stress that the purpose of exemplars is to provide a guide and not a model answer. A related concern is that the exemplars may be interpreted in a very prescriptive manner. Rather than an example, they may be read as a blueprint for success. This may encourage the student to stick rigidly to the structure of the good exemplars and stifle originality. Perhaps to overcome this problem it would be helpful to use two very different essays which both received a very high mark.

Providing the exemplars without carrying out the marking activity would save the tutor valuable time and effort

It might be tempting to think that simply providing the students with the samples of work without carrying out the workshop could capture most of the learning gains but at a far lower cost. However evidence from Rust et al. (2003) suggests that being actively involved in the marking activity is an important determinant of the improved student performance. Simply providing examples of work may only capture a small fraction of the potential learning benefits as it remains a fairly passive way of transmitting the assessment information.

Some suggestions on how to make the activity as effective as possible:

Make sure you spend time focusing on the more difficult aspects of the assessment criteria

Experience from running the workshops has shown that students are far more confident in judging some aspects of the work than others. They found it relatively straightforward to comment on items such as ‘referencing’, ‘labelling on diagrams’, ‘quality of English’ and ‘presentation’. They will be far less confident applying criteria such as analysis, evaluation and synthesis. It is very important that the discussion in the workshop focuses on these more difficult issues and the tutor is able to highlight sections in the exemplars to help illustrate what is meant by these terms in the context of an actual economics assignment.

Make the exemplars as recent as possible

Experience from running this workshop has shown that the economics students prefer the sample assignments to be as recent as possible. Ideally they would be from the previous academic year. This does mean having to change the examples every year.

Try to introduce the activity in more than one module

It may be optimistic to expect many of the students to appreciate fully the meaning of terms such as ‘analysis’ and ‘evaluation’ after just one marking exercise. The activity will probably need to be repeated across a number of modules before they develop an understanding similar to that of the academic staff. Ideally it would be used in both Microeconomics and Macroeconomics modules in the first year of an undergraduate degree programme.

Top tip

Make sure you focus on examples of work which demonstrate and illustrate the parts of the assessment criteria that students find the most difficult to understand, e.g. analysis, evaluation and synthesis. Try to make the exemplars as recent as possible.

3.3.2 Encouraging students to engage with the assessment criteria while completing their coursework

Self-assessment sheets could be used to develop the students’ ability to evaluate their work during the research and write-up stage. The forms can provide a structured framework to help them reflect and critically evaluate their own assignments. Generic versions are available from various websites (for example) and could be easily adapted by tutors for use with specific coursework. One option is to include some simple yes or no questions, for example ‘Please indicate whether you have checked the following:

  • The title of the piece of work i.e. is it correct?
  • Referencing and bibliography – is it in the correct format?
  • Labelling on any diagrams – is it complete and accurate?’

The form could ask the students to assess their own performance against a number of criteria. For example, they might be asked to rate their own work as excellent, good, adequate or poor against the following:

  • quality of the introduction;
  • structure and logical development of the content;
  • relevance of the material to the question;
  • understanding and application of relevant economic theory;
  • depth of analysis;
  • quality of the conclusion.

Some questions could be much more open ended with space left for the student to make some comments, for example ‘Please list what you feel are the strengths of the work’; ‘Which aspects of the work did you find the most difficult or are most dissatisfied with?’ Another alternative would be simply to ask the students to assess their work using the same marking sheet as the tutor. Two ranking lists could be included against each of the criteria, i.e. one for the student to complete and one for the lecturer to complete. An example is shown below.

Please rate your own work using the following scale:

1 = poor           2 = adequate     3 = good          4 = excellent

 
Understanding and application of relevant economic theory
 
Your rating
1       2       3       4
 
1       2       3       4
Tutor’s rating

Whatever version of the form is adopted, once completed, the students should attach a copy to their coursework when it is submitted.

Some reservations about the use of self-assessment sheets:

Will the self-assessment reflect the students true beliefs?

Rather than honest self-evaluation students may behave strategically if they believe that their rankings will have an influence on the tutor. For example, they may think that ranking their own work as poor will increase the likelihood of receiving a low mark from the tutor. It may be more effective to have two separate forms – one for the feedback provided by the tutor and one for the self-assessment sheet. It might also be useful to ask the students to attach the self-assessment sheet at the end of the assignment and the marking feedback sheet at the front of the assignment. The lecturer could explain that they will only look at the self-assessment sheet after they have filled in the marking sheet and awarded a final mark.

Will the students complete the forms?

Evidence from their use in three Geography modules found that completion rates varied between 28% and 75%. You could make submission of the self-assessment sheets part of the assessment criteria and deduct marks if they fail to attach them to the coursework. Alternatively you might explain that you will only release marks once the forms are submitted. However this may create some extra administrative costs for the tutor.

Will the students engage with the process in a meaningful way?

Studies have found that a majority of students admit to filling out the form at the last minute. They also acknowledged that they tended to simply tick the boxes without giving much thought to the self-evaluation process. Feedback from their use on Geography modules suggested that when filling out the forms a number of students realised the potential benefit but printed them out so late that there was no time to change their coursework. Those that had used them more than a few days before the deadline had found it useful in helping them to focus on the precise requirements for the coursework. They had adapted their coursework in response to this structured reflection process. This suggests that it is very important that the potential benefits of self-assessment forms are explained before they are used. However students may simply have to experience the forms and the regret of missing the potential benefits before they will engage with the process on subsequent assignments. It is strongly advised therefore that they are used on a number of modules and especially in the first year of the course. The form could be adapted to the assessment requirements of each particular module.

Top tip

If possible use the self-assessment sheets across a range of modules as students do not seem to fully appreciate the potential benefits until they have failed to take advantage of the sheet on at least one occasion.

3.4 Encouraging students to engage with feedback

The use of exemplars and self-assessment sheets are activities which are designed to help the students engage with and understand the assessment criteria. In the process they should become better prepared to receive feedback on their work. However there is still no guarantee that they will engage with and use the feedback. As previously discussed they may not fully appreciate the potential benefits to their learning. Perhaps there is an element of hyperbolic discounting: they keep meaning to read the feedback the next day. Unfortunately when the next day arrives they delay it again! This process continues with the student never looking at the feedback. Perhaps some assessment activities could be introduced which magnify the potential benefits and incentives of using the feedback. This could be within and across assignments. Also in some instances the feedback may have been read and understood and its full value appreciated. However the student still fails to improve because they simply do not know how to respond to the feedback. The following section will outline and assess some potential methods of dealing with some of these issues. They include:

  • The use of ‘draft assignments’ to increase the incentives to use feedback within an assignment. A detailed case study is included which outlines and evaluates the introduction of a peer review process of draft assignments on an economics programme.
  • The use of ‘pro formas’ and ‘assessments’ to increase the incentives to use feedback across different assignments.
  • The use of ‘peer group discussions’ to help students consider the best way of responding to the feedback.

3.4.1 Increasing the incentive for students to engage with feedback within an assignment

Draft assignments

The student submits a draft version of the assignment a few weeks before the deadline for the final copy. The draft could simply be an outline/plan or a draft of the final copy. The lecturer provides feedback on the draft which can then be used by the student to improve the final copy before submission. The rationale behind this type of assessment design is to magnify the incentives to the students of using the feedback provided by the tutor. The immediate usefulness and ‘feedforward’ of the comments should be more apparent. Hopefully the experience will have a lasting effect. Having seen the usefulness of feedback through the draft assignment it should help them to appreciate the potential usefulness and value of feedback provided on the final copy of subsequent assignments.

Some reservations about the use of draft assignments:

Some students may try and game the system so others believe it is unfair

Students may game the system as they believe that they can effectively get the tutor to write the work for them. This problem could be made worse if multiple drafts were permitted. This fear was expressed very powerfully by an economics student at a focus group:

‘A student could put a small amount of effort into an initial draft that was worth about 40%. They would then receive guidance from the lecturer about how to make this piece of work worth 70%. I could work really hard on my draft copy and produce a piece of work worth 65% and get feedback on how to make it worth 70%. This is not fair because the lecturer is giving the lazy student a short cut to the high mark.’

The comment helps to illustrate an important point about the type of feedback provided on draft assignments. It should focus on how the student can improve their own work rather than feedback that directly improves the work. There is often a fine line between these two objectives.

Impact on the workload of the tutors

Tutors may resist this type of assessment because of the potential increase in marking loads. However this could be overcome to some extent by only providing a mark on the final piece of work or very brief feedback. Also staff may worry that they will get more cases of students complaining on the grounds that they have carried out all the suggested improvements and have still got a relatively low mark. Another innovative way of dealing with the workload issue is to get the students to peer assess and provide feedback on the draft copies.

Top tip

Care needs to be taken with the way feedback is provided on drafts. It needs to focus on the way the student can improve the work rather than being given feedback that directly improves the work.

Case study 2 – Using peer review to provide feedback on draft assignments

Background

This activity is used in the skills module outlined in the first detailed case study. Having completed the exemplar marking activity, the students have to write a 1000-word essay on an economist of their choice as part of the summative assessment. They are told that the essay must include an introduction, conclusion and some personal information about the economist. However it is stressed that the main part of the essay should include a detailed discussion and assessment of the economist’s contribution to the development of the subject. The innovative part of the assessment is the peer review process which is outlined below.

The activity

  • Two weeks before the essay deadline students are instructed to bring a 1000-word draft version of the final copy of their assignments to a workshop. This draft version must have the student’s ID clearly written at the top of the paper but the student’s name must not be visible on the assignment to ensure anonymous marking.
  • It is clearly stated in the assignment guidelines that if the student does not bring a copy of their work that they will be unable to take part in the peer review process. This has implications if the peer review and feedback is part of the summative assessment.
  • The workshop tutor collects the draft essays from the students as they enter the classroom and then randomly redistributes them.
  • Each student is given 35–40 minutes to peer mark the piece of work they have been given. In order to guide them through the process they are given a feedback template which is a simplified version of the assessment criteria. The template has a number of headings with space left below each one so that the students are able to write a number of comments. They are told to write a minimum of three comments under each of the following headings:
    • Introduction – Is there an introduction or does the essay launch straight into the main body of the essay? If an introduction is included to what extent does it outline how the question/topic will be answered?
    • Organisation – Has the material in the essay been structured into paragraphs in a way which allows for a logical development of ideas and arguments? Does it ‘flow’ or is it difficult to follow?
    • Relevance – To what extent is the material provided relevant and applied to the question/topic?
    • Depth of analysis – Have all arguments/points been fully developed? Has the author adopted a critical stance or is it purely descriptive?
    • Language and grammar – Is it written in an appropriate academic style? Is the meaning always clear or is it sometimes difficult to follow? Are the grammar, spelling and punctuation correct?
  • The tutor moves around the room making sure that the students are engaging with the activity and answering any questions. Most of the questions will be about the ‘depth of analysis’ criteria.
  • At the end of the session the tutor collects the assignments with the peer feedback sheets and immediately staples them together. The tutor then simply calls out the ID of the student and they collect their work with the stapled peer feedback.
  • When the students submit the final copy of their coursework they have to explain how they used the peer feedback in order to amend/improve the final version. If they chose to ignore the peer feedback provided they must explain why.

Some reservations about the use of peer review:

Concerns about the use of peer review have been expressed by both tutors and students.

Concerns expressed by tutors – will the students take the process seriously?

A major issue with this type of activity is getting all the students to engage with the process in a meaningful way. Some initial attempts were made in the skills module to assess the quality of the feedback provided, i.e. by grading it and making it worth 10% of the final mark. However this proved difficult for a number of reasons. Deciding on criteria by which the feedback would be marked proved to be very problematic. The students were asked to write a minimum of three comments against each point. This resulted in them focusing on the quantity rather than the quality of the feedback. As one participant in the student focus group stated:

‘I wrote things just for the sake of it because it was being assessed.’

The marking added to the lecturers’ workload and delayed the time it took for the students to receive the feedback. Tutors had to collect the sheets at the end of the class and mark them before they could be returned to the student. It was judged by the tutors involved with the delivery of the module that the costs of assessing the feedback had outweighed the benefits.

Concerns expressed by students – the quality of the feedback and plagiarism

There were a number of comments made in the student focus groups about the quality of the feedback provided by their peers and the worry that their work would be copied. Some students admitted that they would even try and game the system. Comments included:

‘I am worried that other students are not qualified enough to mark my work.’

‘I expect to receive feedback from an expert – not a novice.’

‘Because I would not be confident in the quality of the feedback received from other students, I would be less likely to act upon it.’

‘It would just give the lazy students a chance to copy my work.’

‘I would seriously consider leaving out some of the better bits from the version of my coursework that I submitted for peer review so that it could not be copied.’

One potential solution to the problem suggested by the students themselves would be to get the students from the year above to carry out the peer reviewing. It was felt that they would have more experience and expertise and also would not be able to plagiarise or steal good ideas. In addition when asked about their expectations or experience of this type of activity the students tended to focus on the benefits/weaknesses of receiving the feedback. They tended not to think about the potential benefits from the act of reviewing somebody else’s work. Perhaps some of these benefits need to be made clearer. For example the following advantages of peer reviewing could be stressed:

  • It is valuable because it may reveal issues you might want to consider in your own work.
  • It will develop your own critical thinking and help you to become a better writer yourself.
  • It will help you to better understand the assessment criteria used by the tutor to mark your work.

Top tip

Expect students to have strong reservations about peer review. It is very important that the potential benefits are clearly explained before they take part in the activity. In particular spend some time focusing on the potential benefits from reviewing somebody else’s work.

Alternative ways of organising the peer review process: anonymity vs. discussion/ dialogue

Some students may find the reviewing process both difficult and stressful. They may feel uneasy about making critical comments if their identity is known by the reviewee. The whole process could be made less daunting by making it anonymous. Allocating the papers by student ID does create a certain level of secrecy in the activity previously outlined. However there is still a possibility that each student may become aware of exactly who else in the room is peer reviewing their work. The use of technology could be used to increase the level of anonymity by allowing the activity to take place outside the classroom. The Centre for Academic Writing at Coventry tried to develop software to do this. More recently a ‘Learning and Teaching Development Project’ has been funded by the Economics Network which evaluated the use of an online peer review system on an economics module. The software used in the project was the peer review function embedded within Turnitin which is used by many departments. Setting up peer review within Turnitin is a fairly straightforward three-step process.

  • Step 1 – Assignment: the tutor has to determine which assignment will have the peer review.
  • Step 2 – Distribution: the tutor has to decide how many assignments each student will peer review and how they will be allocated. Turnitin enables the tutor to choose between three allocation systems:
    • automatically and randomly allocated by the ‘Turnitin’ system;
    • manually selected by the instructor;
    • self-selected by the student.
    A combination of the different methods can be chosen.
  • Step 3 – Questions: the tutor has to set the criteria/questions that the students will use to carry out the peer review.

For more details and evaluation see the final project report on the following link http://www.economicsnetwork.ac.uk/projects/mini/chew_international

The drawback of creating anonymity is that it reduces the opportunities for students to compare and discuss the peer review with colleagues. This is something that a number of them said they would find very useful. They also argued that they would have more confidence in the feedback and were more likely to respond to the comments if they were the agreed outcome of joint discussions. An alternative way of running peer review which encourages dialogue is outlined in the following section.

  • The students are asked to bring at least two copies of their assignment.
  • The tutor places them in non-friendship groups of three or four. Each student is asked to read and review the assignments of two other students in the group.
  • The students are given 30 minutes to mark each assignment.
  • The two peer reviewers for each assignment are instructed to spend 15 minutes comparing their thoughts and ideas before producing an agreed set of comments.
  • The lecturer should move from group to group during the activity and check the feedback and marking of one of the assignments in each group to make sure they are appropriate.
  • At the end of the session the student receives the agreed feedback on their work and must explain how they have used this feedback in their final submission.

This activity would require a two-hour workshop and may get logistically complicated as the tutor would have to make sure that two students are looking at the same piece of work. This is more involved than in the exemplar activity where all the students are looking at the same pieces of work.

3.4.2 Increasing the incentive for students to utilise feedback across different assignments

The draft submission clearly increases the incentives for students to engage with feedback as it can be immediately used to improve the final submission of that piece of work. The process happens within an assignment. Ultimately, the aim is to get students to appreciate how feedback can be used across different assignments. We want them to become fully aware of how they can use feedback in one assignment to help them improve their subsequent assignments. The following methods/activities magnify the incentives for the student to use feedback across different assignments.

‘How have you used feedback’ pro forma

As part of the submission for a piece of coursework students are asked to complete a form that asks them what feedback they have received in previous assignments. They are also asked to explain how this feedback has been used to help improve the current assignment. The process could be taken one stage further by setting a whole assignment on the use of feedback. One method that has been used on an economics module is outlined below.

An assessment based on feedback

The students are asked to write a 1000 word reflective essay with the following title ‘Identify your strengths, weaknesses, priorities and actions taken/actions required to close deficiencies with regard to your academic skills’. When answering the question students are asked to focus on the marks and feedback received. Each strength and weakness identified has to be evidenced with reference to a particular feedback comment. They are instructed to attach the feedback sheets in an appendix to the essay. They are also asked to complete a learning style questionnaire e.g. Honey and Mumford. Having identified their dominant learning style the student also has to assess the associated strengths and weaknesses of that learning style.

Alternative ways of running this activity

The timing of this type of activity/assessment can make it difficult for the students to collect enough evidence. If the assessment has to be completed by the end of the first term/semester then many students will have only received limited feedback on their work.

Perhaps the ideal time for this type of activity would be the beginning of the second year. Students could be asked to collect all of their first-year work and identify any key strengths or weaknesses. A really innovative approach would be to ask them to include feedback from their examination papers if that is possible. 

3.4.3 Encouraging students to think about how they should respond to feedback

This final assessment activity uses peer discussion to help students think about the most effective ways of responding to feedback. The activity works as follows:

  • Students are asked to bring copies of their coursework with feedback to a workshop.
  • They are placed in small groups and asked to explain/discuss what feedback they have received. They should also read the feedback on other students work.
  • They are allocated to small groups and asked to discuss what they believe the feedback means on both their own work and that of their peers. They are told to identify their own and the other students’ key strengths and weaknesses. They should also discuss how the feedback could be used to help improve performance in future assignments.

A key part of the discussion could be based around how the student intends to work on improving any weaknesses. The student could be asked to develop an action plan which outlines how they will respond. The appropriate action will depend on the nature of the weakness identified. For example one prompt would be to ask if the feedback refers to any weaknesses or gaps in their understanding and application of economics. If this is the case they could be asked what they think is the most appropriate action. Are they simply going to look at the same textbook/material again or are they going to try and find alternative sources that explain the same principles but maybe in a way they can better understand? Have they thought about making an appointment with the lecturer to discuss the issue or have they considered discussing it with friends on the course. Another prompt would be to ask if some of the weaknesses are more generic in nature and relate to study skills. For example if the comments specify that academic writing skills are poor what could the student do about it? Where could they go for help? Are there any student support centres in the university where they can get help?

4. Summary

This chapter has outlined a number of potential activities that could be used in order to overcome some of the limitations with more traditional methods of assessment and feedback. Two detailed cased studies have been included which outline and evaluate the introduction of two of these activities onto an economics degree programme. In each case, potential limitations have been discussed and alternative ways of running the exercises have been considered. Many tutors may argue that activities such as using exemplars and draft assignments are examples of spoon feeding and may exacerbate problems of learned dependence. However given their prior learning experience we should not expect students to arrive at university with a full understanding and appreciation of the value of feedback. Instead course teams need to consider how a range of assessment and feedback activities can be structured in an appropriate manner as students progress through different levels of a course. In particular an effective assessment strategy needs to be designed to help students manage the transition from a high-dependency learning environment to one of independent and self-directed learning.

References

Black, P. and William, D. (1998) ‘Assessment and classroom learning’, Assessment in Education: Principles, Policy and Practice, vol.5, no. 1, pp.7–74. DOI: 10.1080/0969595980050102

Carless, D. (2006) ‘Different perceptions in the feedback process’, Studies in Higher Education, vol. 31, no.2, pp.219–233. DOI: 10.1080/03075070600572132

Hattie, J., Biggs, J. and Purdie, N. (1996) ‘Effects of learning skills intervention on student learning: a meta-analysis’, Review of Educational Research, vol. 66, no. 2, pp.99–136. DOI: 10.3102/00346543066002099

Hattie, J. and Jaeger, R. (1998) ‘Assessment and classroom learning: a deductive approach’, Assessment in Education, vol. 5, no. 1, pp. 111–122. DOI: 10.1080/0969595980050107

Rust, C., Price, M. and O’Donovan, B. (2003) ‘Improving students’ learning by developing their understanding of assessment criteria and processes’, Assessment and Evaluation in Higher Education, vol. 28, no.2, pp. 147–164. DOI: 10.1080/02602930301671

Sadler, R. (1987) ‘Specifying and promulgating achievement standards’, Oxford Review of Education vol. 13, no. 2, DOI: 10.1080/0305498870130207

Sadler, R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science, vol. 18, pp. 119–144. DOI: 10.1007/BF00117714

Sadler, R. (1998) ‘Formative assessment: revisiting the territory’, Assessment in Higher Education, vol. 5, no.1, pp. 77–84. DOI: 10.1080/0969595980050104

Wojtas, O. (25 September 1998). ‘Feedback? No, just give us the answers’, Times Higher Education Supplement.