Key Contacts Conference 2010

Notes from Wednesday 8th September 2010, University of Bristol

1. Student Feedback from Module Evaluations

Professor John Sloman led this session - the group examined module evaluation forms used in various economics departments. John's Powerpoint can be downloaded here. Conclusions included:

• Mid-module questionnaires can help deal with problems during the module
• The problem of evaluating a teacher who isn’t responsible for the syllabus
• What are the objectives of these forms? If for example, the module and the assessment is easy and fun receives high marks, and a module which pushes students is less popular but ultimately beneficial scores low.
• By good question design you can factor this issue out
• On their own, without context, the evaluations aren’t meaningful
• Filling in online is more efficient and can receive high response rates
• Incentive mechanisms to complete and take seriously evaluations important
• Use of results varies across departments (e.g. some staff and some students can access every individual and course results)
• Some evaluation forms didn’t ask for responses about assessment and feedback which should be crucial to the teaching process
• Some have tried to collect feedback after a module has completely finished, including exams – reflection some time after could be very useful (e.g. through Personal Development Planning)
• At the beginning of a module get students to reflect on what they want to get out of it, so can draw on this throughout to help manage expectations (then they can then compare this at the end)
• A huge variety in what was asked and how it was asked
• A huge variety in how evaluations were then used
• Should be linked to probationary process for new lecturers – though looking at evaluation forms on their own wouldn’t give a fair or adequate picture
• Open questions were most useful
• Not all questions are helpful e.g. was the level of teaching appropriate? Are students in a position to give helpful feedback on this?
• Language used isn’t necessarily accessible to students
• Some questionnaires are just too long and a disincentive to filling in at all, or properly
• Should the questions link to the NSS, preparing them to fill the NSS in in their third year?

The group then examined how the feedback can be beneficially used:

• The incentives are not always appropriate in some universities – if you get negative evaluations you may get less teaching
• Very helpful for probationary process
• Linked to personal reviews with the Head of Department (rather than a staff meeting where nothing is resolved) – linked to changing the cluture of the department
• Overall results go to student-staff committees for discussion and then issues get followed up
• Some use informal mid-module evaluation (what do you like, what do you dislike, what would you change)

2. Results of the NSS and Economics Network Student Surveys

Ros O’Leary discussed the results from the 2009 Economics Student Survey (2,050 responses from 67 departments), highlighting:

• The international profile of many students (around 30%+) and the possible impact on their learning
• The teaching of maths and stats which is still an issue for many students
• Trends in decreasing lectures, assigned reading and essays
• Trends in increasing small classes, set preparatory work, online learning
• Suggestions for improvements including: more interactivity in small classes/seminars; improving the teaching of maths and stats; more frequent and continuous assessment; more written and explicit feedback; increased use of VLEs; increased use of real world examples
• Positive comments about how the course had changed them and about the opportunity to participate in the survey

See the survey report at http://www.economicsnetwork.ac.uk/projects/surveys#National_Surveys_of_Economics_Students

Dr Paul Latreille discussed the NSS results for economics, including rankings of Universities by each question, and a comparison with other subjects. See the full report at: 
http://www.economicsnetwork.ac.uk/projects/surveys#National_Student_Surveys_NSS

The group examined some of the issues raised from the surveys in more detail, including:

• How representative survey results are of the student population – is it possible to explore in more depth for any significance in results, by example ability? The Economics Network’s student survey asks about maths A level which was agreed is one indicator – this was explored in the survey (significant results included those with A level maths found workshops, groupwork projects, teaching of maths and stats, and materials put on VLEs more useful but found their course less relevant to the real world than those without A level maths)
• Average class size would be interesting
• Do students filling in the NSS reflect on the whole 3 years or just the 3rd year? Some universities use NSS question in years 1 and 2, so in this case it would just be a reflection of year 3
• Economics Network focus group common themes included how important students felt about belonging to their department – not only in terms of with other students (some departments have worked hard to engender a sense of community by putting students in small groups which are the same across modules) but also in terms of relating to the research of the department (Heriot-Watt has run research discussions for their student society)

3. Using Feedback from Students to Change Practice

Dr Robbie Mochrie from Heriot-Watt and Dr David McCausland from Aberdeen outlined how they had used feedback from students (including the NSS, the Economics Network student survey and an Economics Network-led student focus group in their respective departments) in their departments to make changes to teaching and curricula.

David discussed changes in student demographics, including an increase in overseas students, and a knock-on effect of these students finding maths and stats teaching at too low a level compared to home students. Students also had difficulties in understanding why they need to write essays as well as essay-writing itself; issues with reading journal articles were also picked up. David’s slides are available here.

Robbie outlined how economics had become more popular at Heriot-Watt, including a conscious effort by the department to attract more students to signing up for the course after the first year. A contributing factor has been the creation of a student economic society by students. The focus group highlighted how students would like a wider choice of advanced level options (e.g. history of economic thought); more real world connections including applied work; podcasts; better libraries. Currently the department is looking at work placements and giving out more student awards for essays etc.

The group then discussed the issues raised in more detail including:

• Will departments be able to deliver what students want (due to decreasing resources)?
• Focus groups could raise expectations of students, particularly in a climate of decreasing resources
• Managing student expectations is key – throughout course/programme and hence reflecting in NSS results/focus groups. Part of this is informing students about work of academics, including research and constraints of academics (e.g. decreasing resource)
• Would be useful to explore assessment – what do they expect?
• Student focus groups useful catalyst for dialogue between staff and students about expectations
• Don’t always get representative sample – departments need to bear this in mind when selecting students/advertising for students and also when considering results
• Helpful to have independent person to run student focus groups
• Could expand scheme through brokering running focus groups between departments (ie departments running them for each other)

To apply for the 2010/2011 Student Focus Group Scheme please go to: 
http://www.economicsnetwork.ac.uk/projects/surveys/focus_group