Case study 1: First Year Economic Theory

Part of the Handbook chapter on Assessment Design and Methods

When considering assessment methods in economics, arguably one of the most interesting potential case studies concerns introductory economic theory. Amongst the various challenges faced by those delivering such a module are the issues of setting the tone for subsequent years of study and addressing the transition between school and university education. For many years, first year undergraduate economic theory at Swansea University has been split into two modules. Both labelled Principles of Economics, the ‘A’ version of the module is taken by students arriving with post-GCSE experience of Economics (A-level, AS-level, foundation year, IB etc.), while students without such a background are enrolled on Principles of Economics B. These year-long modules combine both standard introductory microeconomic and macroeconomic content and are structured in such a fashion as to equally prepare students from differing backgrounds for the demands of their future studies. In an attempt to increase student engagement, 2009–10 saw the introduction of a new assessment scheme for Principles of Economics A. The ‘old’ scheme comprised of a mid-year January multiple-choice examination, an in-class disclosed examination and a standard summer (May/June) examination.^[1] The weightings attached to these components were 30 per cent, 10 per cent and 60 per cent respectively. The new assessment scheme involved the introduction of six within-tutorial mini-assessments spread throughout the year with the best five marks to be included at weighting of 4 per cent each. The revised assessment scheme therefore involved a reduction in the weighting of other assessment components to accommodate the tutorial-based assessment, and consequently involved a corresponding change in the content of these components. The resulting structure was then: tutorial-based exercises (20 per cent), January examination (20 per cent), in-class examination (10 per cent) and summer examination (50 per cent). It should be noted that these summative assessment components are in addition to formative elements contained within the module.

The revised assessment scheme sought to achieve a number of objectives. First, it aimed to increase the extent to which students work consistently throughout the year by introducing an additional six elements of assessment within term. Second, it aimed to introduce a mechanism for students to gauge their understanding of topics and material as soon as possible. As a number of elements of the syllabus will be familiar to students from their earlier studies, there is always a possibility of students focusing their attention and efforts on those topics which are new and paying insufficient attention to previously considered concepts and terminology. Obviously this is not to be encouraged as, aside from the necessity for understanding to be refreshed, it will prove particularly problematic when material is presented in a different, extended or advanced manner to that previously experienced. However, this may only come to light when confronted with assessment which forces closer examination of topics in formal setting. The use of frequent lower weighted assessment allows any misdirection of efforts or gaps in knowledge to be addressed quickly ahead of becoming substantive issues detected in a more weighty assessment component. A third obvious intention of the in-tutorial assessment was to increase the preparedness of students for their more heavily weighted assessments later in the module. The regularity of testing not only ensures a familiarity with the ‘rules and regulations’ of examinations and settles students ahead of the more substantive latter assessment components, but also allows knowledge to build progressively through the year.

In terms of the structure or mechanics of the tutorial-based assessment, this was very straightforward to devise. The module already contained a series of fortnightly tutorials which involved students discussing and working through previously circulated exercise sheets designed to illustrate material presented in lectures. The expectation is that students attempt the exercises ahead of the sessions and then focus more closely with their tutor during tutorials on those elements they feel warrant further analysis to improve their understanding. The tutorial-based assessments were introduced to six sessions (three in each teaching block). This meant that time allocated to the circulated exercises sheets was reduced to allow time to undertake a 10-minute mini-assessment. In each instance, the assessment involved a series of very short questions which were very similar in nature to those covered in the circulated exercise sheet. This reveals a further objective of the assessment in its role as a further prompt to attempt the circulated exercise sheets and engagement with the tutor to resolve any uncertainties concerning material. An example of the nature of the tutorial-based assessment is provided below. This simple mini-test provides students and tutors alike with a quick means of assessing understanding of material covered. In this particular case, the four-part assessment picks up upon issues in consumer choice. Alternatively phrased the questions ask: ‘What is implied by the positioning of an indifference curve?’, ‘What is implied by the slope of an indifference curve?’, ‘What is implied by the shape of an indifference curve?’, ‘How can we construct/manipulate/employ budget lines/constraints’. It can be seen that the exercises provided are simple in nature to assess quickly understanding in a manner that goes beyond that afforded by multiple-choice examination. However, the demands on the tutor are minimal as the marking involved is very straightforward and hence feedback can be provided to all students extremely quickly. Importantly, these are crucial issues and the assessment of student understanding of them can be assessed quickly ahead of formalising the knowledge to move on to consider substitution and income effects.

The feedback on the revised assessment can be considered in two ways. First, changes in module marks can be considered. Information on this is provided in Table 1. However, before considering the figures provided, a number of issues must be recognised. The information to be considered provides broad measures of student performance on the Principles of Economics A module over a three-year period. Table 1 provides three sets of information on the two Principles of Economics modules. The first set of information simply provides the number of students taking the module each year. Clearly version ‘B’ is larger than ‘A’. The second set of information presents the change in the average module mark relative to the year before assessment on Principles of Economics A was revised. In this instance, the other module which does not have a revised assessment method acts as something of a control (albeit in a loose sense). The ‘Average Difference’ column provides information on average difference in the marks students obtain on this module and those they obtain elsewhere. By considering the relative performance both across modules for each year and then over a number of years, some control for cohort effects is present and a snapshot of the general impact of changes in assessment design can be inferred. However, it is recognised that a variety of factors (different students, different questions etc.) make it difficult to identify the true impact of a change in assessment.

Table 1

Level-1 Economic Theory: Student Numbers, Mean Mark and Relative Module Performance

	Principles of Economics A			Principles of Economics B
	Number	Change in Mean Mark	Average Difference	Number	Change in Mean Mark	Average Difference
2010–2011	131	10.1%	–3.2	261	4.9%	–9.2
2009–2010	137	8.5%	–1.4	304	4.9%	–6.8
200–-2010: Introduction of Tutorial-Based Assessment for Principles of Economics A
2008–2009	122	N/A	–7.6	290	N/A	–6.6

An example of the Level 1 tutorial-based assessment

1. The indifference curves below (denoted as E, F and G) relate to differing levels of utility obtained from the consumption of Goods X and Y. One of the indifference curves corresponds to 2 utils, another corresponds to 8 utils, while another corresponds to 6 utils. Which curve depicts 8 utils?

2. Consider the indifference curve depicted in the diagram below. Annotate this diagram by marking two distinct points on this indifference curve. Label these points ‘C’ and ‘D’ in such a way that point ‘D’ corresponds to a lower marginal rate of substitution than point ‘C’. Using a single sentence, explain your answer.

3. It is known that the indifference curve for two goods is L-shaped. What does this tell you about the relationship between these goods?

4. Consider the budget line (BL) depicted in the diagram below for given prices of Goods X and Y and a given level of money income. Suppose that following the drawing of BL, the price of Good X remains the same, the price of Good Y doubles, and the level of money income doubles. Sketch the new budget line on the diagram below.

From consideration of the results presented in Table 1 it can be seen that a dramatic jump in the mean for Principles of Economics A occurred following the introduction of a revised assessment scheme. From inspection of the results for Principles of Economics B, it is apparent that this module experienced an increase in its mean mark at the same time. However, the increases in marks on the modules differ, with the module with in-tutorial assessment experiencing an increase of 8.5 percentage points in its first year following the change, in comparison to the increase of 4.9 percentage points on the other module. This is indicative of an increase above and beyond a cohort effect. Similarly, when considering the average difference between the marks obtained by students on this module and elsewhere, this narrowed for Principles of Economics A (from being an average of 7.6 percentage points below to only 1.4 percent points below) while it widened on the corresponding Principles of Economics B module (on average students scored 6.6 percentage points less in 2008–09, and 6.8 percentage points less in 2009–10). It should be noted that historically these theory-based modules do return lower marks than other application-based modules taken at Level 1. The results obtained in the second year of delivery of the module under the revised assessment policy (2010–11) make for interesting reading. On the one hand, it appears that the noted improvement in the previous year has been reversed, with the gulf between the mark obtained on the module and elsewhere widening (it increases to 3.2 percentage points). In combination with the increased average mark, this is indicative of a cohort effect, with marks increasing in general, but not by as much on this module as elsewhere. However, at the same time the corresponding gulf for the Principles of Economics B module has widened by more (2.4 percentage points from –6.8 per cent to –9.2 per cent). This would then suggest that a general disparity between theoretical and non-theoretical modules has been less acute for the module where a revised assessment policy has been implemented. Considering the results presented for all years available, the introduction of a new assessment structure has led to an increase in marks and a narrowing of the differences between the module concerned and others when considered from its time of implementation to the present day, while the module without changes in assessment has seen the gulf between it and others widen. In terms of qualitative feedback on in-tutorial assessment, its introduction has received favourable comment from students and staff alike.

Considering the level of assessment on the Level 1 economic theory module, the summative assessment is entirely based upon examination, although an element of it is disclosed (10 per cent), six elements are mini-assessment or quizzes (20 per cent) and a further element is multiple-choice examination (20 per cent). Therefore only 50 per cent is based upon an essay-based examination. To consider this assessment split relative to other Economics departments (or groups) in the UK, a survey was conducted involving 32 departments.^[2] The information gleaned from this on the assessment of Level 1 economic theory is reported in Figure 2 below. This chart provides a breakdown of the examination/coursework split for analogous Level 1 economic theory modules along with a figure indicating the percentage of departments adopting this approach. It can be seen that the most popular form of assessment is via 100 per cent examination (28 per cent of surveyed departments adopting this approach), followed by 60:40 and 70:30 splits (both being adopted by 19 per cent of departments). However, the division of assessment between these two components could mask a range of differing numbers of assessments. This is of particular importance as to the extent that one of the aims of assessment should be to increase engagement, the frequency of assessment is of importance. For example, Principles of Economics A, has four forms of assessment but nine points of summative assessment are employed. Figure 3, containing the results for 32 UK Economics departments, surprisingly shows that a single element of summative assessment is the most popularly employed frequency of assessment (25 per cent of departments surveyed), followed by two and then three points of assessment (22 per cent and 19 per cent respectively).

Considering the outcomes noted above and the information contained in the survey of economics departments, the new assessment scheme has proved successful and has also led to a high frequency of assessment relative to the national norm. However, given the improvements in outcomes and student experience, along with the relative ease of implementation, the additional resources required for such a change are a very small price to pay for a huge return.

Figure 2: A survey of Level 1 Economic Theory assessment weightings

(Exam : Coursework; Percentage of institutions)

Figure 3: A survey of Level 1 Economic Theory assessment components

(Number of components; Percentage of institutions)

---