5. Online assessment and the use of media

Online assessment is very common in economics modules. Its key advantage, especially for large classes, is immediate feedback to the students. Moreover, online practice with objective-answer questions, can be very useful for students to reinforce their understanding and can be used by instructors as formative and summative assessment. Indeed, many textbooks provide companion websites that supply supplementary material including assessment based on multiple choice quizzes.[1]

Popular learning environments provided by publishers include Pearson’s MyEconLab (and new arrival Revel), McGraw Hill’s Connect, Cengage’s Aplia and Wiley’s Wiley Plus. Virtual learning environments, such as Blackboard and Moodle, allow instructors to construct assignments from multiple choice, true/false, fill-the-blanks, and other types of questions. In addition, questions can embed figures, diagrams and audio or video clips.

Standard online assignments have drawbacks, however. Questions and feedback tend to be standardized. Especially with MCQs, students are not incentivised to understand the material in its entirety, but only have to identify (often by a process of sequential elimination) the answer most likely to be correct.

There are two recent developments that could allow instructors to move away from standard and repetitive MCQs. The first is the use of more interactive visual material, such as clickable images and interactive diagrams. The second is the possibility for questions to adapt to students’ performance; for example, questions that, if answered incorrectly, provide additional questions and hints to help the student towards a correct answer. The platform MapleTA,[2] built around the mathematical software Maple, supports the creation of algorithmic questions and analytic manipulation.[3] It can also create adaptive questions and clickable diagrams. A very important advantage of MapleTA is full integration with virtual learning environments such as Blackboard. This means that taking an assignment does not require logging in to a different site.

Clickable images / Sketch-the-graph questions

MapleTA allows instructors to create questions that show a visual item (for example a diagram, a figure, even a table with data) where students are asked to identify a particular portion and click on it. Figure 13 shows a question asking students to identify the area of the graph representing the profit of a monopolist. The advantage of this type of question is that students interact directly with the figure, instead of picking the most likely alternative among a set of possible answers.

Figure 13: example of clickable question in MapleTA

The visual content does not have to be a diagram. Figure 14 below shows a question asking students to study the payoff matrix of a very simple 2x2 simultaneous move game and identify a player's best response. Again, students will have to understand and interact with the whole payoff matrix, not just a subset as is often the case with MCQs.

Figure 14: clickable image question with a pay-off matrix

Ideally, instructors may want students to be able to reproduce diagrams. The sketch-the-graph question type requires students to use MapleTA's graphic toolbox to draw a diagram. Figure 15 shows a question asking students to draw the demand function and profit maximisation choice from figure 13.

Figure 15: an example of “sketch the graph” question in MapleTA

Adaptive questions

In a standard multiple choice question, a wrong answer gets no mark (or negative marks) and, possibly, feedback including the correct answer. This may be a little frustrating and disheartening to some students, in particular to those who have a general understanding of the material. Ideally, instructors would give students a combination of encouragement ("You did not get it right. It's OK, let's approach the question step-by-step"), technical information (for example "recall the concept of...") and questions (not necessarily multiple choice) to test their effective understanding. The student would thus learn to identify important concepts and apply them correctly. Eventually, the student should be awarded at least a partial mark. This is what any instructor would do in one-to-one or small group teaching.

This personalised approach cannot be replicated with standard online assignments. Fortunately, MapleTA allows adaptive online questions that do precisely what we are suggesting. Students can be prompted with an additional, and related, sub-question if they get the main question wrong. This way, instructors can guide students step by step and support them in finding alternative ways to answer the original question. If carefully designed, essentially every question in MapleTA can be adaptive and combine various question types.

Figure 16 shows an adaptive version of the graphical question from Figure 15 above.

Figure 16: adaptive question in MapleTA


[1] Olczak (2014) identifies a positive effect of publishers’ web-based resources on student learning.

[2] http://www.maplesoft.com/products/mapleta/index_personas.aspx

[3] These features should be particularly welcome by those instructors teaching modules with mathematical content. The fact that MapleTA can process algebraic expressions implies that expressions 2a+b and b+2a are considered to be equal. This would not be true in assignments created within standard virtual learning environments. In addition, MapleTA allows instructors to access all Maple packages and use algorithms to create randomised mathematical objects such as variables, equations, matrices, etc.