Teaching Econometrics: The role of the 3Rs

1. Introduction

The use of reproduction and replication to support the teaching of quantitative methods has been promoted in a number of studies (see, inter alia, Janz, 2016; Stojmenovska et al., 2019). More recently, Cook and Watson (2023) have proposed a threefold approach to the use of replication as a teaching aid. The first form of replication considered, direct replication, involves the reproduction of empirical findings. The second, step replication, relates to the reproduction of findings that require underlying, hidden steps to be executed. The third form of replication, flexible replication, relates to the creation of resources that can be modified to generate results for subsequent reproduction. Examples of studies supporting the use of flexible replication include Cook (2006, 2019) which provide resources designed to automatically generate results when alternative data sets are input.

All three of these forms of replication offer opportunities to develop and challenge understanding. While direct replication focuses on the ability to accurately apply methods to reproduce results, step replication tests learners’ abilities to derive results from underlying but undisclosed steps, and flexible replication introduces the challenge of reproducing results obtained from modified or different data. However, while alternative forms of reproduction and replication have been discussed in the literature, a single overarching conclusion emerges: significant benefits arise from the incorporation of the 2Rs (reproduction and replication) in the teaching of quantitative methods. In particular, it is argued (see Cook et al., 2019; Cook and Watson, 2023) that by undertaking activities to successfully achieve a specified objective (i.e. reproduce a stated result), self-efficacy (see Bandura, 1978; Zahaciva et al., 2005) – defined as confidence in one’s abilities – can be generated, thereby mitigating potential negative effects arising from ‘anxiety towards quants’ (see, Dreger and Aiken, 1957; Dowker et al., 2016). Additionally, the activity required for producing specified results serves as a means of introducing further active learning in the classroom.

In this paper we consider the use of a third R, redaction, in the teaching of quantitative methods. The specific quantitative discipline we focus on to illustrate redaction is econometrics, using exercises that involve calculating the redacted elements of empirical output. As with the previous two Rs – reproduction and replication – we argue that successful completion of redaction-based exercises can address issues of self-efficacy and ‘anxiety towards quants’. Additionally, these exercises further enhance active learning. However, our aim is not merely to promote the use of redaction as a teaching tool, but also demonstrate how it can be used to generate cognitively challenging exercises.

In this regard, we draw on Mayer (2004, 2021) and the distinction between behavioural and cognitive activities in the context of active learning. The argument here is that while behavioural activities foster active learning, the degree of cognitive engagement determines their effectiveness. Although redaction might initially appear to generate straightforward tasks – for example, combining elements of output to calculate an undisclosed term – we argue that redaction-based exercises can be designed to offer greater challenges. We propose that this added complexity can encourage deeper reflection on understanding and, consequently, lead to more meaningful learning outcomes.

To illustrate redaction-based exercises, we present two examples. The first example not only requires the calculation of an undisclosed value, but also further challenges understanding by incorporating its subsequent use to solve a further problem. The purpose of this example is to highlight the interconnections between elements of empirical output. The second example extends redaction by requiring the undisclosed value to be derived using different approaches. This exercise aims to develop an understanding of the relationships between two test statistics and their methods of calculation. Together, these examples demonstrate that redaction-based exercises can transcend simple substitution to foster deeper engagement and comprehension.

2. Redaction-based exercises

Example 1

The first of our redaction-based exercises considers the topic of unit root testing. The question posed is as follows:

An approach to answering this question is as follows. The lower table presents the ADF testing equation employed in the analysis. This can be stated as follows:

The ADF test statistic is then given as . From the tabulated estimated testing equation, this can be calculated as . This is the redacted value denoted as ‘A’. However, the question requires the p-value associated with this statistic to be calculated. From inspection of the 5% critical value provided in the output, it can be observed that the calculated test statistic and the critical value are very close in value (−2.895 compared to −2.878, rounded to 3 decimal places). Since the calculated value is slightly more negative than the critical value, the p-value can be approximated as close to, but slightly below, 5%. If the unredacted output were available, it would reveal that the actual p-value is 4.8%.

The purpose of this question was to provide a multi-step exercise requiring the consideration of redacted material. Rather than providing a simpler task, such as calculating an undisclosed value by manipulating information in adjoining cells or columns, the question was designed to be more complex. Specifically, it required the calculation of a missing statistic in one table by identifying and manipulating relevant information obtained from a second table. Once this statistic was calculated, the next stage of the analysis involved recognising its relationship to another element of the output to determine an additional redacted value. To design this question, artificial data were generated to create tailored or bespoke output. The calculated test statistic was intentionally designed to be close in value to a stated critical value to support the requirement for an approximation. This approach – carefully tailoring results to support redaction-based exercises – could readily be applied to other topics and materials.

Example 2

A second example of using redacted information is provided by the following question.

The question can be attempted as follows. As three unit root processes have been considered in the analysis, application of the Johansen method results in the generation of three eigenvalues, three Trace test statistics and three maximum-eigenvalue test statistics. Denoting the eigenvalues obtained from this analysis as with , the maximum eigenvalue test statistics obtained using the largest, middle and smallest eigenvalues can be denoted , and respectively. Similarly, the Trace statistics can be denoted as , and respectively where the subscripts reflect the eigenvalues employed in their calculation. Using this notation, the undisclosed test statistic above (denoted as ALPHA) is the second Trace statistic, . This can be calculated in the three following ways.

Method 1: The known expression or formula for can be employed as follows (where denotes the sample size):

= −164 [ln(1 − 0.063748) + ln(1 − 0.001193)]

This first method of calculating results in a value of 10.9985, to 4 decimal places.

Method 2: A second method of calculating involves recognition of its relationship to the maximum-eigenvalue test. can be derived as . Using this expression:

10.80273 + 0.195791

This provides another means of obtaining the value 10.9985.

Method 3: A third means of calculating draws upon recognition that and . Using these expressions:

48.22199 − 37.22347

Again, a calculated value of 10.9985 is obtained.

The purpose of this question was to enhance familiarity with the derivation of the Trace and maximum-eigenvalue test statistics by examining their construction and application across different rounds of sequential hypothesis testing in the Johansen method. This was achieved by requiring the use of alternative methods to determine the undisclosed value in the redacted output.

3. Concluding remarks

Numerous studies have highlighted the benefits of the 2Rs – reproduction and replication – in teaching quantitative methods. This paper introduces redaction as a third R. By providing redacted empirical output, knowledge can be both challenged and developed through tasks involving these redacted elements. However, redaction exercises should not be overly simplistic, such as requiring only the direct use of provided information to determine an undisclosed value. Instead, as the two examples demonstrate, redaction-based exercises can adopt a multi-step format, requiring the synthesis of alternative information or the identification of redacted output through multiple approaches to further challenge and enhance understanding.

References

Bandura, A. 1978. Self-efficacy: toward a unifying theory of behavioral change. Advances in Behaviour Research and Therapy 1, 139-161. https://doi.org/10.1080/23752696.2018.1564880

Cook, S. 2006. Understanding the construction and interpretation of forecast evaluation statistics using computer-based tutorial exercises. Economics Network Ideas Bank https://doi.org/10.53593/n3168a

Cook, S. 2019. Forecast evaluation using Theil’s Inequality Coefficients. Economics Network Ideas Bank https://doi.org/10.53593/n143a

Cook, S. and Watson, D. 2023. The use of online materials to support the development of quantitative skills. In The Handbook of Teaching and Learning Social Research Methods, Nind, M. (ed.). Cheltenham: Edward Elgar. https://doi.org/10.4337/9781800884274.00028

Cook, S., Watson, D. and Vougas, D. 2019. Solving the quantitative skills gap: a flexible learning call to arms! Higher Education Pedagogies Journal 4, 17-31. https://doi.org/10.1080/23752696.2018.1564880

Dowker, A., Sarkar A. and Looi, C. 2016. Mathematics anxiety: what have we learned in 60 years? Frontiers in Psychology 7, 508. https://doi.org/10.3389/fpsyg.2016.00508

Dreger R. and Aiken L. 1957. The identification of number anxiety in a college population. Journal of Educational Psychology 48, 344-351. https://doi.org/10.1037/h0045894

Huang, X. and Mayer, R. 2016. Benefits of adding anxiety-reducing features to a computer-based multimedia lesson on statistics. Computers in Human Behavior 63, 293-303. https://doi.org/10.1016/j.chb.2016.05.034

Janz, N. 2016. Bringing the Gold Standard into the classroom: Replication in University teaching. International Studies Perspectives 17, 392-407. https://doi.org/10.1111/insp.12104

Mayer, R. 2004. Should there be a three-strikes rule against pure discovery learning? American Psychologist 59, 14-19. https://doi.org/10.1037/0003-066x.59.1.14

Mayer, R. 2021. Multimedia Learning (3^rd edition). Cambridge: Cambridge University Press.

Stojmenovska D., Bol T. and Leopold T. 2019. Teaching replication to graduate students. Teaching Sociology, 474, 303-313. https://doi.org/10.1177/0092055X19867996

Zahaciva, A., Lynch, S. and Espenshade, T. 2005. Self-efficacy, stress, and academic success in college. Research in Higher Education 46, 677-706. https://doi.org/10.1007/s11162-004-4139-z