The Handbook for Economics Lecturers

6.4 Getting Students to Evaluate the Quality of their Own Work

How do we get our students to recognise the quality of their own work? My experience suggests that we need to provide guides and models that students can use as indicators of quality. Such guides should run the gamut from general to specific. For example, three handouts I have used to help my students develop a better understanding of what it means to do quality research include:

  • grading scale models
  • a guide of writing tips based on common errors
  • peer/self review guides.

Students are less likely to have accurate understanding of tutors’ definitions of top-quality work if these definitions are not disclosed to them. The course guide I provide all my research students with is over 100 pages and contains descriptions of expectations, research and writing tips, course assignments and evaluations (to be completed by the student, peers and the instructor). The guide aims to:

  • signal that this is a serious endeavour;
  • provide a clear structure for students’ work; and
  • highlight the multiple components of the research process.

One of the very first documents in this guide is a grading scale that specifically identifies what it takes to earn an A, B, C, D or F. Each grade is described in terms of what the written report includes, what it does not include and the way that information should be communicated. There are many such grading scale models available from which one might develop a specific guide for students. See for example:

Since mistakes made by novice researchers tend to be consistent over time, I provide my students with a series of research tips before they even begin their research projects (Figure 7). These tips focus on the research process as well as the expected output. More importantly, the class discussion that accompanies this handout is focused on why these items are important.

Figure 7: Research Tips for Students

While many of these statements are specific to the final research product, now is a good time to get an idea of my expectations and problems to avoid.

  1. Keep track of your research. The semester is short and we have a lot to do. It is in your best interest to keep detailed notes as to the research you have completed, both the process you have used and the results you have found. Don't make the mistake of having to look for a source or a page of a citation at the last minute. If you look ahead in the course schedule you will notice that this will be facilitated through the creation of a research binder that will be turned in at various points during the semester.
  2. Thesis statements. Written assignments must have a thesis, a single major point. A thesis is a guide for the reader, one that tells your audience the purpose of your essay. It is a statement that encourages the reader to continue. Thus, it should be explicitly stated at the onset of your work. The reader should not have to search for your thesis statement. Good thesis statements often answer the questions 'How?' and/or 'Why?'
  3. Your title. This is one place to show your creativity. A title grabs the attention of the reader, makes them want to read your work, and provides information as to the content of the paper.
  4. Organisation. Written work should be organised around a purpose. The beginning should tell the reader what it is you are trying to accomplish in your work. The middle should begin with the background (often in the form of a review of literature) needed to set up the question you are addressing followed by the evidence or arguments used to support your thesis. This should then be followed by a conclusion (more below).
  5. Know your audience. Recognise that the readers for most of your written work are your peers in class as well as Dr. M. You should assume a general familiarity with economic terms, but be sure to include explanations of less commonly known concepts. Do not use jargon or terms not generally familiar to the reader.
  6. Introduction. Don't start with an introduction that is too far reaching or is narrative in format. You are trying to motivate the reader to continue. Introduce the specific issues you will be addressing in your paper in a way that is interesting.
  7. Answering your economic question. It is critical that in the body of the paper you actually address the thesis you put forth in your introduction. This requires detailed evidence in support of your ultimate answer. Do not simply link a bunch of quotes together and assume the reader understands the link to your thesis. Be specific and explicit in your linking of evidence to the economic question. In addition, avoid unsupported statements or ones that are clearly supported only by opinion. Hint: a paper with a lot of quotes implies that you did not understand the evidence enough to put it into your own words. This also suggests that you are unlikely to convince the reader. In her book, Economical Writing, McCloskey notes that: 'Writing is Thinking' (p. 6). Bad writing implies poorly thought-out ideas. If you do not understand the content of your arguments, it will surely be reflected in your writing.
  8. Make a Persuasive Argument. "... evidence should be accurate, authoritative, precise, clearly explained, complete and representative." Greenlaw Ch 4, p.8.
  9. Avoid Logical Fallacies. Greenlaw Ch 4, Appendix.
  10. Tangents. Don't go off on tangents. Be sure that the evidence you include in your paper directly addresses your economic question. Keep asking yourself what evidence this contributes to support the ultimate conclusion.
  11. Take us along for the ride. Don't assume the reader knows where you are going with your arguments. Make the path easy for them to follow. The key to this is to include well-formulated paragraphs in your essay. Each paragraph should develop a specific idea, and the first sentence or two of a paragraph should provide a transition from the previous paragraph and announce what is coming in the current paragraph. The content of the paragraph should then be directly related to what is announced in the first one or two sentences. Hint: read each paragraph and come up with a couple of words to describe their content. Place these words alongside your paragraphs in the margins. Then review the work by reading these summaries from start to finish. Do they flow? (They should if your outline was well formed.)
  12. The conclusion. Don't just rehash the introduction. The conclusion is where you need to wrap it all up. You want to make that final push to convince the reader that you have answered the economic question you proposed in your thesis. One way to accomplish this is to put the work you have completed in a broader context that shows its importance. In other words, make sure the conclusion follows from the evidence. Greenlaw Ch 4, pp. 3-5
  13. First person. Although writing in the fist person is not incorrect, it is best avoided. Using the first person often leads to subjective and imprecise statements. Writing in economics is a formal process and this project requires evidence in answering an economic question. Thus, your project should not be written using the first person.
  14. Format. Your final paper must be typed, double-spaced and include a list of references (at the end of the paper, not as footnotes). The paper should be standard 12 pitch font, New Times Roman with one-inch margins on all sides. Do not justify the right margin. Draft papers and sections must be stapled in the upper-left corner. Final papers must be spiral bound. A title page must include the title of your paper, your name, the date, the course title and the complete pledge must be written in full and signed. To avoid interruption of the flow of your written work, all tables, graphs and charts should appear after the text but prior to the references page. Be sure that each table has the appropriate degree of self-explanation (can it stand alone, independent of the text?) and is appropriately labelled for text reference.
  15. References. Remember to include only those sources actually cited in the paper. These are to be listed in alphabetical order at the end of the paper. Make sure to document all sources, direct quotations, paraphrases and any information that is not your own. Be especially careful about plagiarism, see the course syllabus for more information. When in doubt, cite it!
  16. Computer problems. This will not be an excuse for late work. Do not wait until the last moment to complete or print out your work. Back up your research often, including sending an e-mail copy to yourself!
  17. Writing Centre. Good writing is a form of communication. Your ideas will get lost if the reader keeps getting lost because of a lack of organisation or poor grammar. The writing centre staff are there to help you improve your written communication skills. Even good writers can benefit from this service. For more information visit http://writing.richmond.edu/.
  18. The Writer's Web is an especially useful source of helpful hints and guidelines for writing at various stages of the process. I encourage you to review this resource before you begin your work!

References:

Greenlaw, S. (2006) Research Methodology: A Guide for Undergraduate Economic Research, Houghton-Mifflin.
McCloskey, D. (2000) Economical Writing, 2nd edition, Waveland Press, Inc.

Finally, because students (not unlike ourselves) find it easier to be critical of another person’s work rather than see similar flaws in their own work, I implement a peer review process of all final drafts of research projects. The peer review document in Figure 8 has a combination of general and specific questions relating to both the content and presentation of the economic analysis. Not only does this review provide students with a new set of comments on their papers, but often reading another student’s paper provides another model for organising arguments, shows examples of common mistakes that are made, and stresses the importance of clear and concise communication. After students have reviewed a peer’s work, I ask them to re-read their own work (prior to receiving the peer comments). They are asked to identify (in much the same manner as they did in reviewing their peers’ work) the strengths and weaknesses of their own research. Because the process of reviewing is fresh in their minds, they are more likely to be critical of their own research.

Figure 8: Peer Review of Paper Draft

Title of Paper: _________________________________________________________

Directions:

Answer the following questions thoughtfully, clearly and concisely. Use complete sentences and specific examples (or page references) to make your advice as understandable as possible. You will be evaluated on the thoroughness and helpfulness of your responses to your peer partner. To make your review as beneficial as possible, it is important that you follow the review steps in the order given below:

First, read the paper carefully, without pausing to comment, to get the overall effect of the paper.

In the space below, write down your general feelings about the paper. Is it interesting? Did it generally maintain your interest? Is it easy to follow (organisation is clear)? Did you get lost along the way?






Next, look over all the questions below and then reread each section of the paper and answer the related questions. As you reread the paper make margin notes to the author identifying which paragraphs/sections were clear and which ones were not.

Title and Introduction:

The introduction should present the topic and its importance. It should clearly define relevant concepts, provide an overview of relationships, present the thesis and indicate how it will be developed, and generally prepare the reader for the paper to follow.

  1. Does the title reflect the content of the paper? Is it interesting?
  2. Are the relevant concepts defined? Make a list of the concepts introduced and note which ones are not clearly defined.
  3. Are the relevant relationships described? Make a list of the relationships that were introduced and note which ones were not clearly described.
  4. Is the economic question clearly presented? What is the paper's thesis or main argument? Describe this in your own words in the space below and identify it in the paper using a margin note. (Provide the page number on which this appears.)
  5. Did the introduction adequately convey the importance of this topic? Why or why not?
  6. Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give this introduction? Explain.

Review of Literature:

The review is designed to inform the reader of what other work has been completed on this topic and to motivate the current work. The review presentation should be organised (such as chronologically or by theme) and be directly related to the research project.

  1. Does the depth of review provide enough background to understand the nature of previous studies? Explain.
  2. Is the review sufficiently linked to the current project? Identify links in the paper using margin notes and provide the page numbers where these appear.
  3. Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give this review section? Explain.

The Contribution Section:

In reviewing this section you should mark each sub-section as excellent if it can be clearly identified in the paper, if it accomplishes what is expected, if you are not left with any content questions, and if it is well written (good flow between and within paragraphs, good sentence structure, etc.). Poor sub-sections are those that are not clear because of incoherent content or flow issues, are incomplete leaving you with unanswered questions or issues not sufficiently addressed, or missing altogether.

Please be sure to provide comments for each subsection!

Contribution Section Excellent Good Poor
Statement of what will be demonstrated in the analysis
Comments:
     
Set up of relationships
Comments:
     
Hypothesis development
Comments:
     
Description of method
Comments:
     
Evidence
Comments:
     
Relation of evidence to general hypothesis
Comments:
     
Comparison of findings to previous research
Comments:
     
Effective use of tables, etc. (where applicable)
Comments:
     

Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give this contribution section? Explain.

Conclusion:

A conclusion must go beyond a simple restatement of the overall argument of the paper. It should provide a final push to convince the reader that the economic question proposed in the thesis has been answered.

  1. Is a short reiteration of the topic and hypothesis presented? Identify this in the paper by using a margin note and note on which page this appears.
  2. Is the evidence synthesised sufficiently that you are convinced of the answer to the economic question? Explain.
  3. Is the research contextualised in a broader economic issue that indicates its importance? State this broader issue.
  4. Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give this conclusion section? Explain.

Readability:

A research paper can only effectively convey its message if it is well written.

  1. Was the paper clearly written? Highlight or make margin notes for any paragraph or section that you had to reread to understand or still were not sure what the contribution of the section was to the overall paper. (Note these section pages below.) Explain to the author what was not clear and any questions that you still have.
  2. Is the language use appropriate? And the audience accurately gauged?
  3. Does the paper suffer from grammar or organisational problems that inhibit the message of the author?

    Grammar: _____ yes, very distracting _____ yes, somewhat distracting _____ no

    Organisation: _____ yes, very distracting _____ yes, somewhat distracting _____ no
  4. Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give for readibility? Explain.

Final thoughts:

  1. What is the main strength of this paper?
  2. What is one weakness of this paper?
  3. What was the single most important contribution of this paper?
  4. Rating: On a scale of 1 to 5 (5 being excellent) what rating would you give the paper overall? Explain.