Diploma III is a Computer Assisted Assessment (CAA) software
package, designed by the Brownstone Research Group. My copy
came with the complements of Addison Wesley when I adopted their
Economics textbook by Michael Parkin and David King (ISBN :
0-201-41611-5) as main reference text. Diploma III basically
consists of four individual software programs, making up the main
software, and pulled together by the DIPLOMA executable program.
Other utility executable files are also available, for example
a printer driver. The four individual software packages are
For the purposes of this paper, CAA will be discussed via the use of Proctor software, and the main software is assumed to be installed on the master computer. Basic requirements are a PC compatible with either a hard drive and one floppy drive or, two floppy drives plus a minimum of 512Kb of RAM. DOS requirments: V2.1 or greater. Diploma III is not designed to run on a network.
Supplied with Diploma III software, is a bank of questions, which relate to the above textbook, and are presented in chapter order. References are available within each question, to guide the student to the relevant area within the textbook chapter. I was impressed iwth the quality and quantity of the supplied questions.
Questions can be multiple choice (MC), logical (true or false (TF)), matching (Ma) or short answer (SA). Diploma III software is menu driven.
The questions can be generated into an exam, in either a manual or random fashion. Questions are attributed with a level of difficulty, thus exams can be constructed on a hierarchical level of difficulty, for example as a course develops across time.
Exams, are generated onto a floppy diskette, complete with software to run them (Proctor software).
Exam files have to be given specific names, to allow identification by the gradebook software (and by the students).
I found the best method to use when setting up questions, was to write ten multiple choice questions and save them to a file, then write ten logical questions and save them etc. etc.. This method comes into its own, when an exam is generated (more later).
An area for concern is the editor's inability to produce special characters such as superscripts or subscripts etc.. Hence, writing questions where indices are concerned, although not too difficult, produces an actual display of these characters on the VDU which is difficult to interpret. (Compilation and display of non-mathematical questions is satisfactory).
Similarly, other special characters such as PI (ã), have to be entered as an ASCII code. Specific guiding instructions on how to do this are given in the Diploma handbook.
The editor contains a preview mode to permit a view of the questions as they are being constructed.
When the examiner seeks to set up an exam, the computer supplies adequate prompts to assist. Exams can be constructed by topic and/or by level of difficulty, in either a manual or random fashion. So for example, it is possible to set up an exam covering say four topics at each respective easiest level and the computer will generate the exam, by randomly selecting questions attributed with the least difficulty level for each topic. If the software is instructed to generate a random selection of say ten questions then it is possible to program the computer to permutate a selection, say two questiosn from topic No 1, three questions from topic No 2 etc..
These questions are then copied across (along with the Proctor software), to a floppy diskette, for use by the student. For more than one student, we simply make additional copies of the first diskette.
Potential answers to MC questions can be randomly distributed (scrambled) to deter students from cheating. This means that it is possible for neighbouring students to be viewing the same question on respective VDUs, but respective potential answers may be scrambled. For example the correct answer for the same question may exist at position C on the first VDU and position A on the second VDU. Questions themselves can also be scrambled within the exam.
Comments can be written into each question to assist the student to understand the question. Similarly, reserved space exists within each question for text references.
To deter cheating even further, I decided to generate five exams, using the scramble function across three exam databanks (thirty questions). I used a different coloured label to identify each of the five respective exams when copied to diskette.
For the non computer literate, the exam software offers a simplified exam menu.
Passwords offer some level of security to exist within the system. At the commencement of an exam, all names in the gradebook are presented to each student, but he/she is only able to proceed when the correct password is allocated to the respective name. For security reasons I found it best to allocate each password to the respective student as they entered the computer lab prior to taking the exam.
Passwords also exist within the gradebook, but are hidden from general view. The examiner is able access these passwords. Once the gradebook is set up, it can be easily saved to disk.
The gradebook permits more than one exam to be entered, and each exam can have a different weighting level if required. So, a course (or a module) containing say 4 exams, can be collected within the same gradebook, and each exam can carry a different weighting level if required. So a course (or module) containing say four exams can be collected within the same gradebook and each exam can carry a different weighting.
When students end their exam, and after they have identified their grades (see Proctor below), the examiner collects the floppy diskettes and inserts them into the master computer, and merges each individual student's result into the master gradebook.
Limited statistical analysis such as means, variances and grade ranges are available at each of three levels within the master gradebook i.e.
Provision is allowed to produce character grades such as A, B etc. These grades are defaulted to account for certain limits, but can be altered to accommodate personal percentage limits.
Creation of a Proctor diskette is a matter of following a few simple steps of on screen prompts.
You are offered the opportunity to copy a student gradebook (discussed above), along with the desired exam. By doing so, students are able to access the Proctor software by identifying their name and inputting their unique password. Without the gradebook facility on the diskette anyone can access the Proctor software.
A Proctor exam, can be designed to include (at the beginning), critical information relating to the exam, (for example, the themes and number of topics). During the exam, Proctor displays the questions to the students and invites a response. Responses are via the keyboard characters and Proctor will accept only one of the characters to which the possible answers relate, for example A, B, C and D. An input of E will be ignored. Once Proctor identifies one of the related answers the software records the answer and displays the next question. This pattern continues until the exam is completed. The student is able to move between questions during the examination, to allow a change of answer.
Proctor software not only presents the exam to the student, it also marks the completed exam and displays the result to the student. The software also displays a breakdown of the marks via graphs and indicates to the student where he/she went wrong. Exams can be set up to contain more than one topic. For example the supplied demo disk contains an exam consisting of history, literature and mathematics (three topics in one exam). At the end of the exam the software will display and overall (final) grade, alongside grades per topic (desegregated grades). Performance graphs (i.e. how the student performed in each topic) are similarly displayed. The advantage here is that topics where the students have difficulty can be immediately identified.
Proctor permits the student to review his/her exam via displaying each question alongside the correct answer and, the student's inputted answer. Crafty students might attempt to use this to their advantage, but Proctor will not permit the input of a higher mark in a repeated exam. (However, Proctor will permit the input of a lower mark in a repeated exam).
So in a nutshell, Diploma III allows:
Generation of specific characters are poor. I found it necessary to also supply students with a Word 6 hardcopy of a quantitative mathematics exam, to allow them to identify special characters. Their answers of course, were inputted into Proctor software.
A file import facility is available to transfer files from other word processor software into Diploma III. This attribute allows the software to become flexible. Graphics can also be incorporated into the questions, although I was not too impressed with the standard of graphics displayed with my questions.
As an enthusiast of IT, I usually measure the sophistication of software, by the size of the respective handbook. The Diploma III handbook consists of 200 pages and the additional supplied Quick Start Guide handbook, consists of 32 pages, and instructions are clear and easy to follow. Compared to the handbook for software packages such as Word 6, Diploma III is not a sophisticated package. However, having said that Diploma III does an effective job for me and generated a string interest at our Teaching and Learning Conference (even the Pro Vice- Chancellor attended my session).
Like any other software, the development of Diploma III into a sophisticated package will depend upon fine tuning the current system and tailoring it to meet the requirements of the consumer. This may or may not happen over time. It will probably depend upon the demand and competition of substitutes.
For economists, Diploma III is a 'must'. The supplied databank of questions are offered in (Parkin and King) chapter order. There is no need to write additional questions. Unlike my mathematical exams, generation of an exam with the supplied databank should take a matter of minutes. For keyboard proficient users formulation of a gradebook and Proctor diskettes should take no more than an hour. Similarly, merging the student results into the gradebook takes a few minutes and a hardcopy can be produced for administration purposes. Hence, this is an economic method of supplying an effective assessment.
My next goal is to investigate the possibility for electronic transfer of grades into the campus student records.
The reviewer may be contacted by e-mail at: Jeff Evans@Sunderland.ac.uk