Readers may be interested in uses of business games and economic policy simulations in learning and assessment. Dawson (1995) reported how some 250 students, formed into 44 self-selected groups, on a Level 2 problem-solving skills module had coped with designing, conducting and writing up experiments usins the business games package BOSSCAT. Although this piece relates to one package only, the approach taken by the authors is applicable to all computer-based business and macroeconomic policy games. BOSSCAT is well-adapted to our particular purpose in not requiring the presence of a "referee" every time a student accesses it.
1995/6 faced us with larger numbers and with the need to fashion an assignment of similar nature and difficulty to that set in 1994/5 - which also had to be sufficiently different so that no team could just obtain a disk from the earlier cohort, make a few modifications, and present the work as its own. Preferably the task would also be based on BOSSCAT, but as there is a limited number of variations possible from a given simulation we were not too confident about meeting all the requirements. We did however find a solution and in addition were pleasantly surprised to discover some ideas for next year's assignment, which suggests that simulations and games have more potential than most of us believe.
First we outline the nature of the task set in 1995/6 and how it was designed. Then we discuss our marking scheme. Finally we analyse student performance to see what correlations emerge between types of error.
The 1995/6 cohort numbered over 300 - a 20% rise relative to 1994/5. Thinking the 1994/5 minimum group size (five) had been too high, we set a maximum of four. These two factors raised the number of groups and in the end 95 reports were submitted, of which 7 were by part-time students. In both years the emphasis was very much on students acquiring experience and demonstrating some skills in, controlled experimentation and in making sense of the results. One aspect in which we are particularly interested is whether full-time and part-time students perform equally well. Part timers have fewer classes and with family and job commitments find it more difficult to arrange meetings. On the other hand they tend to be more highly motivated and have practical business experience upon which to draw. The sample size is however very low.
As in 1994/5 we began from the principle that every group should have a slightly different task. This was simple to organise; a six by six matrix of permitted combinations from the lower bound (0.5) varying in steps of 0.2 to the upper bound (1.5) of BOSSCAT's "market elasticity" and "advertising sensitivity" parameters gave 36 unique experimental designs. Doubling this once for the two products (GEMA and LUCY) and once more for two pre-set market multiplier values (1 and 3) gave a total of 144 designs; more than enough (as it proved) to serve our purpose. Each team was issued with a unique alphanumeric code (and an accompanying key) which identified the doll, the market size and the combination of price and advertising sensitivities to be used in all its experiments.(Note 1)
A few groups were still rather unsure whether the row subscript precedes the column subscript in matrix algebra, but in the end almost none got their sensitivities transposed. For example, team L3AA was advising a firm selling LUCY in a market with a multiplier (scale factor) of 3 and both senstivities set at their minimal values of 0.5. Team G1BF had a client operating in a smaller (scale factor 1) market for GEMA characterised by a price sensitivity of 0.7 and an advertising sensitivity of 1.5.
A team's major objective was to find to within 25 pence and 1 advertisement (Note 2) the combination of price and advertising expenditure which would maximise sales revenue (Note 3) in Period 2, subject to a pair of constraints which limited price within the range £2 to £8 and the number of advertisements within the range 3 to 10 (both inclusive). The main purpose of the constraints was to narrow down the potential search area for the students, but a secondary one was to permit similar exercises to be set in future years which would differ only in the constraints. Third, we hoped that groups would recognise the possibility that there might exist superior maxima outwith the zones they had been asked to search. Teams had to produce a report describing and justifying their search method, showing a range of outcomes to illustrate the sensitivity of the results and (equally important) interpret their findings for the firm. To minimise time spent at computer terminals teams were to assume that their clines was a pure monopoly supplier of one doll only.
The experiments we conducted to find out "what the answers should look like" revealed some interesting properties of the market demand function in BOSSCAT. Users of WOODSTOCK may (not) be aware that it shares the same logical structure according to Tony Bushell of Harrison-Macy. Earlier experiments had shown that for each doll there is a ceiling price above which sales are zero - a simple means to ensure that sales are never negative. The surprise element was our discovery that the quantity of dolls sold at that price can also be sold at lower prices; in short the function has a kink (see Figure 1). The range of prices at which this constant quantity can be sold depends on the settings of the sensitivities. For some teams the demand was perfectly ineleastic throughout the permitted range. The perfectly inelastic zone doubtless explains why it is feasible to make profits using high, low or medium price strategies; clearly any business game has to generate lots of outcomes if it is to retain interest. The ceiling price is usually that which maximises revenue. Often it is the sole revenue maximum, but for some settings there is also a low price revenue maximum, which occasionally produces more revenue. Dawson (1991) found that (other things being equal) ten advertisements maximised sales volume.
Figure 1: The Market Demand for Lucy: a typical firm within the prescribed elasticity range
Although each group began with a unique problem and pursued its own strategy to locate the revenue maximising marketing mix, the true maxima vary very little and it is therefore easy to detect a "wrong" answer. This is quite an important consideration when marking scores of assignments. Since students have to submit decision forms, errors should be very simple to identify.
We introduced students to simple search routines, First, beginning with a crude "grid" and refining it around promising locations (pointing out the trade off between initial step size and number of iterations needed). Second, "halving" distances between extremes, then halving around the highest pair of values and so on. Third, taking a given level of price (advertising) varying the level of advertising (price) to locate an appropriate level of the latter. Then holding advertising (price) constant at that locally optimal level to vary the other "instrument" to find its optimal value. We emphasised that although the price and advertising constraints yield 4800 combinations of price and advertising levels which could all be searched, we wanted the "correct" answers found in as few steps as possible. Teams were told to think of themeselves as business consultants selling information in a competitive environment, hence compelled to keep costs under control. To reinforce the message we made it clear that "excessive" numbers of experiments would attract mark deductions. The answers can be (and sometimes were) found in fewer than a dozen steps.
There were three parts, in the first of which the team had to describe and justify its search method essentially to demonstrate to the client firm that it had exercised the minimum level of control required to produce the right answer and that it comprehended the importance of control and that it had conducted its experiments with a keen eye on costs.
Part One is essentially about displaying competence in experimenting. It carried 40 of the 100 available marks; in grading it we notionally gave each team 40 marks and then made set tariff deductions for every experimental error. Although we had predesigned the tasks to a degree; e.g. in limiting the price range to be searched, there are other factors to bear in mind.
First and foremost, to stick to the set objective; second, not to run out of finished goods stocks, for to do so would imply disequilibrium (supply restricted) sales and underestimate the potential revenue. Third, to vary their controls one at a time in order todraw correct inferences. Fourth, to figure out a way to search the "most likely" parts of the grid first. Fifth, to guard against the possibility of multiple maxima by searching at high as well as at low prices.
Part Two involved reporting findings to the client and explaining their significance. Once we had discovered the perfectly inelastic zone we hoped they would do likewise and would stress inter alia the extreme sensitivity of total revenue at the maximising price and advertising mix (a further 1p price rise wipes out sales). We also hoped to see a display of non-zero revenues obtained under alternative marketing mixes. We allocated twenty marks the getting the right answer accompanied by some indicators of sensitivity, and a further ten for making the indications clear to the client. Teams getting and reporting the correct answer accompanied by a range of other strategies to illustrate the sensitivity got twenty marks. Teams getting the wrong answer and/or the correct one but without placing it in relation to other marketing mixes got ten. In some case we compromised on fifteen when there was a combination of the correct answer and some limited but not quite satisfactory attempt at sensitivity analysis. For the tem marks for interpretation we expected teams might explain that the revenue forecast refers only to the second period; that the marketing mix is constrained and thus superior alternatives might exist; that the firm might or might not be able to generate a profit by following the advice.
The third part (thirty marks) was really a bit of economics; to consider whether revenue maximisation was a sustainable long term goal for business.
Of the 88 reports submitted by our full-time students, 41 groups attempted the assignment with respect to the GEMA doll, the remained working on identical problems but using the LUCY doll. The part-time Business Studies students submitted seven reports, all of which were concerned with the LUCY doll.
The aims and objectives of the assignment have been outline earlier. Therefore this section summarises and analyses the students' actual performances. The tow tables show, classified by doll, the frequency of the more important types of experimental errors committed. Since some teams made several of these errors while a few made none, and some forms are error are not recorded, the numbers in the tables do not add up to 95.
As can be seen, it is perfectly feasible (if rather disconcerting!) to arrive at the "correct" answer, but still make some fundamental mistakes, although this is not unknown in reality. Christopher Columbus' voyages could never have "proved" that the world was not flat since he never actually circumnavigated it. For instance, three GEMA groups all found the "correct" answer i.e. the price and advertising mix giving revenue maximisation, but followed the wrong (Note 4) objective! The wrong objective was attempted by 16 (18%) of the groups, although this only caused 9 groups to obtain a wrong answer or no answer at all.
However, for both the LUCY and the GEMA groups, the most common error was an incomplete/inefficient search method i.e. far too many experiments and/or not searching likely revenue maximising areas first. A team having (say) found that 10 adverts and a price of £2 produced more sales revenue than the same number of adverts and prices of £3, £2.75, £2.50 and £2.25 might neglect to search for possibly higher revenue maxima at prices above £3. This type of error was made by 59 (67%) of the groups, although most found the "correct" answer. 8 groups (9%) used the wrong advertising/price parameters and 6 groups (7%) broke the price and/or advertising constraints. Several groups ran out of stock at some point in their experimentation, although the figure of 10 of the groups may underestimate this since a few did not provide records of their unsold goods stock.
Of the 88 groups of full-time students, 21 (23%) did not find the "correct" answer. Of the seven part-time groups, all found the "correct" answer, with no major errors being committed (in fact, only one group made one of the tabulated mistakes, which was to use wrong price/advertising parameters). Indeed the part-time students seemed to have a greater appreciation of the problem and how best to solve it.
Having outlined the basic results it is interesting to look a little closer at what the students actually made of the problem. Although the errors made by the students have been dealt with above, most of the full-time groups failed to appreciate that the constraints imposed upon them might mean that they might not find the global maximisation point for revenue. Very few suggested that the revenue maximisation point might lie outside of the constraints. This did not escape the part-time students however, where perhaps their practical business experience proved helpful. Another omission from many groups was failing to provide a satisfactory rationale for their search method. On the whole most groups seem to have adopted a grid search pattern, although almost inevitably some groups relied upon the tried and tested "hit and hope" method. Finally, although it was emphasised that this was a report to the management of a firm which was expected to pay handsomely for it, some groups still saw fit to produce pieces of work which would not have looked out of place in the waste paper basket.
The third part of the assignment seemd to cause our students some difficulties. The section posed the question of whether the goal of revenue maximisation is sustainable in the long run. Many groups failed to realise the problems of revenue maximisation and allowed themselves to be side-tracked onto less relevant issues such as marketing and product development (means) rather than issues of profit maximisation, market share and managerial goals (ends). Again, the part-time students showed a greater appreciation of these issues than the full-timers.
| Out of Stock | Broke Constraint | Wrong Elasticity | Incomplete Search | Varying P&A Together | Wrong
Objective | |
|---|---|---|---|---|---|---|
| No Answer | - | - | - | 1 | - | 1 |
| Wrong Answer | 3 | 1 | 1 | 10 | 1 | 5 |
| Right
Answer Only | 1 | 2 | 2 | 12 | - | 2 |
| Right answer & Sensistivity Analysis | 1 | - | 1 | 6 | - | - |
| Out of Stock | Broke Constraint | Wrong Elasticity | Incomplete Search | Varying P&A Together | Wrong
Objective |
|
|---|---|---|---|---|---|---|
| No Answer | 1 | 1 | - | 1 | - | 1 |
| Wrong Answer | 2 | 2 | - | 4 | - | 2 |
| Right
Answer Only | 1 | - | 1 | 5 | - | 2 |
| Right answer & Sensistivity Analysis | 1 | - | 3 | 9 | - | 3 |
We intended to set a demanding, although not impossible task for Level Two students. A typical ex post comment from the better students was that once they had thought the problem through and had acquired some familiarity with the running of the package the experimenting, which at first had seemed a very difficult task, proved to be quite a simple one. This is encouraging, although we have hoped that more teams would have come to this conclusion. Some made so many errors one wonders what they had absorbed and even how much of the manual they had read; after all one chapter analyses the economic model underlying the game and another discusses experimentation - complete with worked examples.
We made some decisions in advance, partly in order to reduce the burden upon students but also with an eye to discovering more about the package's properties than we could hope to do unaided; we could also make use of any findings in setting future assignments. For example, predetermining the elasticity combinations removed two decisions. Imposing a well-defined grid of elasticity values ensured that even if not all 144 teask were undertaken the results obtained might reveal some (to us at any rate) previously hidden characteristics.
What did we discover? First, that the revenue-maximising advertising level (ten adverts) appears to be independent of the price, which reduces the number of iterations needed to locate the revenue maximum. Second, that although there are many feasible starting points (which assist in giving each group a unique task) there is only a handful of solutions; unless of course behaviour is radically different outside the specified price and advertising range. Another cohort of students might help us resolve this issue. Third, we found to our surprise that it is also feasible in a small number of iterations to locate a form of profit maximising marketing mix - something we had considered would be too time-consuming to be conducted within the five/six weeks available for a half-semester assignment. Usually (probably always but it needs checking) this will coincide with the price and advertising combination that places the firm on the high-price revenue maximum, but with production scale appropriately. This is a "form" of profit maximum rather than a true maximum, and it is obtained by setting the materials price at £300 per tonne to remove the random component. Were the materials price set lower it is possible that actual costs might be lowered (or raised - the lower the bid price for materials the greater the risks of delayed delivery and unusable quality); but costs and profits become random variables and thus a true maximum cannot be identified, although one could estimate the expected profit at any given material price for a pre-determined marketing mix.
It looks as if BOSSCAT (and by extension similar business games) should be good for another year of two in setting assignments.
Dawson, A J (1991) Through the Looking Glass: Estimating Structures of the Models Embedded in Computer Teaching Packages. British Review of Economic Issues, Volume 13, Part 31, pp 94-104.
Dawson, A J (1995) Controlled Experiments Using BOSSCAT. CHEER, Volume 9, Issue 1, pp 28-29.
BOSSCAT, PROSPEX and WOODSTOCK are published by Harrison Macy Ltd, 217 Silver Road, Norwich NR3 4TL.
1 This is us learning by doing; in 1994 several teams had managed to forget their parameter settings and reminding them had proved rather time-consuming.
2 We had not anticipated that precise answers might readily be found, not then being aware of the perfectly inelastic portion of the market demand curve. Neither had we anticipated that the effectiveness of advertising might be independent of the price - itself clearly a factor making naive search processes highly effective. Therfore we thought it inappropriate to ask for anything more than locating the optimal marketing mix relatively crudely, and specified a location criterion with that in mind. MORAL: conduct the answer generating work BEFORE you finalise the assignment (easier said than done!).
3 Given that even full-time students have only five or six lectures to introduce them to experimenting, search methods and BOSSCAT and an equal number of supervised laboratory sessions, it seemed better not to introduce the extra complications of porfit maximisation. Choosing single period revenue rather than cumulative revenue as the maximand also saved time at the terminals and kept to a minimum the volume of prints required to be generated by the teams.
4 Some groups found it difficult to abandon the profit maximising goal even when the assignment made it quite clear that the client had purchased their expertise specifically in order to pursue the goal of maximising revenue.