Student partnership in the co-production of an AI Module Selection Tool

Running an economics department is very much like living the discipline. Student demands are infinite and staff time is a precious and finite constraint. Programme teams often struggle to accommodate student needs while still fulfilling their teaching and research responsibilities. Finding an initiative which expanded our production possibility frontier was therefore a welcome breath of fresh air. Advancements in generative AI and the new Ignite programme, launched at the University of Southampton, allowed us to do just that. Undergraduate students on this programme are given the opportunity to participate in internal internships within the university in the Easter (Spring) break in their first year. One of the Ignite Interns partnered with the Economics Programmes team to co-produce a new resource for undergraduate students in the department. The following paragraphs outline this project and reflect on the benefits and challenges of involving students in the co-production of student support resources.

Identifying the need

Student feedback in our Student-Staff Liaison Committee (SSLC) has for years highlighted the fact that students were struggling with choosing optional modules. Finding and evaluating information on a large number of modules was challenging and time consuming and they wanted our advice on this selection. This added to the workloads of multiple staff members tasked with advising students. Often staff had incomplete information about modules offered by others and students received varied support based on their advisor's length of experience in the department. Equitable provision of clear and concise information on optional modules was identified as a need by the programmes team.

Looking towards generative AI for solutions

The recent advances in generative AI had been on everyone’s agenda and have been the feature of multiple presentations and communities of practice in economics education. Our school even launched an AI competition to encourage educators to use AI in the classroom. Report of their experiences enriched our understanding of how best to incorporate this tool in our teaching. In light of our increased awareness of this tool we set about leveraging its ability to distil large amounts of information for the good of our students. We experimented with uploading module descriptions and asking Gen AI platforms questions about the modules. We found the tool to be sufficiently good at making deductions on various aspects of the module such as difficulty, suitability for career choices and matching to student strengths.

Role of Student Partnership

Having established the role for Gen AI in supporting students' module choices, we looked towards creating a useful tool for students. Who better than the students themselves to judge what is useful? We therefore set out to work with a student partner to design and develop the tool.

Student partnership captures the role of students as active contributors to the production, design and creation of output (typically education-based) with staff in higher education. (McCulloch, 2009). It is important to distinguish this from student representation where the focus is on students having an input into the governance of a higher education institution, and is a relationship where students are given clear responsibility for the output and is typically accessible only to a small subgroup of students (Matthews and Dollinger, 2023).

It represents a process of engagement where staff and students work together in settings both inside and outside the lecture hall to achieve a clear outcome (Healey, Flint & Harrington, 2014). This relationship is formalised in the Students as Partner (SaP) model which captures the wider span of partnerships that can be established between students and staff members in higher education (teaching staff, university administration, alumni, or employers) (Matthews, 2017). The most typical student partnerships are in the areas of curriculum design and pedagogic consultancy and involve a small number of undergraduate students (1-5) partnering with academic teaching staff (Mercer-Mapstone et al., 2017).

While student partnerships may be successfully used in the field of economics, we were able to find little documentation, with the exception of Halliday (2019) and Pezzino and Riganti (2022). This may stem from the nascent area or with the challenging nature of scholarship publication and is hopefully not an indication of the low take-up of such initiatives, since student partnerships have proven to be able to generate a number of benefits. In their systematic literature review, Mercer-Mapstone et al. (2017) identify 65 papers, books, and reports on the topic. They find that papers document an increased sense of belonging and improved sense of identity for both staff and students, higher levels of engagement and motivation in students, and better relationships between staff and students.

The relationship between staff and students as co-producers of output can also, perhaps controversially, be argued to be a more fitting metaphor of the evolving state of current higher education institutions in the UK than the often-used student as consumers one (McCulloch, 2009) and is one where students actively participate in shaping their own experience. The driving force behind this approach is that there is a shared responsibility for the output, with both students and staff committed to its delivery and success (Matthews and Dollinger, 2023) and a foundation based on the three core pillars for such partnerships; respect, reciprocity, and responsibility (Cook-Sather, Bovill and Felten, 2014).

A basic replication of our tool may be created in a very short time without any prior technological expertise.

The brief given to our student intern

The intern was a first year student in their Easter (Spring) break. The internship took place over 3 weeks and covered 72 hours of work. This meant that the project had to be clear and actionable in a short space of time. We explained what we wanted to achieve to our intern, and together we discussed how it could be achieved. We decided to create a document that contained all the information on the optional modules with clear titles regarding the main evaluation criteria we thought students would care about (eg. Number of credits, or assessment method). We also wanted to test the effectiveness of using Gen AI to use this document to recommend modules to students.

The student contributed to refining the project by

• Outlining which criteria students usually use to evaluate modules (difficulty, assessment type and suitability for career prospects)
• Creating a list of questions that students would typically ask 
• Evaluating the usefulness of the tool from a student perspective
• Suggesting different Gen AI platforms that students may be familiar with

Following their involvement in informing the project brief, the intern worked on the project and produced a document containing each individual programme offered by the department, with all the information required by students. Then tested out the system by using typical questions undergraduate students would have about modules. At the end of the placement the intern presented the project, demonstrating the usefulness of the tool to the entire Economics Department.

The intern also checked various free tools for suitability, since some seemed more capable than the one we suggested initially. He also compared them in the usefulness of the answers of the tools, leading some to be eliminated from consideration. In feedback about the overall experience of the internship, he found it to be an enjoyable, challenging and productive experience that enriched his CV and understanding of generative AI.

Is AI a solution to support student experience in module choice selection?

After the internship, the programme team then compiled a final document. We focused our efforts on a free product and eventually decided to go for chatpdf.com. This website had few relevant restrictions in its free version, and in addition would also indicate to the student where in the training document it found the answer. This allowed students a quick check to rule out hallucinations of the AI, which at this stage of technological progress do happen occasionally. Thus, this project contributes also to awareness of the limitations of AI software. As some of our students use the paid version of ChatGPT, we also used the training document to create a custom GPT and shared it with students.

The tool can give surprisingly good advice on everything related to general questions. For instance, the tool successfully suggests option modules to fit certain career aspirations inside Economics. This might be because the AI can rely on a large set of training data, including websites like Prospects which focus on giving careers advice. It was impressive to see how the Gen AI links this general information to specific courses offered in Southampton.

There are, however, various limitations to consider. The more concrete the question becomes, the higher the chances that the tool, in both the paid and free versions, creates mistakes. It is particularly likely to mix up information that is based on documents that alternate between text and tables. While one could try to reduce this by spending more time to improve the training documents, it may take a few more years until AI can deal better with such specific information. Our view is that our project did not result in a fully production-ready tool that can be used without reservations, but instead in a very impressive technology preview.

We thus decided to share it with students as an experimental project, where students would only get access when joining a designated MS Teams site. This allowed them to ask the Programme Team follow-up questions so we could address concerns. We also included a number of messages warning students to exert caution and to double-check with us when in doubt.

A short How-To

A basic replication of our tool may be created in a very short time without any prior technological expertise. All that is required is a list of the optional modules available in each year of a certain programme together with a short description of contents. This needs to be converted to a PDF and made available to students. All the users need to do is to upload the PDF file to a website like chatpdf.com. If a programme team already has such a document – e.g. slides for an option choice info session – it just takes a few clicks to test the quality of the Gen AI suggestions.

If programme team and students have access to the paid version of ChatGPT, it takes a few more minutes to produce a more refined version of such a tool. The basic document needs to be uploaded to create a custom GPT. Short instructions for the chat-bot as well as conversation starter questions can be added. For example, we instructed the GPT to suggest to double-check all information with conventional sources. The tool can then be made available more elegantly using a shareable link.

Co-production: Lessons going forward

This internship was a win-win experience for both the department and for the student. Reflections and discussions have led us to identify the elements we believe were successful and may be helpful for others taking on interns in the future.

We reflect on a few dimensions that we considered important

• Self-contained project: We identified a project that was self-contained, where the intern could start and finish the project. This gave the intern ownership of the project. It meant they could rightly claim ownership of what they created.
• Marketable project: Developing a tool using generative AI is a skill that is very marketable in the current job market. By using this as our project it allowed the intern to add a useful skill to their CV.
• Match project to the student’s skill set: We consulted our intern before the internship started to ensure that the skills we needed for this project were ones he possessed. We had initially shortlisted 4 projects that we could use for the internship and narrowed it down to this one following discussions with the intern.
• Adjust the challenge dynamically: As output was being produced, and our intern demonstrated the ability to work independently, we offered him more challenging work. We asked if he would be willing to present his project to the department and he was excited by this prospect. Adopting a one-size-fits-all approach would have led to this particular intern being under-challenged.
• Check-in, but only when needed: We realised that our intern was able to self-guide and motivate and did not need too much hand holding. We allowed him to work independently, even if that required trust. Micro-managing can deflate energy, and not checking in can lead to disengagement. Find the right balance by asking then when they would like to touch base.

It was interesting to the authors that these lessons that we identify tie in closely to the four key themes identified by Mercer and Mapstone (2017) in their systematic review of the student-as-partners literature: the importance of reciprocity in partnership; the variability of the reality of partnership; using small scale activities; and the need to move towards more inclusive partnerships in higher education. As both staff and students navigate the new Gen AI landscape, initiatives such as this will help set open communication from both ends about a tool that is very much here to stay.

References

Cook-Sather, A., Bovill, C. and Felten, P., 2014. Engaging students as partners in learning and teaching: A guide for faculty. John Wiley & Sons.

Halliday, S.D., 2019. Promoting an ethical economics classroom through partnership. International Journal for Students as Partners, 3(1), pp. 182-189. https://doi.org/10.15173/ijsap.v3i1.3623

Healey, M., 2014, February. Students as partners in learning and teaching in higher education. In Workshop Presented at University College Cork (Vol. 12, No. 1, p. 15).

Matthews, K.E., 2017. Five propositions for genuine students as partners practice. International Journal for Students as Partners, 1(2), pp. 1-9. https://doi.org/10.15173/ijsap.v1i2.3315

Matthews, K.E. and Dollinger, M., 2023. Student voice in higher education: The importance of distinguishing student representation and student partnership. Higher Education, 85 (3), pp. 555-570. https://doi.org/10.1007/s10734-022-00851-7

McCulloch, A., 2009. The student as co‐producer: Learning from public administration about the student–university relationship. Studies in Higher Education, 34 (2), pp. 171-183. https://doi.org/10.1080/03075070802562857

Mercer-Mapstone, L., Dvorakova, S.L., Matthews, K.E., Abbot, S., Cheng, B., Felten, P., Knorr, K., Marquis, E., Shammas, R. and Swaim, K., 2017. A systematic literature review of students as partners in higher education. International Journal for Students as Partners. https://doi.org/10.15173/ijsap.v1i1.3119

Pezzino, M. and Riganti, A.E., 2022. Co-creation of teaching resources and co-teaching. Ideas Bank The Economics Network https://doi.org/10.53593/n3523a