Context / probleemstelling of aanleiding

Context/probleemstelling of aanleiding:
Students increasingly use generative AI (GenAI) tools, such as ChatGPT, to support their studies. These tools offer clear benefits for personalized learning but also raise concerns about output accuracy, ethical, legal and societal implications (ELSI) and risk of cognitive complacency (Wang, 2023).
To meet these challenges, students must develop GenAI literacy, defined in the EU’s DigComp2.2 framework as the competencies needed to engage with GenAI effectively and ethically. The EU AI Act (Article 4) requires institutions to teach GenAI literacy.
Most literacy interventions focus on conceptual knowledge or practical skills such as prompting, while comprehensive approaches that include ELSI competencies are scarce (Gu & Ericson, 2025).
We therefore developed a project where students design and validate a GenAI tutor. This approach fosters comprehensive GenAI literacy across all five DiComp2.2 competency areas.
Beschrijving van de interventie/innovatie:
The one-week project is part of a required first-year course in the UvA Medical Informatics MSc programme. Students learn GenAI fundamentals, ELSI, and validation methods. Subsequently they design, develop and validate a GenAI-based tutor.
Deliverables:
1. Functioning GenAI tutor
2. Report detailing:
a. Design choices regarding functionalities and ELSI such as bias, copyright, environment and didactical use;
b. Validation outcomes;
Ervaringen/analyse van de implementatie:
Based on 24 matched pre-post responses of students we evaluated the perceived knowledge level and self-efficacy. Lecturers graded the competency level based on a rubric.
Median knowledge level (scale 1-5) of bias recognition, environmental impact, copyright and data security changed respectively from 4.0->3.0(p=0.25), 3.0->4.0(p=0.09), 2.0->3.5(p=0.07), 3.0->4.0(p=0.08). Median self-efficacy (scale 1-7) for bias mitigation, environmental impact, copyright and data security changed respectively from 5.0->5.0(p=0.88), 3.0->5.0(p<0.001), 4.0->5.0(p=0.14), 5.0->5.0(p=0.073). Mean (standard-deviation) rubric score for competency was 7.1(+-1.9)/10 for bias, 7.3(+-2.8)/10 for environmental impact, 9.2(+-1.5)/10 for copyright and 8.3(+-2.1)/10 for data security.
Lessons learned (implicaties voor de praktijk):
Many students lacked experience with probabilistic tools, providing explicit guidance on validation metrics and worked examples of validation was crucial to help them understand non-deterministic outputs and critically evaluate GenAI tutor performance. Self-reported outcomes indicate preliminary positive trends across ELSI knowledge and self-efficacy, but statistical significance was limited by small sample size. Lecturer grades were moderate-high (7.1-9.2/10).
Besides study results, the session will offer tips on: translating a literacy framework into practice, engaging students in applying literacy skills, assessing literacy with a rubric, and teaching validation of GenAI output.
Referenties (max. 2):
Mahadewi, M. P., Aysya, A. A. A., Sofiyani, Z., & Fahmi, F. (2025). The Importance of Literacy on Artificial Intelligence for Higher Education Students: A Systematic Literature Review. International Journal of Advances in Data and Information Systems,6(1),1–14. https://doi.org/10.59395/ijadis.v6i1.1350
Wang, T. (2023). Navigating Generative AI (ChatGPT) in Higher Education: Opportunities and Challenges (p.215–225). Springer Nature. https://doi.org/10.1007/978-981-99-5961-7_28