Abstract

This research investigates the information literacy proficiency levels of engineering students in the context of their use of ChatGPT. With the increasing integration of AI tools like ChatGPT into educational settings, understanding how students engage with and evaluate information is critical. The study employs the DigComp 2.2 framework as a benchmark for measuring information literacy competency, providing a structured approach to assess skills such as information evaluation, and information creation.

To contextualise competency levels, the study examines how students interact with ChatGPT in practical scenarios. A mixed-methods approach is adopted to achieve this: a survey collects data on the frequency, purposes, and types of ChatGPT use among students, while semi-structured interviews provide a deeper exploration of their proficiency levels based on specific tasks and decision-making processes. This combination allows for the triangulation of data, ensuring a comprehensive understanding of information literacy within AI use.

The findings of this study will offer insights into how engineering students navigate the challenges of information literacy in the digital age, particularly in relation to emerging AI technologies. By identifying competency levels and patterns of use, the research aims to inform educational strategies for enhancing information literacy, ultimately contributing to better preparation of students for the demands of the modern engineering workplace.

This presentation examines how students can take an active role in shaping post-AI educational landscapes, emphasising their role as co-designers in defining how generative Artificial Intelligence (genAI) is used to support their learning of programming. Situated in the researcher’s own classroom, this in-depth study takes an ethical approach to exploring the role of genAI in supporting programming education at the post-primary level.

Aligned with UNESCO’s call for human-centered research that is “co-designed by teachers, learners, and researchers” (Miao & Holmes, 2023), this study addresses gaps in the literature regarding student-centered approaches in the area of generative AI and novice programming (Stone, 2024). A design-based research (DBR) methodology is employed, contributing theoretically and practically through exploring this novel space (McKenney & Reeves, 2019). Its focus on co-creation is a key factor in choosing this methodology (Anderson & Shattuck, 2012; Barab & Squire, 2004).

The research progresses through iterative phases of exploration, construction, and reflection (McKenney & Reeves, 2019). The first phase aims to understand the needs and context of the students and explore how they learn about AI before using it. In the second phase, students act as co-creators to develop pedagogical guidelines to support their use of prompts while learning programming. The third phase involves evaluating the pedagogical guidelines. This ethically grounded approach reflects DBR’s focus on “understanding the messiness of real-world practice” (Barab & Squire, 2004, p. 3). Its iterative nature ensures student participation remains at the core, with refinements made to the pedagogical framework throughout the process. Creativity and design remain central to the DBR approach (Hall, 2020), aligning with the aims and objectives of Leaving Certificate Computer Science (Department of Education, 2023).

An overview of the research design will be presented, before sharing preliminary findings from the study’s first phase, offering insights into student attitudes and understandings of generative AI, as well as their use of ChatGPT prompts to support their learning of programming. Paradoxes and dilemmas that surface through the research process will be presented as the researcher engages in a reflexive process (Braun & Clarke, 2013), particularly in balancing ethical considerations with the practical implementation of generative Artificial Intelligence in education. Feedback will be welcomed to inform and refine the next phases of this iterative EdD research.

References

Anderson, T., & Shattuck, J. (2012). Design-Based Research: A Decade of Progress in Education Research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Barab, S., & Squire, K. (2004). Design-Based Research: Putting a Stake in the Ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1

Braun, V., & Clarke, V. (2013). Successful qualitative research: A practical guide for beginners. SAGE.

Department of Education. (2023). Leaving Certificate Computer Science Curriculum Specification. https://www.curriculumonline.ie/getmedia/6eaaa05e-a10b-4bae-bd85-99a1ede0cd67/LC-Computer-Science-specification-updated.pdf

Hall, T. (2020). Bridging Practice and Theory: The Emerging Potential of Design-based Research (DBR) for Digital Innovation in Education. Education Research and Perspectives: An International Journal, 47, 157–173.

McKenney, S. E., & Reeves, T. C. (2019). Conducting educational design research (Second edition). Routledge.

Miao, F., & Holmes, W. (2023). Guidance for generative AI in education and research | UNESCO. UNESCO. https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research

Stone, I. (2024). Exploring Human-Centered Approaches in Generative AI and Introductory Programming Research: A Scoping Review. Proceedings of the 2024 Conference on United Kingdom & Ireland Computing Education Research, 1–7. https://doi.org/10.1145/3689535.3689553

Artificial intelligence (AI) in education is impacting our pedagogical practices (Holmes, 2024; McNamara, 2024). For example, our educational values are being increasingly usurped by the notion that computational processing can be conceptualised as thinking and intelligence. Moreover, these ‘intelligent’ technologies are often presented as superior to human intelligence in terms of speed, efficiency and precision (Selwyn, 2017). Furthermore, these AI algorithms are powerful arbiters of knowledge creation and pedagogical practices through the various ways that they process large streams of online data by way of the classification, creation and dissemination of information and people (Edwards, 2015). This ‘datafication of education’ is situated within an algorithmic culture where everything can be measured and verified against process-driven, goal-oriented pedagogies (Biesta, 2009). However, as Fawns (2018) argues, not everything important is quantifiable. Indeed, this over-reliance on factual data does not adequately consider human ‘value-judgements’ around what is educationally desirable (Biesta, 2009, p.35; O’Leary and Cui, 2020). For example, what cannot be measured is not valued. Thus, the need to rewild our AI pedagogies with more educational values becomes imperative. In response, I propose critical posthuman theory (Braidotti, 2019) to help us think about knowledge and its creation in alternative ways. The posthuman convergence does not position man as its central subject but rather imagines a new collective subject where humans, technology and material matter are inextricably interconnected in and of the world. The posthuman subject is embodied, embedded, relational and differentiated with the capacity to affect and be affected (Braidotti, 2019). The metaphorical figurations of the posthuman subject do not separate the mind from the body, thus thinking capacity cannot be replaced with computational capacity and intelligence is not a fully autonomous force but rather a relational activity. Thus, the embodied and embedded nature of the posthuman subject rejects the instrumental notion of technology. Here, posthuman knowledge cannot be reduced to computational models that adopt an instrumental approach to teaching and learning where human experience is categorised as variables to be counted and processed. This paper is significant in its contribution to how we might collectively rewild AI pedagogies with posthuman values that are more educationally desirable.

References

Biesta, G. (2009) Good education in an age of measurement: on the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability, 21 (1), 33–46. doi.org/10.1007/s11092-008-9064-9

Braidotti, R. (2019) Posthuman knowledge. Cambridge: Polity Press.

Edwards, R. (2015) Software and the hidden curriculum in digital education. Pedagogy, Culture & Society, 23 (2), 265–279. doi.org/10.1080/14681366.2014.977809

Fawns, T. (2018) Postdigital education in design and practice. Postdigital Science and Education, 1 (1), 132–145. doi.org/10.1007/s42438-018-0021-8

Holmes, W. (2024). AIED—Coming of Age? International Journal of Artificial Intelligence in Education. 34, 1–11. https://doi.org/10.1007/s40593-023-00352-3

McNamara, D.S. (2024). From Cognitive Simulations to Learning Engineering, with Humans in the Middle. International Journal of Artificial Intelligence in Education. 34, 42–54. https://doi.org/10.1007/s40593-023-00349-y

O’Leary, M. and Cui, V. (2020) Reconceptualising teaching and learning in higher education: challenging neoliberal narratives of teaching excellence through collaborative observation. Teaching in Higher Education, 25 (2), 141–156. doi.org/10.1080/13562517.2018.1543262

Selwyn, N. (2017) Education and technology: key issues and debates. London: Bloomsbury.

The areas of learning analytics and critical data literacy are growing in focus in higher education, because both society and higher education are becoming increasingly ‘datafied’ (Atenas, Havemann and Timmermann, 2020; Verständig, 2021), particularly through collection of learner data to inform learning analytics. Critical data literacy for individuals has emerged as a way to counter datafication’s effects (Sander, 2020). It is an important part of a person’s wider digital literacies.

With a role of virtual learning environment (VLE) administrator in an Irish university, the author holds a unique perspective on how this particular technology datafies its users. Recognising this, and wider processes of datafication in society, the author sought to respond to calls in the literature for greater critical data literacy education opportunities for students.

An educational intervention for undergraduate students in the Education discipline was developed, drawing upon Pangrazio and Selwyn’s (2018) domains of personal data literacies. It provided a space for students to come together and reflect on their technology use and data practices, through facilitated discussion. Students also explored a personal dashboard of their VLE data, developed by the author as ‘an object to think with’ (Papert, 1980) to prompt further reflection.

Post-intervention interviews were held to analyse the students’ experience and if their critical data literacy had been fostered. Themes of agency, fairness and critical data literacy emerged. Participants had a positive experience of the intervention, and have changed their practice around technology and data as a result. They would welcome further educational opportunities to develop their critical data literacy, including within their undergraduate studies.

This study offers an example of one particular approach to critical data literacy education which shares students’ own data with them. This act of ‘data transparency’ (Prinsloo and Slade, 2015) with students can encourage the university to practice it more widely.

References

Atenas, J., Havemann, L. and Timmermann, C. (2020) ‘Critical literacies for a datafied society: academic development and curriculum design in higher education’, Research in Learning Technology, 28(0). Available at: https://doi.org/10.25304/rlt.v28.2468.

Pangrazio, L. and Selwyn, N. (2019) ‘“Personal data literacies”: A critical literacies approach to enhancing understandings of personal digital data’, New Media & Society, 21(2), pp. 419–437.

Papert, S. (1980) Mindstorms: Children, Computers, and Powerful Ideas.

Prinsloo, P. and Slade, S. (2015) ‘Student privacy self-management: implications for learning analytics’, in Proceedings of the Fifth International Conference on Learning Analytics And Knowledge. LAK ’15: the 5th International Learning Analytics and Knowledge Conference, Poughkeepsie New York: ACM, pp. 83–92.

Sander, I. (2020) ‘Critical big data literacy tools—Engaging citizens and promoting empowered internet usage’, Data & Policy, 2. Available at: https://doi.org/10.1017/dap.2020.5.

Verständig, D. (2021) ‘Critical Data Studies and Data Science in Higher Education: An interdisciplinary and explorative approach towards a critical data literacy’, Seminar.net, 17(2). Available at: https://doi.org/10.7577/seminar.4397.