Revolutionizing Nursing Informatics: Using AI for Students to Develop EHR Alerts and Faculty Grading
Abstract
Problem: Nurses need to develop specialized knowledge and competence to optimize the use of artificial intelligence (AI) technology. This study evaluates the use of AI in a Nursing Informatics graduate-level course assignment focused on designing an EHR alert. The evaluation focuses on both students and faculty. Students conducted a literature search with AI tools and compared them to other databases. AI was also used to create an alert design and workflow analysis. Faculty used AI to grade the assignment utilizing a rubric.
Methods: An IRB-exempt retrospective descriptive design was used with a convenience sample of 103 Doctor of Nurse Practitioner students taking an online informatics course. The assignment objective was to design an electronic alert for clinical decisions on intimate partner violence. The individual portion of the assignment aimed to identify high-risk individuals by searching relevant literature in bibliographic databases and using AI tools, reviewing, and selecting articles. The group portion of the assignment involved identifying alert triggers, designing an EHR alert and intervention screen, and workflow analysis, and writing a proposal to the nursing informatics team. Students’ scores were assigned a value of 1 (Mastered), 2 (Competent), or 3 (Did not Master). Descriptive analyses were performed. Students’ comparisons of the AI tools versus the databases and faculty reflections on grading both assignment components are summarized.
Results: The individual portion of the assignment revealed that 91.4% of the students mastered the assignment, 1.9% were competent, and 6.7% did not master. A summary of students’ comparisons of the use of AI tools versus other databases for a literature search revealed mixed results, as some students found AI inaccurate and incomplete. In contrast, others found AI time efficient and that it provided a template that could be expanded.
The group portion of the assignment revealed that 25 groups (69.4%) mastered the assignment, 7 groups (19.4%) were competent, and 4 groups (11.1%) did not master.
Grading revealed that AI inaccurately determined whether citations were APA formatted. For example, AI tools did not recognize hanging indents and double spacing 100% of the time. When grading the group assignments, AI summarized strong and weak points and provided areas for improvement. Comparing faculty grading to AI, there was agreement 14% of the time, 60% of faculty grades were lower, 26% of faculty grades were higher.
Notes
Reference list included in attached slide deck.
Sigma Membership
Pi Delta, Rho
Type
Presentation
Format Type
Text-based Document
Study Design/Type
Descriptive/Correlational
Research Approach
Mixed/Multi Method Research
Keywords:
Competence, Teaching and Learning Strategies, Nursing Informatics, Nursing Education, Emerging Technologies, Artificial Intelligence, Electronic Health Records
Recommended Citation
Kupferschmid, Barbara; Schoville, Rhonda Rae; and Drake, Jeffrey, "Revolutionizing Nursing Informatics: Using AI for Students to Develop EHR Alerts and Faculty Grading" (2025). Biennial Convention (CONV). 172.
https://www.sigmarepository.org/convention/2025/presentations_2025/172
Conference Name
48th Biennial Convention
Conference Host
Sigma Theta Tau International
Conference Location
Indianapolis, Indiana, USA
Conference Year
2025
Rights Holder
All rights reserved by the author(s) and/or publisher(s) listed in this item record unless relinquished in whole or part by a rights notation or a Creative Commons License present in this item record. All permission requests should be directed accordingly and not to the Sigma Repository. All submitting authors or publishers have affirmed that when using material in their work where they do not own copyright, they have obtained permission of the copyright holder prior to submission and the rights holder has been acknowledged as necessary.
Review Type
Abstract Review Only: Reviewed by Event Host
Acquisition
Proxy-submission
Date of Issue
2025-12-04
Revolutionizing Nursing Informatics: Using AI for Students to Develop EHR Alerts and Faculty Grading
Indianapolis, Indiana, USA
Problem: Nurses need to develop specialized knowledge and competence to optimize the use of artificial intelligence (AI) technology. This study evaluates the use of AI in a Nursing Informatics graduate-level course assignment focused on designing an EHR alert. The evaluation focuses on both students and faculty. Students conducted a literature search with AI tools and compared them to other databases. AI was also used to create an alert design and workflow analysis. Faculty used AI to grade the assignment utilizing a rubric.
Methods: An IRB-exempt retrospective descriptive design was used with a convenience sample of 103 Doctor of Nurse Practitioner students taking an online informatics course. The assignment objective was to design an electronic alert for clinical decisions on intimate partner violence. The individual portion of the assignment aimed to identify high-risk individuals by searching relevant literature in bibliographic databases and using AI tools, reviewing, and selecting articles. The group portion of the assignment involved identifying alert triggers, designing an EHR alert and intervention screen, and workflow analysis, and writing a proposal to the nursing informatics team. Students’ scores were assigned a value of 1 (Mastered), 2 (Competent), or 3 (Did not Master). Descriptive analyses were performed. Students’ comparisons of the AI tools versus the databases and faculty reflections on grading both assignment components are summarized.
Results: The individual portion of the assignment revealed that 91.4% of the students mastered the assignment, 1.9% were competent, and 6.7% did not master. A summary of students’ comparisons of the use of AI tools versus other databases for a literature search revealed mixed results, as some students found AI inaccurate and incomplete. In contrast, others found AI time efficient and that it provided a template that could be expanded.
The group portion of the assignment revealed that 25 groups (69.4%) mastered the assignment, 7 groups (19.4%) were competent, and 4 groups (11.1%) did not master.
Grading revealed that AI inaccurately determined whether citations were APA formatted. For example, AI tools did not recognize hanging indents and double spacing 100% of the time. When grading the group assignments, AI summarized strong and weak points and provided areas for improvement. Comparing faculty grading to AI, there was agreement 14% of the time, 60% of faculty grades were lower, 26% of faculty grades were higher.
Description
Implications: AI is revolutionizing healthcare technology and can be harnessed to transform healthcare. Thus, nurses must be proficient in the use of AI. Using AI to search the literature and develop an alert, students learn the efficiency of AI while experiencing the limitations. AI may produce inaccurate information, and references must be checked. Use of AI for grading has shown that the process can be more efficient, but faculty will need to evaluate both the efficiency and reliability of AI.