Engaging Nurses in Early AI Implementation: Usability, Trust, Acceptability, and Training Evaluation
Abstract
Background: Nurses, the largest healthcare workforce, are often overlooked in AI design and implementation (Ronquillo et al., 2021). Excluding them can hinder adoption, as tools may disrupt rather than enhance workflows. Early engagement fosters trust, usability, and alignment with practice (van Steenis et al., 2025). ChatEHR is an institution-developed large language model integrated with the electronic medical record to help clinicians retrieve information efficiently. It entered beta testing in January 2025 and was introduced in October 2025 to 50 nurses from the Research and Innovation Council (RIC), part of nursing shared governance. Fifty nurses completed a “Prompt-a-Thon” training three weeks after access.
Objective: To evaluate ChatEHR usability, acceptability, trust, intent to use, and skill transfer, and to assess the appropriateness and feasibility of the Prompt-a-Thon.
Methods: This project used a pre-post design with a convenience sample at a tertiary academic health system. Surveys were completed before and after training. Usability was measured with the System Usability Scale (Brooke, 1995). Acceptability and appropriateness were assessed using validated implementation measures (Weiner et al., 2017). Trust and intent to use were measured with the Trust and Acceptance of AI Technology scale (Stevens & Stetson, 2023). Selected Learning Transfer System Inventory items assessed skill transfer (Devos et al., 2007). Descriptive statistics were used to summarize outcomes, and paired t-tests were conducted to evaluate pre-post changes.
Results: Of 50 nurses, 33 (66%) completed both surveys. Usability did not differ between prior and new users (65 vs. 58; p=0.200), both below the benchmark for good usability (Hyzy et al., 2022). Acceptability was higher among prior users (5 vs. 4; p=0.041). Trust did not differ pre- and post-training (3 vs. 3; p=0.364). Participants reported moderately positive views of trust and high intent to use ChatEHR. Most (88%) found the training useful, 77% gained confidence, and 88% agreed it supported learning transfer.
Conclusions: Nurses reported moderate trust but strong willingness to adopt ChatEHR. Usability indicated room for improvement. The Prompt-a-Thon was shown to be effective in building confidence and readiness for AI use.
Implications for Practice: Shared governance groups, such as the RIC, can serve as testing hubs for AI feedback and integration. Early involvement of nurses may enhance trust, usability, and adoption.
Notes
References:
Brooke, J. (1995). SUS: A quick and dirty usability scale. Usability Eval. Ind., 189. https://digital.ahrq.gov/sites/default/files/docs/survey/systemusabilityscale%2528sus%2529_comp%255B1%255D.pdf
Devos, C., Dumay, X., Bonami, M., Bates, R., & Holton, E. (2007). The Learning Transfer System Inventory (LTSI) translated into French: Internal structure and predictive validity. International Journal of Training and Development, 11(3), 181–199. https://doi.org/10.1111/j.1468-2419.2007.00280.x
Hyzy, M., Bond, R., Mulvenna, M., Bai, L., Dix, A., Leigh, S., & Hunt, S. (2022). System usability scale benchmarking for digital health apps: Meta-analysis. JMIR mHealth and uHealth, 10(8), e37290. https://doi.org/10.2196/37290
Ronquillo, C. E., Peltonen, L., Pruinelli, L., Chu, C. H., Bakken, S., Beduschi, A., Cato, K., Hardiker, N., Junger, A., Michalowski, M., Nyrup, R., Rahimi, S., Reed, D. N., Salakoski, T., Salanterä, S., Walton, N., Weber, P., Wiegand, T., & Topaz, M. (2021). Artificial intelligence in nursing: Priorities and opportunities from an international invitational thinktank of the nursing and artificial intelligence leadership collaborative. Journal of Advanced Nursing, 77(9), 3707–3717. https://doi.org/10.1111/jan.14855
Stevens, A. F., & Stetson, P. (2023). Theory of trust and acceptance of artificial intelligence technology (TrAAIT): An instrument to assess clinician trust and acceptance of artificial intelligence. Journal of Biomedical Informatics, 148, 104550. https://doi.org/10.1016/j.jbi.2023.104550
van Steenis, S., Helder, O., Kort, H. S. M., & van Houwelingen, T. (2025). Impact of bottom-up cocreation of nursing technological innovations: Explorative interview study among hospital nurses and managers. JMIR Hum Factors, 12, e60543. https://doi.org/10.2196/60543
Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12(1), 108. https://doi.org/10.1186/s13012-017-0635-3
Sigma Membership
Alpha Eta, Alpha Alpha Lambda at-Large
Type
Presentation
Format Type
Text-based Document
Study Design/Type
Pretest-Posttest
Research Approach
Quantitative Research
Keywords:
Implementation Science, Workforce, Interprofessional, Interdisciplinary, Artificial Intelligence, AI
Recommended Citation
Ly, Quyen; Kim, Kyung Mi; Cunha, Olivia; and Nelson, Michele Diaz, "Engaging Nurses in Early AI Implementation: Usability, Trust, Acceptability, and Training Evaluation" (2026). International Nursing Research Congress (INRC). 85.
https://www.sigmarepository.org/inrc/2026/presentations_2026/85
Conference Name
37th International Nursing Research Congress
Conference Host
Sigma Theta Tau International
Conference Location
Toronto, Ontario, Canada
Conference Year
2026
Rights Holder
All rights reserved by the author(s) and/or publisher(s) listed in this item record unless relinquished in whole or part by a rights notation or a Creative Commons License present in this item record. All permission requests should be directed accordingly and not to the Sigma Repository. All submitting authors or publishers have affirmed that when using material in their work where they do not own copyright, they have obtained permission of the copyright holder prior to submission and the rights holder has been acknowledged as necessary.
Review Type
Abstract Review Only: Reviewed by Event Host
Acquisition
Proxy-submission
Date of Issue
2026-07-29
Engaging Nurses in Early AI Implementation: Usability, Trust, Acceptability, and Training Evaluation
Toronto, Ontario, Canada
Background: Nurses, the largest healthcare workforce, are often overlooked in AI design and implementation (Ronquillo et al., 2021). Excluding them can hinder adoption, as tools may disrupt rather than enhance workflows. Early engagement fosters trust, usability, and alignment with practice (van Steenis et al., 2025). ChatEHR is an institution-developed large language model integrated with the electronic medical record to help clinicians retrieve information efficiently. It entered beta testing in January 2025 and was introduced in October 2025 to 50 nurses from the Research and Innovation Council (RIC), part of nursing shared governance. Fifty nurses completed a “Prompt-a-Thon” training three weeks after access.
Objective: To evaluate ChatEHR usability, acceptability, trust, intent to use, and skill transfer, and to assess the appropriateness and feasibility of the Prompt-a-Thon.
Methods: This project used a pre-post design with a convenience sample at a tertiary academic health system. Surveys were completed before and after training. Usability was measured with the System Usability Scale (Brooke, 1995). Acceptability and appropriateness were assessed using validated implementation measures (Weiner et al., 2017). Trust and intent to use were measured with the Trust and Acceptance of AI Technology scale (Stevens & Stetson, 2023). Selected Learning Transfer System Inventory items assessed skill transfer (Devos et al., 2007). Descriptive statistics were used to summarize outcomes, and paired t-tests were conducted to evaluate pre-post changes.
Results: Of 50 nurses, 33 (66%) completed both surveys. Usability did not differ between prior and new users (65 vs. 58; p=0.200), both below the benchmark for good usability (Hyzy et al., 2022). Acceptability was higher among prior users (5 vs. 4; p=0.041). Trust did not differ pre- and post-training (3 vs. 3; p=0.364). Participants reported moderately positive views of trust and high intent to use ChatEHR. Most (88%) found the training useful, 77% gained confidence, and 88% agreed it supported learning transfer.
Conclusions: Nurses reported moderate trust but strong willingness to adopt ChatEHR. Usability indicated room for improvement. The Prompt-a-Thon was shown to be effective in building confidence and readiness for AI use.
Implications for Practice: Shared governance groups, such as the RIC, can serve as testing hubs for AI feedback and integration. Early involvement of nurses may enhance trust, usability, and adoption.
Description
Nurses with early access to a secure AI tool integrated into the EHR reported moderate trust but strong willingness to use it. The Prompt-a-Thon training was considered feasible, acceptable, and effectively supported skill transfer. Early and ongoing involvement of nurses in AI design, implementation, and structured training may enhance usability, trust, and sustainable adoption, emphasizing the value of nurse-driven evaluation in clinical AI.