“We were expecting AI would reduce administrative burden, but that people would still get the answers eventually with the PDF, but that just wasn’t the case,” Ellithorpe said. “While we don’t know for sure the reason for the difference, we did have a lot more blank responses or ‘I couldn’t find it’ kind of answers in the PDF condition. So, I suspect people using the PDF could not be bothered to continue looking for the answer if they couldn’t find it quickly and easily.”

For Conaty-Buck, a family nurse practitioner and nurse informaticist, the findings reinforce the importance of improving access to reliable health information. 

“Today’s healthcare practice affords clinicians reduced time with patients, so using AI technology to provide evidence-based knowledge improves the patient's ability to practice quality self-care,” Conaty-Buck said.

The project also provided valuable research experience for doctoral students in UD’s communication program. 

“This was my first real opportunity to collect data outside a student sample; recruiting participants was interesting, and working with the advisory board has been enlightening—their input is vital,” said Katelynn Kuijpers, a fourth-year communication doctoral student. 

For communication doctoral candidate Litty Samuel, the project marked her first time working with adults with disabilities and their caregivers. 

“Even as an able-bodied person, it’s at times frustrating to navigate our healthcare system, and for people with disabilities, those challenges are exacerbated,” said Samuel. “There’s such a need in this space, and this project fulfills a unique, timely and important gap.” 

The human factor 

While the chatbot improved accuracy and reduced burden, whether participants ultimately preferred using it remains to be seen. 

Ellithorpe and her team plan to conduct online interviews with study participants to gauge their thoughts. She expects data to match findings from a previous study her team conducted, which showed people appreciate the convenience of AI but still want access to human support and don’t always know how to use AI to its fullest potential. 

“People worry that AI will make it harder to talk to humans,” Ellithorpe said. “Each disability case is so specific, and chatbots often give generic answers that don’t cater to personal cases.”

Researchers say privacy and AI literacy will also be important considerations as organizations adopt AI tools.

“It is our responsibility as researchers and clinicians to ensure people know how to utilize technology safely and efficiently for their care. That’s very challenging in the current environment,” said Conaty-Buck. “We want people to feel empowered to find information but also to understand how that information fits their individual holistic and patient-centered needs." 

As more state disability services offices consider adding chatbots to their websites, the findings underscore a need for literacy training, especially for people with disabilities without prior AI exposure. 

“What we’ve learned is that if humans aren’t available, AI is better than navigating a website or PDF on your own,” said Ellithorpe. “But when it comes to figuring out something as important as food assistance or other benefits, people may still need to talk to a person to obtain the services they need.” 

About the Fund

Maggie E. Neumann Health Sciences Research Fund was established in 2020 to support research designed to improve health and quality of life outcomes for children and adults with physical and developmental disabilities. While the fund resides at the College of Health Sciences, the intent is to support interdisciplinary research across all UD colleges.

The research fund was created with a gift from Donald J. Puglisi and Marichu C. Valencia in honor of their granddaughter, Maggie E. Neumann. Puglisi is Trustee Emeritus, and they both serve on the President’s Leadership Council.