Artificial intelligence (AI) is an evolving, fast paced topic. It includes chatbots, algorithms, deep learning, machine learning / intelligence, predictive analytics, and more. The purpose of this guidance is to share current legal considerations, as well as best practices when conducting human subject research at UCI.

In addition, note the below red flags which specify both current restrictions and sensitive areas that require additional engagement with UCI research partners to help protect human subjects. A restriction means it is currently not allowed at the Institution.  A red flag means it could be allowed but will require further analysis and engagement with Human Research Protections (HRP) and subject matter experts as noted below.

Red Flags - April 2026

  • Use, storage, sharing, selling of identifiable information about UC Student Employees, Research Subjects, Patients, including Protected Health Information (PHI).
  • Do not use, enter or share PHI or other clinical queries to any Large Language Models (LLMs) or AI programs including DoxGPT, ChatGPT or OpenEvidence. All are restricted at UCI Health. Uses of such tools is an Information Security & Privacy violation. Currently there are no  HIPPA-compliant LLM tools available at UCI Health. The Institution is looking at enabling such tools in the future. 
  • Doximity tools are restricted at UCI Health. Do not use Doximity Dialer for phone or video visits. Appropriate replacements are Epic VoIP and Epic Secure Chat, and Epic native video or audio visits.
  • AI making decisions or recommendations that affect research subjects. 
  • Foreign language translations of subject facing documents, including consent forms.

Points to Consider

  • Where did the training data come from?
  • Is there evidence that the system does what it says it can do?
  • De-identification is more difficult with AI; Anonymized data or aggregated data sets could be appropriate.
  • If consent is used, ensure the intent for current and possible future use is well described.  Being too broad however, may exclude a well-informed consent process for future uses however.

AI at UCI Health

  • PHI and restricted data must not be entered into any external or consumer AI tools.
  • Claims that an AI tool is “HIPAA compliant” do not mean it is approved for use at UCI Health.
  • Copying information from Epic or other clinical systems into AI tools is not permitted.
  • Use of AI tools with PHI may be treated as a privacy incident and subject to review
  • The following tools are permitted for use with PHI when signed in with your hs.uci.edu account and using the correct licensed features:
    • Abridge, within approved clinical documentation workflows
    • Epic-embedded AI features, where enabled, including DynaMedex and Dyna.Ai for clinical decision support. AI-generated documentation must be reviewed, edited, and validated in the same manner as copied or templated material. Providers remain fully responsible for verifying the accuracy, relevance, and clinical appropriateness of all AI-assisted content prior to signing the note.

Engage with Procurement

Please note that UCI already offers a selection of contracted AI services. If intending to use AI outside of these available options, please ask for Procurement to review and execute all agreements or contracts with AI vendors.  During the Procurement process, the Terms of Use or Privacy Policy will be reviewed to ensure it does not expose UC to risk.  Contracts must include an Appendix Data Security, and, where PHI is involved, an Appendix - Business Associate Agreement.

Remember, faculty do not have the ability to sign contracts on behalf of UC.  If contract has already been signed, and data involves Red Flags, contact Procurement to protect UC.

Follow Federal, UC, Local and International Security Policies

When providing personal information, including protected health information, UC’s Electronic Information Security Policy (IS-3), along with local UC data security policies for researchers, must be followed.

Importantly, the National Institutes of Health (NIH) have noted that institutions who have been issued a Certificate of Confidentiality (CoC) must consider the CoC protections when selecting 3rd parties or vendors.  Why? These 3rd parties or entities (contractors, online platform vendors) must provide the same CoC protections. This means that they must also protect covered information against compelled disclosure.

Provisions under GDPR and PIPL may also apply if  the data subjects are located internationally (such as EEA, UK, or China).

When AI is used for consent translations, AI translation must be verified via a human translator whose qualifications align with HRP-090 SOP - Informed Consent Process for Research.

Alert Legal, Privacy and Compliance, and Information Security

AI can be helpful, but it may also carry risks.  Think about issues such as privacy, cybersecurity, discrimination, and liability.

AI can be biased and inaccurate. The IRB may ask for evidence of how the AI translation addresses bias; if the data will be stored, for how long; and if the data may be shared.

Research that includes Red Flags must work with UCI Campus Legal Counsel and Privacy Partners as follows: UCI Health Compliance & Privacy Office (for PHI); UCI Registrar’s Office (for student data);  UCI Privacy (all other personal data (including some PHI and student data)); and UCI Information Security to ensure that the use of the AI product is appropriate for use at UCI.

Research involving health data must also work with the Health Data Governance Committee.

Data Use Agreement

AI should be considered as a third-party vendor. 

Any content provided to AI tools can be saved and reused by the tool – and their affiliates.  Accordingly, agreements must be in place to ensure the proper privacy and confidentiality terms, among others.  UC must be able to directly negotiate and impose requirements on vendors to defend and indemnify UC against 3rd party claims. This is important since AI is not always accurate and could infringe on intellectual property rights.

To discuss Data Use Agreements at UCI, please contact Ms. Wanda Seang in the Office of Research.

Questions about getting started?

Please reach out to HRP Staff directly.

Get In Touch