Keeping up with AI in California: 2024 Legislative Recap
Authored By: Rebekah Glick (March 25, 2025)
In a rapidly evolving digital landscape, California is at the forefront of AI. In post, we’ll dive into last year’s legislative session in California to understand what passed last year and how it impacts health care provider policies and procedures moving forward. While Congress did not pass any substantial AI regulatory laws last year, California had an active 2023-2024 legislative session in the area of AI. Stay tuned—there’s a lot to unpack!
Governor Newsom signed 18 AI bills into law, several of which apply directly to health care.
Notably, California has adopted a standard definition of key AI related terms that will be used across government. The definition of AI merged the definition used in the Biden Administration AI Executive order as well as the definition used by the OECD.
California Statutory Definitions
“Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
“Automated decision system” means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decision-making and materially impacts natural persons. “Automated decision system” does not include a spam email filter, firewall, antivirus software, identity and access management tools, calculator, database, dataset, or other compilation of data.
“High-risk automated decision system” means an automated decision system that is used to assist or replace human discretionary decisions that have a legal or similarly significant effect, including decisions that materially impact access to, or approval for, housing or accommodations, education, employment, credit, health care, and criminal justice.
- AB 2885 (Bauer-Kahan): Relating to Artificial Intelligence – Sets standard definitions that apply across current and future laws that would regulate developers and deployers.
- Clear and consistent definitions aid compliance efforts.
- It created a single, broad definition of AI that will be used throughout California statutes and regulations moving forward. Includes additional definitions relevant to AI regulation including terms related to “automated decision system“ and “high risk automated decision system.”
On the health care AI front, there were two laws enacted in 2024 that specifically regulate the use of AI in health care settings:
- AB 3030 (Calderon): Health care services: AI – Requires health facilities deploying generative AI to generate patient communications pertaining to patient clinical information to include disclosures about the use of AI and instructions for how to contact a human health care provider.
- SB 1120 (Becker): Health Care Coverage: Utilization Review – (also known as the Physicians Make Decisions Act) prohibits health insurance plans deploying AI from using AI to deny, delay, or modify healthcare services based on medical necessity. Only licensed health care professionals can make medical necessity determinations.
2024 laws focused on the use of AI and privacy signed by the Governor include:
- AB 1008 (Bauer-Kahan): California Consumer Privacy Act of 2018: Personal Information – expands the definition of “personal information” in the California Consumer Privacy Act of 2018 (CCPA) to state such information can exist in abstract digital formats, including artificial intelligence. Please note that the CCPA does not apply to Personal Health Information regulated under HIPAA or data related to human subjects research.
- SB 1223 (Becker): Consumer Privacy: Sensitive Personal Information: Neural Data – expands on AB 1008 by expanding the definition of “sensitive personal information” to include a consumer’s neural data, which may be stored or used by AI. Please note that the CCPA does not apply to PHI regulated under HIPAA or data related to human subjects’ research.
Several 2024 laws touch on transparency, which will impact the AI tools that health care providers use:
- AB 2013 (Irwin): AI Training Data Transparency – which requires an AI system or service developer to publicly disclose specific information related to the system or service’s training data. This law will provide more transparency to consumers of AI systems by providing documentation about the data used to train the AI systems they use.
- SB 942 (Becker): California AI Transparency Act – requires developers that create, code, or produce a generative AI system with over 1 million monthly users to provide free AI detection tools to detect whether specified content was generated by those systems, which will improve transparency and provide more information to health care providers and the public.
Finally, looking ahead to the future use of AI in state agencies:
- SB 896 (Dodd): California AI Accountability Act – requires state agencies to file reports with the governor and establishes guardrails for state agencies, departments and subdivisions that use or adopt Generate AI. While not directly applicable to health care providers, this law will guide the adoption of AI by California state agencies.
- AB 302 (Ward): Relating to Automated Decision Systems – does not apply to developers or deployers but authorizes the California Department of Technology to conduct a comprehensive inventory of all high-risk Automated Decision Systems used by state agencies.
Leave a Comment