When a Health System Uses an Algorithm: The Nondiscrimination Duty

When a hospital or a clinician uses a clinical algorithm to help make a care decision, a federal nondiscrimination duty attaches to the user, not only to whoever built the tool. Under the Department of Health and Human Services rule finalizing Section 1557 of the Affordable Care Act, published in the Federal Register on May 6, 2024, a covered entity that deploys a patient care decision support tool must make reasonable efforts to identify tools that rely on protected characteristics and to mitigate the risk of discrimination from each tool's use. This is a deployer duty. It places part of the accountability for biased software on the organizations and people who put it into practice.

This piece explains a legal mechanism, not any policy position, party, or administration. It is educational and not medical or legal advice.

What Section 1557 covers, and who is a covered entity

Section 1557 is the civil-rights provision of the Affordable Care Act. It prohibits discrimination on the basis of race, color, national origin, sex, age, or disability in health programs and activities that receive federal financial assistance. In practice that reaches most hospitals, clinics, and health-insurance issuers, because Medicare and Medicaid participation and similar funding streams bring an organization within scope. The people who make care decisions inside those organizations act on the entity's behalf, so the obligation is institutional rather than a matter of any one professional's private choice.

The 2024 rule added a specific application of that older prohibition to decision-support software and to the analog tools that preceded it. The relevant text sits at 45 CFR 92.210, titled Nondiscrimination in the use of patient care decision support tools.

The regulation defines the tool broadly

A "patient care decision support tool" is defined as any automated or non-automated tool, mechanism, method, technology, or combination of these that a covered entity uses to support clinical decision-making. That definition is deliberately wide. It captures machine-learning models and predictive software, and it also captures a paper flowchart used for triage or a printed clinical guideline. The regulation is not aimed only at artificial intelligence. It reaches the logic a health system relies on to sort, score, or route patients, whatever form that logic takes.

The National Health Law Program, in its analysis of the rule, describes the mechanism plainly: covered entities have ongoing and affirmative duties to find the tools that use inputs measuring a protected characteristic and to reduce the discrimination risk from each such tool. That framing captures the shift. The duty is continuous rather than a one-time certification, and it runs to the entity that uses the tool.

The three moving parts of the deployer duty

The regulation breaks into three components.

Paragraph (a) is the general prohibition. A covered entity must not discriminate on the basis of race, color, national origin, sex, age, or disability in its health programs or activities through the use of patient care decision support tools. This is the rule against the outcome.

Paragraph (b) is the identification duty. A covered entity has an ongoing duty to make reasonable efforts to identify uses of these tools that employ input variables or factors that measure race, color, national origin, sex, age, or disability. This is the diligence half: an organization has to look at what its tools take in.

Paragraph (c) is the mitigation duty. For each tool identified under paragraph (b), the entity must make reasonable efforts to mitigate the risk of discrimination resulting from that tool's use. Identifying a risky input is not enough; the entity has to act on what it finds.

The phrase doing the work across all three is "reasonable efforts." The standard is not strict liability for every biased output, and it is not a demand for perfect fairness. It asks whether the organization made a genuine, resourced attempt to find and address the risk.

Developer duty and deployer duty are different

A recurring source of confusion is the assumption that responsibility for a biased algorithm rests entirely with its maker. Section 1557's rule does not adopt that view. The covered entity that uses the tool carries the identification-and-mitigation obligation regardless of who wrote the code. At the same time, the rule does not require a hospital to reverse-engineer a vendor's model or obtain proprietary training data. A separate transparency framework from the Office of the National Coordinator for Health Information Technology requires developers of certified decision-support interventions to disclose specified information about how those interventions work. The two regimes are meant to operate together: developers surface information, and deployers use that information to meet their own duty. Neither one absorbs the other's responsibility.

What "reasonable" is measured against

The rule does not leave "reasonable efforts" entirely open. It points to factors relevant to whether a covered entity satisfied the duty. These include the entity's size and resources, whether it used the tool in the manner the developer intended and regulators approved, whether the developer provided information about the tool's inputs, and whether the entity has a process in place for evaluating such tools. The practical message is that a large integrated system and a small rural clinic are not held to identical operational burdens, but every covered entity is expected to have some process rather than none.

The best-known clinical example of why this matters is the family of race-adjusted algorithms that quietly embedded a protected characteristic as an input, including corrected equations for kidney function and lung function that produced different outputs for patients coded as different races. A tool of that kind is precisely what paragraph (b) is written to surface and paragraph (c) is written to address.

The compliance timeline

The final rule became effective July 5, 2024. The specific obligations under section 92.210 carried a longer runway: covered entities were required to be in compliance by roughly May 1, 2025, about 300 days after the effective date. The gap was built in to let organizations inventory their tools and stand up an evaluation process before enforcement expectations attached.

Why the deployer framing matters

A rule that regulated only software developers would leave a gap: the same model can be safe in one clinical workflow and harmful in another, depending on the population it is applied to and the decision it drives. By placing an identification-and-mitigation duty on the entity that actually uses the tool, Section 1557's rule ties accountability to the point where a real patient is affected. That is the logic of a deployer duty, and it is why "who built it" is no longer the only question a health system has to answer.