Canada’s AI for All strategy has a privacy problem that must be addressed

Key points

• Canada’s federal government recently introduced its AI for All strategy, with the intention of increasing the development and adoption of artificial intelligence (AI) while promising to protect privacy, address algorithmic bias, and support Indigenous self-determination over how AI is built and used.

• These promises are particularly important because the government’s priority for AI development is in the health and life sciences sector.

• To uphold these commitments in the private sector, the government is relying on Bill C-36, the proposed Protecting Privacy and Consumer Data Act

• The Act, however, contains critical gaps that leave sensitive health data exposed, including a problematic new basis of collection, use, and disclosure of personal data; the exclusion of anonymized data; and the erosion of regulatory independence.

• Solutions include leaving the oversight of the Act with the Privacy Commissioner of Canada; ensuring that any exceptions to informed consent do not override the fundamental rights of individuals and Indigenous Peoples; following the European Union’s lead by providing protections for which data should be anonymized; and creating frameworks and standards for AI systems.

On June 4, 2026, Canada’s federal government introduced its AI for All strategy, in an attempt to increase the development and adoption of artificial intelligence (AI) while promising to protect privacy, address algorithmic bias, and “support Indigenous self-determination over how AI is built and used.”¹ These are critical commitments because the government’s first priority sector is health and life sciences, where AI systems use extremely sensitive information.¹ Accordingly, many nonprofit and government organizations that create AI systems are working to put safeguards in place.²

How private-sector entities handle health data remains a concern to people in Canada, however.³ Such entities may hold highly detailed, patient-level records from millions of people in Canada and other countries. The data may be used internally or deidentified and sold to industry, governments, and academics for generation of AI tools, or for research or marketing purposes. ⁴ The government’s privacy strategy for such entities is found in Bill C-36, the Protecting Privacy and Consumer Data Act.⁵ The proposed Act will replace the current federal Personal Information Protection and Electronic Documents Act (PIPEDA) and apply to all data-handling activities in the private sector. Some exceptions exist; for example, when companies provide clinical services (i.e., they are health data custodians) or act as agents to health data custodians (such as by providing AI decision-support systems), their activities would fall under provincial privacy laws (in provinces that have enacted legislation that is substantially similar to PIPEDA).^6,7

The Protecting Privacy and Consumer Data Act, however, contains critical gaps that leave sensitive health data exposed. We highlight 3 failures: a problematic new basis for the collection, use, and disclosure of personal data; the exclusion of anonymized data from the Act; and the erosion of regulatory independence.

Bill C-36 allows companies to process — collect, use, and share — personal data without the knowledge or consent of individuals when companies have a “legitimate interest that outweighs any reasonably foreseeable adverse effect on the individual.” ⁵ In Bill C-27, an earlier attempt at privacy reform, this exception was confined to the collection and use of data. The “legitimate interest” basis for processing data introduces additional privacy risks and undermines the ability of individuals to exercise control over their personal data. The 2025 purchase of Ambry Genetics in the United States by an AI company provides an example of what could happen when individuals have limited ability to control what happens to their data. Without patient consent, the new owners used the data to create AI systems and licensed patients’ genetic data to pharmaceutical companies.⁸ Although the exception for legitimate interest in Canada’s Bill C-36 has guardrails, they fall short of expressly affirming the human rights of individuals, and it is too early to know how they will be interpreted.

Conversely, in the European Union (EU)’s General Data Protection Regulation (GDPR), legitimate interests for processing data cannot override the fundamental rights and freedoms of the individual.⁹ Under Article 21 of the GDPR, individuals have a right to object to processing on the basis of legitimate interests.⁹ Nothing similar is found in Bill C-36.

Further, Bill C-36 uses a definition of deidentification that is similar to pseudonymization. This means that although direct identifiers such as names and health card information are removed, individuals can often be reidentified through indirect means such as partial postal codes and unusual health conditions. The First Nations Information Governance Centre has noted that this approach to deidentification overlooks the fact that such data could still include indirect identifiers unique to Indigenous Peoples (e.g., postal codes, remote or fly-in community designations, and claims to programs that are offered exclusively to Indigenous Peoples), allowing aggregation of the data according to First Nations status.¹⁰ This would be contrary to principles of Indigenous Data Sovereignty; however, Bill C-36 is silent on the issue.

Bill C-36 also excludes anonymized data from the Act under a relative definition of anonymized data, which allows reuse of data with no governance to ensure that such use is reasonable or appropriate.⁵ Anonymization is different from deidentification in Bill C-36; the bill defines anonymization as a process “to irreversibly and permanently modify personal information to ensure that there is no reasonably foreseeable risk in the circumstances that an individual can be identified from the information, whether directly or indirectly, by any means.”⁵ As a result, anonymized data — the very data used to build AI systems — will continue to fall entirely outside the scope of the law. One relevant example involves a chain of clinics in Ontario that sells deidentified patient records to IQVIA, an international data broker and creator of AI systems. Because the chain’s medical records are considered to be deidentified under a similar definition in Ontario’s Personal Health Information Protection Act,^11,12 they are likely also to be considered anonymized for the purposes of Bill C-36. Thus, IQVIA’s activities will continue to be exempt from regulatory oversight. Deliberate attempts to reidentify anonymized data, however, will be penalized.⁵

Despite the exclusion of anonymized data from the Act, their use can cause serious harm. Anonymized data often reflect societal biases that, if incorporated into AI systems, can lead to discrimination against people with shared characteristics. For example, a widely used algorithm to allocate hospital health resources in the US discriminated against Black people by allocating them fewer resources than to White people.¹³ Anonymized data can also cause harm when used to produce AI systems not in the public interest. Accordingly, a qualitative study has shown that people in Canada are much less comfortable with the private sector’s use of their health data than the public and health sectors, largely because of concerns that private profit would be the primary motive, not public benefit.³

To provide protections for anonymized data, and address algorithmic bias and problematic data uses, the government could look to the EU. Although anonymized data are excluded from the EU’s GDPR, the law still protects these data by limiting purposes for which sensitive data, like personal health information, can be anonymized.⁹ The EU also has AI governance legislation to address the development of high-risk AI systems, such as those used in health care.¹⁴

The third failure in Bill C-36 is that it will also remove oversight from the experienced and independent Privacy Commissioner of Canada, turning it over to a new commission that reports to the governing party rather than acting as an independent agent of Parliament.⁵ The Bill is aligned with the new AI strategy to promote and support the AI industry in Canada, with indications that the government will play a greater role in shaping how privacy decisions are made. For example, the parameters for decision-making by the new commission include not only privacy, but also economic and trade treaty considerations.

The move to a less independent regulator and increased focus on economic goals risks undermining privacy protection and adversely affecting public trust. One solution is to leave oversight of the Act with the Privacy Commissioner of Canada and create a separate tribunal to impose monetary penalties. The separate tribunal would avoid any conflict between the commissioner’s role as an independent agent and the imposition of substantial fines on private-sector actors. This is what was proposed in the previous attempt to amend PIPEDA, Bill C-27.

Furthermore, the exception for legitimate interests should not be available when those interests conflict with human rights, including privacy and nondiscrimination. The government must also engage in meaningful consultation with First Nations, Inuit, and Métis communities to determine what safeguards are needed in the law to protect against unconsented use of Indigenous Peoples’ data.

Canada’s government must move fast to build frameworks for AI governance that provide meaningful assurance that AI systems are safe, reliable, and protective of privacy. The government’s chosen path may be to develop standards and compliance certification rather than to legislate — if so, urgent action is needed to build these frameworks and encourage compliance. Specific AI governance legislation that provides oversight and recourse may still be required.

These changes would help ensure that AI systems produced by private-sector companies do not cause harm in their pursuit of private gain. The changes would reduce the risk of privacy loss, algorithmic bias, and problematic uses of patient data, as well as support the right of Indigenous Peoples to self-determination. The changes may also help to build public trust and increase support for AI adoption in health care.