Memorisation in AI models

Polity & Governance glossary

Also called: Regurgitation, Model memorisation · Topic: Fundamental Rights and Constitutional Remedies · NCERT: Beyond NCERT

Meaning

Memorisation in AI models happens when a model stores pieces of its training data, including personal details, inside its model weights. Model weights are the millions of numbers a model learns during training. When the model later repeats those stored details in its answers, this is called regurgitation. It matters because a querying a model with "anonymised" data can pull out memorised details and help identify a person. The OECD flags this risk as especially serious for chatbots built on open-source models.

Example

Under the DPDP Rules, 2025, a Data Fiduciary (the company that decides how your data is used) must act on an erasure request within 90 days. Suppose a company deletes a person's records from its training files on time. Its trained chatbot can still reveal that person's details, because the data now lives in the weights and not in a file. So the erasure right can be met on paper while the person stays unprotected. This sits uneasily with the right to privacy, which the Supreme Court made part of Art. 21 in the K.S. Puttaswamy case (2017).

Don't confuse with

  • Web scraping: scraping is the input stage, where personal data is collected from the internet to train a model. Memorisation is about what the trained model stores and later reproduces.
  • Right to be forgotten: this is a person's right to have old or personal information removed. Memorisation is the technical problem that makes this right hard to enforce against a trained model.

Read more