data licensing
In plain wordsThe legal permissions that say who may use a dataset or a body of content, for what purposes and on what terms — including whether it may be used to train AI.
The terms, set by contract or license grant, under which data or content may be copied, used, modified, shared and commercialized, and the market practice of paying for such rights. Open datasets usually carry public licenses: Creative Commons variants (CC BY, CC BY-SA, CC0, with or without non-commercial clauses), Open Data Commons licenses such as ODC-By, and, for code, permissive (MIT, Apache 2.0) or copyleft (GPL) licenses. A dataset's license covers the compilation; the underlying works may carry different rights, which is why data provenancedata provenance, n.: The documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system. tracing matters — a 2023 audit of more than 1,800 text datasets found licenses omitted 70% or more of the time on popular hosting sites. Since 2023, AI developers have also signed deals with news publishers, stock-media libraries, academic publishers and online platforms for training and display rights, while the lawfulness of unlicensed training remains contested (see fair use and AI trainingfair use and AI training, phr.: The unsettled US legal question of whether AI companies may copy books, articles, pictures and music to train their models without permission, under the rule that allows some unlicensed uses of copyrighted work., text and data mining exceptiontext and data mining exception, n.: A rule in many countries' copyright laws that lets people use computers to copy and analyze large amounts of text, images or data without asking permission — sometimes only for research, sometimes unless owners opt out.).
The corpus is ODC-By, but several subsets inherit non-commercial terms from their sources.
Origin
Public licenses for creative content began with Creative Commons (2002) and for databases with the Open Data Commons licenses (from about 2008). AI-specific practice took shape after 2023 through publisher deals and audits such as Shayne Longpre and colleagues' Data Provenance Initiative (2023).
Usage note
A permissive license on a dataset does not settle whether training on it is lawful, and attribution or share-alike terms raise unresolved questions about what a trained model owes its sources. A model licensemodel license, n.: The legal terms under which an AI model may be downloaded and used — for example, whether businesses may use it, whether changes may be shared, and which uses are banned. governs model weights, not data.
Related
- data provenancedata provenance, n.: The documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system.
- opt-outopt-out, n.: A way for creators, websites or individuals to say 'do not use my work or data to train AI' — instead of being asked for permission first, which would be opt-in.
- fair use and AI trainingfair use and AI training, phr.: The unsettled US legal question of whether AI companies may copy books, articles, pictures and music to train their models without permission, under the rule that allows some unlicensed uses of copyrighted work.
- text and data mining exceptiontext and data mining exception, n.: A rule in many countries' copyright laws that lets people use computers to copy and analyze large amounts of text, images or data without asking permission — sometimes only for research, sometimes unless owners opt out.
- AI copyright litigationAI copyright litigation, n.: The many lawsuits in which authors, artists, news organizations, record labels and others accuse AI companies of copying their work without permission to build or run AI models.
- robots.txtrobots.txt, n.: A small public file in which a website posts its rules for automated visitors, such as search-engine and AI crawlers, listing areas they are asked to stay out of. Obeying it is voluntary.
- model licensemodel license, n.: The legal terms under which an AI model may be downloaded and used — for example, whether businesses may use it, whether changes may be shared, and which uses are banned.
Sources
- Longpre, S. et al. (2023). The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. arXiv:2310.16787. arxiv.org/abs/2310.16787
Mentioned in
AI copyright litigationAI copyright litigation, n.: The many lawsuits in which authors, artists, news organizations, record labels and others accuse AI companies of copying their work without permission to build or run AI models., fair use and AI trainingfair use and AI training, phr.: The unsettled US legal question of whether AI companies may copy books, articles, pictures and music to train their models without permission, under the rule that allows some unlicensed uses of copyrighted work., open-source AIopen-source AI, n.: AI released, ideally with its code and details of its training data and not just the finished model, so that anyone may use, inspect, alter, share and even rebuild it. How open is open enough is debated., data provenancedata provenance, n.: The documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system., model licensemodel license, n.: The legal terms under which an AI model may be downloaded and used — for example, whether businesses may use it, whether changes may be shared, and which uses are banned., opt-outopt-out, n.: A way for creators, websites or individuals to say 'do not use my work or data to train AI' — instead of being asked for permission first, which would be opt-in., robots.txtrobots.txt, n.: A small public file in which a website posts its rules for automated visitors, such as search-engine and AI crawlers, listing areas they are asked to stay out of. Obeying it is voluntary., text and data mining exceptiontext and data mining exception, n.: A rule in many countries' copyright laws that lets people use computers to copy and analyze large amounts of text, images or data without asking permission — sometimes only for research, sometimes unless owners opt out., web scrapingweb scraping, n.: Using software to automatically copy information from websites — text, prices, images — in bulk, which is how much of the data used to train AI was gathered., Books3Books3, prop. n.: A collection of nearly 200,000 pirated e-books, assembled in 2020 and used to train several AI language models, which became central to lawsuits by authors over AI training., Open Source InitiativeOpen Source Initiative, prop. n.: A nonprofit group that decides what counts as 'open source' software and, in 2024, published a definition of what makes an AI system genuinely open source., The StackThe Stack, prop. n.: A large free collection of computer source code from public repositories, selected mainly for permissive licenses and released from 2022 onward by the BigCode project for training coding models.