information extraction
In plain wordsAutomatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database.
The task of turning free text, and semi-structured documents such as forms and web pages, into database-ready records by identifying entities, their attributes, and the relations and events that connect them. Component tasks include named-entity recognitionnamed-entity recognition, n.: Software that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date., entity linking to a knowledge base, coreference resolutioncoreference resolution, n.: Working out which words in a text are talking about the same someone or something — for instance, that 'she', 'the senator' and 'Ms. Okafor' all mean one woman., relation extraction (for example, that one company acquired another), event extraction with roles such as time and place, and template or slot filling. Early systems used hand-crafted patterns for each domain; statistical and neural extractors trained on annotated corpora followed, including distant supervision, which labels text automatically from existing databases. LLMs now perform much extraction directly into JSON schemas via structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork., evaluated by field-level precision and recall. The output feeds knowledge graphsknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over., search, analytics and document-processing pipelines.
Information extraction turns each contract into parties, dates, fees and renewal terms in our database.
Origin
Precursors include Naomi Sager's Linguistic String Project at New York University (launched 1965), which developed methods for turning scientific and later clinical text into structured data. The DARPA-funded Message Understanding Conferences (MUC, 1987–1997) defined the field's tasks and metrics, from template filling to named entities and coreference.
Related
- named-entity recognitionnamed-entity recognition, n.: Software that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date.
- coreference resolutioncoreference resolution, n.: Working out which words in a text are talking about the same someone or something — for instance, that 'she', 'the senator' and 'Ms. Okafor' all mean one woman.
- knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over.
- structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
- document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed.
- natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence.
Sources
- Grishman, R. & Sundheim, B. (1996). Message Understanding Conference-6: A Brief History. COLING 1996. aclanthology.org/C96-1079
Mentioned in
natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence., knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over., named-entity recognitionnamed-entity recognition, n.: Software that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date., coreference resolutioncoreference resolution, n.: Working out which words in a text are talking about the same someone or something — for instance, that 'she', 'the senator' and 'Ms. Okafor' all mean one woman., Message Understanding ConferenceMessage Understanding Conference, prop. n.: A series of US government-funded contests held from 1987 to 1998 in which research teams built programs to pull key facts, such as who did what, where and when, out of news and military messages., part-of-speech taggingpart-of-speech tagging, n.: Labeling each word in a sentence with its grammatical role — noun, verb, adjective and so on — as in 'Time (noun) flies (verb) like (preposition) an arrow'.