data science
In plain wordsThe practice of collecting, cleaning, analyzing and interpreting data, using statistics, programming and machine learning, to answer questions and guide decisions.
The discipline of turning data into knowledge and decisions by combining statistics, computing and expertise in the area under study. Its workflow spans framing questions, acquiring and cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results. data, exploratory analysis and visualization, statistical modeling and machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats., experiments such as A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better., and communicating results. Typical tools include Python or R, SQL, notebooksJupyter notebook, n.: An interactive document, opened in a web browser, that mixes runnable code with its results, charts and notes, so people can experiment step by step and share their work. and libraries such as pandas and scikit-learnscikit-learn, prop. n.: A free, widely taught Python library of classic machine-learning methods — such as decision trees, regression and clustering — that all work in the same simple way.. Data science overlaps with statistics, though with more emphasis on computation and large, messy data; with machine learning, though with more emphasis on causal inferencecausal inference, n.: Working out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time., uncertainty and communication than on predictive accuracy alone; and with data miningdata mining, n.: Searching large collections of data with computers to discover useful patterns, such as products often bought together or signs that a transaction may be fraudulent.. Since 2023 many routine analyses are carried out with language-model assistants and agents that write and run analysis code.
The data science team found that most churn happened in the week after a price change.
Origin
Peter Naur used 'datalogy' and 'data science' for the science of dealing with data, notably in Concise Survey of Computer Methods (1974). The statisticians C. F. Jeff Wu (1997) and William Cleveland (2001) proposed 'data science' as the name for an expanded statistics. The job title 'data scientist' is credited to DJ Patil and Jeff Hammerbacher around 2008, and a 2012 Harvard Business Review article made it famous.
Usage note
A loosely bounded term: job titles range from analysts producing dashboards to engineers building production models. Contrast AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., which centers on building applications from pretrained models.
Related
- machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats.
- data miningdata mining, n.: Searching large collections of data with computers to discover useful patterns, such as products often bought together or signs that a transaction may be fraudulent.
- big databig data, n.: Data that piles up too fast, in too many forms or in too great a quantity for one ordinary computer to store and study, and the methods invented to cope with it.
- data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
- A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better.
- Jupyter notebookJupyter notebook, n.: An interactive document, opened in a web browser, that mixes runnable code with its results, charts and notes, so people can experiment step by step and share their work.
- causal inferencecausal inference, n.: Working out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time.
- KaggleKaggle, prop. n.: A popular website, owned by Google, where people compete to build the most accurate prediction models on shared datasets, publish their code and data, and learn data science.
- AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale.
Sources
- Cleveland, W. S. (2001). Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics. International Statistical Review, 69(1), 21–26.
- Davenport, T. H. & Patil, D. J. (2012). Data Scientist: The Sexiest Job of the 21st Century. Harvard Business Review, October 2012.
Mentioned in
datadata, n.: Recorded facts, measurements, text, images or sounds that a computer can store and process. AI systems learn their skills from large amounts of data rather than from rules written by hand., machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats., AI engineerAI engineer, n.: A software developer who builds products using ready-made AI models — connecting them to data and tools and testing that they work well — rather than inventing or training the models., association rule learningassociation rule learning, n.: Finding 'if this, then often that' patterns in large records of transactions, such as noticing that shoppers who buy pasta and tomatoes often buy parmesan too., big databig data, n.: Data that piles up too fast, in too many forms or in too great a quantity for one ordinary computer to store and study, and the methods invented to cope with it., data miningdata mining, n.: Searching large collections of data with computers to discover useful patterns, such as products often bought together or signs that a transaction may be fraudulent., machine learning engineermachine learning engineer, n.: An engineer who turns machine-learning ideas into working systems: preparing data, training and testing models, and running them reliably inside real products.