Skip to content
Last Dictionary
Plane
Vision, Speech & PerceptionEvaluation & BenchmarksMachine Learning

mean average precision

nounAlso: mAP, average precision, AP, AP50, AP75, box AP, mask AP

In plain wordsThe standard score for object detectors: it rewards finding all the objects while raising few false alarms, averaged over every type of object. Higher is better, up to 100.

  1. 1

    Object detection

    The mean, over object classes, of average precision (AP), where AP summarizes one class's precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare.. Detections are ranked by confidence; each is marked a true or false positive by matching it to an unclaimed ground-truth box at a given intersection over unionintersection over union, n.: A score from 0 to 1 for how well two shapes overlap, such as a predicted box and the true box around an object: the shared area divided by the combined area. threshold; AP is the area under the resulting, interpolated precision–recall curve. PASCAL VOC used a single IoU threshold of 0.5 (AP50) and, at first, 11-point interpolation. COCOCOCO, prop. n.: A widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems.'s primary metric, written simply 'AP', also averages over ten IoU thresholds from 0.50 to 0.95 in steps of 0.05, rewarding tight localization, and is reported alongside AP50, AP75 and AP for small, medium and large objects. Applied to masks the same procedure gives mask AP for instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob.; applied to keypoints, with object keypoint similarity in place of IoU, it gives keypoint AP.

    The new detection head adds 1.8 box AP on COCO val, almost all of it on small objects.

  2. 2

    Information retrieval

    The mean over a set of queries of each query's average precision, which is the average of the precision values at the ranks where relevant documents are retrieved; a standard measure of ranked results in information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up. evaluations such as TREC.

mAP=1C∑c=1CAPc,APc=∫01pc(r) dr\text{mAP} = \frac{1}{C}\sum_{c=1}^{C}\text{AP}_c, \qquad \text{AP}_c = \int_0^1 p_c(r)\,dr

Origin

Average precision comes from information retrieval, where its mean over queries was a standard measure in the Text REtrieval Conference (TREC) evaluations from the 1990s. The PASCAL VOC challenge (2005–2012) adapted it to detection, and the COCO benchmark (2014–2015) introduced averaging over IoU thresholds.

Usage note

COCO does not distinguish 'AP' from 'mAP'; both denote the class-averaged value. Scores computed with different IoU thresholds, interpolation rules or data splits are not comparable, a frequent source of misleading results tables.

Related

See also
  • precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.
  • recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.
  • precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare.
  • intersection over unionintersection over union, n.: A score from 0 to 1 for how well two shapes overlap, such as a predicted box and the true box around an object: the shared area divided by the combined area.
  • object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene.
  • COCOCOCO, prop. n.: A widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems.
Contrast with
  • accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others.

Sources

  1. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J. & Zisserman, A. (2010). The PASCAL Visual Object Classes (VOC) Challenge. International Journal of Computer Vision 88, 303–338.
  2. Lin, T.-Y. et al. (2014). Microsoft COCO: Common Objects in Context. ECCV 2014. arxiv.org/abs/1405.0312

Mentioned in

object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene., COCOCOCO, prop. n.: A widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems., instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob., intersection over unionintersection over union, n.: A score from 0 to 1 for how well two shapes overlap, such as a predicted box and the true box around an object: the shared area divided by the combined area., keypoint detectionkeypoint detection, n.: Finding specific, recognizable points in an image, such as the corners of a building, a person's elbows and knees, or the tip of a nose., precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare., TRECTREC, prop. n.: A yearly contest, run by a US government standards agency since 1992, in which search-engine researchers test their systems on the same documents and questions, with people judging which results are actually relevant.