# fluid.metrics¶

## Accuracy¶

class `paddle.fluid.metrics.``Accuracy`(name=None)[源代码]

• name — 度量标准的名称

```import paddle.fluid as fluid
# 假设有batch_size = 128
batch_size=128
accuracy_manager = fluid.metrics.Accuracy()
# 假设第一个batch的准确率为0.9
batch1_acc = 0.9
accuracy_manager.update(value = batch1_acc, weight = batch_size)
print("expect accuracy: %.2f, get accuracy: %.2f" % (batch1_acc, accuracy_manager.eval()))
# 假设第二个batch的准确率为0.8
batch2_acc = 0.8
accuracy_manager.update(value = batch2_acc, weight = batch_size)
#batch1和batch2的联合准确率为(batch1_acc * batch_size + batch2_acc * batch_size) / batch_size / 2
print("expect accuracy: %.2f, get accuracy: %.2f" % ((batch1_acc * batch_size + batch2_acc * batch_size) / batch_size / 2, accuracy_manager.eval()))
#重置accuracy_manager
accuracy_manager.reset()
#假设第三个batch的准确率为0.8
batch3_acc = 0.8
accuracy_manager.update(value = batch3_acc, weight = batch_size)
print("expect accuracy: %.2f, get accuracy: %.2f" % (batch3_acc, accuracy_manager.eval()))
```
`update`(value, weight)

• value (float|numpy.array) – 每个mini batch的正确率
• weight (int|float) – batch 大小
`eval`()

## Auc¶

class `paddle.fluid.metrics.``Auc`(name, curve='ROC', num_thresholds=4095)[源代码]

auc函数创建四个局部变量true_positives, true_negatives, false_positives和false_negatives，用于计算AUC。对于离散化AUC曲线，临界值线性间隔设置以便计算召回率和准确率的值，用false positive率的召回值高度计算ROC曲线面积，用recall的准确值高度计算PR曲线面积。

• name - 度量名
• curve - 将要计算的曲线名的详情，曲线包括ROC（默认）或者PR（Precision-Recall-curve）。

```import paddle.fluid as fluid
import numpy as np
# 初始化auc度量
auc_metric = fluid.metrics.Auc("ROC")

# 假设batch_size为128
batch_num = 100
batch_size = 128

for batch_id in range(batch_num):

class0_preds = np.random.random(size = (batch_size, 1))
class1_preds = 1 - class0_preds

preds = np.concatenate((class0_preds, class1_preds), axis=1)

labels = np.random.randint(2, size = (batch_size, 1))
auc_metric.update(preds = preds, labels = labels)

# 应为一个接近0.5的值，因为preds是随机指定的
print("auc for iteration %d is %.2f" % (batch_id, auc_metric.eval()))
```
`update`(preds, labels)

• preds – 形状为(batch_size, 2)的numpy数组，preds[i][j]表示将实例i划分为类别j的概率。
• labels – 形状为(batch_size, 1)的numpy数组，labels[i]为0或1，代表实例i的标签。
`eval`()

## ChunkEvaluator¶

class `paddle.fluid.metrics.``ChunkEvaluator`(name=None)[源代码]

```import paddle.fluid as fluid
# 初始化chunck-level的评价管理。
metric = fluid.metrics.ChunkEvaluator()

# 假设模型预测10个chuncks，其中8个为正确，且真值有9个chuncks。
num_infer_chunks = 10
num_label_chunks = 9
num_correct_chunks = 8

metric.update(num_infer_chunks, num_label_chunks, num_correct_chunks)
numpy_precision, numpy_recall, numpy_f1 = metric.eval()

print("precision: %.2f, recall: %.2f, f1: %.2f" % (numpy_precision, numpy_recall, numpy_f1))

# 下一个batch，完美地预测了3个正确的chuncks。
num_infer_chunks = 3
num_label_chunks = 3
num_correct_chunks = 3

metric.update(num_infer_chunks, num_label_chunks, num_correct_chunks)
numpy_precision, numpy_recall, numpy_f1 = metric.eval()

print("precision: %.2f, recall: %.2f, f1: %.2f" % (numpy_precision, numpy_recall, numpy_f1))
```
`update`(num_infer_chunks, num_label_chunks, num_correct_chunks)

• num_infer_chunks (int|numpy.array): 给定minibatch的Interface块数。
• num_label_chunks (int|numpy.array): 给定minibatch的Label块数。
• num_correct_chunks （int|float|numpy.array）: 给定minibatch的Interface和Label的块数

## CompositeMetric¶

class `paddle.fluid.metrics.``CompositeMetric`(name=None)[源代码]

```import paddle.fluid as fluid
import numpy as np
preds = [[0.1], [0.7], [0.8], [0.9], [0.2],
[0.2], [0.3], [0.5], [0.8], [0.6]]
labels = [[0], [1], [1], [1], [1],
[0], [0], [0], [0], [0]]
preds = np.array(preds)
labels = np.array(labels)

comp = fluid.metrics.CompositeMetric()
precision = fluid.metrics.Precision()
recall = fluid.metrics.Recall()

comp.update(preds=preds, labels=labels)
numpy_precision, numpy_recall = comp.eval()
print("expect precision: %.2f, got %.2f" % ( 3. / 5, numpy_precision ) )
print("expect recall: %.2f, got %.2f" % (3. / 4, numpy_recall ) )
```
`add_metric`(metric)

• metric – MetricBase的一个实例。
`update`(preds, labels)

• preds (numpy.array) - 当前mini batch的预测
• labels (numpy.array) - 当前minibatch的label，如果标签是one-hot或soft-laebl 编码，应该自定义相应的更新规则。
`eval`()

## DetectionMAP¶

class `paddle.fluid.metrics.``DetectionMAP`(input, gt_label, gt_box, gt_difficult=None, class_num=None, background_label=0, overlap_threshold=0.5, evaluate_difficult=True, ap_version='integral')[源代码]

1. 根据detectors中的输入和label，计算 true positive 和 false positive
2. 计算map，支持 ‘11 point’ and ‘integral’

• input (Variable) – detection的结果，一个 shape=[M, 6] 的 lodtensor。布局为[label, confidence, xmin, ymin, xmax, ymax]
• gt_label (Variable) – ground truth label 的索引，它是一个形状为[N, 1]的lodtensor
• gt_box (Variable) – ground truth bounds box (bbox)，是一个具有形状的lod张量[N, 4]。布局是[xmin, ymin, xmax, ymax]
• gt_difficult (Variable|None) – 指定这个ground truth是否是一个difficult bounding bbox，它可以是一个 shape=[N, 1]的LoDTensor，也可以不被指定。如果设置为None，则表示所有的ground truth标签都不是difficult bbox。
• class_num (int) – 检测类别的数目
• background_label (int) – 背景标签的索引，背景标签将被忽略。如果设置为-1，则所有类别将被考虑，默认为0。
• overlap_threshold (float) – 判断真假阳性的阈值，默认为0.5
• evaluate_difficult (bool) – 是否考虑 difficult ground truth 进行评价，默认为 True。当 gt_difficult 为 None 时，这个参数不起作用。
• ap_version (string) – 平均精度的计算方法，必须是 "integral" 或 "11point"。详情请查看 https://sanchom.wordpress.com/tag/average-precision/。 其中，11point为：11-point 插值平均精度。积分: precision-recall曲线的自然积分。

```import paddle.fluid as fluid

batch_size = -1 # 可以为任意大小
image_boxs_num = 10
bounding_bboxes_num = 21

pb = layers.data(name='prior_box', shape=[image_boxs_num, 4],
append_batch_size=False, dtype='float32')

pbv = layers.data(name='prior_box_var', shape=[image_boxs_num, 4],
append_batch_size=False, dtype='float32')

loc = layers.data(name='target_box', shape=[batch_size, bounding_bboxes_num, 4],
append_batch_size=False, dtype='float32')

scores = layers.data(name='scores', shape=[batch_size, bounding_bboxes_num, image_boxs_num],
append_batch_size=False, dtype='float32')

nmsed_outs = fluid.layers.detection_output(scores=scores,
loc=loc, prior_box=pb, prior_box_var=pbv)

gt_box = fluid.layers.data(name="gt_box", shape=[batch_size, 4], dtype="float32")
gt_label = fluid.layers.data(name="gt_label", shape=[batch_size, 1], dtype="float32")
difficult = fluid.layers.data(name="difficult", shape=[batch_size, 1], dtype="float32")

exe = fluid.Executor(fluid.CUDAPlace(0))
map_evaluator = fluid.metrics.DetectionMAP(nmsed_outs, gt_label, gt_box, difficult, class_num = 3)
cur_map, accum_map = map_evaluator.get_map_var()

# 更详细的例子请参见
```
`get_map_var`()

`reset`(executor, reset_program=None)

• executor (Executor) – 执行reset_program的执行程序
• reset_program (Program|None) – 单一 program 的 reset 过程。如果设置为 None，将创建一个 program

## EditDistance¶

class `paddle.fluid.metrics.``EditDistance`(name)[源代码]

1. distance：一个形状为（batch_size, 1）的numpy.array，每个元素表示两个序列之间的编辑距离；
2. seq_num：一个整型/浮点型数，代表序列对的数目，并返回多个序列对的整体编辑距离。

• name - 度量标准名称

```import paddle.fluid as fluid
import numpy as np

# 假设batch_size为128
batch_size = 128

# 初始化编辑距离管理器
distance_evaluator = fluid.metrics.EditDistance("EditDistance")
# 生成128个序列对间的编辑距离，此处的最大距离是10
edit_distances_batch0 = np.random.randint(low = 0, high = 10, size = (batch_size, 1))
seq_num_batch0 = batch_size

distance_evaluator.update(edit_distances_batch0, seq_num_batch0)
avg_distance, wrong_instance_ratio = distance_evaluator.eval()
print("the average edit distance for batch0 is %.2f and the wrong instance ratio is %.2f " % (avg_distance, wrong_instance_ratio))
edit_distances_batch1 = np.random.randint(low = 0, high = 10, size = (batch_size, 1))
seq_num_batch1 = batch_size

distance_evaluator.update(edit_distances_batch1, seq_num_batch1)
avg_distance, wrong_instance_ratio = distance_evaluator.eval()
print("the average edit distance for batch0 and batch1 is %.2f and the wrong instance ratio is %.2f " % (avg_distance, wrong_instance_ratio))

distance_evaluator.reset()
```
`distance_evaluator.``reset`()
```import paddle.fluid as fluid
edit_distances_batch2 = np.random.randint(low = 0, high = 10, size = (batch_size, 1))
seq_num_batch2 = batch_size
distance_evaluator.update(edit_distances_batch2, seq_num_batch2)
avg_distance, wrong_instance_ratio = distance_evaluator.eval()
print("the average edit distance for batch2 is %.2f and the wrong instance ratio is %.2f " % (avg_distance, wrong_instance_ratio))
```
`update`(distances, seq_num)

• distances – 一个形状为(batch_size, 1)的numpy.array，每个元素代表两个序列间的距离。(edit) –
• seq_num – 一个整型/浮点型值，代表序列对的数量。
`eval`()

## MetricBase¶

class `paddle.fluid.metrics.``MetricBase`(name)[源代码]

• name (str) - metric实例名。例如准确率（accuracy）。如果想区分一个模型里不同的metrics，则需要实例名。
`reset`()

reset()清除度量（metric）的状态（state）。默认情况下，状态（state）包含没有 `_` 前缀的metric。reset将这些状态设置为初始状态。如果不想使用隐式命名规则，请自定义reset接口。

`get_config`()

`update`(preds, labels)

• preds (numpy.array) - 当前minibatch的预测
• labels (numpy.array) - 当前minibatch的标签，如果标签为one-hot或者soft-label，应该自定义相应的更新规则。
`eval`()

## Precision¶

class `paddle.fluid.metrics.``Precision`(name=None)[源代码]

Precision(也称为 positive predictive value,正预测值)是被预测为正样例中实际为正的比例。 https://en.wikipedia.org/wiki/Evaluation_of_binary_classifiers 该类管理二分类任务的precision分数。

```import paddle.fluid as fluid
import numpy as np

metric = fluid.metrics.Precision()

# 生成预测值和标签

preds = [[0.1], [0.7], [0.8], [0.9], [0.2],
[0.2], [0.3], [0.5], [0.8], [0.6]]

labels = [[0], [1], [1], [1], [1],
[0], [0], [0], [0], [0]]

preds = np.array(preds)
labels = np.array(labels)

metric.update(preds=preds, labels=labels)
numpy_precision = metric.eval()

print("expct precision: %.2f and got %.2f" % ( 3.0 / 5.0, numpy_precision))
```

## Recall¶

class `paddle.fluid.metrics.``Recall`(name=None)[源代码]

https://en.wikipedia.org/wiki/Precision_and_recall

```import paddle.fluid as fluid
import numpy as np

metric = fluid.metrics.Recall()
# 生成预测值和标签
preds = [[0.1], [0.7], [0.8], [0.9], [0.2],
[0.2], [0.3], [0.5], [0.8], [0.6]]
labels = [[0], [1], [1], [1], [1],
[0], [0], [0], [0], [0]]

preds = np.array(preds)
labels = np.array(labels)

metric.update(preds=preds, labels=labels)
numpy_precision = metric.eval()

print("expct precision: %.2f and got %.2f" % ( 3.0 / 4.0, numpy_precision))
```