# Metric¶

Base class for metric, encapsulates metric logic and APIs Usage:

```m = SomeMetric()
for prediction, label in ...:
m.update(prediction, label)
m.accumulate()
```

Advanced usage for `compute`:

Metric calculation can be accelerated by calculating metric states from model outputs and labels by build-in operators not by Python/NumPy in `compute`, metric states will be fetched as NumPy array and call `update` with states in NumPy format. Metric calculated as follows (operations in Model and Metric are indicated with curly brackets, while data nodes not):

```   inputs & labels              || ------------------
|                      ||
{model}                   ||
|                      ||
outputs & labels              ||
|                      ||    tensor data
{Metric.compute}              ||
|                      ||
metric states(tensor)           ||
|                      ||
{fetch as numpy}              || ------------------
|                      ||
metric states(numpy)            ||    numpy data
|                      ||
{Metric.update}               \/ ------------------
```

Examples

For `Accuracy` metric, which takes `pred` and `label` as inputs, we can calculate the correct prediction matrix between `pred` and `label` in `compute`. For examples, prediction results contains 10 classes, while `pred` shape is [N, 10], `label` shape is [N, 1], N is mini-batch size, and we only need to calculate accurary of top-1 and top-5, we could calculate the correct prediction matrix of the top-5 scores of the prediction of each sample like follows, while the correct prediction matrix shape is [N, 5].

```def compute(pred, label):
# sort prediction and slice the top-5 scores
# calculate whether the predictions are correct
correct = pred == label
```

With the `compute`, we split some calculations to OPs (which may run on GPU devices, will be faster), and only fetch 1 tensor with shape as [N, 5] instead of 2 tensors with shapes as [N, 10] and [N, 1]. `update` can be define as follows:

```def update(self, correct):
accs = []
for i, k in enumerate(self.topk):
num_corrects = correct[:, :k].sum()
num_samples = len(correct)
accs.append(float(num_corrects) / num_samples)
self.total[i] += num_corrects
self.count[i] += num_samples
return accs
```
abstract reset ( )

Reset states and result

abstract update ( *args )

Update states for metric

Inputs of `update` is the outputs of `Metric.compute`, if `compute` is not defined, the inputs of `update` will be flatten arguments of output of mode and label from data: `update(output1, output2, ..., label1, label2,...)`

see `Metric.compute`

abstract accumulate ( )

Accumulates statistics, computes and returns the metric value

abstract name ( )

Returns metric name

compute ( *args )

This API is advanced usage to accelerate metric calculating, calulations from outputs of model to the states which should be updated by Metric can be defined here, where Paddle OPs is also supported. Outputs of this API will be the inputs of “Metric.update”.

If `compute` is defined, it will be called with outputs of model and labels from data as arguments, all outputs and labels will be concatenated and flatten and each filed as a separate argument as follows: `compute(output1, output2, ..., label1, label2,...)`

If `compute` is not defined, default behaviour is to pass input to output, so output format will be: `return output1, output2, ..., label1, label2,...`

see `Metric.update`