Meters¶

AverageValueMeter¶

class AverageValueMeter¶

An implementation of average value meter, which measures the mean and variance of a sequence of values.

This meter takes as input a stream of i.i.d data \( X = {x_i} \) with unormalized weights \( W = {w_i} \) ( \( w_i \ge 0 \)). Suppose \( p_i = w_i / \sum_{j = 1}^n w_j \), it maintains the following variables:

unbiased mean \( \tilde{mu} = \sum_{i = 1}^n p_i x_i \)
unbiased second momentum \( \tilde{mu}_2 = \sum_{i = 1}^n p_i x_i^2 \)
sum of weights \( Sum(W) = \sum_{i = 1}^n w_i} \)
sum of squared weights \( Sum(W^2) = \sum_{i = 1}^n w_i^2} \)

Thus, we have \( Sum(P^2) = \sum_{i = 1}^n p_i^2} = Sum(W^2) / Sum(W)^2 \).

Example usage:

AverageValueMeter meter();
for (double sample : data) {
  meter.add(sample);
}
double mean = meter.value()[0];

Public Functions

AverageValueMeter()¶: Constructor of AverageValueMeter.

void add(const double val, const double w = 1.0)¶: Updates counters with the given value val with weight w.

void add(const Tensor &vals)¶: Updates counters with all values in vals with equal weights.

std::vector<double> value() const¶

Returns a vector of four values:

unbiased mean: \( \tilde{mu} \)
unbiased variance: \( \tilde{sigma}^2 = \frac{(\tilde{mu}_2 - \tilde{mu}^2)}{1 - Sum(P^2)} \)
weight_sum: \( Sum(W) \)
weight_squared_sum: \( Sum(W^2) \)

void reset()¶: Sets all the counters to 0.

CountMeter¶

class CountMeter¶

An implementation of count meter, which measures the total value of each category.

Example usage:

CountMeter meter(10);  // 10 categories in total
meter.add(4, 6);  // add 6 count to category 4
meter.add(7, 2);  // add 2 count to category 7
meter.add(4, -1);  // add -1 count to category 4

auto counts = meter.value();
std::cout << counts[4];  // prints 5

Public Functions

CountMeter(int num)¶

Constructor of CountMeter.

num specifies the total number of categories.

void add(int id, int64_t val)¶

Adds value val to category id.

Note that id should be in range [0, num - 1].

std::vector<int64_t> value() const¶: Returns a vector of num values, representing the total value of each category.

void reset()¶: Sets the value of each category to 0.

EditDistanceMeter¶

class EditDistanceMeter¶

An implementation of edit distance meter, which measures the edit distance between targets and predictions made by the model.

Example usage:

EditDistanceMeter meter();
for (auto& sample : data) {
  auto prediction = model(sample.input);
  meter.add(sample.target, prediction);
}
double letterErrorRate = meter.value()[0];

Public Functions

EditDistanceMeter()¶

Constructor of EditDistanceMeter.

An instance will maintain five counters initialized to 0:

n: total target lengths
ndel: total deletion error
nins: total insertion error
nsub: total substitution error

void add(const Tensor &output, const Tensor &target)¶: Computes edit distance between two arrayfire arrays output and target and updates the counters.

void add(const int64_t n, const int64_t ndel, const int64_t nins, const int64_t nsub)¶: Updates all the counters with inputs sharing the same meaning.

void add(const ErrorState &es, const int64_t n)¶: Updates all the counters with an ErrorState.

std::vector<double> errorRate() const¶

Returns a vector of five values:

error rate: \( \frac{(ndel + nins + nsub)}{n} \times 100.0 \)
total length: \( n \)
deletion rate: \( \frac{ndel}{n} \times 100.0\)
insertion rate: \( \frac{nins}{n} \times 100.0 \)
substitution rate: \( \frac{nsub}{n} \times 100.0 \)

std::vector<int64_t> value() const¶

Returns a vector of five values:

edit distance: \( (ndel + nins + nsub)\)
total length: \( n \)
number of deletions: \( ndel \)
number of insertions: \( nins \)
number of substitution: \( nsub \)

template<typename T, typename S> void add(const T &output, const S &target, const size_t olen, const size_t tlen)¶: Computes edit distance between two arrays output and target, with length olen and tlen respectively, and updates the counters.

template<typename T> void add(const std::vector<T> &output, const std::vector<T> &target)¶: Computes edit distance between two vectors output and target and updates the counters.

void reset()¶: Sets all the counters to 0.

struct ErrorState¶

A structure storing number of different type of errors when computing edit distance.

Public Functions

ErrorState()¶

int64_t sum() const¶: Sums up all the errors.

Public Members

int64_t ndel¶: Number of deletion error.

int64_t nins¶: Number of insertion error.

int64_t nsub¶: Number of substitution error.

MSEMeter¶

class MSEMeter¶

An implementation of mean square error meter, which measures the mean square error between targets and predictions made by the model.

Example usage:

MSEMeter meter();
for (auto& sample : data) {
  auto prediction = model(sample.input);
  meter.add(sample.target, prediction);
}
double mse = meter.value();

Public Functions

MSEMeter()¶

Constructor of MSEMeter.

An instance will maintain two counters initialized to 0:

n: total samples
mse: mean square error of samples

void add(const Tensor &output, const Tensor &target)¶

Computes mean square error between two arrayfire arrays output and target and updates the counters.

Note that the shape of the two input arrays should be identical.

double value() const¶: Returns a single value of mean square error.

void reset()¶: Sets all the counters to 0.

FrameErrorMeter¶

class FrameErrorMeter¶

An implementation of frame error meter, which measures the frame-level or element-level mismatch between targets and predictions made by the model.

Example usage:

FrameErrorMeter meter();
for (auto& sample : data) {
  auto prediction = model(sample.input);
  meter.add(sample.target, prediction);
}
double frameErrorRate = meter.value();

Public Functions

FrameErrorMeter(bool accuracy = false)¶

Constructor of FrameErrorMeter.

Flag accuracy indicates if the meter computes and returns accuracy or error rate instead. An instance will maintain two counters initialized to 0:

n: total samples
sum: total mismatches

void add(const Tensor &output, const Tensor &target)¶

Computes frame-level mismatch between two arrayfire arrays output and target and updates the counters.

Note that the shape of the two input arrays should be identical.

double value() const¶

Returns a single value in percentage.

If accuracy is True, the value returned is accuracy, error otherwise.

void reset()¶: Sets all the counters to 0.

TimeMeter¶

class TimeMeter¶

An implementation of timer, which measures the wall clock time.

Example usage:

TimeMeter meter();
meter.resume();
// Do something here;
meter.stop();
double time = meter.value();

Public Functions

TimeMeter(bool unit = false)¶

Constructor of TimeMeter.

An instance will maintain a timer which is initialized as stopped. The flag unit indicates if there is multiple units running in sequential in the current timing period.

double value() const¶

Stops the timer if still running.

If unit is True, returns the average time spend per unit, otherwise the total time in the current timing period. Time is measured in seconds.

void reset()¶: Refreshes the counters and stops the timer.

void incUnit(int64_t num = 1)¶: Increases the number of units by num.

void resume()¶: Starts the timer.

void stop()¶: Stops the timer.

void set(double val, int64_t num = 1)¶: Sets the number of units by num and the total time spend by val.

void stopAndIncUnit(int64_t num = 1)¶: Stops the timer and increase the number of units by num.