Meters¶
AverageValueMeter¶
-
class
AverageValueMeter
¶ An implementation of average value meter, which measures the mean and variance of a sequence of values.
This meter takes as input a stream of i.i.d data \( X = {x_i} \) with unormalized weights \( W = {w_i} \) ( \( w_i \ge 0 \)). Suppose \( p_i = w_i / \sum_{j = 1}^n w_j \), it maintains the following variables:
unbiased mean \( \tilde{mu} = \sum_{i = 1}^n p_i x_i \)
unbiased second momentum \( \tilde{mu}_2 = \sum_{i = 1}^n p_i x_i^2 \)
sum of weights \( Sum(W) = \sum_{i = 1}^n w_i} \)
sum of squared weights \( Sum(W^2) = \sum_{i = 1}^n w_i^2} \)
Thus, we have \( Sum(P^2) = \sum_{i = 1}^n p_i^2} = Sum(W^2) / Sum(W)^2 \).
Example usage:
AverageValueMeter meter(); for (double sample : data) { meter.add(sample); } double mean = meter.value()[0];
Public Functions
-
AverageValueMeter
()¶ Constructor of
AverageValueMeter
.
-
void
add
(const double val, const double w = 1.0)¶ Updates counters with the given value
val
with weightw
.
-
std::vector<double>
value
() const¶ Returns a vector of four values:
unbiased mean
: \( \tilde{mu} \)unbiased variance
: \( \tilde{sigma}^2 = \frac{(\tilde{mu}_2 - \tilde{mu}^2)}{1 - Sum(P^2)} \)weight_sum
: \( Sum(W) \)weight_squared_sum
: \( Sum(W^2) \)
-
void
reset
()¶ Sets all the counters to 0.
CountMeter¶
-
class
CountMeter
¶ An implementation of count meter, which measures the total value of each category.
Example usage:
CountMeter meter(10); // 10 categories in total meter.add(4, 6); // add 6 count to category 4 meter.add(7, 2); // add 2 count to category 7 meter.add(4, -1); // add -1 count to category 4 auto counts = meter.value(); std::cout << counts[4]; // prints 5
Public Functions
-
CountMeter
(int num)¶ Constructor of
CountMeter
.num
specifies the total number of categories.
-
void
add
(int id, int64_t val)¶ Adds value
val
to categoryid
.Note that
id
should be in range [0,num
- 1].
-
std::vector<int64_t>
value
() const¶ Returns a vector of
num
values, representing the total value of each category.
-
void
reset
()¶ Sets the value of each category to 0.
-
EditDistanceMeter¶
-
class
EditDistanceMeter
¶ An implementation of edit distance meter, which measures the edit distance between targets and predictions made by the model.
Example usage:
EditDistanceMeter meter(); for (auto& sample : data) { auto prediction = model(sample.input); meter.add(sample.target, prediction); } double letterErrorRate = meter.value()[0];
Public Functions
-
EditDistanceMeter
()¶ Constructor of
EditDistanceMeter
.An instance will maintain five counters initialized to 0:
n
: total target lengthsndel
: total deletion errornins
: total insertion errornsub
: total substitution error
-
void
add
(const Tensor &output, const Tensor &target)¶ Computes edit distance between two arrayfire arrays
output
andtarget
and updates the counters.
-
void
add
(const int64_t n, const int64_t ndel, const int64_t nins, const int64_t nsub)¶ Updates all the counters with inputs sharing the same meaning.
-
void
add
(const ErrorState &es, const int64_t n)¶ Updates all the counters with an
ErrorState
.
-
std::vector<double>
errorRate
() const¶ Returns a vector of five values:
error rate
: \( \frac{(ndel + nins + nsub)}{n} \times 100.0 \)total length
: \( n \)deletion rate
: \( \frac{ndel}{n} \times 100.0\)insertion rate
: \( \frac{nins}{n} \times 100.0 \)substitution rate
: \( \frac{nsub}{n} \times 100.0 \)
-
std::vector<int64_t>
value
() const¶ Returns a vector of five values:
edit distance
: \( (ndel + nins + nsub)\)total length
: \( n \)number of deletions
: \( ndel \)number of insertions
: \( nins \)number of substitution
: \( nsub \)
-
template<typename
T
, typenameS
>
voidadd
(const T &output, const S &target, const size_t olen, const size_t tlen)¶ Computes edit distance between two arrays
output
andtarget
, with lengtholen
andtlen
respectively, and updates the counters.
-
template<typename
T
>
voidadd
(const std::vector<T> &output, const std::vector<T> &target)¶ Computes edit distance between two vectors
output
andtarget
and updates the counters.
-
void
reset
()¶ Sets all the counters to 0.
-
struct
ErrorState
¶ A structure storing number of different type of errors when computing edit distance.
-
MSEMeter¶
-
class
MSEMeter
¶ An implementation of mean square error meter, which measures the mean square error between targets and predictions made by the model.
Example usage:
MSEMeter meter(); for (auto& sample : data) { auto prediction = model(sample.input); meter.add(sample.target, prediction); } double mse = meter.value();
Public Functions
-
MSEMeter
()¶ Constructor of
MSEMeter
.An instance will maintain two counters initialized to 0:
n
: total samplesmse
: mean square error of samples
-
void
add
(const Tensor &output, const Tensor &target)¶ Computes mean square error between two arrayfire arrays
output
andtarget
and updates the counters.Note that the shape of the two input arrays should be identical.
-
double
value
() const¶ Returns a single value of mean square error.
-
void
reset
()¶ Sets all the counters to 0.
-
FrameErrorMeter¶
-
class
FrameErrorMeter
¶ An implementation of frame error meter, which measures the frame-level or element-level mismatch between targets and predictions made by the model.
Example usage:
FrameErrorMeter meter(); for (auto& sample : data) { auto prediction = model(sample.input); meter.add(sample.target, prediction); } double frameErrorRate = meter.value();
Public Functions
-
FrameErrorMeter
(bool accuracy = false)¶ Constructor of
FrameErrorMeter
.Flag
accuracy
indicates if the meter computes and returns accuracy or error rate instead. An instance will maintain two counters initialized to 0:n
: total samplessum
: total mismatches
-
void
add
(const Tensor &output, const Tensor &target)¶ Computes frame-level mismatch between two arrayfire arrays
output
andtarget
and updates the counters.Note that the shape of the two input arrays should be identical.
-
double
value
() const¶ Returns a single value in percentage.
If
accuracy
isTrue
, the value returned is accuracy, error otherwise.
-
void
reset
()¶ Sets all the counters to 0.
-
TimeMeter¶
-
class
TimeMeter
¶ An implementation of timer, which measures the wall clock time.
Example usage:
TimeMeter meter(); meter.resume(); // Do something here; meter.stop(); double time = meter.value();
Public Functions
-
TimeMeter
(bool unit = false)¶ Constructor of
TimeMeter
.An instance will maintain a timer which is initialized as stopped. The flag
unit
indicates if there is multiple units running in sequential in the current timing period.
-
double
value
() const¶ Stops the timer if still running.
If
unit
isTrue
, returns the average time spend per unit, otherwise the total time in the current timing period. Time is measured in seconds.
-
void
reset
()¶ Refreshes the counters and stops the timer.
-
void
incUnit
(int64_t num = 1)¶ Increases the number of units by
num
.
-
void
resume
()¶ Starts the timer.
-
void
stop
()¶ Stops the timer.
-
void
set
(double val, int64_t num = 1)¶ Sets the number of units by
num
and the total time spend byval
.
-
void
stopAndIncUnit
(int64_t num = 1)¶ Stops the timer and increase the number of units by
num
.
-