Variable

class Variable

Variable wraps an Arrayfire array and facilitates easy backpropagation.

Variable is a wrapper around Arrayfire array and supports many operations (Functions) on arrays. When a Function is applied on input Variable(s), the output Variable(s) records it’s inputs and a gradient function which can be used to compute gradient propagated from output to each of its inputs.

Thus, Variable and Functions build a computation graph which is a DAG. Following chain rule, backpropagation through this graph can be done easily by traversing in a topologically sorted order starting from output Variable(s).

NOTE: Variable holds the underlying array and gradient as shared_ptrs. Thus, calling the copy constructor of Variable creates a shallow copy with the same underlying data!

Example :

fl::Variable aVar(af::randu(2), true); // Creates a variable
af::print("aVar", aVar.array());
// aVar
// [2 1 1 1]
//     0.6010
//     0.0278
auto bVar = 1.0 + log(aVar); // Perform some arithmetic operations
bVar.backward(); // Perform backward pass to compute the gradients
af::print("bVar Grad", bVar.grad().array());
// bVar Grad
// [2 1 1 1]
//    1.0000
//    1.0000
af::print("aVar Grad", aVar.grad().array());
// aVar Grad
// [2 1 1 1]
//    1.6640
//   36.0246

Public Types

using GradFunc = std::function<void(std::vector<Variable> &inputs, const Variable &grad_output)>
using GradHook = std::function<void(Variable &grad)>

Public Functions

Variable()

Creates an empty Variable.

The underlying array is empty and isCalcGrad() is false.

Variable(af::array data, bool calcGrad)

Creates a Variable which wraps the array specified.

Parameters
  • [in] data: array to be stored in the Variable

  • [in] calcGrad: specifies whether to the gradient is required for this Variable

Variable(af::array data, std::vector<Variable> inputs, GradFunc gradFunc)

Creates a Variable which wraps the array and inputs specified.

Parameters
  • [in] data: array to the stored in the Variable

  • [in] inputs: a vector specifying inputs for this Variable

  • [in] gradFunc: function specifying how to calculate gradient of the input Variables

Variable operator()(const af::index &s0, const af::index &s1 = af::span, const af::index &s2 = af::span, const af::index &s3 = af::span) const

Indexing operator on the Arrayfire Array wrapped by the Variable.

Return

Variable storing the result after indexing operation

Parameters
  • [in] s0: sequence of indices along first dimension

  • [in] s1: sequence of indices along second dimension

  • [in] s2: sequence of indices along third dimension

  • [in] s3: sequence of indices along fourth dimension

af::array &array() const

Return

a reference to the underlying Arrayfire array.

Variable &grad() const

Return

a reference to the underlying gradient Variable.

bool isCalcGrad() const

Returns whether the gradient calculation for the Variable is enabled.

bool isGradAvailable() const

Returns whether the gradient has been calculated for the Variable.

af::dim4 dims() const

Returns the dimension of the array wrapped by the Variable.

bool isempty() const

Returns whether the array wrapped by the Variable is empty.

bool isLinear() const

Returns whether the array wrapped by the Variable is contiguous in memory in C order.

Variable linear() const

Returns a Variable with contiguous array containing the same data as self array.

af::dtype type() const

Returns the type of the array wrapped by the Variable (e.g.

f32 for float, f64 for double). Full list: http://arrayfire.org/docs/defines_8h.htm (search dtype)

dim_t elements() const

Returns the total number of elements stored in array wrapped by the Variable.

size_t bytes() const

Returns the total number of bytes stored in array wrapped by the Variable.

unsigned numdims() const

Returns the number of dimension of array wrapped by the Variable.

dim_t dims(unsigned dim) const

Returns the dimension of array wrapped by the Variable.

void eval() const

Evaluates any expressions in the array wrapped by the Variable.

template<typename T>
T *host() const

Copies the array to the host and return the pointer.

Must eventually be freed with af::freeHost().

template<typename T>
void host(T *ptr) const

Copies the array to the existing host pointer ptr

template<typename T>
T scalar() const

Get the first element of the array as a scalar.

void zeroGrad()

Remove the gradient stored by the Variable.

void setCalcGrad(bool calcGrad)

Set whether to calculate gradient for the Variable.

void addGrad(const Variable &childGrad)

Add the gradient childGrad to the Variable.

No-op if this->isCalcGrad() is false.

void registerGradHook(const GradHook &hook)

Registers a lambda function hook to be applied on the gradient w.r.t Variable after it is computed during backward pass.

void clearGradHook()

Clears the gradient hook stored in the variable.

void backward(const Variable &grad, bool retainGraph = false)

Run backward pass on the Variable.

Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed.

Parameters
  • [in] grad: gradient w.r.t to the Variable

  • [in] retainGraph: If False, clears the input Variables stored by the Variable

void backward(bool retainGraph = false)

Run backward pass on the Variable.

Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed. Gradient w.r.t the all the elements in the variable is set to 1.0

Parameters
  • [in] retainGraph: If False, clears the input Variables stored by the Variable

Variable col(int index) const

Return a column from an array based on index.

This can also be seen as the result of doing input(af::span, index, af::span, af::span)

Return

Variable storing the result

Parameters
  • [in] index: index of the row

Variable cols(int first, int last) const

Returns a sequence of colums from an array based on first and last indices.

This can also be seen as the result of doing input(af::span, af::seq(first, last), af::span, af::span)

Return

Variable storing the result

Parameters
  • [in] first: start index of the rows

  • [in] last: end index of the rows

Variable row(int index) const

Returns a row from an array based on index.

This can also be seen as the result of doing input(index, af::span, af::span, af::span)

Return

Variable storing the result

Parameters
  • [in] index: index of the slice

Variable rows(int first, int last) const

Returns a sequence of rows from an array based on first and last indices.

This can also be seen as the result of doing input(af::seq(first, last), af::span, af::span, af::span)

Return

Variable storing the result

Parameters
  • [in] first: start index of the rows

  • [in] last: end index of the rows

Variable slice(int index) const

Return slice in volume from an array based on index.

This can also be seen as the result of doing input(af::span, af::span, index, af::span)

Return

Variable storing the result

Parameters
  • [in] index: index of the slice

Variable slices(int first, int last) const

Return slices in volume from an array based on first and last indices.

This can also be seen as the result of doing input(af::span, af::span, af::seq(first, last), af::span)

Return

Variable storing the result

Parameters
  • [in] first: start index of the slices

  • [in] last: end index of the slices

Variable withoutData() const

Returns a copy of this variable after removing its underlying array.

The new Variable is used to store the inputs for a Variable which doesn’t need the output.