Variable¶

class Variable¶

Variable wraps an Arrayfire array and facilitates easy backpropagation.

Variable is a wrapper around Arrayfire array and supports many operations (Functions) on arrays. When a Function is applied on input Variable(s), the output Variable(s) records it’s inputs and a gradient function which can be used to compute gradient propagated from output to each of its inputs.

Thus, Variable and Functions build a computation graph which is a DAG. Following chain rule, backpropagation through this graph can be done easily by traversing in a topologically sorted order starting from output Variable(s).

NOTE: Variable holds the underlying array and gradient as shared_ptrs. Thus, calling the copy constructor of Variable creates a shallow copy with the same underlying data!

Example :

fl::Variable aVar(fl::rand({2}), true); // Creates a variable
std::cout << "aVar" << aVar.tensor());
// aVar
// [2 1 1 1]
//     0.6010
//     0.0278
auto bVar = 1.0 + log(aVar); // Perform some arithmetic operations
bVar.backward(); // Perform backward pass to compute the gradients
std::cout << "bVar Grad" << bVar.grad().tensor());
// bVar Grad
// [2 1 1 1]
//    1.0000
//    1.0000
std::cout << "aVar Grad" << aVar.grad().tensor());
// aVar Grad
// [2 1 1 1]
//    1.6640
//   36.0246

Public Types

using GradFunc = std::function<void(std::vector<Variable> &inputs, const Variable &grad_output)>¶

using GradHook = std::function<void(Variable &grad)>¶

Public Functions

Variable() = default¶

Creates an empty Variable.

The underlying array is empty and isCalcGrad() is false.

Variable(Tensor data, bool calcGrad)¶

Creates a Variable which wraps the specified Tensor.

Parameters:

data – [in] Tensor to be stored in the Variable
calcGrad – [in] specifies whether to the gradient is required for this Variable

Variable(Tensor data, std::vector<Variable> inputs, GradFunc gradFunc)¶

Creates a Variable which wraps the specified Tensor and inputs.

Parameters:

data – [in] Tensor to the stored in the Variable
inputs – [in] a vector specifying inputs for this Variable
gradFunc – [in] function specifying how to calculate gradient of the input Variables

Variable operator()(const std::vector<Index> &indices) const¶

template<typename ...Ts> inline Variable operator()(const Ts&... args) const¶

Indexing operator on a flattened Variable.

Parameters:: indices – [in] a variable number of indices.
Returns:: Variable storing the result after indexing operation

Variable flat(const fl::Index &index) const¶

Indexing operator on a flattened Variable.

Parameters:: index – [in] index with which to index the flattened tensor
Returns:: Variable storing the result after indexing operation

Tensor &tensor() const¶

Returns:: a reference to the underlying Flashlight Tensor.

Variable astype(fl::dtype type) const¶

Creates a new variable based on the current variable whose type will be adjusted based on the input type.

Parameters:: type – [in] target data type
Returns:: returns the casted variable.

Variable &grad() const¶

Returns:: a reference to the underlying gradient Variable.

bool isCalcGrad() const¶: Returns whether the gradient calculation for the Variable is enabled.

bool isGradAvailable() const¶: Returns whether the gradient has been calculated for the Variable.

Shape shape() const¶: Returns the dimension of the array wrapped by the Variable.

bool isEmpty() const¶: Returns whether the array wrapped by the Variable is empty.

bool isContiguous() const¶: Returns whether the array wrapped by the Variable is contiguous in memory in C order.

Variable asContiguous() const¶: Returns a Variable with contiguous array containing the same data as self array.

fl::dtype type() const¶

Returns the type of the Tensor wrapped by the Variable (e.g.

f32 for float, f64 for double).

See fl/tensor/Types.h.

Dim elements() const¶: Returns the total number of elements stored in array wrapped by the Variable.

size_t bytes() const¶: Returns the total number of bytes stored in array wrapped by the Variable.

unsigned ndim() const¶: Returns the number of dimension of array wrapped by the Variable.

Dim dim(unsigned dim) const¶: Returns the dimension of array wrapped by the Variable.

void eval() const¶: Evaluates any expressions in the array wrapped by the Variable.

template<typename T> inline T *host() const¶

Copies the array to the host and return the pointer.

Must eventually be freed manually via free or a related call.

template<typename T> inline void host(T *ptr) const¶: Copies the array to the existing host pointer ptr

template<typename T> inline T scalar() const¶: Get the first element of the array as a scalar.

void zeroGrad()¶: Remove the gradient stored by the Variable.

void setCalcGrad(bool calcGrad)¶: Set whether to calculate gradient for the Variable.

void addGrad(const Variable &childGrad)¶

Add the gradient childGrad to the Variable.

No-op if this->isCalcGrad() is false.

void registerGradHook(const GradHook &hook)¶: Registers a lambda function hook to be applied on the gradient w.r.t Variable after it is computed during backward pass.

void clearGradHook()¶: Clears the gradient hook stored in the variable.

void backward(const Variable &grad, bool retainGraph = false)¶

Run backward pass on the Variable.

Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed.

Parameters:

grad – [in] gradient w.r.t to the Variable
retainGraph – [in] If False, clears the input Variables stored by the Variable

void backward(bool retainGraph = false)¶

Run backward pass on the Variable.

Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed. Gradient w.r.t the all the elements in the variable is set to 1.0

Parameters:: retainGraph – [in] If False, clears the input Variables stored by the Variable

Variable withoutData() const¶

Returns a copy of this variable after removing its underlying array.

The new Variable is used to store the inputs for a Variable which doesn’t need the output.