Variable¶
-
class
Variable
¶ Variable wraps an Arrayfire array and facilitates easy backpropagation.
Variable is a wrapper around Arrayfire array and supports many operations (Functions) on arrays. When a Function is applied on input Variable(s), the output Variable(s) records it’s inputs and a gradient function which can be used to compute gradient propagated from output to each of its inputs.
Thus, Variable and Functions build a computation graph which is a DAG. Following chain rule, backpropagation through this graph can be done easily by traversing in a topologically sorted order starting from output Variable(s).
NOTE: Variable holds the underlying array and gradient as shared_ptrs. Thus, calling the copy constructor of Variable creates a shallow copy with the same underlying data!
Example :
fl::Variable aVar(fl::rand({2}), true); // Creates a variable std::cout << "aVar" << aVar.tensor()); // aVar // [2 1 1 1] // 0.6010 // 0.0278 auto bVar = 1.0 + log(aVar); // Perform some arithmetic operations bVar.backward(); // Perform backward pass to compute the gradients std::cout << "bVar Grad" << bVar.grad().tensor()); // bVar Grad // [2 1 1 1] // 1.0000 // 1.0000 std::cout << "aVar Grad" << aVar.grad().tensor()); // aVar Grad // [2 1 1 1] // 1.6640 // 36.0246
Public Types
Public Functions
-
Variable
()¶ Creates an empty Variable.
The underlying array is empty and isCalcGrad() is false.
-
Variable
(Tensor data, std::vector<Variable> inputs, GradFunc gradFunc)¶ Creates a Variable which wraps the specified Tensor and inputs.
-
template<typename ...
Ts
>
Variableoperator()
(const Ts&... args) const¶ Indexing operator on a flattened Variable.
- Return
Variable storing the result after indexing operation
- Parameters
[in] indices
: a variable number of indices.
-
Variable
flat
(const fl::Index &index) const¶ Indexing operator on a flattened Variable.
- Return
Variable storing the result after indexing operation
- Parameters
[in] index
: index with which to index the flattened tensor
-
Variable
astype
(fl::dtype type) const¶ Creates a new variable based on the current variable whose type will be adjusted based on the input type.
- Return
returns the casted variable.
- Parameters
[in] type
: target data type
-
bool
isContiguous
() const¶ Returns whether the array wrapped by the Variable is contiguous in memory in C order.
-
Variable
asContiguous
() const¶ Returns a Variable with contiguous array containing the same data as self array.
-
fl::dtype
type
() const¶ Returns the type of the
Tensor
wrapped by the Variable (e.g.f32 for float, f64 for double).
See
fl/tensor/Types.h
.
-
template<typename
T
>
T *host
() const¶ Copies the array to the host and return the pointer.
Must eventually be freed manually via
free
or a related call.
-
void
addGrad
(const Variable &childGrad)¶ Add the gradient
childGrad
to the Variable.No-op if
this->isCalcGrad()
is false.
-
void
registerGradHook
(const GradHook &hook)¶ Registers a lambda function
hook
to be applied on the gradient w.r.t Variable after it is computed during backward pass.
-
void
clearGradHook
()¶ Clears the gradient hook stored in the variable.
-
void
backward
(const Variable &grad, bool retainGraph = false)¶ Run backward pass on the Variable.
Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed.
-
void
backward
(bool retainGraph = false)¶ Run backward pass on the Variable.
Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed. Gradient w.r.t the all the elements in the variable is set to 1.0
- Parameters
[in] retainGraph
: If False, clears the input Variables stored by the Variable
-