Variable¶
-
class Variable¶
Variable wraps an Arrayfire array and facilitates easy backpropagation.
Variable is a wrapper around Arrayfire array and supports many operations (Functions) on arrays. When a Function is applied on input Variable(s), the output Variable(s) records it’s inputs and a gradient function which can be used to compute gradient propagated from output to each of its inputs.
Thus, Variable and Functions build a computation graph which is a DAG. Following chain rule, backpropagation through this graph can be done easily by traversing in a topologically sorted order starting from output Variable(s).
NOTE: Variable holds the underlying array and gradient as shared_ptrs. Thus, calling the copy constructor of Variable creates a shallow copy with the same underlying data!
Example :
fl::Variable aVar(fl::rand({2}), true); // Creates a variable std::cout << "aVar" << aVar.tensor()); // aVar // [2 1 1 1] // 0.6010 // 0.0278 auto bVar = 1.0 + log(aVar); // Perform some arithmetic operations bVar.backward(); // Perform backward pass to compute the gradients std::cout << "bVar Grad" << bVar.grad().tensor()); // bVar Grad // [2 1 1 1] // 1.0000 // 1.0000 std::cout << "aVar Grad" << aVar.grad().tensor()); // aVar Grad // [2 1 1 1] // 1.6640 // 36.0246
Public Types
Public Functions
-
Variable() = default¶
Creates an empty Variable.
The underlying array is empty and isCalcGrad() is false.
-
Variable(Tensor data, std::vector<Variable> inputs, GradFunc gradFunc)¶
Creates a Variable which wraps the specified Tensor and inputs.
-
template<typename ...Ts>
inline Variable operator()(const Ts&... args) const¶ Indexing operator on a flattened Variable.
- Parameters:
indices – [in] a variable number of indices.
- Returns:
Variable storing the result after indexing operation
-
Variable flat(const fl::Index &index) const¶
Indexing operator on a flattened Variable.
- Parameters:
index – [in] index with which to index the flattened tensor
- Returns:
Variable storing the result after indexing operation
-
Variable copy() const¶
Creates a copy of this variable, but detached from the computation graph.
- Returns:
returns the cloned and detached variable.
-
Variable astype(fl::dtype type) const¶
Creates a new variable based on the current variable whose type will be adjusted based on the input type.
- Parameters:
type – [in] target data type
- Returns:
returns the casted variable.
-
bool isContiguous() const¶
Returns whether the array wrapped by the Variable is contiguous in memory in C order.
-
Variable asContiguous() const¶
Returns a Variable with contiguous array containing the same data as self array.
-
fl::dtype type() const¶
Returns the type of the
Tensorwrapped by the Variable (e.g.f32 for float, f64 for double).
See
fl/tensor/Types.h.
-
template<typename T>
inline T *host() const¶ Copies the array to the host and return the pointer.
Must eventually be freed manually via
freeor a related call.
-
template<typename T>
inline void host(T *ptr) const¶ Copies the array to the existing host pointer
ptr
-
void addGrad(const Variable &childGrad)¶
Add the gradient
childGradto the Variable.No-op if
this->isCalcGrad()is false.
-
void registerGradHook(const GradHook &hook)¶
Registers a lambda function
hookto be applied on the gradient w.r.t Variable after it is computed during backward pass.
-
void clearGradHook()¶
Clears the gradient hook stored in the variable.
-
void backward(const Variable &grad, bool retainGraph = false)¶
Run backward pass on the Variable.
Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed.
-
void backward(bool retainGraph = false)¶
Run backward pass on the Variable.
Gradient of all the inputs in the computation graph leading up to the Variable on which the function is computed. Gradient w.r.t the all the elements in the variable is set to 1.0
- Parameters:
retainGraph – [in] If False, clears the input Variables stored by the Variable
-
Variable() = default¶