Tensor

Flashlight’s Tensor abstraction is itself an API. See the README in flashlight/fl/tensor for more on implementing a Tensor backend. The below documentation documents Tensor functions and usage of the Tensor class.

A Tensor in FLashlight is a multidimensional container for data that has a shape. Memory locations and operations on Tensors are defined in per a backend’s implementation.

The Tensor class

class Tensor

A Tensor on which computation can be performed.

Underlying implementations of tensor operations are implemented via types derived from TensorAdapterBase; these implementations also store state associated with the tensor. Tensor stores a pointer to the implementation, which can be swapped out if the implementation of backend changes.TensorAdapterBase implementations may differ across tensor libraries, hardware-specific libraries or compilers and DSLs.

Todo:

: Improve documentation throughout.

Warning

This API may break and is not yet stable.

Public Functions

Tensor(const Tensor &tensor)

Copy constructor - calls the implementation-defined copy constructor for the TensorAdapter.

Tensor(Tensor &&tensor) noexcept

Move constructor - moves the pointer to the TensorAdapter - performs no other operations.

Tensor()

Construct an empty tensor with the default tensor backend’s tensor adapter.

explicit Tensor(const Shape &shape, fl::dtype type = fl::dtype::f32)

Construct a tensor of a given shape (and optionally type) without populating its data.

Parameters:
  • shape[in] the shape of the tensor

  • type[in] (optional) the type of the tensor

explicit Tensor(fl::dtype type)

Construct an empty tensor of a given type.

Parameters:

type[in] (optional) the type of the tensor

Tensor(const Dim nRows, const Dim nCols, const Tensor &values, const Tensor &rowIdx, const Tensor &colIdx, StorageType storageType)

Construct a sparse tensor.

Todo:

Expand this API with getters as needed.

Parameters:
  • nRows[in] the number of rows of the tensor

  • nCols[in] the number of columns of the tensor

  • values[in] the values associated with the tensor

  • rowIdx[in] the row indices of the sparse array

  • colIdx[in] the the column indices of the sparse array

  • storageType[in] the storage type of the underlying tensor

Tensor copy() const

Deep-copies the tensor, including underlying data.

const Shape &shape() const

Get the shape of a tensor.

Returns:

the shape of the tensor

Location location() const

Get a tensor’s location, host or some device.

Returns:

the tensor’s location

size_t elements() const

Get the number of elements in the tensor.

Returns:

the size of the tensor in elements.

Dim dim(const size_t dim) const

Get the size of a given dimension of a tensor in the number of arguments.

Throws if the given dimension is larger than the number of tensor dimensions.

Returns:

the number of elements at the given dimension

int ndim() const

Get the number of directions of the tensor.

Returns:

the number of dimensions

bool isEmpty() const

Returns true if the tensor has zero elements, else false.

Returns:

true if the tensor is empty

bool hasAdapter() const

Returns true if the tensor has an associated underlying adapter.

Returns:

true if the tensor has a valid adapter

size_t bytes() const

Get the tensor size in bytes.

Returns:

the size of the tensor in bytes.

dtype type() const

Get the data type of tensor.

Returns:

the dtype of the tensor

bool isSparse() const

Returns whether or not the tensor is sparse.

Returns:

true if the tensor is a sparse tensor, else false

Shape strides() const

Get this tensor’s strides - the number of elements/coefficients to step when moving along each dimension when traversing the tensor.

Returns:

a Shape containing strides in each dimension.

virtual const Stream &stream() const

Get the stream which contains(ed) the computation required to realize an up-to-date value for this tensor.

For instance, device() may not yield a pointer to the up-to-date value — to use this pointer, Stream::sync or Stream::relativeSync is required.

Returns:

an immutable reference to the stream that contains(ed) the computations which create this tensor.

Tensor astype(const dtype type) const

Returns a tensor with elements cast as a particular type.

Parameters:

type[in] the type to which to cast the tensor

Returns:

a tensor with element-wise cast to the new type

Tensor operator()(const std::vector<Index> &indices) const

Index into a tensor using a vector of fl::Index references.

Parameters:

indices[in] a vector of fl::Index references with which to index.

Returns:

an indexed tensor

template<typename ...Ts>
inline Tensor operator()(const Ts&... args) const

Index into a tensor using a variable number of fl::Index.

Parameters:

args[in] fl::Index instances to use

Returns:

an indexed tensor

Tensor flatten() const

Returns a representation of the tensor in 1 dimension.

Returns:

a 1D version of this tensor 1D-indexed with the given index.

Tensor flat(const Index &idx) const

Returns a tensor indexed from this tensor but indexed as a 1D/flattened tensor.

Returns:

an indexed, 1D version of this tensor.

Tensor asContiguousTensor() const

Return a copy (depending on copy-on-write behavior of the underlying implementation) of this tensor that is contigous in memory.

Returns:

an identical tensor that is contiguous in memory

TensorBackendType backendType() const

Gets the backend enum from the underlying TensorAdapter.

Returns:

the backend in question

template<typename T>
inline T &getAdapter() const

Gets the underlying tensor adapter implementation.

Returns:

the tensor adapter.

TensorBackend &backend() const

Return the TensorBackend associated with this tensor.

Returns:

a TensorBackend.

template<typename T>
T scalar() const

Return a scalar of a specified type for the tensor.

If the tensor has more than one element, returns the first element as a scalar.

Throws an exception if the specified type does not match the dtype trait of the underlying tensor. To implicitly cast the scalar regardless of the underlying Tensor’s dtype, use asScalar.

Returns:

a scalar of the first element in the tensor.

template<typename T>
inline T asScalar() const

Return a scalar of the specified type of the tensor.

If the specified type does not match the tensor’s underlying dtype, the scalar value is implicitly cast.

Returns:

a scalar of the first element in the tensor cast to the specified type.

template<typename T>
T *device() const

Return a pointer to the tensor’s underlying data per a certain type.

This pointer exists on the computation device.

Note

The memory allocated here will not be freed until Tensor:unlock() is called.

Returns:

the requested pointer on the device.

template<typename T>
void device(T **ptr) const

Populate a pointer value with the address of a Tensor’s underlying buffer on the computation device.

Note

The memory allocated here will not be freed until Tensor:unlock() is called.

Parameters:

ptr[in] the pointer to populate with the Tensor’s buffer location on device.

template<typename T>
T *host() const

Returns a pointer to the tensor’s underlying data, but on the host.

If the tensor is located on a device, makes a copy of device memory and returns a buffer on the host containing the relevant memory.

Returns:

the requested pointer on the host.

template<typename T>
void host(T *ptr) const

Populates an existing buffer with the tensor’s underlying data, but on the host.

If the tensor is located on a device, makes a copy of device memory and returns a buffer on the host containing the relevant memory.

Parameters:

ptr[in] a pointer to the region of memory to populate with tensor values

template<typename T>
inline std::vector<T> toHostVector() const

Returns a vector on the host contaning a flat representation of the tensor.

The resulting vector is a copy of the underlying tensor memory, even if on the host.

Returns:

a vector in host memory containing

void unlock() const

Unlocks any device memory associated with the tensor that was acquired with Tensor::device(), making it eligible to be freed.

bool isLocked() const

Returns true if the tensor has been memory-locked per a call to Tensor::device<T>().

After unlocking via Tensor::unlock(), the tensor is no longer locked.

Returns:

true if the tensor is locked and a device pointer is active.

bool isContiguous() const

Returns if the Tensor is contiguous in its memory-based representation.

Returns:

a bool denoting Tensor contiguousness

void setContext(void *data)

Stores arbitrary data on a tensor.

For internal use/benchmarking only. This may be a no-op for some backends.

Parameters:

data[in] a pointer to arbitrary data to pass to a tensor impl.

void *getContext() const

Gets arbitrary data stored on a tensor.

For internal use/benchmarking only. This may be a no-op for some backends.

Returns:

a pointer to some implementation-defined data, else nullptr if a no-op.

std::string toString() const

Returns a string representation of a Tensor.

Note

This is backend-dependent. See Flashlight’s serialization utilities for ways to serialize Tensors that are portable across Tensor backends.

Returns:

a string representation of the Tensor.

std::ostream &operator<<(std::ostream &ostr) const

Write a string representation of a tensor to an output stream.

Public Static Functions

template<typename T>
static inline Tensor fromVector(Shape s, std::vector<T> v)

Create a tensor from a vector of values.

Parameters:
  • s[in] the shape of the resulting tensor.

  • v[in] values with which to populate the tensor.

Returns:

a tensor with values and shape as given.

template<typename T>
static inline Tensor fromBuffer(Shape s, const T *ptr, Location memoryLocation)

Create a tensor from an existing buffer.

Parameters:
  • s[in] the shape of the resulting tensor.

  • ptr[in] the buffer containing the data

  • memoryLocation[in] the location in memory where the input buffer with which to create the tensor resides.

Returns:

a tensor with values and shape as given.

static inline Tensor fromBuffer(Shape s, fl::dtype t, const uint8_t *ptr, Location memoryLocation)

Create a tensor from an existing byte buffer given a type.

Parameters:
  • s[in] the shape of the resulting tensor.

  • t[in] the type of the underlying tensor

  • ptr[in] the buffer of bytes containing the data

  • memoryLocation[in] the location in memory where the input buffer with which to create the tensor resides.

Returns:

a tensor with values and shape as given.

Indexing

Flashlight Tensors can be indexed by index literal values, ranges, spans (fl::span) or other Tensors (advanced indexing).

class range

An entity representing a contiguous or strided sequence of indices.

Assuming an axis has N elements, this is the mapping from negative to

Public Functions

range() = default

Default ctor.

explicit range(const Dim &idx)

Construct a range with the indices [0, idx) (i.e.

[0, idx - 1])

Parameters:

idx[in] the end index of the range, which will start from 0

range(const Dim &start, const idx &end)

Construct a range with the indices [start, end) (i.e.

[start, end - 1])

Parameters:
  • start[in] the starting index of the range

  • end[in] the end index of the range, which will start from 0

range(const Dim &start, const idx &end, const Dim stride)

Construct a range with the indices [start, end) (i.e.

[start, end - 1]) with the given stride.

Parameters:
  • start[in] the starting index of the range

  • end[in] the end index of the range, which will start from 0

  • stride[in] the interval over which successive range elements appear

struct Index

An entity used to index a tensor.

An index can be of a few different types, which are implicitly converted via Index’s constructors:

  • fl::Tensor if doing advanced indexing, where elements from a tensor are indexed based on values in the indexing tensor

  • fl::range which refers to a contiguous (or strided) sequence of indices.

  • An index literal, which refers to a single subtensor of the tensor being indexed.

Public Functions

Index &operator=(const Index&) = default

Default copy assignment operator.

Index(Index &&index) noexcept

Move constructor - moves the index data.

detail::IndexType type() const

Get the index type for this index.

Returns:

the index type.

bool isSpan() const

Returns true if the index represents a span.

template<typename T>
inline const T &get() const

Get the internal data for a particular Index.

Parameterized by type. Will throw as per std::variant if the type doesn’t match this Index’s underlying type.

Functions on Tensors

enum class PadType

Padding types for the pad operator.

Values:

enumerator Constant

pad with a constant zero value.

enumerator Edge

pad with the values at the edges of the tensor

enumerator Symmetric

pad with a reflection of the tensor mirrored along each edge

enum class SortMode

Sorting mode for sorting-related functions.

Values:

enumerator Descending
enumerator Ascending
enum class MatrixProperty

Transformations to apply to Tensors (i.e. matrices) before applying certain operations (i.e. matmul).

Values:

enumerator None
enumerator Transpose
template<typename T> FL_API Tensor fromScalar (const T &val, const dtype type=dtype_traits< T >::ctype)

Creates a new scalar Tensor with a particular value.

Scalar tensors have an empty Shape and 1 element by definition.

Parameters:
  • val[in] the value with which to fill the tensor

  • type[in] the type of the tensor to create. Defaults to a value based on the value type

Returns:

a tensor of the specified shape filled with the specified value

template<typename T> FL_API Tensor full (const Shape &dims, const T &val, const dtype type=dtype_traits< T >::ctype)

Creates a new Tensor with a given Shape and filled with a particular value.

Parameters:
  • dims[in] the shape of the tensor to create

  • val[in] the value with which to fill the tensor

  • type[in] the type of the tensor to create. Defaults to a value based on the value type

Returns:

a tensor of the specified shape filled with the specified value

FL_API Tensor identity (const Dim dim, const dtype type=dtype::f32)

Return a the identity tensor of a given size and type.

Parameters:
  • dim[in] the size of the dimension of the matrix (dim x dim)

  • type[in] the type of the resulting matrix

template<typename T> FL_API Tensor arange (const T &start, const T &end, const T &step=1, const dtype type=dtype_traits< T >::ctype)

Return evenly-spaced values in a given interval.

Generate values in the interval [start, stop) steppping each element by the passed step.

Parameters:
  • start[in] the start of the range

  • end[in] the end of the range, inclusive

  • step[in] the increment for each consecutive value in the range

  • type[in] the dtype of the resulting tensor

Returns:

a tensor containing the evenly-spaced values

FL_API Tensor arange (const Shape &shape, const Dim seqDim=0, const dtype type=dtype::f32)

Create a tensor with [0, N] values along dimension given by seqDim and tiled along the other dimensions.

N is equal to the size of the shape along the seqDim dimension.

Parameters:
  • shape[in] the shape of the output tensor.

  • seqDim[in] (optional) the dimension along which the sequence increases

  • type[in] (optional) the dtype of the resulting tensor

Returns:

a tensor with the given shape with the sequence along the given dimension, tiled along other dimensions.

FL_API Tensor iota (const Shape &dims, const Shape &tileDims={1}, const dtype type=dtype::f32)

Creates a sequence with the range [0, dims.elements()) sequentially in the shape given by dims, then tiles the result along the specified tile dimensions.

Todo:

an optimized version of this function is implemented only with the ArrayFire backend.

Parameters:
  • dims[in] the dimensions of the range

  • tileDims[in] the dimensions along which to tile

  • type[in] the dtype of the resulting tensoe

Returns:

FL_API Tensor reshape (const Tensor &tensor, const Shape &shape)

Change a tensor’s shape without changing its underlying data.

Parameters:
  • tensor[in] the tensor to reshape

  • shape[in] the new shape for the tensor

Returns:

the reshaped tensor

FL_API Tensor transpose (const Tensor &tensor, const Shape &axes={})

Permute the axes of a tensor.

If no arguments are given, reverses the axes of a tensor.

Parameters:
  • tensor[in] the tensor to transpose

  • axes[in] (optional) the permuted indices of the tensor the kth access of the output tensor will correspond to dims[k] in the input tensor. If this argument is not passed, the axes of the input tensor will be reversed.

Returns:

the permuted tensor

FL_API Tensor tile (const Tensor &tensor, const Shape &shape)

Repeat the contents of a tensor a given number of times along specified dimensions.

Parameters:
  • tensor[in] the tensor to tile

  • shape[in] the number of times, along each dimension, which to tile the tensor

Returns:

the tiled tensor

FL_API Tensor concatenate (const std::vector< Tensor > &tensors, const unsigned axis=0)

Join or concatenate tensors together along a particular axis.

Parameters:
  • tensors[in] a vector of tensors to concatenate

  • axis[in] the axis along which to concatenate tensors

Returns:

a concatenated tensor

template<typename ...Ts>
Tensor concatenate(unsigned axis, const Ts&... args)

Join or concatenate tensors together along a particular axis.

Parameters:
  • axis[in] the axis along which to concatenate tensors

  • args[in] tensors to concatenate

Returns:

a concatenated tensor

FL_API Tensor nonzero (const Tensor &tensor)

Return the indices of elements that are non-zero.

Indices correspond to a flattened version of the input tensor.

Parameters:

tensor[in] input tensor

Returns:

a tensor containing the indices of the nonzero elements

FL_API Tensor pad (const Tensor &input, const std::vector< std::pair< int, int >> &padWidths, const PadType type=PadType::Constant)

Pad a tensor with zeros.

Parameters:
  • input[in] the input tensor to pad

  • padWidths[in] a vector of tuples representing padding (before, after) tuples for each axis

  • type[in] the padding mode with which to pad the tensor - see PadType

Returns:

the padded tensor

FL_API Tensor negative (const Tensor &tensor)

Element-wise negation of a tensor.

Parameters:

tensor[in] the input tensor to negate.

Returns:

a tensor with elements negated.

inline Tensor operator-(const Tensor &tensor)
FL_API Tensor logicalNot (const Tensor &tensor)

Performs element-wise logical-not on the elements of a tensor.

Parameters:

tensor[in] the tensor on which to perform logical not

Returns:

a tensor with element-wise logical not of the input

inline Tensor operator!(const Tensor &tensor)
FL_API Tensor exp (const Tensor &tensor)

Compute the element-wise exponential of a tensor.

Parameters:

tensor[in] the tensor to exponentiate

Returns:

the exponentiated tensor

FL_API Tensor log (const Tensor &tensor)

Compute the element-wise natural logarithm of a tensor.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor log1p (const Tensor &tensor)

Returns the natural logarithm of one plus the input, element-wise.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor sin (const Tensor &tensor)

Returns the element-wise sine of the input.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor cos (const Tensor &tensor)

Returns the element-wise cosine of the input.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor sqrt (const Tensor &tensor)

Returns the element-wise non-negative square root of the input.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor tanh (const Tensor &tensor)

Returns the element-wise hyperbolic tangent of the input.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor floor (const Tensor &tensor)

Returns the element-wise floor of the input.

Parameters:

tensor[in] the tensor on which to compute the floor

Returns:

the resulting tensor

FL_API Tensor ceil (const Tensor &tensor)

Returns the element-wise ceiling of the input.

Parameters:

tensor[in] the tensor on which to compute the ceiling

Returns:

the resulting tensor

FL_API Tensor rint (const Tensor &tensor)

Returns the tensor with element-wise rounding to the nearest integer.

Parameters:

tensor[in] the input tensor

Returns:

the resulting tensor

FL_API Tensor absolute (const Tensor &tensor)

Returns the element-wise absolute value of the input.

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

inline Tensor abs(const Tensor &tensor)
FL_API Tensor sigmoid (const Tensor &tensor)

Returns the element-wise sigmoid the input:

\[ out_i = \frac{1}{1 + \exp(-var_i)} \]

Parameters:

tensor[in] the tensor on which to compute

Returns:

the resulting tensor

FL_API Tensor erf (const Tensor &tensor)

Computes the element-wise error function the input: see here for details.

Parameters:

tensor[in] the tensor on which to compute

Returns:

ther resulting tensor

FL_API Tensor flip (const Tensor &tensor, const unsigned dim)

Flip a Tensor along a specified dimension.

Parameters:
  • tensor[in] the tensor on which to compute

  • dim[in] the dimension along which to flip the tensor

Returns:

the resulting flipped tensor

FL_API Tensor clip (const Tensor &tensor, const Tensor &low, const Tensor &high)

Clip (limit) the values of a tensor.

Given some interval of values, set values outside of that interval to be the boundaries of the interval. All values larger than the max become the max, and all values smaller than the min become the min. Minimum and maximum values are determined element-wise in the input low and high input tensors.

Todo:

Require, enforce, and document broadcasting behavior in testing.

Parameters:
  • tensor[in] the tensor to clip

  • low[in] a tensor containing minimum values used element-wise in clipping

  • high[in] a tensor containing maximum values used element-wise in clipping

Returns:

a tensor with all values clipped between high and low

FL_API Tensor clip (const Tensor &tensor, const Tensor &low, const double &high)

Clip (limit) the values of a tensor.

Given some interval of values, set values outside of that interval to be the boundaries of the interval. All values larger than the max become the max, and all values smaller than the min become the min. Minimum values are determined element-wise in the low input tensor and the upper bound scalar.

Parameters:
  • tensor[in] the tensor to clip

  • low[in] a tensor containing minimum values used element-wise in clipping

  • high[in] a scalar to use as the maximum value in clipping

Returns:

a tensor with all values clipped between high and low

FL_API Tensor clip (const Tensor &tensor, const double &low, const Tensor &high)

Clip (limit) the values of a tensor.

Given some interval of values, set values outside of that interval to be the boundaries of the interval. All values larger than the max become the max, and all values smaller than the min become the min. Minimum values are given by the lower bound scalar and element-wise in the high input tensor.

Parameters:
  • tensor[in] the tensor to clip

  • low[in] a scalar to use as the minimum value in clipping

  • high[in] a tensor containing maximum values used element-wise in clipping

Returns:

a tensor with all values clipped between high and low

FL_API Tensor clip (const Tensor &tensor, const double &low, const double &high)

Clip (limit) the values of a tensor.

Given some interval of values, set values outside of that interval to be the boundaries of the interval. All values larger than the max become the max, and all values smaller than the min become the min. Minimum values are determined by the passed scalars.

Parameters:
  • tensor[in] the tensor to clip

  • low[in] a scalar to use as the minimum value in clipping

  • high[in] a scalar to use as the maximum value in clipping

Returns:

a tensor with all values clipped between high and low

FL_API Tensor roll (const Tensor &tensor, const int shift, const unsigned axis)

Rolls (or shifts) a tensor by a certain amount along a given axis, moving elements that would be shifted out of bounds to the beginning of the axis in a circular fashion.

Parameters:
  • tensor[in] the tensor to roll shift

  • shift[in] the amount by which to shift

  • axis[in] the axis along which to perform the shift

Returns:

a tensor with values shifted by the given amount in a circular fashion

FL_API Tensor isnan (const Tensor &tensor)

Returns a boolean tensor which is true where the input tensor was NaN, and false otherwise.

Parameters:

tensor[in] the input tensor

Returns:

a boolean tensor with true in positions that contained NaN in the input tensor

FL_API Tensor isinf (const Tensor &tensor)

Returns a boolean tensor which is true where the input tensor was infinity, and false otherwise.

Parameters:

tensor[in] the input tensor

Returns:

a boolean tensor with true in positions that contained Inf in the input tensor

FL_API Tensor sign (const Tensor &tensor)

Returns a tensor that contains -1 if an element is less than 0, 0 if an element is 0, and 1 if an element is greater than zero.

Returns NaN for NaN values.

Parameters:

tensor[in] the input tensor

Returns:

a tensor containing element-wise sign values.

FL_API Tensor tril (const Tensor &tensor)

Returns an upper triangular version of the tensor.

For tensors that have greater than two dimensions, this function outputs a tensor with lower triangular submatrices along the last two dimensions of the input tensor.

Parameters:

tensor[in] the input tensor

Returns:

a copy of the input tensor with elements above the diagonal zeroed out

FL_API Tensor triu (const Tensor &tensor)

Returns an upper triangular version of the tensor.

For tensors that have greater than two dimensions, this function outputs a tensor with upper triangular submatrices along the last two dimensions of the input tensor.

Parameters:

tensor[in] the input tensor

Returns:

a copy of the input tensor with elements below the diagonal zeroed out

FL_API Tensor where (const Tensor &condition, const Tensor &x, const Tensor &y)

Conditionally return elements from one of two tensors based on a condition.

Parameters:
  • condition[in] a tensor that, where true, uses values from x positionally, else values from y. This tensor must be of type dtype::b8 else an exception is thrown.

  • x[in] the tensor from which values are chosen for true values in condition

  • y[in] the tensor from which values are chosen for false values in condition

Returns:

the resulting tensor that contains elements of x where condition is true and elements of y where condition is false.

FL_API Tensor where (const Tensor &condition, const Tensor &x, const double &y)

Conditionally return elements from a tensor or passed scalar based on a condition.

Parameters:
  • condition[in] a tensor that, where true, uses the scalar value x, else positional values from y. This tensor must be of type dtype::b8 else an exception is thrown.

  • x[in] the tensor from which values are chosen for true values in the condition

  • y[in] the scalar returned for false values in the condition

Returns:

the resulting tensor that contains elements of x where condition is true and the scalar value y where the condition is false.

FL_API Tensor where (const Tensor &condition, const double &x, const Tensor &y)

Conditionally return elements from a scalar or passed tensor based on a condition.

Parameters:
  • condition[in] a tensor that, where true, uses values from x positionally, else the scalar value y. This tensor must be of type dtype::b8 else an exception is thrown.

  • x[in] the scalar returned for true values in the condition

  • y[in] the tensor from which values are chosen for false values in the condition

Returns:

the resulting tensor that contains elements of x where condition is true and the scalar value y where the condition is false.

FL_API void topk (Tensor &values, Tensor &indices, const Tensor &input, const unsigned k, const Dim axis, const SortMode sortMode=SortMode::Descending)

Get the top-k values and indices from a Tensor.

Parameters:
  • values[out] the sorted tensor

  • indices[out] the indices corresponding to the sorted ordering

  • input[in] the input tensor to sort

  • k[in] the top number of elements to return

  • axis[in] the axis along which to sort.

  • sortMode[in] the ordering with which to sort. In descending mode, the topk highest values are returned; else the topk lowest values are returned. Defaults to descending.

FL_API Tensor sort (const Tensor &input, const Dim axis, const SortMode sortMode=SortMode::Ascending)

Sort the values of a tensor, and return the sorted tensor.

Parameters:
  • input[in] the input Tensor

  • axis[in] the axis along which to sort

  • sortMode[in] the ordering with which to sort. Defaults to ascending

FL_API void sort (Tensor &values, Tensor &indices, const Tensor &input, const Dim axis, const SortMode sortMode=SortMode::Ascending)

Sort the values of a tensor, and return the sorted tensor and sorted indices.

Parameters:
  • values[out] the sorted tensor

  • indices[out] the indices corresponding to the sorted ordering

  • input[in] the input Tensor

  • axis[in] the axis along which to sort

  • sortMode[in] the ordering with which to sort. Defaults to ascending

FL_API Tensor argsort (const Tensor &input, const Dim axis, const SortMode sortMode=SortMode::Ascending)

Sort the values of a tensor and return the sorted indices.

Parameters:
  • input[in] the input Tensor

  • axis[in] the axis along which to sort

  • sortMode[in] the ordering with which to sort. Defaults to ascending

FL_API Tensor minimum (const Tensor &lhs, const Tensor &rhs)

Returns the element-wise minimum of tensor elements.

Todo:

Require, enforce, and document broadcasting behavior in testing.

Parameters:
  • lhs[in] left hand side tensor for the minimum

  • rhs[in] right hand side tensor for the minimum

Returns:

a tensor containing the minimum values in each tensor

FL_API Tensor minimum (const Tensor &lhs, const double &rhs)

Returns the element-wise minimum of tensor elements with some scalar.

Parameters:
  • lhs[in] the tensor

  • rhs[in] a scalar value

Returns:

a tensor containing the minimum values element-wise with the tensor and a scalar.

FL_API Tensor minimum (const double &lhs, const Tensor &rhs)

Returns the element-wise minimum of tensor elements with some scalar.

Parameters:
  • lhs[in] a scalar value

  • rhs[in] the tensor

Returns:

a tensor containing the minimum values element-wise with the tensor and a scalar.

FL_API Tensor maximum (const Tensor &lhs, const Tensor &rhs)

Returns the element-wise maximum of tensor elements.

Todo:

Require, enforce, and document broadcasting behavior in testing.

Parameters:
  • lhs[in] left hand side tensor for the minimum

  • rhs[in] right hand side tensor for the minimum

Returns:

a tensor containing the maximum values in each tensor

FL_API Tensor maximum (const Tensor &lhs, const double &rhs)

Returns the element-wise maximum of tensor elements with some scalar.

Parameters:
  • lhs[in] the tensor

  • rhs[in] a scalar value

Returns:

a tensor containing the maximum values element-wise with the tensor and a scalar.

FL_API Tensor maximum (const double &lhs, const Tensor &rhs)

Returns the element-wise maximum of tensor elements with some scalar.

Parameters:
  • lhs[in] a scalar value

  • rhs[in] the tensor

Returns:

a tensor containing the maximum values element-wise with the tensor and a scalar.

FL_API Tensor power (const Tensor &lhs, const Tensor &rhs)

Returns the element-wise exponentiation of tensors; the left hand tensor is exponentiated to the power of the right hand tensor, element-wise.

Parameters:
  • lhs[in] the base tensor

  • rhs[in] the exponent tensor

Returns:

a tensor containing the exponentiated values

FL_API Tensor power (const Tensor &lhs, const double &rhs)

Returns the element-wise exponentiation of tensors raised to some scalar power.

Parameters:
  • lhs[in] the base tensor

  • rhs[in] a scalar exponent

Returns:

a tensor containing the exponentiated values

FL_API Tensor power (const double &lhs, const Tensor &rhs)

Returns the element-wise exponentiation of a scalar raised element-wise to values from a tensor.

Parameters:
  • lhs[in] a scalar base

  • rhs[in] the tensor containing exponent values

Returns:

a tensor containing the exponentiated values

FL_API Tensor matmul (const Tensor &lhs, const Tensor &rhs, MatrixProperty lhsProp=MatrixProperty::None, MatrixProperty rhsProp=MatrixProperty::None)

Perform matrix multiplication between two tensors.

Parameters:
  • lhs[in] the Tensor on the left hand side

  • rhs[in] the Tensor on the right hand side

  • lhsProp[in] the MatrixProperty to apply to the tensor on the left-hand side

  • rhsProp[in] the MatrixProperty to apply to the tensor on the right-hand side

Returns:

an output tensor containing the matrix product.

FL_API Tensor amin (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Compute the minimum value along multiple axes.

If axes is left empty, computes the minumum along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the max(es)

FL_API Tensor amax (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Compute the maximum value along multiple axes.

If axes is left empty, computes the maximum along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the max(es)

FL_API void min (Tensor &values, Tensor &indices, const Tensor &input, const unsigned axis, const bool keepDims=false)

Compute the maximum value along multiple axes for a tensor, returning both the maximum values and the indices of the input tensor in which they appear.

Parameters:
  • values[out] a Tensor into which to populate the max values from the tensor along the specified axes

  • indices[out] a Tensor into which to populate the indices of the max values from the tensor along the specified axes

  • input[in] the input tensor

  • axis[in] the axis along which to find minimum values

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

FL_API void max (Tensor &values, Tensor &indices, const Tensor &input, const unsigned axis, const bool keepDims=false)

Compute the maximum value along multiple axes for a tensor, returning both the maximum values and the indices of the input tensor in which they appear.

Parameters:
  • values[out] a Tensor into which to populate the max values from the tensor along the specified axes

  • indices[out] a Tensor into which to populate the indices of the max values from the tensor along the specified axes

  • input[in] the input tensor

  • axis[in] the axis along which to find maximum values

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

FL_API Tensor argmax (const Tensor &input, const unsigned axis, const bool keepDims=false)

Return the indices of the maximum values along an axis.

Parameters:
  • input[in] the input tensor

  • axis[in] the axis along which to find maximum values

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the indices of the max values along each axis

FL_API Tensor argmin (const Tensor &input, const unsigned axis, const bool keepDims=false)

Return the indices of the minimum values along an axis.

Parameters:
  • input[in] the input tensor

  • axis[in] the axis along which to find minimum values

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the indices of the max values along each axis

FL_API Tensor sum (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Sum of tensor over given axes.

If axes is left empty, computes the sum along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the sum(s)

FL_API Tensor cumsum (const Tensor &input, const unsigned axis)

Compute the cumulative sum (or the prefix sum, scan, or inclusive scan) of a tensor along a given axis.

Parameters:
  • input[in] the input tensor

  • axis[in] the axis along which to accumulate

Returns:

a tensor of the same shape containing the accumulated sum

FL_API Tensor mean (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Mean of tensor over given axes.

If axes is left empty, computes the mean along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the mean(s)

FL_API Tensor median (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Median of tensor over given axes.

If axes is left empty, computes the median along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the median(s)

FL_API Tensor var (const Tensor &input, const std::vector< int > &axes={}, const bool bias=false, const bool keepDims=false)

Variance of an tensor over given axes.

If axes is left empty, computes the variance along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • bias[in] defaults false. Compute biased or unbiased variance

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the variance(s)

FL_API Tensor std (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Standard deviation of an tensor over given axes.

If axes is left empty, computes the standard deviation along all axes.

Parameters:
  • input[in] the input along which to operate

  • axes[in] the dimension along which to reduce. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the standard deviation(s)

FL_API Tensor norm (const Tensor &input, const std::vector< int > &axes={}, double p=2, const bool keepDims=false)

Perform Lp-norm computation, reduced over specified dimensions.

If axes is left blank, computes the norm along all dimensions.

Parameters:
  • input[in] tensor on which the Lp norm is going to be computed.

  • p[in] the p value of the Lp norm.

  • axes[in] dimensions over which the reduction is performed.

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the norm(s)

FL_API Tensor countNonzero (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Counts the number of nonzero elements in a tensor.

If k axes are passed, returns a tensor of size k with element-wise nonzero counts along each axis.

Parameters:
  • input[in] the tensor on which to operate.

  • axes[in] (optional) the axis along which to give nonzeros.

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a tensor containing the number of nonzero elements along each axis or over the entire tensor.

FL_API Tensor any (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Checks for any true values in a tensor along one or more axes; returns true if any exist.

If k axes are passed, returns a tensor of size k with truthiness checks along each axis. If axes is left empty, computes the variance along all axes.

Parameters:
  • input[in] the input tensor

  • axes[in] the axes along which to check for truthy values. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a bool tensor containing axis-wise values denoting truthy values along that axis in the input tensor.

FL_API Tensor all (const Tensor &input, const std::vector< int > &axes={}, const bool keepDims=false)

Checks if all values are true in a tensor along one or more axes; returns true if all are true and false otherwise.

If k axes are passed, returns a tensor of size k with all-true checks along each axis. If axes is left empty, computes the variance along all axes.

Parameters:
  • input[in] the input tensor

  • axes[in] the axes along which to check. If empty, computes along all axes

  • keepDims[in] defaults false. Keeps the dimensions being reduced over as singleton dimensions rather than collapsing them

Returns:

a bool tensor containing axis-wise values with true along axes that contain only true values.

FL_API std::ostream & operator<< (std::ostream &ostr, const Tensor &t)

Write a string representation of a tensor to an output stream.

FL_API void print (const Tensor &tensor)

Print a string representation of a tensor to standard out.

Parameters:

tensor[in] the tensor to print

FL_API bool allClose (const fl::Tensor &a, const fl::Tensor &b, const double absTolerance=1e-5)

Returns of two tensors are close.

Checks:

  • Tensor data types

  • Tensor shapes

  • Emptiness

  • Absolute distance between elements

Parameters:
  • a[in] lhs tensor

  • b[in] rhs tensor

  • absTolerance[in] the maximum-allowable distance between the tensors

FL_API bool isInvalidArray (const Tensor &tensor)
Returns:

if a Tensor contains any NaN or Inf values.

FL_API std::string tensorBackendTypeToString (const TensorBackendType type)

Get a string representation of a tensor backend type.

Parameters:

type[in] the tensor backend type.

Returns:

a string representing the given tensor backend type.

FL_API std::ostream & operator<< (std::ostream &os, const TensorBackendType type)

Write a string representation of a tensor backend type to an output stream.

Parameters:
  • os[out] the output stream.

  • type[in] the tensor backend type.

Returns:

the output stream.

template<typename T>
Tensor to(Tensor &&t)

Convert a tensor from one type to another.

Requires moving the input Tensor

  • destroys the resulting tensor and creates another tensor of the desired tensor type.

Parameters:

t[in] the tensor to convert

Returns:

a tensor backed by the specified compile time type