Contrib¶
Modules¶
Layers¶
-
class Residual : public fl::Container¶
A module for creating a generic Residual block as given by He et al (2015) and Kumar et al (2015).
Example:
auto res = Residual(); // Add multiple layers res.add(Conv2D(30, 50, 9, 7, 2, 3, 3, 2)); res.add(BatchNorm(2, 50)); res.add(ReLU()); // Add a shortcut from the input to the block to the third layer res.addShortcut(0, 3); // Add a shortcut from the second layer to the output res.addShortcut(2, 4); // Scale the inputs to the third layer by some constant res.addScale(3, 0.5); // Create a model Sequential model; // ... // Add our residual block as needed model.add(res); model.add(Pool2D(2, 3, 1, 1, 1, 1, PoolingMode::MAX)); model.add(res); // ...
Public Functions
-
void addScale(int beforeLayer, float scale)¶
Adds a scaling factor to all residual connections connecting to a layer given by some index index.
Given some scale \( \alpha \), the input to
beforeLayer
becomes \( (x + f(x)) * \alpha \).- Parameters:
beforeLayer – [in] the index of the layer to which to scale the input and residual connection output.
scale – [in] the value by which to scale the sum of the previous layer and output of the residual connection.
-
void addShortcut(int fromLayer, int toLayer)¶
Adds a shortcut between two layers.
- Parameters:
fromLayer – [in] the layer index from which the skip connection will originate; must be in the range \( [0, N_{layers} - 1] \). If the index 0 is used, the input to the shortcut will be equal to the input to the residual block.
toLayer – [in] the layer index to which the skip connection outputs a tensor; must be in the range \( [1, N_{layers} + 1] \). If the index \( N_{layers} + 1 \) is used, the output of the shortcut will be added to the output of the entire residual block.
Adds a shortcut connection between two layers such that tensors passed through the connection are forwarded through a passed module before being added to the resultant module’s input.
Can be used to reshape the output of input module to match the input dimensions for the output module.
- Parameters:
fromLayer – [in] the layer index from which the shortcut connection will originate; must be in the range \( [0, N_{layers} - 1] \). If the index 0 is used, the input to the shortcut will be equal to the input to the residual block.
toLayer – [in] the layer index to which the shortcut connection outputs a tensor; must be in the range \( [1, N_{layers} + 1] \). If the index \( N_{layers} + 1 \) is used, the output of the shortcut will be added to the output of the entire residual block.
module – [in] a specified module through which the input to the shortcut connection will be forwarded before being added to the input to the destination module.
-
virtual std::vector<Variable> forward(const std::vector<Variable> &inputs) override¶
Performs forward computation for the module, given some inputs.
- Parameters:
inputs – the values to compute forward computation for the module.
- Returns:
a vector of
Variable
tensors containing the result of the forward computation
-
void addScale(int beforeLayer, float scale)¶