Activation Functions

Activation functions introduce non-linearity into neural networks, enabling them to learn complex patterns. The athena library provides a variety of commonly used activation functions.

Available Activation Functions

• Gaussian Activation
• Linear Activation
• ReLU Activation
• Leaky ReLU Activation
• Sigmoid Activation
• Tanh Activation
• Softmax Activation
• Swish Activation
• SELU Activation
• No Activation

Creating custom activation functions

The athena library is designed with extensibility in mind, allowing users to create custom activation functions by extending the base_actv_type.

See the tutorial: Creating Custom Activations