Building a Basic Network

This tutorial will guide you through creating your first neural network using the athena library.

Prerequisites

Make sure you have athena installed. See Installation for details.

Creating a Simple Network

Let’s build a simple feedforward neural network for a classification problem.

Step 1: Import the Library

program basic_network
  use athena
  implicit none

  ! Variable declarations will go here

end program basic_network

Step 2: Define the Network Variable

We now define a variable to hold the network:

type(network_type) :: net

Step 3: Build the Network Architecture

And build the network layer by layer:

integer :: num_inputs, num_hidden, num_outputs

! Define network dimensions
num_inputs = 784    ! e.g., 28x28 flattened image
num_hidden = 128
num_outputs = 10    ! 10 classes

! Add first hidden layer with ReLU activation
call net%add(full_layer_type( &
     num_inputs=num_inputs, &
     num_outputs=num_hidden, &
     activation="relu"))

! Add second hidden layer
call net%add(full_layer_type( &
     num_outputs=num_hidden, &
     activation="relu"))

! Add output layer with softmax for classification
call net%add(full_layer_type( &
     num_outputs=num_outputs, &
     activation="softmax"))

Note

For simple networks (i.e. single input layer), the input layer is created automatically during the compile step.

Step 4: Compile the Network

Initialise the network with an optimiser and loss function. The loss function can either be specified as a string or as a loss object (the currently supported loss functions are listed in Loss Functions). The optimiser must be specified as an optimiser object (the currently supported optimisers are listed in Optimisers).

An example using the Adam optimiser and categorical cross-entropy loss is shown below:

type(adam_optimiser_type) :: optimiser
type(cce_loss_type) :: loss

! Set up optimiser
optimiser = adam_optimiser_type(learning_rate=0.001)

! Set up loss function
loss = cce_loss_type()

! Compile network
call net%compile(optimiser=optimiser, loss_method=loss, accuracy_method="mse")

The accuracy method is optional for train() and test().

If accuracy_method is provided during compilation, athena computes and prints accuracy alongside loss. If it is not provided, training and testing run in loss-only mode.

See train() Subroutine for full logging and print options.

Complete Example

Here’s the complete program:

program basic_network
  use athena
  implicit none

  type(network_type) :: net
  type(adam_optimiser_type) :: optimiser
  type(cce_loss_type) :: loss
  integer :: num_inputs, num_hidden, num_outputs

  ! Define dimensions
  num_inputs = 784
  num_hidden = 128
  num_outputs = 10

  ! Build network
  call net%add(full_layer_type(num_inputs=num_inputs, num_outputs=num_hidden, activation="relu"))
  call net%add(full_layer_type(num_outputs=num_hidden, activation="relu"))
  call net%add(full_layer_type(num_outputs=num_outputs, activation="softmax"))

  ! Compile
  optimiser = adam_optimiser_type(learning_rate=0.001)
  loss = cce_loss_type()
  call net%compile(optimiser=optimiser, loss=loss)

  ! Print network summary
  call net%print_summary()

end program basic_network

Network Variations

Below are some variations you can try to modify the network architecture, optimiser, and loss function.

Different Architectures

Different architectures can be achieved through adding or modifying layers in the network. For a full list of available layers, see Layers or refer to the API.

Deeper Network:

call net%add(full_layer_type(num_inputs=num_inputs, num_outputs=256, activation="relu"))
call net%add(full_layer_type(num_outputs=128, activation="relu"))
call net%add(full_layer_type(num_outputs=64, activation="relu"))
call net%add(full_layer_type(num_outputs=num_outputs, activation="softmax"))

With Dropout for Regularisation:

call net%add(input_layer_type(input_shape=[num_inputs]))
call net%add(full_layer_type(num_outputs=128, activation="relu"))
call net%add(dropout_layer_type(rate=0.5, num_masks=10))
call net%add(full_layer_type(num_outputs=128, activation="relu"))
call net%add(dropout_layer_type(rate=0.5, num_masks=10))
call net%add(full_layer_type(num_outputs=num_outputs, activation="softmax"))

With Batch Normalisation:

call net%add(input_layer_type(input_shape=[num_inputs]))
call net%add(full_layer_type(num_outputs=128, activation="relu"))
call net%add(batchnorm1d_layer_type(num_channels=128))
call net%add(full_layer_type(num_outputs=128, activation="relu"))
call net%add(batchnorm1d_layer_type(num_channels=128))
call net%add(full_layer_type(num_outputs=num_outputs, activation="softmax"))

For more complex architectures, such as convolution, residual, or physics-informed networks, refer to the respective tutorials in the Layers section.

Different Optimisers

Different optimisation algorithms can be used to train the network. The different optimisers are suited to different types of problems and datasets. For a full list of available optimisers, see Optimisers.

SGD with Momentum:

optimiser = sgd_optimiser_type( &
     learning_rate=0.01, &
     momentum=0.9, &
     nesterov=.true.)

RMSprop:

optimiser = rmsprop_optimiser_type( &
     learning_rate=0.001, &
     beta=0.9)

Different Loss Functions

Choice of loss function depends on the task at hand. Different loss functions guide the learning process in different ways and can be used, in some cases, to encode prior knowledge about the problem. For a full list of available loss functions, see Loss Functions.

For Regression:

type(mse_loss_type) :: loss
loss = mse_loss_type()

For Binary Classification:

type(bce_loss_type) :: loss
loss = bce_loss_type()

Next Steps

Now that you’ve created a basic network, learn how to:

See Also