From One Neuron to a Network: A Multilayer Forward Pass With NumPy

In the previous posts, we built a single neuron and explored why one neuron cannot solve every pattern. The next step is to connect neurons into layers. A layer takes the values from the layer before it, computes one weighted sum for each neuron, and passes the results onward. Repeating that operation gives us a forward pass: input goes in, layer by layer, and a prediction comes out.

Describe the network by its layer sizes

We will use a network with three input values, two hidden neurons, and one output neuron. The list [3, 2, 1] describes those sizes from left to right. It does not count a separate neuron for the bias: each neuron's bias is stored alongside its weights.

layer_sizes = [3, 2, 1]

Each pair of neighboring sizes determines a weight matrix and a bias vector. For a transition from n_in inputs to n_out neurons, the weight matrix has shape (n_out, n_in). Each row contains all the weights for one neuron. The bias vector has one value per neuron, so its shape is (n_out,).

# 3 inputs -> 2 neurons: weights (2, 3), biases (2,)
# 2 neurons -> 1 neuron: weights (1, 2), biases (1,)

Initialize each layer

The constructor walks through each adjacent pair in layer_sizes. NumPy's random number generator creates the weights and biases, and giving it a seed makes the same numbers appear every time the network is created. Keeping each layer's parameters in a list makes it possible to handle networks with different numbers of layers.

rng = np.random.default_rng(seed)

for n_in, n_out in zip(layer_sizes[:-1], layer_sizes[1:]):
    weights.append(rng.random((n_out, n_in)))
    bias.append(rng.random(n_out))

This uniform random initialization is convenient for a first forward-pass example, but it is not a recommended general-purpose training initialization. In practice, methods such as He or Xavier initialization choose a scale based on the layer width so signals and gradients are less likely to grow or shrink too much.

What one layer computes

For a neuron with weight vector w, input vector x, and bias b, the weighted sum is the dot product plus the bias:

z = np.dot(x, w) + b

The first layer has two weight rows, one per neuron. Taking the dot product of the input with each row produces one number per neuron. The hidden layer applies ReLU, max(0, z), which keeps positive values and turns negative ones into zero. The final layer in this example returns its weighted sum unchanged. That is a linear output, often useful for a regression output; a classifier would typically use an output activation appropriate to its task.

Implement the forward pass

Here is the complete network. On every layer, last_output is the input to that layer. After computing all its neurons, the new outputs become the input to the next layer.

import numpy as np


class MultipleNeuron:
    def __init__(self, layer_sizes: list[int], seed: int = 42) -> None:
        self.layer_sizes = layer_sizes
        self.weights = []
        self.bias = []

        rng = np.random.default_rng(seed)

        for n_in, n_out in zip(layer_sizes[:-1], layer_sizes[1:]):
            self.weights.append(rng.random((n_out, n_in)))
            self.bias.append(rng.random(n_out))

    def activation_function(
        self, forward_measure: float, is_output_layer: bool = False
    ) -> float:
        if is_output_layer:
            return forward_measure
        return max(0.0, forward_measure)

    def forward_pass(self, single_input: np.ndarray) -> np.ndarray:
        last_output = single_input

        for i, layer_weights in enumerate(self.weights):
            outputs = []
            is_output_layer = i == len(self.weights) - 1

            for j, neuron_weights in enumerate(layer_weights):
                forward_measure = (
                    np.dot(last_output, neuron_weights) + self.bias[i][j]
                )
                outputs.append(
                    self.activation_function(
                        forward_measure,
                        is_output_layer=is_output_layer,
                    )
                )

            last_output = np.array(outputs)

        return last_output

The nested loops make the calculation visible. The outer loop selects a layer. The inner loop selects one row of that layer's weight matrix, computes that neuron's weighted sum, adds its bias, and applies the activation. Once every neuron has produced an output, np.array(outputs) gathers them into the vector passed to the next layer.

Trace a prediction

With the default seed, we can pass in three input values and print the final one-element output:

nn = MultipleNeuron(layer_sizes=[3, 2, 1])
example_input = np.array([10.0, 2.0, 9.0])

print("Input:", example_input)
print("Prediction:", nn.forward_pass(example_input))

The hidden layer produces approximately [17.1058, 16.7287], and the output layer produces approximately [10.0967]. These numbers are reproducible because the initializer uses seed 42. The returned value is a NumPy array because the network's output layer contains one neuron.

Notice that this is only a forward pass, not a trained model. The parameters are random, so the prediction has no learned meaning. We have not provided target values, a loss function, or a way to adjust the weights. Forward propagation answers “what does this network predict with its current parameters?” Training will answer “how should those parameters change to make the predictions better?”

A useful shape check

For this architecture, the input has shape (3,). Multiplying it by a row of the first layer's weights gives one scalar per hidden neuron, so the first layer's output has shape (2,). The second layer consumes that vector and returns shape (1,). Each layer's output therefore has exactly the size declared in layer_sizes.

This implementation expects a single input example at a time. It does not yet support a batch of inputs, and the type annotation does not validate that the input length matches the first layer. Those are useful next improvements once the single-example calculation is clear.

From layers to learning

We now have the structure that was missing from a single neuron: one layer can transform the inputs, and a later layer can combine those transformed values. The forward pass is the prediction half of a neural network. In the next step, we can compare that prediction with a target and use the error to update the weights and biases.