Artificial Neuron

A mathematical processing unit that scales inputs by synaptic weights, adds a bias threshold, and fires an output through a non-linear activation function. It translates incoming numerical signals into a single decision boundary.

Why Does This Exist?

In 1943, neurophysiologist Warren McCulloch and logician Walter Pitts set out to explain how animal nervous systems could generate complex computational behavior from interconnected, microscopic biological cells. A biological neuron receives electrical impulses along branched dendrites, accumulates ionic charge in the cell body (soma), and abruptly fires an action potential down its axon when the net membrane voltage surpasses a metabolic firing threshold.

Without a mathematical formalization of this mechanism, early computing machines could only execute hand-wired boolean logic gates. The artificial neuron replaced rigid circuits with parameterized, continuous mathematical functions. By assigning a real-valued numerical weight to each incoming connection and adding a trainable bias offset, a single artificial unit can modulate the relative importance of diverse sensory inputs and define an oriented hyperplane dividing multi-dimensional space.

Connecting artificial neurons in layers forms the foundation of modern multi-layer perceptrons. Understanding the single unit's arithmetic—its weighted sum, its bias translation, and its non-linear activation—is an essential prerequisite before studying activation functions, gradient descent, and backpropagation.

Think of It Like This

A loan committee judge weighing dossier scores

Picture a senior credit officer evaluating a commercial loan application. The dossier contains three separate numbers: annual revenue, existing corporate debt, and years of continuous operation.

The officer does not treat every number equally. Instead, they assign an explicit weighting factor to each metric: high revenue receives a large positive multiplier, heavy debt receives an aggressive negative penalty, and operating years provide a modest positive boost.

After multiplying each dossier metric by its personal weight, the officer sums the results into a single composite score and adds a baseline institutional policy adjustment—the bias. If economic conditions are tightening, the baseline bias shifts downward, requiring every applicant to present stronger raw numbers just to stay afloat. Finally, the officer submits the final composite score to a strict underwriting protocol: if the tally exceeds zero, the loan is authorized; otherwise, it is rejected on the spot.

How It Actually Works

Affine Combination and the Decision Surface

An artificial neuron maps an input vector $x = [x_1, x_2, \dots, x_d]^\top \in \mathbb{R}^d$ to a scalar output $y \in \mathbb{R}$ through two consecutive stages: an affine linear combination followed by an element-wise non-linear activation.

First, the neuron calculates the pre-activation scalar $z$ , representing the inner product of the input vector $x$ and the learnable synaptic weight vector $w = [w_1, w_2, \dots, w_d]^\top \in \mathbb{R}^d$ , shifted by a scalar bias $b \in \mathbb{R}$ :

z = \sum_{i=1}^d w_i x_i + b = w^\top x + b

The weight vector $w$ establishes the orientation and sensitivity of the neuron to each feature dimension, while the bias term $b$ translates the decision boundary away from the coordinate origin.

Second, the scalar $z$ passes into a non-linear activation function $\sigma: \mathbb{R} \to \mathbb{R}$ :

y = \sigma(z) = \sigma(w^\top x + b)

The historical evolution of the activation function defines the core milestones of connectionist architectures:

McCulloch-Pitts Neuron (1943): Binary threshold with fixed, non-learnable excitatory and inhibitory weights: $\sigma(z) = 1$ if $z \ge 0$ , else $0$ .
Rosenblatt Perceptron (1958): Learnable weights updated by mistake-driven corrections, using the Heaviside step function $\Theta(z)$ .
Modern Artificial Neuron: Smooth, continuously differentiable activations such as the logistic sigmoid $\sigma(z) = \frac{1}{1 + e^{-z}}$ , the hyperbolic tangent $\tanh(z)$ , or the Rectified Linear Unit $\text{ReLU}(z) = \max(0, z)$ .

Geometrically, setting $z = 0$ defines a $(d-1)$ -dimensional linear hyperplane in $\mathbb{R}^d$ :

w^\top x + b = 0

The weight vector $w$ serves as the normal vector perpendicular to this hyperplane, pointing in the direction of the positive half-space where $z > 0$ . The perpendicular Euclidean distance from the origin to the hyperplane is given by $\frac{|b|}{\|w\|_2}$ .

Worked Example

Consider a neuron with $d = 3$ input features, operating with weights $w$ , bias $b$ , and a logistic sigmoid activation function $\sigma(z) = \frac{1}{1 + e^{-z}}$ .

Let the inputs, weights, and bias be:

x = \begin{bmatrix} 2.0 \\ -1.5 \\ 0.5 \end{bmatrix}, \quad w = \begin{bmatrix} 0.8 \\ -1.2 \\ 0.4 \end{bmatrix}, \quad b = -0.50

Compute the pairwise products:

w_1 x_1 = 0.8 \times 2.0 = 1.60

w_2 x_2 = (-1.2) \times (-1.5) = 1.80

w_3 x_3 = 0.4 \times 0.5 = 0.20

Sum the weighted inputs:

\sum_{i=1}^3 w_i x_i = 1.60 + 1.80 + 0.20 = 3.60

Add the scalar bias $b$ :

z = 3.60 + (-0.50) = 3.10

Pass $z = 3.10$ through the logistic sigmoid function:

e^{-3.10} \approx 0.045049

y = \sigma(3.10) = \frac{1}{1 + 0.045049} = \frac{1}{1.045049} \approx 0.9569

Because $y \approx 0.9569 > 0.5$ , the neuron strongly fires a positive classification. Under a ReLU activation, the output would be $\max(0, 3.10) = 3.10$ .

To observe gradient propagation during training, the derivative with respect to weight $w_1$ evaluates to:

\frac{\partial y}{\partial w_1} = \sigma'(z) \cdot x_1 = y(1 - y) \cdot x_1 = 0.9569 \times (1 - 0.9569) \times 2.0 \approx 0.0825

Code

1import numpy as np2
3def artificial_neuron(4    x: np.ndarray,5    w: np.ndarray,6    b: float,7    activation: str = "sigmoid"8) -> tuple[float, float]:9    """Compute the forward pass of a single artificial neuron."""10    # Pre-activation affine combination: z = w^T * x + b11    z: float = float(np.dot(w, x) + b)12
13    if activation == "sigmoid":14        y: float = 1.0 / (1.0 + np.exp(-z))15    elif activation == "relu":16        y = max(0.0, z)17    elif activation == "step":18        y = 1.0 if z >= 0.0 else 0.019    else:20        raise ValueError(f"Unsupported activation: {activation}")21
22    return z, round(y, 4)23
24# Feature vector, weight parameters, and bias25x_input = np.array([2.0, -1.5, 0.5], dtype=np.float64)26weights = np.array([0.8, -1.2, 0.4], dtype=np.float64)27bias = -0.528
29z_val, y_sigmoid = artificial_neuron(x_input, weights, bias, activation="sigmoid")30_, y_relu = artificial_neuron(x_input, weights, bias, activation="relu")31_, y_step = artificial_neuron(x_input, weights, bias, activation="step")32
33print(f"Pre-activation z: {z_val:.2f}")34# -> Pre-activation z: 3.1035
36print(f"Sigmoid output: {y_sigmoid}")37# -> Sigmoid output: 0.956938
39print(f"ReLU output: {y_relu}")40# -> ReLU output: 3.141
42print(f"Step output: {y_step}")43# -> Step output: 1.0

Watch Out For

Omitting the bias term collapses the decision hyperplane through the origin

When the bias term $b$ is omitted, the neuron's decision boundary reduces to $w^\top x = 0$ . This forces the separating hyperplane to pass directly through the coordinate origin $(0, 0, \dots, 0)$ regardless of the weight values.

In real-world classification problems, data clusters rarely align symmetrically around the origin. For instance, if all benign examples have feature values near $(5.0, 5.0)$ and all anomalous examples cluster near $(4.0, 4.0)$ , an origin-constrained hyperplane cannot separate them without misclassifying substantial portions of the distribution. Always ensure that the bias $b$ is instantiated as a trainable parameter or incorporate an augmented constant feature $x_0 = 1$ with weight $w_0$ .

The Quick Version

An artificial neuron computes an affine combination $z = w^\top x + b$ before applying a non-linear activation function $\sigma(z)$ .
The weight vector $w$ determines the tilt and orientation of the separating hyperplane, while the scalar bias $b$ translates the boundary relative to the origin.
Non-linear activations enable networks of neurons to escape linear separability limitations and approximate complex decision surfaces.