PyTorch Tensors for Vision

PyTorch packs photos as channels-first tensors on the GPU and learns from them with gradients, so one shape convention governs the whole training loop.

A PyTorch image is channels first: 3 planes of 224 by 224 hold 150528 numbers

Why Does This Exist?

Training moves millions of numbers through the same operations thousands of times, and it needs those numbers on the GPU, tracked for gradients, in a layout the fast kernels expect. PyTorch's tensor is that container: a shaped, typed array that lives on CPU or GPU, records every operation for differentiation, and batches neatly. The training loop, the data pipeline, and the saved model all speak tensor.

The one convention to internalize is channels-first. Where everyday arrays store $(H, W, 3)$ , PyTorch wants $(N, C, H, W)$ : batch, channels, height, width. A single photo is $(3, 224, 224)$ , which is $3 \times 224 \times 224 = 150{,}528$ numbers, exactly as the diagram counts. This page covers tensors, the layout, loading batches, and the training step. Reusing trained weights belongs to transfer learning.

Think of It Like This

Sheet music rewritten for a different instrument

A piano score and a trumpet part hold the same notes in different orders: the piano stacks hands vertically, the trumpet runs one line. A pianist sight-reading the trumpet part plays valid, wrong music, because every symbol is legal and the order is off.

Everyday arrays are the piano score, channels last. PyTorch tensors are the trumpet part, channels first. Feeding one where the other belongs runs without error and sounds wrong: predictions scramble while shapes still multiply. The analogy stops at transposition: permute rewrites the part exactly, with no interpretation needed.

How It Actually Works

A tensor is a NumPy-style array with two extras: a device (CPU or GPU memory) and a gradient flag. Marking a weight tensor for gradients makes every operation record itself, so one backward call fills every gradient by the chain rule. A DataLoader stacks single $(C, H, W)$ images into $(N, C, H, W)$ batches with shuffling and parallel loading, so the GPU never waits on the disk.

One training step then reads the same every time. Move the batch to the GPU, run the model forward, score predictions against labels with a loss, run backward to fill gradients, and step the optimizer. Forgetting the gradient reset accumulates stale gradients across batches, which is why the zeroing call sits inside the loop rather than outside it.

Counting the layout

A 224 by 224 photo: $224^2 = 50{,}176$ positions per plane, times 3 channel planes gives $150{,}528$ numbers per image. Channels-first stores all 50,176 reds, then all greens, then all blues. Channels-last interleaves them pixel by pixel. Same count, different walk order, and the kernels are compiled for the first.

Code

The layout move, shown with the runnable array library underneath:

1import numpy as np2
3hwc = np.zeros((224, 224, 3), dtype=np.uint8)  # everyday order, channels last4nchw = np.transpose(hwc, (2, 0, 1))            # channels-first order5print(hwc.shape, nchw.shape)                   # -> (224, 224, 3) (3, 224, 224)6print(nchw.size)                               # -> 150528

The transpose is what every loader does before the batch reaches the model, and the 150,528 repeats the diagram's count. Real code spells it permute(2, 0, 1) on the tensor; the axes move identically.

Watch Out For

Feeding channels-last into a channels-first model

The symptom is confident garbage: loss falls a little, predictions look random, and every shape in the error-free run multiplies cleanly. Square inputs make it worse, because $(224, 224, 3)$ mistyped still looks plausible next to $(3, 224, 224)$ at a glance.

Fix it by printing the tensor shape right before the model call and reading all four numbers. $N$ first and $C$ second is the contract; anything else gets permuted before it travels further.

Evaluating with training still switched on

Dropout masks and batch statistics change every forward pass in training mode, so scores wobble between runs and degrade on single images. The symptom is an accuracy that shifts by points between identical evaluations, with no code change in between.

Fix it by switching the model to evaluation mode and disabling gradient tracking for every measurement run. Training randomness belongs to training; measurement must be deterministic.

The Quick Version

A tensor is a shaped array with a device and gradient tracking, so training, data and models share one container.
Photos ride channels-first: $(3, 224, 224)$ holds 150,528 numbers, reds then greens then blues.
Batches stack to $(N, C, H, W)$ through a loader that shuffles and prefetches off the critical path.
One step reads forward, loss, backward, optimizer step, with gradients zeroed inside the loop.
Channels-last input scrambles predictions without error, so read all four shape numbers first.
Evaluation runs in evaluation mode with gradients off, or training noise pollutes the score.