Every time a machine reads a photograph — a coral reef, a chest scan, a sorting belt — it does not see what you see. It sees a grid of numbers. That sentence is not a figure of speech. By the end of this chapter you will know exactly what the grid contains, how the machine extracts patterns from it, and how you can build and test a real image-classification tool yourself.
Think of the chapter as an eye-test chart with three rows. The top row is the whole photograph — the friendly object you recognise instantly. The middle row is the pixel grid — the machine's version of that photograph. The bottom row is a patch of numbers inside one corner of the grid — what arithmetic actually runs on. Moving down one row means moving one step closer to what the machine truly holds. This chapter is that downward trip.
Why does the machine need numbers at all? Because a machine cannot hold a photograph the way a camera holds light — it needs a version that arithmetic can work on, one where every part of the image has a precise, storable value. The pixel is the unit that makes this possible.
A *pixel* is the smallest unit of a digital image: a single coloured dot. An image is a rectangular grid of pixels. The machine's version of a photograph is this grid — nothing more. When an image is zoomed in far enough, the individual dots become visible. The machine does not zoom: it begins from the dots.
Note the exam trap here. A pixel is a unit of a digital image, not a unit of human vision. How finely a human eye can resolve detail is a different question entirely. A pixel is a data construct — a cell in a table — and the image IS that table.
*Resolution* is the number of pixels in an image, stated as width × height. A 640 × 480 image contains 640 × 480 = 307,200 pixels. Higher resolution means more pixels, which means more detail — but also more numbers for the machine to process at every stage of the pipeline. Resolution names the cost of processing an image, and that trade-off never goes away. A useful anchor: a small pixel-art canvas of 8 × 8 holds just 64 numbers. A standard-definition photograph at 640 × 480 holds 307,200. Double the width and height and you quadruple the count. Every extra pixel is more arithmetic — which is why CV systems often resize images to a fixed small resolution before passing them to a model.
What does each pixel actually contain? A pixel's value is a number. In a *grayscale image* each pixel holds exactly one number: `0` = black, `255` = white, values in between are shades of grey. The machine reads any grayscale photograph as a 2D array — a table of rows and columns — of these numbers. This is what "the machine sees numbers, not a scene" means in practice: the image IS the number grid.
Notice what this implies. Two photographs — a face and a building — look completely different to you. To the machine, both are patterns of numbers between `0` and `255`. A face where the background is bright and the skin is mid-tone is a different distribution of numbers from a building with strong vertical contrast — but in both cases what the machine holds is a table of integers, nothing more. The scene is never present in memory.
Grayscale images carry one number per pixel. Colour images carry three. An *RGB image* stores three values at every pixel position: a Red value, a Green value, and a Blue value, each in the range `0` to `255`. A 640 × 480 colour image is therefore three overlaid grids — one for red intensities, one for green, one for blue — stacked on top of each other. The machine stores 640 × 480 × 3 = 921,600 numbers in total.
The three channels are not blended together in storage. The red grid, the green grid, and the blue grid are three separate arrays. Any colour visible on screen is produced by combining one value from each channel at the same pixel position. For exam purposes, the calculation that is almost always tested is the ×3 step: `width × height × 3`. A 100 × 80 RGB image stores 100 × 80 × 3 = 24,000 numbers — not 8,000. Omitting the ×3 is the standard mark-losing move.
Worth pausing here — this one fools almost everyone. "The machine sees shapes and colours in the photograph — it just recognises them like we do, only faster." That is not what happens.
The machine holds a grid of numbers. It has no concept of "shape" or "colour" as perception. Shape and colour emerge only as statistical patterns in how the numbers vary across positions. The machine never has a visual experience of the scene. It runs arithmetic on integers.
This is not a philosophical point with no consequences. It has a direct practical consequence. The machine cannot tell that two very different-looking pixel arrays show the same object. It can do this only if it has learned the statistical pattern from training examples. Consider a photograph of a face under strong side-lighting. The same face under soft frontal light has different pixel values. To the machine, these can look like two unrelated grids of numbers. That is why training data diversity matters. It is also why a model trained on indoor photographs fails on outdoor ones. The pixel statistics are different.
You now know that an image is a grid of numbers. But a raw pixel grid is not very useful by itself. Two photographs of the same face, taken under different lighting, will have different pixel values. The face itself is the same. The machine needs a way to extract patterns that survive these variations — edges, textures, shapes. *Convolution* is the operation that does this.
Convolution is a mathematical operation. A small grid of numbers, called the *kernel or filter, slides across the pixel grid one position at a time. At each position, the kernel multiplies its own values against the pixel values beneath it. It then sums all the results. That sum is one output number. After the kernel has slid across the whole image, the full set of output numbers forms a new grid. This grid is called the feature map*.
Three things to hold onto: the kernel, the slide, and the feature map. The kernel is the small grid doing the detecting. The slide means it moves one step at a time. The feature map is the output. At each position the arithmetic is just nine multiplications and one sum. The power comes from repeating this at every position across a 640 × 480 image — hundreds of thousands of times. Many different kernels run in parallel, each doing this same simple arithmetic.
What is the kernel, exactly? A kernel is a small grid of numbers — typically 3×3 or 5×5. Its values encode what kind of pattern it is looking for. Different kernels detect different things. An edge-detection kernel highlights positions where pixel values change sharply — the edge between a light and dark region. A blur kernel averages nearby pixels and smooths the image. A sharpening kernel amplifies local contrast.
The key insight: a kernel IS a feature detector. Applying a convolution with a particular kernel asks one specific question of the pixel grid: "does this pattern appear here?" The answer, at each position, is the output number in the feature map.
This connects to what you already know about features. In earlier classes, "extracting features" was treated as a step the pipeline simply did. In a CV pipeline, the convolution kernel IS the extractor. Its numbers define what counts as a feature. The feature map records where that feature was found. In a CNN, the kernel values are not hand-designed. They are learned from training data.
The output of applying one convolution kernel to the full pixel grid is the *feature map*: a new grid with one number per position. Each number records how strongly the kernel's pattern was present at that location in the original image.
In an edge-detection feature map, bright positions mark where sharp edges were found; dark positions mean no edge there. In a texture-detection map, bright positions mark the texture the kernel was tuned to find. A CNN applies many kernels to the same image, producing many feature maps — each one a different "question answered" about the pixel grid.
A worked example makes this concrete. Take a 5×5 patch of pixel values from an image and a 3×3 edge-detection kernel with top row `[-1, -2, -1]`, middle row `[0, 0, 0]`, bottom row `[1, 2, 1]`. Place the kernel on the top-left 3×3 sub-patch.
Multiply each kernel value against the pixel value beneath it — nine multiplications. Sum the nine products — one number. That number is the feature-map value at this position. Slide the kernel one step right and repeat. After all positions, the feature map is complete.
In Chapter 4 you met the neural network: layers of nodes, weighted connections, and a learning process that adjusts those weights. Computer Vision uses a specialised version of the same idea. A *CNN (Convolutional Neural Network)* stacks convolution operations as its first layers. A plain neural network starts from raw numbers fed straight in. A CNN starts from pixel grids instead. It applies learned kernels to extract feature maps first, then feeds those maps into the same fully-connected layers. The word "convolutional" names exactly this front end.
A CNN's vision front end has three kinds of layers, in order. First, the *convolutional layer: applies learned kernels to the pixel grid and produces feature maps — one map per kernel. Second, the pooling layer: shrinks each feature map by keeping only the strongest value in each small region. For example, it keeps the maximum value in each 2×2 patch. This reduces computation while preserving the key signals. Third, the fully-connected layer*: takes the flattened feature maps and classifies them, exactly like a plain neural network. The sequence is worth memorising: kernels first, pooling second, fully-connected last.
Why not feed raw pixels directly into a dense layer? A 640 × 480 × 3 image is 921,600 numbers. Connecting every one of those to a dense layer would be computationally enormous. Convolution layers reduce the data to compact feature maps that capture local patterns. The dense layers then classify from those compact representations, not from the raw 921,600 numbers.
A second point trips up exam answers: "A CNN has stored a picture of a cat — that is how it knows what a cat looks like." A CNN learns numerical kernel weights. These weights fire strongly under one condition. The pixel statistics of the input must match the statistics of past training examples for that class label.
The convolutional layers detect edges first. Then they detect textures. As the maps pass through deeper layers, they detect increasingly abstract shapes. But there is no stored image anywhere in this process. There are only weights that activate on matching pixel patterns. There is no picture in the machine, just as there was no scene. There is only arithmetic on numbers.
The practical consequence is the same. The model fails when the pixel statistics of a new image fall outside the distribution it was trained on. A CNN trained on studio photographs of cats may fail on photographs taken in dim light. The cat is not missing. The pixel statistics are different enough that the learned weights do not fire.
You have traced the theory: image → pixel grid → convolution → feature maps → classification. Three tools put that pipeline in your hands without requiring any code. *Teachable Machine* (teachablemachine.withgoogle.com) lets you collect webcam images for two or more classes. You train a CNN classifier right in the browser. Then you test with your live camera and watch the confidence bars shift in real time. The confidence bar is the model's output — "probability that this input belongs to each class." An ambiguous image splits the bar between classes.
*Lobe* (lobe.ai) lets you import a folder of labelled images. It trains automatically and shows you a Results panel with the examples the model got wrong. Examining the wrong examples is model evaluation in practice. It is the same skill as reading a confusion matrix, shown to you directly.
*Orange Data Mining* (orangedatamining.com) runs the coral-bleaching image dataset through an Image Embedding node, then a classifier, then a confusion matrix. It connects the CV pipeline to SDG 14 (Life Below Water). Early detection of coral bleaching can support conservation decisions.
All three share one pipeline shape: collect labelled images → train a model → test it → interpret the results. The new Grade 10 skill is the last step. In Class 8 you described what a pipeline did. In Class 10 you decide whether the result is trustworthy. That means reading the output carefully, not just clicking Train.
The confusion matrix, precision, and recall you learned in Chapter 6 apply directly to image classifiers. The coral-bleaching project in Orange is where this becomes concrete. Suppose a model correctly identifies 95% of healthy coral images. It identifies only 60% of bleached ones.
Overall accuracy might still read 90%. That is because healthy corals are far more numerous in the dataset. Doing well on the common class is enough to push the aggregate number up. The 40% miss-rate on bleached corals is hidden inside that 90%. Bleached corals are the class that matters most for conservation.
Reporting "overall accuracy = 90%" here is the honest-sounding but misleading crop. The honest report names precision and recall per class, or shows the full confusion matrix. A bleach-detection tool that misses 40% of bleached corals is dangerous for conservation decisions. That is true whatever the overall number says.
This is not a new concept. It is the same precision-and-recall logic from Chapter 6. Here it applies to a pixel-based classifier rather than a tabular one. Evaluation tools are domain-agnostic. That is why the viva questions for this chapter overlap with both Chapter 6 and Chapter 8.