I Flipped One Bit in a Neural Network Until It Developed a Favorite Animal

TL;DR

• I loaded a pinned CIFAR-10 ResNet-20 checkpoint, reproduced its 92.60% test accuracy, and exhaustively searched every bit in all 640 weights of its final classifier layer.
• The selected float32 mutation changed one exponent bit in one weight, turning 0.3142956197 into 1.0694925739e38; the live model then predicted bird for 9,994 of 10,000 test images, while accuracy fell to 10.03%.
• The result has a very plain mechanism: final feature 8 was positive for exactly those 9,994 images, so the mutated bird weight overwhelmed every competing class score. It was zero for the other six images.
• Float16 produced the same kind of collapse in this isolated final-layer test. The best int8 mutation did not; that is a measurement for this quantizer and checkpoint, not a certificate that int8 models are safe.
• The complete artifact includes the runner, Colab notebook, 35,840-row candidate search, 10,000-row per-image trace, results, and offline verifier.

I changed this:

0x3ea0eb5b

into this:

0x7ea0eb5b

The neural network responded by identifying 9,994 images as birds, including the cars, ships, trucks, frogs, horses, and cats. It had not become better at ornithology; one number in the final layer had become so large that the other nine classes were basically submitting polite suggestions.

That result sounds like a cheap magic trick unless the search, bit pattern, checkpoint, and evaluation are inspectable, so I packaged all of them. Researchers have already demonstrated that targeted weight-bit attacks can hurt models; this experiment adds a complete trail from one flipped exponent bit to every changed prediction, without summoning the usual fog machine around neural networks.

The result before the mechanism

The model is the 272,474-parameter cifar10_resnet20 checkpoint from Yaofo Chen's PyTorch CIFAR Models repository, pinned at commit 786c162. Its repository reports 92.60% top-1 accuracy, and my untouched evaluation reproduced 92.60% on the complete CIFAR-10 test batch.

After the selected one-bit mutation, the same live model produced this:

The 10.03% accuracy is not a mysterious floor. CIFAR-10 has exactly 1,000 test images per class, so an almost-always-bird classifier gets almost every bird right by accident and almost everything else wrong. Three of its 1,000 correct answers came from the six images that escaped the collapse, which is how the result landed at 1,003 correct rather than exactly 1,000.

The official CIFAR-10 dataset page describes the test batch as 10,000 images with 1,000 examples from each of ten classes. Those images are only 32 by 32 pixels, which matters when interpreting the result: this is a clean fault experiment on a small vision benchmark, not a safety evaluation for a production perception system.

A classifier ends with ten weighted sums

The simplest useful model of the final layer is a spreadsheet with ten rows. The ResNet backbone turns each image into 64 feature values, then the last layer computes one score, or logit, for each class:

class_score[c] = bias[c] + sum(feature[j] * weight[c, j])

The predicted class is whichever row gets the largest score. There is a row for airplane, a row for automobile, a row for bird, and so on; nothing in this last operation is doing vision anymore, because the feature extractor has already done that work.

The tested architecture follows the residual-network family introduced by He, Zhang, Ren, and Sun in Deep Residual Learning for Image Recognition. In this particular implementation, the last residual block uses ReLU before global average pooling, so the 64 values entering the linear classifier are nonnegative. That small implementation detail is the entire bird factory.

The selected mutation changed fc.weight[2, 8], which connects feature 8 to class 2, or bird. For any image, the mutation adds this amount to the bird score:

feature[8] * (mutated_weight - original_weight)

Feature 8 was positive for 9,994 test images and exactly zero for six; it was never negative. The live mutated model predicted bird on exactly the same 9,994 rows where that feature was positive, while the six zero rows received no added bird score and kept an ordinary competition among the classes.

That one-to-one match is recorded in the per-image trace, including the true class, original prediction, feature activation, old bird logit, new bird logit, and final prediction for every test image. The mechanism is not an interpretation laid over the result afterward; it is an equality the verifier checks across all 10,000 rows.

The bit changed the exponent, not the bird knowledge

The selected float32 value started as:

0.31429561972618103

Its 32 stored bits were:

0 | 01111101 | 01000001110101101011011

Float32 divides those bits into one sign bit, eight exponent bits, and 23 fraction bits. The IEEE 754-2019 standard defines the floating-point formats and arithmetic behavior; the short version needed here is that the exponent controls the scale while the fraction controls the significant digits.

The mutation flipped bit 30, the highest bit of the exponent field:

0 | 11111101 | 01000001110101101011011

The sign and fraction stayed identical. The biased exponent moved from 125 to 253, so the value was multiplied by 2^128 and became:

1.0694925739330808e38

The new stored weight is finite, but 64 of the 100,000 output logits overflowed to infinity when the live model multiplied that weight by larger feature activations. I kept those outputs in the result instead of filtering them into respectability; argmax still selects the bird score, and the non-finite count is written into results.json.

PyTorch exposes the underlying representation cleanly enough for this experiment. Its Tensor.view(dtype) documentation describes viewing the same tensor data through a different dtype, and the runner uses equivalent NumPy views to XOR exactly one storage bit, verify the before-and-after hexadecimal words, write the mutated value into the live PyTorch parameter, evaluate it, and restore the original bits.

Nothing here improved, erased, or rewired a learned bird feature. One coefficient attached to an already-active feature became approximately 3.4e38 times larger; the final argmax behaved correctly, which was unfortunately the problem.

Exactly what I searched

I limited the search to the final 10 x 64 weight matrix because that scope is small enough to exhaust, explain, and publish as raw data. I did not search the convolutional backbone, biases, batch-normalization parameters, or every parameter in the checkpoint.

The evaluation had two stages:

1. I built a deterministic calibration set from the first 100 test images in each class, giving 1,000 balanced images.
2. I scored every candidate on that calibration set, selected one candidate with a fixed rule, and then evaluated that candidate once on all 10,000 test images.

For the premise, a candidate had to target one of the six CIFAR-10 animal rows: bird, cat, deer, dog, frog, or horse. Among finite candidates, the selection rule maximized the target-class share, then minimized calibration accuracy, then chose the smaller flat weight index and bit index. This is a targeted adversarial search, not a random cosmic ray with excellent comedic timing.

I repeated the final-layer storage search in three formats while keeping the captured ResNet backbone features in float32:

The complete candidate-search CSV contains the storage format, target class, weight index, bit index and field, old and new bit words, old and new effective values, finiteness, calibration accuracy, and target share. Ten float32 candidates and ten float16 candidates produced non-finite stored values and were excluded from selection; they remain in the CSV with finite=false.

The model code is pinned to one commit, while the 1,139,055-byte checkpoint must match this complete SHA-256 before the runner proceeds:

4118986f0df73003d572b0e397f0ac7b3f60af1f31aff3d2da164536e36f6ec8

The CIFAR-10 archive is also checked against the MD5 published by its creators, and results.json records an additional SHA-256 for both the archive and extracted test batch. That is slightly annoying, which is a healthy property for an experiment about a single corrupted bit.

Bird won the tie-break; every animal had candidates

The title says the model developed a favorite animal, but I should not pretend the search discovered a buried emotional commitment to birds. On the calibration set, float32 had at least one perfect-collapse candidate for every animal row:

Bird won because it is class index 2, the first animal row, and the deterministic tie-break eventually preferred its smaller flat weight index. The full 10,000-image evaluation was performed for that selected bird candidate; I am not reporting the other five calibration collapses as full-test results.

This actually makes the mechanism less magical and more useful. The checkpoint does not contain one cursed bird bit; a positive feature feeding an exponent-amplified weight can turn several output rows into dictators. Bird was simply first in line at the administrative desk.

Float32 and float16 escaped; int8 stayed in its cage

The float16 search found the same selected connection, feature 8 into bird, and flipped exponent bit 14. That changed the rounded weight from 0.314208984375 to 20,592, producing bird on 9,992 of 10,000 full-test images and 10.05% accuracy.

The int8 comparison behaved differently. I quantized the final-layer weights with one symmetric per-tensor scale, flipped every one of their 5,120 stored bits, and dequantized each candidate back into the float32 calculation. The strongest animal-target candidate changed one cat-row weight from approximately -0.309 to 1.213; cat reached 12.1% on calibration and 11.67% on the full test, while accuracy remained 92.26%.

The reason is dynamic range. An exponent-bit flip can multiply a floating-point value by an absurd power of two while preserving its remaining significant bits; this int8 scheme has only 256 stored patterns and one fixed scale, so a one-bit mutation stays within that bounded codebook.

That does not make int8 immune to weight corruption. Rakin, He, and Fan's Bit-Flip Attack paper specifically studied malicious bit selection in quantized neural-network weights and reported severe degradation with a small number of flips. Their attack, models, search method, layers, and number of mutations differ from mine; my int8 result only says that no single final-layer bit among these 5,120 candidates produced the floating-point-style class collapse under this quantizer.

Reproduce the experiment

The experiment directory contains the exact method and passing conditions. On Windows:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe run_experiment.py
.\.venv\Scripts\python.exe verify_results.py

On macOS or Linux, replace .venv\Scripts\python.exe with .venv/bin/python. The Colab notebook performs the same setup, run, verification, and chart display; upload it to Colab or open it from the published repository beside the runner.

The complete experiment also lives in the public Blog Labs repository. For an immutable citation, use v1.0.0 and its CITATION.cff; for presentations, download the embeddable SVG and exact summary CSV.

The first run downloads the 163 MB CIFAR-10 archive and 1.09 MB checkpoint. The recorded cached run took 13.79 seconds on an NVIDIA GeForce RTX 3050 Laptop GPU with Python 3.10.11, PyTorch 2.10.0+cu128, torchvision 0.25.0+cpu, and NumPy 1.26.4; download speed will dominate a cold run, while CPU feature extraction will take longer.

The output package has three useful levels:

• results.json states the claim, exact scope, checkpoint and dataset hashes, environment, bit words, selected mutations, precision comparison, feature statistics, and control outcomes.
• The CSV files expose every searched candidate, the animal-target summary, exact class distributions, per-class accuracy, and every test image's winning feature and prediction.
• verify_results.py checks all published hashes, candidate counts, the one-bit XOR invariant, distribution totals, accuracy threshold, and the equality between positive feature 8 and bird predictions without downloading the model or dataset.

Rerunning the full experiment validates the model execution; running the verifier validates that the packaged outputs have not drifted. Those are different jobs, so the artifact provides both.

What this result does not establish

The strongest defensible claim is narrow: for this exact ResNet-20 checkpoint, an exhaustive search of the final classifier found a finite one-bit float32 mutation that made bird account for 99.94% of predictions on the CIFAR-10 test batch.

It does not establish any of the following:

• that a random bit flip is likely to hit one of these candidates;
• that every neural network has an equivalent single-bit failure;
• that a stored checkpoint, CPU, GPU, DRAM module, or accelerator will experience this physical fault at a particular rate;
• that ECC, checksums, redundant execution, framework serialization, or deployment hardware would miss or correct it;
• that the float16 and int8 results represent full mixed-precision or quantized deployments, because only final-layer storage changed while the backbone features remained float32;
• that 1,000 calibration images prove behavior outside the evaluated 10,000-image test batch;
• that output collapse is the only damaging failure shape, since subtler class-specific errors may matter more in practice.

The search is exhaustive only inside a finite box: 640 final-layer weights times the bits in each tested storage format. It is empirical evaluation, not a proof about neural networks in general.

There is also selection pressure by construction. I searched for animal-target candidates because that was the premise, and I chose the best calibration result before touching the full test evaluation. The raw search makes that process auditable; it does not turn the process into random sampling.

What I would actually do with this

The obvious check is a cryptographic hash on the model artifact before loading it. That detects a corrupted file, and the runner refuses its checkpoint unless all 256 hash bits match. A hash does not catch a memory fault that occurs after verification, though, so it is one boundary rather than a complete runtime defense.

The less obvious lesson is that torch.isfinite(weights) would not have caught this selected parameter. The mutated weight itself remained finite; only some output logits overflowed. Useful runtime checks therefore depend on the system boundary:

1. Verify serialized model files and configuration with pinned hashes before load.
2. Validate parameter ranges or layer norms against a known-good manifest when the deployment threat model includes in-memory corruption.
3. Reject non-finite intermediate or output tensors when the application can fail closed.
4. Monitor aggregate prediction distributions for sudden entropy collapse, while remembering that a distribution alarm can miss targeted errors and natural data shifts can also trigger it.
5. Test the actual deployment format and hardware path, including quantization, compilation, and error-correction behavior, instead of treating a float32 notebook as a hardware qualification report.

The experiment also suggests a cheap regression fixture for model-serving stacks: mutate one controlled bit in a disposable copy of a classifier head, confirm that integrity or tensor-health checks fire, then restore the original artifact and verify its hash. The goal is not to teach production models to love birds; it is to make sure the alarm notices when one does.

FAQ

Can one bit flip break a neural network?

One bit broke this pinned model under this targeted final-layer search: accuracy fell from 92.60% to 10.03%, and bird took 99.94% of predictions. That measurement does not provide a universal probability or guarantee for other architectures, weights, layers, formats, or hardware.

Why did the model choose bird?

The search found perfect calibration candidates for all six animal rows. Bird won a deterministic tie-break because it had the smallest eligible class and weight index; the selected bird weight then dominated whenever feature 8 was positive.

Did the mutation change only one parameter and one bit?

Yes. 0x3ea0eb5b XOR 0x7ea0eb5b equals 0x40000000, which has exactly one set bit. The runner wrote that pattern into one live parameter, checked that live predictions matched the analytical final-layer calculation, and restored the original pattern afterward.

Is int8 safe from bit flips?

No such conclusion follows. This one-bit, final-layer, symmetric int8 search did not find a class-collapse candidate, but prior research has demonstrated damaging multi-bit attacks on quantized networks under different models and search methods.

Sources and artifact map

• Pinned PyTorch CIFAR Models source and ResNet-20 checkpoint
• CIFAR-10 dataset description, test split, and archive checksum
• IEEE 754-2019 floating-point standard
• PyTorch Tensor.view documentation
• Deep Residual Learning for Image Recognition
• Bit-Flip Attack: Crushing Neural Network With Progressive Bit Search
• Reproducible runner, Colab notebook, raw candidate data, per-image trace, results, and verifier

Conclusion

One exponent bit changed a perfectly ordinary bird weight from 0.3142956197 to 1.0694925739e38; because its input feature was positive on 9,994 test images, bird won those same 9,994 class competitions and the model's accuracy collapsed to 10.03%.

The model appears to acquire a personality, which makes the result memorable; the useful explanation is less mystical because floating-point representation, a nonnegative feature, and an argmax account for every changed prediction. Float16 repeated the escape, this int8 comparison did not, and none of that extends beyond the published scope without another experiment.

The bird is not conscious. It is one exponent bit standing on a very large number.