Art and Heritage Analysis
Vision reads paintings and ruins to name artists, spot fakes, and rebuild broken pieces, so curators preserve more with less handling.
Why Does This Exist?
Museums hold millions of fragile items. Unrolling a scroll risks cracks. Scraping pigment risks loss. Art vision exists to read without touching: attribute a painting, flag a fake, reassemble fresco shards, transcribe manuscripts from images.
Style differs from objects. Image classification names a cat. Image retrieval finds lookalikes. This page is the heritage stack: high resolution capture, texture and pigment features, and expert in the loop calls.
Basics first: brushstroke texture is measured by local patterns at 1 mm scale. Pigment is read by multispectral bands beyond RGB. Varnish yellows both, so old photos need recalibration.
Think of It Like This
A conservator with a magnifier and a lamp
A conservator leans in with a magnifier for cracks, shines a side lamp for impasto ridges, and checks a pigment chart for era blues. Vision is that bench at scale: texture is the magnifier, spectra are the lamp, and the archive is the chart. The analogy stops at value: the conservator weighs history, while the model only scores pixels and must defer on contested works.
How It Actually Works
Capture uses raking light for relief, multispectral stacks for pigment, and X ray for underdrawings. A texture branch scores stroke statistics. A spectra branch maps pigment mixtures. A layout branch reads composition. Scores fuse, then a curator reviews borderline calls with provenance records.
Varnish and restoration are the assumptions. Yellowed varnish shifts blues to greens. Repainted areas carry modern pigments. Systems flag restored regions from UV fluorescence and downweight them in attribution.
Worked attribution set
Seventy four disputed paintings go through review. The model holds 62 with experts, mistakes 7 varnished works, and misses 5 heavily restored ones. Agreement is 62 divided by 74 which equals 0.838. Varnish errors are 7 divided by 12 which equals 0.583 of all errors, so the next capture fix is varnish correction, not a bigger backbone.
Watch Out For
Training on clean museum scans only
Lab scans are flat lit and unvarnished, while church walls are curved and smoky. The symptom is strong test scores with field failures on frescoes. Fix it by mixing raking light and aged surfaces in training and by reporting sites separately.
The Quick Version
- Heritage vision fuses texture, pigment, and layout for attribution.
- Raking light and multispectral capture matter more than backbone size.
- Our set agreed on 62 of 74, rate 0.838.
- Varnish caused most errors, so correct it before scaling models.
- Keep experts in the loop for contested and restored works.