Building a Lorcana card scanner that runs entirely in the browser

I had a small web app that turned typing into a CSV. Pick a set, type a collector number, check the preview, add the card. Fine for a handful of cards. Miserable for a shoebox full of them.

So I wanted to point a camera at a card instead.

What I ended up with recognises any of 3,188 Lorcana cards (at the moment of writing in august 2026) in about 8ms, runs at 10fps in a Web Worker, works offline after first load, and ships no ML model, no OpenCV, no WASM. The whole reference index is 898 KB.

Here’s how it works, plus the six times I got it wrong.

The reframing that made it easy

First instinct with “recognise a card from a photo” is to reach for a neural net, or at least OpenCV. I measured first instead, and the measurement rewrote the whole design.

Lorcana has 3,192 cards across 22 sets. That’s a tiny search space. If comparing one card to another is cheap, then comparing against all of them is still cheap. So I downloaded every card image and threw the candidate approaches at synthetic camera degradation, at full index scale.

Four of the findings deleted work:

Photometric robustness is free. A descriptor that’s mean-subtracted and scaled to unit length, compared with normalised cross-correlation, scored 98.5% top-1 under genuinely nasty abuse: brightness 0.72, gamma 1.5, hue shift, 1.9px blur, JPEG quality 30, downscaled to 240px. Brightness, white balance, blur and compression all get absorbed for free. No histogram equalisation. No local contrast tricks. Nothing.

1. 1Clean render
2. 2Its fingerprint
3. 3All six abuses at once
4. 4Its fingerprint. Near-identical

Geometric alignment is the entire problem. A 4% error in where you crop the card costs seven points of accuracy. An 8% error costs twelve. Every hour of effort belonged there, not in photometrics.

Brute force is fast enough. An exhaustive scan over all 3,186 references measured 2.3ms in plain JavaScript. I’d planned a two-stage perceptual hash prefilter and deleted it. It added a code path and a recall risk to optimise something that was already free.

Colour is useless here. A 4x6 RGB signature scored 0.9999 on exactly the pairs I needed to tell apart. Everything converts to grayscale in step one and never looks back.

A fifth finding shaped the product rather than the algorithm: roughly 18% of Lorcana cards share their artwork with a reprint, and no descriptor is ever going to separate those. That one echoes through everything below.

The fingerprint

Each card becomes 288 bytes.

Take the card, crop off the printed black border (a uniform 2.45mm, which is 19px of a 488px render), average what’s left into a 12x16 grid of grayscale tiles. That’s 192 bytes. Then take four bands around the outside of the full card (left, right, top, bottom) and average those into 96 more.

1. 1The card as the index has it
2. 2Grayscale. Colour is thrown away
3. 3Border cropped, 12×16 tiles, 192 bytes
4. 4The ring: 4 bands, 24 tiles each, 96 bytes

3,188 cards x 288 bytes = 898 KB, fetched once and cached.

Two things worth calling out.

Fingerprint the whole card face, not just the artwork. I measured both. Full card scored 98.8%, art window alone 97.5%. The frame, the nameplate and the text box all carry real signal, and the alignment search deals with crop error better than shrinking the region does.

The ring is a tiebreak, never a matcher. Two random cards score a median 0.766 on the ring, way too weak to identify anything. But it captures the frame and treatment that near-identical printings differ in, which turns out to matter a lot.

The comparison is ZNCC: subtract the mean, divide by the norm, take the dot product. The useful property is that it’s invariant to a·x + b applied to the whole descriptor. Make the card twice as bright, or lift every value by 40, and the score doesn’t budge. That one property is why all that photometric abuse costs nothing.

For calibration: two genuinely different Lorcana cards score around 0.75. A card against itself under heavy degradation scores 0.99. That gap is huge, which is the whole reason any of this works.

0.700.800.901.000.75two different cards0.766the ring, two random cards0.99the same card, abused

Matching: making alignment nearly free

Alignment is the real problem, so the matcher is built around searching over alignments cheaply.

The trick is a summed-area table. Build one integral image over the 176x246 grayscale patch, about 43,000 additions and roughly 0.2ms, and after that the average of any rectangle costs four lookups. Building a 192-tile descriptor for some new hypothetical crop becomes 768 array reads.

Which makes searching over crops affordable, so I search over a lot of them:

5 scales (0.88 to 1.12) x 5 dx x 5 dy = 125 alignment hypotheses

Scoring 125 hypotheses against all 3,188 cards would not be affordable, so the pipeline is a cascade:

1. the indexevery card3,188
2. pooled prefilter6×8 tiles, 48 dims400
3. full resolution12×16 tiles, 192 dims96
4. sparse refine45 hypotheses each16
5. dense refine125 hypotheses each1

The pooled prefilter (averaging 2x2 tile blocks into a quarter-size descriptor) earned its keep: total time halved from 14.3ms to 7.2ms with identical accuracy on every single scenario. That’s the good kind of optimisation, where the output doesn’t change at all.

Nice freebie: cards come out of a stack upside down constantly. Rotating a descriptor 180° maps tile (x, y) to (W-1-x, H-1-y), which in row-major order is just the reversed array. So every query gets scored both ways for the cost of one extra dot product, and “upside down” scores 100%.

The camera pipeline

<video> ─┬─► drawImage → 192x268 canvas → grayscale → edge-snap ──┐
         │                                                        │ snapped rect
         └─► drawImage(rect) → 176x246 canvas → grayscale ────────┘
                                    │
                          transfer ArrayBuffer (43 KB)
                                    ▼
                          Web Worker: match() → candidates
                                    ▼
                       temporal voting → confirm → add

The optics decision

Obvious design: big guide frame, fill the view with the card, get maximum pixels. That’s wrong, and the math says why.

A Lorcana card is 88mm tall. A typical webcam has roughly a 45° vertical field of view. For the card to fill 82% of the frame it has to sit about 13cm from the lens, which is closer than most built-in webcams can focus. They’re fixed-focus units set up for a face at arm’s length. You get a big, blurry card.

At 30cm the card covers about 45% of frame height, which on a 1080p feed is still 486 pixels. The matcher only ever sees a 176x246 patch. That’s roughly double the resolution I need.

So the guide frame is small: 45% of frame height by default, adjustable down to 30%, and the camera gets asked for 1920x1080. Distance is nearly free. Focus isn’t.

Scheduling

video.requestVideoFrameCallback() fires once per decoded frame and won’t hand you the same frame twice, unlike requestAnimationFrame. Firefox doesn’t have it, so there’s a setInterval fallback guarded on currentTime actually having moved.

Throttled to ~10Hz. The loop is single-flight: if a scan is still out in the worker, the tick gets skipped rather than queued. A backed-up queue makes the on-screen readout describe the past, which is worse than just running slower.

Where the time actually goes

Measured against a live 1280x720 stream:

stage	cost
drawImage(video → 192x268)	2.83ms
getImageData(192x268)	0.07ms
grayscale, 51k px	0.10ms
drawImage(video → 176x246)	2.82ms
getImageData(176x246)	0.05ms
main thread total	~5.9ms of a 100ms budget

I’d assumed getImageData was the expensive bit, the classic GPU-to-CPU readback warning everyone repeats. It isn’t. drawImage from a video element costs about 40x more than reading the pixels back, because that’s where the frame actually gets decoded and rescaled. Reading back off an already-rasterised small canvas is nearly free.

OffscreenCanvas and MediaStreamTrackProcessor are deliberately unused. The latter is Chromium-only and iOS Safari was a target. The former doesn’t help, because the <video> element lives on the main thread and drawImage has to happen there regardless.

The preview isn’t mirrored, either. Mirroring is a selfie convention. Here it’d just render every card’s text backwards.

Edge-snap

Finding the card’s real borders inside the guide, in about 120 lines, no CV library.

Sum the absolute horizontal gradient down each column, and the vertical gradient across each row. You get two 1-D projection profiles where card edges show up as peaks.

The nice part is looking for two candidate edges per side:

• the card’s outer boundary against the table
• the black border to artwork transition just inside it

The second one is the reliable one. It’s a fixed, high-contrast feature of the card itself, where the outer boundary vanishes completely when a black-bordered card sits on a dark desk. Whichever peak is more prominent wins, and if it’s the inner one you subtract the known border width to get back to where the true edge must be. Borderless cards have no inner transition, so the outer peak wins there on its own.

Then it validates: size within 0.78x to 1.18x nominal, aspect within 8% of 63:88, every peak at least 2.2x the profile median. Fail any of that and it falls back to the plain guide rectangle. Falling back beats refusing, because the alignment search already tolerates a loose crop and refusing just makes the scanner feel broken.

Temporal voting, and the playset trap

One good frame isn’t enough. Three of the last five frames have to agree on the same card (four of six, stricter margin, when edge-snap didn’t lock). Costs a fraction of a second, kills basically every single-frame fluke.

The subtle bit is re-arming, deciding when the next card is allowed to be accepted. Obvious rule: wait until the top match changes. That rule is a bug, and a very domain-specific one. A playset is four identical cards in a row, which is the normal case when you’re sorting a collection. It would silently swallow three of them.

So re-arm needs two consecutive frames scoring below threshold, meaning the card physically left the frame, plus a 600ms cooldown.

Six wrong turns

The design above reads pretty tidy. Getting there did not.

1. Widening the refine grid, when the coarse pass was the problem

A badly cropped card, 8% of background bleeding in around it, scored 30% top-1. Obvious fix was to widen the alignment search, so I did, and it went to 56%. Better. Not fixed.

The actual bottleneck was somewhere else entirely. Refinement only ever re-ranks the shortlist it’s handed, and the coarse pass that produced that shortlist sampled a single scale. The true card never made the top 48, so no amount of refinement was ever going to recover it.

Sweeping a coarse scale ladder too took it to 100%.

Lesson: when a ranking stage underperforms, check recall before precision.

2. Being confidently wrong

I fed the scanner a photo of “Mickey Mouse - Brave Little Tailor” from set 1. It answered D23 #1. Confidently. No prompt, no hesitation.

Three printings of that card exist: set 1 #115, promo P1 #1, and D23 #1. The build step groups cards whose artwork is indistinguishable, using a 0.985 correlation threshold. Set 1 and P1 scored 0.9862 and got grouped. D23 scored 0.9755, just under the bar, so it was treated as a perfectly distinguishable card. A slightly degraded photo was all it took to flip the ranking.

Fix was to stop trusting build-time grouping on its own. The border ring had only been used to separate members of a known group, so I generalised it to arbitrate any close contest, whatever the index thought. If the ring can’t separate the field, the scanner stops and asks instead of guessing.

Lesson: a threshold that classifies pairs at build time will always have something sitting just underneath it.

3. A feature that did absolutely nothing

Roughly 18% of Lorcana cards share artwork with a reprint. Set 9 “Fabled” alone reprints 50 cards from set 1, and the promo sets pile on more. The mitigation is a “sorting set” hint that narrows matching to the set you’re physically holding.

It silently did nothing at all.

The prefilter admits candidates above a score threshold found by histogram. When the scoped set had fewer cards than the prefilter quota, no threshold was ever reached, so the function returned a sentinel meaning “admit everything”. The exclusion test was scores[card] < threshold, and against that sentinel it excluded precisely nothing.

So the entire reprint strategy, the thing the whole product decision rested on, was inert. And it looked completely fine.

Lesson: features that fail open fail invisibly. The test that caught it asserted on the shortlist contents, not on the final answer.

4. Measuring the wrong thing

My accuracy harness reported “confidently wrong” answers regardless of score, including matches so weak the app would never display them. Which made a scenario look like it was failing a safety gate when the user would just have seen nothing at all.

Making the metric apply the same acceptance threshold the UI uses wasn’t moving the goalposts. It was measuring the product that actually exists. And it immediately surfaced something real the old metric had been hiding: at 5° of rotation, a big chunk of frames score too low to report at all.

Lesson: a metric that doesn’t model the product will lie to you in both directions.

5. Edge-snap had never worked. Not once.

The projection-profile code shipped, looked reasonable, and quietly never located a single card edge in its life.

Band boundaries were computed as fractions of the buffer, w * 0.86 and friends, then handed straight into a loop:

for (let i = Math.max(1, from); i < Math.min(profile.length - 1, to); i++)

from was 165.12. So i took the values 165.12, 166.12, 167.12, and every single profile[i] came back undefined. undefined > best is always false, so the search returned “no peak found” for the right and bottom edge of every card, forever.

The fallback was good enough to hide it completely. Uploads had been matching at 96.7% purely on the guide rectangle. One Math.floor took that to 98.2%.

Lesson: a well-designed fallback will happily conceal the failure of the thing it’s backing up. Assert that the primary path actually ran.

6. Presence is not the same as certainty

Once live scanning worked, a single card held still in front of the camera got added four times.

Re-arm was written as “the card has left when frames score low or edge-snap fails”. But edge-snap failing means the crop is uncertain, not that the card is gone. On a feed where snapping was marginal, that clause fired constantly, the scanner re-armed while the card was still sitting right there, and added it again.

Presence gets judged on score alone now.

Foils, and getting a second opinion

The descriptor is immune to a·x + b applied to the whole card. Holographic foil does not do that. It lays a bright sheen across part of the card, so different regions get different gain and some tiles clip to pure white. No global offset undoes a local effect.

Simulating it, a specular band with a Gaussian profile that clips where it peaks, reproduced the problem exactly:

scenario	top-1
foil, soft sheen	99.5%
foil, bright	12.5%
foil, offset band	3.0%

Textbook fix is local contrast normalisation: judge each tile against its neighbours instead of against the card as a whole. It fixed foils and wrecked everything else. A 4%-shifted card dropped from 93% to 76%, and correct answers started scoring too low to even report. I swept the strength across four settings and every one was a bad trade, because local normalisation throws away the low-frequency detail that makes clean cards easy in the first place.

So the matcher now keeps two descriptors and tries them in order. Full-signal one runs first. Only if that fails to produce a usable answer does the illumination-robust one get a go.

scenario	before	after
foil, bright	12.5%	99.6%
foil, offset band	3.0%	99.4%
foil + glare spot	9.0%	96.8%
4% shift	93%	94.6%

Total cost: 0.3ms, because the fallback only fires on frames the first pass couldn’t handle anyway.

One trap this created: the two descriptors produce different score ranges, so a single acceptance threshold would have thrown away correct foil answers. Results now carry a flag for which descriptor answered, and a shared helper applies the right threshold. Temporal voting compares margin past threshold rather than raw score, since raw scores across the two spaces aren’t comparable.

There’s a hard limit though. Under a near-total whiteout, a sheen strong enough to clip most of the artwork, accuracy stays near zero and no descriptor fixes that. Where the highlight clips to 255 the information is physically gone. The answer there is to tilt the card, which the live loop handles fine because it’s re-matching ten times a second anyway.

Testing a camera without a camera

Chrome accepts --use-file-for-fake-video-capture=<file.y4m>, which drives getUserMedia off a video file. Y4M is a trivially simple uncompressed container, a text header then raw I420 frames, so you can synthesise one from a card image in about thirty lines. Including a simulated foil sheen.

That gets you fully deterministic end-to-end tests of a live camera pipeline, in CI, with zero hardware. The tests assert things like:

• the right card gets identified and written to localStorage
• a card left in frame for four more seconds is not added twice
• a reprint pauses the loop and offers three thumbnails
• with a set hint, that same reprint resolves silently
• the audio cue fires with the expected oscillator frequencies

The accuracy harness is separate, runs in Node against the real index, and replays every card through a matrix of degradations: brightness, gamma, hue, blur, JPEG, resolution, scale error, translation, rotation, 180° flips, foil sheens. It imports the same match() module the browser worker uses, so there’s no second implementation quietly drifting out of sync.

Four gates have to pass before anything ships, and the one that matters most is “confidently wrong ≤ 1%”. Being slow is recoverable. Silently writing the wrong card into someone’s collection is not.

What I chose not to build

Automatic foil detection. Lorcana foils and normals share the exact same artwork. The API even serves one image for both. They’re genuinely indistinguishable to any descriptor. The scanner does flag cards that needed the illumination-robust path, since a sheen usually means a foil, and offers a one-key correction. But it never sets the variant itself. A wrong variant is invisible until you import the CSV, which makes it exactly the wrong thing to guess at.

Collector-number OCR. Would eliminate reprint ambiguity entirely by reading the printed “23/204”. It’s the highest-value thing still undone, and the matcher already returns a ranked candidate list specifically so it can slot in as a verifier later without touching the descriptor, the index format, or the worker protocol.

The numbers

cards indexed	3,188 (22 sets)
index size	898 KB
bytes per card	288
match time	7.5ms
live frame rate	10fps
main-thread cost per frame	5.9ms
top-1, photometric abuse	100%
top-1, ±8% crop error	98.6%
top-1, foil with bright sheen	99.6%
confidently wrong, worst case	0.4%
runtime dependencies added	0

Accuracy figures are top-1 over 500 randomly sampled cards per scenario, each matched against the full 3,188-card index.

Doing the UX last

I normally start with the interface. This time I started with “can it even recognise a card”, and everything about using the thing waited until the matcher worked. That order is fine while you’re the only user. It stops being fine the moment someone else wants to sort their own shoebox.

The fix came out of one observation: a scanner that says nothing makes you watch the screen, and if you’re watching the screen you may as well have typed the number in. Which was the entire thing I was trying to get away from.

So every accepted card announces itself now. A toast names the card as it lands in the list. A short beep goes with it, and the beep is the part that matters, because it tells you a card landed without you looking at all. Eyes stay on the stack, and you only glance up when you hear something, to check the right card went in.

The list got editable for the same reason. Confidently wrong is 0.4% in the worst case, which is small and isn’t zero, and the CSV is the thing you actually keep. One click deletes a row.

Foil rows get flagged rather than decided. If the illumination-robust descriptor was the one that answered, a sheen was probably involved, so the row is marked as a likely foil and a single key confirms it. The scanner still doesn’t set the variant itself, for the reason above: a wrong variant hides until you open the CSV.

And the shimmer on those rows is decoration. It costs nothing and it makes three hundred lines of collection nicer to look at, which is a good enough reason.

None of this touched the matcher. All of it decided whether anyone else could use it.