My background cutter kept eating the furniture

My whole pitch for Overserved is “the lens changes everything.” Every market plays different — Silicon Valley is not the cosmic frontier, China is not Africa. Then you looked at the board and saw the same four grey blocks scattered on every map of every country: a glass-and-steel Valley dressed in the exact same rubble as deep space, with long stretches of empty void between them.

A pitch that dies the moment you open the game isn’t a pitch. So I gave every world its own scenery — and that decision is the actual point here, before the cutter even enters. Decor isn’t garnish. It’s the cheapest, highest-leverage way to make a market read as a place. A board the player learns to recognize at a glance — neon and lanterns means China, acacias and solar towers means Africa — does the worldbuilding that the rules can only imply. Same mechanics, five identities, for the price of art.

80 pieces of furniture

The plan stayed cheap on purpose. I already had four size tiers, small prop up to giant landmark, that control a prop’s footprint and how often it spawns. I kept those and gave each tier four variants per country: four tiers times four variants times five worlds, eighty props. A hash picks which variant lands on each tile, so a board shows a varied skyline instead of one model stamped a hundred times. Then I roughly doubled the scatter rate and re-baked all hundred campaign maps so the boards actually felt lived-in.

That was the easy part. I had eighty pieces of art on flat colored backgrounds and now I had to cut them out. That’s where it turned into a fight.

The bench that exploded

I generate each prop on a solid color, then run a remover to punch out the background. Standard. Except mine kept committing crimes.

First USA pass: six of sixteen props came out so dark they vanished into the dark board — my fault, I’d literally prompted “dark futuristic palette.” Switched to light bodies with glowing accents. Then the domed arena rendered its glow rings green, the same green I was keying out, so the cutter ate the glow with the background and left a ragged halo. Annoying, but both of those were me feeding the model bad instructions.

The real disaster showed up when I stopped squinting at thumbnails and inspected props at full resolution: a teal bench shattered into confetti, glowing landmarks wore halos, a topiary sat serenely on its colored background like nobody had asked it to leave, a temple had a roof floating a clean inch above its walls. The raw renders were clean. The cutter was doing every bit of this — and unlike the dark-palette goof, I hadn’t done anything wrong in the prompt. The method itself was broken.

Why it was eating teal

Here’s the part worth keeping. The old cutter keyed by hue — a cheap green-screen test, roughly “how much greener is this pixel than it is red or blue.” That’s fast and it’s how a lot of homemade background removers work, because comparing one channel against the others is a couple of subtractions per pixel. The catch is that hue is a terrible proxy for foreground or background. Teal is heavy in both green and blue; to a “how green” test it reads as green-ish, indistinguishable from the green screen behind it. So the cutter looked at my lovely teal bench, decided it was basically backdrop, and machine-gunned it. The temple’s roof keyed out anywhere its color happened to drift hue-ward into the key; the topiary’s green simply was the key.

The fix I reached for was to stop asking “what hue is this pixel” and start asking “how far is this pixel’s color from the actual background.” I sampled the real background from the corners and measured straight-line distance to it across all three channels. Teal sits miles from pure green out along the blue axis, so it clears the threshold and survives; only colors genuinely near the backdrop get removed. Re-cut all eighty from the clean raws, and the bench came back whole.

The tree I painted magenta

One more, because it’s too good not to confess. The acacia’s prompt said “on a magenta background.” It never said the tree was green. So the model, taking me at my word, gave me a fully magenta tree on a dark background — and the magenta cut dutifully mangled it into a navy smear. Fix: “bright green canopy, brown trunk.” Name the subject’s color whenever you name the background’s. The model won’t assume a tree is green just because every tree you’ve ever seen is; an LLM fills silence with the nearest token, and the nearest token to “magenta background, tree” is a magenta tree.

What stuck

• Decor is worldbuilding, not garnish. Per-country scenery makes each market read as its own place for the price of art — the highest-leverage identity work in the game.
• Color is the wrong question for background removal. Hue, distance, any single-color rule keys on what a pixel is, when what you actually need to know is whether it’s subject or backdrop — and those aren’t the same thing. A teal bench proves it.
• Don’t dress the subject in the color you’re keying out, and never name a background color without naming the subject’s — both are the same failure: a collision between what you want kept and what you want gone.
• Inspect at full resolution. The shattered bench looked perfect at thumbnail size; the pixels that betray a cut are smaller than a contact sheet.

The feature shipped: eighty themed props, denser boards, every market visibly its own world. And the clever color-distance cutter I’m so proud of up there? I retired it a day later.

Because “color distance” is still answering the wrong question. It’s a better proxy for subject-vs-background than hue, but it’s still a proxy — give it a prop whose paint happens to sit near the backdrop and it’ll fail the same way, just less often. The actual answer is to stop reasoning about color at all and segment by subject: an AI matting model that looks at the image, decides what the object is, and lifts it off whatever’s behind it. I render on neutral grey now and let the segmenter do it, no thresholds to tune. It has its own quirks — it leaves see-through interior gaps, so solid props cut cleaner than slatted ones — but it never mistakes teal for absence, because it was never looking at the color.

That’s how it always goes. You win the battle with a clever threshold, then realize the whole category of fix was the problem, and the war moves somewhere better.