An Ethereal Window

Why a window works

A photo of a room looks the same no matter where you stand. A window doesn’t: lean left and you see more of the right side of the room, stand up and you see more of the floor. That shift, motion parallax, is one of the strongest cues your brain uses to judge depth, and it doesn’t need two eyes or a headset. It only needs the picture to change as you move, exactly the way the real world would.

Today every laptop has a camera, and a browser can find a face in that camera’s video many times a second. That is everything Johnny Lee’s Wii Remote did, already sitting on top of the screen. Ethereal is the experiment of putting those pieces together: find your eyes, work out where they are in front of the screen, and redraw the scene from exactly that spot, every frame.

How it works

Finding your eyes. Google’s MediaPipe face tracker runs right in the browser and maps hundreds of points on your face. Ethereal uses a handful around the eyes. How far apart your eyes appear tells it how far away you are sitting, since most adults’ pupils are about 63mm apart.

Holding still. Raw tracking shakes, and when the whole world is drawn from your eye position, the whole world shimmers. A One Euro filter smooths the position heavily when you are still and barely at all when you move, so the room is calm when you sit and responsive when you lean.

Bending the camera. This is the trick that makes it a window. Simply sliding a 3D camera around as you move looks wrong; the scene seems to swivel. A window always shows the room through the same fixed frame, so instead the camera’s view is rebuilt every frame to pass through the four physical edges of the screen from wherever your eye happens to be. When you move off to one side, the view skews:

The math for this “off-axis projection” fits in a few lines:

const ratio = near / eye.z;
const left = (-halfWidth - eye.x) * ratio;
const right = (halfWidth - eye.x) * ratio;
const bottom = (-halfHeight - eye.y) * ratio;
const top = (halfHeight - eye.y) * ratio;

camera.projectionMatrix.makePerspective(left, right, top, bottom, near, far);

With that in place, anything drawn at the surface of the screen stays pinned to the glass, anything behind it recedes into a room, and anything in front of it floats out toward you. The first demo is a tribute to Johnny Lee’s: a wireframe room the size of your screen, with bullseye targets hanging at different depths.

Beyond 3D scenes

The same tracked position can drive plain HTML too. CSS 3D transforms already have the idea of a viewer’s distance and viewpoint built in, so ordinary buttons, cards, and text can sit at real depths with no 3D engine at all.

That led to the demo I find most exciting: spatial navigation. Every card you click opens a new page in front of you and pushes the previous one back into the distance, fainter and blurrier with each step. With head tracking on, your history is no longer a back button; it is a place behind the current page that you can lean over and look into. It is a small glimpse of interfaces that use depth to mean something, rather than as decoration.

Both modes are packaged as a library, so the window isn’t limited to my demos. It installs straight from GitHub, along with the face tracking model it runs on:

npm install github:jasondecamp/desktop-vr @mediapipe/tasks-vision

Built entirely by Claude Code

Ethereal was two experiments at once. Besides head tracking, it was one of my first attempts at building a genuinely complex project entirely with Claude Code. I never touched the code by hand. Over seven days and 42 commits, Claude Code wrote the face tracking, the projection math, the filters, the calibration tools, the library packaging, and every demo, while I described what I wanted and judged the results.

That judging turned out to be the interesting part. Claude can read the code and reason about the math, but it can’t sit in my chair and move its head. Whether the room felt solid or swimmy, whether a layer floated or just looked flat, was something only I could see. So much of the work was a conversation about feel. In one afternoon the dock icons in a parallax desktop experiment went from 160 pixels in front of the screen to 10, then 5, then back to 10, one commit at a time, until they sat just right.

The rest was turning feel into numbers we could both check. The calibration tools came from that need: a crosshair you line up to set your natural sitting position, a ruler you drag to match a real inch so the room is the right size, and a diagnostics screen with a crosshair drawn exactly at the surface of the glass. If everything is right, that crosshair doesn’t move at all as you move your head, and a drift meter shows how far it wanders when it isn’t.

What it accomplished

It works. Sitting in front of a laptop, the room really does seem to sit behind the screen and the targets really do seem to hang in space, and the first instinct is to lean all the way to one side to look around the corner. It takes nothing but the camera you already have and a web page.

It also made the limits clear. It works for one viewer at a time, and anyone watching over your shoulder sees a scene that warps for no reason. Both eyes see the same image, so it is convincing when you move and flatter when you hold still. And asking a visitor for camera access is a real hurdle.

The biggest gap is harder to name. The effect is genuinely impressive, but it hasn’t crossed the line from cool to real. The calibration and the sense of scale are close, and close turns out to be its own kind of problem: the room is the right size, almost; it holds still as you move, almost. It lands in an uncanny valley of virtual reality, real-ish but just far enough off that you always know you are looking at a simulation. Getting across that valley, so the window stops being a trick and simply becomes a window, would take a lot more work on calibration and scale.

As a proof of concept, it answered the question I started with: a browser and a webcam are enough to make a window. Along the way it showed me what building with an AI agent could feel like when the hardest part of the work is something only the person in the chair can judge. Next I would take it toward interfaces where depth means something, starting with that spatial navigation idea.