Track o’ Lantern
Happy Halloween! I turned an old CRT into a fun stoop accoutrement to amuse our trick-or-treaters. It features a 3D animation of a jack o’ lantern that follows your head motions and facial expressions, much like an animoji. It also plays spooky Halloween classics via a little speaker hidden inside.
Track o’ Lantern lives inside a Javelin VM-9A monitor, circa 1976, a clone of the Sanyo VM-4209 that was often paired with Apple I and early Apple IIs (and is in the collection of the Victoria and Albert Museum!). My father got it from his brother-in-law in the early eighties and, I recently learned, never used it. Lol. Anyway I found it in my parents’ basement this spring and knew I needed to do something with it.
Inside the monitor’s enclosure lives the Raspberry Pi 5 that powers the whole thing, as well as a Waveshare USB audio card, a small 8ohm/3watt speaker, and an 18v>5v buck converter. (The Javelin CRT features an intriguing optional DC input and the AC mains gets routed through a big transformer in the case, which was much easier to hijack. These monitors were originally designed for use in boats and big rigs and other closed-circuit applications, long before anyone had even a whiff of the personal computer revolution.) An infrared-sensitive Pi camera sticks up through the back of the case.
I knew I wanted to package it all inside the monitor for greater theatrical effect, so the Pi is soldered to DC power as well as composite video directly via the display’s motherboard, avoiding the need for extra wires and mounting hardware.
Track o’ Lantern uses Mediapipe for facial landmark detection (you can play with the very similar demo here) and three.js to animate a 3D pumpkin model based on those detected movements. I thought it’d be dead-simple to find a 3D model similar to the raccoon Google uses in their Mediapipe demo, but since when is anything simple? You need a 3D model that’s been rigged to match the facial landmarks that Mediapipe identifies, as seen in this ARkit demo page:
Fortunately I didn’t have to figure out how to create keyframes for each of the supported facial movements. After searching high and low I was able to find a wonderful pre-rigged pumpkin model on ko-fi, designed by Maakurika. All I had to do was adjust the origin in Blender and it was a magical drop-in replacement for the demo’s raccoon head!
The real trouble came with getting everything working smoothly on the Pi. I figured I’d be able to adapt the Python demo, which by default provides a face mesh but no avatar animating:
In the end, I simply adapted the Javascript demo rather than reinvent the three.js wheel. This mostly worked out-of-the-box, except the Pi 5 had some trouble with WebGL, which seemed to be about Firefox and Chromium on the Pi not supporting requestVideoFrameCallback. Claude helped me quickly resolve that! After that, it was just a matter of getting the browsers to recognize the Pi camera, which is no small feat. (So maddening!) I’m still having trouble getting it to reliably work on boot with a systemd, but for my use case it’s totally fine to VNC in and manually launch Firefox in kiosk mode via the GUI’s terminal app.
Adding audio support was a last-minute decision. Fortunately the CRT’s case had enough room for me to just pop a USB audio card into the thing, rather than have to dissemble it again to install an audio DAC hat! I discovered Raspotify which was joyously simple to set up, and now I can deejay any Spotify track or playlist to my beautiful little pumpkin tv.
And that’s about it! I wish I had photos of trick-or-treaters and revelers enjoying Track o’ Lantern, but I wanted to get this post up before Halloween in case anyone wanted to use the web version for anything. Fortunately my most important beta tester seemed to enjoy it!
(Meanwhile, if you’re in Brooklyn, swing by. I have way more candy than I’m going to be able to get rid of on Thursday night and to make matters worse I’ve already broken into the stash. Rookie mistake.)
