Six Degrees of Syncopation
I made a Spotify remote that tweaks song recommendations across six axes, based on Spotify’s audio features: danceability, energy, acousticness, duration, valence and popularity. It uses the Spacetec Spaceball 2003, an incredible six-degrees-of-freedom CAD mouse from circa 1991.
The Spaceball’s freedom of motion allows each axis of motion to tweak your music recommendations, while the eight buttons offer recommendations pulled from your most frequently-played tracks. (A ninth button on the ball itself controls play/pause.) Here’s a demo — note how the chart on the screen depicts each audio feature changing (most notably whichever feature we’re optimizing for during each twist of the ball):
So what is the Spaceball? It looks like a regular trackball, but it’s not! The ball doesn’t roll freely, and it doesn’t just move the X/Y axes. Instead, you can fully control six degrees of freedom. What does that mean? Imagine you’re holding the ball in your hand: you can move it in any direction you’d be able to move a physical ball, and that movement will be measured. That’s translation and rotation across X, Y and Z:
It’s also just so gosh-darn cool looking. A friend texted me this photo about a month ago, asking if I’d ever tried it. I hadn’t, and neither had he:
“I’m the most prolific of trackballers, but even for me it’s too off the wall to risk.”
– My friend. (Challenge accepted.)
While my project is only sending a new song recommendation each time the ball moves to the extreme end of any given axis, the Spaceball has much finer-grained control. It was originally designed for CAD projects, and its successors are still sold.
There’s still a lot of love for the Spaceball 2003 and its cousins. The device connects via a 9-pin serial port, but others have done a ton of work keeping the flame alive. Vic Putz has been building drivers for these devices for literally decades, and most recently he built the Orbotron 9001, which uses an Adafruit QT Py, some level shifters and CircuitPython to convert the serial input to USB HID, allowing you to use the SpaceOrb and more recent Spaceball devices as regular gamepads. (Another fan, Evan Allen, pushed this further and developed proper support for the Spaceball 2003.) Adafruit’s ladyada posted a great video parsing the Spaceball’s serial input, too:
None of this quite worked right for me. The Orbotron code created a gamepad that worked on Gamepad Tester but which my Mac seemingly couldn’t recognize. None of the joystick-mapping apps seemed to help, until I found BetterTouchTool, which has a tool for parsing generic USB HID devices and scripting actions. But because this solution was in user-space, rather than a system driver, it just felt too laggy for me. So I embarked on trying to parse the serial data myself and create my own gamepad driver.
This was a journey. I had to try to make sense of the serial packets the Spaceball emitted, tweak the input data, and then figure out how to create USB HID descriptors in CircuitPython that would yield a natively-recognizable USB device on my Mac. CircuitPython and Adafruit make this reasonably easy but it’s still a trip. I used ChatGPT heavily to help me create descriptors like this, and to help me pack my axis and button data into exactly the right format to match:
import usb_hid
SPACEBALL_2003_DESCRIPTOR = bytes((
0x05, 0x01, # Usage Page (Generic Desktop Ctrls)
0x09, 0x08, # Usage (Multi-axis Controller)
0xA1, 0x01, # Collection (Application)
0xA1, 0x00, # Collection (Physical)
0x85, 0x01, # Report ID (1)
0x16, 0xA2, 0xFE, # Logical Minimum (-350)
0x26, 0x5E, 0x01, # Logical Maximum (350)
0x36, 0x88, 0xFA, # Physical Minimum (-1400)
0x46, 0x78, 0x05, # Physical Maximum (1400)
0x55, 0x0C, # Unit Exponent (-4)
0x65, 0x11, # Unit (System: SI Linear, Length: Centimeter)
0x09, 0x30, # Usage (X)
0x09, 0x31, # Usage (Y)
0x09, 0x32, # Usage (Z)
0x75, 0x10, # Report Size (16)
0x95, 0x03, # Report Count (3)
0x81, 0x06, # Input (Data,Var,Rel,No Wrap,Linear,Preferred State,No Null Position)
0xC0, # End Collection
0xA1, 0x00, # Collection (Physical)
0x85, 0x02, # Report ID (2)
0x09, 0x33, # Usage (Rx)
0x09, 0x34, # Usage (Ry)
0x09, 0x35, # Usage (Rz)
0x75, 0x10, # Report Size (16)
0x95, 0x03, # Report Count (3)
0x81, 0x06, # Input (Data,Var,Rel,No Wrap,Linear,Preferred State,No Null Position)
0xC0, # End Collection
0xA1, 0x02, # Collection (Logical)
0x85, 0x03, # Report ID (3)
0x05, 0x09, # Usage Page (Button)
0x19, 0x01, # Usage Minimum (Button 1)
0x29, 0x08, # Usage Maximum (Button 8)
0x15, 0x00, # Logical Minimum (0)
0x25, 0x01, # Logical Maximum (1)
0x75, 0x01, # Report Size (1)
0x95, 0x08, # Report Count (8)
0x81, 0x02, # Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0xC0, # End Collection
0xC0, # End Collection
))
usb_hid.enable((usb_hid.Device.CONSUMER_CONTROL,
usb_hid.Device.MOUSE,
# usb_hid.Device.KEYBOARD,
usb_hid.Device.GAMEPAD,
usb_hid.Device.VENDOR_DEFINED_DEVICE),
report_descriptor=SPACEBALL_2003_DESCRIPTOR)Eventually, I created several working drivers. I decided to try to mimic existing gamepads in hopes that’d trick my Mac into natively recognizing them, so I used mac-hid-dump to parse the drivers of various devices I had on hand like my PS4 and 8Bitdo SN30 controllers. I even got my hands on a new Spacemouse Pro to see how it reports its USB HID data! But 3dconnexion’s Mac support is so flaky that even my fresh-out-of-the-box Spacemouse Pro didn’t work properly on macOS Sonoma, not to mention my sneaky hacky Spaceball 2003. I also made a variant that acts as a generic gamepad, and one that ignores most of the axes and acts like a regular 2D mouse. That one’s my favorite. All of these can be found on github, here, and mostly they work. The Mac does best with the 2D mouse and 8Bitdo versions! (Most of them have uf2 files that represent custom builds of CircuitPython. This is because I was experimenting with mimicking how the USB device reported itself, borrowing the product, manufacturer, version, PID and VID strings from the devices I was trying to emulate.)
After a couple of weeks of messing with all of this, I realized that what I really wanted was a more compelling way to use six axes of data. I didn’t need to emulate a trackball, and I don’t use CAD tools (or PC action games) often enough to actually need precise 6DOF controls. I toyed with other ideas like controlling my smarthome (twist one way for lights? pull up to pull the shades up in the bedroom? Nudge forward for the weather?). But as I was playing with the device, music controls seemed so natural. Twisting a knob to tune a radio or adust volume, pressing buttons to shift through tracks on a CD player… it all felt right. So I got to thinking about messing with Spotify data, and I started exploring ways to get interesting recommendations from my music via the Spotify Web API, and that got me here.
Six Degrees of Syncopation — which, sorry audiophiles, really isn’t about syncopation — is a little Flask app that does a few simple things. It shows the user’s currently-playing Spotify track, including a visualizer of that track’s audio features, based on Lily Zhang and Victoria Cabales’s work. (I’ve added a fun dancing WebGL cube that reacts to live audio input, piped in through a Loopback device to get around the way the Mac’s built-in mic filters system audio.) On the backend, the Python script listens for and parses data coming in from the Spacemouse. (I’m using the Orbotron for this, but since this use case doesn’t require USB HID I probably should’ve just reverted back to a normal DB9 serial <> USB-C adapter and cut out the middleman.) It maps the six axes to Spotify’s audio features (danceability, energy, acousticness, duration_ms, valence, popularity), and sends a trigger for a new song recommendation when any axis reaches its maximum point. The buttons cycle through recommendations based on the user’s top-played Spotify tracks: first we sort by date [mostly because all my recs would be yacht rock otherwise, lol], then filter for tracks by distinct artists, then use those tracks as the seed for the recommendation. This is a hacky way for me to approximate genre and year-of-release support, since neither can be directly used as inputs for track recommendations by Spotify’s Web API. (Trust me, I tried every trick in the book here, including reading many fascinating papers that attempted to predict things like genre or decade-of-release based on Spotify’s audio features. I piped that data into ChatGPT and tried to get it to combine audio features as proxy metrics, but wasn’t satisfied with what I got back in return. I figured that decibels alone would be enough to approximate year of release, given the loudness wars! Alas.)
It’s ridiculously fun to play with. I want to do so many different things with this device now! I suspect you’ll see it playing a role in other upcoming projects where I need interesting ways to capture user input. Spotify recs have of course gotten me thinking about TV/movie recs. Twist right for more drama? Tilt up for more comedy? Nudge left for older movies?
If you happen to find a Spacemouse on eBay, grab it! You can flash my code to an Orbotron and very quickly get a functioning 2D mouse or gamepad going. Worth it!