Astral Planes is a playable album disguised as a cozy puzzle game. You move through musical terrain, carving into it to uncover ideas, chasing those ideas down into notes, and deciding which notes become part of the record. There isn't one correct solution. Each playthrough is another way the music could be.

I started the project wanting to figure out how my own music could become part of a game rather than simply sit underneath one. What followed was a long series of prototypes involving MIDI, procedural music, AI-assisted programming, questionable ideas, abandoned systems, and eventually something that began to feel like playing music.

Origin

I didn't realize that Astral Planes was going to bring so many pieces of my creative practice together. I had been working reimagining Moonlight Flight with a game design student, trying to figure out a proper hook. We went down a narrative path for a bit that would allow the player to explore a galaxy. I wanted to work with a gamepad, I was tired of touch controls. I knew if I wanted to go any further with making the music tie into game mechanics, MIDI was my best bet.

I found the Maestro TK plugin and set about prototyping a system where I could control the music through player actions. My use of Ableton Live threw a wrench into things at first. The MIDI export function is limited to single clips, so composing a multitrack MIDI sequence was going to require some work. To use what Maestro had available, my best bet was to ditch Ableton Live. Going the path of code would require a deep dive into MIDI.

ChatGPT had just started gaining traction for programming and I wanted to see what it was capable of. I fed it the core classes of Moonlight Flight and got it to read the MIDI clips and spawn notes.

AI Enchantment

I was immediately taken back by how quickly I could prototype playable code. The LLM could hold the essential mechanics of the Moonlight Flight system while also writing effective code that made experimenting with MIDI interaction. My programming skill level is good enough that I can rapidly prototype a simple game. While wearing the hat of a designer, I often constrained ideas. Without constraint, I would just be increasing the amount of programming I would have to do. Especially when it came to ideas that would require spending more time learning how to translate a concept into code.

Procedural Music Generation

When I went down the MIDI path, I hoped to land at a place where I could import the Moonlight Flight project into Unity and have each instrument could be a separate MIDI player loaded to a specific preset.

ChatGPT and Music Theory

Up until this point, I had been cutting and pasting code back and forth in the browser based chat interface. I would attach the appropriate class files for reference and ask ChatGPT to generate a new feature, which I would then copy and paste into the project.

I kept iterating in the same conversation, attaching new classes as needed. It was able to keep a fairly accurate representation of the entire game system in its context window. I started asking design questions about music theory in the same conversation. It was able to generate code with the math required to do things like transpose chords, play notes in a particular key, and generate arpeggios. These are all things that I understand at a fairly basic level. I can play chords and scales, but I can't read music. It allowed me to design a system based on my musical intuition.

Configurable Notesets

I decided to explore the possibilities of a system that generated notes based on a set of musical rules. I created constraints based on scale, chord patterns, rhythm style, and note behavior. The core mechanic was collecting what was spawned. Collected notes were added to the loop and different tools would allow you to change different things about the track.

Creative Velocity

Each of the videos above have a month or less between them. I was habitually doing design/code sprints. The ability to propose a feature and have a playable prototype in less than an hour was exhilirating. It was creative cocaine.

It's the effort you know it would have taken you. And potentially what things might not have been possible without enlisting someone else's help. You move forward so fast in terms of engineering that it seems like shipping it is just around the corner.

But it isn't.

Playtesting

I playtested the prototype with game design students and it was fairly well received as "cool" but they had no idea what was going on musically. I had a lot of explaining to do. I had been experimenting with visualizing the notes as a soundwave but it made no sense. The next version addressed this by placing notes on a timeline and drawing lines between what could be collected to where it would end up.

This was also the point where I changed the instructions in ChatGPT to be highly critical. I realized that I was chasing a different idea that was far from the original of incorporating my own music into a game. I focused less on procedural music generation and began work on importing MIDI clips. I also started to think about what the central challenge of gameplay would be. The next version explored the idea of a grid that you carved into.

Mechanics as Metaphors

I spent more time using LLMs to explore ideas focused on making the whole system more coherent. There was a digression into game mechanics as metaphors. Those metaphors were useful as a design language.

Metaphors to Concept

Those metaphors became useful as I worked on the legibility of the musical system underneath. I wondered if the game itself had a philosophy that I could turn into a concept. I landed at my philosophy of the creative process. The universe gives us the raw material that we explore/dig/carve into. The idea we uncover leads to a discovery and from that a new creation is made.

AI is Sloppy

When software developers complain about AI slop, it often comes from a mental model of production grade code. Corporations using AI to write code for platforms we use everyday terrifies me. The person tasked with fixing a bug affecting potentially millions of people? It's already a moment filled with existential dread, let alone fixing something that gets exponentially sloppier the faster you iterate. This isn't necessarily a problem for game development though. If the game works, it works. If you don't plan on updating the game, it doesn't really matter because it's your own dirty mess.

Open Source

When I started importing MIDI clips that I authored as sets in Ableton Live, the code "worked." If you play the game, you make music and it's in key. But was it following the correct timing? Not always. There were many functions written by AI that mixed up references, especially in regards to timing. I'm an open source advocate, and it would be really embarrassing to claim that certain parts of my game did something that someone else could point to and say "no, it doesn't." I had switched over to Claude Code, which now had access to the full project. It had a much better chance at cleaning the codebase.

Context Window Creep

One of the quirks of vibe coding is that you never know when the LLM is going to go stupid. I feel like I was riding the wave of Big Tech tech's perpetual beta products. There were moments when I felt that I might have fell into a hole I couldn't dig my way out of. It's a strange moment of despair. You mourn the time you have lost trying to fix what you broke instead of creating more potential problems. These are the moments that remind you that there are limits and you could be walking off a cliff.

Fortunately, I didn't fall off the cliff. I completed the Ableton Live -> Unity import process where I could author something and it would sound the same in both places.

Back to High School

As I worked through the game design, I found myself reflecting on my history of making music. There are certain rhythms that I find myself returning to. There's something in the iteration process that expresses how things could be different. There isn't perfect, just different ways of being.

A Familiar Pattern

With a dependable musical system built, I started creating levels based on musical terrain that I had musically played around with a lot. I wanted to see what it would feel like to play it as a game.

A Cozy Puzzle

As I played through musical terrain I was familiar with, I realized that I was hiding too much information. I had convinced myself (or maybe the LLM had?) that the primary mechanic of carving needed to have hidden information that was discovered. The musical system was already complex, I didn't need to make anything else more complicated to understand.

Instead of starting gray, the dust appeared showing its musical color. The player can decide what dust to harvest. Flowers bloom, which are sought after by the stars. When the stars consume enough, they produce an idea that the player chases down. When the player has fully consumed that idea, it reveals the notes as a musical phrase. The player collects the notes and commits some or all of them to a record in whatever sequence they decide.

Behind the scenes, each star is assigned to a musical role and has a set of midi clips to choose from. This is the musical phrase that is revealed to the player. Each instrument track can expand up to 4 interpretations of this clip. The bins follow a chord progression. This is where my experimentation with procedural music generation became extremely helpful. If I author a clip that has 4 C notes, the first bin will have 4 C notes. If I assign a chord progression of I-II-III-IV, the second bin is transposed to 4 D notes, the third bin is transposed to 4 E notes, and the final bin is tranposed as 4 F notes. When a clip has more than one note value, the system transposes each note into the correct key.

Many Possible Solutions

In the playthrough above, I authored 10 total MIDI clips (4 Bass, 3 Harmony, 2 Lead, 1 Groove, and 1 Rhythm). The number of rings that the player assembles is 8. If you count the ways to split this up among 5 tracks, it gives you a possible 320 distributions. When you factor in the combinatorics of the notes available to place in each midi clip, there are 5,547 possibilities.

The Feeling of Playing Music

There is a flow state that comes with learning to play an instrument. A person that learns to play a few chords on a guitar might learn a song that someone else can sing to. That shared act of creation gets increasingly more profound at higher skill levels, I can't begin to imagine the intensity of flow that might come from playing with a band of jazz with virtuosos.

This is the core idea I have been chasing with Astral Planes. When you strip away all of the required knowledge to make music and translate it into game mechanics - what does the player feel? When they play together, what does their shared experience feel like?