We are at an intersection point in LLM adoption, where we are throwing everything at the wall and seeing what sticks, and yet the high cost of inference and always-online requirement are keeping AI’s most powerful results behind expensive and normie-unfriendly tools like claude code. These tools require genuine knowledge of the medium in which they operate to use the effectively. Like hiring a staff of developers without being able to properly define their tasks.

There is a point of friction and frustration which I assume we’ve all felt in picking up a new piece of software and thinking “damnit I wish the button for that was here”. Like every time I try a different Android phone and the geniuses at the company du jour deciced to bury the setting for the default launcher down an unimaginably contrived submenu. I bet it made sense to someone.

I think there is a new paradigm shift waiting to happen here, where user interfaces morph to user desires. The reason I think this is that I already do it.

I have been working on tooling to boost my capabilities in photography. As I built these tools alongside the growth of LLMs, I quickly adopted a new methodology. I’d use my tool with a tmux session open next to it. Every time I was frustrated, I got my agent to re-write that function to fit my flow. Sometimes it wasn’t a matter of right or wrong, but of how I felt about that task that day.

I made it create 3d games to fly through my photo’s embeddings, a tiktok style brainrot UI showing me videos made from my pics, a tinder style swiping game to tournament out my best photos. All of these were just thoughts I had whilst arduously star ratings shots from a photoshoot.

That same day, I got some feedback about the Pixel phone I insisted my friend gets to replace their aging iPhone. My friend was complaining that the swipe gestures on the homescreen just wouldn’t act like she expected on her iPhone. That moment, my first instinct was to hop into my AI coding tool and give it the feedback, until I realized we can’t do that on a live phone environment… yet.

Now’s when we imagine the future.

I am on my homescreen, and I am pissed off that swiping up doesn’t immediately bring up a beautiful search UI, maybe with my mic already listening for voice input and the keyboard already up to lightning type “minecraft” before I am too far into my dump for the loading time to be worth it. I press and hold the power key and Gemini springs to life, and in an instant of hearing out my complaint, she springs to action spinning up a local dev agent to reconfig my gestures and make my phone just right.

💡 On the tech side, this doesn’t have to be full on Kotlin compiles. If we redo our UIs in a DSL (domain specific language) which deals in modules, components, and gesture handlers at a high level (similar to the philosophy Android Wear took with WFF the watchface format), we can reduce risk of dev agents playing with our live phone uis and constrain the options just enough to be both helpful and stable.

This brought me back to 2015. I was using an original Moto G (shoutout Motorola for that banger), and I absolutely adored running paranoid android with pie gesture navigation. I lauded my 8.4% greater screen real estate and quick gesture for the flashlight over everyone. In fact, the way I used gestures on that Moto G looks a lot like how I flick around in my Android or iOS UI today. Though I realize, that joy was mine only through relentless troubleshooting and hacking to get the damn thing to run, which any sane person would not put up with. This is where the magic moment happens.

I felt the magic on my moto g in 2013, making the UI feel just right

I felt another kind of magic with my coding agents, turning my most boring photography workflow into a real adventure

And when I used Gemini on the Pixel replacing Google Assistant, that was a step change which unlocked the bridge between the two.

I hope that in 5 years, my mother can talk to her phone and make it hers, just like I talked to the kind folks on XDADevelopers when I was a kid to make it happen on mine.

💡 Who’s to say I, or you have good taste? We may or may not. The beauty of Android and Linux has always been that we present options. For the first time, that beauty of customization has a chance to be for everybody. It’s a fool’s errand to think one size will fit all, now we have a chance to make those pants from Back to the Future which resize to fit the wearer.