Like I said in The “Hate” of My Love/Hate of AI post, I do actually like AI tech. The issue is just that there are snake oil salespeople who have managed to convince people (especially investors) that AGI is just around the corner, and as a result, everyone’s gone bonkers over it.
But it really does have its uses (and potential uses). I do genuinely think bringing the tools we now collectively call AI to more general availability can be useful in the right contexts. Automating the tedious task of fitting armor on disparate body types, so game artists can work on actually creative things, for example. It’s just that the actually useful tools aren’t going to be so flashy as all the hype-driven tools are right now. I wager most of them will be largely boring and invisible, embedded seamlessly into workflows.
The Right Tools For The Job
I think the big issue right now is that people are trying to shove LLMs into everything and trying to make it do everything, even when better systems exist. As a result, they’re chasing diminishing returns on a technology that, at least in its current state, is horribly inefficient at scale, trying to reach an end it simply cannot achieve.
But what if we use the tools for what they’re actually good at? Novel idea, I know.
In my opinion, LLMs shine brightest when used a) to augment humans, not replace them, and b) in scenarios where accuracy is either easily verifiable or irrelevant.
Two of my favorite uses so far are:
- A natural-language search engine that can find answers to more sophisticated questions and link to sources I can then verify against and use. This, to me, is a huge upgrade from keyword-based search engines for complex ideas, often shaving off minutes, or occasionally even hours of searching for just the right keyword combination on abstract or obscure topics.
- Brainstorming. Specifically, I’ve used it to help kickstart worldbuilding for creative writing projects, and help develop constructed languages. In these scenarios, “accuracy” isn’t really a thing, because I’m creating something out of whole cloth and I determine the rules. So, I’ve done things like feed it grammar rules I’ve already come up with as I’ve built a constructed language, and have it come up with other linguistic artifacts, then refine it as its responses uncover things that I had subconsciously decided, but hadn’t realized it (kind of like that adjective order thing in English). It really satisfies that part of my brain that loves the logical consistency that machines are pretty great at.
Small and Local
I genuinely think the future is local or semi-local (as in, local to a particular SaaS or application, if not the end user), smaller language models, customizable agents, and specialized tools. Small language models are proving to have comparable accuracy to the behemoths, using far, far fewer resources, allowing them to run locally. Models like Qwen, Cogito, Command-R, and Llama provide a balance of power and small footprint.
Their small size would also allow them to be paired with other tools, like natural language processing, dictation, and other machine learning programs on consumer hardware to create rich agents and devices, based on what the user needs. Siri on steroids, or build-your-own alternatives, tools that automatically adapt to differing needs over time and incorporate previously-learned things to make the connection to the new topics.
RAG Reigns Supreme
Vectara has a “Hallucination Leaderboard” that shows the models only having a 0.7-2.2% hallucination rate, but if you look closely, you’ll find that these are for summarization tasks using documents they fed into the models. This is the essence of Retrieval Augmented Generation, and it clearly shows this is a place where language models really shine (unsurprisingly, really, when you think about it).
This is a big reason, I think, why tools like Narrative or (certain parts of) GitHub Copilot work so comparatively well and are actually useful. Imagine having a local language model work over your second brain collection to surface notes you’ve saved in the past that are relevant to work you’re doing now? Or being able to ask the model to find the notes on, or related to, a particular topic? Maybe it could even save notes on things you revisit frequently or dive particularly deep into, like what Pieces strives to do and what the AI tools in Obsidian and Notion touch on.
Accessible Technology
Imagine a world where the barriers to accessibility aren’t so high, where auto-generated closed captions are more accurate, or voice chats are accurately transcribed in real time, without having to pay what would be a prohibitive cost for an individual or small organization to pay a human or hire a third party company to do, or which there are few humans who can do it. Imagine existing dictation tools that use language model technology to make their transcriptions more accurate, reducing the time needed to edit for simple things like punctuation and misheard words. Or a text to speech system that can read physical media, with the help of something akin to LeapFrog’s LeapReader pen, but for anything, as opposed to proprietary books with limited, dedicated audio files, and can do so with natural-sounding audio, instead of the flat, robotic audio that’s been historically available on most text-to-speech software.
These are the kinds of things where I see this tech (or tech like it) being not just actually useful, but downright beneficial. Technology should help humans, not replace them.
But this doesn’t just extend to accessibility tools. It could really extend to most (digital) tools. I have a great many qualms with how these tools got their corpus of training data and the general, blatant copyright violations because of it, but I do genuinely believe that anyone should be able to make their vision come to life, as long as people understand the limitations of these tools and don’t try to pass the results off as on par with their “artisanal” versions. Wonder Bread is not and will never be a handmade focaccia, plain and simple, but that doesn’t mean it doesn’t make a perfectly good peanut butter and jelly sandwich.
Orchestrated Modules
Language models are but one tool in the AI and machine learning arsenal. RAG systems use rerankers to improve their results, while MCP servers give them access to the Internet and interoperability with other tools.
Think Home Assistant, but for other things (or hell, even in Home Assistant, collecting data and recommending schedules or making note of things that use excess energy, etc using machine learning and predictive analysis). Ideally, these would all come bundled into one application, perhaps with some customization (default models, with options to switch them out for more suitable ones, for example), or some type of “build your own suite” setup that bundles the different parts into one interface.
Human Augmentation, Not Replacement
Contrary to popular business belief, this generation of tools can’t actually replace humans. In fact, it technically doesn’t even augment them – despite being able to generate code faster, the overhead created at the stability and overall throughput levels means productivity across the entire SDLC decreases.
They also said tools like Dreamweaver and FrontPage would replace us, too. And offshoring would take all of our jobs. And robots before that. And that the cloud would replace infrastructure people.
Not only did it not happen with any of those, they all turned into net positives for us.
“Computer” used to be a job role, held by humans (mostly women) who spent their days solving complex math problems. Then “computers” the machine came along, and while the job title of “Computer” went away, the work itself didn’t. “Computers” quickly evolved into “Computer Programmers,” which then evolved from glorified secretarial work to the prestigious, high-paid work that we know it as today.
I’m still highly skeptical of this technology’s ability to be even remotely the panacea the marketing groups of the world seem to so desperately want it to be. Hell, maybe once this bubble bursts, it’ll just die out entirely, kind of like how the whole Metaverse thing did. Until then, I’ll keep playing around, seeing where I can fit it into my workflow and how much use I really get out of it, and dreaming about the ways in which it could actually improve the world.