Thinking about an LLM
About This Guide
This is an introductory guide that helps you reframe how to think about an LLM to most effectively use it.
Full Disclosure
I wrote this myself, but I used Claude (Opus) to give me suggestions for content that I didn't consider, and had it point out mistakes and inconsistencies. It never touched the actual manuscript, and I never copy/pasted anything it generated.
All the images were created using some combination of AI diffusion models, 3D, 2D compositing and editing software, and whatever else I needed to make it work.
This guide is also available in 📄 PDF format here
Intro
For most of our lives, we've interacted with computers and software - and really most technology - in the same way: We get a new system, read or watch a primer on how it works (or don't), start using it, run into issues, seek external help, and then slowly get better at knowing which buttons to press in what order to get what we were expecting. If a system was unreliable or gave unexpected results, we called it "buggy", "flaky", or something more, uh, colorful.
Generative AI requires a different mindset to work with because it was BUILT to be variable and unexpected, but for very good reason.
The #1 thing to understand about LLMs is that they do not KNOW anything. They work by storing little bits of language in a big network and then traveling through the connections between them to arrive at a response based on the input given.
The #2 thing to understand is that they don't inherently know that they don't know something (because, again, they don't know anything). Sometimes they carry internal signals of their own uncertainty, but they're under no obligation to tell you this.
The #3 thing is that they're trained to sound like they do know things. An LLM's initial training comes from millions of works written by people. People who write about things sound confident (often well-founded since most of them do their research before presenting findings). Users also like and trust confidence, so in further levels of training that involve people ranking answers, confident answers were often favored over true ones. Ergo, vis-a-vis, i.e. and e.g., LLMs often sound confident, even when they're flat-out wrong.
So why would we knowingly pay good money for an unreliable system?
Turns out, they're great for helping us make connections that we wouldn't have reached on our own. Asking for facts might be dicey, but asking for inspiration is something other systems simply can't do. LLMs can also write code reasonably well, help us refine our art style by driving diffusion models, write their own deterministic tools if consistency really is needed, and a ton of other useful things.
It's not that they're unreliable all the time, it's more that we really, honestly, and seriously do need to double-check factual claims.
There are also lots of things we can do to the framework that surrounds the LLM to influence the responses it gives. The model can be told to double-check instead of relying on its training. It can also be pointed at a set of known good data to work from. The core technology might have a bit of variability, but that's actually a good thing when used right.
Two Examples
Let's just look at two scenarios that compare and contrast a database, a search engine, a human, and an LLM.
Inspiration
If you ask a massive movie database to give you a list of everything Tilda Swinton has an acting credit in, it will do that accurately, and every time without fail. If you ask for a handful of movies you could watch to inspire you to be more creative, it'll look at you like you're on something ("no entries found" is machine speak for that).
Asked the same question, an average human will rattle off a handful of movies, and probably get some of them wrong. Your movie buff friend Sasha might be able to recall 30 or 40, but not get to IMDb level where she knows the exact number of acting credits.
Unlike the database, though, your friend is more than capable of making suggestions for your inspiration question. She also knows you, so she knows what you've been into lately, which makes her suggestions even more relevant. This type of interaction is often much more useful and fulfilling than knowing a simple fact.
The exciting part about LLMs is that they can also do something similar to your friend. They'll go through their internal training or out to the web and find interesting and obscure connections between seemingly unrelated things that a database, spreadsheet, or any other digital tool we've used to date is incapable of providing. This is where the variability of the system really shines - asking it a few times may yield different or more interesting results.
Even better, an LLM can also ask you follow-up questions and carry on a conversation about other movies that may have triggered this response to get more context and even better results. You might get really dumb suggestions, but you might strike gold too. Just like a human :D
Practicality
If you ask a search engine whether a washing machine will fit in your Jeep, after all the sponsored results, it will direct you to the manufacturer's specs for the machine in question and your Jeep. Further down the page it will link to blogs, Reddit posts, videos, and forums where people recount tales of getting their washing machine to fit in their trucks. Maybe it's your machine, maybe it's your Jeep... probably not both. You'll get suggestions and ideas from all this, and then start to do some research. You'll end up doing your own calculations until you decide to either give it a go or ask a friend to borrow their pickup.
If you ask your uncle Dale this same question, he'll probably say "yep, it'll totally fit", and then when it doesn't he'll show up with a crowbar and some duct tape. He's (probably) not being a jerk, he really thought it would fit and didn't think it was worth researching, and didn't want to lose face by saying he didn't know. When challenged, he went into fix-it mode to either prove he was right or because he was feeling guilty about it.
If you ask an LLM, one of a few things will happen.
If it's a cheap-to-run free model, it'll likely say yes without bothering to check (kind of like Dale). When you say "It didn't fit, and now I'm stuck in the parking lot with this washer.", it'll say "I'm so sorry! I totally dropped the ball there!". If you then immediately ask "but will it fit?", it'll probably say yes again.
It's also not being a jerk, and it's not stupid. It's just how an LLM works. It runs the input through its network, finds the strongest connections between the tokens that make up the sentence "will this washing machine fit in my Jeep" based solely on its training, and picks a plausible response. It'll often weigh pleasing the user over fact-checking, and because it's fast and cheap, it's not great at double-checking and extended reasoning. Basically, this is the wrong model for this type of complex question, also kind of like Dale :)
If it's an advanced model with better training, more thinking capabilities, and access to tools and data sources, the experience will be a LOT better (and more costly in both time and money). It'll first do the same thing as the cheap model, but then it'll stop and check itself. It will look for any tools it has or any sites with better information and run through a bunch of checks and balances. Eventually it will say something like "the manufacturer's specs indicate that it should, but there may be other things preventing a good fit. Do you want me to keep looking?".
At this point, it'd probably get you about as far as a search engine and a few hours of research, but would speed up the process considerably by homing in on the issues to look at.
A really advanced model will do even deeper research, and may even hit on a real showstopper issue by itself because of the higher level of reasoning. It would also likely end up costing even more money and take longer to process, but possibly save a bit of time in the long run because it gives you a better starting point to work from.
We have to keep in mind that even the best LLM out there still doesn't know whether it will fit, it'll just have better odds of coming to the right conclusion than a lesser one.
It's still up to you to think about whether the information is feasible enough to give it a try.
Wrap Up
In all cases, working with an LLM requires a skill that we haven't had to flex really hard over the last several years: Critical thinking. This is going to be the best thing to fall back on when we're dealing with this new concept of a computer that works more like a brain than a calculator.
Think about whether something passes the sniff test, think about what an LLM is good for and what it's not, verify claims, and don't take anything personally. When used as intended, it's a HUGE boon to our daily lives and working processes. When we try to use it for something it's not good at, we're in for a world of frustration.
Strap in, things are going to get weird :)