Step Zero
On your journey to understanding Language Models
As I sat down to record a YouTube series designed to help folks get started with “GPT” from the ground up, I was collating through notes, and running through my own memories of helping friends and family over the past few months. A couple of key themes jumped out.
So as a primer, I wanted to put out a quick newsletter this week tailored to readers out there that want to try, or are at least curious about this “Chat GPT” thing they’ve been hearing about, but are either nervous about interacting with a “advanced super intelligence” they keep hearing about, or simply don’t know how to get started.
If this is you, then read on and see if this article helps. If this is not you but you, too, field a lot of questions on how to get started, then consider sharing this post.
The challenge, as I see it, is how do we demystify these “large language models” so that we can easily separate everything that they are, from the things they are not. I think we can do that together.
Thanks for your support,
-DH
Mindset
Before we even open a browser window, let’s start with a basic, idea that may not be 100% exact on how AI works, but will give us an easy place to start, and we will build up from there.
Library 1.0
Imagine you walk into a library, or a book store like Barnes & Noble, because you have a topic in mind that you want to research.
Now, this place is a lot of books, and you could just go wandering about, finding the right section, then scanning through the shelves to find the right topic area, and then picking out several books to see if each one is what you’re looking for.
And certainly there’s nothing wrong with this approach - and in fact this is how most people approach it.
Let’s take the scenario so far and align it to searching Google. Google is the library, you are typing in some key phrases and clicking through various links, like wandering round in that library until you find what you’re looking for.
Library 2.0
Ok - same library. But this time, instead of wandering through the sections and shelves and poking about, you approach the librarian.
You let her know what topic you’re trying to research, and since she’s very knowledgeable, she directs you right over to the section of books that you need.
Maybe she even knows a little bit about the topic, and offers some additional tips or recommendations on authors or specific books. Cool, right? Let’s think about what we gained by interacting with the Librarian:
Less time spent lost or wandering, by having someone direct us right to the section we were looking for, especially if our topic of choice could fall under several possible categories.
Less chance for distraction along the way since many shelves on unrelated topics might catch your eye and distract you from your original goal.
Recommendations based on popularity or insights into the topic that might save us time and energy trying to compare a bunch of books on the same topic.
And as with all things, let’s be fair and list out a couple of things we traded for this time convenience:
Less chance of a happy coincidence while wandering. Sometimes, wandering about in the library or book store leads to unexpected gems, or triggering memories of other topics you wanted to look into.
Recommendation sometimes leads to missing out on less popular gems, or simply may not be aligned with what you were really looking for.
In the digital world, this might be like asking Siri or Alexa about a topic. Or finding a Reddit post, Youtube summary video, or Twitter/X thread covering your topic that’s making recommendations.
Like the Librarian, Siri might have a couple of quick answers and general knowledge, but ultimately, they best they can do is point you towards a more authoritative source and it’s up to you to continue the search from there.
Library 3.0
Ok, we’re almost there. Last scenario. This time, you walk into the library with a topic in mind. Maybe you have an essay due on the topic of Gravity.
You walk up to the information desk, but instead of your typical librarian, you get Einstein.
You ask him about your Gravity essay, and two things become apparent-
He could point you to books on Gravity, but it’s pretty clear that he knows a lot about the topic and has pretty much read everything in the entire science section.
Rather than speaking to you like a super brainiac scientist, he can somehow talk to you at a level you feel the most comfortable with.
After about 20 min of exchange, you snap back and realize that you’ve been monopolizing the poor guys' time and haven’t made it to any particular book in the library. But why bother? You have friggin’ Einstein sitting here!
You ask him why he’s wearing the headset, and he mentions that he’s essentially here to do just this; answer questions in an AMA or “ask me anything” capacity, and that in addition to scientific topics, he’s spent the last year reading pretty much every other book in the library, and can likely help answer questions on any other topics you might be curious about.
This, in the modern world is a Large Language Model, or LLM.
Unpacking AMA Einstein
As we did before, let’s break Library 3.0 down into a set of benefits, and a set of trade-offs.
On the upside:
You are likely able to get everything you need for your Gravity essay, at least conceptually, from Einstein rather than having to crack open a single book. After all, he’s already read them all.
You can likely explore concepts and ask Einstein to explain things you’re struggling to understand from the material, which is not something you can do with a fixed book or article.
Since Einstein is well read on a wide range of topics, you can even link together similar topics and thoughts conversationally with Einstein which otherwise may have required you to find other related books and do that comparison alone.
Finally, Einstein has infinite patience to explain concepts and simplify them for you for as long as it takes until you grasp them.
As amazing as this sounds, there are still tradeoffs:
Since Einstein has read so many books, and so many books on the same topic are similar, it may be difficult for him to pinpoint exactly where he read a particular fact. This is like seeing a familiar actor in a show, but struggling to place exactly where you’ve seen them last.
Because Einstein has been exposed to such a vast amount of information in his training, he may mistakingly be inferring information that isn’t actually found in a specific book. That is, he may confuse a new original thought for a memory, and it may sound convincing.
If your goal is to gain general knowledge or understanding, then all this might be fine, but if your research paper requires footnotes with references, you may still have to find specific books and papers to cite, since you can’t just write “AMA Einstein from the Library” as your Source.
Just because AMA Einstein has reed most of the books in the library, it doesn’t mean he’s read them all. Therefore you might be fooled into assuming that if Einstein gives you an answer, it must be “complete”.
These very same considerations apply to “Chat GPT”, except it’s a bit less magical than hanging out with Einstein.
Large Language Models are trained on a massive library, a good portion of the internet, and it’s in all of that training that GPT seems “worldly” and deeply insightful.
And with that, we’re going to pause here, and I’ll invite you to to ponder the Library 1.0-3.0 scenarios as a pre-cursor to our upcoming Video series.
In Step 1, we are starting from scratch and learn how to interact with Chat GPT and more importantly, how to ask and interact with it in natural ways. We’ll continue to demystify aspects of it as the lessons progress, and by the end of the journey, you’ll have a confident understanding of its capabilities, even if you decide it isn’t for you.
Thank you for reading, and have a great weekend!
👍👍
You also have a great weekend!