Small Language Model

 

   Large Language Models (LLM) are a type of Artificial Intelligence (AI) that has been trained to write sentences, paragraphs and essays. To be able to do such an amazing thing, LLMs study trillions of words in all kinds of books, magazines, encyclopedias,... everything it can get its hands on, to understand how language works.

   The value of AI is that I can ask it anything and it will give me an answer. It never says "I don't know." I wondered if LLMs had watched TV shows during its LLM training, so I asked it to summarize the famous "Soup Nazi" episode on the old Seinfeld show. AI gave me a short summary of the episode then referred me to Wikipedia for the complete plot. So the LLM had trained on Wikipedia but did not watch the show itself. The LLM probably didn't have time to waste on TV.

   The premise of Seinfeld is that it's about nothing, which is not really true. On the other hand the book Finnegans Wake by James Joyce is about everything, to the point that it becomes unreadable. I asked AI to summarize the first Book of Finnegans Wake, and in 82 words AI gave a succinct summary of the 214 pages of Book 1, then referred me to Wikipedia for more details.

   I asked AI why this book was even written if it's unreadable. AI said, "James Joyce wrote Finnegans Wake not to tell a traditional story, but to map the chaotic, multilingual, and cyclical logic of human dreams and the subconscious mind." AI found that quote on Online Literature Network. OK, that's fine. But if it’s unreadable, why waste time trying to read it. AI found a reader on Reddit who said, "...the pleasure comes not from a straightforward narrative, but from decoding the rich, musical wordplay and uncovering the hidden layers." Right.

   I asked AI about Hitler's antisemitic manifesto, Mein Kampf. Yes, the book is in the LLMs, but there are safety guardrails to prevent AI from generating hate speech, though AI admits the guardrails can be jumped by trolls.

 AI developers admit that what AI does once it gets going is so complicated that even they don't understand how it works. If that's true, what hope is there of understanding how human learning works?Maybe AI can help. First off, it says, we have good brains. We understand how they work. Then there's DNA which somehow makes us what we are, but we don't yet fully understand how DNA works. We have good senses to help us learn about our surroundings and perform trial and error experiments: "Og ate that mushroom and died. Tell everyone." We write things down and pass them on to the kids so they don't have to keep starting over. And finally, there's Science which most of us leave to the scientists.

   I personally was trained on a Small Language Model staring in first grade. Did I need to go beyond eighth grade? Not really, but school saved me from having to get a job. Four years in the Navy was my real education. I saw the world, or enough of it to realize it was strange and very big. I could self-train from that point on, like the LLMs do.

   I'm able to learn from those smarter than me. When I was a kid I had an irrational fear of dogs. Not so irrational really. Dogs are strong and have sharp teeth. My mother told me a dog could smell my fear which would make it want to take a bite of me even more. Keeping a stiff upper lip is one thing but I am still unable to squelch my fear when a large barking dog come running at me. I relax if it's wagging its tail.

   I used to take a short cut through the woods to get to my cousin's house. The path crossed about fifty feet of the territory a mid-sized, non-tail wagging dog who always confronted me. I froze on the path while the dog barked and bared its teeth. Every time it looked away, I moved a foot up the path. This went on for several minutes until I reached the far side of the dog's territory at which point the dog ran home. I was not saving any time with this shortcut. If AI had been around back then I could have picked up some dog-attack tips. Carrying a stick would have helped. Or a bag of meat scraps. Whenever James Joyce went out he had a cane with him. An irrational fear of dogs is something Joyce and I have in common. I've learned over the years that the best way to squelch fear is to become familiar with, even friends with, the object of my fear.