I have successfully built my first ever large language model, but don’t expect to find it on Hugging Face anytime soon.
Why? Because the model, which I’ve named lariat-1.7b, was built for one very specific job: detecting the writing of Will Rogers.
Will Rogers is one of America’s greatest humorists but many people have never heard of him. In the 1920s and 30s, Rogers was one of the most famous entertainers in the world. He wrote a mix of daily and weekly syndicated columns, published books, regularly contributed to magazines such as The Saturday Evening Post and Life, starred on Broadway, radio, and film, and by 1935 was one of the biggest box office draws in Hollywood. Tragically, in August 1935, at the peak of his fame, he died in a plane crash in Alaska.
I’ve been fascinated with Will Rogers ever since I took an elementary school field trip to his memorial and museum. Like me, he was born and raised in northeastern Oklahoma and his good humor and generosity of spirit always struck me as the kind of person, and the kind of Oklahoman, I wanted to be.
His work is also a fascinating window into the period between the World Wars, which is exactly why it fell out of popular culture so easily. Much of his work was just riffs on the day’s news, and reading it a hundred years later is a lot watching late-night monologues from before you were born. Even if you can recognize the main ideas, smaller stuff just goes right over your head because you don’t have the context. The world his jokes depended on started unraveling almost as soon as he died with the build-up to World War II kicking off just weeks after he was laid to rest.
In short, Will Rogers got memory holed. I want to fix that.
the cowboy philosopher
A couple of years ago, I was wandering through Barnes & Noble and ended up in the philosophy section. All the usual suspects were there: Marcus Aurelius, Epictetus, Seneca. But a little voice in the back of my head asked, “What about the cowboy philosopher?” I searched for Will Rogers books on my phone and found they’d been out of print for decades. A few eBay sellers had copies that looked new, but they were clearly print-on-demand garbage. That’s how I learned that much of his work had entered the public domain in just the last few years.
It made me wonder why it was easier to find writers from two thousand years ago, whose words had been copied and recopied across centuries, materials, and languages, than a man from my home state who spoke my language less than a century ago. In an instant, I knew the answer. Every time humanity went through a major format change, it was only because someone cared enough to carry those works forward and repackage them for the next generation. And they didn’t just preserve the words. They preserved the context, because words from the past only carry their meaning when we can read them the way they were meant to be read.
Since then, I’ve been on a mission to gather, index, and preserve his work, with the goal of launching a complete digital archive anyone can access for free.
So where does a language model fit into this? I certainly never set out to fine-tune a model, and I have no interest in simulating Rogers to generate new text. But part of my research has been recovering some of his less publicized writing, and that’s where I found the use case.
From May 1925 to January 1927, Rogers had a syndicated column called “The Worst Story I Have Heard Today.” It ran in newspapers almost every day, and the conceit was simple: Rogers would tell you about someone who told him a joke, and then he’d tell you the joke. Each installment was about 200 words.
When Oklahoma State University published The Writings of Will Rogers in the 1970s, the definitive collection of his work, the editors left the Worst Story columns out. It’s not hard to see why. In her biography of Will, his wife Betty said she supplied many of the jokes herself, out of old joke books she hunted down in used bookstores. She also wrote that Will never liked the format and eventually gave it up, but he was still under contract, so the feature kept running under his name. Ben Yagoda, one of his biographers, wrote that an editor at the syndicate wrote the last of them and the two split the money.
I’ve assembled and transcribed every one of these columns from various archives, but finding the point where Will stopped wasn’t as obvious as I expected.
My first attempt used a classic stylometry method called Burrows’s Delta. It compares how often writers use common words, and it comes from the same tradition scholars have used to argue over who wrote the Federalist Papers and which parts of Shakespeare’s plays were collaborations.
Stylometry rated some of the suspect columns as more Rogers than the real Rogers. When I considered it, it made perfect sense. The later columns used the word “guy” about 20 times more often than Rogers did, and the phrase “I reckon” 35 times more often. The phrase “this here” showed up just twice in 2.6 million words of his real telegrams and columns but appeared 29 times in 20,000 words of the suspect columns. It was a caricature. Whoever ghostwrote these knew what Rogers was supposed to sound like and laid it on thick. It’s probably enough to fool someone casually skimming the paper over breakfast, but the stylometry really only confirmed my smell test and I’d need something stronger to pin it down to dates.
how lariat works
A language model, more than anything, is a tool for predicting the next word. Train it on enough of one person’s writing and it gets very good at predicting how that person writes.
The catch is that I didn’t want it to decide any 1920s newspaper English was Will Rogers, or every columnist of his era would pass. So Lariat is actually two models. I started with SmolLM2, a small open model with 1.7 billion parameters, and trained two adapters on top of it. One learned Will Rogers from about 1.4 million words of his daily telegrams, weekly columns, magazine articles and books. The other learned his contemporaries, about 1.7 million words in the public domain from other humorists published in the same timeframe.
To score a passage, I show it to both models and see which one is less surprised by it. If that sounds familiar, it’s basically how a lot of AI detection tools work. Since both models learned from 1920s writing, the 1920s part should wash out, and whatever’s left over is the part that sounds like Will. If the Rogers model wins by a big enough margin, I count it as probably his.
Training was the easy part. Cleaning the data was not. Rogers’s published work is packed with words that aren’t his. He quotes letters from readers, telegrams, news dispatches and speeches. His editors added headlines and notes and captions. Every daily telegram starts with a dateline and ends with some version of “Yours, Will Rogers.” It’s a lot. My first model trained on all of it and basically learned to spot a Will Rogers column by how it was laid out on the page.
So for round two, I ran everything through Cloudflare’s Clef model one paragraph at a time and had it sort each one: Will talking, Will quoting somebody, an editor’s addition, or no idea. If a paragraph was a mix, I broke it down and sorted it sentence by sentence. Only the stuff that was clearly Will went into training.
I also kept some of the collection out of training entirely so I could test it. A model that only recognizes text it’s already seen isn’t worth much, so before I ever pointed it at the Worst Story columns, I gave it Rogers writing it had never seen mixed in with other 1920s humorists it had never seen. It sorted them correctly about 99% of the time.
When I finally tested the suspect Worst Story columns with Lariat, I found a long stretch that matched, and then it got spotty. The columns were getting extremely vague about names and places, and on September 23, 1926, the column opened with a joke from “a Scotchman in London” and scores well below the line. So do the next two. Then on September 27 and 28, the old Rogers seems to be back. One of those credits the singer Irene Franklin as “a fellow Concert Artist of mine,” which is exactly how the real Rogers talked, but after that, the column stops getting specific for good. From October 2 on, every column but two scored as not Rogers, and those two only barely passed.
My conclusion is that his editor took over around September 23 with a few Rogers’ columns banked. This way, they could intersperse them over the following week to see if anyone noticed. The timing also fits because the following October is when his daily telegram to The New York Times went into national syndication, giving him even less reason to keep up a daily joke column.
from the desk of the will rogers archivist
This is the first time I’ve written publicly about the archive, and while I’ve been at it for a couple years now there’s still a lot more I want to do before I share anything else widely. I’ve used a lot of different tools in this project, from reel-to-reel tape machines at the National Archives in Maryland to VHS capture devices and even building my own non-destructive book scanner. It’s just funny to me that a large language model also turned out to be one of the tools. Something people think is only good at pretending to write like other people helped me catch somebody pretending to write like Will Rogers.
It’s a reminder that every format change in history has looked like a threat to the written word, and every one of them has also been a chance for somebody to carry it forward. Even writing itself was controversial. Socrates opposed it, believing it would weaken his students’ memories and give them the illusion of wisdom rather than real understanding. We only know that because Plato wrote it down anyway.
Will Rogers never worried about new formats. He went from the rodeo, to the stage, to the silent movie screen, to newspapers, then radio, then talking pictures, and he was rather good at every one of them because he was always himself in every format. I don’t know what he’d make of the Internet and language models reading his old columns, but I’m pretty sure he’d have found something to make a joke about it.
There’s even a bit of an epilogue. I also ran Lariat against The Bull’s Eye, a series of Bull Durham tobacco ads from 1924 to 1926 presented as columns Rogers wrote, or at least signed. These had also been excluded from The Writings of Will Rogers without any explanation, so I fully expected some of it to fail, since ad copy goes through an agency that could easily twist or replace his words. As best I can tell, it didn’t. More than ninety percent of the ad text scored as Rogers.