The Era of Truthiness

Stephen Colbert’s Late Show ends tonight – and the Late Show itself – ending one of New York’s largest comedic institutions after 33 years. I hope that the Ed Sullivan theater will find a suitable new tenant that continues to have as much joy and respect for its audience and terroir as the Late Show, in both incarnations. Unfortunately, I have little confidence that the Ellison CBS will treat the theater with any more respect than it is with the 11:30 timeslot, which is simply renting to Byron Allen.

While it is disappointing that the Late Show is ending, having time to prepare enabled David Letterman to come back to the Ed Sullivan Theater last week to throw more things off the roof. The Boss performed Streets of Minneapolis. David Byrne came by to burn down the house. Lewis Cato and the Great Big Joy Machine talked about their time on the show (and dropped an album). It was fitting that Monday’s preantepenultimate Late Show was for the LSSC staff more than anyone else. While Colbert will certainly have many opportunities, the show’s cancellation leaves talented and funny Late Show staffers out of work.

We are well past peak-late night talk show. Conan O’Brien, Samantha Bee, The Late Late Show, After Midnight, and all of Comedy Central’s post-Colbert Report attempts at 11:30 shows have ended. The Late Show is by far the most prestigious of these properties to shutter – and it is frustrating that it is due to not merely to financial pressure, but to oligarchs trying to curry favor with a President who lacks the ability to laugh at himself.

Back in the very first episode of the Colbert Report introduced The Word segment and the word Truthiness to the world1. . As aptly summarized by Wikipedia, Truthiness is “the belief or assertion that a particular statement is true based on the intuition or perceptions of some individual or individuals, without regard to evidence, logic, intellectual examination, or facts.”

Truthiness is even more broadly relevant today than it was in 2005. We now live in an era of truthiness.

LLM chatbots confidently and quickly generate text that has the appearance of accuracy and knowledge. But this is just a facade. LLMs do not know anything, they return the most statistically likely response to a query. In many cases, the responses are good to excellent. How leading-edge models turn natural language into software code is nearly magical and can be readily tested. In other areas, they produce output that resembles what a good response should appear to be.

Using an LLM to produce legal advocacy often produces a properly structured argument that usually identifies the applicable legal theories and cites to authority where appropriate. However, the LLM work product often creates cites to non-existent cases that feel like authority. The LLM will claim that a doctrine supports the desired result, regardless of what the actual doctrine might be.

LLMs produce copious amounts of truthiness with speed and authority, but too many people treat it as truth.

Colbert’s time in late night may be ending, but now we all live in a world of truthiness.

  1. Unfortunately, Comedy Central does not have Colbert Report episodes or clips readily available online -- and certainly not in the highly-linkable and embeddable way that CBS posts The Late Show content on Youtube. It will be interesting to see what happens with the Colbert Late Show channel after the show ends.
Andrew Raff @andrewraff