Lessons from science fiction for the AI hype

The people I follow on Bluesky have been having a gleeful week making fun of and swapping articles about the obvious errors created by Google’s AI Overview. I’ve had a chuckle or two myself. Of all the reasons for STEM majors should take humanities courses, one of the most urgent is the need for literature classes to teach them about satire and jokes—so that they can program their AIs to avoid The Onion and Cracked. After the schadenfreude faded, however, I recalled two works of science fiction that have some very important lessons about technology and the commodification of information: The Word Exchange, by Alena Graedon, and Rainbows End, by Vernor Vinge.

Both of these novels show apocalyptic (to me, anyway) visions of what might happen if the general population loses access to information unmediated by technology. In The Word Exchange, dictionary apps lead to a global aphasia. Words become unmoored from their meanings when the owners make it impossible for anyone to look them up without their apps. In Rainbows End, the Theodore Geisel Library at the University of California-San Diego is literally chewed up and pulped by machines that read the information and upload the contents to a proprietary database. If you can’t afford access to the database, then you’re cut off from everything you might’ve learned from the library.

Of the two, Rainbows End hit me harder. I work for a university library and I know very well how much it costs for our university to subscribe to our suite of databases. Some of them run to tens of thousands of dollars a year for access. The majority of our access to journals, magazines, and newspapers is tied up in these databases. If we can no longer pay these subscriptions, we don’t have print copies to fall back on the way we used to twenty years ago. It’s very easy for me to see a future like the one Vinge created, with gargantuan tech companies mining libraries for everything of value and cutting off anyone who can’t pay.

Right now, Google is free. (Before anyone quotes this at me, I’m well aware of the adage that if you’re not paying for the product, you are the product.) But there is a non-monetary cost that we don’t think about most of the time. Google’s algorithm has, for years, done a lot of the heavy lifting in helping us get to the information we want. This algorithm is a deliberately black box. Google doesn’t reveal all of their special herbs and spices to try and prevent people from gaming the system. This worked wonderfully for a long time, I think, until the company started to add things: sponsored ads, people also asked, knowledge graph, video spotlights from YouTube, etc. Now there’s the AI Overview, which scrapes the internet to create short answers without always making it clear where that information comes from and, thus, making it hard to evaluate the quality of the information it provides.

In both novels, the technology becomes unsustainable. Any benefits they might’ve provided are swept away by their enormous costs or by the outsized harms they caused. The only remedy is to roll back the clock on the tech. The Luddites are proven right. I can’t entirely agree with a complete rollback on technology. I love having access to the internet and there are some nifty pharmaceuticals keeping me mostly cancer-free. But I’ve also been buying fountain pens and keeping a journal. I’ve been buying more hardbacks than I have in recent years because there are books I never want to lose access to if the ebooks blip off of my ereader. I believe that there’s a middle ground between Ludditism and techno-utopias.

I would argue that science fiction like The Word Exchange and Rainbows End are just the kind of books I would want in the hands of AI programmers and tech futurists (as well as a solid explainer of satire). Hopefully, stories like these might make them pause for thought about the possible consequences of their products.