Issue 588  December 9th 2022

Let me say before I dive into this topic again. ChatGPT is already a remarkable piece of software, and I’ve lost count of how many times I’ve been amazed at screenshots from it in the last few days.

That said, again, I have a really uneasy feeling that I can’t shake.

The possibilities for this software are mind-blowing. It feels like we just jumped leaps and bounds from our conversations with voice assistants, where we ask a single question and listen to it read the opening paragraph of Wikipedia back to us. Yes, that’s simplifying it, but having conversations span several cycles back and forth with sensible and believable results is remarkable.

At the same time, I can’t help feeling like all the breathless praise of the last couple of weeks is very premature. I remember chatting with Kim Silverman in the WWDC labs in 2008 about speech synthesis and recognition. He talked about the early days of speech synthesis in the late 1970s and how developers quickly progressed to 90% of the way there and then spent the next 30 years getting to 95%. 😬 Using AI technology like ChatGPT, DALL·E, or Copilot often makes me think back to that conversation with Kim.

Self-driving car software has the same issue. It’s been a while since self-driving cars were feasible, and here we are 40+ years later, and it still feels “a few years away”. I bet someone said that back in the 1980s, too! I’m not saying there’s not been progress, just that it has moved slower than everyone expected.

Naturally, given how new it is, some of what ChatGPT comes up with is dead wrong but where things get problematic is with how confidently it presents answers. Take the first question that Ben Thompson asked was incorrect as a perfect example. He describes the answer he received as:

This is a confident answer, complete with supporting evidence and a citation to Hobbes work, and it is completely wrong.

If you’ve seen the AlphaGo documentary, then you’ll remember where they can’t figure out why the AI is doing what it’s doing during a game. It’s not only about code, either. It’s the training data and model that’s the problem. From what I can learn about ChatGPT training, it almost certainly has that same problem. Here comes that deeply uneasy feeling again. Yes, people are fact-checking it and examining its output now, but how long before we blindly trust it? 😬

Finally, I’ve seen people suggest that Google is in trouble with this on the horizon. That may be true, but our use of a search engine is fundamentally different to how ChatGPT works. With search, we type a query and get back a set of results, but it’s our responsibility to figure out which results contain accurate, unbiased information. To think that any single training model could be impartial and accurate enough to replace that process seems impossible, or at least well beyond what we see here.

I’ll be the first to admit I’m not an expert on this subject, and I am sure people are working hard on the issues I have mentioned here. I see how quickly people are rushing to find ways this type of technology can integrate with everything we do, and I can’t shake that uneasy feeling. Or maybe I should lighten up and assume it’ll all be fine. 😬

Dave Verwer  

Note: Thanks so much to Dave DeLong, Carter Jernigan, and Daniel Jalkut for helping me pinpoint who I spoke to at WWDC 2008 about speech synthesis! I may not have remembered his name well, but I have thought about that conversation we had a lot over the years.