This article was originally featured on MIT Press.
Back in 2017, Google researchers unveiled the “transformer,” a revolutionary machine-learning program designed to process language. Originally intended to improve machine translation, the transformer’s capabilities quickly caught the attention of the AI community.
Trained on vast collections of documents to predict what comes next based on preceding context, it developed an uncanny knack for the rhythm of the written word. You could start a thought, and like a friend who knows you exceptionally well, the transformer could complete your sentences. If your sequence began with a question, then the transformer would spit out an answer. Even more surprisingly, if you began describing a program, it would pick up where you left off and output that program.
It’s long been recognized that programming is difficult, however, with its arcane notation and unforgiving attitude toward mistakes. It’s well documented that novice programmers can struggle to correctly specify even a simple task like computing a numerical average. Even professional programmers have written buggy code that has resulted in crashing spacecraft, cars, and even the internet itself.
So when it was discovered that transformer-based systems like ChatGPT could turn casual human-readable descriptions into working code, there was much reason for excitement. It’s exhilarating to think that, with the help of generative AI, anyone who can write can also write programs. Andrej Karpathy, one of the architects of the current wave of AI, declared, “The hottest new programming language is English.” With amazing advances announced seemingly daily, you’d be forgiven for believing that the era of learning to program is behind us. But while recent developments have fundamentally changed how novices and experts might code, the democratization of programming has made learning to code more important than ever because it’s empowered a much broader set of people to harness its benefits. Generative AI makes things easier, but it doesn’t make it easy.
There are three main reasons I’m skeptical of the idea that people without coding experience could trivially use a transformer to code. First is the problem of hallucination. Transformers are notorious for spitting out reasonable-sounding gibberish, especially when they aren’t really sure what’s coming next. After all, they are trained to make educated guesses, not to admit when they are wrong. Think of what that means in the context of programming.
Say you want to produce a program that computes averages. You explain in words what you want and a transformer writes a program. Outstanding! But is the program correct? Or has the transformer hallucinated in a bug? The transformer can show you the program, but if you don’t already know how to program,…