Research has gone into compilers for over 75 years. Most is available in papers, books, articles, and in the source of flagship open-source compilers such as LLVM and GCC. High-level and assembly languages are well-defined and documented.
All of this has been fed into the latest LLMs. So rather than compile a program by running a
compiler, can an LLM reason its way through a program and emit assembly directly, either from
source or even natural language? I couldn’t get working programs out of GPT-4o, but with o3-mini
it has just started to work. I found success with C and “natural language” (English), but it
struggles with higher-level languages. Grok 3 is even better but I don’t have API access yet.
Even if this approach continues to improve it’s unlikely to replace compilers altogether. At a minimum, some of their work is compute-intensive in a mathematical way that LLMs seem to struggle with. More importantly, languages/runtimes often place constraints on behaviors that greatly improve reliability and security. However, maybe “LLM compilation to assembly” could facilitate the development of future programming languages, allowing language features to be refined and tested even before a real compiler exists. The LLM is not actually compiling the language, but using the source as the prompt to generate the WebAssembly. But it seems like compiling if you squint from a distance.
See for yourself! The programs below were all compiled with this technique. You can change the input for any of the examples and the program will run on each change. To try a new program, just change the source. Cut the LLM some slack — it’s SLOW and not perfect; you might need to try a couple of times before it gets it right. To help prevent abuse of my credits, compiling a new program requires you to sign in with X or GitHub.
C is L33T
This simple in-place transformation worked on the first try. Note that the resulting program is just over 200 bytes uncompressed.
Partial Reverse
This program reverses the string between matching angle brackets. The LLM struggled a bit here:
initially in the r function I wrote *l++ = *r; *r-- = t; but it translated that incorrectly until
I broke it out into separate steps.
Natural Language Party To 11
With the LLM the language is just a way of prompting the model; so natural language or made-up programming languages work as well, to the limits of their specificity and the model’s understanding.
What’s Next?
In a year or two, maybe LLMs will be trained on compiled binaries and be able to generate tons of perfect assembly for any requirement. Perhaps this could replace containerized Python environments in some scenarios where that is too heavy-weight. Or, perhaps LLMs will reap the same benefits as humans do from the higher-level languages and will forever rely on traditional compilers. I look forward to trying these experiments with new models as they become available, and we’ll see where we end up!
If you liked this article, let’s connect on X. Thanks for reading!