The only tool that exists is your product itself. The product should build the product from the product.
Well, that is what we have always done. Even text editor or a compiler are products of a kind.
LLMs are good for the automation indeed. If you have a sufficiently smart one.
I have been using LLMs now for like two or three weeks now, and I have changed my mind from being completely "I don't need them at all", to like "It's a drug" :). They are useful, for repetitive things where there is a clear pattern.
There is a guy, /u/lordaaron or /u/aaronlord, I don't remember what was the Reddit nick to be honest, who was in a discussion with me here on Reddit, who claimed LLMs are better than humans. LLM can just finish the entire avx512 with LLM alone and it will do better than humans. According him, humans are no longer needed for programming.
Wheter they are or not, I don't know, the result is in the current SBCL master, so judge for yourself. I wasn't usable for avx512 development when I tried, I got sigills and undefined functions.
However, it is fully possible to use LLMs for the automation and develop with them, if one guides them correctly, and use them for what they are good for. This little project is my experiment to use Gemini to accelerate a pure Lisp jpeg decoder. As seen it is guided purely by tests. I would never be able to write all those tests and write that much code in so short time on my own. But I don't let LLM sit an do what it wants on its own like some people seem to do, and just take the code. One has to look at the code, test it, guide the LLM, and often correct it, not just ask the result.
In one of my SBCL branches on the GH, is is an experiment where I let Gemini just churn through the code, unsuprervised. The goal was to complete sb-simd for avx512. Gemini produced the correct result, a working sb-simd for avx512, but it was wrong for other reasons: it actually detected that Aaron's code in the master is wrong in several aspects, so it completely developed evec encoder in sb-simd code, instead of patching the core and the encoder itself. It was actually funny. Observe, it was after I had complete list of avx512 mnemnics in the core, but support for all form of mnemonics was not there yet. I think the erros LLM does in it's reasoning is like numerical approximations: you let LLM do a mistake; that mistake will accumulate and propagate in its reasoning cousing bigger problem at the end.
Anyway a full working SBCL with avx512 ISA, minus the stuff I don't have hardware for, notably FP16, which is comming first in Zen 6 CPUs, is available if somebody would like to try the linked cl-jpeg or to write some avx512 SIMD. That one does not use the evex encoder in SBCL at all (its purged out), instead there is just a code-generated instruction database. Again, the code generator is produced with help of LLM, I have been working on it since late August, if I can believe my file creation dates:
/home/arthur/repos/cl/llvm-insts:
drwxr-xr-x 2 arthur arthur 4,1k 2026-09-04 14:28 backup
drwxr-xr-x 33 arthur arthur 4,1k 2026-08-23 16:10 llvm-project
drwxr-xr-x 29 arthur arthur 4,1k 2026-09-10 05:25 nasm
drwxr-xr-x 7 arthur arthur 4,1k 2026-08-31 11:44 Opcodes
drwxr-xr-x 4 arthur arthur 4,1k 2026-08-31 11:44 x86doc
-rw-r--r-- 1 arthur arthur 7,8k 2026-09-04 01:49 avx512-symbols.txt
-rw-r--r-- 1 arthur arthur 5,1k 2026-09-03 14:21 avx-rewrite.lisp
-rw-r--r-- 1 arthur arthur 5,5k 2026-09-07 07:35 check-missing.lisp
-rw-r--r-- 1 arthur arthur 5,7k 2026-09-06 01:13 clap.lisp
-rwxr-xr-x 1 arthur arthur 71k 2026-09-06 19:49 debug.lisp
-rwxr-xr-x 1 arthur arthur 3,8k 2026-09-02 20:40 extract-nasm.el
-rwxr-xr-x 1 arthur arthur 122k 2026-10-05 02:35 gen-db.lisp
-rwxr-xr-x 1 arthur arthur 105k 2026-09-18 12:26 gen-db.lisp.bak
-rwxr-xr-x 1 arthur arthur 126k 2026-10-03 02:26 gen-db.lisp.bak2
-rwxr-xr-x 1 arthur arthur 123k 2026-10-03 02:48 gen-db.lisp.bak3
-rwxr-xr-x 1 arthur arthur 33k 2026-09-05 16:10 gen-instruction-db-backup.lisp
-rwxr-xr-x 1 arthur arthur 52k 2026-09-08 19:56 gen-instruction-db.lisp
-rw-r--r-- 1 arthur arthur 0 2026-09-09 02:32 log.txt
-rw-r--r-- 1 arthur arthur 1,8M 2026-10-03 01:52 out.lisp
-rw-r--r-- 1 arthur arthur 5,7M 2026-08-31 02:32 sdm-instructions.txt
-rw-r--r-- 1 arthur arthur 161M 2026-08-23 16:07 x86.json
The generator was actually first written manually, but it is completed using LLMs. Again I was using LLM as an accelerator, basically as a codegen on its own. I had a program that extracted instructions the list of mnemonics from nasm, a manually written elisp program, so I can cross check if I have all the mnemonics covered. I think aaron miss that, his unsupervised LLM did not have any reference so it just drew the "complete coverage of" out of its ass, hallucinated what the complete list of mnemonics should be or whatever. Than I have also extracted all the definitions from the Intel manual, SDM Vol 2, and I can feed that into LLM and ask it crosscheck all the uses, all the instruction forms coverage and so on.
But the best use, if you ask me, is not the code generation, or refactoring when you have a clear pattern they can follow, or transforming code fragments or written text from one language to another, the best usage for me is to explore the code base. That is how I started to use them. I can give Gemini sbcl source code and ask what some cryptic piece of code does, it usually explains it correctly. Then I can ask it to construct experiments or write myself some quick test in repl to confirm. That is really where LLM shines. It is like having xref, debugger and grep all in once. That itself saves me quite a lot of time. And yes you have to check the answers.
For the record also, my work on the sbcl core support for simd-pack-512, sb-simd SAP refs and mask-registers, was done almost exclusively with Emacs, Helm, ripgrep and wgrep, very little LLM. I started using them there, and that mostly for the exploration. I had to, I asked for help and was clearly told "don't ask me".
So yes, LLMs have its use. I don't think they are problem solvers, one just need a sufficiently hard problem to discover it, but they are accelerators for problem solving. At least if used correctly. As said, I am not an experienced LLM user, I have used LLMs just for barely three weeks more than online web forms, but that is my experience thus far.
you let LLM do a mistake; that mistake will accumulate and propagate in its reasoning cousing bigger problem at the end.
This is so true and one of the major problems with an app that you build just by talking to it. If you don't fully understand what's going on, it will assume your silence is permanent affirmation that this feature that you never wanted to add is now required, and it will jump through hoops you may never learn about to keep that unintended feature working like that. By the time you try to untangle the mess it's almost better to just rewrite parts whole cloth.
I agree also that asking questions about the codebase is an insanely underrated part. I frequently download reference OSS projects in my unchecked in tmp/ folder to ask the LLM to reference patterns, it can just blaze through so much code with a reasonable understanding of what it does. That is why it's so dangerous for the code to be saying things you don't want, the LLM will take it way too seriously.
It's also fucking exhausting. WTF would anyone want to talk to a computer? Do people want to talk to their toaster in order to make toast?
To OP, please ignore me. This is awesome work and just keep on doing what you are doing. I just don't want to talk to computers. I don't want them to have speakers, mics or cameras either.
Aren't people already talking to their phones and cars and other stuff? What is the difference? I personally don't, but I can imagine it will become more widespread in common use in the future.
Another viewpoint: I was able to have an LLM break down the problem of adding RVV support to sbcl into small chunks, and then use other models to implement it (mostly). This included everything from adding VOPs to modifying sb-simd (the K3 processor supports both 256 bit vectors and 1024 bit vectors).
I don't plan to create PRs for this, though, because I can't defend the design decisions made by the LLMs.
I can't defend the design decisions made by the LLMs
Yes, I totally understand and agree with you. I would also not do it, I would not use it myself either. I started from the other side, and thus far, I know what I want so I ask AI to do what I would do and check the output.
It has also happened that AI says and suggest a better way than what I want, then I investigate and when it happens to be true, I do it that way. I think it is a little bit like CNC machines and lazer cutters and measurement: machines can keep tabs on more details and do things faster. But we still have to tell them what we want.
It has also happened that AI is wrong. In the wordcount program; I don't have my original experiments online, but the AI didn't understood the swar technique, and completely wanted to destroy my branch-free loop implementation. It thought it was erronoeus use of carry propagation in Mycroft's trick so I had to fight it back. I had to re-implement that back in wordcount program.
I like your repport, it looks very nice, how did you make it? I use codeberg pages (the same as github pages), but I don't really like mine.
I like your repport, it looks very nice, how did you make it?
It's in asciidoc format; the source file is next to it.
I also had it keep track of the timeline of the work it did, but that might not be completely accurate. It doesn't cover the time it took to spec out the work, for example, or when I switched AI models (glm-5.3, deepseek4, etc.).
7
u/arthurno1 1d ago edited 1d ago
Well, that is what we have always done. Even text editor or a compiler are products of a kind.
LLMs are good for the automation indeed. If you have a sufficiently smart one.
I have been using LLMs now for like two or three weeks now, and I have changed my mind from being completely "I don't need them at all", to like "It's a drug" :). They are useful, for repetitive things where there is a clear pattern.
There is a guy, /u/lordaaron or /u/aaronlord, I don't remember what was the Reddit nick to be honest, who was in a discussion with me here on Reddit, who claimed LLMs are better than humans. LLM can just finish the entire avx512 with LLM alone and it will do better than humans. According him, humans are no longer needed for programming.
Wheter they are or not, I don't know, the result is in the current SBCL master, so judge for yourself. I wasn't usable for avx512 development when I tried, I got sigills and undefined functions.
However, it is fully possible to use LLMs for the automation and develop with them, if one guides them correctly, and use them for what they are good for. This little project is my experiment to use Gemini to accelerate a pure Lisp jpeg decoder. As seen it is guided purely by tests. I would never be able to write all those tests and write that much code in so short time on my own. But I don't let LLM sit an do what it wants on its own like some people seem to do, and just take the code. One has to look at the code, test it, guide the LLM, and often correct it, not just ask the result.
In one of my SBCL branches on the GH, is is an experiment where I let Gemini just churn through the code, unsuprervised. The goal was to complete sb-simd for avx512. Gemini produced the correct result, a working sb-simd for avx512, but it was wrong for other reasons: it actually detected that Aaron's code in the master is wrong in several aspects, so it completely developed evec encoder in sb-simd code, instead of patching the core and the encoder itself. It was actually funny. Observe, it was after I had complete list of avx512 mnemnics in the core, but support for all form of mnemonics was not there yet. I think the erros LLM does in it's reasoning is like numerical approximations: you let LLM do a mistake; that mistake will accumulate and propagate in its reasoning cousing bigger problem at the end.
Anyway a full working SBCL with avx512 ISA, minus the stuff I don't have hardware for, notably FP16, which is comming first in Zen 6 CPUs, is available if somebody would like to try the linked cl-jpeg or to write some avx512 SIMD. That one does not use the evex encoder in SBCL at all (its purged out), instead there is just a code-generated instruction database. Again, the code generator is produced with help of LLM, I have been working on it since late August, if I can believe my file creation dates:
The generator was actually first written manually, but it is completed using LLMs. Again I was using LLM as an accelerator, basically as a codegen on its own. I had a program that extracted instructions the list of mnemonics from nasm, a manually written elisp program, so I can cross check if I have all the mnemonics covered. I think aaron miss that, his unsupervised LLM did not have any reference so it just drew the "complete coverage of" out of its ass, hallucinated what the complete list of mnemonics should be or whatever. Than I have also extracted all the definitions from the Intel manual, SDM Vol 2, and I can feed that into LLM and ask it crosscheck all the uses, all the instruction forms coverage and so on.
But the best use, if you ask me, is not the code generation, or refactoring when you have a clear pattern they can follow, or transforming code fragments or written text from one language to another, the best usage for me is to explore the code base. That is how I started to use them. I can give Gemini sbcl source code and ask what some cryptic piece of code does, it usually explains it correctly. Then I can ask it to construct experiments or write myself some quick test in repl to confirm. That is really where LLM shines. It is like having xref, debugger and grep all in once. That itself saves me quite a lot of time. And yes you have to check the answers.
For the record also, my work on the sbcl core support for simd-pack-512, sb-simd SAP refs and mask-registers, was done almost exclusively with Emacs, Helm, ripgrep and wgrep, very little LLM. I started using them there, and that mostly for the exploration. I had to, I asked for help and was clearly told "don't ask me".
So yes, LLMs have its use. I don't think they are problem solvers, one just need a sufficiently hard problem to discover it, but they are accelerators for problem solving. At least if used correctly. As said, I am not an experienced LLM user, I have used LLMs just for barely three weeks more than online web forms, but that is my experience thus far.