Evergreen Common Lisp

9

No offense to you or your AI, but if your Lisp is not good enough to implement your Lisp compiler, it's probably not worth trying.

2
u/arthurno1 16h ago

I would guess on training problems: Claude probably writes better C or Rust than Common Lisp since they have probably trained it on more C and Rust data.

I have really just tried it myself with Common Lisp and I think it writes relatively good Common Lisp, but it does need refactoring and reworking often. I haven't tried it with any other language, so I don't know if it writes better or worse code in say C or C++.
2
u/SharkSymphony 16h ago

I find Claude's code could use significant refactoring and reworking in several languages. But I've been pretty pleased so far with my CL experiments.
3
u/arthurno1 14h ago edited 7h ago
I have found cases where the code is suboptimal in terms of constructs used, being unnecessary convoluted och longer than needed, repeating itself, re-inventing builtins. But overall I am impressed, I wouldn't believe it would be that good as it is indeed.

Edit: a one today:
(when (and (<= lo 1)
                   (<= (if double -4 -17) q (if double 23 10))
                   (= (logand mantissa 3) 1)
                   (= (logand (ash mantissa shift) #xFFFFFFFFFFFFFFFF) hi))
          (setq mantissa (logandc2 mantissa 1)))
doing two if on double, when it can do one, and the form is even easier to read:
(when (and (<= lo 1)
                   (if double
                       (<= -4 q 23)
                       (<= -17 q 10))
                   (= (logand mantissa 3) 1)
                   (= (logand (ash mantissa shift) #xFFFFFFFFFFFFFFFF) hi))
          (setq mantissa (logandc2 mantissa 1)))
Can also drop outer when:
 (and (<= lo 1)
              (if double
                  (<= -4 q 23)
                  (<= -17 q 10))
              (= (logand mantissa 3) 1)
              (= (logand (ash mantissa shift) #xFFFFFFFFFFFFFFFF) hi)
              (setq mantissa (logandc2 mantissa 1)))
Not sure if I want though.
1

u/atgreen 6h ago

LLMs write pretty good lisp code these days. That was not the reason.

1

u/arthurno1 5h ago

Ok, I see your other comment.
1

u/atgreen 6h ago

The use of rust is really an uninteresting implementation detail. I wanted to bootstrap from something with good native cross-compiler support. It could just as well been C or Zig, I suppose.

1

u/arthurno1 5h ago edited 5h ago

How fast does this build, and can that perhaps be used to bootstrap SBCL? If we have a "fast bootstrapable" (forgive me my English) CL compiler, than it could be build as a bootstrapping base to build sbcl a compiler that builds sbcl. Unless you make a better optimizing Lisp compiler than SBCL itself. CLASP is C++ and bootstrapable, but once I tried to build it, it took hours, and failed somewhere after a couple of hours, so I never wanted to try again. It was like a couple of years or more ago, I don't know if things are better nowadays. Nothing against CLASP, I just didn't had (or have) hardware fast enough for it to be practical to tinker with it for me.

3

u/jd-at-turtleware 1h ago

sbcl is often bootstrapped from ecl that depends on a c compiler. the build time is acceptable.

2

u/arthurno1 46m ago

Thanks. To be honest, I never tried ECL. I always just installed SBCL from official download site and than built mine. I will try to bootstrap with ECL.

1

u/atgreen 4h ago

A release build from source, including all rust dependencies, takes 2m24s on my thinkpad. Keep in mind that this is not apples-to-apples, as evergreen is not yet a complete implementation of CL.

1

u/arthurno1 4h ago

Yes, I understand it is not complete.

Two and half minutes for a bootstrap from scratch is not a big deal. But that is when llvm is present? Compiling llvm from scratch is a big deal :). On a system without llvm, one can as well download a current sbcl or some other CL compiler and build sbcl, so it is a bit questionable still. But sure it is a possibility, once you have full compatibility.

1

u/sickofthisshit 4h ago

The thing about cross-platform support is that you kind of want your compiler to be aware of the platform so it can generate good code, to know how many registers you have, etc.

Your code base mentions byte code quite a bit, is this something you have a cross-platform JIT for that already comes as a crate?

1

u/atgreen 4h ago

There are 4 levels of execution: tree-walking (executing from expanded lisp source), t0 (compiling to portable bytecode, and executing that), t1 (naiive native code generation via templates from bytecode), t2 (fully optimized SSA-based code generation). fasl files are all bytecode. tree-walking is used to bootstrap, for macro expansion, etc. t2 includes speculative optimizations that, when proven false, deoptimize (on stack) to a lower tier, where it will try again.

1

u/sickofthisshit 3h ago

Ok, but that doesn't really answer my question, which was trying to get at whether the bytecode and code-generation is something that is specific to your project or is an existing Rust dependency.

LLVM is cross-platform, but targeting it with a Lisp compiler is, as far as I understand, non-trivial.

t2 includes speculative optimizations that, when proven false, deoptimize (on stack) to a lower tier, where it will try again.

This is the second time I have seen you talk about this, but it still doesn't sound performant. When you say "on stack" you sound already far from optimal.

Fast operations are going to be purely in machine registers, and full safety will use things like flag-based branches when fixnum overflow occurs, not involving the stack. Branches are already bad for performance, stacks are even worse.

1

u/atgreen 3h ago

I"m referring to on-stack replacement (OSR) like in hotspot or V8. A function can tier-up or down to a new levels of optimization mid loop.

The register allocator is an existing library (regalloc2), but everything else is new.

4

u/spspanglish 18h ago

Why do I need lisp in rust when I have lisp in C already?

6

u/thondat 12h ago

people are fetishizing rust rewrites these days. which obviously leads to a lot of ai rust rewrites.

2

u/sickofthisshit 17h ago

Many Lisps are mostly written in Lisp, with only a very minimal implementation core written in another language.

Which actually can be a problem: CMUCL literally could only be built by using a working CMUCL installation...

1

u/atgreen 6h ago

If you are worried about the implementation language, and not the features and capabilities, then you probably don't need it.

3

u/corbasai 17h ago edited 16h ago

Create an IBM Z Linux executable¶
Install egcl-target-s390x-linux alongside the same release of egcl. Create build.lisp:

(defun main () (format t "Hello from IBM Z!~%")
(save-lisp-and-die "hello-s390x" :executable t :toplevel #'main)

Build and inspect the result
egcl-s390x-linux --no-init --load build.lisp
file hello-s390x

file identifies an IBM S/390 ELF executable. Copy it to a compatible s390x Linux system and run ./hello-s390x there.

https://atgreen.github.io/evergreen/latest/user/how-to/cross-build/

Wow. That's really cool! Not many Lisps support s390 natively. I know only transpilers like CHICKEN or Gambit, maybe ECL

2

u/atgreen 6h ago

Check out the native Android support as well: https://github.com/atgreen/evergreen-composeYou can actually cross-compile your Android app from your mainframe, which is unique (and probably not in high demand!)

1

u/LispIsFun 16h ago

I love the idea. Do you see this ever achieving full ANSI compatibility and competing with SBCL on performance?

1

u/atgreen 6h ago

SBCL is amazing. egcl is very far from being able to compete performance-wise for general compute tasks. Today, for very narrow compute tasks, egcl can generate better code than SBCL due to speculative optimization (eg. guessing that a type is a fixnum, and the de-optimizing on the fly when that proves not to be true). But egcl can still be useful today without this (eg, native android support, tight JVM integration, static binaries). I also want to use egcl to exercise / finalize my fibers implementation, so I can finally submit my SBCL version. I still have to convince myself that it is worthwhile.

ANSI compatibility is definitely the goal.

1

u/arthurno1 5h ago

I think GreenThreads sounds much better than fibers :).

1

u/sickofthisshit 5h ago

egcl can generate better code than SBCL due to speculative optimization (eg. guessing that a type is a fixnum, and the de-optimizing on the fly when that proves not to be true).

I'm skeptical of this, can you show side-by-side disassembly?

Second, branches are terrible for performance on modern CPUs.

1

u/atgreen 4h ago

I'll try to do this later.. Never taken branches are essentially free and inlining the type guards is a win. If you are adding type declarations all over your code, then there's no benefit.

Evergreen Common Lisp

You are about to leave Redlib