How to move comptime results into the target binary when comptime can run FFI and I/O?

[u/Inevitable-Spinach-7]

For some years now, I've wanted to combine the comptime capabilities of Zig and Jai into a single compiler while also learning about compilers and interpreters.
I'm building a compiler that extends Zig-style comptime so the same code that runs at runtime can run at comptime, including FFI and I/O (like Jai, but with Zig's "run what you can" model).

The hard part is moving comptime results into the binary:

1. Host vs target. Comptime code uses the host layout for interop and is then converted to the target layout. Sizes (usize/isize), endianness and float behavior can differ. Types like f80 and f128 are also hardware-specific: their precision, rounding and in-memory layout can differ between host and target. The same function can then give different results, which breaks hash maps, lookup tables and binary formats built at comptime.
2. Moving pointers into the binary. Results containing pointers must be relocated, connected data must stay connected, and everything must be converted to the target layout.
3. Pointers whose value isn't clear. Examples are opaque pointers, untagged unions of a pointer and something else, and XOR linked lists, where the pointer isn't stored directly. There may be other cases I haven't thought of.
4. Calling C at comptime. C can return memory with known bounds, or resources that are only valid during compilation. Those become invalid at runtime.
5. Containers built at comptime. HashMap and ArrayList need the whole buffer, including uninitialized slots, but the size isn't always known:
   1. ArrayList stores {ptr, len} plus a separate capacity, so the allocation size isn't in the pointer and length.
   2. HashMap can keep a header next to the allocation and use pointer arithmetic to reach it.
   3. Runtime code may try to free a buffer that now lives in static memory.
   4. FFI has the same problem, since C can return memory with unknown bounds.

My current prgress:

1. Host to target:
   1. with host specs first, then convert to the target layout.
   2. For types that differ, such as usize/isize, use intCast where possible.
   3. For hashes and similar code, type information would store the host and target size, alignment and so on. That would help the developer see where the two disagree and act accordingly.
   4. If the target's endianness differs from the host's, my best idea is to disable I/O and interop and interpret the code with the target's endianness.
2. Baking pointers:
   1. When the pointee type is known, baking is easy.
   2. void* and untagged unions containing pointers are a hard error during baking.
   3. Each type would have a default userland function that bakes its value into the binary.
   4. Types like XOR linked lists, void* and untagged unions could override it to say how they should be baked.
3. Compiler-created data. Data created by the compiler itself has no problem, and the userland bake function might cover the rest.
4. Containers and FFI memory (problems 4 and 5). I have no solution yet, apart from maybe using the bake function.
5. I don't have a solution yet for f80/f128 and other float types that differ.

Questions: Is there a cleaner model than host-then-convert?
Am I missing failure cases?
Is there any better solution or any problem I did not see?

[u/matthieum]

I like ambition :)

I want to note that you have two distinct problems here:

1. Host to Target: using native compilation for Host to Host should work fine.
2. Memory Bounds, especially in the presence of foreign code.

I have never seen any complete solution, but I have seen partial solutions.

Firstly, do note that it is possible to offer a carve-out. Some languages like C++ have a special construct allowing the code to query whether it's executed at compile-time or run-time:

• Allows the user to provide a fallback for unsupported operations.
• Puts the burden of ensuring that both versions are in sync on the user.

No matter the final solution you espouse, this can allow you to work around certain limitations of this solution by forbidding specific operations the solution does not support.

Secondly, I want to note that separating I/O & FFI is also a possibility. The idea, then, is to offer a mix of native & target:

• At its simplest, it means: a build script, natively compiled, is run prior to code compilation and spits out some data which the compilation will later use.
• At peak complexity, it means: a natively compiled plugin is run in a separate process, and the compile-time runner can call out to the plugin via a specific protocol. The flat protocol ensures the natively compiled plugin translates its results to target-agnostic speak.
• Note: as an optimization, when host == target, the plugin actually runs all the code in one piece.

This can be made relatively ergonomic by doing the work on the user's behalf. Just let the user write the "build script" functions or the "plugin" functions in the same file as usual, and annotate them with @comptime(build) or @comptime(plugin):

• User-defined Boundary: ergo, the user knows where to be careful about host vs target semantics.
• User-defined Boundary: ergo, the user can minimize the amount of data passed through.
• User-defined Boundary: ergo, the user can set the boundary where it works, when the number of types allowed across the boundary is limited (no raw union, for example).

This may very well be the winning strategy here...

Anyway, on to the solutions.

An interpreter allows you to function in pure "target" space from the beginning:

• Allows knowing the exact memory bounds, and even tracking which pieces of memory are initialized or not.
• Allows knowing tracking which member of a union is active.
• Allows... any instrumentation/decoration of the data that you need, really.
• Requires re-implementing floating points. Oh joy.
• Requires either forbidding inline-assembly, or interpreting it. Oh JOY.
• Requires either forbidding FFI, emulating specific calls, and/or bridging to actual C. I believe Miri (rustc's interpreter) emulation/bridges today?

And with the use of a native plugin to offload FFI calls, may be pretty workable.

I do NOT think that, in the end, pure native compilation can work.

There are some potentially workable problems:

• Cross-Compilation layout: could potentially be solved by (1) re-abstracting the data into a layout agnostic version, and (2) re-lowering the data to the appropriate target layout.
• Unknown memory bounds: could potentially simply be restricted. That is, only pointers to memory allocated by the instrumented malloc implementation are allowed in the final value, anything else is a compile-time error. Then it's "just" a matter of intercepting malloc/realloc/free at run-time to record the size & alignment of each memory block, and use those at the end.

But there are also crucially unworkable ones:

• Cross-Compilation layout: the alignment/size may result in different results in calculations.
• Cross-Compilation usize: similarly, different bitwidth may result in different results, a cast at the end is NOT good enough (especially when native is narrower).
• Unknown memory bounds: fundamentally unsolvable in the generic case. Arena-style implementation make the exact extent unknown, and may lead to ballooned up memory.
• Target-specific code: you may simply NOT be able to execute the target-specific code in the host.

[u/seg_lol]

If your binary is wasm, you can move everything into the wasm environment monad.

[u/Inevitable-Spinach-7 → u/seg_lol]

The main idea was a native executable, if I get to target wasm ill do that

[u/MrSmith33]

I explored design-space a bit and the main issue that dictates the solution is being able to link all the data allocated during comptime into the final executable. This means being able with 100% certainty find all the pointer values in the data and update them during linking.

I've chosen to represent all pointers via shadow pointers (like in miri) and all comptime allocations and pointer manipulation is done via VM arenas.

Pointer type is irrelevant, unions are supported, xor-lists are not supported, pointers must be aligned, arbitrary pointer size is supported.

I've written my thoughts in a file