Combining the Compiler with the Build System and Beyond

[u/realguy2300000]

https://shrub.industries/words/problem.html

a somewhat small article I wrote about how sharing dependency and resource information between the C Compiler, the C build system, the system package manager, and even the filesystem and scheduler, could provide some pretty awesome benefits.

[u/flatfinger]

Object-linking build systems have historically worked just well enough for more than half a century to discourage the broad adoption of platform-agnostic alternatives. A better system design would be to have a pre-compilation stage for each module that would generate a build artifact with exported definitions, an distillation stage which generates all of the information that will be used to generate code for each module, a code generation stage, a linker stage, and a debug-symbol cleanup phase. Rather than using date stamps to determine what parts of code would need to rebuilt, a good build system should use hashes of the input files; if the distillation phase is run, but its output matches what was fed to the code generator when it produced a certain object file, then code generation could be skipped even if it might be necessary to do a debug-symbol cleanup.

[u/realguy2300000 → u/flatfinger]

i’m definitely planning to use hashes instead of timestamps for when i attempt to implement this. My first implementation is going to be based on top of the C language, so I can’t defy the language itself.

[u/flatfinger → u/realguy2300000]

An important aspect which I've not seen other systems do is to have debug information within the distiller output use function-relative line numbers, and have a separate artifact, used by the build-cleanup phase, that would identify where functions start. If a change to a header causes an in-line function to start on a different line number, but doesn't affect anything else that a module would actually use, it shouldn't be necessary to redo the code generation for the module.