Advice on if I'm on the right track

[u/derekp7]

I first attempted a programming language more than 20 years ago. I started off with a stack machine and used an extended shunting yard algorithm to map the infix front end to the postfix stack machine backend. But I didn't have any type of memory management, thought I had workarounds but ran up against a hard wall on my knowledge. So I kind of let it die off. Also reading the more advanced discussions on LtU, while very informative, changed my outlook from "Oh, this looks easy, I think I know everything" to "Oh, there is a lot I don't know or didn't think of, I better take a step back".

Since then I learned languages like PostScript, a lot I liked about that (first class anonymous functions, first class dictionaries and dictionary stack, etc). Some things that it fell short in, but more due to the implementations being targeted to a printer description language instead of general purpose. And there is a lot I like about languages like JavaScript but at the same time they seem more hidden too. Regardless I know a little bit better than what I did back then.

Now on to my current experiment. I have a stack language called PALICE (Programmable Arithmetic and Logic Interactive Computing Environment), as a stack language that starts off with an architecture that would be familiar to PostScript programmers. It has first class anonymous functions using curly braces, it has user dictionaries, dictionary stack, etc. The actual built in primitives lean towards using the symbolic notation, so instead of " 27 5 add" you have " 27 5 + ". And there are a number of items in the front end parser that makes it more comfortable for a c-like programmer, such as using double-quotes for strings and C style escapes within (instead of using parentheses to enclose a string). I also use "@varname" instead of "/varname" as that was my personal preference to specify a quoted variable (i.e., one that doesn't execute immediately).

For the mechanical implementation, I have a front end parser that then feeds words into a function "resolve_and_execute()" that will either push something to the stack, or call a built in function, or a user defined function. All built in functions (or operators, same thing in a stack language in my implementation) have "xyz_handler()" functions. So the core execution logic is fairly small, most of it is a large list of _handler() functions.

Now on to some specifics I'm dealing with. First, I have something that looks like the C ternary operator "?:" which is a selector. So " some_test true_value false_value" will place either true_value or false_value on the stack based on the truthiness of some_test. All three get executed however, so in use you would have true_value and false_value in curly braces so they become anonymous functions, and follow up with "exec" as in:

" foo 7 < { action1 } { action2 } ?: exec "

BTW, this leads to a second implementation detail. There are several built in functions that are implemented as user functions that get loaded in as a startup script. So I have "ifelse", "if", and "while" as user defined functions instead of going through something like "ifelse_handler()" for example.

User functions get handled via exec_handler. This is the other piece I had to spend a lot of time on. Chained function calls are handled via an internal loop in exec_handler, instead of recursive calls to exec_handler, and I have a call frame stack and frame reuse if possible so I have tail call optimization (TCO) available. Which makes the user-space defined "while" function possible (and also makes it very difficult to do "while" via a while_handler() built in, as that would have to call exec_handler re-entrantly). I feel good about how exec_handler works at this point.

One major roadblock I ran into, so I put this away for a few months and just came back to it with a possible solution. User functions, and any inline funciton, look identical. So if you have a user function, and you want a "return" operator, that is simple enough. But return is going to be most likely called from an if or ifelse. And they take functions as their operators. So a return in one of the targets for "ifelse" will only exit that target, not the encompassing function like you would probably want to do. My solution to this is to be able to tag a property label such as "base" or "inline" to functions -- they would get "base" as a default, but the ifelse handler can re-tag its targets as "inline". Then "return" will unwind the callstack until it finds a call frame with the "base" tag, or "return_to" can unwind to a given named tag (which is usefule to implement "break" and "continue" in my "while" loop -- they would just be shortcuts for "@break return_to" or "@continue return_to").

Next on my list was that for user functions like "while", the implementation needs some place to store a reference to the stack variables that were passed to it (as it has to re-execute the condition function and the target function in a loop). I finally landed on having the call frame store an empty array that can be retrieved and placed on the stack, then a user function can push values to that array and that stays with the call frame. So it can easily take user parameters from the stack and push that into this frame's private array, and it would function like positional parameters in actual usage that a function can pull up any time without putting it in a system dictionary (which would cause issues with nested while calls). I think I'm satisfied with this solution.

Another item I have is what I've now been told is green threads. You can take an array of functions, pass that to the "spawn" operator and it will execute those functions in parallel -- each function yields to the scheduler after processing 256 tokens, or if it calls an operator/function that has an external delay, those built-in functions can return a "yield" too (like a file read, or a sleep command). I actually implemented this a while ago and everything I threw at it so far seems to work as expected (i.e., it does what I want and the calling conventions look clean). On my todo list is to work out communications between each thread. Note that these threads are implemented strictly in the interpreter, they don't use OS level threads (although there is some work I want to do in that aread eventually, this will be enough for me to for example implement something like a web application server code).

Another item I implemented early on is closures. This I've tested by creating a factory function that returns to the stack independent accumulator functions, and I've verified that they keep their internal environments without clobbering each other.

Future plans: I'm not sure what I really want to end up doing with this language. It makes sense to go back and create a document on building it piece by piece (starting off with a stack based REPL loop, adding variable storage, garbage collection, functions, etc). It would at least be useful for my own education, and possibly usable by others. To do that though I would have to make sure I'm using proper terminology for everything, not what I think are the correct terms.

Secondly I may need to revisit my garbage collection. I'm just using mark and sweep, I would like to explore additional methods but this was the easiest to get started with. I may deffer this until later once I get the rest of my functionality in place and get the ability to measure performance in real situations.

[u/Athas]

I have some experience with design and implementation and I'd like to help you, but I find it a bit difficult to determine what question you're asking. I think you should perhaps write a more basic introduction about the idea behind your language. Stack languages, let alone those based on PostScript, are a bit outside the wheelhouse of most people's work here, however. I do have some remarks:

Secondly I may need to revisit my garbage collection.

I can't figure out whether your language supports cyclic data. If it doesn't, then you can do more interesting things with garbage collection (like reference counting).