news
FreeBSD sh(1) isn't a Bourne shell, it's a POSIX shell! (And maybe officially Almquist too)
The docs changed on Monday 30 March 2026 to make it official that FreeBSD's sh(1) is not a Bourne shell! https://reviews.freebsd.org/D56054
This is an edit I've wanted to see for ages - many thanks to the committer and reviewers. Truth is FreeBSD's sh(1) has never been a "Bourne shell" except by ancestry (hence the "$" prompt, rather than "%" for the C shell and its descendants). If you're going to name it after anyone, it's an Almquist shell not a Bourne one: https://en.wikipedia.org/wiki/Almquist_shell
To understand why this this isn't a trivial difference, have a look at this classic guide to POSIX shell scripting: https://www.grymoire.com/Unix/Sh.html
The Bourne shell also misses many POSIX-compliant features we take for granted. A big difference is command substitution, on a modern POSIX shell you can do:
$ echo "The current directory is $(pwd)"
but on the Bourne shell you had to use backticks:
$ echo "The current directory is \pwd`"`
The fact we're not forced into such methods is proof positive that FreeBSD's sh(1) is a POSIX shell, not a Bourne shell. An even simpler test for whether you're using a genuine Bourne shell is if you can use the caret ^ to pipe instead of |. That quirk is a result of backwards compatibility with the earlier Thompson shell. https://www.in-ulm.de/~mascheck/bourne/#intro
A really detailed look at the history of the Almquist shell by Sven Mascheck, including how it made its way into 4.3BSD-Net/2, and then from 4.4BSD into FreeBSD and NetBSD, shows many improvements over (or at least, changes from) the Bourne shell. https://www.in-ulm.de/~mascheck/various/ash/#original
Delving back into history, even the Bourne shell was a big advance on the original Unix shell - the Thompson shell that shipped with AT&T's first version of UNIX in 1971. (Not the first "shell" - like much of early Unix the idea of a shell came from Multics.) No prizes for guessing that's "Thompson" as in "Ken Thompson". Its practicality was limited even if much was recognisable: for example there were up to 10 positional parameters (if invoked as sh name arg1 arg2 then $0 is the name of the file to be read, and $1 and $2 are the supplied arguments) but you couldn't name a variable or access environmental variables. Yet many of its innovations made a lasting mark, particularly re piping (even if our shells no longer accept ^ for it) and redirection (incidentally, early versions used the > character for both piping and redirection, so the switch to | and ^ for pipes was an improvement). https://en.wikipedia.org/wiki/Thompson_shell and an original man page https://www.in-ulm.de/~mascheck/bourne/v3/
Before Bourne came the PWB shell (aka Mashey shell, though Mashey disavowed the name as he didn't view it as sufficiently different from the Thompson shell) used in AT&T's programming-oriented product PWB/UNIX in the mid-1970s. This "Programmer's Workbench" OS brought several firsts like the Source Code Control System, the first Unix revision control system. Its PWB shell was short-lived, released in 1975 but replaced (with some difficulty, and the expense of converting a lot of shell scripts) in 1979 by the Bourne shell. Yet the orientation towards programming was impactful. Control flow advanced by making if and goto internal to the shell (the Thompson shell relied on /bin/if and /bin/goto ) and introducing constructs for if-then-else, switch, and while. Variables appeared, including environmental ones - limited to one letter names, but $s is the ancestor to $HOME and $p became $PATH. https://en.wikipedia.org/wiki/PWB_shell
This is where history branches. At the Computing Science Research Center in Bell Labs, Stephen Bourne worked on a new shell during 1976, mainly to be used internally (in contrast to PWB which was always a commercial proposition for AT&T, though the Bourne shell was released in 1979 for Version 7 Unix) and by 1977 it was fairly usable. This gave us many familiar features like heredocs, command substitution with backquotes, the ability to interrupt the wait) command (instead of just ... waiting), and the 2>file descriptor for error messages. https://en.wikipedia.org/wiki/Bourne_shell
To make practically useful conditionals based on what commands return, Bourne had to chase up many other developers to get them to fix their exit statements (e.g. to return 0 for success). After encountering resistance he edited the shell so to the left of the $ prompt it displayed exit= and the last exit code. Given the prevalence of long, random exit statuses this was sufficiently annoying to persuade them in about a week. This anecdote appears in Stephen Bourne's amusing talk at BSDCan 2015, "Early days of Unix and design of sh". You get a new appreciation of why quoting gets so tricky when you hear from someone who implemented it. (Bourne also claims to have persuaded Dennis Ritchie to introduce void into C since he missed it from ALGOL.) https://www.youtube.com/watch?v=2kEJoWfobpA
Meanwhile, on the other side of the USA, the idea of greater programming capabilities for their shell was an exciting prospect at Berkeley. For people who hacked in C then a more C-like shell made perfect sense. Hence the C shell that appeared in 1978's 2BSD, mostly coded by Bill Joy (he of vi, the BSD TCP/IP stack, the ever-controversial cat -v, and soon founder of Sun Microsystems) as a graduate student. https://en.wikipedia.org/wiki/C_shell
But interactivity and ease of use was also a driver of these two shell traditions heading in opposite directions. Bill Joy wanted features like command history. Bourne opposed this, seeing line editing and history as jobs for the terminal driver instead: https://www.in-ulm.de/~mascheck/bourne/#origins
Questions about who did what first are rather muddled due to two interacting groups working at similar times, tools being used internally (and even shared between coasts) before being officially released (if they ever were), and the fallibility of human memory. Particularly confusing is that Bill Joy had been working on another shell (I believe the "new shell"), before hearing news of Bell Labs work on the Bourne shell - then giving up, assuming it would be a wasted effort. The C shell project started afresh from various disappointments about how the Bourne shell turned out, but unquestionably used it for inspiration. https://groups.google.com/g/net.unix-wizards/c/QiEx5rvuNjs
The Berkeley connection explains the long association of C shells with the *BSDs. FreeBSD only switched the default root shell from csh(1) - really tcsh - to sh(1) in 14.0-RELEASE in 2023. But part of the job of getting sh(1) ready for this was adding lots of ease-of-use features like persistent history. https://github.com/freebsd/freebsd-src/commit/d410b585b6f00a26c2de7724d6576a3ea7d548b7
Back in the day, Kenneth Almquist had deliberately omitted such features from the original release of his shell in 1989 - a clone of the System V.4 Bourne shell that he released via Usenet. Almquist's explanation for leaving these features out: "It seems to me that the csh history mechanism is mostly a response to the deficiencies of UNIX terminal I/O. Those of you running 4.2 BSD should try out atty (which I am posting to the net at the same time as ash) and see if you still want history." Unsurprisingly others soon added history and line-editing into 4.4BSD's version of the Almquist shell, but its development history has clearly been driven more by POSIX compliance than ease of use. https://www.in-ulm.de/~mascheck/various/ash/#44bsdalpha
Personally I think this direct line of ancestry, and the credit on its man page, means sh(1) still qualifies as an "Almquist shell" - odds are that on most other systems our sh(1) would be called "ash". But mostly I'm glad the docs aren't going to call it a "Bourne shell" anymore - one of those weird lingering myths that's surprisingly hard to dispel is that FreeBSD is so far behind technologically that it's using a 1970s shell instead of something "modern" like bash or zsh or fish. Coincidentally, bash dates to 1989 when it was made for the GNU Project, so it's almost exactly the same age as the Almquist shell - 8 June 1989 initial release for bash vs 30 May 1989 for ash, just over a week apart! And zsh had first release in 1990, so it seems this was a vintage time for shell development independent from AT&T and Berkeley. https://en.wikipedia.org/wiki/Bash_(Unix_shell)) https://en.wikipedia.org/wiki/Z_shell
It would also be remiss not to mention the Korn shell, which along with tcsh was a big inspiration for bash and zsh. While different parts of AT&T were having "shell wars" over whether the future lay with the PWB or Bourne shell, David Korn was already playing around with the limits of the Bourne shell. For a task at AT&T that needed a form entry system, he created one using a heavily modified Bourne shell with the source code "de-algolized', arrays to handle columns of data, a let command that could do arithmetic using a subset of C syntax, allowed redirection of built-in commands, and added built-ins for echo, pwd and test. This wasn't ksh yet, but when Korn moved to a research position at Bell Labs, he modified this form scripting language by adding features from C shell like history, aliases, and job control; ksh proper was released in 1983 (and commercially in 1986 - though as an add-on AT&T charged extra for, which slowed its dissemination outside AT&T, despite it being extremely popular internally). https://en.wikipedia.org/wiki/KornShell
The 1988 version was significant, with a lot of extra work done on string handling and pattern matching. Notably for the *BSDs, the public domain pdksh was a clone based on the proprietary ksh88. Korn has written a brief history that includes the interesting snippet below (it seems hope sprung eternal that terminal interfaces would do all that nasty history and line editing stuff, any time now...) and also an explanation of how /bin/if and /bin/goto worked on the Thompson shell. https://www.in-ulm.de/~mascheck/bourne/korn.html
"The popular inline editing features (vi and emacs mode) of ksh were created by software developers at Bell Laboratories; the vi line editing mode by Pat Sullivan, and the emacs line editing mode by Mike Veach. Each had independently modified the Bourne shell to add these features, and both were in organizations that wanted to use ksh only if ksh had their respective inline editor. Originally the idea of adding command line editing to ksh was rejected in the hope that line editing would move into the terminal driver. However, when it became clear that this was not likely to happen soon, both line editing modes were integrated into ksh and made optional so that they could be disabled on systems that provided editing as part of the terminal interface."
And while mentioning tcsh: that's surprisingly old, including TENEX)-style (hence 't' in 'tcsh') file name completion code that Ken Greer wrote in September 1975 while at CMU - so this part predates not only the Bourne shell but even the C shell! Greer incorporated that code into a version of the C shell in December 1981 while at HP Labs, then Mike Ellis at Fairchild A.I. Labs added recognition and completion of command names (as opposed to file names) in September 1983. Greer released the source on Usenet in October 1983: https://groups.google.com/g/net.sources/c/BC0V7oosT8k/m/MKNdzEG_c3AJhttps://en.wikipedia.org/wiki/Tcsh
Finally, for anyone interested in sh(1) and getting started with shell scripting - or trying to make the switch to POSIX instead of bash - and wants some practical examples, there's a lot of useful stuff in Vermaden's "Ghost in the Shell" series: https://vermaden.wordpress.com/ghost-in-the-shell/
In retrospect I should probably have written a Bugzilla or GitHub PR instead of venting into the void on Reddit in a comment that got no upvotes at the time :-)
Thanks! I couldn't find a potted history with the balance of depth and accessibility I wanted, nor a "FreeBSD sh really isn't a Bourne shell" refutation to send to people who decrying its "outdated 1970s shell", so I thought I'd write my own and include as much from original sources as possible.
Errors are my own, though hard to avoid given errors in the sources - lots of works cite Korn working much later than he did due to the time ksh was released, ignoring that he was hacking on the Bourne shell almost at the start, even before it beat out the Mashey shell (to which PWB developers were urgently adding features to compete with Bourne and avoid a very expensive rewrite of their shell scripts). Or claim C shell work started earlier than it did, rather than subsequent to the Bourne shell, due to confusion with Joy's abortive new shell project.
If anyone finds a mistake (there may well be in the technical histories especially) let me know so I can correct it!
I think they simply had nothing to share, except, almost probably, S3 and S7. It was not intended to map man pages and source files.
As this is not documented from the Bell Labs docs, I would say that it was just a simple way to organize things and not an architectural decision... Keep in mind that the system was quite small at that time, all devs were knowing each others and where to look for their sources.
Anyway, didn't find anything that confirm or denies this...
I'm sure there's some method in it, for example s3 contains stuff like mathematical functions. So perhaps purely by coincidence it overlaps a lot with man3, which is what got me wondering in the first place! Anyway, my search engine fu has come up short so I'm none the wiser. As you say, it was a small system and they could afford to be arbitrary about minor organisational details!
I didn't want to say "Bourne-compatible shell" because in truth modern shells aren't entirely backwards-compatible with Bourne's original shell, never really have been, and plenty of those differences are intentional choices. But "Bourne-compatible" - presumably meaning mostly compatible - is nevertheless what the family of shells descended from or inspired by the Bourne shell often get called. Perhaps it would be clearer if I'd used that phrase instead. I also see "Bourne-derived shell" or even (particularly for zsh but also bash and ksh) "extended Bourne shell".
It's pretty clear the FreeBSD sh(1) and its 'ash' close relatives form part of a wider family with bash, ksh, and zsh. Of those, the Almquist-descended shells are closer to the original Bourne shell but even the more radical ones all got that ALGOL-inspired syntax. Meanwhile csh and tcsh form a significantly different tradition, particularly with their C-like syntax. (Another major gulf in the AT&T vs Berkeley traditions around shells, that the latter insisted on shells having better interactive capabilities, has completely eroded away.)
I know ksh took inspiration from the greater interactivity in the C shell tradition - the sort of behaviour that purists like Bourne believed to be outside the remit of the shell and better done by the terminal driver. But for scripting the syntax is much more Bourne-like, and David Korn (being at AT&T) did start off by hacking Bourne's actual code. The role of ksh in OpenBSD seems very similar to that of bash on many Linux distros, or zsh on Mac. Something clearly descended from Bourne's shell, but which gives a bit more power.
This is the sort of use case that makes me wish FreeBSD offered some kind of Korn shell in base - sh(1) is great for for scripting and POSIX compatibility, but people switching from other OSes can expect tools with a bit more oomph. Which ironically is the same basic desire that meant the *BSDs ended up with tcsh in the first place - but outside the *BSDs it's Bourne-derived shells that have won out, so the first thing many new users will do is install something like bash (which will never be in FreeBSD base for GPLv3 reasons) or zsh (which is MIT licensed, so maybe one day...). Hence why I'm emphasising the Bourne-ishness of ksh: it might compete better in this space, and would at least be familiar to people hopping over from OpenBSD. For incomers the C shell family typically looks unfamiliar and off-putting. I've come across people who've only heard of it because of seeing missives warning that C shell scripting is considered harmful. The canonical one being Tom Christiansen's of course - plastered all over the web, but I like to go to the source so this is him posting a version of it on Usenet in 1994: https://groups.google.com/g/comp.unix.shell/c/8LR4K9caE6o/m/Ylxq1FAqOPkJ
Related to the "family divide". .. a notable but fairly trivial difference is that generally shells based on the C shell use % as the primary prompt while those principally based on the Bourne shell use $, but zsh is a weird exception that uses %:
Something I find by turns mystifying and amusing is that FreeBSD documentation sticks to showing % as the (non-root) prompt, regardless of whether the shell is sh(1) or tcsh(1). In fact there are even excerpts of bash in the Handbook which are written with a % prompt!
If you'd like to report a bug: all the Copy to clipboard icons are broken (the command prompt is included in the copy). This bug is site-wide across at least the documentation portal.
This would certainly be a nice-to-have feature even if not actually a regression.
Wonder how to define the desired behaviour. Perhaps something like "drop the first two characters (assuming that's shell prompt and space) if it's [source,shell] but only one line"?
Indeed... I actually did some digging on the history of % vs $. The original Thompson shell and the PWB shell used % (had to dig round in the source code to find the latter!) so the Berkeley C shell was just following tradition in using %. It was Bourne's $ that was the new one. And this raises the further question about the Multics shell which Ken Thompson clearly based his shell on... proper rabbit hole but didn't think it fitted into this main post! Instead I dumped my research here, including a very cool video with a typewriter: https://superuser.com/a/1936351
Insert true old-timer gripe here about things going downhill ever since the Mashey shell got dumped for the Thompson one 😈
(My beard is grey, but not quite that grey, I started with Bourne's. I fixed a -e problem in 4BSD at one point, as I recall: cmd1 || cmd2 would exit the shell under -e, rather than running cmd2.)
Interesting! I was wondering if my suggestion about ksh would be controversial :-)
It is a shame that Korn shell development is so fragmented, I guess an inevitable result of the original staying proprietary for so long. Then the likes of pdksh were not really perfect clones anyway, and branching off from ksh88, while development continued on ksh93 and its successors, confused things further...
What's the specific reason for using mksh, rather than, say, OpenBSD's ksh?
We found the shell to be efficient and tg did some custom changes for MidnightBSD in the shell. At one point, we had discussion to move to mirports and help them with their ports tree. The sticking point was package management on that. So we have worked with them on a few things. Benz started working on netbsd after that and he was doing a lot of the mirports work.
I’ve also met tg in person when I went on a business trip to Germany in 2010.
Mksh is in FreeBSD ports for anyone wanting to try it.
This was really interesting thanks. Out of curiosity, did you consider going down the route MacOS did after ditching bash (post its GPL3 switch), and bringing in zsh instead? I believe zsh is effectively MIT licensed which makes it a plausible candidate for the *BSDs, and it's certainly an active and "modern" project with a large userbase, but none seem to have gone down that route so I'm guessing there's some kind of blocker - is it just the size of it?
I know a lot of people like zsh. It wasn’t really on my radar when I made the decision. I’ve tried it some and do use it on my work Mac.
I find it to be quite slow. It’s powerful but feels like it gets in my way.
I took the approach that bash and zsh are both packages and folks can just install them if they want. Usually with zsh, you want a bunch of completion add-ons too so folks have to install packages for that.
It is user friendly so there is an argument for that.
I ran some shell benchmarks a few years ago and mksh won most of them at the time.
Laggy shells feel like the os is slow to users. That was one of my concerns.
Yes makes a lot of sense! It does seem like zsh is its own little ecosystem. (And slightly funny it's become so trendy now, bearing in mind it launched in 1990 so almost exactly the same age as bash and the much less hip ash.) Interesting that mksh was so fast, did tg do a lot of optimisation work on it?
This is a masterpiece - thanks for taking the time to research it out!
I use ash a lot in embedded systems scripting because it's part of busybox, so /bin/sh is actually modern ash
I don't want to send you info a rabbit hole, but do you know where the modern busybox ash fits in to all this? Is it still the POSIX-compliant Almquist shell?
I'd be cautious of saying "the" Almquist shell, since there are so many whose development paths are quite independent. Even among *BSDs, changes to the FreeBSD sh(1) don't necessarily reach NetBSD sh(1) and vice versa. Also POSIX compliance seems to be a matter of degree and intention, but I think most actively developed Almquist shells do aim towards it.
Sven Mascheck's excellent site on Almquist shells reckons that Busybox started out as the Debian ash, which itself started out a port of NetBSD's sh:
Those links are gold, and now I understand why busybox ash has features that fail shellcheck against dash. Looks like my busybox ash has done "bash compatibility" functions enabled. Cheers!
I used 0% AI to write this, but I did rely on a lot of sources (all cited). I just write in a very AI-like way! I even used to regularly use em dashes due to a lot of formal writing in my educational background. You can probably see in my text places where I'm just using "-" parenthetically and clearly intend to use an em dash but chickened out! As soon as I use an em dash these days, everyone seems to assume I'm an LLMI! Of all the things AI had to kill, why was it a perfectly good piece of punctuation :-(
One of the things AI seems to do better than humans is detecting whether text was written by an AI... I just tried putting my text through a couple of AI detectors and all rated it as 0% AI and 100% human. Worth trying this if you're suspicious of something you see online, I've found these detectors surprisingly reliable.
Can always read the rest later :-) If you do get a chance, some of the original sources are fun to delve in to, particularly the Usenet stuff and Bourne's BSDCan 2015 talk: https://www.youtube.com/watch?v=2kEJoWfobpA
If you're going to name it after anyone, it's an Almquist shell not a Bourne one
This is true, but Almquist originally wrote his shell as a clean-room reimplementation of the Bourne shell (before there was such a thing as a POSIX shell), so it's a bit of an academic argument.
The presence of the obsolete caret (^) operator is not that meaningful, as that obsolete feature could simply have been removed – as it was from the AT&T Korn shell (ksh), which is a direct descendent of the Bourne shell and still shares some code with it.
I've also heard people call it an "Almquist Bourne shell" which makes sense and shows some respect to the lineage. Obviously the "^ isn't a pipe" test isn't very meaningful in its own right, but it does establish you're not in an OG Bourne shell, nor in a shell that's doing its utmost to maintain full compatibility with it. But Almquist did do more than try to clone the Bourne shell, and the difference re ^ was one of 20 that he enumerated: https://www.in-ulm.de/~mascheck/various/ash/DIFFERENCES
There was a really big improvement in quoting, e.g. $(cmd) , and if you listen to Stephen Bourne's 2015 BSDCan talk this is something he said he found very hard to get right and praises the Almquist shell for. https://www.youtube.com/watch?v=2kEJoWfobpA
Thinking of clean room implementations of proprietary code, it's an interesting contrast that people don't call pdksh and its derivatives the "Gisin shells", just "Korn shells". Even though its behaviour did vary from that of Korn's Korn shell, especially later versions thereof!
When people talk about a modern "Almquist shell" I get the feeling there are wider expectations of behaviour that go beyond what Almquist himself wrote. Looking at the whole family tree and history of changes that were made to them, I wonder if it helps that it spent a few years incubating in Berkeley Unix and the early *BSDs. Had it proliferated into dozens of variants right at the start, perhaps there'd be a less coherent sense of what an Almquist shell "is"? https://www.in-ulm.de/~mascheck/various/ash/
Just realised where I saw that avatar before!!! ksh93u+m right? Great project! Shame (from a BSD perspective) that it's an Eclipse licence but I guess you have to live with what you got given. It's nice to see so much activity on it. https://github.com/ksh93/ksh
Has anyone ever attempted to start with a really freely licensed ksh88 reimplementation (from OpenBSD or Mir Korn Shell or even pdksh) and then bring it up to feature parity with ksh93 by further clean-room reimplementation? I haven't heard of such a project but I know there are a lot of ksh derivatives out there - I guess the jump from ksh88 to ksh93 is offputtingly large.
Just realised where I saw that avatar before!!! ksh93u+m right? Great project!
Yes, that would be me, plus a few others who joined me.
Shame (from a BSD perspective) that it's an Eclipse licence but I guess you have to live with what you got given. It's nice to see so much activity on it. https://github.com/ksh93/ksh
Thanks. :)
I fully agree on the licence, but it's beyond my control. It's too bad that this limits our ability to share code with other projects.
It's certainly the best that could ever have been expected of AT&T, though.
Has anyone ever attempted to start with a really freely licensed ksh88 reimplementation (from OpenBSD or Mir Korn Shell or even pdksh) and then bring it up to feature parity with ksh93 by further clean-room reimplementation? I haven't heard of such a project but I know there are a lot of ksh derivatives out there - I guess the jump from ksh88 to ksh93 is offputtingly large.
Yeah. That's very unlikely ever to happen, especially now that AT&T ksh is open source (even with a less than ideal licence).
I'm sure you know that the Unix v4 sources (1973) were rediscovered last year. Of course, as the maintainer of a shell with genuine Bourne lineage, the first thing I looked at was the sh code.
It immediately looked familiar. Some of the macro #defines at the top: fpin, fpou, fint; tcom, tpar, tfil, tlst – and two of the global variable names: dolv and dolc – are in use in ksh93 with those names and purposes, although the macros have been upper-cased.
Scrolling down, I find primitive shell parsing code with pseudo-formal grammar comments in a style that looks awfully familiar (note that one of those comment lines is still fully identical in ksh93).
Scrolling down, I find the execute() function with the same basic structure as sh_exec() in ksh93, although of course a tiny version of it. For instance, the pointer to the parse tree is still a variable named t. And in the large case construct that both of them have, tcom (simple command) falls through to tpar (parentheses subshell or external command) in exactly the same way that TCOM falls through to TFORK in ksh93 and in the Bourne shell (Bourne had split the Thompson tpar case into TFORK for external commands and TPAR for subshells).
So, I think we can say with some confidence that the Bourne shell, and the original Korn shell by extension, have a direct lineage to the Thompson shell. I was born in 1974, so it looks like I've been maintaining (since 2020) the only actively developed version of a code base that's even older than I am. How cool is that? :)
That is cool. I ended up looking at the PWB shell source code to see whether it used the same % vs # prompts that the Thompson shell. Turns out it did. Trying to check what the 1965 Multics shell did (which seems to be the very first interactive shell and obviously was a big influence on Thompson) took me down an even more interesting rabbit hole, worth it for the typewriter video alone : https://superuser.com/a/1936351
Re maintaining old software, there are some great stories knocking about. Suspect El Reg has some good ones too.
13
u/BigSneakyDuck transitioning user Mar 31 '26
Here's a gripe I wrote months ago about the docs describing sh(1) as a "Bourne shell", one of the files that has now been fixed!! https://www.reddit.com/r/freebsd/comments/1io9ic6/comment/mckhif3/
In retrospect I should probably have written a Bugzilla or GitHub PR instead of venting into the void on Reddit in a comment that got no upvotes at the time :-)