The Joy Of Limitations: Writing an ARexx REPL in ARexx
Making things while making do
Autumn 2021
If someone gives you two gum wrappers and three sticks and tells you to make the Pietà with this, you will take two gum wrappers and three sticks as far as they can possibly go.
- Mark Ferrari on creating EGA art for Loom and Monkey Island
An unsurprising rant about old hardware
I like limited computers. Of course all computers are limited in some sense: there's finite amounts of memory, disk space and screen real estate on all machines. When I'm talking limited, I usually mean home computers made during the 1990s and 1980s, machines that are now far surpassed by even the cheapest system on a chip.
There are several reasons for this. One is familiarity: they're the type of machines I started my computing career on. Having used them and the software they run for three decades means that sitting down in front of one feels like coming home.
Another one is scale: simpler machines are, well, simpler. When resources are scarce, there's less room for overengineered bloat, telemetry collection, "trusted computing" and other frivolities. More importantly, simple things are easier to understand, learn and remember.
The creativity of limits
The most important one, however, is creativity. I can be fairly creative on any computer and I wouldn't want to do modern web development on an Amiga, but when it comes to the things I get intense joy (and sometimes pride) from creating, some platform limitations nearly always apply. Some of them are pure hardware limitations, such as when creating pixel graphics for the Commodore 64 or Amiga. Some are partly enforced by hardware, such as CPU speed and memory consumption. Others yet are silly limitations I impose on myself simply because I believe it to be a constructive exercise.
Limitations themselves won't magically conjure up creativity. There has to be some idea, some inspiration to start with - but I do think that limitations, over time, will help the creative process if handled correctly. Necessity may be the mother of all inventions, but limitations are the, uh, nanny of problem solving skills. They force us to think differently, to apply our knowledge in unexpected ways and to seek new knowledge in the hope of overcoming them.
A limited attention span
Limitations can also help creativity - or at least productivity - by keeping distractions at a minimum. Even though the Amiga has a multitasking OS, my 14 MHz machine isn't something on which I'm likely to fire up a web browser or even launch a program that's not necessary to complete the task at hand. There's no doomscrolling, no notifications, no chat software and no Wikipedia rabbit hole to fall down. Even better, there's no Stackoverflow from which to nick a finished solution. If I want something done, I have to figure it out myself by thinking, experimenting and reading actual documentation. In short, staying focused is much easier on the Amiga than on my modern, interwebs-enabled PC.
A detailed example
The latest (somewhat artificial) limitation I've imposed on myself was conceived when writing Little Things That Made Amiga Great. Yammering on at length about how great the Amiga's operating system was (and still is!) made me wonder how would it be to program something meaningful with nothing but a standard installation of AmigaOS 3.1. After all, it does have all the tools needed: three text editors, a more than capable command line, various helpful utilities and, importantly, a competent scripting language called ARexx.
Like any popular scripting language, there are several third party libraries that extends its capabilities. The Amiga has several powerful text editors available, ranging from ports of Vim and GNU Emacs to native powerhouses such as CygnusEd and TurboText.
An Amiga 1200 has a 14 MHz 68EC020 CPU, but can be accelerated to impressive speeds using various expansion cards, both old and new. I've got a coveted Blizzard 1260, powered by Motorola's last 68k CPU, the superscalar 68060 with 8 KB data and instruction caches, running at a stonking 50 MHz.
The OS itself can be tweaked, improved and configured in minute detail using a number of third party programs and patches.
Of course, no such luxuries would be allowed. Both my machine and OS were thus kept in a state that could best be described as having sex in a Lehman Brothers office in 2008: Fucking close to the defaults.
Choosing a project
Lots of modern scripting languages come with a REPL, something I've become increasingly used to. They're unbeatable for quickly testing things out or just standing in as calculators when doing other programming. ARexx, being from 1987, doesn't come with one - a lack that seriously hampers my workflow. Even when not working with ARexx as such, for example when coding an intro in AMOS, it's a useful tool for everything from quick calculations and radix conversions to generating repetitive AMOS code, thanks to its excellent functions for string parsing. A REPL would certainly be handy in such situations.
When writing an ARexx REPL in ARexx, it helps that ARexx has the INTERPRET instruction, which is basically an equivalent of JavaScript's eval(). It's also got a number of other useful tools for introspection, which might not be expected considering its age. I've written a few really simple ARexx read-eval loops before, but this time I decided to create a true REPL that's as close to a modern counterpart as possible. I wanted it to seamlessly and automatically echo back variable assignments and function return values, handle multi-line input, report errors in a meaningful way, and so on.
Choosing an editor
AmigaOS comes with three text editors. Out of those, I opted for MEmacs. The main reason for this is that MEmacs opens its own screen. This means two things: One, I can switch between the editor and the shell without using the mouse - think of it as alt-tabbing between two maximized applications in Windows. Two, I can fit 80 columns of text on screen. Using Ed would mean running it in a window, which effectively cuts of a few characters. On a 640x256 display, screen real estate matters.
MEmacs proved perfectly capable for the task. It's not a very Amiga-ish program, which is a bit of a setback, but the main threshold proved to be the nonstandard keyboard shortcuts: turns out it isn't exactly Emacs-ish either, at least not by modern standards. Ctrl-C is quit, not Ctrl-X-C, which still confuses me sometimes. Even worse, Ctrl-X-S saves a file as expected, but Ctrl-X-F doesn't load a file - it performs a write-quit. It took a few ruined files to adapt to that particular fact.
Apart from this, once I got to grips with the basic commands for navigating, searching, copying, pasting and so on, the editing proved to be if not pleasant, then at least not frustrating. Considering it's from 1986, I'd say it's surprisingly usable even by modern standards.
In the end, I'm satisfied with my choice. I'm usually happy to help people out with their Amigas on retro gatherings and demo parties which oftentimes means editing various system script files. While Ed is fine for most tasks, knowing more about another standard editor is a certainly a good thing.
Amiga MicroEMACS
Refreshing my memory
Manuals are indispensable when developing. These days, the web is usually our manual and whatever info we might need is just a quick search away. Not so when doing purist retro development on an Amiga. If I had bought AmigaOS 3.1 as a stand-alone update kit or with an A4000, I would've gotten an ARexx manual. It sadly wasn't delivered with my A1200, so I had to cheat: I've bought an ARexx manual elsewhere and the make-believe scenario in which I'm working could be bent to allow its use.
It's a fine language reference, spiral bound so it can easily be left open on a particular page. The examples are perhaps a bit sparse at times, but all in all it offers a pleasant experience.
Apart from describing the language itself, it also contained some interesting information about ARexx. I was writing my REPL using an A1200 with several megabytes of memory and a hard drive. ARexx itself was apparently written on a 7 MHz Amiga 1000 with half a meg of RAM and two floppy disk drives. The same computer was used to typeset the ARexx manual using AmigaTEX. I guess there's room for an even more hardcore purist project in the future.
End results
Download the REPL, along with some other ARexx development tools, from here. Or just read the REPL source code here. It's got plenty of comments so following along should be easy. You might even pick up a bit of ARexx when doing so!
Example REPL session.
Lessons learned
Creating an ARexx REPL in ARexx means you can't really write a lexer and parser, because it'd be too slow. It also means the end user can alter and break the REPL and there's not much you can do about it. I still think I was fairly successful. My REPL understands multi-line statements, comments, correctly handles quoting, semicolon-separated statements - and it traps errors and lets the user continue with an untouched state. A lot of this comes for free because of how ARexx works, but some of it did require some work and I really did learn a lot of new things about ARexx.
For example when optimizing a (still rather slow) tokenization loop for handling quotes: I desperately scoured the manual for useful hints and unexpected tricks - and found one. Through the standard extension library, ARexx allows for memory management and pointer arithmetic. My hopes weren't high but I was willing to try anything. Alas, the overhead of doing it through ARexx ultimately proved slower than my initial approach - but learning about this feature gave me new and interesting ideas.
Writing the code in MEmacs made me more inclined than usual to keep the code less than 80 columns wide, and I learned about ARexx' line continuation behavior. Short summary: it's somewhat erratic.
Among other things, I also learned more about introspection and the performance of various string operations.
Working with an almost-default AmigaOS is something I'm used to, so that offered no significant threshold. I did allow myself a patch called NoClick, which silences the internal floppy drive by locking the drive head. Without it a constant, regular clicking sound is heard when the drive checks for a freshly inserted floppy disk. Other than that, my setup was based completely on Commodore's standard OS distribution.
Watch this space for more ARexx silliness. In the meantime, happy hacking!