Symbolics Genera Virtual Lisp Machine on macOS with Apple Silicon

✎︎
Note

This is a work in progress.

⚡︎
The patched emulator contains unfixed bugs.

In the worst case, you may lose your data or corrupt your OS/hardware. Backup your important data!

1The News

A decade later, I've got Open Genera running again — this time on the latest macOS with an M4 chip, rather than an aging x86_64 Linux box as in genera-install.

Teasing you with some images.

2The setup guide

2.1Preparation

You'll need a Mac with macOS, a compiler toolchain, and XQuartz installed. Download the source and run the usual ./configure && make.

With og2vlm you will be able to use the system NFSv3 service directly, and the image boots without the need for inetd. You can find the Genera 2.0 ISO image on the Wayback Machine, which holds the distribution tape. With the LMFS patches pre-applied, you can restore the tape onto the native Lisp Machine File System, so you won't need to load Lisp systems from the host file system via NFS every time.

2.2Setup your macOS

The og2vlm-setup-v4.0.md already has the most important instructions on how to enable NFS. However, you do not have to edit /etc/hosts on your macOS, because Open Genera does not need that file to connect to the host computer, and you'll later learn Genera resolves IP addresses using Namespace Objects. You will only need to edit the NFS /etc/exports file. After that, unpack og2vlm, remember to install the Genera fonts, then edit the example dot.VLM, and rename it to .VLM.

The vital difference is this line (the host= would let the Open Genera know the host IP):

genera.network: 192.168.2.2;mask=255.255.255.0;gateway=192.168.2.1;host=192.168.2.1

When you start the VLM with sudo (which is required for the vmnet.Framework network bridge to work), it will print a line that tells you the exact network address to match:

net #0 vmnet subnet 192.168.2.1 - 192.168.2.254 mask 255.255.255.0 (gateway = 192.168.2.1);
align Genera's static address with this range

If the printed address range doesn't match your .VLM configuration, stop the VLM and edit the configuration file first.

Meanwhile, create /etc/exports with something like:

/Users/ldbeth/Public/symbolics -alldirs -mapall=ldbeth -network 192.168.2.0 -mask 255.255.255.0

The key is that the address after -network need to match the subnet of vmnet. The rest is regular NFS setup. After NFS service is up, verify the UDP feature is enabled, i.e.

rpcinfo -p localhost | grep nfs

This should print:

    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs

2.3Get the manuals

At this point, you should get Open Genera running for the first time. I recommend downloading and reading the Genera manuals in PDF format. Although you can access most of the manuals inside Open Genera itself, switching back and forth between the Document Examiner and other Activities is awkward.

2.4Get the LMFS ready and apply your first patches

✎︎
Note

You can practice multiple times until you're familiar with the operations and get the desired results.

Basically you create a FEP disk image that would become your host drive. Although you will later access facilities that enables back up the file system to tape, the easiest way to backup is tar the FEP disk.

⚡︎
Warning

Before install a patch, you should compile the patch file first. This not only enhances the runtime efficiency, some patches to critical system components only works as expected when compiled.

2.4.1The GC bug

The modification to VLM introduced a problem that reordering memory during Optimize World could write wrong tags to memory. The solution is to load the full-gc-patch.lisp which fixes corrupted memory regions and reorder-mem-patch.lisp prevents GC introduces more corruptions (both compiled). After running full GC a couple of times, you can (zl:fset 'si:make-static-regions-dynamic #'ignore) to disable marking all regions dynamic every time.

2.4.2Enable Telnet

After login, you can enable Telnet by:

> Enable Services (disabled services [default All]) ALL
> (NET:REMOTE-LOGIN-ON)

Then you can telnet 192.168.2.2. Use ^] to enter telnet menu and set mode char. Then at the telnet window:

> Set Remote Terminal Options 
Console supports X3.64 display codes [default Yes]: Yes
Console has a meta key [default Yes]: Yes
Width in characters [default 140]: 140
Height in characters [default 49]: 49
More processing [default Yes]: Yes
Display a status line [default Yes]: Yes
  Frequency of status line updates [default 5 seconds]: 5 seconds

This would make life much easier when need to run copied Lisp code on Genera.

What about the Abort key?

Table 1Genera remote-terminal special keys (press Ctrl_ first)
Key Genera Key
H Help
E End
A Abort
S Suspend
R Resume
C Complete
I Clear-Input
X Escape
L Line
P Page
F Refresh
B Backspace
N Network
1 Square
2 Circle
3 Triangle

3The bug fixes explained

If you are interested in how the VLM has been ported to macOS, here is the explanation.

⚠︎
Not verified

This section is based on my observations of the fixes generated by LLMs (Claude Opus 4.8 and Fable 5). The fixes worked before I ever tried to fully understand them.

Besides, I have rather limited knowledge of OS implementation details, so either the LLM or I could be wrong.

3.1The emulator memory

According to the leaked top secret document Lisp-Machine Data Types, now available from bitsavers.org, the Mac Ivory and the emulated VLM use 32-bit words, and each word is tagged by an 8-bit tag that consists of a 2-bit cdr code and 6-bit type information, forming a 40-bit word. On the Ivory, the tag and the payload are probably ground together; on the VLM they are allocated on two different segments (or arrays, if you prefer simpler terminology).

I can't tell if the author of the VLM was being creative or just lazy. The two memory segments sit at hardcoded addresses high enough in the address space to be safely reserved with mmap(). The VLM then relies on SIGSEGV to learn when a segment is actually needed, at which point it creates the mapping on demand with mprotect().

Fable 5 decides that since the Arm64 macOS has 16k OS page size (Linux has 4k), while Ivory uses 8k for tag space page size. This means the GC barrier and stuffs need to be redesigned for

This design also creates a second problem when Clang is used to compile the instruction emulator. The emulator was translated from Alpha assembly to C by a Common Lisp program written with x86 assumptions in mind. Clang's optimizer for ARM64 has no way of knowing that an ordinary memory write can trigger a UNIX signal that transfers control to a specific C label, so it optimizes the write away — which then causes a memory access fault.

The solution to this is to use asm goto.

Glossary

Activities

They are the Applications.

Namespace Objects

In Genera, these are the records that store the configuration of servers and Lisp Machines.

Bibliography

OG2VLM: Open Genera 2 Virtual Lisp Machine emulator for Debian Linux. https://github.com/JMlisp/og2vlm/.

Running Open Genera 2.0 on Linux. . https://archives.loomcom.com/genera/genera-install.html.