diff options
| -rw-r--r-- | README.baremetal.md | 45 | ||||
| -rw-r--r-- | README.md | 73 | ||||
| -rw-r--r-- | linux/README.md | 27 | ||||
| -rw-r--r-- | linux/bootstrap/README.md | 4 | ||||
| -rw-r--r-- | linux/bootstrap/bootstrap.md (renamed from bootstrap.md) | 0 | ||||
| -rw-r--r-- | linux/subx_debugging.md (renamed from subx_debugging.md) | 1 | ||||
| -rw-r--r-- | subx_bare.md | 29 | ||||
| -rw-r--r-- | x86_approx.md | 5 |
8 files changed, 77 insertions, 107 deletions
diff --git a/README.baremetal.md b/README.baremetal.md deleted file mode 100644 index 5cdb75ed..00000000 --- a/README.baremetal.md +++ /dev/null @@ -1,45 +0,0 @@ -Some apps written in SubX and Mu. Where the rest of this repo relies on a few -Linux syscalls, the apps in this subdirectory interface directly with hardware. -We still need the top-level and apps to build them. - -I'd like to eventually test these programs on real hardware, and to that end -they are extremely parsimonious in the hardware they assume: - - 0. Lots (more than 640KB/1MB[1]) of RAM - 1. Pure-graphics video mode (1024x768 pixels) in 256-color mode. At 8x8 - pixels per grapheme, this will give us 160x128 graphemes. But it's still - an open question if it's reasonably widely supported by modern hardware. - If it isn't, I'll downsize. - 2. Keyboard. Just a partial US keyboard for now. Main qwerty zone only. No - number pad, no function keys, no ctrl/alt/meta/fn/super/capslck/numlck. - -That's it: - * No wifi, no networking - * No multitouch, no touchscreen, no mouse - * No graphics acceleration - * No virtual memory, no memory reclamation - -Just your processor, gigabytes of RAM[1], a moderately-sized monitor and a -keyboard. (The mouse should also be easy to provide.) - -We can't yet read from or write to disk, except for the initial load of the -program. Enabling access to lots of RAM gives up access to BIOS helpers for -the disk. - -These programs don't convert to formats like ELF that can load on other -operating systems. There's also currently no code/data segment separation, -just labels and bytes. I promise not to write self-modifying code. Security -and sandboxing is still an open question. - -Programs start executing at address 0x9400. See baremetal/boot.hex for -details. - -Mu programs always run all their automated tests first. `main` only runs if -there are no failing tests. See baremetal/mu-init.subx for details. - -So far the programs have only been tested in Qemu and Bochs emulators. - -[1] Though we might need to start thinking of [the PC memory map](https://wiki.osdev.org/Memory_Map_(x86)) -as our programs grow past the first 32MB of memory. Mu doesn't yet make any -attempt to understand how much RAM the underlying computer has. Also, writing -to random locations can damage hardware or corrupt storage devices. diff --git a/README.md b/README.md index 83115d14..ca2caf4e 100644 --- a/README.md +++ b/README.md @@ -10,18 +10,33 @@ Running the code you want to run, and nothing else. ```sh $ git clone https://github.com/akkartik/mu $ cd mu -$ ./translate_mu apps/ex2.mu # emit a.elf -$ ./a.elf # add 3 and 4 -$ echo $? -7 +$ ./translate life.mu # emit a bootable disk.img +$ qemu-system-i386 disk.img ``` +<img alt='screenshot of Game of Life running on the Mu computer' src='html/baremetal-life.png'> + +([Colorized sources.](http://akkartik.github.io/mu/html/baremetal/life.mu.html) +This is memory-safe code, and most statements map to a single instruction of +machine code.) + Rather than start from some syntax and introduce layers of translation to implement it, Mu starts from the processor's instruction set and tries to get to _some_ safe and clear syntax with as few layers of translation as possible. The emphasis is on internal consistency at any point in time rather than compatibility with the past. ([More details.](http://akkartik.name/akkartik-convivial-20200607.pdf)) +Tests are a key mechanism here for creating a computer that others can make +their own. I want to encourage a style of active and interactive reading with +Mu. If something doesn't make sense, try changing it and see what tests break. +Any breaking change should break some well-named test somewhere. Consequently, +any manual test should be easy to turn into a reproducible automated test. Mu +is a testbed for providing this guarantee. It exposes testable interfaces for +hardware using dependency injection so that tests can run on -- and make +assertions against -- fake hardware. It also is an experiment in [automated +white-box testing](http://akkartik.name/post/tracing-tests) which promises +robust tests for performance, concurrency, fault-tolerance, etc. + Currently Mu requires a 32-bit x86 processor. ## Goals @@ -60,7 +75,7 @@ In priority order: ## Toolchain The Mu stack consists of: -- the Mu type-safe language; +- the Mu type-safe and memory-safe language; - SubX, an unsafe notation for a subset of x86 machine code; and - _bare_ SubX, a more rudimentary form of SubX without certain syntax sugar. @@ -72,38 +87,31 @@ emulator for Mu's supported subset of x86, that's useful for [debugging SubX programs](subx_debugging.md). Mu programs build natively either on Linux or on Windows using [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10). -For Macs and other Unix-like systems, use the emulator: +For Macs and other Unix-like systems, use the (much slower) emulator: ```sh -$ ./translate_mu_emulated apps/ex2.mu # ~2 mins to emit a.elf -$ ./bootstrap run ./a.elf # run in the emulator -$ echo $? +$ ./translate_mu_emulated ex2.mu # ~2 mins to emit disk.img ``` Mu programs can be written for two very different environments: -* With just a Linux kernel. This is the environment that Mu bootstraps itself - into, and it's the option for programming with stdin, stdout and file - descriptors. - -* Without an OS, by interacting directly with the screen and keyboard. This - is the option for rudimentary pixel graphics. There's currently no mouse, no - hardware acceleration, no virtual memory, no process separation, no multi-tasking, - no persistent storage, no network. These programs have not yet been tested - on native hardware, only on on Qemu and Bochs. But these _baremetal_ - programs build from scratch, without any reliance on C. This is the future. - Here is Conway's Game of Life on Mu: - - ```sh - $ ./translate_mu_baremetal baremetal/life.mu # emit disk.img - $ qemu-system-i386 disk.img - ``` +* At the top-level, Mu programs emit a bootable image that runs without an OS + (under emulation; I haven't tested on native hardware yet). There's just a + screen and a keyboard, and that's it. No mouse, no hardware acceleration, no + virtual memory, no process separation, no multi-tasking, no persistent + storage, no network. - <img alt='screenshot of Game of Life running on Mu without any intervening Operating System' src='html/baremetal-life.png'> +* The top-level is built using tools created under the linux/ sub-directory. + This sub-directory contains an entirely separate set of standard libraries + intended for building programs that run with just a Linux kernel, reading + from stdin and writing to stdout. The Mu compiler is such a program, at + linux/mu.subx. -Use `translate_mu` to build programs for Linux, and `translate_mu_baremetal` -for running without Linux. The standard libraries are totally separate for the -two options, so programs for one won't run on the other. +While I currently focus on programs without an OS, the `linux/` sub-directory +is fairly ergonomic. There's a couple of dozen example programs to try out +there. It is likely to be the option for a network stack in the foreseeable +future; I have no idea how to write to disk or interact on the network without +Linux. ## Syntax @@ -173,15 +181,16 @@ If you're still reading, here are some more things to check out: $ ./a.elf screen ``` -- [How to get your text editor set up for Mu and SubX programs.](editor.md) +- [How to get your text editor set up for Mu and SubX programs.](editor/editor.md) - [Some tips for debugging SubX programs.](subx_debugging.md) - [Shared vocabulary of data types and functions shared by Mu programs.](vocabulary.md) Mu programs can transparently call low-level functions written in SubX. -- [A summary](mu_instructions) of how the Mu compiler translates instructions - to SubX. ([colorized version](http://akkartik.github.io/mu/html/mu_instructions.html)) +- [A summary](mu_instructions) of how the Mu compiler translates statements + to SubX. Most Mu statements map to a single x86 instruction. + ([colorized version](http://akkartik.github.io/mu/html/mu_instructions.html)) - [Some starter exercises for learning SubX](https://github.com/akkartik/mu/pulls) (labelled `hello`). Feel free to [ping me](mailto:ak@akkartik.com) with any questions. diff --git a/linux/README.md b/linux/README.md index 23b61058..5eabd571 100644 --- a/linux/README.md +++ b/linux/README.md @@ -1,7 +1,8 @@ +A set of standard libraries for building programs that run with just a Linux +kernel. Most programs here read from stdin and write to stdout. One of these +programs is the Mu compiler ([colorized sources](http://akkartik.github.io/mu/html/apps/mu.subx.html)). - - -Some apps written in SubX and Mu. Check out: +Other apps beyond the Mu toolchain: * `tile`: [An experimental live-updating postfix shell environment](https://mastodon.social/@akkartik/105108305362341204) that updates as you type. Prototype. Look at this to see what is currently @@ -17,15 +18,13 @@ Some apps written in SubX and Mu. Check out: * `factorial*`: A simple program to compute factorials in 5 versions, showing all the different syntax sugars and what they expand to. -* Code unique to phases of our build toolchain: - * Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests` - * Syntax sugar for SubX: `sigils`, `calls`, `braces` - * More ambitious translator for a memory-safe language (in progress): `mu` - -* Miscellaneous test programs. +The Mu toolchain is also here in the following phases: +* Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests` +* Syntax sugar for SubX: `sigils`, `calls`, `braces` +* More ambitious translator for a memory-safe language (in progress): `mu` -All SubX apps include binaries. At any commit, an example's binary should be -identical bit for bit with the result of translating the corresponding `.subx` -file. The binary should also be natively runnable on a Linux system running on -Intel x86 processors, either 32- or 64-bit. If either of these invariants is -violated, it's a bug. +The toolchain includes binaries in the repo. At any commit, the binary should +be identical bit for bit with the result of translating the corresponding +`.subx` file. The binary should also be natively runnable on a Linux system +running on Intel x86 processors, either 32- or 64-bit. If either of these +invariants is violated, it's a bug. diff --git a/linux/bootstrap/README.md b/linux/bootstrap/README.md index fdc3213a..ca4ea4b3 100644 --- a/linux/bootstrap/README.md +++ b/linux/bootstrap/README.md @@ -4,4 +4,6 @@ a) An emulator for SubX, the subset of the 32-bit x86 instruction set used by Mu. b) A second translator for SubX programs that emits identical binaries to the -self-hosting versions in the parent directory. +self-hosting versions in the parent directory. Having two diverse compilers +(one in a familiar language, one with minimal syscall surface area) that emit +identical binaries should help gain confidence in Mu. diff --git a/bootstrap.md b/linux/bootstrap/bootstrap.md index ca9320e5..ca9320e5 100644 --- a/bootstrap.md +++ b/linux/bootstrap/bootstrap.md diff --git a/subx_debugging.md b/linux/subx_debugging.md index acd6f88d..e179df54 100644 --- a/subx_debugging.md +++ b/linux/subx_debugging.md @@ -27,6 +27,7 @@ rudimentary but hopefully still workable toolkit: mode (`bootstrap run`): ``` + $ cd linux $ ./translate_subx_debug file1.subx file2.subx ... # generating a.elf $ ./bootstrap --trace run a.elf arg1 arg2 saving trace to 'last_run' diff --git a/subx_bare.md b/subx_bare.md index 535b14fd..f8f00864 100644 --- a/subx_bare.md +++ b/subx_bare.md @@ -107,8 +107,9 @@ reader's burden. Here's the order I've been using after opcodes: Try running this example now: ```sh -$ ./bootstrap translate init.linux apps/ex3.subx -o apps/ex3 -$ ./bootstrap run apps/ex3 +$ cd linux +$ bootstrap/bootstrap translate 000init.subx ex3.subx -o ex3 +$ bootstrap/bootstrap run ex3 $ echo $? 55 ``` @@ -116,7 +117,8 @@ $ echo $? If you're on Linux you can also run it natively: ```sh -$ ./apps/ex3 +$ chmod +x ex3 +$ ./ex3 $ echo $? 55 ``` @@ -127,21 +129,22 @@ low-level SubX programs. ## Translating SubX programs This repo includes two translators for bare SubX. The first is [the bootstrap -translator](bootstrap.md) implemented in C++. In addition, you can use SubX to -translate itself. For example, running natively on Linux: +translator](bootstrap/bootstrap.md) implemented in C++. In addition, you can +use SubX to translate itself. For example, running natively on Linux: ```sh # generate translator phases using the C++ translator -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/hex.subx -o hex -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/survey_elf.subx -o survey_elf -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/pack.subx -o pack -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/assort.subx -o assort -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/dquotes.subx -o dquotes -$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/tests.subx -o tests +$ cd linux +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx hex.subx -o hex +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx survey_elf.subx -o survey_elf +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx pack.subx -o pack +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx assort.subx -o assort +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx dquotes.subx -o dquotes +$ bootstrap/bootstrap translate [01]*.subx subx-params.subx tests.subx -o tests $ chmod +x hex survey_elf pack assort dquotes tests # use the generated translator phases to translate SubX programs -$ cat init.linux apps/ex1.subx |./tests |./dquotes |./assort |./pack |./survey_elf |./hex > a.elf +$ cat 000init.subx ex1.subx |./tests |./dquotes |./assort |./pack |./survey_elf |./hex > a.elf $ chmod +x a.elf $ ./a.elf $ echo $? @@ -158,7 +161,7 @@ Or, running in a VM on other platforms (much slower): ```sh $ ./translate_subx_emulated init.linux apps/ex1.subx # generates identical a.elf to above -$ ./bootstrap run a.elf +$ bootstrap/bootstrap run a.elf $ echo $? 42 ``` diff --git a/x86_approx.md b/x86_approx.md index a0512d60..059c5e6b 100644 --- a/x86_approx.md +++ b/x86_approx.md @@ -21,8 +21,9 @@ Here's a test Mu program that prints out the bits for 0.5: It gives different results when emulated and run natively: ``` - $ ./translate_mu_debug x.mu # debug mode = error checking - $ ./bootstrap run a.elf + $ cd linux + $ ./translate_debug x.mu # debug mode = error checking + $ bootstrap/bootstrap run a.elf 0x3f000000 # correct $ ./a.elf 0x3efff000 # wrong |