diff options
Diffstat (limited to 'linux')
| -rw-r--r-- | linux/README.md | 27 | ||||
| -rw-r--r-- | linux/bootstrap/README.md | 4 | ||||
| -rw-r--r-- | linux/bootstrap/bootstrap.md | 17 | ||||
| -rw-r--r-- | linux/subx_debugging.md | 127 |
4 files changed, 160 insertions, 15 deletions
diff --git a/linux/README.md b/linux/README.md index 23b61058..5eabd571 100644 --- a/linux/README.md +++ b/linux/README.md @@ -1,7 +1,8 @@ +A set of standard libraries for building programs that run with just a Linux +kernel. Most programs here read from stdin and write to stdout. One of these +programs is the Mu compiler ([colorized sources](http://akkartik.github.io/mu/html/apps/mu.subx.html)). - - -Some apps written in SubX and Mu. Check out: +Other apps beyond the Mu toolchain: * `tile`: [An experimental live-updating postfix shell environment](https://mastodon.social/@akkartik/105108305362341204) that updates as you type. Prototype. Look at this to see what is currently @@ -17,15 +18,13 @@ Some apps written in SubX and Mu. Check out: * `factorial*`: A simple program to compute factorials in 5 versions, showing all the different syntax sugars and what they expand to. -* Code unique to phases of our build toolchain: - * Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests` - * Syntax sugar for SubX: `sigils`, `calls`, `braces` - * More ambitious translator for a memory-safe language (in progress): `mu` - -* Miscellaneous test programs. +The Mu toolchain is also here in the following phases: +* Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests` +* Syntax sugar for SubX: `sigils`, `calls`, `braces` +* More ambitious translator for a memory-safe language (in progress): `mu` -All SubX apps include binaries. At any commit, an example's binary should be -identical bit for bit with the result of translating the corresponding `.subx` -file. The binary should also be natively runnable on a Linux system running on -Intel x86 processors, either 32- or 64-bit. If either of these invariants is -violated, it's a bug. +The toolchain includes binaries in the repo. At any commit, the binary should +be identical bit for bit with the result of translating the corresponding +`.subx` file. The binary should also be natively runnable on a Linux system +running on Intel x86 processors, either 32- or 64-bit. If either of these +invariants is violated, it's a bug. diff --git a/linux/bootstrap/README.md b/linux/bootstrap/README.md index fdc3213a..ca4ea4b3 100644 --- a/linux/bootstrap/README.md +++ b/linux/bootstrap/README.md @@ -4,4 +4,6 @@ a) An emulator for SubX, the subset of the 32-bit x86 instruction set used by Mu. b) A second translator for SubX programs that emits identical binaries to the -self-hosting versions in the parent directory. +self-hosting versions in the parent directory. Having two diverse compilers +(one in a familiar language, one with minimal syscall surface area) that emit +identical binaries should help gain confidence in Mu. diff --git a/linux/bootstrap/bootstrap.md b/linux/bootstrap/bootstrap.md new file mode 100644 index 00000000..ca9320e5 --- /dev/null +++ b/linux/bootstrap/bootstrap.md @@ -0,0 +1,17 @@ +## Running + +`bootstrap` currently has the following sub-commands: + +- `bootstrap help`: some helpful documentation to have at your fingertips. + +- `bootstrap test`: runs all automated tests. + +- `bootstrap translate <input files> -o <output ELF binary>`: translates `.subx` + files into an executable ELF binary. + +- `bootstrap run <ELF binary> <args>`: simulates running the ELF binaries emitted + by `bootstrap translate`. Useful for testing and debugging. + + Remember, not all 32-bit Linux binaries are guaranteed to run. I'm not + building general infrastructure here for all of the x86 instruction set. + SubX is about programming with a small, regular subset of 32-bit x86. diff --git a/linux/subx_debugging.md b/linux/subx_debugging.md new file mode 100644 index 00000000..e179df54 --- /dev/null +++ b/linux/subx_debugging.md @@ -0,0 +1,127 @@ +## A few hints for debugging SubX programs + +Writing programs in SubX is surprisingly pleasant and addictive. Reading +programs is a work in progress, and hopefully the extensive unit tests help. +However, _debugging_ programs is where one really faces up to the low-level +nature of SubX. Even the smallest modifications need testing to make sure they +work. In my experience, there is no modification so small that I get it working +on the first attempt. And when it doesn't work, there are no clear error +messages. Machine code is too simple-minded for that. You can't use a debugger, +since SubX's simplistic ELF binaries contain no debugging information. So +debugging requires returning to basics and practicing with a new, more +rudimentary but hopefully still workable toolkit: + +- Start by nailing down a concrete set of steps for reproducibly obtaining the + error or erroneous behavior. + +- If possible, turn the steps into a failing test. It's not always possible, + but SubX's primary goal is to keep improving the variety of tests one can + write. + +- Start running the single failing test alone. This involves modifying the top + of the program (or the final `.subx` file passed in to `bootstrap translate`) by + replacing the call to `run-tests` with a call to the appropriate `test-` + function. + +- Generate a trace for the failing test while running your program in emulated + mode (`bootstrap run`): + + ``` + $ cd linux + $ ./translate_subx_debug file1.subx file2.subx ... # generating a.elf + $ ./bootstrap --trace run a.elf arg1 arg2 + saving trace to 'last_run' + ``` + + The ability to generate a trace is the essential reason for the existence of + `bootstrap run` mode. It gives far better visibility into program internals than + running natively. + + Here's a sample of the contents of `last_run`, with a few boxes highlighted: + + <img alt='trace example' src='html/trace.png'> + + Each of the green boxes shows the trace emitted for a single instruction. + It starts with a line of the form `run: inst: ___` followed by the opcode + for the instruction, the state of registers before the instruction executes, + and various other facts deduced during execution. Some instructions first + print a matching label. In the above screenshot, the red boxes show that + address `0x0900005e` maps to label `$loop` and presumably marks the start of + some loop. Function names get similar `run: == label` lines. + +- One quick trick when scanning a trace for the first time: + + ``` + $ grep label last_run + ``` + + This is useful for quickly showing you the control flow for the run, and the + function executing when the error occurred. I find it useful to start with + this information, only looking at the complete trace after I've gotten + oriented on the control flow. Did it get to the loop I just modified? How + many times did it go through the loop? + +- Once you have SubX displaying labels in traces, it's a short step to modify + the program to insert more labels just to gain more insight. For example, + consider the following function: + + <img alt='control example -- before' src='html/control0.png'> + + This function contains a series of jump instructions. If a trace shows + `is-hex-lowercase-byte?` being encountered, and then `$is-hex-lowercase-byte?:end` + being encountered, it's still ambiguous what happened. Did we hit an early + exit, or did we execute all the way through? To clarify this, add temporary + labels after each jump: + + <img alt='control example -- after' src='html/control1.png'> + + Now the trace should have a lot more detail on which of these labels was + reached, and precisely when the exit was taken. + +- If you find yourself wondering, "when did the contents of this memory + address change?", `bootstrap run` has some rudimentary support for _watch + points_. Just insert a label starting with `$watch-` before an instruction + that writes to the address, and its value will start getting dumped to the + trace after every instruction thereafter. + +- Once we have a sense for precisely which instructions we want to look at, + it's time to look at the trace as a whole. Key is the state of registers + before each instruction. If a function is receiving bad arguments it becomes + natural to inspect what values were pushed on the stack before calling it, + tracing back further from there, and so on. + + I occasionally want to see the precise state of the stack segment, in which + case I uncomment a commented-out call to `dump_stack()` in the `vm.cc` + layer. It makes the trace a lot more verbose and a lot less dense, necessitating + a lot more scrolling around, so I keep it turned off most of the time. + +- If the trace seems overwhelming, try [browsing it](https://github.com/akkartik/mu/blob/master/tools/browse_trace.readme.md) + in the 'time-travel debugger'. + +- Don't be afraid to slice and dice the trace using Unix tools. For example, + say you have a SubX binary that dies while running tests. You can see what + test it's segfaulting at by compiling it with debug information using + `./translate_subx_debug`, and then running: + + ``` + grep 'label test-' |tail + ``` + + Just read out the last test printed out before the segfault. + + Even outside of tests, I can often quickly debug an error just by scanning + the end of a trace for labels: + + ``` + $ grep label last_run |tail + ``` + + Knowing _where_ the error occurred is often enough to put me on the right + track to debugging an error. + +Hopefully these hints are enough to get you started. The main thing to +remember is to not be afraid of modifying the sources. A good debugging +session gets into a nice rhythm of generating a trace, staring at it for a +while, modifying the sources, regenerating the trace, and so on. Email +[me](mailto:mu@akkartik.com) if you'd like another pair of eyes to stare at a +trace, or if you have questions or complaints. |