7842 - new directory organization

Baremetal is now the default build target and therefore has its sources
at the top-level. Baremetal programs build using the phase-2 Mu toolchain
that requires a Linux kernel. This phase-2 codebase which used to be at
the top-level is now under the linux/ directory. Finally, the phase-2 toolchain,
while self-hosting, has a way to bootstrap from a C implementation, which
is now stored in linux/bootstrap. The bootstrap C implementation uses some
literate programming tools that are now in linux/bootstrap/tools.

So the whole thing has gotten inverted. Each directory should build one
artifact and include the main sources (along with standard library). Tools
used for building it are relegated to sub-directories, even though those
tools are often useful in their own right, and have had lots of interesting
programs written using them.

A couple of things have gotten dropped in this process:
  - I had old ways to run on just a Linux kernel, or with a Soso kernel.
    No more.
  - I had some old tooling for running a single test at the cursor. I haven't
    used that lately. Maybe I'll bring it back one day.

The reorg isn't done yet. Still to do:
  - redo documentation everywhere. All the README files, all other markdown,
    particularly vocabulary.md.
  - clean up how-to-run comments at the start of programs everywhere
  - rethink what to do with the html/ directory. Do we even want to keep
    supporting it?

In spite of these shortcomings, all the scripts at the top-level, linux/
and linux/bootstrap are working. The names of the scripts also feel reasonable.
This is a good milestone to take stock at.

1 files changed, 269 insertions, 0 deletions

diff --git a/linux/103kernel-string-equal.subx b/linux/103kernel-string-equal.subx
new file mode 100644
index 00000000..7e1b9d6f
--- /dev/null
+++ b/linux/103kernel-string-equal.subx
@@ -0,0 +1,269 @@
+# Checking null-terminated strings.
+#
+# By default we create strings as arrays of bytes, and all arrays have a 4-byte
+# size prefix.
+#
+# However, we sometimes need to deal with null-terminated strings when
+# interacting with the Linux kernel. This layer implements a function for
+# comparing a null-terminated 'kernel string' with a size-prefixed 'SubX
+# string'.
+#
+# To run (from the subx directory):
+#   $ ./bootstrap translate 05[0-2]*.subx -o /tmp/tmp52
+#   $ ./bootstrap run /tmp/tmp52  # runs a series of tests
+#   ......  # all tests pass
+#
+# (We can't yet run the tests when given a "test" commandline argument,
+# because checking for it would require the function being tested! Breakage
+# would cause tests to not run, rather than to fail as we'd like.)
+
+== code
+#   instruction                     effective address                                                   register    displacement    immediate
+# . op          subop               mod             rm32          base        index         scale       r32
+# . 1-3 bytes   3 bits              2 bits          3 bits        3 bits      3 bits        2 bits      2 bits      0/1/2/4 bytes   0/1/2/4 bytes
+
+Entry:  # run all tests
+    e8/call  run-tests/disp32  # 'run-tests' is a function created automatically by SubX. It calls all functions that start with 'test-'.
+    # syscall(exit, Num-test-failures)
+    8b/copy                         0/mod/indirect  5/rm32/.disp32            .             .           3/r32/ebx   Num-test-failures/disp32          # copy *Num-test-failures to ebx
+    e8/call  syscall_exit/disp32
+
+kernel-string-equal?:  # s: (addr kernel-string), benchmark: (addr array byte) -> result/eax: boolean
+    # pseudocode:
+    #   n = benchmark->size
+    #   s1 = s
+    #   s2 = benchmark->data
+    #   i = 0
+    #   while (i < n)
+    #     c1 = *s1
+    #     c2 = *s2
+    #     if (c1 == 0) return false
+    #     if (c1 != c2) return false
+    #     ++s1, ++s2, ++i
+    #   return *s1 == 0
+    #
+    # registers:
+    #   i: ecx
+    #   n: edx
+    #   s1: edi
+    #   s2: esi
+    #   c1: eax
+    #   c2: ebx
+    #
+    # . prologue
+    55/push-ebp
+    89/copy                         3/mod/direct    5/rm32/ebp    .           .             .           4/r32/esp   .               .                 # copy esp to ebp
+    # . save registers
+    51/push-ecx
+    52/push-edx
+    53/push-ebx
+    56/push-esi
+    57/push-edi
+    # var s1/edi: (addr byte) = s
+    8b/copy                         1/mod/*+disp8   5/rm32/ebp    .           .             .           7/r32/edi   8/disp8         .                 # copy *(ebp+8) to edi
+    # var n/edx: int = benchmark->size
+    8b/copy                         1/mod/*+disp8   5/rm32/ebp    .           .             .           2/r32/edx   0xc/disp8       .                 # copy *(ebp+12) to edx
+    8b/copy                         0/mod/indirect  2/rm32/edx    .           .             .           2/r32/edx   .               .                 # copy *edx to edx
+    # var s2/esi: (addr byte) = benchmark->data
+    8b/copy                         1/mod/*+disp8   5/rm32/ebp    .           .             .           6/r32/esi   0xc/disp8       .                 # copy *(ebp+12) to esi
+    81          0/subop/add         3/mod/direct    6/rm32/esi    .           .             .           .           .               4/imm32           # add to esi
+    # var i/ecx: int = 0
+    b9/copy-to-ecx  0/imm32/exit
+    # var c1/eax: byte = 0
+    b8/copy-to-eax  0/imm32
+    # var c2/ebx: byte = 0
+    bb/copy-to-ebx  0/imm32
+$kernel-string-equal?:loop:
+    # if (i >= n) break
+    39/compare                      3/mod/direct    1/rm32/ecx    .           .             .           2/r32/edx   .               .                 # compare ecx with edx
+    7d/jump-if->=  $kernel-string-equal?:break/disp8
+    # c1 = *s1
+    8a/copy-byte                    0/mod/indirect  7/rm32/edi    .           .             .           0/r32/AL    .               .                 # copy byte at *edi to AL
+    # c2 = *s2
+    8a/copy-byte                    0/mod/indirect  6/rm32/esi    .           .             .           3/r32/BL    .               .                 # copy byte at *esi to BL
+    # if (c1 == 0) return false
+    3d/compare-eax-and  0/imm32/null
+    74/jump-if-=  $kernel-string-equal?:false/disp8
+    # if (c1 != c2) return false
+    39/compare                      3/mod/direct    0/rm32/eax    .           .             .           3/r32/ebx   .               .                 # compare eax and ebx
+    75/jump-if-!=  $kernel-string-equal?:false/disp8
+    # ++i
+    41/increment-ecx
+    # ++s1
+    47/increment-edi
+    # ++s2
+    46/increment-esi
+    eb/jump  $kernel-string-equal?:loop/disp8
+$kernel-string-equal?:break:
+    # return *s1 == 0
+    8a/copy-byte                    0/mod/indirect  7/rm32/edi    .           .             .           0/r32/AL    .               .                 # copy byte at *edi to AL
+    3d/compare-eax-and  0/imm32/null
+    75/jump-if-!=  $kernel-string-equal?:false/disp8
+$kernel-string-equal?:true:
+    b8/copy-to-eax  1/imm32
+    eb/jump  $kernel-string-equal?:end/disp8
+$kernel-string-equal?:false:
+    b8/copy-to-eax  0/imm32
+$kernel-string-equal?:end:
+    # . restore registers
+    5f/pop-to-edi
+    5e/pop-to-esi
+    5b/pop-to-ebx
+    5a/pop-to-edx
+    59/pop-to-ecx
+    # . epilogue
+    89/copy                         3/mod/direct    4/rm32/esp    .           .             .           5/r32/ebp   .               .                 # copy ebp to esp
+    5d/pop-to-ebp
+    c3/return
+
+# - tests
+
+test-compare-null-kernel-string-with-empty-array:
+    # eax = kernel-string-equal?(Null-kernel-string, "")
+    # . . push args
+    68/push  ""/imm32
+    68/push  Null-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 1, msg)
+    # . . push args
+    68/push  "F - test-compare-null-kernel-string-with-empty-array"/imm32
+    68/push  1/imm32/true
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-null-kernel-string-with-non-empty-array:
+    # eax = kernel-string-equal?(Null-kernel-string, "Abc")
+    # . . push args
+    68/push  "Abc"/imm32
+    68/push  Null-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 0, msg)
+    # . . push args
+    68/push  "F - test-compare-null-kernel-string-with-non-empty-array"/imm32
+    68/push  0/imm32/false
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-kernel-string-with-equal-array:
+    # eax = kernel-string-equal?(_test-Abc-kernel-string, "Abc")
+    # . . push args
+    68/push  "Abc"/imm32
+    68/push  _test-Abc-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 1, msg)
+    # . . push args
+    68/push  "F - test-compare-kernel-string-with-equal-array"/imm32
+    68/push  1/imm32/true
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-kernel-string-with-inequal-array:
+    # eax = kernel-string-equal?(_test-Abc-kernel-string, "Adc")
+    # . . push args
+    68/push  "Adc"/imm32
+    68/push  _test-Abc-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 0, msg)
+    # . . push args
+    68/push  "F - test-compare-kernel-string-with-equal-array"/imm32
+    68/push  0/imm32/false
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-kernel-string-with-empty-array:
+    # eax = kernel-string-equal?(_test-Abc-kernel-string, "")
+    # . . push args
+    68/push  ""/imm32
+    68/push  _test-Abc-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 0, msg)
+    # . . push args
+    68/push  "F - test-compare-kernel-string-with-equal-array"/imm32
+    68/push  0/imm32/false
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-kernel-string-with-shorter-array:
+    # eax = kernel-string-equal?(_test-Abc-kernel-string, "Ab")
+    # . . push args
+    68/push  "Ab"/imm32
+    68/push  _test-Abc-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 0, msg)
+    # . . push args
+    68/push  "F - test-compare-kernel-string-with-shorter-array"/imm32
+    68/push  0/imm32/false
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+test-compare-kernel-string-with-longer-array:
+    # eax = kernel-string-equal?(_test-Abc-kernel-string, "Abcd")
+    # . . push args
+    68/push  "Abcd"/imm32
+    68/push  _test-Abc-kernel-string/imm32
+    # . . call
+    e8/call  kernel-string-equal?/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               8/imm32           # add to esp
+    # check-ints-equal(eax, 0, msg)
+    # . . push args
+    68/push  "F - test-compare-kernel-string-with-longer-array"/imm32
+    68/push  0/imm32/false
+    50/push-eax
+    # . . call
+    e8/call  check-ints-equal/disp32
+    # . . discard args
+    81          0/subop/add         3/mod/direct    4/rm32/esp    .           .             .           .           .               0xc/imm32         # add to esp
+    c3/return
+
+== data
+
+Null-kernel-string:  # (addr kernel-string)
+    00/null
+
+_test-Abc-kernel-string:  # (addr kernel-string)
+    41/A 62/b 63/c 00/null
+
+# . . vim:nowrap:textwidth=0