7842 - new directory organization

Baremetal is now the default build target and therefore has its sources
at the top-level. Baremetal programs build using the phase-2 Mu toolchain
that requires a Linux kernel. This phase-2 codebase which used to be at
the top-level is now under the linux/ directory. Finally, the phase-2 toolchain,
while self-hosting, has a way to bootstrap from a C implementation, which
is now stored in linux/bootstrap. The bootstrap C implementation uses some
literate programming tools that are now in linux/bootstrap/tools.

So the whole thing has gotten inverted. Each directory should build one
artifact and include the main sources (along with standard library). Tools
used for building it are relegated to sub-directories, even though those
tools are often useful in their own right, and have had lots of interesting
programs written using them.

A couple of things have gotten dropped in this process:
  - I had old ways to run on just a Linux kernel, or with a Soso kernel.
    No more.
  - I had some old tooling for running a single test at the cursor. I haven't
    used that lately. Maybe I'll bring it back one day.

The reorg isn't done yet. Still to do:
  - redo documentation everywhere. All the README files, all other markdown,
    particularly vocabulary.md.
  - clean up how-to-run comments at the start of programs everywhere
  - rethink what to do with the html/ directory. Do we even want to keep
    supporting it?

In spite of these shortcomings, all the scripts at the top-level, linux/
and linux/bootstrap are working. The names of the scripts also feel reasonable.
This is a good milestone to take stock at.

1 files changed, 424 insertions, 0 deletions

diff --git a/linux/test_apps b/linux/test_apps
new file mode 100755
index 00000000..e72975d4
--- /dev/null
+++ b/linux/test_apps
@@ -0,0 +1,424 @@
+#!/bin/sh
+# Build and test all included SubX programs on Linux:
+#   translate them into ELF binaries
+#   if possible, compare the generated binaries with what's already in git
+#   run/test the ELF binaries in emulated mode
+#   run/test the ELF binaries in native mode
+#
+# Example usage:
+#   test_apps
+#   test_apps record  # don't compare with what's in git
+
+set -e
+cd `dirname $0`
+
+echo "== translating using the bootstrap C++ translator"
+
+# example programs
+
+echo ex1
+bootstrap/bootstrap translate 000init.subx ex1.subx  -o ex1
+{
+  bootstrap/bootstrap run ex1  ||  ret=$?
+  test $ret -eq 42  # life, the universe and everything
+}
+{
+  chmod +x ex1
+  ./ex1  ||  ret=$?
+  test $ret -eq 42  # life, the universe and everything
+}
+
+echo ex2
+bootstrap/bootstrap translate 000init.subx ex2.subx  -o ex2
+{
+  bootstrap/bootstrap run ex2  ||  ret=$?
+  test $ret -eq 7  # 3 + 4
+}
+{
+  chmod +x ex2
+  ./ex2  ||  ret=$?
+  test $ret -eq 7  # 3 + 4
+}
+
+echo ex3
+bootstrap/bootstrap translate 000init.subx ex3.subx  -o ex3
+{
+  bootstrap/bootstrap run ex3  ||  ret=$?
+  test $ret -eq 55  # 1 + 2 + ... + 10
+}
+{
+  chmod +x ex3
+  ./ex3  ||  ret=$?
+  test $ret -eq 55  # 1 + 2 + ... + 10
+}
+
+echo ex4
+bootstrap/bootstrap translate 000init.subx ex4.subx  -o ex4
+{
+  echo a | bootstrap/bootstrap run ex4 >ex4.out  ||  true
+  test `cat ex4.out` = 'a'
+}
+{
+  chmod +x ex4
+  echo a | ./ex4 >ex4.out  ||  true
+  test `cat ex4.out` = 'a'
+}
+
+echo ex5
+bootstrap/bootstrap translate 000init.subx ex5.subx  -o ex5
+{
+  echo a | bootstrap/bootstrap run ex5 >ex5.out  ||  true
+  test `cat ex5.out` = 'a'
+}
+{
+  chmod +x ex5
+  echo a | ./ex5 >ex5.out  ||  true
+  test `cat ex5.out` = 'a'
+}
+
+echo ex6
+bootstrap/bootstrap translate 000init.subx ex6.subx  -o ex6
+{
+  bootstrap/bootstrap run ex6 >ex6.out  ||  true
+  test "`cat ex6.out`" = 'Hello, world!'
+}
+{
+  chmod +x ex6
+  ./ex6 >ex6.out  ||  true
+  test "`cat ex6.out`" = 'Hello, world!'
+}
+
+echo ex7
+bootstrap/bootstrap translate 000init.subx ex7.subx  -o ex7
+{
+  bootstrap/bootstrap run ex7  ||  ret=$?
+  test $ret -eq 97  # 'a'
+}
+{
+  chmod +x ex7
+  ./ex7  ||  ret=$?
+  test $ret -eq 97  # 'a'
+}
+
+echo ex8
+bootstrap/bootstrap translate 000init.subx ex8.subx  -o ex8
+{
+  bootstrap/bootstrap run ex8 abcd  ||  ret=$?
+  test $ret -eq 4  # length('abcd')
+}
+{
+  chmod +x ex8
+  ./ex8 abcd  ||  ret=$?
+  test $ret -eq 4  # length('abcd')
+}
+
+echo ex9
+bootstrap/bootstrap translate 000init.subx ex9.subx  -o ex9
+{
+  bootstrap/bootstrap run ex9 z x  ||  ret=$?
+  test $ret -eq 2  # 'z' - 'x'
+}
+{
+  chmod +x ex9
+  ./ex9 z x  ||  ret=$?
+  test $ret -eq 2  # 'z' - 'x'
+}
+
+echo ex10
+bootstrap/bootstrap translate 000init.subx ex10.subx  -o ex10
+{
+  bootstrap/bootstrap run ex10 abc abc  ||  ret=$?
+  test $ret -eq 1  # equal
+  bootstrap/bootstrap run ex10 abc abcd  # 0; not equal
+}
+{
+  chmod +x ex10
+  ./ex10 abc abc  ||  ret=$?
+  test $ret -eq 1  # equal
+  ./ex10 abc abcd  # 0; not equal
+}
+
+echo ex11
+bootstrap/bootstrap translate 000init.subx ex11.subx  -o ex11
+{
+  bootstrap/bootstrap run ex11
+  echo
+}
+{
+  chmod +x ex11
+  ./ex11
+  echo
+}
+
+echo ex12
+bootstrap/bootstrap translate 000init.subx ex12.subx  -o ex12
+bootstrap/bootstrap run ex12  # final byte of mmap'd address is well-nigh guaranteed to be 0
+chmod +x ex12
+./ex12
+
+echo ex13
+bootstrap/bootstrap translate 000init.subx ex13.subx  -o ex13
+{
+  bootstrap/bootstrap run ex13  ||  ret=$?
+  test $ret -eq 1  # 3 == 3
+}
+{
+  chmod +x ex13
+  ./ex13  ||  ret=$?
+  test $ret -eq 1  # 3 == 3
+}
+
+# Larger apps that use the standard library.
+
+echo factorial
+bootstrap/bootstrap translate [01]*.subx factorial.subx  -o factorial
+{
+  bootstrap/bootstrap run factorial  ||  ret=$?
+  test $ret -eq 120  # factorial(5)
+  bootstrap/bootstrap run factorial test
+  echo
+}
+{
+  chmod +x factorial
+  ./factorial  ||  ret=$?
+  test $ret -eq 120  # factorial(5)
+  ./factorial test
+  echo
+}
+
+echo crenshaw2-1
+bootstrap/bootstrap translate [01]*.subx crenshaw2-1.subx  -o crenshaw2-1
+{
+  bootstrap/bootstrap run crenshaw2-1 test
+  echo
+}
+{
+  chmod +x crenshaw2-1
+  ./crenshaw2-1 test
+  echo
+}
+
+echo crenshaw2-1b
+bootstrap/bootstrap translate [01]*.subx crenshaw2-1b.subx  -o crenshaw2-1b
+{
+  bootstrap/bootstrap run crenshaw2-1b test
+  echo
+}
+{
+  chmod +x crenshaw2-1b
+  ./crenshaw2-1b test
+  echo
+}
+
+echo random
+bootstrap/bootstrap translate [01]*.subx random.subx  -o random
+# don't run
+
+# Phases of the self-hosted SubX translator.
+
+for phase in hex survey_elf pack assort dquotes tests
+do
+  echo $phase
+  bootstrap/bootstrap translate [01]*.subx subx-params.subx $phase.subx -o $phase
+  test "$1" = 'record'  ||  git diff --exit-code $phase
+  {
+    bootstrap/bootstrap run $phase test
+    echo
+  }
+  {
+    chmod +x $phase
+    ./$phase test
+    echo
+  }
+done
+
+# One phase variant for translating SubX to baremetal. (Other phases are reused.)
+
+for phase in survey_baremetal
+do
+  echo $phase
+  bootstrap/bootstrap translate [01]*.subx subx-params.subx $phase.subx -o $phase
+  test "$1" = 'record'  ||  git diff --exit-code $phase
+  {
+    bootstrap/bootstrap run $phase test
+    echo
+  }
+  {
+    chmod +x $phase
+    ./$phase test
+    echo
+  }
+done
+
+# Higher-level syntax.
+
+# Certain phases of translation run native beyond this point. We're starting
+# to go beyond functionality of the C++ bootstrap_bin.
+
+echo sigils
+bootstrap/bootstrap translate [012]*.subx subx-params.subx sigils.subx  -o sigils
+test "$1" = 'record'  ||  git diff --exit-code sigils
+{
+  bootstrap/bootstrap run sigils test
+  echo
+}
+{
+  chmod +x sigils
+  ./sigils test
+  echo
+}
+
+echo calls
+cat [012]*.subx subx-params.subx calls.subx  |  sigils  > a.sigils
+bootstrap/bootstrap translate a.sigils -o calls
+test "$1" = 'record'  ||  git diff --exit-code calls
+{
+  bootstrap/bootstrap run calls test
+  echo
+}
+{
+  chmod +x calls
+  ./calls test
+  echo
+}
+
+echo braces
+cat [012]*.subx subx-params.subx braces.subx  |  calls  |  sigils  > a.sigils
+bootstrap/bootstrap translate a.sigils -o braces
+test "$1" = 'record'  ||  git diff --exit-code braces
+{
+  bootstrap/bootstrap run braces test
+  echo
+}
+{
+  chmod +x braces
+  ./braces test
+  echo
+}
+
+echo mu
+cat [0-2]*.subx mu.subx  |  braces  |  calls  |  sigils  > a.sigils
+bootstrap/bootstrap translate a.sigils -o mu
+test "$1" = 'record'  ||  git diff --exit-code mu
+{
+  bootstrap/bootstrap run mu test
+  echo
+}
+{
+  chmod +x mu
+  ./mu test
+  echo
+}
+
+echo "== translating using the self-hosted translator"
+
+# example programs
+
+for n in `seq 1 12`
+do
+  echo ex$n
+  ./translate_subx 000init.subx ex$n.subx
+  diff ex$n a.elf
+done
+
+# Larger apps that use the standard library.
+
+for app in factorial crenshaw2-1 crenshaw2-1b
+do
+  echo $app
+  ./translate_subx [01]*.subx $app.subx
+  diff $app a.elf
+done
+
+# Phases of the self-hosted SubX translator.
+
+for app in hex survey_elf pack assort dquotes tests
+do
+  echo $app
+  ./translate_subx [01]*.subx subx-params.subx $app.subx
+  diff $app a.elf
+done
+
+# One phase variant for translating SubX to baremetal. (Other phases are reused.)
+
+for app in survey_baremetal
+do
+  echo $app
+  ./translate_subx [01]*.subx subx-params.subx $app.subx
+  diff $app a.elf
+done
+
+for app in sigils calls braces
+do
+  echo $app
+  ./translate_subx [012]*.subx subx-params.subx $app.subx
+  diff $app a.elf
+done
+
+# Mu translator
+echo mu
+./translate_subx [0-2]*.subx mu.subx
+diff mu a.elf
+
+# Mu programs
+
+echo ex1.mu
+./translate ex1.mu
+{
+  bootstrap/bootstrap run a.elf  ||  ret=$?
+  test $ret -eq 42  # life, the universe and everything
+}
+{
+  ./a.elf  ||  ret=$?
+  test $ret -eq 42  # life, the universe and everything
+}
+
+echo ex2.mu
+./translate ex2.mu
+{
+  bootstrap/bootstrap run a.elf  ||  ret=$?
+  test $ret -eq 7
+}
+{
+  ./a.elf  ||  ret=$?
+  test $ret -eq 7
+}
+
+echo ex3.mu
+./translate ex3.mu
+{
+  bootstrap/bootstrap run a.elf  ||  ret=$?
+  test $ret -eq 55
+}
+{
+  ./a.elf  ||  ret=$?
+  test $ret -eq 55
+}
+
+echo ex3.2.mu
+./translate ex3.2.mu
+{
+  bootstrap/bootstrap run a.elf  ||  ret=$?
+  test $ret -eq 55
+}
+{
+  ./a.elf  ||  ret=$?
+  test $ret -eq 55
+}
+
+echo factorial.mu
+./translate factorial.mu
+{
+  bootstrap/bootstrap run a.elf  ||  ret=$?
+  test $ret -eq 120
+  bootstrap/bootstrap run a.elf test
+  echo
+}
+{
+  ./a.elf  ||  ret=$?
+  test $ret -eq 120
+  ./a.elf test
+  echo
+}
+
+exit 0