Mu explores ways to turn arbitrary manual tests into reproducible automated
tests. Hoped-for benefits:
1. Projects release with confidence without requiring manual QA or causing
regressions for their users.
1. Open source projects become easier for outsiders to comprehend, since they
can more confidently try out changes with the knowledge that they'll get
rapid feedback if they break something. As a result projects become more
*rewrite-friendly*: it's easier to leave your project's historical
accidents and other baggage behind if you can be confident of not causing
regressions.
1. It becomes easier to teach programming by emphasizing tests far earlier
than we do today.
In this quest, Mu is currently experimenting with the following mechanisms:
1. New, testable interfaces for the operating system. Currently manual tests
are hard to automate because a file you assumed might vanish, the network
might go down, etc. To make manual tests reproducible it suffices to
improve the 15 or so OS syscalls through which a computer talks to the
outside world. We have to allow programs to transparently write to a fake
screen, read from a fake disk/network, etc. In Mu, printing to screen
explicitly takes a screen object, so it can be called on the real screen,
or on a fake screen inside tests, so that we can then check the expected
state of the screen at the end of a test. Here's a test for a little
text-mode chessboard program in Mu (delimiting the edge of the 'screen'
with dots):
<br> <img alt='a screen test' src='html/chessboard-test.png'>
<br>I'm building up similarly *dependency-injected* interfaces to the
keyboard, mouse, touch screen, disk, network, etc.
1. Support for testing side-effects like performance, deadlock-freedom,
race-freeness, memory usage, etc. Mu's *white-box tests* can check not just
the results of a function call, but also the presence or absence of
specific events in the log of its progress. For example, here's a test that
our string-comparison function doesn't scan individual characters unless it
has to:
<br> <img alt='white-box test' src='html/tracing-test.png'>
<br>Another example: if a sort function logs each swap, a performance test can
ensure that the number of swaps doesn't quadruple when the size of the
input doubles.
<p>Besides expanding the scope of tests, this ability also allows more
radical refactoring without needing to modify tests. All Mu's tests call a
top-level function rather than individual sub-systems directly. As a result
the way the subsystems are invoked can be radically changed (interface
changes, making synchronous functions asynchronous, etc.). As long as the
new versions emit the same implementation-independent events in the logs,
the tests will continue to pass. ([More information.](http://akkartik.name/post/tracing-tests))
1. Organizing code and tests in layers of functionality, so that outsiders can
build simple and successively more complex versions of a project, gradually
enabling more peripheral features. Think of it as a cleaned-up `git log`
for the project. ([More information.](http://akkartik.name/post/wart-layers))
Since I don't understand how Linux and other modern platforms work, Mu is
built on an idealized VM while I learn. Eventually the plan is to transplant
what I learn back to Linux.
To minimize my workload, Mu doesn't have a high-level language yet. Instead,
I've been programming directly in the VM's idealized assembly language. I
expected this to be painful, but it's had some surprising benefits. First,
programs as lists of instructions seem to be easier for non-programmers to
comprehend than programs as trees of expressions. Second, I've found that
Literate Programming using layers makes assembly much more ergonomic. Third,
labels for gotos turn out to be great waypoints to insert code at from future
layers; when I tried to divide C programs into layers, I sometimes had to
split statements in two so I could insert code between them. Labels also seem
a promising representation for providing advanced mechanisms like
continuations and lisp-like macros.
High level languages today seem to provide three kinds of benefits:
expressiveness (e.g. nested expressions, classes), safety (e.g. type checking)
and automation (e.g. garbage collection). An idealized assembly language gives
up some expressiveness, but doesn't s<style>pre { line-height: 125%; }
td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
.highlight .hll { background-color: #ffffcc }
.highlight .c { color: #888888 } /* Comment */
.highlight .err { color: #a61717; background-color: #e3d2d2 } /* Error */
.highlight .k { color: #008800; font-weight: bold } /* Keyword */
.highlight .ch { color: #888888 } /* Comment.Hashbang */
.highlight .cm { color: #888888 } /* Comment.Multiline */
.highlight .cp { color: #cc0000; font-weight: bold } /* Comment.Preproc */
.highlight .cpf { color: #888888 } /* Comment.PreprocFile */
.highlight .c1 { color: #888888 } /* Comment.Single */
.highlight .cs { color: #cc0000; font-weight: bold; background-color: #fff0f0 } /* Comment.Special */
.highlight .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */
.highlight .ge { font-style: italic } /* Generic.Emph */
.highlight .ges { font-weight: bold; font-style: italic } /* Generic.EmphStrong */
.highlight .gr { color: #aa0000 } /* Generic.Error */
.highlight .gh { color: #333333 } /* Generic.Heading */
.highlight .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */
.highlight .go { color: #888888 } /* Generic.Output */
.highlight .gp { color: #555555 } /* Generic.Prompt */
.highlight .gs { font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #666666 } /* Generic.Subheading */
.highlight .gt { color: #aa0000 } /* Generic.Traceback */
.highlight .kc { color: #008800; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #008800 } /* Keyword.Pseudo */
.highlight .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #888888; font-weight: bold } /* Keyword.Type */
.highlight .m { color: #0000DD; font-weight: bold } /* Literal.Number */
.highlight .s { color: #dd2200; background-color: #fff0f0 } /* Literal.String */
.highlight .na { color: #336699 } /* Name.Attribute */
.highlight .nb { color: #003388 } /* Name.Builtin */
.highlight .nc { color: #bb0066; font-weight: bold } /* Name.Class */
.highlight .no { color: #003366; font-weight: bold } /* Name.Constant */
.highlight .nd { color: #555555 } /* Name.Decorator */
.highlight .ne { color: #bb0066; font-weight: bold } /* Name.Exception */
.highlight .nf { color: #0066bb; font-weight: bold } /* Name.Function */
.highlight .nl { color: #336699; font-style: italic } /* Name.Label */
.highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */
.highlight .py { color: #336699; font-weight: bold } /* Name.Property */
.highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #336699 } /* Name.Variable */
.highlight .ow { color: #008800 } /* Operator.Word */
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
.highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */
.highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */
.highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */
.highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */
.highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */
.highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */
.highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */
.highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */
.highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */
.highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */
.highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */
.highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */
.highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */
.highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */
.highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */
.highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */
.highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */
.highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */
.highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */
.highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */
.highlight .vc { color: #336699 } /* Name.Variable.Class */
.highlight .vg { color: #dd7700 } /* Name.Variable.Global */
.highlight .vi { color: #3333bb } /* Name.Variable.Instance */
.highlight .vm { color: #336699 } /* Name.Variable.Magic */
.highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */</style><div class="highlight"><pre><span></span><span class="c1">//: Everything this project/binary supports.</span>
<span class="c1">//: This should give you a sense for what to look forward to in later layers.</span>
<span class="o">:</span><span class="p">(</span><span class="n">before</span><span class="w"> </span><span class="s">"End Commandline Parsing"</span><span class="p">)</span>
<span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">argc</span><span class="w"> </span><span class="o"><=</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">||</span><span class="w"> </span><span class="n">is_equal</span><span class="p">(</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span><span class="w"> </span><span class="s">"--help"</span><span class="p">))</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="c1">// this is the functionality later layers will provide</span>
<span class="w"> </span><span class="c1">// currently no automated tests for commandline arg parsing</span>
<span class="w"> </span><span class="n">cerr</span><span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">"To load files and run 'main':</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">" mu file1.mu file2.mu ...</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">"To run all tests:</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">" mu test</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">"To load files and then run all tests:</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="s">" mu test file1.mu file2.mu ...</span><span class="se">\n</span><span class="s">"</span>
<span class="w"> </span><span class="p">;</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">//:: Helper function used by the above fragment of code (and later layers too,</span>
<span class="c1">//:: who knows?).</span>
<span class="c1">//: The :(code) directive appends function definitions to the end of the</span>
<span class="c1">//: project. Regardless of where functions are defined, we can call them</span>
<span class="c1">//: anywhere we like as long as we format the function header in a specific</span>
<span class="c1">//: way: put it all on a single line without indent, end the line with ') {'</span>
<span class="c1">//: and no trailing whitespace. As long as functions uniformly start this</span>
<span class="c1">//: way, our makefile contains a little command to automatically generate</span>
<span class="c1">//: declarations for them.</span>
<span class="o">:</span><span class="p">(</span><span class="n">code</span><span class="p">)</span>
<span class="kt">bool</span><span class="w"> </span><span class="n">is_equal</span><span class="p">(</span><span class="kt">char</span><span class="o">*</span><span class="w"> </span><span class="n">s</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="o">*</span><span class="w"> </span><span class="n">lit</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">strncmp</span><span class="p">(</span><span class="n">s</span><span class="p">,</span><span class="w"> </span><span class="n">lit</span><span class="p">,</span><span class="w"> </span><span class="n">strlen</span><span class="p">(</span><span class="n">lit</span><span class="p">))</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="o">:</span><span class="p">(</span><span class="n">before</span><span class="w"> </span><span class="s">"End Includes"</span><span class="p">)</span>
<span class="cp">#include</span><span class="cpf"><assert.h></span>
<span class="cp">#include</span><span class="cpf"><iostream></span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">istream</span><span class="p">;</span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">ostream</span><span class="p">;</span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">iostream</span><span class="p">;</span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">cin</span><span class="p">;</span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="p">;</span>
<span class="k">using</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">cerr</span><span class="p">;</span>
<span class="cp">#include</span><span class="cpf"><cstring></span>
<span class="cp">#include</span><span class="cpf"><string></span>
<span class="k">using</span><span class="w"