Compiler plugin for implementing incremental computation in user space (#10819)

This plugin provides essential building block for implementing incremental computations in your programs. The idea behind incremental computations is that if you do the same calculation multiple times but with slightly different inputs you don't have to recompute everything from scratch. Also you don't want to adopt special algorithms either, you would like to write your code in standard from scratch manner and get incrementality for free when it is possible.

The plugin computes the digest of the proc bodies, recursively hashing all called procs as well . Such digest with the digest of the argument values gives a good "name" for the result. Terminology loosely follows paper "Incremental Computation with Names" link below. It works well if you have no side effects in your computations. If you have global state in your computations then you will need problem specific workarounds to represent global state in set of "names" . SideEffect tracking in Nim also useful in this topic.

Classical examples:

Dashboard with ticking data. New data arrives non stop and you would like to update the dashboard recomputing only changed outputs.
Excel spreadsheet where user changes one cell and you would like to recompute all cells that are affected by the change, but do not want to recompute every cell in the spreadsheet.

3 files changed, 29 insertions, 1 deletions

diff --git a/lib/core/macros.nim b/lib/core/macros.nim
index 6525770eb..d27d97f2f 100644
--- a/lib/core/macros.nim
+++ b/lib/core/macros.nim
@@ -360,6 +360,13 @@ when defined(nimHasSignatureHashInMacro):
     ## the owning module of the symbol and others. The same identifier is
     ## used in the back-end to produce the mangled symbol name.
 
+proc symBodyHash*(s: NimNode): string {.noSideEffect.} =
+  ## Returns a stable digest for symbols derived not only from type signature
+  ## and owning module, but also implementation body. All procs/varibles used in
+  ## the implementation of this symbol are hashed recursively as well, including
+  ## magics from system module.
+  discard
+
 proc getTypeImpl*(n: typedesc): NimNode {.magic: "NGetType", noSideEffect.}
   ## Version of ``getTypeImpl`` which takes a ``typedesc``.
 
diff --git a/lib/pure/md5.nim b/lib/pure/md5.nim
index 794d839f8..ae8e12707 100644
--- a/lib/pure/md5.nim
+++ b/lib/pure/md5.nim
@@ -197,7 +197,9 @@ proc `$`*(d: MD5Digest): string =
     add(result, digits[d[i].int and 0xF])
 
 proc getMD5*(s: string): string =
-  ## Computes an MD5 value of `s` and returns its string representation.
+  ## Computes an MD5 value of `s` and returns its string representation. 
+  ## .. note::
+  ## available at compile time
   ##
   ## See also:
   ## * `toMD5 proc <#toMD5,string>`_ which returns the `MD5Digest` of a string
diff --git a/lib/system/io.nim b/lib/system/io.nim
index e93f602ae..4497b1b0b 100644
--- a/lib/system/io.nim
+++ b/lib/system/io.nim
@@ -612,6 +612,25 @@ proc writeFile*(filename, content: string) {.tags: [WriteIOEffect], benign.} =
   else:
     sysFatal(IOError, "cannot open: " & filename)
 
+
+proc readLines*(filename: string, n = 1.Natural): seq[TaintedString] =
+  ## read `n` lines from the file named `filename`. Raises an IO exception
+  ## in case of an error. Raises EOF if file does not contain at least `n` lines.
+  ## Available at compile time. A line of text may be delimited by ``LF`` or ``CRLF``.
+  ## The newline character(s) are not part of the returned strings.
+  var f: File
+  if open(f, filename):
+    try:
+      result = newSeq[TaintedString](n)
+      for i in 0 .. n - 1:
+        if not readLine(f, result[i]):
+          raiseEOF()
+    finally:
+      close(f)
+  else:
+    sysFatal(IOError, "cannot open: " & filename)
+
+
 iterator lines*(filename: string): TaintedString {.tags: [ReadIOEffect].} =
   ## Iterates over any line in the file named `filename`.
   ##