Templates
=========
A template is a simple form of a macro: It is a simple substitution
mechanism that operates on Nim's abstract syntax trees. It is processed in
the semantic pass of the compiler.
The syntax to *invoke* a template is the same as calling a procedure.
Example:
.. code-block:: nim
template `!=` (a, b: expr): expr =
# this definition exists in the System module
not (a == b)
assert(5 != 6) # the compiler rewrites that to: assert(not (5 == 6))
The ``!=``, ``>``, ``>=``, ``in``, ``notin``, ``isnot`` operators are in fact
templates:
| ``a > b`` is transformed into ``b < a``.
| ``a in b`` is transformed into ``contains(b, a)``.
| ``notin`` and ``isnot`` have the obvious meanings.
The "types" of templates can be the symbols ``expr`` (stands for *expression*),
``stmt`` (stands for *statement*) or ``typedesc`` (stands for *type
description*). These are "meta types", they can only be used in certain
contexts. Real types can be used too; this implies that expressions are
expected.
Ordinary vs immediate templates
-------------------------------
There are two different kinds of templates: immediate templates and
ordinary templates. Ordinary templates take part in overloading resolution. As
such their arguments need to be type checked before the template is invoked.
So ordinary templates cannot receive undeclared identifiers:
.. code-block:: nim
template declareInt(x: expr) =
var x: int
declareInt(x) # error: unknown identifier: 'x'
An ``immediate`` template does not participate in overload resolution and so
its arguments are not checked for semantics before invocation. So they can
receive undeclared identifiers:
.. code-block:: nim
template declareInt(x: expr) {.immediate.} =
var x: int
declareInt(x) # valid
Passing a code block to a template
----------------------------------
If there is a ``stmt`` parameter it should be the last in the template
declaration, because statements are passed to a template via a
special ``:`` syntax:
.. code-block:: nim
template withFile(f, fn, mode: expr, actions: stmt): stmt {.immediate.} =
var f: TFile
if open(f, fn, mode):
try:
actions
finally:
close(f)
else:
quit("cannot open: " & fn)
withFile(txt, "ttempl3.txt", fmWrite):
txt.writeln("line 1")
txt.writeln("line 2")
In the example the two ``writeln`` statements are bound to the ``actions``
parameter.
Symbol binding in templates
---------------------------
A template is a `hygienic`:idx: macro and so opens a new scope. Most symbols are
bound from the definition scope of the template:
.. code-block:: nim
# Module A
var
lastId = 0
template genId*: expr =
inc(lastId)
lastId
.. code-block:: nim
# Module B
import A
echo genId() # Works as 'lastId' has been bound in 'genId's defining scope
As in generics symbol binding can be influenced via ``mixin`` or ``bind``
statements.
Identifier construction
-----------------------
In templates identifiers can be constructed with the backticks notation:
.. code-block:: nim
template typedef(name: expr, typ: typedesc) {.immediate.} =
type
`T name`* {.inject.} = typ
`P name`* {.inject.} = ref `T name`
typedef(myint, int)
var x: PMyInt
In the example ``name`` is instantiated with ``myint``, so \`T name\` becomes
``Tmyint``.
Lookup rules for template parameters
------------------------------------
A parameter ``p`` in a template is even substituted in the expression ``x.p``.
Thus template arguments can be used as field names and a global symbol can be
shadowed by the same argument name even when fully qualified:
.. code-block:: nim
# module 'm'
type
TLev = enum
levA, levB
var abclev = levB
template tstLev(abclev: TLev) =
echo abclev, " ", m.abclev
tstLev(levA)
# produces: 'levA levA'
But the global symbol can properly be captured by a ``bind`` statement:
.. code-block:: nim
# module 'm'
type
TLev = enum
levA, levB
var abclev = levB
template tstLev(abclev: TLev) =
bind m.abclev
echo abclev, " ", m.abclev
tstLev(levA)
# produces: 'levA levB'
Hygiene in templates
--------------------
Per default templates are `hygienic`:idx:\: Local identifiers declared in a
template cannot be accessed in the instantiation context:
.. code-block:: nim
template newException*(exceptn: typedesc, message: string): expr =
var
e: ref exceptn # e is implicitly gensym'ed here
new(e)
e.msg = message
e
# so this works:
let e = "message"
raise newException(EIO, e)
Whether a symbol that is declared in a template is exposed to the instantiation
scope is controlled by the `inject`:idx: and `gensym`:idx: pragmas: gensym'ed
symbols are not exposed but inject'ed are.
The default for symbols of entity ``type``, ``var``, ``let`` and ``const``
is ``gensym`` and for ``proc``, ``iterator``, ``converter``, ``template``,
``macro`` is ``inject``. However, if the name of the entity is passed as a
template parameter, it is an inject'ed symbol:
.. code-block:: nim
template withFile(f, fn, mode: expr, actions: stmt): stmt {.immediate.} =
block:
var f: TFile # since 'f' is a template param, it's injected implicitly
...
withFile(txt, "ttempl3.txt", fmWrite):
txt.writeln("line 1")
txt.writeln("line 2")
The ``inject`` and ``gensym`` pragmas are second class annotations; they have
no semantics outside of a template definition and cannot be abstracted over:
.. code-block:: nim
{.pragma myInject: inject.}
template t() =
var x {.myInject.}: int # does NOT work
To get rid of hygiene in templates, one can use the `dirty`:idx: pragma for
a template. ``inject`` and ``gensym`` have no effect in ``dirty`` templates.
Limitations of the method call syntax
-------------------------------------
The expression ``x`` in ``x.f`` needs to be semantically checked (that means
symbol lookup and type checking) before it can be decided that it needs to be
rewritten to ``f(x)``. Therefore the dot syntax has some limiations when it
is used to invoke templates/macros:
.. code-block:: nim
template declareVar(name: expr): stmt =
const name {.inject.} = 45
# Doesn't compile:
unknownIdentifier.declareVar
Another common example is this:
.. code-block:: nim
from sequtils import toSeq
iterator something: string =
yield "Hello"
yield "World"
var info = toSeq(something())
The problem here is that the compiler already decided that ``something()`` as
an iterator is not callable in this context before ``toSeq`` gets its
chance to convert it into a sequence.
Macros
======
A macro is a special kind of low level template. Macros can be used
to implement `domain specific languages`:idx:. Like templates, macros come in
the 2 flavors *immediate* and *ordinary*.
While macros enable advanced compile-time code transformations, they
cannot change Nim's syntax. However, this is no real restriction because
Nim's syntax is flexible enough anyway.
To write macros, one needs to know how the Nim concrete syntax is converted
to an abstract syntax tree.
There are two ways to invoke a macro:
(1) invoking a macro like a procedure call (`expression macros`)
(2) invoking a macro with the special ``macrostmt`` syntax (`statement macros`)
Expression Macros
-----------------
The following example implements a powerful ``debug`` command that accepts a
variable number of arguments:
.. code-block:: nim
# to work with Nim syntax trees, we need an API that is defined in the
# ``macros`` module:
import macros
macro debug(n: varargs[expr]): stmt =
# `n` is a Nim AST that contains the whole macro invocation
# this macro returns a list of statements:
result = newNimNode(nnkStmtList, n)
# iterate over any argument that is passed to this macro:
for i in 0..n.len-1:
# add a call to the statement list that writes the expression;
# `toStrLit` converts an AST to its string representation:
add(result, newCall("write", newIdentNode("stdout"), toStrLit(n[i])))
# add a call to the statement list that writes ": "
add(result, newCall("write", newIdentNode("stdout"), newStrLitNode(": ")))
# add a call to the statement list that writes the expressions value:
add(result, newCall("writeln", newIdentNode("stdout"), n[i]))
var
a: array [0..10, int]
x = "some string"
a[0] = 42
a[1] = 45
debug(a[0], a[1], x)
The macro call expands to:
.. code-block:: nim
write(stdout, "a[0]")
write(stdout, ": ")
writeln(stdout, a[0])
write(stdout, "a[1]")
write(stdout, ": ")
writeln(stdout, a[1])
write(stdout, "x")
write(stdout, ": ")
writeln(stdout, x)
Arguments that are passed to a ``varargs`` parameter are wrapped in an array
constructor expression. This is why ``debug`` iterates over all of ``n``'s
children.
BindSym
-------
The above ``debug`` macro relies on the fact that ``write``, ``writeln`` and
``stdout`` are declared in the system module and thus visible in the
instantiating context. There is a way to use bound identifiers
(aka `symbols`:idx:) instead of using unbound identifiers. The ``bindSym``
builtin can be used for that:
.. code-block:: nim
import macros
macro debug(n: varargs[expr]): stmt =
result = newNimNode(nnkStmtList, n)
for i in 0..n.len-1:
# we can bind symbols in scope via 'bindSym':
add(result, newCall(bindSym"write", bindSym"stdout", toStrLit(n[i])))
add(result, newCall(bindSym"write", bindSym"stdout", newStrLitNode(": ")))
add(result, newCall(bindSym"writeln", bindSym"stdout", n[i]))
var
a: array [0..10, int]
x = "some string"
a[0] = 42
a[1] = 45
debug(a[0], a[1], x)
The macro call expands to:
.. code-block:: nim
write(stdout, "a[0]")
write(stdout, ": ")
writeln(stdout, a[0])
write(stdout, "a[1]")
write(stdout, ": ")
writeln(stdout, a[1])
write(stdout, "x")
write(stdout, ": ")
writeln(stdout, x)
However, the symbols ``write``, ``writeln`` and ``stdout`` are already bound
and are not looked up again. As the example shows, ``bindSym`` does work with
overloaded symbols implicitly.
Statement Macros
----------------
Statement macros are defined just as expression macros. However, they are
invoked by an expression following a colon.
The following example outlines a macro that generates a lexical analyzer from
regular expressions:
.. code-block:: nim
import macros
macro case_token(n: stmt): stmt =
# creates a lexical analyzer from regular expressions
# ... (implementation is an exercise for the reader :-)
discard
case_token: # this colon tells the parser it is a macro statement
of r"[A-Za-z_]+[A-Za-z_0-9]*":
return tkIdentifier
of r"0-9+":
return tkInteger
of r"[\+\-\*\?]+":
return tkOperator
else:
return tkUnknown
**Style note**: For code readability, it is the best idea to use the least
powerful programming construct that still suffices. So the "check list" is:
(1) Use an ordinary proc/iterator, if possible.
(2) Else: Use a generic proc/iterator, if possible.
(3) Else: Use a template, if possible.
(4) Else: Use a macro.
Macros as pragmas
-----------------
Whole routines (procs, iterators etc.) can also be passed to a template or
a macro via the pragma notation:
.. code-block:: nim
template m(s: stmt) = discard
proc p() {.m.} = discard
This is a simple syntactic transformation into:
.. code-block:: nim
template m(s: stmt) = discard
m:
proc p() = discard