diff options
Diffstat (limited to 'doc')
-rw-r--r-- | doc/advopt.txt | 76 | ||||
-rw-r--r-- | doc/apis.md (renamed from doc/apis.rst) | 11 | ||||
-rw-r--r-- | doc/astspec.txt | 516 | ||||
-rw-r--r-- | doc/backends.md (renamed from doc/backends.rst) | 307 | ||||
-rw-r--r-- | doc/basicopt.txt | 7 | ||||
-rw-r--r-- | doc/contributing.md (renamed from doc/contributing.rst) | 492 | ||||
-rw-r--r-- | doc/destructors.md (renamed from doc/destructors.rst) | 446 | ||||
-rw-r--r-- | doc/docgen.md | 891 | ||||
-rw-r--r-- | doc/docgen.rst | 412 | ||||
-rw-r--r-- | doc/docgen_sample.nim | 2 | ||||
-rw-r--r-- | doc/docs.md (renamed from doc/docs.rst) | 24 | ||||
-rw-r--r-- | doc/docstyle.md (renamed from doc/docstyle.rst) | 76 | ||||
-rw-r--r-- | doc/drnim.md (renamed from doc/drnim.rst) | 67 | ||||
-rw-r--r-- | doc/effects.txt | 5 | ||||
-rw-r--r-- | doc/estp.md | 206 | ||||
-rw-r--r-- | doc/estp.rst | 197 | ||||
-rw-r--r-- | doc/filelist.txt | 10 | ||||
-rw-r--r-- | doc/filters.md (renamed from doc/filters.rst) | 200 | ||||
-rw-r--r-- | doc/grammar.txt | 128 | ||||
-rw-r--r-- | doc/hcr.md (renamed from doc/hcr.rst) | 52 | ||||
-rw-r--r-- | doc/idetools.md (renamed from doc/idetools.rst) | 292 | ||||
-rw-r--r-- | doc/intern.md | 679 | ||||
-rw-r--r-- | doc/intern.rst | 796 | ||||
-rw-r--r-- | doc/koch.md (renamed from doc/koch.rst) | 43 | ||||
-rw-r--r-- | doc/lib.md | 682 | ||||
-rw-r--r-- | doc/lib.rst | 603 | ||||
-rw-r--r-- | doc/manual.md (renamed from doc/manual.rst) | 4098 | ||||
-rw-r--r-- | doc/manual/var_t_return.md (renamed from doc/manual/var_t_return.rst) | 4 | ||||
-rw-r--r-- | doc/manual_experimental.md (renamed from doc/manual_experimental.rst) | 2193 | ||||
-rw-r--r-- | doc/manual_experimental_strictnotnil.md (renamed from doc/manual_experimental_strictnotnil.rst) | 89 | ||||
-rw-r--r-- | doc/markdown_rst.md | 349 | ||||
-rw-r--r-- | doc/mm.md | 95 | ||||
-rw-r--r-- | doc/nep1.md (renamed from doc/nep1.rst) | 115 | ||||
-rw-r--r-- | doc/nimc.md (renamed from doc/nimc.rst) | 512 | ||||
-rw-r--r-- | doc/nimdoc.cls | 196 | ||||
-rw-r--r-- | doc/nimdoc.css | 376 | ||||
-rw-r--r-- | doc/nimfix.rst | 58 | ||||
-rw-r--r-- | doc/nimgrep.md | 128 | ||||
-rw-r--r-- | doc/nimgrep.rst | 52 | ||||
-rw-r--r-- | doc/nimgrep_cmdline.txt | 136 | ||||
-rw-r--r-- | doc/niminst.md (renamed from doc/niminst.rst) | 58 | ||||
-rw-r--r-- | doc/nims.md (renamed from doc/nims.rst) | 194 | ||||
-rw-r--r-- | doc/nimsuggest.md (renamed from doc/nimsuggest.rst) | 54 | ||||
-rw-r--r-- | doc/overview.md (renamed from doc/overview.rst) | 2 | ||||
-rw-r--r-- | doc/packaging.md (renamed from doc/packaging.rst) | 40 | ||||
-rw-r--r-- | doc/pegdocs.txt | 100 | ||||
-rw-r--r-- | doc/readme.txt | 4 | ||||
-rw-r--r-- | doc/refc.md (renamed from doc/gc.rst) | 113 | ||||
-rw-r--r-- | doc/regexprs.txt | 14 | ||||
-rw-r--r-- | doc/rstcommon.rst | 51 | ||||
-rw-r--r-- | doc/sets_fragment.txt | 76 | ||||
-rw-r--r-- | doc/spawn.txt | 8 | ||||
-rw-r--r-- | doc/subexes.txt | 5 | ||||
-rw-r--r-- | doc/testament.md | 390 | ||||
-rw-r--r-- | doc/testament.rst | 252 | ||||
-rw-r--r-- | doc/tools.md (renamed from doc/tools.rst) | 34 | ||||
-rw-r--r-- | doc/tut1.md (renamed from doc/tut1.rst) | 786 | ||||
-rw-r--r-- | doc/tut2.md (renamed from doc/tut2.rst) | 133 | ||||
-rw-r--r-- | doc/tut3.md (renamed from doc/tut3.rst) | 121 |
59 files changed, 11150 insertions, 6906 deletions
diff --git a/doc/advopt.txt b/doc/advopt.txt index 108c07222..e4d11081a 100644 --- a/doc/advopt.txt +++ b/doc/advopt.txt @@ -4,9 +4,13 @@ Advanced commands: //compileToOC, objc compile project to Objective C code //js compile project to Javascript //e run a Nimscript file + //md2html convert a Markdown file to HTML + use `--docCmd:skip` to skip compiling snippets //rst2html convert a reStructuredText file to HTML use `--docCmd:skip` to skip compiling snippets - //rst2tex convert a reStructuredText file to TeX + //md2tex convert a Markdown file to LaTeX + //rst2tex convert a reStructuredText file to LaTeX + //doc2tex extract the documentation to a LaTeX file //jsondoc extract the documentation to a json file //ctags create a tags file //buildIndex build an index for the whole documentation @@ -14,7 +18,8 @@ Advanced commands: module dependency graph //dump dump all defined conditionals and search paths see also: --dump.format:json (useful with: `| jq`) - //check checks the project for syntax and semantic + //check checks the project for syntax and semantics + (can be combined with --defusages) Runtime checks (see -x): --objChecks:on|off turn obj conversion checks on|off @@ -27,28 +32,45 @@ Runtime checks (see -x): --infChecks:on|off turn Inf checks on|off Advanced options: + --defusages:FILE,LINE,COL + find the definition and all usages of a symbol -o:FILE, --out:FILE set the output filename --outdir:DIR set the path where the output file will be written --usenimcache will use `outdir=$$nimcache`, whichever it resolves to after all options have been processed --stdout:on|off output to stdout --colors:on|off turn compiler messages coloring on|off - --listFullPaths:on|off list full paths in messages + --filenames:abs|canonical|legacyRelProj + customize how filenames are rendered in compiler messages, + defaults to `abs` (absolute) + --processing:dots|filenames|off + show files as they're being processed by nim compiler + --unitsep:on|off use the ASCII unit separator (31) between error + messages, useful for IDE-like tooling --declaredLocs:on|off show declaration locations in messages - --spellSuggest|:num show at most `num >= 0` spelling suggestions on typos. + --spellSuggest:num show at most `num >= 0` spelling suggestions on typos. if `num` is not specified (or `auto`), return an implementation defined set of suggestions. + --hints:on|off|list. `on|off` enables or disables hints. + `list` reports which hints are selected. + --hint:X:on|off turn specific hint X on|off. `hint:X` means `hint:X:on`, + as with similar flags. `all` is the set of all hints + (only `all:off` is supported). + --hintAsError:X:on|off turn specific hint X into an error on|off -w:on|off|list, --warnings:on|off|list - turn all warnings on|off or list all available - --warning[X]:on|off turn specific warning X on|off - --hints:on|off|list turn all hints on|off or list all available - --hint[X]:on|off turn specific hint X on|off - --warningAsError[X]:on|off turn specific warning X into an error on|off - --hintAsError[X]:on|off turn specific hint X into an error on|off + `on|off` enables or disables warnings. + `list` reports which warnings are selected. + --warning:X:on|off turn specific warning X on|off. `warning:X` means `warning:X:on`, + as with similar flags. `all` is the set of all warning + (only `all:off` is supported). + --warningAsError:X:on|off + turn specific warning X into an error on|off --styleCheck:off|hint|error produce hints or errors for Nim identifiers that do not adhere to Nim's official style guide https://nim-lang.org/docs/nep1.html + --styleCheck:usages only enforce consistent spellings of identifiers, + do not enforce the style on declarations --showAllMismatches:on|off show all mismatching candidates in overloading resolution @@ -83,7 +105,8 @@ Advanced options: if path == @default (the default and most useful), will use best match among @pkg,@path. if these are nonexistent, will use project path - -b, --backend:c|cpp|js|objc sets backend to use with commands like `nim doc` or `nim r` + -b, --backend:c|cpp|js|objc + sets backend to use with commands like `nim doc` or `nim r` --docCmd:cmd if `cmd == skip`, skips runnableExamples else, runs runnableExamples with given options, e.g.: `--docCmd:"-d:foo --threads:on"` @@ -93,28 +116,26 @@ Advanced options: --lineDir:on|off generation of #line directive on|off --embedsrc:on|off embeds the original source code as comments in the generated output - --threadanalysis:on|off turn thread analysis on|off --tlsEmulation:on|off turn thread local storage emulation on|off --implicitStatic:on|off turn implicit compile time evaluation on|off --trmacros:on|off turn term rewriting macros on|off --multimethods:on|off turn multi-methods on|off - --memTracker:on|off turn memory tracker on|off --hotCodeReloading:on|off turn support for hot code reloading on|off --excessiveStackTrace:on|off stack traces use full file paths --stackTraceMsgs:on|off enable user defined stack frame msgs via `setFrameMsg` - --seqsv2:on|off use the new string/seq implementation based on - destructors --skipCfg:on|off do not read the nim installation's configuration file --skipUserCfg:on|off do not read the user's configuration file --skipParentCfg:on|off do not read the parent dirs' configuration files --skipProjCfg:on|off do not read the project's configuration file - --gc:refc|arc|orc|markAndSweep|boehm|go|none|regions - select the GC to use; default is 'refc' - --exceptions:setjmp|cpp|goto + --mm:orc|arc|refc|markAndSweep|boehm|go|none|regions + select which memory management to use; default is 'orc' + --exceptions:setjmp|cpp|goto|quirky select the exception handling implementation - --index:on|off turn index file generation on|off + --index:on|off|only docgen: turn index file generation on|off (`only` means + not generate output files like HTML) + --noImportdoc:on|off turn loading documentation ``.idx`` files on|off --putenv:key=value set an environment variable --NimblePath:PATH add a path for Nimble support --noNimblePath deactivate the Nimble path @@ -122,6 +143,8 @@ Advanced options: --cppCompileToNamespace:namespace use the provided namespace for the generated C++ code, if no namespace is provided "Nim" will be used + --nimMainPrefix:prefix use `{prefix}NimMain` instead of `NimMain` in the produced + C/C++ code --expandMacro:MACRO dump every generated AST from MACRO --expandArc:PROCNAME show how PROCNAME looks like after diverse optimizations before the final backend phase (mostly ARC/ORC specific) @@ -146,12 +169,11 @@ Advanced options: enable experimental language feature --legacy:$2 enable obsolete/legacy language feature - --useVersion:1.0|1.2 emulate Nim version X of the Nim compiler, for testing - --profiler:on|off enable profiling; requires `import nimprof`, and - works better with `--stackTrace:on` - see also https://nim-lang.github.io/Nim/estp.html - --benchmarkVM:on|off enable benchmarking of VM code with cpuTime() - --profileVM:on|off enable compile time VM profiler - --sinkInference:on|off en-/disable sink parameter inference (default: on) + --benchmarkVM:on|off turn benchmarking of VM code with cpuTime() on|off + --profileVM:on|off turn compile time VM profiler on|off --panics:on|off turn panics into process terminations (default: off) - --deepcopy:on|off enable 'system.deepCopy' for ``--gc:arc|orc`` + --deepcopy:on|off enable 'system.deepCopy' for ``--mm:arc|orc`` + --jsbigint64:on|off toggle the use of BigInt for 64-bit integers for + the JavaScript backend (default: on) + --nimBasePattern:nimbase.h + allows to specify a custom pattern for `nimbase.h` diff --git a/doc/apis.rst b/doc/apis.md index d01e75d78..f0b8c93e5 100644 --- a/doc/apis.rst +++ b/doc/apis.md @@ -1,9 +1,10 @@ -.. default-role:: code - ================= API naming design ================= +.. default-role:: code +.. include:: rstcommon.rst + The API is designed to be **easy to use** and consistent. Ease of use is measured by the number of calls to achieve a concrete high-level action. @@ -17,9 +18,9 @@ been renamed to fit this scheme. The ultimate goal is that the programmer can *guess* a name. -------------------- ------------ -------------------------------------- +=================== ============ ====================================== English word To use Notes -------------------- ------------ -------------------------------------- +=================== ============ ====================================== initialize initT `init` is used to create a value type `T` new newP `new` is used to create a @@ -81,4 +82,4 @@ literal lit string str identifier ident indentation indent -------------------- ------------ -------------------------------------- +=================== ============ ====================================== diff --git a/doc/astspec.txt b/doc/astspec.txt index c49f7bcc2..7a7053a2d 100644 --- a/doc/astspec.txt +++ b/doc/astspec.txt @@ -6,8 +6,7 @@ The AST consists of nodes (``NimNode``) with a variable number of children. Each node has a field named ``kind`` which describes what the node contains: -.. code-block:: nim - + ```nim type NimNodeKind = enum ## kind of a node; only explanatory nnkNone, ## invalid node kind @@ -32,6 +31,7 @@ contains: strVal: string ## the string literal else: sons: seq[NimNode] ## the node's sons (or children) + ``` For the ``NimNode`` type, the ``[]`` operator has been overloaded: ``n[i]`` is ``n``'s ``i``-th child. @@ -50,9 +50,9 @@ A leaf of the AST often corresponds to a terminal symbol in the concrete syntax. Note that the default ``float`` in Nim maps to ``float64`` such that the default AST for a float is ``nnkFloat64Lit`` as below. ------------------ --------------------------------------------- +================= ============================================= Nim expression Corresponding AST ------------------ --------------------------------------------- +================= ============================================= ``42`` ``nnkIntLit(intVal = 42)`` ``42'i8`` ``nnkInt8Lit(intVal = 42)`` ``42'i16`` ``nnkInt16Lit(intVal = 42)`` @@ -72,7 +72,7 @@ Nim expression Corresponding AST ``nil`` ``nnkNilLit()`` ``myIdentifier`` ``nnkIdent(strVal = "myIdentifier")`` ``myIdentifier`` after lookup pass: ``nnkSym(strVal = "myIdentifier", ...)`` ------------------ --------------------------------------------- +================= ============================================= Identifiers are ``nnkIdent`` nodes. After the name lookup pass these nodes get transferred into ``nnkSym`` nodes. @@ -86,17 +86,19 @@ Command call Concrete syntax: -.. code-block:: nim + ```nim echo "abc", "xyz" + ``` AST: -.. code-block:: nim + ```nim nnkCommand( nnkIdent("echo"), nnkStrLit("abc"), nnkStrLit("xyz") ) + ``` Call with ``()`` @@ -104,17 +106,19 @@ Call with ``()`` Concrete syntax: -.. code-block:: nim + ```nim echo("abc", "xyz") + ``` AST: -.. code-block:: nim + ```nim nnkCall( nnkIdent("echo"), nnkStrLit("abc"), nnkStrLit("xyz") ) + ``` Infix operator call @@ -122,29 +126,32 @@ Infix operator call Concrete syntax: -.. code-block:: nim + ```nim "abc" & "xyz" + ``` AST: -.. code-block:: nim + ```nim nnkInfix( nnkIdent("&"), nnkStrLit("abc"), nnkStrLit("xyz") ) + ``` Note that with multiple infix operators, the command is parsed by operator precedence. Concrete syntax: -.. code-block:: nim + ```nim 5 + 3 * 4 + ``` AST: -.. code-block:: nim + ```nim nnkInfix( nnkIdent("+"), nnkIntLit(5), @@ -154,6 +161,7 @@ AST: nnkIntLit(4) ) ) + ``` As a side note, if you choose to use infix operators in a prefix form, the AST behaves as a @@ -162,12 +170,13 @@ behaves as a Concrete syntax: -.. code-block:: nim + ```nim `+`(3, 4) + ``` AST: -.. code-block:: nim + ```nim nnkCall( nnkAccQuoted( nnkIdent("+") @@ -175,22 +184,25 @@ AST: nnkIntLit(3), nnkIntLit(4) ) + ``` Prefix operator call -------------------- Concrete syntax: -.. code-block:: nim + ```nim ? "xyz" + ``` AST: -.. code-block:: nim + ```nim nnkPrefix( nnkIdent("?"), nnkStrLit("abc") ) + ``` Postfix operator call @@ -201,16 +213,18 @@ Postfix operator call Concrete syntax: -.. code-block:: nim + ```nim identifier* + ``` AST: -.. code-block:: nim + ```nim nnkPostfix( nnkIdent("*"), nnkIdent("identifier") ) + ``` Call with named arguments @@ -218,12 +232,13 @@ Call with named arguments Concrete syntax: -.. code-block:: nim + ```nim writeLine(file=stdout, "hallo") + ``` AST: -.. code-block:: nim + ```nim nnkCall( nnkIdent("writeLine"), nnkExprEqExpr( @@ -232,6 +247,7 @@ AST: ), nnkStrLit("hallo") ) + ``` Call with raw string literal ---------------------------- @@ -242,29 +258,33 @@ This is used, for example, in the ``bindSym`` examples Concrete syntax: -.. code-block:: nim + ```nim echo"abc" + ``` AST: -.. code-block:: nim + ```nim nnkCallStrLit( nnkIdent("echo"), nnkRStrLit("hello") ) + ``` Dereference operator ``[]`` --------------------------- Concrete syntax: -.. code-block:: nim + ```nim x[] + ``` AST: -.. code-block:: nim + ```nim nnkDerefExpr(nnkIdent("x")) + ``` Addr operator @@ -272,13 +292,15 @@ Addr operator Concrete syntax: -.. code-block:: nim + ```nim addr(x) + ``` AST: -.. code-block:: nim + ```nim nnkAddr(nnkIdent("x")) + ``` Cast operator @@ -286,13 +308,15 @@ Cast operator Concrete syntax: -.. code-block:: nim + ```nim cast[T](x) + ``` AST: -.. code-block:: nim + ```nim nnkCast(nnkIdent("T"), nnkIdent("x")) + ``` Object access operator ``.`` @@ -300,13 +324,15 @@ Object access operator ``.`` Concrete syntax: -.. code-block:: nim + ```nim x.y + ``` AST: -.. code-block:: nim + ```nim nnkDotExpr(nnkIdent("x"), nnkIdent("y")) + ``` If you use Nim's flexible calling syntax (as in ``x.len()``), the result is the same as above but wrapped in an ``nnkCall``. @@ -317,31 +343,77 @@ Array access operator ``[]`` Concrete syntax: -.. code-block:: nim + ```nim x[y] + ``` AST: -.. code-block:: nim + ```nim nnkBracketExpr(nnkIdent("x"), nnkIdent("y")) + ``` Parentheses ----------- -Parentheses for affecting operator precedence or tuple construction -are built with the ``nnkPar`` node. +Parentheses for affecting operator precedence use the ``nnkPar`` node. + +Concrete syntax: + + ```nim + (a + b) * c + ``` + +AST: + + ```nim + nnkInfix(nnkIdent("*"), + nnkPar( + nnkInfix(nnkIdent("+"), nnkIdent("a"), nnkIdent("b"))), + nnkIdent("c")) + ``` + +Tuple Constructors +------------------ + +Nodes for tuple construction are built with the ``nnkTupleConstr`` node. Concrete syntax: -.. code-block:: nim - (1, 2, (3)) + ```nim + (1, 2, 3) + (a: 1, b: 2, c: 3) + () + ``` AST: -.. code-block:: nim - nnkPar(nnkIntLit(1), nnkIntLit(2), nnkPar(nnkIntLit(3))) + ```nim + nnkTupleConstr(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3)) + nnkTupleConstr( + nnkExprColonExpr(nnkIdent("a"), nnkIntLit(1)), + nnkExprColonExpr(nnkIdent("b"), nnkIntLit(2)), + nnkExprColonExpr(nnkIdent("c"), nnkIntLit(3))) + nnkTupleConstr() + ``` + +Since the one tuple would be syntactically identical to parentheses +with an expression in them, the parser expects a trailing comma for +them. For tuple constructors with field names, this is not necessary. + + ```nim + (1,) + (a: 1) + ``` + +AST: + ```nim + nnkTupleConstr(nnkIntLit(1)) + nnkTupleConstr( + nnkExprColonExpr(nnkIdent("a"), nnkIntLit(1))) + ``` Curly braces ------------ @@ -350,28 +422,32 @@ Curly braces are used as the set constructor. Concrete syntax: -.. code-block:: nim + ```nim {1, 2, 3} + ``` AST: -.. code-block:: nim + ```nim nnkCurly(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3)) + ``` When used as a table constructor, the syntax is different. Concrete syntax: -.. code-block:: nim + ```nim {a: 3, b: 5} + ``` AST: -.. code-block:: nim + ```nim nnkTableConstr( nnkExprColonExpr(nnkIdent("a"), nnkIntLit(3)), nnkExprColonExpr(nnkIdent("b"), nnkIntLit(5)) ) + ``` Brackets @@ -381,13 +457,15 @@ Brackets are used as the array constructor. Concrete syntax: -.. code-block:: nim + ```nim [1, 2, 3] + ``` AST: -.. code-block:: nim + ```nim nnkBracket(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3)) + ``` Ranges @@ -399,21 +477,23 @@ AST, construction with ``..`` as an infix operator should be used instead. Concrete syntax: -.. code-block:: nim + ```nim 1..3 + ``` AST: -.. code-block:: nim + ```nim nnkInfix( nnkIdent(".."), nnkIntLit(1), nnkIntLit(3) ) + ``` Example code: -.. code-block:: nim + ```nim macro genRepeatEcho() = result = newNimNode(nnkStmtList) @@ -432,6 +512,7 @@ Example code: # 3 # 3 # 3 + ``` If expression @@ -441,17 +522,19 @@ The representation of the ``if`` expression is subtle, but easy to traverse. Concrete syntax: -.. code-block:: nim + ```nim if cond1: expr1 elif cond2: expr2 else: expr3 + ``` AST: -.. code-block:: nim + ```nim nnkIfExpr( nnkElifExpr(cond1, expr1), nnkElifExpr(cond2, expr2), nnkElseExpr(expr3) ) + ``` Documentation Comments ---------------------- @@ -462,19 +545,21 @@ comments are ignored. Concrete syntax: -.. code-block:: nim + ```nim ## This is a comment ## This is part of the first comment stmt1 ## Yet another + ``` AST: -.. code-block:: nim + ```nim nnkCommentStmt() # only appears once for the first two lines! stmt1 nnkCommentStmt() # another nnkCommentStmt because there is another comment # (separate from the first) + ``` Pragmas ------- @@ -485,30 +570,33 @@ objects, but the standalone ``emit`` pragma shows the basics with the AST. Concrete syntax: -.. code-block:: nim + ```nim {.emit: "#include <stdio.h>".} + ``` AST: -.. code-block:: nim + ```nim nnkPragma( nnkExprColonExpr( nnkIdent("emit"), nnkStrLit("#include <stdio.h>") # the "argument" ) ) + ``` As many ``nnkIdent`` appear as there are pragmas between ``{..}``. Note that the declaration of new pragmas is essentially the same: Concrete syntax: -.. code-block:: nim + ```nim {.pragma: cdeclRename, cdecl.} + ``` AST: -.. code-block:: nim + ```nim nnkPragma( nnkExprColonExpr( nnkIdent("pragma"), # this is always first when declaring a new pragma @@ -516,6 +604,7 @@ AST: ), nnkIdent("cdecl") ) + ``` Statements ========== @@ -528,7 +617,7 @@ there is no ``else`` branch, no ``nnkElse`` child exists. Concrete syntax: -.. code-block:: nim + ```nim if cond1: stmt1 elif cond2: @@ -537,16 +626,18 @@ Concrete syntax: stmt3 else: stmt4 + ``` AST: -.. code-block:: nim + ```nim nnkIfStmt( nnkElifBranch(cond1, stmt1), nnkElifBranch(cond2, stmt2), nnkElifBranch(cond3, stmt3), nnkElse(stmt4) ) + ``` When statement @@ -560,13 +651,15 @@ Assignment Concrete syntax: -.. code-block:: nim + ```nim x = 42 + ``` AST: -.. code-block:: nim + ```nim nnkAsgn(nnkIdent("x"), nnkIntLit(42)) + ``` This is not the syntax for assignment when combined with ``var``, ``let``, or ``const``. @@ -576,15 +669,17 @@ Statement list Concrete syntax: -.. code-block:: nim + ```nim stmt1 stmt2 stmt3 + ``` AST: -.. code-block:: nim + ```nim nnkStmtList(stmt1, stmt2, stmt3) + ``` Case statement @@ -592,7 +687,7 @@ Case statement Concrete syntax: -.. code-block:: nim + ```nim case expr1 of expr2, expr3..expr4: stmt1 @@ -602,10 +697,11 @@ Concrete syntax: stmt3 else: stmt4 + ``` AST: -.. code-block:: nim + ```nim nnkCaseStmt( expr1, nnkOfBranch(expr2, nnkRange(expr3, expr4), stmt1), @@ -613,6 +709,7 @@ AST: nnkElifBranch(cond1, stmt3), nnkElse(stmt4) ) + ``` The ``nnkElifBranch`` and ``nnkElse`` parts may be missing. @@ -622,14 +719,16 @@ While statement Concrete syntax: -.. code-block:: nim + ```nim while expr1: stmt1 + ``` AST: -.. code-block:: nim + ```nim nnkWhileStmt(expr1, stmt1) + ``` For statement @@ -637,14 +736,16 @@ For statement Concrete syntax: -.. code-block:: nim + ```nim for ident1, ident2 in expr1: stmt1 + ``` AST: -.. code-block:: nim + ```nim nnkForStmt(ident1, ident2, expr1, stmt1) + ``` Try statement @@ -652,7 +753,7 @@ Try statement Concrete syntax: -.. code-block:: nim + ```nim try: stmt1 except e1, e2: @@ -663,10 +764,11 @@ Concrete syntax: stmt4 finally: stmt5 + ``` AST: -.. code-block:: nim + ```nim nnkTryStmt( stmt1, nnkExceptBranch(e1, e2, stmt2), @@ -674,6 +776,7 @@ AST: nnkExceptBranch(stmt4), nnkFinally(stmt5) ) + ``` Return statement @@ -681,13 +784,15 @@ Return statement Concrete syntax: -.. code-block:: nim + ```nim return expr1 + ``` AST: -.. code-block:: nim + ```nim nnkReturnStmt(expr1) + ``` Yield statement @@ -695,8 +800,9 @@ Yield statement Like ``return``, but with ``nnkYieldStmt`` kind. -.. code-block:: nim + ```nim nnkYieldStmt(expr1) + ``` Discard statement @@ -704,8 +810,9 @@ Discard statement Like ``return``, but with ``nnkDiscardStmt`` kind. -.. code-block:: nim + ```nim nnkDiscardStmt(expr1) + ``` Continue statement @@ -713,26 +820,30 @@ Continue statement Concrete syntax: -.. code-block:: nim + ```nim continue + ``` AST: -.. code-block:: nim + ```nim nnkContinueStmt() + ``` Break statement --------------- Concrete syntax: -.. code-block:: nim + ```nim break otherLocation + ``` AST: -.. code-block:: nim + ```nim nnkBreakStmt(nnkIdent("otherLocation")) + ``` If ``break`` is used without a jump-to location, ``nnkEmpty`` replaces ``nnkIdent``. @@ -741,13 +852,15 @@ Block statement Concrete syntax: -.. code-block:: nim + ```nim block name: + ``` AST: -.. code-block:: nim + ```nim nnkBlockStmt(nnkIdent("name"), nnkStmtList(...)) + ``` A ``block`` doesn't need an name, in which case ``nnkEmpty`` is used. @@ -756,18 +869,20 @@ Asm statement Concrete syntax: -.. code-block:: nim + ```nim asm """ some asm """ + ``` AST: -.. code-block:: nim + ```nim nnkAsmStmt( nnkEmpty(), # for pragmas nnkTripleStrLit("some asm"), ) + ``` Import section -------------- @@ -777,37 +892,42 @@ on what keywords are present. Let's start with the simplest form. Concrete syntax: -.. code-block:: nim - import math + ```nim + import std/math + ``` AST: -.. code-block:: nim + ```nim nnkImportStmt(nnkIdent("math")) + ``` With ``except``, we get ``nnkImportExceptStmt``. Concrete syntax: -.. code-block:: nim - import math except pow + ```nim + import std/math except pow + ``` AST: -.. code-block:: nim + ```nim nnkImportExceptStmt(nnkIdent("math"),nnkIdent("pow")) + ``` -Note that ``import math as m`` does not use a different node; rather, +Note that ``import std/math as m`` does not use a different node; rather, we use ``nnkImportStmt`` with ``as`` as an infix operator. Concrete syntax: -.. code-block:: nim - import strutils as su + ```nim + import std/strutils as su + ``` AST: -.. code-block:: nim + ```nim nnkImportStmt( nnkInfix( nnkIdent("as"), @@ -815,6 +935,7 @@ AST: nnkIdent("su") ) ) + ``` From statement -------------- @@ -823,15 +944,17 @@ If we use ``from ... import``, the result is different, too. Concrete syntax: -.. code-block:: nim - from math import pow + ```nim + from std/math import pow + ``` AST: -.. code-block:: nim + ```nim nnkFromStmt(nnkIdent("math"), nnkIdent("pow")) + ``` -Using ``from math as m import pow`` works identically to the ``as`` modifier +Using ``from std/math as m import pow`` works identically to the ``as`` modifier with the ``import`` statement, but wrapped in ``nnkFromStmt``. Export statement @@ -842,26 +965,30 @@ the ``export`` syntax is pretty straightforward. Concrete syntax: -.. code-block:: nim + ```nim export unsigned + ``` AST: -.. code-block:: nim + ```nim nnkExportStmt(nnkIdent("unsigned")) + ``` Similar to the ``import`` statement, the AST is different for ``export ... except``. Concrete syntax: -.. code-block:: nim + ```nim export math except pow # we're going to implement our own exponentiation + ``` AST: -.. code-block:: nim + ```nim nnkExportExceptStmt(nnkIdent("math"),nnkIdent("pow")) + ``` Include statement ----------------- @@ -870,25 +997,28 @@ Like a plain ``import`` statement but with ``nnkIncludeStmt``. Concrete syntax: -.. code-block:: nim + ```nim include blocks + ``` AST: -.. code-block:: nim + ```nim nnkIncludeStmt(nnkIdent("blocks")) + ``` Var section ----------- Concrete syntax: -.. code-block:: nim + ```nim var a = 3 + ``` AST: -.. code-block:: nim + ```nim nnkVarSection( nnkIdentDefs( nnkIdent("a"), @@ -896,6 +1026,7 @@ AST: nnkIntLit(3), ) ) + ``` Note that either the second or third (or both) parameters above must exist, as the compiler needs to know the type somehow (which it can infer from @@ -913,12 +1044,13 @@ This is equivalent to ``var``, but with ``nnkLetSection`` rather than Concrete syntax: -.. code-block:: nim + ```nim let a = 3 + ``` AST: -.. code-block:: nim + ```nim nnkLetSection( nnkIdentDefs( nnkIdent("a"), @@ -926,18 +1058,20 @@ AST: nnkIntLit(3), ) ) + ``` Const section ------------- Concrete syntax: -.. code-block:: nim + ```nim const a = 3 + ``` AST: -.. code-block:: nim + ```nim nnkConstSection( nnkConstDef( # not nnkConstDefs! nnkIdent("a"), @@ -945,6 +1079,7 @@ AST: nnkIntLit(3), # required in a const declaration! ) ) + ``` Type section ------------ @@ -954,12 +1089,13 @@ and ``const``. Concrete syntax: -.. code-block:: nim + ```nim type A = int + ``` AST: -.. code-block:: nim + ```nim nnkTypeSection( nnkTypeDef( nnkIdent("A"), @@ -967,18 +1103,20 @@ AST: nnkIdent("int") ) ) + ``` Declaring ``distinct`` types is similar, with the last ``nnkIdent`` wrapped in ``nnkDistinctTy``. Concrete syntax: -.. code-block:: nim + ```nim type MyInt = distinct int + ``` AST: -.. code-block:: nim + ```nim # ... nnkTypeDef( nnkIdent("MyInt"), @@ -987,17 +1125,19 @@ AST: nnkIdent("int") ) ) + ``` If a type section uses generic parameters, they are treated here: Concrete syntax: -.. code-block:: nim + ```nim type A[T] = expr1 + ``` AST: -.. code-block:: nim + ```nim nnkTypeSection( nnkTypeDef( nnkIdent("A"), @@ -1012,6 +1152,7 @@ AST: expr1, ) ) + ``` Note that not all ``nnkTypeDef`` utilize ``nnkIdent`` as their parameter. One of the most common uses of type declarations @@ -1019,12 +1160,13 @@ is to work with objects. Concrete syntax: -.. code-block:: nim + ```nim type IO = object of RootObj + ``` AST: -.. code-block:: nim + ```nim # ... nnkTypeDef( nnkIdent("IO"), @@ -1037,29 +1179,39 @@ AST: nnkEmpty() ) ) + ``` Nim's object syntax is rich. Let's take a look at an involved example in its entirety to see some of the complexities. Concrete syntax: -.. code-block:: nim - type Obj[T] = object {.inheritable.} + ```nim + type Obj[T] {.inheritable.} = object name: string case isFat: bool of true: m: array[100_000, T] of false: m: array[10, T] + ``` AST: -.. code-block:: nim + ```nim # ... + nnkPragmaExpr( + nnkIdent("Obj"), + nnkPragma(nnkIdent("inheritable")) + ), + nnkGenericParams( + nnkIdentDefs( + nnkIdent("T"), + nnkEmpty(), + nnkEmpty()) + ), nnkObjectTy( - nnkPragma( - nnkIdent("inheritable") - ), + nnkEmpty(), nnkEmpty(), nnkRecList( # list of object parameters nnkIdentDefs( @@ -1103,55 +1255,61 @@ AST: ) ) ) + ``` Using an ``enum`` is similar to using an ``object``. Concrete syntax: -.. code-block:: nim + ```nim type X = enum First + ``` AST: -.. code-block:: nim + ```nim # ... nnkEnumTy( nnkEmpty(), nnkIdent("First") # you need at least one nnkIdent or the compiler complains ) + ``` The usage of ``concept`` (experimental) is similar to objects. Concrete syntax: -.. code-block:: nim + ```nim type Con = concept x,y,z (x & y & z) is string + ``` AST: -.. code-block:: nim + ```nim # ... nnkTypeClassTy( # note this isn't nnkConceptTy! - nnkArglist( + nnkArgList( # ... idents for x, y, z ) # ... ) + ``` Static types, like ``static[int]``, use ``nnkIdent`` wrapped in ``nnkStaticTy``. Concrete syntax: -.. code-block:: nim + ```nim type A[T: static[int]] = object + ``` AST: -.. code-block:: nim + ```nim # ... within nnkGenericParams nnkIdentDefs( nnkIdent("T"), @@ -1161,13 +1319,14 @@ AST: nnkEmpty() ) # ... + ``` In general, declaring types mirrors this syntax (i.e., ``nnkStaticTy`` for ``static``, etc.). Examples follow (exceptions marked by ``*``): -------------- --------------------------------------------- +============= ============================================= Nim type Corresponding AST -------------- --------------------------------------------- +============= ============================================= ``static`` ``nnkStaticTy`` ``tuple`` ``nnkTupleTy`` ``var`` ``nnkVarTy`` @@ -1180,7 +1339,7 @@ Nim type Corresponding AST ``proc`` ``nnkProcTy`` ``iterator`` ``nnkIteratorTy`` ``object`` ``nnkObjectTy`` -------------- --------------------------------------------- +============= ============================================= Take special care when declaring types as ``proc``. The behavior is similar to ``Procedure declaration``, below, but does not treat ``nnkGenericParams``. @@ -1188,12 +1347,13 @@ Generic parameters are treated in the type, not the ``proc`` itself. Concrete syntax: -.. code-block:: nim - type MyProc[T] = proc(x: T) + ```nim + type MyProc[T] = proc(x: T) {.nimcall.} + ``` AST: -.. code-block:: nim + ```nim # ... nnkTypeDef( nnkIdent("MyProc"), @@ -1203,38 +1363,75 @@ AST: nnkProcTy( # behaves like a procedure declaration from here on nnkFormalParams( # ... - ) + ), + nnkPragma(nnkIdent("nimcall")) ) ) + ``` The same syntax applies to ``iterator`` (with ``nnkIteratorTy``), but *does not* apply to ``converter`` or ``template``. +Type class versions of these nodes generally share the same node kind but +without any child nodes. The ``tuple`` type class is represented by +``nnkTupleClassTy``, while a ``proc`` or ``iterator`` type class with pragmas +has an ``nnkEmpty`` node in place of the ``nnkFormalParams`` node of a +concrete ``proc`` or ``iterator`` type node. + + ```nim + type TypeClass = proc {.nimcall.} | ref | tuple + ``` + +AST: + + ```nim + nnkTypeDef( + nnkIdent("TypeClass"), + nnkEmpty(), + nnkInfix( + nnkIdent("|"), + nnkProcTy( + nnkEmpty(), + nnkPragma(nnkIdent("nimcall")) + ), + nnkInfix( + nnkIdent("|"), + nnkRefTy(), + nnkTupleClassTy() + ) + ) + ) + ``` + Mixin statement --------------- Concrete syntax: -.. code-block:: nim + ```nim mixin x + ``` AST: -.. code-block:: nim + ```nim nnkMixinStmt(nnkIdent("x")) + ``` Bind statement -------------- Concrete syntax: -.. code-block:: nim + ```nim bind x + ``` AST: -.. code-block:: nim + ```nim nnkBindStmt(nnkIdent("x")) + ``` Procedure declaration --------------------- @@ -1244,12 +1441,13 @@ a feel for how procedure calls are broken down. Concrete syntax: -.. code-block:: nim + ```nim proc hello*[T: SomeInteger](x: int = 3, y: float32): int {.inline.} = discard + ``` AST: -.. code-block:: nim + ```nim nnkProcDef( nnkPostfix(nnkIdent("*"), nnkIdent("hello")), # the exported proc name nnkEmpty(), # patterns for term rewriting in templates and macros (not procs) @@ -1277,6 +1475,7 @@ AST: nnkEmpty(), # reserved slot for future use nnkStmtList(nnkDiscardStmt(nnkEmpty())) # the meat of the proc ) + ``` There is another consideration. Nim has flexible type identification for its procs. Even though ``proc(a: int, b: int)`` and ``proc(a, b: int)`` @@ -1284,12 +1483,13 @@ are equivalent in the code, the AST is a little different for the latter. Concrete syntax: -.. code-block:: nim + ```nim proc(a, b: int) + ``` AST: -.. code-block:: nim + ```nim # ...AST as above... nnkFormalParams( nnkEmpty(), # no return here @@ -1301,24 +1501,27 @@ AST: ) ), # ... + ``` When a procedure uses the special ``var`` type return variable, the result is different from that of a var section. Concrete syntax: -.. code-block:: nim + ```nim proc hello(): var int + ``` AST: -.. code-block:: nim + ```nim # ... nnkFormalParams( nnkVarTy( nnkIdent("int") ) ) + ``` Iterator declaration -------------------- @@ -1328,17 +1531,19 @@ replacing ``nnkProcDef``. Concrete syntax: -.. code-block:: nim + ```nim iterator nonsense[T](x: seq[T]): float {.closure.} = ... + ``` AST: -.. code-block:: nim + ```nim nnkIteratorDef( nnkIdent("nonsense"), nnkEmpty(), ... ) + ``` Converter declaration --------------------- @@ -1347,16 +1552,18 @@ A converter is similar to a proc. Concrete syntax: -.. code-block:: nim + ```nim converter toBool(x: float): bool + ``` AST: -.. code-block:: nim + ```nim nnkConverterDef( nnkIdent("toBool"), # ... ) + ``` Template declaration -------------------- @@ -1369,12 +1576,13 @@ the ``nnkEmpty()`` as the second argument to ``nnkProcDef`` and Concrete syntax: -.. code-block:: nim + ```nim template optOpt{expr1}(a: int): int + ``` AST: -.. code-block:: nim + ```nim nnkTemplateDef( nnkIdent("optOpt"), nnkStmtList( # instead of nnkEmpty() @@ -1382,6 +1590,7 @@ AST: ), # follows like a proc or iterator ) + ``` If the template does not have types for its parameters, the type identifiers inside ``nnkFormalParams`` just becomes ``nnkEmpty``. @@ -1395,8 +1604,9 @@ Macros behave like templates, but ``nnkTemplateDef`` is replaced with Hidden Standard Conversion -------------------------- -.. code-block:: nim + ```nim var f: float = 1 + ``` The type of "f" is ``float`` but the type of "1" is actually ``int``. Inserting ``int`` into a ``float`` is a type error. Nim inserts the ``nnkHiddenStdConv`` diff --git a/doc/backends.rst b/doc/backends.md index 896b0f834..9f0c54835 100644 --- a/doc/backends.rst +++ b/doc/backends.md @@ -1,5 +1,3 @@ -.. default-role:: code - ================================ Nim Backend Integration ================================ @@ -7,27 +5,33 @@ :Author: Puppet Master :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst +.. no syntax highlighting here by default: + .. contents:: - "Heresy grows from idleness." -- Unknown. + +> "Heresy grows from idleness." -- Unknown. Introduction ============ -The `Nim Compiler User Guide <nimc.html>`_ documents the typical -compiler invocation, using the `compile` or `c` command to transform a -`.nim` file into one or more `.c` files which are then compiled with the +The [Nim Compiler User Guide](nimc.html) documents the typical +compiler invocation, using the `compile`:option: +or `c`:option: command to transform a +``.nim`` file into one or more ``.c`` files which are then compiled with the platform's C compiler into a static binary. However, there are other commands to compile to C++, Objective-C, or JavaScript. This document tries to concentrate in a single place all the backend and interfacing options. The Nim compiler supports mainly two backend families: the C, C++ and -Objective-C targets and the JavaScript target. `The C like targets -<#backends-the-c-like-targets>`_ creates source files that can be compiled -into a library or a final executable. `The JavaScript target -<#backends-the-javascript-target>`_ can generate a `.js` file which you -reference from an HTML file or create a `standalone Node.js program -<http://nodejs.org>`_. +Objective-C targets and the JavaScript target. [The C like targets]( +#backends-the-c-like-targets) creates source files that can be compiled +into a library or a final executable. [The JavaScript target]( +#backends-the-javascript-target) can generate a ``.js`` file which you +reference from an HTML file or create a [standalone Node.js program]( +http://nodejs.org). On top of generating libraries or standalone applications, Nim offers bidirectional interfacing with the backend targets through generic and @@ -42,35 +46,37 @@ The C like targets The commands to compile to either C, C++ or Objective-C are: - //compileToC, cc compile project with C code generator - //compileToCpp, cpp compile project to C++ code - //compileToOC, objc compile project to Objective C code +//compileToC, cc compile project with C code generator +//compileToCpp, cpp compile project to C++ code +//compileToOC, objc compile project to Objective C code The most significant difference between these commands is that if you look -into the `nimcache` directory you will find `.c`, `.cpp` or `.m` +into the ``nimcache`` directory you will find ``.c``, ``.cpp`` or ``.m`` files, other than that all of them will produce a native binary for your -project. This allows you to take the generated code and place it directly +project. This allows you to take the generated code and place it directly into a project using any of these languages. Here are some typical command- -line invocations:: +line invocations: - $ nim c hallo.nim - $ nim cpp hallo.nim - $ nim objc hallo.nim + ```cmd + nim c hallo.nim + nim cpp hallo.nim + nim objc hallo.nim + ``` The compiler commands select the target backend, but if needed you can -`specify additional switches for cross-compilation -<nimc.html#crossminuscompilation>`_ to select the target CPU, operative system +[specify additional switches for cross-compilation]( +nimc.html#crossminuscompilation) to select the target CPU, operative system or compiler/linker commands. The JavaScript target --------------------- -Nim can also generate `JavaScript`:idx: code through the `js` command. +Nim can also generate `JavaScript`:idx: code through the `js`:option: command. Nim targets JavaScript 1.5 which is supported by any widely used browser. Since JavaScript does not have a portable means to include another module, -Nim just generates a long `.js` file. +Nim just generates a long ``.js`` file. Features or modules that the JavaScript platform does not support are not available. This includes: @@ -83,22 +89,24 @@ available. This includes: * some modules of the standard library * proper 64-bit integer arithmetic -To compensate, the standard library has modules `catered to the JS backend -<lib.html#pure-libraries-modules-for-js-backend>`_ +To compensate, the standard library has modules [catered to the JS backend]( +lib.html#pure-libraries-modules-for-js-backend) and more support will come in the future (for instance, Node.js bindings to get OS info). -To compile a Nim module into a `.js` file use the `js` command; the -default is a `.js` file that is supposed to be referenced in an `.html` +To compile a Nim module into a ``.js`` file use the `js`:option: command; the +default is a ``.js`` file that is supposed to be referenced in an ``.html`` file. However, you can also run the code with `nodejs`:idx: -(`<http://nodejs.org>`_):: +(http://nodejs.org): + ```cmd nim js -d:nodejs -r examples/hallo.nim + ``` If you experience errors saying that `globalThis` is not defined, be sure to run a recent version of Node.js (at least 12.0). - + Interfacing =========== @@ -112,14 +120,14 @@ component?). Nim code calling the backend ---------------------------- -Nim code can interface with the backend through the `Foreign function -interface <manual.html#foreign-function-interface>`_ mainly through the -`importc pragma <manual.html#foreign-function-interface-importc-pragma>`_. +Nim code can interface with the backend through the [Foreign function +interface](manual.html#foreign-function-interface) mainly through the +[importc pragma](manual.html#foreign-function-interface-importc-pragma). The `importc` pragma is the *generic* way of making backend symbols available -in Nim and is available in all the target backends (JavaScript too). The C++ -or Objective-C backends have their respective `ImportCpp -<manual.html#implementation-specific-pragmas-importcpp-pragma>`_ and -`ImportObjC <manual.html#implementation-specific-pragmas-importobjc-pragma>`_ +in Nim and is available in all the target backends (JavaScript too). The C++ +or Objective-C backends have their respective [ImportCpp]( +manual.html#implementation-specific-pragmas-importcpp-pragma) and +[ImportObjC](manual.html#implementation-specific-pragmas-importobjc-pragma) pragmas to call methods from classes. Whenever you use any of these pragmas you need to integrate native code into @@ -131,66 +139,67 @@ However, for the C like targets you need to link external code either statically or dynamically. The preferred way of integrating native code is to use dynamic linking because it allows you to compile Nim programs without the need for having the related development libraries installed. This is done -through the `dynlib pragma for import -<manual.html#foreign-function-interface-dynlib-pragma-for-import>`_, though -more specific control can be gained using the `dynlib module <dynlib.html>`_. +through the [dynlib pragma for import]( +manual.html#foreign-function-interface-dynlib-pragma-for-import), though +more specific control can be gained using the [dynlib module](dynlib.html). -The `dynlibOverride <nimc.html#dynliboverride>`_ command line switch allows +The [dynlibOverride](nimc.html#dynliboverride) command line switch allows to avoid dynamic linking if you need to statically link something instead. -Nim wrappers designed to statically link source files can use the `compile -pragma <manual.html#implementation-specific-pragmas-compile-pragma>`_ if +Nim wrappers designed to statically link source files can use the [compile +pragma](manual.html#implementation-specific-pragmas-compile-pragma) if there are few sources or providing them along the Nim code is easier than using a system library. Libraries installed on the host system can be linked in with -the `PassL pragma <manual.html#implementation-specific-pragmas-passl-pragma>`_. +the [PassL pragma](manual.html#implementation-specific-pragmas-passl-pragma). -To wrap native code, take a look at the `c2nim tool <https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst>`_ which helps +To wrap native code, take a look at the [c2nim tool]( +https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst) which helps with the process of scanning and transforming header files into a Nim interface. -C invocation example -~~~~~~~~~~~~~~~~~~~~ +### C invocation example -Create a `logic.c` file with the following content: +Create a ``logic.c`` file with the following content: -.. code-block:: c + ```c int addTwoIntegers(int a, int b) { return a + b; } + ``` -Create a `calculator.nim` file with the following content: - -.. code-block:: nim +Create a ``calculator.nim`` file with the following content: + ```nim {.compile: "logic.c".} proc addTwoIntegers(a, b: cint): cint {.importc.} when isMainModule: echo addTwoIntegers(3, 7) + ``` -With these two files in place, you can run `nim c -r calculator.nim` and -the Nim compiler will compile the `logic.c` file in addition to -`calculator.nim` and link both into an executable, which outputs `10` when +With these two files in place, you can run `nim c -r calculator.nim`:cmd: and +the Nim compiler will compile the ``logic.c`` file in addition to +``calculator.nim`` and link both into an executable, which outputs `10` when run. Another way to link the C file statically and get the same effect would -be to remove the line with the `compile` pragma and run the following typical -Unix commands:: +be to remove the line with the `compile` pragma and run the following +typical Unix commands: - $ gcc -c logic.c - $ ar rvs mylib.a logic.o - $ nim c --passL:mylib.a -r calculator.nim + ```cmd + gcc -c logic.c + ar rvs mylib.a logic.o + nim c --passL:mylib.a -r calculator.nim + ``` -Just like in this example we pass the path to the `mylib.a` library (and we -could as well pass `logic.o`) we could be passing switches to link any other +Just like in this example we pass the path to the ``mylib.a`` library (and we +could as well pass ``logic.o``) we could be passing switches to link any other static C library. -JavaScript invocation example -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Create a `host.html` file with the following content: +### JavaScript invocation example -.. code-block:: +Create a ``host.html`` file with the following content: + ``` <html><body> <script type="text/javascript"> function addTwoIntegers(a, b) @@ -200,30 +209,31 @@ Create a `host.html` file with the following content: </script> <script type="text/javascript" src="calculator.js"></script> </body></html> + ``` -Create a `calculator.nim` file with the following content (or reuse the one +Create a ``calculator.nim`` file with the following content (or reuse the one from the previous section): -.. code-block:: nim - + ```nim proc addTwoIntegers(a, b: int): int {.importc.} when isMainModule: echo addTwoIntegers(3, 7) + ``` Compile the Nim code to JavaScript with `nim js -o:calculator.js -calculator.nim` and open `host.html` in a browser. If the browser supports +calculator.nim`:cmd: and open ``host.html`` in a browser. If the browser supports javascript, you should see the value `10` in the browser's console. Use the -`dom module <dom.html>`_ for specific DOM querying and modification procs -or take a look at `karax <https://github.com/pragmagic/karax>`_ for how to +[dom module](dom.html) for specific DOM querying and modification procs +or take a look at [karax](https://github.com/pragmagic/karax) for how to develop browser-based applications. Backend code calling Nim ------------------------ -Backend code can interface with Nim code exposed through the `exportc -pragma <manual.html#foreign-function-interface-exportc-pragma>`_. The +Backend code can interface with Nim code exposed through the [exportc +pragma](manual.html#foreign-function-interface-exportc-pragma). The `exportc` pragma is the *generic* way of making Nim symbols available to the backends. By default, the Nim compiler will mangle all the Nim symbols to avoid any name collision, so the most significant thing the `exportc` pragma @@ -237,32 +247,32 @@ Also, C code requires you to specify a forward declaration for functions or the compiler will assume certain types for the return value and parameters which will likely make your program crash at runtime. -The Nim compiler can generate a C interface header through the `--header` -command-line switch. The generated header will contain all the exported -symbols and the `NimMain` proc which you need to call before any other -Nim code. +The name `NimMain` can be influenced via the `--nimMainPrefix:prefix` switch. +Use `--nimMainPrefix:MyLib` and the function to call is named `MyLibNimMain`. +When compiling to static or dynamic libraries, they don't call destructors of global variables as normal Nim programs would do. A C API `NimDestroyGlobals` is provided to call these global destructors. -Nim invocation example from C -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Create a `fib.nim` file with the following content: +### Nim invocation example from C -.. code-block:: nim +Create a ``fib.nim`` file with the following content: + ```nim proc fib(a: cint): cint {.exportc.} = if a <= 2: result = 1 else: result = fib(a - 1) + fib(a - 2) + ``` -Create a `maths.c` file with the following content: +Create a ``maths.c`` file with the following content: -.. code-block:: c - - #include "fib.h" + ```c #include <stdio.h> + int fib(int a); + void NimMain(); + int main(void) { NimMain(); @@ -270,63 +280,66 @@ Create a `maths.c` file with the following content: printf("Fib of %d is %d\n", f, fib(f)); return 0; } + ``` Now you can run the following Unix like commands to first generate C sources from the Nim code, then link them into a static binary along your main C -program:: +program: - $ nim c --noMain --noLinking --header:fib.h fib.nim - $ gcc -o m -I$HOME/.cache/nim/fib_d -Ipath/to/nim/lib $HOME/.cache/nim/fib_d/*.c maths.c + ```cmd + nim c --noMain --noLinking fib.nim + gcc -o m -I$HOME/.cache/nim/fib_d -Ipath/to/nim/lib $HOME/.cache/nim/fib_d/*.c maths.c + ``` The first command runs the Nim compiler with three special options to avoid -generating a `main()` function in the generated files, avoid linking the -object files into a final binary, and explicitly generate a header file for C -integration. All the generated files are placed into the `nimcache` -directory. That's why the next command compiles the `maths.c` source plus -all the `.c` files from `nimcache`. In addition to this path, you also -have to tell the C compiler where to find Nim's `nimbase.h` header file. +generating a `main()`:c: function in the generated files and to avoid linking the +object files into a final binary. All the generated files are placed into the ``nimcache`` +directory. That's why the next command compiles the ``maths.c`` source plus +all the ``.c`` files from ``nimcache``. In addition to this path, you also +have to tell the C compiler where to find Nim's ``nimbase.h`` header file. -Instead of depending on the generation of the individual `.c` files you can -also ask the Nim compiler to generate a statically linked library:: +Instead of depending on the generation of the individual ``.c`` files you can +also ask the Nim compiler to generate a statically linked library: - $ nim c --app:staticLib --noMain --header fib.nim - $ gcc -o m -Inimcache -Ipath/to/nim/lib libfib.nim.a maths.c + ```cmd + nim c --app:staticLib fib.nim + gcc -o m -Inimcache -Ipath/to/nim/lib maths.c libfib.nim.a + ``` The Nim compiler will handle linking the source files generated in the -`nimcache` directory into the `libfib.nim.a` static library, which you can -then link into your C program. Note that these commands are generic and will +``nimcache`` directory into the ``libfib.nim.a`` static library, which you can +then link into your C program. Note that these commands are generic and will vary for each system. For instance, on Linux systems you will likely need to -use `-ldl` too to link in required dlopen functionality. - +use `-ldl`:option: too to link in required dlopen functionality. -Nim invocation example from JavaScript -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Create a `mhost.html` file with the following content: +### Nim invocation example from JavaScript -.. code-block:: +Create a ``mhost.html`` file with the following content: + ``` <html><body> <script type="text/javascript" src="fib.js"></script> <script type="text/javascript"> alert("Fib for 9 is " + fib(9)); </script> </body></html> + ``` -Create a `fib.nim` file with the following content (or reuse the one +Create a ``fib.nim`` file with the following content (or reuse the one from the previous section): -.. code-block:: nim - + ```nim proc fib(a: cint): cint {.exportc.} = if a <= 2: result = 1 else: result = fib(a - 1) + fib(a - 2) + ``` -Compile the Nim code to JavaScript with `nim js -o:fib.js fib.nim` and -open `mhost.html` in a browser. If the browser supports javascript, you -should see an alert box displaying the text `Fib for 9 is 34`. As mentioned +Compile the Nim code to JavaScript with `nim js -o:fib.js fib.nim`:cmd: and +open ``mhost.html`` in a browser. If the browser supports javascript, you +should see an alert box displaying the text ``Fib for 9 is 34``. As mentioned earlier, JavaScript doesn't require an initialization call to `NimMain` or a similar function and you can call the exported Nim proc directly. @@ -337,8 +350,8 @@ Nimcache naming logic The `nimcache`:idx: directory is generated during compilation and will hold either temporary or final files depending on your backend target. The default name for the directory depends on the used backend and on your OS but you can -use the `--nimcache` `compiler switch -<nimc.html#compiler-usage-commandminusline-switches>`_ to change it. +use the `--nimcache`:option: [compiler switch]( +nimc.html#compiler-usage-commandminusline-switches) to change it. Memory management @@ -356,32 +369,22 @@ aware of who controls what to avoid crashing. Strings and C strings --------------------- -The manual mentions that `Nim strings are implicitly convertible to -cstrings <manual.html#types-cstring-type>`_ which makes interaction usually +The manual mentions that [Nim strings are implicitly convertible to +cstrings](manual.html#types-cstring-type) which makes interaction usually painless. Most C functions accepting a Nim string converted to a `cstring` will likely not need to keep this string around and by the time -they return the string won't be needed anymore. However, for the rare cases -where a Nim string has to be preserved and made available to the C backend -as a `cstring`, you will need to manually prevent the string data from being -freed with `GC_ref <system.html#GC_ref,string>`_ and `GC_unref -<system.html#GC_unref,string>`_. +they return the string won't be needed anymore. A similar thing happens with C code invoking Nim code which returns a `cstring`. Consider the following proc: -.. code-block:: nim - + ```nim proc gimme(): cstring {.exportc.} = result = "Hey there C code! " & $rand(100) + ``` -Since Nim's garbage collector is not aware of the C code, once the +Since Nim's reference counting mechanism is not aware of the C code, once the `gimme` proc has finished it can reclaim the memory of the `cstring`. -However, from a practical standpoint, the C code invoking the `gimme` -function directly will be able to use it since Nim's garbage collector has -not had a chance to run *yet*. This gives you enough time to make a copy for -the C side of the program, as calling any further Nim procs *might* trigger -garbage collection making the previously returned string garbage. Or maybe you -are `yourself triggering the collection <gc.html>`_. Custom data types @@ -389,43 +392,15 @@ Custom data types Just like strings, custom data types that are to be shared between Nim and the backend will need careful consideration of who controls who. If you want -to hand a Nim reference to C code, you will need to use `GC_ref -<system.html#GC_ref,ref.T>`_ to mark the reference as used, so it does not get -freed. And for the C backend you will need to expose the `GC_unref -<system.html#GC_unref,ref.T>`_ proc to clean up this memory when it is not +to hand a Nim reference to C code, you will need to use [GC_ref]( +system.html#GC_ref,ref.T) to mark the reference as used, so it does not get +freed. And for the C backend you will need to expose the [GC_unref]( +system.html#GC_unref,ref.T) proc to clean up this memory when it is not required anymore. Again, if you are wrapping a library which *mallocs* and *frees* data structures, you need to expose the appropriate *free* function to Nim so you can clean it up. And of course, once cleaned you should avoid accessing it from Nim (or C for that matter). Typically C data structures have their own -`malloc_structure` and `free_structure` specific functions, so wrapping +`malloc_structure`:c: and `free_structure`:c: specific functions, so wrapping these for the Nim side should be enough. - - -Thread coordination -------------------- - -When the `NimMain()` function is called Nim initializes the garbage -collector to the current thread, which is usually the main thread of your -application. If your C code later spawns a different thread and calls Nim -code, the garbage collector will fail to work properly and you will crash. - -As long as you don't use the threadvar emulation Nim uses native thread -variables, of which you get a fresh version whenever you create a thread. You -can then attach a GC to this thread via - -.. code-block:: nim - - system.setupForeignThreadGc() - -It is **not** safe to disable the garbage collector and enable it after the -call from your background thread even if the code you are calling is short -lived. - -Before the thread exits, you should tear down the thread's GC to prevent memory -leaks by calling - -.. code-block:: nim - - system.tearDownForeignThreadGc() diff --git a/doc/basicopt.txt b/doc/basicopt.txt index a2db1c92e..e8133d227 100644 --- a/doc/basicopt.txt +++ b/doc/basicopt.txt @@ -1,10 +1,9 @@ -:: nim command [options] [projectfile] [arguments] Command: //compile, c compile project with default code generator (C) - //r compile to $nimcache/projname, run with [arguments] + //r compile to $nimcache/projname, run with `arguments` using backend specified by `--backend` (default: c) //doc generate the documentation for inputfile for backend specified by `--backend` (default: c) @@ -27,11 +26,11 @@ Options: -a, --assertions:on|off turn assertions on|off --opt:none|speed|size optimize not at all or for speed|size Note: use -d:release for a release build! - --debugger:native Use native debugger (gdb) + --debugger:native use native debugger (gdb) --app:console|gui|lib|staticlib generate a console app|GUI app|DLL|static library -r, --run run the compiled program with given arguments - --eval:cmd evaluates nim code directly; e.g.: `nim --eval:"echo 1"` + --eval:cmd evaluate nim code directly; e.g.: `nim --eval:"echo 1"` defaults to `e` (nimscript) but customizable: `nim r --eval:'for a in stdin.lines: echo a'` --fullhelp show all command line switches diff --git a/doc/contributing.rst b/doc/contributing.md index b7188a436..420c1438e 100644 --- a/doc/contributing.rst +++ b/doc/contributing.md @@ -1,14 +1,18 @@ -.. default-role:: code - ============ Contributing ============ +.. default-role:: code +.. include:: rstcommon.rst + .. contents:: -Contributing happens via "Pull requests" (PR) on github. Every PR needs to be +Contributing happens via "Pull requests" (PR) on GitHub. Every PR needs to be reviewed before it can be merged and the Continuous Integration should be green. +The title of a PR should contain a brief description. If it fixes an issue, +in addition to the number of the issue, the title should also contain a description +of the issue. The PR has to be approved by two core developers or by Araq. @@ -19,21 +23,23 @@ Writing tests There are 4 types of tests: -1. `runnableExamples` documentation comment tests, ran by `nim doc mymod.nim` +1. `runnableExamples` documentation comment tests, ran by `nim doc mymod.nim`:cmd: These end up in documentation and ensure documentation stays in sync with code. -2. separate test files, e.g.: `tests/stdlib/tos.nim`. - In nim repo, `testament` (see below) runs all `$nim/tests/*/t*.nim` test files; +2. separate test files, e.g.: ``tests/stdlib/tos.nim``. + In nim repo, `testament`:cmd: (see below) runs all + ``$nim/tests/*/t*.nim`` test files; for nimble packages, see https://github.com/nim-lang/nimble#tests. -3. (deprecated) tests in `when isMainModule:` block, ran by `nim r mymod.nim`. - `nimble test` can run those in nimble packages when specified in a +3. (deprecated) tests in `when isMainModule:` block, ran by `nim r mymod.nim`:cmd:. + `nimble test`:cmd: can run those in nimble packages when specified in a `task "test"`. -4. (not preferred) `.. code-block:: nim` RST snippets; these should only be used in rst sources, +4. (not preferred) ``.. code-block:: nim`` RST snippets; + these should only be used in rst sources, in nim sources `runnableExamples` should now always be preferred to those for several reasons (cleaner syntax, syntax highlights, batched testing, and - `rdoccmd` allows customization). + parameter `rdoccmd` allows customization). Not all the tests follow the convention here, feel free to change the ones that don't. Always leave the code cleaner than you found it. @@ -41,8 +47,8 @@ that don't. Always leave the code cleaner than you found it. Stdlib ------ -Each stdlib module (anything under `lib/`, e.g. `lib/pure/os.nim`) should -preferably have a corresponding separate test file, e.g. `tests/stdlib/tos.nim`. +Each stdlib module (anything under ``lib/``, e.g. ``lib/pure/os.nim``) should +preferably have a corresponding separate test file, e.g. ``tests/stdlib/tos.nim``. The old convention was to add a `when isMainModule:` block in the source file, which only gets executed when the tester is building the file. @@ -53,8 +59,7 @@ things like `echo "done"`. Don't use `unittest.suite` and `unittest.test`. Sample test: -.. code-block:: nim - + ```nim block: # foo doAssert foo(1) == 10 @@ -70,42 +75,42 @@ Sample test: @[false, false], @[false, false]] # doAssert with `not` can now be done as follows: doAssert not (1 == 2) + ``` -Always refer to a GitHub issue using the following exact syntax: `bug #1234` as shown +Always refer to a GitHub issue using the following exact syntax: ``bug #1234`` as shown above, so that it's consistent and easier to search or for tooling. Some browser extensions (e.g. https://github.com/sindresorhus/refined-github) will even turn those in clickable links when it works. Rationale for using a separate test file instead of `when isMainModule:` block: * allows custom compiler flags or testing options (see details below) -* faster CI since they can be joined in `megatest` (combined into a single test) +* faster CI since they can be joined in ``megatest`` (combined into a single test) * avoids making the parser do un-necessary work when a source file is merely imported * avoids mixing source and test code when reporting line of code statistics or code coverage Compiler -------- -The tests for the compiler use a testing tool called `testament`. They are all -located in `tests/` (e.g.: `tests/destructor/tdestructor3.nim`). +The tests for the compiler use a testing tool called `testament`:cmd:. They are all +located in ``tests/`` (e.g.: ``tests/destructor/tdestructor3.nim``). Each test has its own file. All test files are prefixed with `t`. If you want to create a file for import into another test only, use the prefix `m`. At the beginning of every test is the expected behavior of the test. Possible keys are: -- `cmd`: A compilation command template e.g. `nim $target --threads:on $options $file` +- `cmd`: A compilation command template e.g. `nim $target --threads:on $options $file`:cmd: - `output`: The expected output (stdout + stderr), most likely via `echo` - `exitcode`: Exit code of the test (via `exit(number)`) - `errormsg`: The expected compiler error message - `file`: The file the errormsg was produced at - `line`: The line the errormsg was produced at -For a full spec, see here: `testament/specs.nim` +For a full spec, see here: ``testament/specs.nim`` An example of a test: -.. code-block:: nim - + ```nim discard """ errormsg: "type mismatch: got (PTest)" """ @@ -117,6 +122,7 @@ An example of a test: var buf: PTest buf.test() + ``` Running tests @@ -124,79 +130,78 @@ Running tests You can run the tests with -:: - + ```cmd ./koch tests + ``` which will run a good subset of tests. Some tests may fail. If you only want to see the output of failing tests, go for -:: - + ```cmd ./koch tests --failing all + ``` You can also run only a single category of tests. A category is a subdirectory -in the `tests` directory. There are a couple of special categories; for a -list of these, see `testament/categories.nim`, at the bottom. +in the ``tests/`` directory. There are a couple of special categories; for a +list of these, see ``testament/categories.nim``, at the bottom. -:: - - ./koch tests c lib # compiles/runs stdlib modules, including `isMainModule` tests + ```cmd + ./koch tests c lib # compiles / runs stdlib modules, including `isMainModule` tests ./koch tests c megatest # runs a set of tests that can be combined into 1 + ``` To run a single test: -:: - + ```cmd ./koch test run <category>/<name> # e.g.: tuples/ttuples_issues ./koch test run tests/stdlib/tos.nim # can also provide relative path + ``` For reproducible tests (to reproduce an environment more similar to the one -run by Continuous Integration on travis/appveyor), you may want to disable your -local configuration (e.g. in `~/.config/nim/nim.cfg`) which may affect some +run by Continuous Integration on GitHub actions/azure pipelines), you may want to disable your +local configuration (e.g. in ``~/.config/nim/nim.cfg``) which may affect some tests; this can also be achieved by using -`export XDG_CONFIG_HOME=pathtoAlternateConfig` before running `./koch` +`export XDG_CONFIG_HOME=pathtoAlternateConfig`:cmd: before running `./koch`:cmd: commands. Comparing tests =============== -Test failures can be grepped using `Failure:`. +Test failures can be grepped using ``Failure:``. The tester can compare two test runs. First, you need to create a reference test. You'll also need to the commit id, because that's what the tester needs to know in order to compare the two. -:: - + ```cmd git checkout devel DEVEL_COMMIT=$(git rev-parse HEAD) ./koch tests + ``` Then switch over to your changes and run the tester again. -:: - + ```cmd git checkout your-changes ./koch tests + ``` -Then you can ask the tester to create a `testresults.html` which will +Then you can ask the tester to create a ``testresults.html`` which will tell you if any new tests passed/failed. -:: - + ```cmd ./koch tests --print html $DEVEL_COMMIT + ``` Deprecation =========== -Backward compatibility is important, so instead of a rename you need to deprecate -the old name and introduce a new name: - -.. code-block:: nim +Backwards compatibility is important. When renaming types, procedures, etc. the old name +must be marked as deprecated using the `deprecated` pragma: + ```nim # for routines (proc/template/macro/iterator) and types: proc oldProc(a: int, b: float): bool {.deprecated: "deprecated since v1.2.3; use `newImpl: string -> int` instead".} = discard @@ -208,9 +213,10 @@ the old name and introduce a new name: # (likewise with object types and their fields): type Bar {.deprecated.} = enum bar0, bar1 type Barz = enum baz0, baz1 {.deprecated.}, baz2 + ``` -See also `Deprecated <manual.html#pragmas-deprecated-pragma>`_ +See also [Deprecated](manual.html#pragmas-deprecated-pragma) pragma in the manual. @@ -219,72 +225,127 @@ Documentation When contributing new procs, be sure to add documentation, especially if the proc is public. Even private procs benefit from documentation and can be -viewed using `nim doc --docInternal foo.nim`. +viewed using `nim doc --docInternal foo.nim`:cmd:. Documentation begins on the line following the `proc` definition, and is prefixed by `##` on each line. Runnable code examples are also encouraged, to show typical behavior with a few test cases (typically 1 to 3 `assert` statements, depending on complexity). -These `runnableExamples` are automatically run by `nim doc mymodule.nim` -as well as `testament` and guarantee they stay in sync. +These `runnableExamples` are automatically run by `nim doc mymodule.nim`:cmd: +as well as `testament`:cmd: and guarantee they stay in sync. -.. code-block:: nim + ```nim proc addBar*(a: string): string = ## Adds "Bar" to `a`. runnableExamples: assert "baz".addBar == "bazBar" result = a & "Bar" + ``` -See `parentDir <os.html#parentDir,string>`_ example. +See [parentDir](os.html#parentDir,string) example. The RestructuredText Nim uses has a special syntax for including code snippets -embedded in documentation; these are not run by `nim doc` and therefore are +embedded in documentation; these are not run by `nim doc`:cmd: and therefore are not guaranteed to stay in sync, so `runnableExamples` is almost always preferred: -.. code-block:: nim - + ````nim proc someProc*(): string = ## Returns "something" ## - ## .. code-block:: - ## echo someProc() # "something" + ## ``` + ## echo someProc() # "something" + ## ``` result = "something" # single-hash comments do not produce documentation + ```` -The `.. code-block:: nim` followed by a newline and an indentation instructs the -`nim doc` command to produce syntax-highlighted example code with the -documentation (`.. code-block::` is sufficient from inside a nim module). +The \`\`\` followed by a newline and an indentation instructs the +`nim doc`:cmd: command to produce syntax-highlighted example code with the +documentation (\`\`\` is sufficient inside a ``.nim`` module, while from +a ``.md`` one needs to set the language explicitly as \`\`\`nim). When forward declaration is used, the documentation should be included with the first appearance of the proc. -.. code-block:: nim - + ```nim proc hello*(): string ## Put documentation here proc nothing() = discard proc hello*(): string = ## ignore this echo "hello" + ``` The preferred documentation style is to begin with a capital letter and use the third-person singular. That is, between: -.. code-block:: nim - + ```nim proc hello*(): string = ## Returns "hello" result = "hello" + ``` or -.. code-block:: nim - + ```nim proc hello*(): string = ## say hello result = "hello" + ``` the first is preferred. +When you specify an *RST role* (highlighting/interpretation marker) do it +in the postfix form for uniformity, that is after \`text in backticks\`. +For example an ``:idx:`` role for referencing a topic ("SQLite" in the +example below) from [Nim Index] can be used in doc comment this way: + + ```nim + ## A higher level `SQLite`:idx: database wrapper. + ``` + +.. _`Nim Index`: https://nim-lang.org/docs/theindex.html + +Inline monospaced text can be input using \`single backticks\` or +\`\`double backticks\`\`. The former are syntactically highlighted, +the latter are not. +To avoid accidental highlighting follow this rule in ``*.nim`` files: + +* Use single backticks for fragments of code in Nim and other + programming languages, including identifiers, in ``*.nim`` files. + + For languages other than Nim add a role after final backtick, + e.g. for C++ inline highlighting: + + `#include <stdio.h>`:cpp: + + For a currently unsupported language add the `:code:` role, + like for SQL in this example: + + `SELECT * FROM <table_name>;`:code: + + Highlight shell commands by ``:cmd:`` role; for command line options use + ``:option:`` role, e.g.: \`--docInternal\`:option:. + +* Use double backticks: + + * For file names: \`\`os.nim\`\` + * For fragments of strings **not** enclosed by `"` and `"` and not + related to code, e.g. text of compiler messages + * When code ends with a standalone ``\`` (otherwise a combination of + ``\`` and a final \` would get escaped) + +.. Note:: ``*.rst`` files have ``:literal:`` as their default role. + So for them the rule above is only applicable if the ``:nim:`` role + is set up manually as the default [^1]: + + .. role:: nim(code) + :language: nim + .. default-role:: nim + + The first 2 lines are for other RST implementations, + including Github one. + + [^1]: this is fulfilled when ``doc/rstcommon.rst`` is included. Best practices ============== @@ -298,85 +359,85 @@ New `defined(foo)` symbols need to be prefixed by the nimble package name, or by `nim` for symbols in nim sources (e.g. compiler, standard library). This is to avoid name conflicts across packages. -.. code-block:: nim - + ```nim # if in nim sources when defined(allocStats): discard # bad, can cause conflicts when defined(nimAllocStats): discard # preferred - # if in a pacakge `cligen`: + # if in a package `cligen`: when defined(debug): discard # bad, can cause conflicts when defined(cligenDebug): discard # preferred + ``` .. _noimplicitbool: Take advantage of no implicit bool conversion -.. code-block:: nim - + ```nim doAssert isValid() == true doAssert isValid() # preferred + ``` .. _design_for_mcs: Design with method call syntax chaining in mind -.. code-block:: nim - + ```nim proc foo(cond: bool, lines: seq[string]) # bad proc foo(lines: seq[string], cond: bool) # preferred # can be called as: `getLines().foo(false)` + ``` .. _avoid_quit: -Use exceptions (including assert / doAssert) instead of `quit` +Use exceptions (including `assert` / `doAssert`) instead of `quit` rationale: https://forum.nim-lang.org/t/4089 -.. code-block:: nim - + ```nim quit() # bad in almost all cases doAssert() # preferred + ``` .. _tests_use_doAssert: Use `doAssert` (or `unittest.check`, `unittest.require`), not `assert` in all -tests so they'll be enabled even with `--assertions:off`. - -.. code-block:: nim +tests, so they'll be enabled even with `--assertions:off`:option:. + ```nim block: # foo assert foo() # bad doAssert foo() # preferred + ``` .. _runnableExamples_use_assert: -An exception to the above rule is `runnableExamples` and `code-block` rst blocks +An exception to the above rule is `runnableExamples` and ``code-block`` rst blocks intended to be used as `runnableExamples`, which for brevity use `assert` -instead of `doAssert`. Note that `nim doc -d:danger main` won't pass `-d:danger` to the -`runnableExamples`, but `nim doc --doccmd:-d:danger main` would, and so would the +instead of `doAssert`. Note that `nim doc -d:danger main`:cmd: won't pass `-d:danger`:option: to the +`runnableExamples`, but `nim doc --doccmd:-d:danger main`:cmd: would, and so would the second example below: -.. code-block:: nim - + ```nim runnableExamples: doAssert foo() # bad assert foo() # preferred runnableExamples("-d:danger"): doAssert foo() # `assert` would be disabled here, so `doAssert` makes more sense + ``` .. _delegate_printing: Delegate printing to caller: return `string` instead of calling `echo` rationale: it's more flexible (e.g. allows the caller to call custom printing, -including prepending location info, writing to log files, etc). - -.. code-block:: nim +including prepending location info, writing to log files, etc.). + ```nim proc foo() = echo "bar" # bad proc foo(): string = "bar" # preferred (usually) + ``` .. _use_Option: -[Ongoing debate] Consider using Option instead of return bool + var argument, +(Ongoing debate) Consider using Option instead of return bool + var argument, unless stack allocation is needed (e.g. for efficiency). -.. code-block:: nim - + ```nim proc foo(a: var Bar): bool proc foo(): Option[Bar] + ``` .. _use_doAssert_not_echo: Tests (including in testament) should always prefer assertions over `echo`, @@ -384,14 +445,14 @@ except when that's not possible. It's more precise, easier for readers and maintainers to where expected values refer to. See for example https://github.com/nim-lang/Nim/pull/9335 and https://forum.nim-lang.org/t/4089 -.. code-block:: nim - + ```nim echo foo() # adds a line for testament in `output:` block inside `discard`. doAssert foo() == [1, 2] # preferred, except when not possible to do so. + ``` -The Git stuff -============= +The `git`:cmd: stuff +==================== General commit rules -------------------- @@ -399,14 +460,14 @@ General commit rules 1. Important, critical bugfixes that have a tiny chance of breaking somebody's code should be backported to the latest stable release branch (currently 1.4.x) and maybe also all the way back to the 1.0.x branch. - The commit message should contain the tag `[backport]` for "backport to all - stable releases" and the tag `[backport:$VERSION]` for backporting to the - given $VERSION. + The commit message should contain the tag ``[backport]`` for "backport to the latest + stable release" and the tag ``[backport:$VERSION]`` for backporting back to the + given $VERSION (and all newer releases). 2. If you introduce changes which affect backward compatibility, - make breaking changes, or have PR which is tagged as `[feature]`, - the changes should be mentioned in `the changelog - <https://github.com/nim-lang/Nim/blob/devel/changelog.md>`_. + make breaking changes, or have PR which is tagged as ``[feature]``, + the changes should be mentioned in [the changelog]( + https://github.com/nim-lang/Nim/blob/devel/changelog.md). 3. All changes introduced by the commit (diff lines) must be related to the subject of the commit. @@ -415,44 +476,44 @@ General commit rules your editor reformatted automatically the code or whatever different reason, this should be excluded from the commit. - *Tip:* Never commit everything as is using `git commit -a`, but review - carefully your changes with `git add -p`. + *Tip:* Never commit everything as-is using `git commit -a`:cmd:, but review + carefully your changes with `git add -p`:cmd:. 4. Changes should not introduce any trailing whitespace. - Always check your changes for whitespace errors using `git diff --check` - or add the following `pre-commit` hook: + Always check your changes for whitespace errors using `git diff --check`:cmd: + or add the following ``pre-commit`` hook: - .. code-block:: sh - - #!/bin/sh - git diff --check --cached || exit $? + ```cmd + #!/bin/sh + git diff --check --cached || exit $? + ``` 5. Describe your commit and use your common sense. Example commit message: - `Fixes #123; refs #124` + Fixes #123; refs #124 - indicates that issue `#123` is completely fixed (GitHub may automatically - close it when the PR is committed), wheres issue `#124` is referenced + indicates that issue ``#123`` is completely fixed (GitHub may automatically + close it when the PR is committed), whereas issue ``#124`` is referenced (e.g.: partially fixed) and won't close the issue when committed. -6. PR body (not just PR title) should contain references to fixed/referenced github - issues, e.g.: `fix #123` or `refs #123`. This is so that you get proper cross - referencing from linked issue to the PR (github won't make those links with just - PR title, and commit messages aren't always sufficient to ensure that, e.g. - can't be changed after a PR is merged). +6. PR body (not just PR title) should contain references to fixed/referenced GitHub + issues, e.g.: ``fix #123`` or ``refs #123``. This is so that you get proper + cross-referencing from linked issue to the PR (GitHub won't make those links + with just a PR title, and commit messages aren't always sufficient to ensure + that, e.g. can't be changed after a PR is merged). -7. Commits should be always be rebased against devel (so a fast forward +7. Commits should be always be rebased against devel (so a fast-forward merge can happen) - e.g.: use `git pull --rebase origin devel`. This is to avoid messing up + e.g.: use `git pull --rebase origin devel`:cmd:. This is to avoid messing up git history. Exceptions should be very rare: when rebase gives too many conflicts, simply squash all commits using the script shown in https://github.com/nim-lang/Nim/pull/9356 8. Do not mix pure formatting changes (e.g. whitespace changes, nimpretty) or - automated changes (e.g. nimfix) with other code changes: these should be in + automated changes with other code changes: these should be in separate commits (and the merge on GitHub should not squash these into 1). @@ -461,14 +522,17 @@ Continuous Integration (CI) 1. Continuous Integration is by default run on every push in a PR; this clogs the CI pipeline and affects other PR's; if you don't need it (e.g. for WIP or - documentation only changes), add `[ci skip]` to your commit message title. - This convention is supported by `Appveyor - <https://www.appveyor.com/docs/how-to/filtering-commits/#skip-directive-in-commit-message>`_ - and `Travis <https://docs.travis-ci.com/user/customizing-the-build/#skipping-a-build>`_. + documentation only changes), add ``[skip ci]`` to your commit message title. + This convention is supported by our GitHub actions pipelines and our azure pipeline + (using custom logic, which should complete in < 1mn) as well as our former other pipelines: + [Appveyor]( + https://www.appveyor.com/docs/how-to/filtering-commits/#skip-directive-in-commit-message) + and [Travis]( + https://docs.travis-ci.com/user/customizing-the-build/#skipping-a-build). 2. Consider enabling CI (azure, GitHub actions and builds.sr.ht) in your own Nim fork, and waiting for CI to be green in that fork (fixing bugs as needed) before - opening your PR in the original Nim repo, so as to reduce CI congestion. Same + opening your PR in the original Nim repo, to reduce CI congestion. Same applies for updates on a PR: you can test commits on a separate private branch before updating the main PR. @@ -481,18 +545,20 @@ Debugging CI failures, flaky tests, etc 2. If CI failure seems unrelated to your PR, it could be caused by a flaky test. File a bug for it if it isn't already reported. A PR push (or opening/closing PR) will re-trigger all CI jobs (even successful ones, which can be wasteful). Instead, + request collaboration from the Nim team. The Nim team should follow these instructions to only restart the jobs that failed: - * Azure: if on your own fork, it's possible from inside azure console - (e.g. `dev.azure.com/username/username/_build/results?buildId=1430&view=results`) via `rerun failed jobs` on top. - If either on you own fork or in Nim repo, it's possible from inside GitHub UI - under checks tab, see https://github.com/timotheecour/Nim/issues/211#issuecomment-629751569 - * GitHub actions: under "Checks" tab, click "Re-run jobs" in the right. - * builds.sr.ht: create a sourcehut account so you can restart a PR job as illustrated. - builds.sr.ht also allows you to ssh to a CI machine which can help a lot for debugging - issues, see docs in https://man.sr.ht/builds.sr.ht/build-ssh.md and - https://drewdevault.com/2019/08/19/Introducing-shell-access-for-builds.html; see - https://man.sr.ht/tutorials/set-up-account-and-git.md to generate and upload ssh keys. + * Azure: if on your own fork, it's possible from inside azure console + (e.g. ``dev.azure.com/username/username/_build/results?buildId=1430&view=results``) via + ``rerun failed jobs`` on top. + If either on you own fork or in Nim repo, it's possible from inside GitHub UI + under checks tab, see https://github.com/timotheecour/Nim/issues/211#issuecomment-629751569 + * GitHub actions: under "Checks" tab, click "Re-run jobs" in the right. + * builds.sr.ht: create a SourceHut account so that you can restart a PR job as illustrated. + builds.sr.ht also allows you to ssh to a CI machine which can help a lot for debugging + issues, see docs in https://man.sr.ht/builds.sr.ht/build-ssh.md and + https://drewdevault.com/2019/08/19/Introducing-shell-access-for-builds.html; see + https://man.sr.ht/tutorials/set-up-account-and-git.md to generate and upload ssh keys. Code reviews @@ -503,26 +569,25 @@ Code reviews https://forum.nim-lang.org/t/4317 2. When reviewing large diffs that may involve code moving around, GitHub's interface - doesn't help much as it doesn't highlight moves. Instead, you can use something - like this, see visual results `here <https://github.com/nim-lang/Nim/pull/10431#issuecomment-456968196>`_: - - .. code-block:: sh + doesn't help much, as it doesn't highlight moves. Instead, you can use something + like this, see visual results [here]( + https://github.com/nim-lang/Nim/pull/10431#issuecomment-456968196): - git fetch origin pull/10431/head && git checkout FETCH_HEAD - git diff --color-moved-ws=allow-indentation-change --color-moved=blocks HEAD^ + ```cmd + git fetch origin pull/10431/head && git checkout FETCH_HEAD + git diff --color-moved-ws=allow-indentation-change --color-moved=blocks HEAD^ + ``` 3. In addition, you can view GitHub-like diffs locally to identify what was changed - within a code block using `diff-highlight` or `diff-so-fancy`, e.g.: - - .. code-block:: sh + within a code block using `diff-highlight`:cmd: or `diff-so-fancy`:cmd:, e.g.: - # put this in ~/.gitconfig: - [core] - pager = "diff-so-fancy | less -R" # or: use: `diff-highlight` + # put this in ~/.gitconfig: + [core] + pager = "diff-so-fancy | less -R" # or: use: `diff-highlight` -.. include:: docstyle.rst +.. include:: docstyle.md Evolving the stdlib @@ -556,7 +621,7 @@ Time handling, especially the `Time` type are also covered by this rule. Existing, battle-tested modules stay ------------------------------------ -Reason: There is no benefit in moving them around just to fullfill some design +Reason: There is no benefit in moving them around just to fulfill some design fashion as in "Nim's core MUST BE SMALL". If you don't like an existing module, don't import it. If a compilation target (e.g. JS) cannot support a module, document this limitation. @@ -570,7 +635,7 @@ Syntactic helpers can start as experimental stdlib modules Reason: Generally speaking as external dependencies they are not exposed to enough users so that we can see if the shortcuts provide enough benefit -or not. Many programmers avoid external dependencies, even moreso for +or not. Many programmers avoid external dependencies, especially for "tiny syntactic improvements". However, this is only true for really good syntactic improvements that have the potential to clean up other parts of the Nim library substantially. If in doubt, new stdlib modules should start @@ -595,7 +660,7 @@ to existing modules is acceptable. For two reasons: ("Why does sequtils lack a `countIt`? Because version 1.0 happens to have lacked it? Silly...") 2. To encourage contributions. Contributors often start with PRs that - add simple things and then they stay and also fix bugs. Nim is an + add simple things, then they stay and also fix bugs. Nim is an open source project and lives from people's contributions and involvement. Newly introduced issues have to be balanced against motivating new people. We know where to find perfectly designed pieces of software that have no bugs -- these are the systems @@ -603,16 +668,131 @@ to existing modules is acceptable. For two reasons: Conventions ----------- -1. New stdlib modules should go under `Nim/lib/std/`. The rationale is to +1. New stdlib modules should go under ``Nim/lib/std/``. The rationale is to require users to import via `import std/foo` instead of `import foo`, which would cause potential conflicts with nimble packages. Note that this still applies for new modules in existing logical - directories, e.g.: use `lib/std/collections/foo.nim`, - not `lib/pure/collections/foo.nim`. + directories, e.g.: use ``lib/std/collections/foo.nim``, + not ``lib/pure/collections/foo.nim``. 2. New module names should prefer plural form whenever possible, e.g.: - `std/sums.nim` instead of `std/sum.nim`. In particular, this reduces + ``std/sums.nim`` instead of ``std/sum.nim``. In particular, this reduces chances of conflicts between module name and the symbols it defines. Furthermore, module names should use `snake_case` and not use capital letters, which cause issues when going from an OS without case sensitivity to an OS with it. + + +Breaking Changes +================ + +Introducing breaking changes, no matter how well-intentioned, +creates long-term problems for the community, in particular those looking to promote +reusable Nim code in libraries: In the Nim distribution, critical security and bugfixes, +language changes and community improvements are bundled in a single distribution - it is +difficult to make partial upgrades with only benign changes. When one library depends on +a legacy behavior, it can no longer be used together with another library that does not, +breaking all downstream applications - the standard library is unique in that it sits at +the root of **all** dependency trees. + +There is a big difference between compile-time breaking changes and run-time breaking +changes. + + +Run-time breaking changes +------------------------- + +Run-time breaking changes are to be avoided at almost all costs: Nim is used for +mission critical applications which depend on behaviours that +are not covered by the test suite. As such, it's important that changes to the +*stable* parts of the standard library are made avoiding changing the existing +behaviors, even when the test suite continues to pass. + +Examples of run-time breaking changes: + +- Raising exceptions of a new type, compared to what's currently being raised. + +- Adding unconstrained or poorly constrained generic procs or macros + ("hash now works for all `ref T`"): This may cause code to behave differently + depending only on which modules are imported - common examples include `==` and `hash`. + +- Changing behavior of existing functions like: + + * "Nim's path handling procs like `getXDir` now consistently lack the trailing slash" + * "Nim's strformat implementation is now more consistent with Python" + +Instead, write new code that explicitly announces the feature you think we announced but +didn't. For example, `strformat` does not say "it's compatible with Python", it +says "inspired by Python's f-strings". This new code can be submitted to the stdlib +and the old code can be deprecated or published as a Nimble package. + +Sometimes, a run-time breaking change is most desirable: For example, a string +representation of a floating point number that "roundtrips" is much better than +a string representation that doesn't. These run-time breaking changes must start in the +state "opt-in" via a new `-d:nimPreviewX` or command line flag and then should become +the new default later, in follow-up versions. This way users can track +regressions more easily. ("git bisect" is not an acceptable alternative, that's for +Nim compiler developers, not for Nim users.) + +Above all else, additive approaches that don't change existing behaviors +should be preferred. + + +Compile-time breaking changes +----------------------------- + +Compile-time breaking changes are usually easier to handle, but for large code bases +they can also involve a large amount of work and can hinder the adoption of a new +Nim release. +Additive approaches are to be preferred here as well. + +Examples of compile-time breaking changes include (but are not limited to): + +* Renaming functions and modules, or moving things. Instead of a direct rename, + deprecate the old name and introduce a new one. +* Renaming the parameter names: Thanks to Nim's "named parameter" calling syntax + like `f(x = 0, y = 1)` this is a breaking change. Instead, live with the existing + parameter names. +* Adding an enum value to an existing enum. Nim's exhaustive case statements stop + compiling after such a change. Instead, consider to introduce new `bool` + fields/parameters. This can be impractical though, so we use good judgement + and our list of "important packages" to see if it doesn't break too much code + out there in practice. +* Adding a new proc to an existing stdlib module. However, for aesthetic reasons + this is often preferred over introducing a new module with just a single proc + inside. We use good judgement and our list of "important packages" to see if + it doesn't break too much code out there in practice. The new procs need to + be annotated with a `.since` annotation. + + +Compiler/language spec bugfixes +------------------------------- + +This can even be applied to compiler "bugfixes": If the compiler should have been +"pickier" in its handling of `typedesc`, instead of "fixing typedesc handling bugs", +consider the following solution: + +- Spec out how `typedesc` should really work and also spec out the cases where it + should not be allowed! +- Deprecate `typedesc` and name the new metatype something new like `typeArg`. +- Implement the spec. + + +Non-breaking changes +-------------------- + +Examples of changes that are considered non-breaking (or acceptable breaking changes) include: + +* Creating a new module. +* Adding an overload to an already overloaded proc. +* Adding new default parameters to an existing proc. It is assumed that you do not + use Nim's stdlib procs's addresses (that you don't use them as first class entities). +* Changing the calling convention from `nimcall` to `inline` + (but first RFC https://github.com/nim-lang/RFCs/issues/396 needs to be implemented). +* Changing the behavior from "crashing" into some other, well documented result (including + raising a Defect, but not raising an exception that does not inherit from Defect). +* Adding new fields to an existing object. + +Nim's introspection facilities imply that strictly speaking almost every addition can +break somebody's code. It is impractical to care about these cases, a change that only +affects introspection is not considered to be a breaking change. diff --git a/doc/destructors.rst b/doc/destructors.md index 01e2d2ee9..e192fd362 100644 --- a/doc/destructors.rst +++ b/doc/destructors.md @@ -1,5 +1,3 @@ -.. default-role:: code - ================================== Nim Destructors and Move Semantics ================================== @@ -7,18 +5,20 @@ Nim Destructors and Move Semantics :Authors: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: About this document =================== -This document describes the upcoming Nim runtime which does +This document describes the ARC/ORC Nim runtime which does not use classical GC algorithms anymore but is based on destructors and -move semantics. The new runtime's advantages are that Nim programs become +move semantics. The advantages are that Nim programs become oblivious to the involved heap sizes and programs are easier to write to make effective use of multi-core machines. As a nice bonus, files and sockets and -the like will not require manual `close` calls anymore. +the like can be written not to require manual `close` calls anymore. This document aims to be a precise specification about how move semantics and destructors work in Nim. @@ -30,23 +30,31 @@ Motivating example With the language mechanisms described here, a custom seq could be written as: -.. code-block:: nim - + ```nim test type myseq*[T] = object len, cap: int data: ptr UncheckedArray[T] - proc `=destroy`*[T](x: var myseq[T]) = + proc `=destroy`*[T](x: myseq[T]) = if x.data != nil: for i in 0..<x.len: `=destroy`(x.data[i]) dealloc(x.data) + proc `=wasMoved`*[T](x: var myseq[T]) = + x.data = nil + + proc `=trace`[T](x: var myseq[T]; env: pointer) = + # `=trace` allows the cycle collector `--mm:orc` + # to understand how to trace the object graph. + if x.data != nil: + for i in 0..<x.len: `=trace`(x.data[i], env) + proc `=copy`*[T](a: var myseq[T]; b: myseq[T]) = # do nothing for self-assignments: if a.data == b.data: return `=destroy`(a) - wasMoved(a) + `=wasMoved`(a) a.len = b.len a.cap = b.cap if b.data != nil: @@ -54,11 +62,19 @@ written as: for i in 0..<a.len: a.data[i] = b.data[i] + proc `=dup`*[T](a: myseq[T]): myseq[T] {.nodestroy.} = + # an optimized version of `=wasMoved(tmp); `=copy(tmp, src)` + # usually present if a custom `=copy` hook is overridden + result = myseq[T](len: a.len, cap: a.cap, data: nil) + if a.data != nil: + result.data = cast[typeof(result.data)](alloc(result.cap * sizeof(T))) + for i in 0..<result.len: + result.data[i] = `=dup`(a.data[i]) + proc `=sink`*[T](a: var myseq[T]; b: myseq[T]) = # move assignment, optional. # Compiler is using `=destroy` and `copyMem` when not provided `=destroy`(a) - wasMoved(a) a.len = b.len a.cap = b.cap a.data = b.data @@ -79,13 +95,14 @@ written as: x.data[i] = y proc createSeq*[T](elems: varargs[T]): myseq[T] = - result.cap = elems.len - result.len = elems.len - result.data = cast[typeof(result.data)](alloc(result.cap * sizeof(T))) + result = myseq[T]( + len: elems.len, + cap: elems.len, + data: cast[typeof(result.data)](alloc(result.cap * sizeof(T)))) for i in 0..<result.len: result.data[i] = elems[i] proc len*[T](x: myseq[T]): int {.inline.} = x.len - + ``` Lifetime-tracking hooks @@ -93,9 +110,10 @@ Lifetime-tracking hooks The memory management for Nim's standard `string` and `seq` types as well as other standard collections is performed via so-called -"Lifetime-tracking hooks", which are particular `type bound operators <manual.html#procedures-type-bound-operators>`_. +"Lifetime-tracking hooks", which are particular [type bound operators]( +manual.html#procedures-type-bound-operators). -There are 3 different hooks for each (generic or concrete) object type `T` (`T` can also be a +There are 6 different hooks for each (generic or concrete) object type `T` (`T` can also be a `distinct` type) that are called implicitly by the compiler. (Note: The word "hook" here does not imply any kind of dynamic binding @@ -111,22 +129,47 @@ other associated resources. Variables are destroyed via this hook when they go out of scope or when the routine they were declared in is about to return. -The prototype of this hook for a type `T` needs to be: - -.. code-block:: nim - - proc `=destroy`(x: var T) +A `=destroy` hook is allowed to have a parameter of a `var T` or `T` type. Taking a `var T` type is deprecated. The prototype of this hook for a type `T` needs to be: + ```nim + proc `=destroy`(x: T) + ``` The general pattern in `=destroy` looks like: -.. code-block:: nim - - proc `=destroy`(x: var T) = + ```nim + proc `=destroy`(x: T) = # first check if 'x' was moved to somewhere else: if x.field != nil: freeResource(x.field) + ``` +A `=destroy` is implicitly annotated with `.raises: []`; a destructor +should not raise exceptions. For backwards compatibility the compiler +produces a warning for a `=destroy` that does raise. + +A `=destroy` can explicitly list the exceptions it can raise, if any, +but this of little utility as a raising destructor is implementation defined +behavior. Later versions of the language specification might cover this case precisely. + + +`=wasMoved` hook +---------------- + +A `=wasMoved` hook sets the object to a state that signifies to the destructor there is nothing to destroy. + +The prototype of this hook for a type `T` needs to be: + + ```nim + proc `=wasMoved`(x: var T) + ``` + +Usually some pointer field inside the object is set to `nil`: + + ```nim + proc `=wasMoved`(x: var T) = + x.field = nil + ``` `=sink` hook @@ -139,31 +182,30 @@ not free the resources afterward by setting the object to its default value default value is written as `wasMoved(x)`. When not provided the compiler is using a combination of `=destroy` and `copyMem` instead. This is efficient hence users rarely need to implement their own `=sink` operator, it is enough to -provide `=destroy` and `=copy`, compiler will take care of the rest. +provide `=destroy` and `=copy`, the compiler will take care of the rest. The prototype of this hook for a type `T` needs to be: -.. code-block:: nim - + ```nim proc `=sink`(dest: var T; source: T) - + ``` The general pattern in `=sink` looks like: -.. code-block:: nim + ```nim proc `=sink`(dest: var T; source: T) = `=destroy`(dest) wasMoved(dest) dest.field = source.field - + ``` **Note**: `=sink` does not need to check for self-assignments. How self-assignments are handled is explained later in this document. `=copy` hook ---------------- +------------ The ordinary assignment in Nim conceptually copies the values. The `=copy` hook is called for assignments that couldn't be transformed into `=sink` @@ -171,42 +213,139 @@ operations. The prototype of this hook for a type `T` needs to be: -.. code-block:: nim - + ```nim proc `=copy`(dest: var T; source: T) - + ``` The general pattern in `=copy` looks like: -.. code-block:: nim - + ```nim proc `=copy`(dest: var T; source: T) = # protect against self-assignments: if dest.field != source.field: `=destroy`(dest) wasMoved(dest) dest.field = duplicateResource(source.field) - + ``` The `=copy` proc can be marked with the `{.error.}` pragma. Then any assignment that otherwise would lead to a copy is prevented at compile-time. This looks like: -.. code-block:: nim - + ```nim proc `=copy`(dest: var T; source: T) {.error.} + ``` but a custom error message (e.g., `{.error: "custom error".}`) will not be emitted by the compiler. Notice that there is no `=` before the `{.error.}` pragma. + +`=trace` hook +------------- + +A custom **container** type can support Nim's cycle collector `--mm:orc` via +the `=trace` hook. If the container does not implement `=trace`, cyclic data +structures which are constructed with the help of the container might leak +memory or resources, but memory safety is not compromised. + +The prototype of this hook for a type `T` needs to be: + + ```nim + proc `=trace`(dest: var T; env: pointer) + ``` + +`env` is used by ORC to keep track of its internal state, it should be passed around +to calls of the built-in `=trace` operation. + +Usually there will only be a need for a custom `=trace` when a custom `=destroy` that deallocates +manually allocated resources is also used, and then only when there is a chance of cyclic +references from items within the manually allocated resources when it is desired that `--mm:orc` +is able to break and collect these cyclic referenced resources. Currently however, there is a +mutual use problem in that whichever of `=destroy`/`=trace` is used first will automatically +create a version of the other which will then conflict with the creation of the second of the +pair. The workaround for this problem is to forward declare the second of the "hooks" to +prevent the automatic creation. + +The general pattern in using `=destroy` with `=trace` looks like: + + ```nim + type + Test[T] = object + size: Natural + arr: ptr UncheckedArray[T] # raw pointer field + + proc makeTest[T](size: Natural): Test[T] = # custom allocation... + Test[T](size: size, arr: cast[ptr UncheckedArray[T]](alloc0(sizeof(T) * size))) + + + proc `=destroy`[T](dest: Test[T]) = + if dest.arr != nil: + for i in 0 ..< dest.size: dest.arr[i].`=destroy` + dealloc dest.arr + + proc `=trace`[T](dest: var Test[T]; env: pointer) = + if dest.arr != nil: + # trace the `T`'s which may be cyclic + for i in 0 ..< dest.size: `=trace`(dest.arr[i], env) + + # following may be other custom "hooks" as required... + ``` + +**Note**: The `=trace` hooks (which are only used by `--mm:orc`) are currently more experimental and less refined +than the other hooks. + + +`=dup` hook +----------- + +A `=dup` hook duplicates an object. `=dup(x)` can be regarded as an optimization replacing a `wasMoved(dest); =copy(dest, x)` operation. + +The prototype of this hook for a type `T` needs to be: + + ```nim + proc `=dup`(x: T): T + ``` + +The general pattern in implementing `=dup` looks like: + + ```nim + type + Ref[T] = object + data: ptr T + rc: ptr int + + proc `=dup`[T](x: Ref[T]): Ref[T] = + result = x + if x.rc != nil: + inc x.rc[] + ``` + Move semantics ============== A "move" can be regarded as an optimized copy operation. If the source of the copy operation is not used afterward, the copy can be replaced by a move. This document uses the notation `lastReadOf(x)` to describe that `x` is not -used afterwards. This property is computed by a static control flow analysis +used afterward. This property is computed by a static control flow analysis but can also be enforced by using `system.move` explicitly. +One can query if the analysis is able to perform a move with `system.ensureMove`. +`move` enforces a move operation and calls `=wasMoved` whereas `ensureMove` is +an annotation that implies no runtime operation. An `ensureMove` annotation leads to a static error +if the compiler cannot prove that a move would be safe. + +For example: + + ```nim + proc main(normalParam: string; sinkParam: sink string) = + var x = "abc" + # valid: + let valid = ensureMove x + # invalid: + let invalid = ensureMove normalParam + # valid: + let alsoValid = ensureMove sinkParam + ``` + Swap ==== @@ -245,8 +384,7 @@ not a linear type system. The employed static analysis is limited and only concerned with local variables; however, object and tuple fields are treated as separate entities: -.. code-block:: nim - + ```nim proc consume(x: sink Obj) = discard "no implementation" proc main = @@ -254,16 +392,16 @@ however, object and tuple fields are treated as separate entities: consume tup[0] # ok, only tup[0] was consumed, tup[1] is still alive: echo tup[1] - + ``` Sometimes it is required to explicitly `move` a value into its final position: -.. code-block:: nim - + ```nim proc main = var dest, src: array[10, string] # ... for i in 0..high(dest): dest[i] = move(src[i]) + ``` An implementation is allowed, but not required to implement even more move optimizations (and the current implementation does not). @@ -273,20 +411,19 @@ Sink parameter inference ======================== The current implementation can do a limited form of sink parameter -inference. But it has to be enabled via `--sinkInference:on`, either +inference. But it has to be enabled via `--sinkInference:on`:option:, either on the command line or via a `push` pragma. To enable it for a section of code, one can -use `{.push sinkInference: on.}`...`{.pop.}`. +use `{.push sinkInference: on.}` ... `{.pop.}`. The `.nosinks`:idx: pragma can be used to disable this inference for a single routine: -.. code-block:: nim - + ```nim proc addX(x: T; child: T) {.nosinks.} = x.s.add child - + ``` The details of the inference algorithm are currently undocumented. @@ -303,58 +440,57 @@ Rewrite rules The current implementation follows strategy (2). This means that resources are destroyed at the scope exit. -:: - var x: T; stmts - --------------- (destroy-var) - var x: T; try stmts - finally: `=destroy`(x) + var x: T; stmts + --------------- (destroy-var) + var x: T; try stmts + finally: `=destroy`(x) - g(f(...)) - ------------------------ (nested-function-call) - g(let tmp; - bitwiseCopy tmp, f(...); - tmp) - finally: `=destroy`(tmp) + g(f(...)) + ------------------------ (nested-function-call) + g(let tmp; + bitwiseCopy tmp, f(...); + tmp) + finally: `=destroy`(tmp) - x = f(...) - ------------------------ (function-sink) - `=sink`(x, f(...)) + x = f(...) + ------------------------ (function-sink) + `=sink`(x, f(...)) - x = lastReadOf z - ------------------ (move-optimization) - `=sink`(x, z) - wasMoved(z) + x = lastReadOf z + ------------------ (move-optimization) + `=sink`(x, z) + `=wasMoved`(z) - v = v - ------------------ (self-assignment-removal) - discard "nop" + v = v + ------------------ (self-assignment-removal) + discard "nop" - x = y - ------------------ (copy) - `=copy`(x, y) + x = y + ------------------ (copy) + `=copy`(x, y) - f_sink(g()) - ----------------------- (call-to-sink) - f_sink(g()) + f_sink(g()) + ----------------------- (call-to-sink) + f_sink(g()) - f_sink(notLastReadOf y) - -------------------------- (copy-to-sink) - (let tmp; `=copy`(tmp, y); - f_sink(tmp)) + f_sink(notLastReadOf y) + -------------------------- (copy-to-sink) + (let tmp = `=dup`(y); + f_sink(tmp)) - f_sink(lastReadOf y) - ----------------------- (move-to-sink) - f_sink(y) - wasMoved(y) + f_sink(lastReadOf y) + ----------------------- (move-to-sink) + f_sink(y) + `=wasMoved`(y) Object and array construction @@ -367,7 +503,7 @@ function has `sink` parameters. Destructor removal ================== -`wasMoved(x);` followed by a `=destroy(x)` operation cancel each other +`=wasMoved(x)` followed by a `=destroy(x)` operation cancel each other out. An implementation is encouraged to exploit this in order to improve efficiency and code sizes. The current implementation does perform this optimization. @@ -376,11 +512,11 @@ optimization. Self assignments ================ -`=sink` in combination with `wasMoved` can handle self-assignments but +`=sink` in combination with `=wasMoved` can handle self-assignments but it's subtle. The simple case of `x = x` cannot be turned -into `=sink(x, x); wasMoved(x)` because that would lose `x`'s value. +into `=sink(x, x); =wasMoved(x)` because that would lose `x`'s value. The solution is that simple self-assignments that consist of - Symbols: `x = x` @@ -394,8 +530,7 @@ The complex case looks like a variant of `x = f(x)`, we consider `x = select(rand() < 0.5, x, y)` here: -.. code-block:: nim - + ```nim proc select(cond: bool; a, b: sink string): string = if cond: result = a # moves a into result @@ -407,21 +542,19 @@ The complex case looks like a variant of `x = f(x)`, we consider var y = "xyz" # possible self-assignment: x = select(true, x, y) - + ``` Is transformed into: - -.. code-block:: nim - + ```nim proc select(cond: bool; a, b: sink string): string = try: if cond: `=sink`(result, a) - wasMoved(a) + `=wasMoved`(a) else: `=sink`(result, b) - wasMoved(b) + `=wasMoved`(b) finally: `=destroy`(b) `=destroy`(a) @@ -435,15 +568,16 @@ Is transformed into: `=sink`(y, "xyz") `=sink`(x, select(true, let blitTmp = x - wasMoved(x) + `=wasMoved`(x) blitTmp, let blitTmp = y - wasMoved(y) + `=wasMoved`(y) blitTmp)) echo [x] finally: `=destroy`(y) `=destroy`(x) + ``` As can be manually verified, this transformation is correct for self-assignments. @@ -465,8 +599,7 @@ that the pointer does not outlive its origin. No destructor call is injected for expressions of type `lent T` or of type `var T`. -.. code-block:: nim - + ```nim test type Tree = object kids: seq[Tree] @@ -474,13 +607,13 @@ for expressions of type `lent T` or of type `var T`. proc construct(kids: sink seq[Tree]): Tree = result = Tree(kids: kids) # converted into: - `=sink`(result.kids, kids); wasMoved(kids) + `=sink`(result.kids, kids); `=wasMoved`(kids) `=destroy`(kids) proc `[]`*(x: Tree; i: int): lent Tree = result = x.kids[i] # borrows from 'x', this is transformed into: - result = addr x.kids[i] + # result = addr x.kids[i] # This means 'lent' is like 'var T' a hidden pointer. # Unlike 'var' this hidden pointer cannot be used to mutate the object. @@ -491,42 +624,43 @@ for expressions of type `lent T` or of type `var T`. # everything turned into moves: let t = construct(@[construct(@[]), construct(@[])]) echo t[0] # accessor does not copy the element! + ``` -The .cursor annotation -====================== - -Under the `--gc:arc|orc` modes Nim's `ref` type is implemented via the same runtime -"hooks" and thus via reference counting. This means that cyclic structures cannot be freed -immediately (`--gc:orc` ships with a cycle collector). With the `.cursor` annotation -one can break up cycles declaratively: +The cursor pragma +================= -.. code-block:: nim +Under the `--mm:arc|orc`:option: modes Nim's `ref` type is implemented +via the same runtime "hooks" and thus via reference counting. +This means that cyclic structures cannot be freed +immediately (`--mm:orc`:option: ships with a cycle collector). +With the `cursor` pragma one can break up cycles declaratively: + ```nim type Node = ref object left: Node # owning ref right {.cursor.}: Node # non-owning ref + ``` But please notice that this is not C++'s weak_ptr, it means the right field is not involved in the reference counting, it is a raw pointer without runtime checks. Automatic reference counting also has the disadvantage that it introduces overhead -when iterating over linked structures. The `.cursor` annotation can also be used +when iterating over linked structures. The `cursor` pragma can also be used to avoid this overhead: -.. code-block:: nim - + ```nim var it {.cursor.} = listRoot while it != nil: use(it) it = it.next + ``` - -In fact, `.cursor` more generally prevents object construction/destruction pairs +In fact, `cursor` more generally prevents object construction/destruction pairs and so can also be useful in other contexts. The alternative solution would be to use raw pointers (`ptr`) instead which is more cumbersome and also more dangerous -for Nim's evolution: Later on, the compiler can try to prove `.cursor` annotations +for Nim's evolution: Later on, the compiler can try to prove `cursor` pragmas to be safe, but for `ptr` the compiler has to remain silent about possible problems. @@ -534,25 +668,25 @@ problems. Cursor inference / copy elision =============================== -The current implementation also performs `.cursor` inference. Cursor inference is +The current implementation also performs `cursor` inference. Cursor inference is a form of copy elision. To see how and when we can do that, think about this question: In `dest = src` when do we really have to *materialize* the full copy? - Only if `dest` or `src` are mutated -afterwards. If `dest` is a local variable that is simple to analyze. And if `src` is a +afterward. If `dest` is a local variable that is simple to analyze. And if `src` is a location derived from a formal parameter, we also know it is not mutated! In other words, we do a compile-time copy-on-write analysis. This means that "borrowed" views can be written naturally and without explicit pointer indirections: -.. code-block:: nim - + ```nim proc main(tab: Table[string, string]) = - let v = tab["key"] # inferred as .cursor because 'tab' is not mutated. + let v = tab["key"] # inferred as cursor because 'tab' is not mutated. # no copy into 'v', no destruction of 'v'. use(v) useItAgain(v) + ``` Hook lifting @@ -576,8 +710,7 @@ Hook generation The ability to override a hook leads to a phase ordering problem: -.. code-block:: nim - + ```nim type Foo[T] = object @@ -586,11 +719,11 @@ The ability to override a hook leads to a phase ordering problem: # error: destructor for 'f' called here before # it was seen in this module. - proc `=destroy`[T](f: var Foo[T]) = + proc `=destroy`[T](f: Foo[T]) = discard + ``` - -The solution is to define `proc `=destroy`[T](f: var Foo[T])` before +The solution is to define ``proc `=destroy`[T](f: Foo[T])`` before it is used. The compiler generates implicit hooks for all types in *strategic places* so that an explicitly provided hook that comes too "late" can be detected reliably. These *strategic places* @@ -611,8 +744,7 @@ The experimental `nodestroy`:idx: pragma inhibits hook injections. This can be used to specialize the object traversal in order to avoid deep recursions: -.. code-block:: nim - + ```nim test type Node = ref object x, y: int32 left, right: Node @@ -620,18 +752,19 @@ used to specialize the object traversal in order to avoid deep recursions: type Tree = object root: Node - proc `=destroy`(t: var Tree) {.nodestroy.} = + proc `=destroy`(t: Tree) {.nodestroy.} = # use an explicit stack so that we do not get stack overflows: var s: seq[Node] = @[t.root] while s.len > 0: let x = s.pop if x.left != nil: s.add(x.left) if x.right != nil: s.add(x.right) - # free the memory explicit: - dispose(x) + # free the memory explicitly: + `=dispose`(x) # notice how even the destructor for 's' is not called implicitly # anymore thanks to .nodestroy, so we have to call it on our own: `=destroy`(s) + ``` As can be seen from the example, this solution is hardly sufficient and @@ -641,63 +774,28 @@ should eventually be replaced by a better solution. Copy on write ============= -String literals are implemented as [copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write). +String literals are implemented as "copy on write". When assigning a string literal to a variable, a copy of the literal won't be created. Instead the variable simply points to the literal. The literal is shared between different variables which are pointing to it. The copy operation is deferred until the first write. -.. code-block:: nim - var x = "abc" # no copy - var y = x # no copy - -The string literal "abc" is stored in static memory and not allocated on the heap. -The variable `x` points to the literal and the variable `y` points to the literal too. -There is no copy during assigning operations. +For example: -.. code-block:: nim + ```nim var x = "abc" # no copy var y = x # no copy y[0] = 'h' # copy + ``` -The program above shows when the copy operations happen. -When mutating the variable `y`, the Nim compiler creates a fresh copy of `x`, -the variable `y` won't point to the string literal anymore. -Instead it points to the copy of `x` of which the memory can be mutated -and the variable `y` becomes a mutable string. - -.. Note:: The abstraction fails for `addr x` because whether the address is going to be used for mutations is unknown. +The abstraction fails for `addr x` because whether the address is going to be used for mutations is unknown. +`prepareMutation` needs to be called before the "address of" operation. For example: -Let's look at a silly example demonstrating this behaviour: - -.. code-block:: nim - var x = "abc" - var y = x - - moveMem(addr y[0], addr x[0], 3) - -The program fails because we need to prepare a fresh copy for the variable `y`. -`prepareMutation` should be called before the address operation. - -.. code-block:: nim - var x = "abc" - var y = x - - prepareMutation(y) - moveMem(addr y[0], addr x[0], 3) - assert y == "abc" - -Now `prepareMutation` solves the problem. -It manually creates a fresh copy and makes the variable `y` mutable. - -.. code-block:: nim + ```nim var x = "abc" var y = x prepareMutation(y) moveMem(addr y[0], addr x[0], 3) - moveMem(addr y[0], addr x[0], 3) - moveMem(addr y[0], addr x[0], 3) assert y == "abc" - -No matter how many times `moveMem` is called, the program compiles and runs. + ``` diff --git a/doc/docgen.md b/doc/docgen.md new file mode 100644 index 000000000..3cc75fc18 --- /dev/null +++ b/doc/docgen.md @@ -0,0 +1,891 @@ +=================================== + Nim DocGen Tools Guide +=================================== + +:Author: Erik O'Leary +:Version: |nimversion| + +.. default-role:: code +.. include:: rstcommon.rst +.. contents:: + +.. importdoc:: markdown_rst.md, compiler/docgen.nim + +Introduction +============ + +This document describes the `documentation generation tools`:idx: built into +the [Nim compiler](nimc.html), which can generate HTML, Latex and JSON output +from input ``.nim`` files and projects. +The output documentation will include the module +dependencies (`import`), any top-level documentation comments (`##`), and +exported symbols (`*`), including procedures, types, and variables. + +=================== ============== +command output format +=================== ============== +`nim doc`:cmd: ``.html`` HTML +`nim doc2tex`:cmd: ``.tex`` LaTeX +`nim jsondoc`:cmd: ``.json`` JSON +=================== ============== + +Nim can generate HTML and LaTeX from input Markdown and +RST (reStructuredText) files as well, which is intended for writing +standalone documents like user's guides and technical specifications. +See [Nim-flavored Markdown and reStructuredText] document for the description +of this feature and particularly section [Command line usage] for the full +list of supported commands. + +Quick start +----------- + +Generate HTML documentation for a file: + + ```cmd + nim doc <filename>.nim + ``` + +Generate HTML documentation for a whole project: + + ```cmd + # delete any htmldocs/*.idx file before starting + nim doc --project --index:on --git.url:<url> --git.commit:<tag> --outdir:htmldocs <main_filename>.nim + # this will generate html files, a theindex.html index, css and js under `htmldocs` + # See also `--docroot` to specify a relative root. + # to get search (dochacks.js) to work locally, you need a server otherwise + # CORS will prevent opening file:// urls; this works: + python3 -m http.server 7029 --directory htmldocs + # When --outdir is omitted it defaults to $projectPath/htmldocs, + # or `$nimcache/htmldocs` with `--usenimcache` which avoids clobbering your sources; + # and likewise without `--project`. + # Adding `-r` will open in a browser directly. + # Use `--showNonExports` to show non-exported fields of an exported type. + ``` + +Documentation Comments +---------------------- + +Any comments which are preceded by a double-hash (`##`), are interpreted as +documentation. Comments are parsed as RST (see [reference]( +http://docutils.sourceforge.net/docs/user/rst/quickref.html)), providing +Nim module authors the ability to easily generate richly formatted +documentation with only their well-documented code! +Basic Markdown syntax is also supported inside the doc comments. + +Example: + + ```nim + type Person* = object + ## This type contains a description of a person + name: string + age: int + ``` + +Outputs: + + Person* = object + name: string + age: int + + This type contains a description of a person + +Field documentation comments can be added to fields like so: + + ```nim + var numValues: int ## \ + ## `numValues` stores the number of values + ``` + +Note that without the `*` following the name of the type, the documentation for +this type would not be generated. Documentation will only be generated for +*exported* types/procedures/etc. + +It's recommended to always add exactly **one** space after `##` for readability +of comments — this extra space will be cropped from the parsed comments and +won't influence RST formatting. + +.. note:: Generally, this baseline indentation level inside a documentation + comment may not be 1: it can be any since it is determined by the offset + of the first non-whitespace character in the comment. + After that indentation **must** be consistent on the following lines of + the same comment. + If you still need to add an additional indentation at the very beginning + (for RST block quote syntax) use backslash \\ before it: + + ```nim + ## \ + ## + ## Block quote at the first line. + ## + ## Paragraph. + ``` + +Structuring output directories +------------------------------ + +Basic directory for output is set by `--outdir:OUTDIR`:option: switch, +by default `OUTDIR` is ``htmldocs`` sub-directory in the directory of +the processed file. + +There are 2 basic options as to how generated HTML output files are stored: + +1) complex hierarchy when docgen-compiling with `--project`:option:, + which follows directory structure of the project itself. + So `nim doc`:cmd: replicates project's directory structure + inside `--outdir:OUTDIR`:option: directory. + `--project`:option: is well suited for projects that have 1 main module. + File name clashes are impossible in this case. + +2) flattened structure, where user-provided script goes through all + needed input files and calls commands like `nim doc`:cmd: + with `--outdir:OUTDIR`:option: switch, thus putting all HTML (and + ``.idx``) files into 1 directory. + + .. Important:: Make sure that you don't have files with same base name + like ``x.nim`` and ``x.md`` in the same package, otherwise you'll + have name conflict for ``x.html``. + + .. Tip:: To structure your output directories and avoid file name + clashes you can split your project into + different *packages* -- parts of your repository that are + docgen-compiled with different `--outdir:OUTDIR`:option: options. + + An example of such strategy is Nim repository itself which has: + + * its stdlib ``.nim`` files from different directories and ``.md`` + documentation from ``doc/`` directory are all docgen-compiled + into `--outdir:web/upload/<version>/`:option: directory + * its ``.nim`` files from ``compiler/`` directory are docgen-compiled + into `--outdir:web/upload/<version>/compiler/`:option: directory. + Interestingly, it's compiled with complex hierarchy using + `--project`:option: switch. + + Contents of ``web/upload/<version>`` are then deployed into Nim's + Web server. + + This output directory structure allows to work correctly with files like + ``compiler/docgen.nim`` (implementation) and ``doc/docgen.md`` (user + documentation) in 1 repository. + + +Index files +----------- + +Index (``.idx``) files are used for 2 different purposes: + +1. easy cross-referencing between different ``.nim`` and/or ``.md`` / ``.rst`` + files described in [Nim external referencing] +2. creating a whole-project index for searching of symbols and keywords, + see [Buildindex command]. + + +Document Types +============== + +Example of Nim file input +------------------------- + +The following examples will generate documentation for this sample +*Nim* module, aptly named ``doc/docgen_sample.nim``: + + ```nim file=docgen_sample.nim + ``` + +All the below commands save their output to ``htmldocs`` directory relative to +the directory of file; +hence the output for this sample will be in ``doc/htmldocs``. + +HTML +---- + +The generation of HTML documents is done via the `doc`:option: command. This command +takes either a single ``.nim`` file, outputting a single ``.html`` file with the same +base filename, or multiple ``.nim`` files, outputting multiple ``.html`` files and, +optionally, an index file. + +The `doc`:option: command: + + ```cmd + nim doc docgen_sample.nim + ``` + +Partial Output: + + ... + proc helloWorld(times: int) {.raises: [], tags: [].} + ... + +The full output can be seen here: [docgen_sample.html](docgen_sample.html). +It runs after semantic checking and includes pragmas attached implicitly by the +compiler. + +LaTeX +----- + +LaTeX files are intended to be converted to PDF, especially for offline +reading or making hard copies. (LaTeX output is oftentimes better than +HTML -> PDF conversion). + +The `doc2tex`:option: command: + + ```cmd + nim doc2tex docgen_sample.nim + cd htmldocs + xelatex docgen_sample.tex + xelatex docgen_sample.tex + # It is usually necessary to run `xelatex` 2 times (or even 3 times for + # large documents) to get all labels generated. + # That depends on this warning in the end of `xelatex` output: + # LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right. + ``` + +The output is ``docgen_sample.pdf``. + +JSON +---- + +The generation of JSON documents is done via the `jsondoc`:option: command. +This command takes in a ``.nim`` file and outputs a ``.json`` file with +the same base filename. +Note that this tool is built off of the `doc`:option: command +(previously `doc2`:option:), and contains the same information. + +The `jsondoc`:option: command: + + ```cmd + nim jsondoc docgen_sample.nim + ``` + +Output: + + { + "orig": "docgen_sample.nim", + "nimble": "", + "moduleDescription": "This module is a sample", + "entries": [ + { + "name": "helloWorld", + "type": "skProc", + "line": 5, + "col": 0, + "description": "Takes an integer and outputs as many "hello world!"s", + "code": "proc helloWorld(times: int) {.raises: [], tags: [].}" + } + ] + } + +Similarly to the old `doc`:option: command, the old `jsondoc`:option: command has been +renamed to `jsondoc0`:option:. + +The `jsondoc0`:option: command: + + ```cmd + nim jsondoc0 docgen_sample.nim + ``` + +Output: + + [ + { + "comment": "This module is a sample." + }, + { + "name": "helloWorld", + "type": "skProc", + "description": "Takes an integer and outputs as many "hello world!"s", + "code": "proc helloWorld*(times: int)" + } + ] + +Note that the `jsondoc`:option: command outputs its JSON without pretty-printing it, +while `jsondoc0`:option: outputs pretty-printed JSON. + + +Simple documentation links +========================== + +It's possible to use normal Markdown/RST syntax to *manually* +reference Nim symbols using HTML anchors, however Nim has an *automatic* +facility that makes referencing inside ``.nim`` and ``.md/.rst`` files and +between them easy and seamless. +The point is that such links will be resolved automatically +by `nim doc`:cmd: (or `md2html`:option:, or `jsondoc`:option:, +or `doc2tex`:option:, ...). And, unlike manual links, such automatic +links **check** that their target exists -- a warning is emitted for +any broken link, so you avoid broken links in your project. + +Nim treats both ``.md/.rst`` files and ``.nim`` modules (their doc comment +part) as *documents* uniformly. +Hence all directions of referencing are equally possible having the same syntax: + +1. ``.md/rst`` -> itself (internal). See [Markup local referencing]. +2. ``.md/rst`` -> external ``.md/rst``. See [Markup external referencing]. + To summarize, referencing in `.md`/`.rst` files was already described in + [Nim-flavored Markdown and reStructuredText] + (particularly it described usage of index files for referencing), + while in this document we focus on Nim-specific details. +3. ``.md/rst`` -> external ``.nim``. See [Nim external referencing]. +4. ``.nim`` -> itself (internal). See [Nim local referencing]. +5. ``.nim`` -> external ``.md/rst``. See [Markup external referencing]. +6. ``.nim`` -> external ``.nim``. See [Nim external referencing]. + +To put it shortly, local referencing always works out of the box, +external referencing requires to use ``.. importdoc:: <file>`` +directive to import `file` and to ensure that the corresponding +``.idx`` file was generated. + +Syntax for referencing is basically the same as for normal markup. +Recall from [Referencing] that our parser supports two equivalent syntaxes +for referencing, Markdown and RST one. +So to reference ``proc f`` one should use something like that, +depending on markup type: + + Markdown RST + + Ref. [proc f] or [f] Ref. `proc f`_ or just f_ for a one-word case + +Nim local referencing +--------------------- + +You can reference Nim identifiers from Nim documentation comments +inside their ``.nim`` file (or inside a ``.rst`` file included from +a ``.nim``). +This pertains to any exported symbol like `proc`, `const`, `iterator`, etc. +Link text is either one word or a group of words enclosed by delimiters +(brackets ``[...]`` for Markdown or backticks `\`...\`_` for RST). +Link text will be displayed *as is* while *link target* will be set to +the anchor [^1] of Nim symbol that corresponds to link text. + +[^1] anchors' format is described in [HTML anchor generation] section below. + +If you have a constant: + + ```Nim + const pi* = 3.14 + ``` + +then it should be referenced in one of the 2 forms: + +A. non-qualified (no symbol kind specification): + + pi_ + +B. qualified (with symbol kind specification): + + `const pi`_ + +For routine kinds there are more options. Consider this definition: + + ```Nim + proc foo*(a: int, b: float): string + ``` + +Generally following syntax is allowed for referencing `foo`: + +* short (without parameters): + + A. non-qualified: + + foo_ + + B. qualified: + + `proc foo`_ + +* longer variants (with parameters): + + A. non-qualified: + + 1) specifying parameters names: + + `foo(a, b)`_ + + 2) specifying parameters types: + + `foo(int, float)`_ + + 3) specifying both names and types: + + `foo(a: int, b: float)`_ + + 4) output parameter can also be specified if you wish: + + `foo(a: int, b: float): string`_ + + B. qualified: all 4 options above are valid. + Particularly you can use the full format: + + `proc foo(a: int, b: float): string`_ + +.. Tip:: Avoid cluttering your text with extraneous information by using + one of shorter forms: + + binarySearch_ + `binarySearch(a, key, cmp)`_ + + Brevity is better for reading! If you use a short form and have an + ambiguity problem (see below) then just add some additional info. + +Symbol kind like `proc` can also be specified in the postfix form: + + `foo proc`_ + `walkDir(d: string) iterator`_ + +.. Warning:: An ambiguity in resolving documentation links may arise because of: + + 1. clash with other RST anchors + * manually setup anchors + * automatically set up, e.g. section names + 2. collision with other Nim symbols: + + * routines with different parameters can exist e.g. for + `proc` and `template`. In this case they are split between their + corresponding sections in output file. Qualified references are + useful in this case -- just disambiguate by referring to these + sections explicitly: + + See `foo proc`_ and `foo template`_. + + * because in Nim `proc` and `iterator` belong to different namespaces, + so there can be a collision even if parameters are the same. + Use `\`proc foo\`_`:literal: or `\`iterator foo\`_`:literal: then. + + Any ambiguity is always reported with Nim compiler warnings and an anchor + with higher priority is selected. Manual anchors have highest + priority, then go automatic RST anchors; then Nim-generated anchors + (while procs have higher priority than other Nim symbol kinds). + +Generic parameters can also be used. All in all, this long form will be +recognized fine: + + `proc binarySearch*[T; K](a: openArray[T], key: K, cmp: proc(T, K)): int`_ + +**Limitations**: + +1. The parameters of a nested routine type can be specified only with types + (without parameter names, see form A.2 above). + E.g. for this signature: + + ```Nim + proc binarySearch*[T, K](a: openArray[T]; key: K; + cmp: proc (x: T; y: K): int {.closure.}): int + ~~ ~~ ~~~~~ + ``` + + you cannot use names underlined by `~~` so it must be referenced with + ``cmp: proc(T, K)``. Hence these forms are valid: + + `binarySearch(a: openArray[T], key: K, cmp: proc(T, K))`_ + `binarySearch(openArray[T], K, proc(T, K))`_ + `binarySearch(a, key, cmp)`_ +2. Default values in routine parameters are not recognized, one needs to + specify the type and/or name instead. E.g. for referencing `proc f(x = 7)` + use one of the mentioned forms: + + `f(int)`_ or `f(x)`_ or `f(x: int)`_. +3. Generic parameters must be given the same way as in the + definition of referenced symbol. + + * their names should be the same + * parameters list should be given the same way, e.g. without substitutions + between commas (,) and semicolons (;). + +.. Note:: A bit special case is operators + (as their signature is also defined with `\``): + + ```Nim + func `$`(x: MyType): string + func `[]`*[T](x: openArray[T]): T + ``` + + A short form works without additional backticks: + + `$`_ + `[]`_ + + However for fully-qualified reference copy-pasting backticks (`) into other + backticks will not work in our RST parser (because we use Markdown-like + inline markup rules). You need either to delete backticks or keep + them and escape with backslash \\: + + no backticks: `func $`_ + escaped: `func \`$\``_ + no backticks: `func [][T](x: openArray[T]): T`_ + escaped: `func \`[]\`[T](x: openArray[T]): T`_ + +.. Note:: Types that defined as `enum`, or `object`, or `tuple` can also be + referenced with those names directly (instead of `type`): + + type CopyFlag = enum + ... + ## Ref. `CopyFlag enum`_ + +Nim external referencing +------------------------ + +Just like for [Markup external referencing], which saves markup anchors, +the Nim symbols are also saved in ``.idx`` files, so one needs +to generate them beforehand, and they should be loaded by +an ``.. importdoc::`` directive. Arguments to ``.. importdoc::`` is a +comma-separated list of Nim modules or Markdown/RST documents. + +`--index:only`:option: tells Nim to only generate ``.idx`` file and +do **not** attempt to generate HTML/LaTeX output. +For ``.nim`` modules there are 2 alternatives to work with ``.idx`` files: + +1. using [Project switch] implies generation of ``.idx`` files, + however, if ``importdoc`` is called on upper modules as its arguments, + their ``.idx`` are not yet created. Thus one should generate **all** + required ``.idx`` first: + ```cmd + nim doc --project --index:only <main>.nim + nim doc --project <main>.nim + ``` +2. or run `nim doc --index:only <module.nim>`:cmd: command for **all** (used) + Nim modules in your project. Then run `nim doc <module.nim>` on them for + output HTML generation. + + .. Warning:: A mere `nim doc --index:on`:cmd: may fail on an attempt to do + ``importdoc`` from another module (for which ``.idx`` was not yet + generated), that's why `--index:only`:option: shall be used instead. + + For ``.md``/``.rst`` markup documents point 2 is the only option. + +Then, you can freely use something like this in ``your_module.nim``: + + ```nim + ## .. importdoc:: user_manual.md, another_module.nim + + ... + ## Ref. [some section from User Manual]. + + ... + ## Ref. [proc f] + ## (assuming you have a proc `f` in ``another_module``). + ``` + +and compile it by `nim doc`:cmd:. Note that link text will +be automatically prefixed by the module name of symbol, +so you will see something like "Ref. [another_module: proc f](#)" +in the generated output. + +It's also possible to reference a whole module by prefixing or +suffixing full canonical module name with "module": + + Ref. [module subdir/name] or [subdir/name module]. + +Markup documents as a whole can be referenced just by their title +(or by their file name if the title was not set) without any prefix. + +.. Tip:: During development process the stage of ``.idx`` files generation + can be done only *once*, after that you use already generated ``.idx`` + files while working with a document *being developed* (unless you do + incompatible changes to *referenced* documents). + +.. Hint:: After changing a *referenced* document file one may need + to regenerate its corresponding ``.idx`` file to get correct results. + Of course, when referencing *internally* inside any given ``.nim`` file, + it's not needed, one can even immediately use any freshly added anchor + (a document's own ``.idx`` file is not used for resolving its internal links). + +If an ``importdoc`` directive fails to find a ``.idx``, then an error +is emitted. + +In case of such compilation failures please note that: + +* **all** relative paths, given to ``importdoc``, relate to insides of + ``OUTDIR``, and **not** project's directory structure. + +* ``importdoc`` searches for ``.idx`` in `--outdir:OUTDIR`:option: directory + (``htmldocs`` by default) and **not** around original modules, so: + + .. Tip:: look into ``OUTDIR`` to understand what's going on. + +* also keep in mind that ``.html`` and ``.idx`` files should always be + output to the same directory, so check this and, if it's not true, check + that both runs *with* and *without* `--index:only`:option: have all + other options the same. + +To summarize, for 2 basic options of [Structuring output directories] +compilation options are different: + +1) complex hierarchy with `--project`:option: switch. + + As the **original** project's directory structure is replicated in + `OUTDIR`, all passed paths are related to this structure also. + + E.g. if a module ``path1/module.nim`` does + ``.. importdoc:: path2/another.nim`` then docgen tries to load file + ``OUTDIR/path1/path2/another.idx``. + + .. Note:: markup documents are just placed into the specified directory + `OUTDIR`:option: by default (i.e. they are **not** affected by + `--project`:option:), so if you have ``PROJECT/doc/manual.md`` + document and want to use complex hierarchy (with ``doc/``), + compile it with `--docroot`:option:\: + ```cmd + # 1st stage + nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ + --index:only PROJECT/doc/manual.md + ... + # 2nd stage + nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ + PROJECT/doc/manual.md + ``` + + Then the output file will be placed as ``OUTDIR/doc/manual.idx``. + So if you have ``PROJECT/path1/module.nim``, then ``manual.md`` can + be referenced as ``../doc/manual.md``. + +2) flattened structure. + + E.g. if a module ``path1/module.nim`` does + ``.. importdoc:: path2/another.nim`` then docgen tries to load + ``OUTDIR/path2/another.idx``, so the path ``path1`` + does not matter and providing ``path2`` can be useful only + in the case it contains another package that was placed there + using `--outdir:OUTDIR/path2`:option:. + + The links' text will be prefixed as ``another: ...`` in both cases. + + .. Warning:: Again, the same `--outdir:OUTDIR`:option: option should + be provided to both `doc --index:only`:option: / + `md2html --index:only`:option: and final generation by + `doc`:option:/`md2html`:option: inside 1 package. + +To temporarily disable ``importdoc``, e.g. if you don't need +correct link resolution at the moment, use a `--noImportdoc`:option: switch +(only warnings about unresolved links will be generated for external references). + +Related Options +=============== + +Project switch +-------------- + + ```cmd + nim doc --project filename.nim + ``` + +This will recursively generate documentation of all Nim modules imported +into the input module that belong to the Nimble package that ``filename.nim`` +belongs to. The index files and the corresponding ``theindex.html`` will +also be generated. + + +Index switch +------------ + + ```cmd + nim doc --index:on filename.nim + ``` + +This will generate an index of all the exported symbols in the input Nim +module, and put it into a neighboring file with the extension of ``.idx``. The +index file is line-oriented (newlines have to be escaped). Each line +represents a tab-separated record of several columns, the first two mandatory, +the rest optional. See the [Index (idx) file format] section for details. + +.. Note:: `--index`:option: switch only affects creation of ``.idx`` + index files, while user-searchable Index HTML file is created by + `buildIndex`:option: command. + +Buildindex command +------------------ + +Once index files have been generated for one or more modules, the Nim +compiler command `nim buildIndex directory`:cmd: can be run to go over all the index +files in the specified directory to generate a [theindex.html](theindex.html) +file: + + ```cmd + nim buildIndex -o:path/to/htmldocs/theindex.html path/to/htmldocs + ``` + +See source switch +----------------- + + ```cmd + nim doc --git.url:<url> filename.nim + ``` + +With the `git.url`:option: switch the *See source* hyperlink will appear below each +documented item in your source code pointing to the implementation of that +item on a GitHub repository. +You can click the link to see the implementation of the item. + +The `git.commit`:option: switch overrides the hardcoded `devel` branch in +``config/nimdoc.cfg``. +This is useful to link to a different branch e.g. `--git.commit:master`:option:, +or to a tag e.g. `--git.commit:1.2.3`:option: or a commit. + +Source URLs are generated as ``href="${url}/tree/${commit}/${path}#L${line}"`` +by default and thus compatible with GitHub but not with GitLab. + +Similarly, `git.devel`:option: switch overrides the hardcoded `devel` branch +for the `Edit` link which is also useful if you have a different working +branch than `devel` e.g. `--git.devel:master`:option:. + +Edit URLs are generated as ``href="${url}/tree/${devel}/${path}#L${line}"`` +by default. + +You can edit ``config/nimdoc.cfg`` and modify the ``doc.item.seesrc`` value +with a hyperlink to your own code repository. + +In the case of Nim's own documentation, the `commit` value is just a commit +hash to append to a formatted URL to https://github.com/nim-lang/Nim. + + +Other Input Formats +=================== + +The *Nim compiler* also has support for RST (reStructuredText) files with +the `rst2html`:option: and `rst2tex`:option: commands. Documents like this one are +initially written in a dialect of RST which adds support for Nim source code +highlighting with the ``.. code-block:: nim`` prefix. ``code-block`` also +supports highlighting of a few other languages supported by the +[packages/docutils/highlite module](highlite.html). + +See [Markdown and RST markup languages](markdown_rst.html) for +usage of those commands. + +HTML anchor generation +====================== + +When you run the `rst2html`:option: command, all sections in the RST document will +get an anchor you can hyperlink to. Usually, you can guess the anchor lower +casing the section title and replacing spaces with dashes, and in any case, you +can get it from the table of contents. But when you run the `doc`:option: +command to generate API documentation, some symbol get one or two anchors at +the same time: a numerical identifier, or a plain name plus a complex name. + +The numerical identifier is just a random number. The number gets assigned +according to the section and position of the symbol in the file being processed +and you should not rely on it being constant: if you add or remove a symbol the +numbers may shuffle around. + +The plain name of a symbol is a simplified version of its fully exported +signature. Variables or constants have the same plain name symbol as their +complex name. The plain name for procs, templates, and other callable types +will be their unquoted value after removing parameters, return types, and +pragmas. The plain name allows short and nice linking of symbols that works +unless you have a module with collisions due to overloading. + +If you hyperlink a plain name symbol and there are other matches on the same +HTML file, most browsers will go to the first one. To differentiate the rest, +you will need to use the complex name. A complex name for a callable type is +made up of several parts: + + (**plain symbol**)(**.type**),(**first param**)?(**,param type**)\* + +The first thing to note is that all callable types have at least a comma, even +if they don't have any parameters. If there are parameters, they are +represented by their types and will be comma-separated. To the plain symbol a +suffix may be added depending on the type of the callable: + +============== ============== +Callable type Suffix +============== ============== +`proc`, `func` *empty string* +`macro` ``.m`` +`method` ``.e`` +`iterator` ``.i`` +`template` ``.t`` +`converter` ``.c`` +============== ============== + +The relationship of type to suffix is made by the proc `complexName` in the +``compiler/docgen.nim`` file. Here are some examples of complex names for +symbols in the [system module](system.html). + +* `type SomeSignedInt = int | int8 | int16 | int32 | int64` **=>** + [#SomeSignedInt](system.html#SomeSignedInt) +* `var globalRaiseHook: proc (e: ref E_Base): bool {.nimcall.}` **=>** + [#globalRaiseHook](system.html#globalRaiseHook) +* `const NimVersion = "0.0.0"` **=>** + [#NimVersion](system.html#NimVersion) +* `proc getTotalMem(): int {.rtl, raises: [], tags: [].}` **=>** + [#getTotalMem](system.html#getTotalMem) +* `proc len[T](x: seq[T]): int {.magic: "LengthSeq", noSideEffect.}` **=>** + [#len,seq[T]](system.html#len,seq[T]) +* `iterator pairs[T](a: seq[T]): tuple[key: int, val: T] {.inline.}` **=>** + [#pairs.i,seq[T]](iterators.html#pairs.i,seq[T]) +* `template newException[](exceptn: typedesc; message: string; + parentException: ref Exception = nil): untyped` **=>** + [#newException.t,typedesc,string,ref.Exception]( + system.html#newException.t,typedesc,string,ref.Exception) + + +Index (idx) file format +======================= + +Files with the ``.idx`` extension are generated when you use the [Index +switch] along with commands to generate +documentation from source or text files. You can programmatically generate +indices with the [setIndexTerm()]( +rstgen.html#setIndexTerm,RstGenerator,string,string,string,string,string) +and `writeIndexFile() <rstgen.html#writeIndexFile,RstGenerator,string>`_ procs. +The purpose of `idx` files is to hold the interesting symbols and their HTML +references so they can be later concatenated into a big index file with +[mergeIndexes()](rstgen.html#mergeIndexes,string). This section documents +the file format in detail. + +Index files are line-oriented and tab-separated (newline and tab characters +have to be escaped). Each line represents a record with 6 fields. +The content of these columns is: + +0. Discriminator tag denoting type of the index entry, allowed values are: + `markupTitle` + : a title for ``.md``/``.rst`` document + `nimTitle` + : a title of ``.nim`` module + `heading` + : heading of sections, can be both in Nim and markup files + `idx` + : terms marked with :idx: role + `nim` + : a Nim symbol + `nimgrp` + : a Nim group for overloadable symbols like `proc`s +1. Mandatory term being indexed. Terms can include quoting according to + Nim's rules (e.g. \`^\`). +2. Base filename plus anchor hyperlink (e.g. ``algorithm.html#*,int,SortOrder``). +3. Optional human-readable string to display as a hyperlink. If the value is not + present or is the empty string, the hyperlink will be rendered + using the term. Prefix whitespace indicates that this entry is + not for an API symbol but for a TOC entry. +4. Optional title or description of the hyperlink. Browsers usually display + this as a tooltip after hovering a moment over the hyperlink. +5. A line number of file where the entry was defined. + +The index generation tools differentiate between documentation +generated from ``.nim`` files and documentation generated from ``.md`` or +``.rst`` files by tag `nimTitle` or `markupTitle` in the 1st line of +the ``.idx`` file. + +.. TODO Normal symbols are added to the index with surrounding whitespaces removed. An + exception to this are the table of content (TOC) entries. TOC entries are added to + the index file with their third column having as much prefix spaces as their + level is in the TOC (at least 1 character). The prefix whitespace helps to + filter TOC entries from API or text symbols. This is important because the + amount of spaces is used to replicate the hierarchy for document TOCs in the + final index, and TOC entries found in ``.nim`` files are discarded. + + +Additional resources +==================== + +* [Nim Compiler User Guide](nimc.html#compiler-usage-commandminusline-switches) + +* already mentioned documentation for + [Markdown and RST markup languages](markdown_rst.html), which also + contains the list of implemented features of these markup languages. + +* the implementation is in [module compiler/docgen]. + +The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and +``config/nimdoc.tex.cfg`` configuration files. You can add and modify these +files to your project to change the look of the docgen output. + +You can import the [packages/docutils/rstgen module](rstgen.html) in your +programs if you want to reuse the compiler's documentation generation procs. diff --git a/doc/docgen.rst b/doc/docgen.rst deleted file mode 100644 index c002ddb83..000000000 --- a/doc/docgen.rst +++ /dev/null @@ -1,412 +0,0 @@ -.. default-role:: code - -=================================== - Nim DocGen Tools Guide -=================================== - -:Author: Erik O'Leary -:Version: |nimversion| - -.. contents:: - - -Introduction -============ - -This document describes the `documentation generation tools`:idx: built into -the `Nim compiler <nimc.html>`_, which can generate HTML and JSON output -from input .nim files and projects, as well as HTML and LaTeX from input RST -(reStructuredText) files. The output documentation will include the module -dependencies (`import`), any top-level documentation comments (##), and -exported symbols (*), including procedures, types, and variables. - -Quick start ------------ - -Generate HTML documentation for a file: - -:: - nim doc <filename>.nim - -Generate HTML documentation for a whole project: - -:: - # delete any htmldocs/*.idx file before starting - nim doc --project --index:on --git.url:<url> --git.commit:<tag> --outdir:htmldocs <main_filename>.nim - # this will generate html files, a theindex.html index, css and js under `htmldocs` - # See also `--docroot` to specify a relative root. - # to get search (dochacks.js) to work locally, you need a server otherwise - # CORS will prevent opening file:// urls; this works: - python3 -m http.server 7029 --directory htmldocs - # When --outdir is omitted it defaults to $projectPath/htmldocs, - or `$nimcache/htmldocs` with `--usenimcache` which avoids clobbering your sources; - and likewise without `--project`. - Adding `-r` will open in a browser directly. - - -Documentation Comments ----------------------- - -Any comments which are preceded by a double-hash (##), are interpreted as -documentation. Comments are parsed as RST (see `reference -<http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_), providing -Nim module authors the ability to easily generate richly formatted -documentation with only their well-documented code. - -Example: - -.. code-block:: nim - type Person* = object - ## This type contains a description of a person - name: string - age: int - -Outputs:: - Person* = object - name: string - age: int - -This type contains a description of a person - -Field documentation comments can be added to fields like so: - -.. code-block:: nim - var numValues: int ## \ - ## `numValues` stores the number of values - -Note that without the `*` following the name of the type, the documentation for -this type would not be generated. Documentation will only be generated for -*exported* types/procedures/etc. - -It's recommended to always add exactly **one** space after `##` for readability -of comments — this extra space will be cropped from the parsed comments and -won't influence RST formatting. - -.. note:: Generally, this baseline indentation level inside a documentation - comment may not be 1: it can be any since it is determined by the offset - of the first non-whitespace character in the comment. - After that indentation **must** be consistent on the following lines of - the same comment. - If you still need to add an additional indentation at the very beginning - (for RST block quote syntax) use backslash \\ before it: - - .. code-block:: nim - ## \ - ## - ## Block quote at the first line. - ## - ## Paragraph. - -Nim file input ------------------ - -The following examples will generate documentation for the below contrived -*Nim* module, aptly named 'sample.nim' - -sample.nim: - -.. code-block:: nim - ## This module is a sample. - - import std/strutils - - proc helloWorld*(times: int) = - ## Takes an integer and outputs - ## as many "hello world!"s - - for i in 0 .. times-1: - echo "hello world!" - - helloWorld(5) - - -Document Types -============== - - -HTML ----- - -The generation of HTML documents is done via the `doc` command. This command -takes either a single .nim file, outputting a single .html file with the same -base filename, or multiple .nim files, outputting multiple .html files and, -optionally, an index file. - -The `doc` command:: - nim doc sample - -Partial Output:: - ... - proc helloWorld(times: int) {.raises: [], tags: [].} - ... - -The full output can be seen here: `docgen_sample.html <docgen_sample.html>`_. -It runs after semantic checking and includes pragmas attached implicitly by the -compiler. - - -JSON ----- - -The generation of JSON documents is done via the `jsondoc` command. This command -takes in a .nim file and outputs a .json file with the same base filename. Note -that this tool is built off of the `doc` command (previously `doc2`), and -contains the same information. - -The `jsondoc` command:: - nim jsondoc sample - -Output:: - { - "orig": "docgen_sample.nim", - "nimble": "", - "moduleDescription": "This module is a sample", - "entries": [ - { - "name": "helloWorld", - "type": "skProc", - "line": 5, - "col": 0, - "description": "Takes an integer and outputs as many "hello world!"s", - "code": "proc helloWorld(times: int) {.raises: [], tags: [].}" - } - ] - } - -Similarly to the old `doc` command, the old `jsondoc` command has been -renamed to `jsondoc0`. - -The `jsondoc0` command:: - nim jsondoc0 sample - -Output:: - [ - { - "comment": "This module is a sample." - }, - { - "name": "helloWorld", - "type": "skProc", - "description": "Takes an integer and outputs as many "hello world!"s", - "code": "proc helloWorld*(times: int)" - } - ] - -Note that the `jsondoc` command outputs it's JSON without pretty-printing it, -while `jsondoc0` outputs pretty-printed JSON. - -Related Options -=============== - -Project switch --------------- - -:: - nim doc --project filename.nim - -This will recursively generate documentation of all nim modules imported -into the input module that belong to the Nimble package that `filename.nim` -belongs to. - - -Index switch ------------- - -:: - nim doc --index:on filename.nim - -This will generate an index of all the exported symbols in the input Nim -module, and put it into a neighboring file with the extension of `.idx`. The -index file is line-oriented (newlines have to be escaped). Each line -represents a tab-separated record of several columns, the first two mandatory, -the rest optional. See the `Index (idx) file format`_ section for details. - -Once index files have been generated for one or more modules, the Nim -compiler command `buildIndex directory` can be run to go over all the index -files in the specified directory to generate a `theindex.html <theindex.html>`_ -file. - -See source switch ------------------ - -:: - nim doc --git.url:<url> filename.nim - -With the `git.url` switch the *See source* hyperlink will appear below each -documented item in your source code pointing to the implementation of that -item on a GitHub repository. -You can click the link to see the implementation of the item. - -The `git.commit` switch overrides the hardcoded `devel` branch in config/nimdoc.cfg. -This is useful to link to a different branch e.g. `--git.commit:master`, -or to a tag e.g. `--git.commit:1.2.3` or a commit. - -Source URLs are generated as `href="${url}/tree/${commit}/${path}#L${line}"` by default and this compatible with GitHub but not with GitLab. - -Similarly, `git.devel` switch overrides the hardcoded `devel` branch for the `Edit` link which is also useful if you have a different working branch than `devel` e.g. `--git.devel:master`. - -Edit URLs are generated as `href="${url}/tree/${devel}/${path}#L${line}"` by default. - -You can edit `config/nimdoc.cfg` and modify the `doc.item.seesrc` value with a hyperlink to your own code repository. - -In the case of Nim's own documentation, the `commit` value is just a commit -hash to append to a formatted URL to https://github.com/nim-lang/Nim. The -`tools/nimweb.nim` helper queries the current git commit hash during the doc -generation, but since you might be working on an unpublished repository, it -also allows specifying a `githash` value in `web/website.ini` to force a -specific commit in the output. - - -Other Input Formats -=================== - -The *Nim compiler* also has support for RST (reStructuredText) files with -the `rst2html` and `rst2tex` commands. Documents like this one are -initially written in a dialect of RST which adds support for nim source code -highlighting with the `.. code-block:: nim` prefix. `code-block` also -supports highlighting of C++ and some other c-like languages. - -Usage:: - nim rst2html docgen.txt - -Output:: - You're reading it! - -The `rst2tex` command is invoked identically to `rst2html`, but outputs -a .tex file instead of .html. - - -HTML anchor generation -====================== - -When you run the `rst2html` command, all sections in the RST document will -get an anchor you can hyperlink to. Usually, you can guess the anchor lower -casing the section title and replacing spaces with dashes, and in any case, you -can get it from the table of contents. But when you run the `doc` -command to generate API documentation, some symbol get one or two anchors at -the same time: a numerical identifier, or a plain name plus a complex name. - -The numerical identifier is just a random number. The number gets assigned -according to the section and position of the symbol in the file being processed -and you should not rely on it being constant: if you add or remove a symbol the -numbers may shuffle around. - -The plain name of a symbol is a simplified version of its fully exported -signature. Variables or constants have the same plain name symbol as their -complex name. The plain name for procs, templates, and other callable types -will be their unquoted value after removing parameters, return types, and -pragmas. The plain name allows short and nice linking of symbols that works -unless you have a module with collisions due to overloading. - -If you hyperlink a plain name symbol and there are other matches on the same -HTML file, most browsers will go to the first one. To differentiate the rest, -you will need to use the complex name. A complex name for a callable type is -made up of several parts: - - (**plain symbol**)(**.type**),(**first param**)?(**,param type**)\* - -The first thing to note is that all callable types have at least a comma, even -if they don't have any parameters. If there are parameters, they are -represented by their types and will be comma-separated. To the plain symbol a -suffix may be added depending on the type of the callable: - -------------- -------------- -Callable type Suffix -------------- -------------- -proc *empty string* -macro `.m` -method `.e` -iterator `.i` -template `.t` -converter `.c` -------------- -------------- - -The relationship of type to suffix is made by the proc `complexName` in the -`compiler/docgen.nim` file. Here are some examples of complex names for -symbols in the `system module <system.html>`_. - -* `type SomeSignedInt = int | int8 | int16 | int32 | int64` **=>** - `#SomeSignedInt <system.html#SomeSignedInt>`_ -* `var globalRaiseHook: proc (e: ref E_Base): bool {.nimcall.}` **=>** - `#globalRaiseHook <system.html#globalRaiseHook>`_ -* `const NimVersion = "0.0.0"` **=>** - `#NimVersion <system.html#NimVersion>`_ -* `proc getTotalMem(): int {.rtl, raises: [], tags: [].}` **=>** - `#getTotalMem, <system.html#getTotalMem>`_ -* `proc len[T](x: seq[T]): int {.magic: "LengthSeq", noSideEffect.}` **=>** - `#len,seq[T] <system.html#len,seq[T]>`_ -* `iterator pairs[T](a: seq[T]): tuple[key: int, val: T] {.inline.}` **=>** - `#pairs.i,seq[T] <iterators.html#pairs.i,seq[T]>`_ -* `template newException[](exceptn: typedesc; message: string; - parentException: ref Exception = nil): untyped` **=>** - `#newException.t,typedesc,string,ref.Exception - <system.html#newException.t,typedesc,string,ref.Exception>`_ - - -Index (idx) file format -======================= - -Files with the `.idx` extension are generated when you use the `Index -switch <#related-options-index-switch>`_ along with commands to generate -documentation from source or text files. You can programmatically generate -indices with the `setIndexTerm() -<rstgen.html#setIndexTerm,RstGenerator,string,string,string,string,string>`_ -and `writeIndexFile() <rstgen.html#writeIndexFile,RstGenerator,string>`_ procs. -The purpose of `idx` files is to hold the interesting symbols and their HTML -references so they can be later concatenated into a big index file with -`mergeIndexes() <rstgen.html#mergeIndexes,string>`_. This section documents -the file format in detail. - -Index files are line-oriented and tab-separated (newline and tab characters -have to be escaped). Each line represents a record with at least two fields -but can have up to four (additional columns are ignored). The content of these -columns is: - -1. Mandatory term being indexed. Terms can include quoting according to - Nim's rules (e.g. \`^\`). -2. Base filename plus anchor hyperlink (e.g. `algorithm.html#*,int,SortOrder`). -3. Optional human-readable string to display as a hyperlink. If the value is not - present or is the empty string, the hyperlink will be rendered - using the term. Prefix whitespace indicates that this entry is - not for an API symbol but for a TOC entry. -4. Optional title or description of the hyperlink. Browsers usually display - this as a tooltip after hovering a moment over the hyperlink. - -The index generation tools try to differentiate between documentation -generated from `.nim` files and documentation generated from `.txt` or -`.rst` files. The former are always closely related to source code and -consist mainly of API entries. The latter are generic documents meant for -human reading. - -To differentiate both types (documents and APIs), the index generator will add -to the index of documents an entry with the title of the document. Since the -title is the topmost element, it will be added with a second field containing -just the filename without any HTML anchor. By convention, this entry without -anchor is the *title entry*, and since entries in the index file are added as -they are scanned, the title entry will be the first line. The title for APIs -is not present because it can be generated concatenating the name of the file -to the word **Module**. - -Normal symbols are added to the index with surrounding whitespaces removed. An -exception to this are the table of content (TOC) entries. TOC entries are added to -the index file with their third column having as much prefix spaces as their -level is in the TOC (at least 1 character). The prefix whitespace helps to -filter TOC entries from API or text symbols. This is important because the -amount of spaces is used to replicate the hierarchy for document TOCs in the -final index, and TOC entries found in `.nim` files are discarded. - - -Additional resources -==================== - -`Nim Compiler User Guide <nimc.html#compiler-usage-commandminusline-switches>`_ - -`RST Quick Reference -<http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_ - -The output for HTML and LaTeX comes from the `config/nimdoc.cfg` and -`config/nimdoc.tex.cfg` configuration files. You can add and modify these -files to your project to change the look of the docgen output. - -You can import the `packages/docutils/rstgen module <rstgen.html>`_ in your -programs if you want to reuse the compiler's documentation generation procs. diff --git a/doc/docgen_sample.nim b/doc/docgen_sample.nim index 7a167cb45..06b8d7f8e 100644 --- a/doc/docgen_sample.nim +++ b/doc/docgen_sample.nim @@ -1,6 +1,6 @@ ## This module is a sample. -import strutils +import std/strutils proc helloWorld*(times: int) = ## Takes an integer and outputs diff --git a/doc/docs.rst b/doc/docs.md index b34383253..b6ff6d2c7 100644 --- a/doc/docs.rst +++ b/doc/docs.md @@ -1,38 +1,38 @@ The documentation consists of several documents: -- | `Tutorial (part I) <tut1.html>`_ +- | [Tutorial (part I)](tut1.html) | The Nim tutorial part one deals with the basics. -- | `Tutorial (part II) <tut2.html>`_ +- | [Tutorial (part II)](tut2.html) | The Nim tutorial part two deals with the advanced language constructs. -- | `Tutorial (part III) <tut3.html>`_ +- | [Tutorial (part III)](tut3.html) | The Nim tutorial part three about Nim's macro system. -- | `Language Manual <manual.html>`_ +- | [Language Manual](manual.html) | The Nim manual is a draft that will evolve into a proper specification. -- | `Library documentation <lib.html>`_ +- | [Library documentation](lib.html) | This document describes Nim's standard library. -- | `Compiler user guide <nimc.html>`_ +- | [Compiler user guide](nimc.html) | The user guide lists command line arguments, special features of the compiler, etc. -- | `Tools documentation <tools.html>`_ +- | [Tools documentation](tools.html) | Description of some tools that come with the standard distribution. -- | `GC <gc.html>`_ - | Additional documentation about Nim's multi-paradigm memory management strategies +- | [Memory management](mm.html) + | Additional documentation about Nim's memory management strategies | and how to operate them in a realtime setting. -- | `Source code filters <filters.html>`_ +- | [Source code filters](filters.html) | The Nim compiler supports source code filters as a simple yet powerful builtin templating system. -- | `Internal documentation <intern.html>`_ +- | [Internal documentation](intern.html) | The internal documentation describes how the compiler is implemented. Read this if you want to hack the compiler. -- | `Index <theindex.html>`_ +- | [Index](theindex.html) | The generated index. **Index + (Ctrl+F) == Joy** diff --git a/doc/docstyle.rst b/doc/docstyle.md index 7f3fa8cf2..291a34cf6 100644 --- a/doc/docstyle.rst +++ b/doc/docstyle.md @@ -4,51 +4,52 @@ Documentation Style General Guidelines ------------------ -* See also `nep1<https://nim-lang.github.io/Nim/nep1.html>`_ which should probably be merged here. +* See also [nep1](nep1.html) which should probably be merged here. * Authors should document anything that is exported; documentation for private - procs can be useful too (visible via `nim doc --docInternal foo.nim`). + procs can be useful too (visible via `nim doc --docInternal foo.nim`:cmd:). * Within documentation, a period (`.`) should follow each sentence (or sentence fragment) in a comment block. The documentation may be limited to one sentence fragment, but if multiple sentences are within the documentation, each sentence after the first should be complete and in present tense. * Documentation is parsed as a custom ReStructuredText (RST) with partial markdown support. * In nim sources, prefer single backticks to double backticks since it's simpler - and `nim doc` supports it. Likewise with rst files: `nim rst2html` will render those as monospace, and - adding `.. default-role:: code` to an rst file will also make those render as monospace when rendered directly + and `nim doc`:cmd: supports it. Likewise with ``rst`` files: `nim rst2html`:cmd: will render those as monospace, and + adding ``.. default-role:: code`` to an ``rst`` file will also make those render as monospace when rendered directly in tools such as github. -* In nim sources, for links, prefer `[link text](link.html)` to `` `link text<link.html>`_ `` - since the syntax is simpler and markdown is more common (likewise, `nim rst2html` also supports it in rst files). - -.. code-block:: nim +* (debatable) In nim sources, for links, prefer ``[link text](link.html)`` to `\`link text<link.html>\`_`:code: + since the syntax is simpler and markdown is more common (likewise, `nim rst2html`:cmd: also supports it in ``rst`` files). + ```nim proc someproc*(s: string, foo: int) = ## Use single backticks for inline code, e.g.: `s` or `someExpr(true)`. ## Use a backlash to follow with alphanumeric char: `int8`\s are great. + ``` Module-level documentation -------------------------- Documentation of a module is placed at the top of the module itself. Each line of documentation begins with double hashes (`##`). -Sometimes `##[ multiline docs containing code ]##` is preferable, see `lib/pure/times.nim`. +Sometimes `##[ multiline docs containing code ]##` is preferable, see ``lib/pure/times.nim``. Code samples are encouraged, and should follow the general RST syntax: -.. code-block:: Nim - + ````Nim ## The `universe` module computes the answer to life, the universe, and everything. ## - ## .. code-block:: - ## doAssert computeAnswerString() == 42 + ## ``` + ## doAssert computeAnswerString() == 42 + ## ``` + ```` Within this top-level comment, you can indicate the authorship and copyright of the code, which will be featured in the produced documentation. -.. code-block:: Nim - + ```Nim ## This is the best module ever. It provides answers to everything! ## ## :Author: Steve McQueen ## :Copyright: 1965 ## + ``` Leave a space between the last line of top-level documentation and the beginning of Nim code (the imports, etc.). @@ -57,38 +58,38 @@ Procs, Templates, Macros, Converters, and Iterators The documentation of a procedure should begin with a capital letter and should be in present tense. Variables referenced in the documentation should be surrounded by single tick marks: -.. code-block:: Nim - + ```Nim proc example1*(x: int) = ## Prints the value of `x`. echo x + ``` Whenever an example of usage would be helpful to the user, you should include one within the documentation in RST format as below. -.. code-block:: Nim - + ````Nim proc addThree*(x, y, z: int8): int = ## Adds three `int8` values, treating them as unsigned and ## truncating the result. ## - ## .. code-block:: - ## # things that aren't suitable for a `runnableExamples` go in code-block: - ## echo execCmdEx("git pull") - ## drawOnScreen() + ## ``` + ## # things that aren't suitable for a `runnableExamples` go in code block: + ## echo execCmdEx("git pull") + ## drawOnScreen() + ## ``` runnableExamples: - # `runnableExamples` is usually preferred to `code-block`, when possible. + # `runnableExamples` is usually preferred to code blocks, when possible. doAssert addThree(3, 125, 6) == -122 result = x +% y +% z + ```` -The command `nim doc` will then correctly syntax highlight the Nim code within the documentation. +The command `nim doc`:cmd: will then correctly syntax highlight the Nim code within the documentation. Types ----- Exported types should also be documented. This documentation can also contain code samples, but those are better placed with the functions to which they refer. -.. code-block:: Nim - + ```Nim type NamedQueue*[T] = object ## Provides a linked data structure with names ## throughout. It is named for convenience. I'm making @@ -96,12 +97,12 @@ Exported types should also be documented. This documentation can also contain co name*: string ## The name of the item val*: T ## Its value next*: ref NamedQueue[T] ## The next item in the queue + ``` You have some flexibility when placing the documentation: -.. code-block:: Nim - + ```Nim type NamedQueue*[T] = object ## Provides a linked data structure with names @@ -110,11 +111,11 @@ You have some flexibility when placing the documentation: name*: string ## The name of the item val*: T ## Its value next*: ref NamedQueue[T] ## The next item in the queue + ``` Make sure to place the documentation beside or within the object. -.. code-block:: Nim - + ```Nim type ## Bad: this documentation disappears because it annotates the `type` keyword ## above, not `NamedQueue`. @@ -123,14 +124,14 @@ Make sure to place the documentation beside or within the object. ## is not what we want. val*: T ## Its value next*: ref NamedQueue[T] ## The next item in the queue + ``` Var, Let, and Const ------------------- When declaring module-wide constants and values, documentation is encouraged. The placement of doc comments is similar to the `type` sections. -.. code-block:: Nim - + ```Nim const X* = 42 ## An awesome number. SpreadArray* = [ @@ -138,25 +139,26 @@ When declaring module-wide constants and values, documentation is encouraged. Th [2,3,1], [3,1,2], ] ## Doc comment for `SpreadArray`. + ``` Placement of comments in other areas is usually allowed, but will not become part of the documentation output and should therefore be prefaced by a single hash (`#`). -.. code-block:: Nim - + ```Nim const BadMathVals* = [ 3.14, # pi 2.72, # e 0.58, # gamma ] ## A bunch of badly rounded values. + ``` Nim supports Unicode in comments, so the above can be replaced with the following: -.. code-block:: Nim - + ```Nim const BadMathVals* = [ 3.14, # Ï€ 2.72, # e 0.58, # γ ] ## A bunch of badly rounded values (including Ï€!). + ``` diff --git a/doc/drnim.rst b/doc/drnim.md index d33a6066e..1dc2b550f 100644 --- a/doc/drnim.rst +++ b/doc/drnim.md @@ -1,5 +1,3 @@ -.. default-role:: code - =================================== DrNim User Guide =================================== @@ -7,6 +5,8 @@ :Author: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: @@ -14,7 +14,7 @@ Introduction ============ This document describes the usage of the *DrNim* tool. DrNim combines -the Nim frontend with the `Z3 <https://github.com/Z3Prover/z3>`_ proof +the Nim frontend with the [Z3](https://github.com/Z3Prover/z3) proof engine, in order to allow verify/validate software written in Nim. DrNim's command-line options are the same as the Nim compiler's. @@ -22,11 +22,11 @@ DrNim's command-line options are the same as the Nim compiler's. DrNim currently only checks the sections of your code that are marked via `staticBoundChecks: on`: -.. code-block:: nim - + ```nim {.push staticBoundChecks: on.} # <--- code section here ----> {.pop.} + ``` DrNim currently only tries to prove array indexing or subrange checks, overflow errors are *not* prevented. Overflows will be checked for in @@ -43,7 +43,8 @@ your code. Installation ============ -Run `koch drnim`, the executable will afterwards be in `$nim/bin/drnim`. +Run `koch drnim`:cmd:, the executable will afterwards be +in ``$nim/bin/drnim``. Motivating Example @@ -52,8 +53,7 @@ Motivating Example The follow example highlights what DrNim can easily do, even without additional annotations: -.. code-block:: nim - + ```nim {.push staticBoundChecks: on.} proc sum(a: openArray[int]): int = @@ -63,11 +63,12 @@ without additional annotations: {.pop.} echo sum([1, 2, 3]) + ``` This program contains a famous "index out of bounds" bug. DrNim -detects it and produces the following error message:: +detects it and produces the following error message: - cannot prove: i <= len(a) + -1; counter example: i -> 0 a.len -> 0 [IndexCheck] + cannot prove: i <= len(a) + -1; counter example: i -> 0 a.len -> 0 [IndexCheck] In other words for `i == 0` and `a.len == 0` (for example!) there would be an index out of bounds error. @@ -124,8 +125,7 @@ Example: insertionSort **Note**: This example does not yet work with DrNim. -.. code-block:: nim - + ```nim import std / logic proc insertionSort(a: var openArray[int]) {. @@ -141,27 +141,28 @@ Example: insertionSort {.invariant: forall(j in 1..k, i in 0..<j, j == t or a[i] <= a[j]).} swap a[t], a[t-1] dec t + ``` Unfortunately, the invariants required to prove that this code is correct take more code than the imperative instructions. However, this effort can be compensated by the fact that the result needs very little testing. Be aware though that -DrNim only proves that after `insertionSort` this condition holds:: +DrNim only proves that after `insertionSort` this condition holds: - forall(i in 1..<a.len, a[i-1] <= a[i]) + forall(i in 1..<a.len, a[i-1] <= a[i]) This is required, but not sufficient to describe that a `sort` operation was performed. For example, the same postcondition is true for this proc which doesn't sort at all: -.. code-block:: nim - + ```nim import std / logic proc insertionSort(a: var openArray[int]) {. ensures: forall(i in 1..<a.len, a[i-1] <= a[i]).} = # does not sort, overwrites `a`'s contents! for i in 0..<a.len: a[i] = i + ``` @@ -169,23 +170,23 @@ Syntax of propositions ====================== The basic syntax is `ensures|requires|invariant: <prop>`. -A `prop` is either a comparison or a compound:: - - prop = nim_bool_expression - | prop 'and' prop - | prop 'or' prop - | prop '->' prop # implication - | prop '<->' prop - | 'not' prop - | '(' prop ')' # you can group props via () - | forallProp - | existsProp - - forallProp = 'forall' '(' quantifierList ',' prop ')' - existsProp = 'exists' '(' quantifierList ',' prop ')' - - quantifierList = quantifier (',' quantifier)* - quantifier = <new identifier> 'in' nim_iteration_expression +A `prop` is either a comparison or a compound: + + prop = nim_bool_expression + | prop 'and' prop + | prop 'or' prop + | prop '->' prop # implication + | prop '<->' prop + | 'not' prop + | '(' prop ')' # you can group props via () + | forallProp + | existsProp + + forallProp = 'forall' '(' quantifierList ',' prop ')' + existsProp = 'exists' '(' quantifierList ',' prop ')' + + quantifierList = quantifier (',' quantifier)* + quantifier = <new identifier> 'in' nim_iteration_expression `nim_iteration_expression` here is an ordinary expression of Nim code diff --git a/doc/effects.txt b/doc/effects.txt index 4ed1d09f1..df624e8b0 100644 --- a/doc/effects.txt +++ b/doc/effects.txt @@ -11,13 +11,14 @@ Iff a proc is side effect free and all its argument are evaluable at compile time, it can be evaluated by the compiler. However, really difficult is the ``newString`` proc: If it is simply wrapped, it should not be evaluated at compile time! On other occasions it can -and should be evaluted: +and should be evaluated: -.. code-block:: nim + ```nim proc toUpper(s: string): string = result = newString(len(s)) for i in 0..len(s) - 1: result[i] = toUpper(s[i]) + ``` No, it really can always be evaluated. The code generator should transform ``s = "\0\0\0..."`` back into ``s = newString(...)``. diff --git a/doc/estp.md b/doc/estp.md new file mode 100644 index 000000000..8a986bdf3 --- /dev/null +++ b/doc/estp.md @@ -0,0 +1,206 @@ +=================================================== + Embedded Stack Trace Profiler (ESTP) User Guide +=================================================== + +.. default-role:: code +.. include:: rstcommon.rst + +:Author: Andreas Rumpf +:Version: |nimversion| + + +Nim comes with a platform independent profiler - +the Embedded Stack Trace Profiler (ESTP). The profiler +is *embedded* into your executable. To activate the profiler you need to do: + +* compile your program with the `--profiler:on --stackTrace:on`:option: command + line options +* import the `nimprof` module +* run your program as usual. + +You can in fact look at `nimprof`'s source code to see how to implement +your own profiler. + +The setting `--profiler:on`:option: defines the conditional symbol `profiler`. +You can use `when compileOption("profiler")` to make the switch seamless. +If `profiler`:option: is `off`:option:, your program runs normally. +Otherwise your program is profiled. + +```nim +when compileOption("profiler"): + import std/nimprof +``` + +After your program has finished the profiler will create a +file ``profile_results.txt`` containing the profiling results. + +Since the profiler works by examining stack traces, it's essential that +the option `--stackTrace:on`:option: is active! Unfortunately this means that a +profiling build is much slower than a release build. + + +Memory profiler +=============== + +You can also use ESTP as a memory profiler to see which stack traces allocate +the most memory and thus create the most GC pressure. It may also help to +find memory leaks. To activate the memory profiler you need to do: + +* compile your program with the + `--profiler:off --stackTrace:on -d:memProfiler`:option: + command line options. Yes it's `--profiler:off`:option:. +* import the `nimprof` module +* run your program as usual. + +Define the symbol `ignoreAllocationSize` so that only the number of +allocations is counted and the sizes of the memory allocations do not matter. + + +Example results file +==================== + +The results file lists stack traces ordered by significance. + +The following example file has been generated by profiling the Nim compiler +itself: It shows that in total 5.4% of the runtime has been spent +in `crcFromRope` or its children. + +In general the stack traces show you immediately where the problem is because +the trace acts like an explanation; in traditional profilers you can only find +expensive leaf functions easily but the *reason* why they are invoked +often remains mysterious. + + total executions of each stack trace: + Entry: 0/3391 Calls: 84/4160 = 2.0% [sum: 84; 84/4160 = 2.0%] + newCrcFromRopeAux + crcFromRope + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 1/3391 Calls: 46/4160 = 1.1% [sum: 130; 130/4160 = 3.1%] + updateCrc32 + newCrcFromRopeAux + crcFromRope + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 2/3391 Calls: 41/4160 = 0.99% [sum: 171; 171/4160 = 4.1%] + updateCrc32 + updateCrc32 + newCrcFromRopeAux + crcFromRope + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 3/3391 Calls: 41/4160 = 0.99% [sum: 212; 212/4160 = 5.1%] + crcFromFile + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 4/3391 Calls: 41/4160 = 0.99% [sum: 253; 253/4160 = 6.1%] + updateCrc32 + crcFromFile + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 5/3391 Calls: 32/4160 = 0.77% [sum: 285; 285/4160 = 6.9%] + pop + newCrcFromRopeAux + crcFromRope + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + nim + Entry: 6/3391 Calls: 17/4160 = 0.41% [sum: 302; 302/4160 = 7.3%] + doOperation + forAllChildrenAux + pop + newCrcFromRopeAux + crcFromRope + writeRopeIfNotEqual + shouldRecompile + writeModule + myClose + closePasses + processModule + CompileModule + CompileProject + CommandCompileToC + MainCommand + HandleCmdLine + ... + nim + Entry: 7/3391 Calls: 14/4160 = 0.34% [sum: 316; 316/4160 = 7.6%] + Contains + isAccessible + interiorAllocatedPtr + gcMark + markStackAndRegisters + collectCTBody + collectCT + rawNewObj + newObj + newNode + copyTree + matchesAux + matches + resolveOverloads + semOverloadedCall + semOverloadedCallAnalyseEffects + ... + CommandCompileToC + MainCommand + HandleCmdLine diff --git a/doc/estp.rst b/doc/estp.rst deleted file mode 100644 index 8146562b6..000000000 --- a/doc/estp.rst +++ /dev/null @@ -1,197 +0,0 @@ -.. default-role:: code - -=================================================== - Embedded Stack Trace Profiler (ESTP) User Guide -=================================================== - -:Author: Andreas Rumpf -:Version: |nimversion| - - -Nim comes with a platform independent profiler - -the Embedded Stack Trace Profiler (ESTP). The profiler -is *embedded* into your executable. To activate the profiler you need to do: - -* compile your program with the `--profiler:on --stackTrace:on` command - line options -* import the `nimprof` module -* run your program as usual. - -You can in fact look at `nimprof`'s source code to see how to implement -your own profiler. - -The setting `--profiler:on` defines the conditional symbol `profiler`. - -After your program has finished the profiler will create a -file `profile_results.txt` containing the profiling results. - -Since the profiler works by examining stack traces, it's essential that -the option `--stackTrace:on` is active! Unfortunately this means that a -profiling build is much slower than a release build. - - -Memory profiler -=============== - -You can also use ESTP as a memory profiler to see which stack traces allocate -the most memory and thus create the most GC pressure. It may also help to -find memory leaks. To activate the memory profiler you need to do: - -* compile your program with the `--profiler:off --stackTrace:on -d:memProfiler` - command line options. Yes it's `--profiler:off`. -* import the `nimprof` module -* run your program as usual. - -Define the symbol `ignoreAllocationSize` so that only the number of -allocations is counted and the sizes of the memory allocations do not matter. - - -Example results file -==================== - -The results file lists stack traces ordered by significance. - -The following example file has been generated by profiling the Nim compiler -itself: It shows that in total 5.4% of the runtime has been spent -in `crcFromRope` or its children. - -In general the stack traces show you immediately where the problem is because -the trace acts like an explanation; in traditional profilers you can only find -expensive leaf functions easily but the *reason* why they are invoked -often remains mysterious. - -:: - total executions of each stack trace: - Entry: 0/3391 Calls: 84/4160 = 2.0% [sum: 84; 84/4160 = 2.0%] - newCrcFromRopeAux - crcFromRope - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 1/3391 Calls: 46/4160 = 1.1% [sum: 130; 130/4160 = 3.1%] - updateCrc32 - newCrcFromRopeAux - crcFromRope - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 2/3391 Calls: 41/4160 = 0.99% [sum: 171; 171/4160 = 4.1%] - updateCrc32 - updateCrc32 - newCrcFromRopeAux - crcFromRope - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 3/3391 Calls: 41/4160 = 0.99% [sum: 212; 212/4160 = 5.1%] - crcFromFile - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 4/3391 Calls: 41/4160 = 0.99% [sum: 253; 253/4160 = 6.1%] - updateCrc32 - crcFromFile - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 5/3391 Calls: 32/4160 = 0.77% [sum: 285; 285/4160 = 6.9%] - pop - newCrcFromRopeAux - crcFromRope - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - nim - Entry: 6/3391 Calls: 17/4160 = 0.41% [sum: 302; 302/4160 = 7.3%] - doOperation - forAllChildrenAux - pop - newCrcFromRopeAux - crcFromRope - writeRopeIfNotEqual - shouldRecompile - writeModule - myClose - closePasses - processModule - CompileModule - CompileProject - CommandCompileToC - MainCommand - HandleCmdLine - ... - nim - Entry: 7/3391 Calls: 14/4160 = 0.34% [sum: 316; 316/4160 = 7.6%] - Contains - isAccessible - interiorAllocatedPtr - gcMark - markStackAndRegisters - collectCTBody - collectCT - rawNewObj - newObj - newNode - copyTree - matchesAux - matches - resolveOverloads - semOverloadedCall - semOverloadedCallAnalyseEffects - ... - CommandCompileToC - MainCommand - HandleCmdLine diff --git a/doc/filelist.txt b/doc/filelist.txt index c863ba0cf..5522414fe 100644 --- a/doc/filelist.txt +++ b/doc/filelist.txt @@ -5,7 +5,7 @@ Short description of Nim's modules Module Description ============== ========================================================== nim main module: parses the command line and calls - ``main.MainCommand`` + `main.MainCommand` main implements the top-level command dispatching nimconf implements the config file reader syntaxes dispatcher for the different parsers and filters @@ -31,16 +31,14 @@ semstmts contains the semantic checking phase for statements semtypes contains the semantic checking phase for types seminst instantiation of generic procs and types semfold contains code to deal with constant folding -semthreads deep program analysis for threads -evals contains an AST interpreter for compile time evaluation +sempass2 Second semantic checking pass over the AST +vm contains an AST interpreter for compile time evaluation pragmas semantic checking of pragmas idents implements a general mapping from identifiers to an internal - representation (``PIdent``) that is used so that a simple + representation (`PIdent`) that is used so that a simple id-comparison suffices to establish whether two Nim identifiers are equivalent -ropes implements long strings represented as trees for - lazy evaluation; used mainly by the code generators transf transformations on the AST that need to be done before code generation diff --git a/doc/filters.rst b/doc/filters.md index 52704411f..9482b0b47 100644 --- a/doc/filters.rst +++ b/doc/filters.md @@ -1,32 +1,32 @@ -.. default-role:: code - =================== Source Code Filters =================== +.. include:: rstcommon.rst +.. default-role:: code .. contents:: A `Source Code Filter (SCF)` transforms the input character stream to an in-memory output stream before parsing. A filter can be used to provide templating systems or preprocessors. -To use a filter for a source file the `#?` notation is used:: +To use a filter for a source file the `#?` notation is used: - #? stdtmpl(subsChar = '$', metaChar = '#') - #proc generateXML(name, age: string): string = - # result = "" - <xml> - <name>$name</name> - <age>$age</age> - </xml> + #? stdtmpl(subsChar = '$', metaChar = '#') + #proc generateXML(name, age: string): string = + # result = "" + <xml> + <name>$name</name> + <age>$age</age> + </xml> As the example shows, passing arguments to a filter can be done just like an ordinary procedure call with named or positional arguments. The available parameters depend on the invoked filter. Before version 0.12.0 of the language `#!` was used instead of `#?`. -**Hint:** With `--hint[codeBegin]:on` or `--verbosity:2` -(or higher) while compiling or `nim check`, Nim lists the processed code after +**Hint:** With `--hint:codeBegin:on`:option: or `--verbosity:2`:option: +(or higher) while compiling or `nim check`:cmd:, Nim lists the processed code after each filter application. Usage @@ -41,23 +41,24 @@ recognize it as Nim source file). If we use `generateXML` code shown above and call the SCF file `xmlGen.nimf` In your `main.nim`: -.. code-block:: nim + ```nim include "xmlGen.nimf" echo generateXML("John Smith","42") + ``` Pipe operator ============= -Filters can be combined with the `|` pipe operator:: +Filters can be combined with the `|` pipe operator: - #? strip(startswith="<") | stdtmpl - #proc generateXML(name, age: string): string = - # result = "" - <xml> - <name>$name</name> - <age>$age</age> - </xml> + #? strip(startswith="<") | stdtmpl + #proc generateXML(name, age: string): string = + # result = "" + <xml> + <name>$name</name> + <age>$age</age> + </xml> Available filters @@ -70,11 +71,11 @@ The replace filter replaces substrings in each line. Parameters and their defaults: - `sub: string = ""` - the substring that is searched for +* `sub: string = ""` + : the substring that is searched for - `by: string = ""` - the string the substring is replaced with +* `by: string = ""` + : the string the substring is replaced with Strip filter @@ -85,15 +86,15 @@ each line. Parameters and their defaults: - `startswith: string = ""` - strip only the lines that start with *startswith* (ignoring leading +* `startswith: string = ""` + : strip only the lines that start with *startswith* (ignoring leading whitespace). If empty every line is stripped. - `leading: bool = true` - strip leading whitespace +* `leading: bool = true` + : strip leading whitespace - `trailing: bool = true` - strip trailing whitespace +* `trailing: bool = true` + : strip trailing whitespace StdTmpl filter @@ -107,50 +108,50 @@ statements need `end X` delimiters. Parameters and their defaults: - `metaChar: char = '#'` - prefix for a line that contains Nim code - - `subsChar: char = '$'` - prefix for a Nim expression within a template line - - `conc: string = " & "` - the operation for concatenation - - `emit: string = "result.add"` - the operation to emit a string literal - - `toString: string = "$"` - the operation that is applied to each expression - -Example:: - - #? stdtmpl | standard - #proc generateHTMLPage(title, currentTab, content: string, - # tabs: openArray[string]): string = - # result = "" - <head><title>$title</title></head> - <body> - <div id="menu"> - <ul> - #for tab in items(tabs): - #if currentTab == tab: - <li><a id="selected" - #else: - <li><a - #end if - href="${tab}.html">$tab</a></li> - #end for - </ul> - </div> - <div id="content"> - $content - A dollar: $$. - </div> - </body> +* `metaChar: char = '#'` + : prefix for a line that contains Nim code + +* `subsChar: char = '$'` + : prefix for a Nim expression within a template line + +* `conc: string = " & "` + : the operation for concatenation + +* `emit: string = "result.add"` + : the operation to emit a string literal + +* `toString: string = "$"` + : the operation that is applied to each expression + +Example: + + #? stdtmpl | standard + #proc generateHTMLPage(title, currentTab, content: string, + # tabs: openArray[string]): string = + # result = "" + <head><title>$title</title></head> + <body> + <div id="menu"> + <ul> + #for tab in items(tabs): + #if currentTab == tab: + <li><a id="selected" + #else: + <li><a + #end if + href="${tab}.html">$tab</a></li> + #end for + </ul> + </div> + <div id="content"> + $content + A dollar: $$. + </div> + </body> The filter transforms this into: -.. code-block:: nim + ```nim proc generateHTMLPage(title, currentTab, content: string, tabs: openArray[string]): string = result = "" @@ -173,6 +174,7 @@ The filter transforms this into: " A dollar: $.\n" & " </div>\n" & "</body>\n") + ``` Each line that does not start with the meta character (ignoring leading @@ -181,36 +183,36 @@ whitespace) is converted to a string literal that is added to `result`. The substitution character introduces a Nim expression *e* within the string literal. *e* is converted to a string with the *toString* operation which defaults to `$`. For strong type checking, set `toString` to the -empty string. *e* must match this PEG pattern:: +empty string. *e* must match this PEG pattern: - e <- [a-zA-Z\128-\255][a-zA-Z0-9\128-\255_.]* / '{' x '}' - x <- '{' x+ '}' / [^}]* + e <- [a-zA-Z\128-\255][a-zA-Z0-9\128-\255_.]* / '{' x '}' + x <- '{' x+ '}' / [^}]* To produce a single substitution character it has to be doubled: `$$` produces `$`. The template engine is quite flexible. It is easy to produce a procedure that -writes the template code directly to a file:: - - #? stdtmpl(emit="f.write") | standard - #proc writeHTMLPage(f: File, title, currentTab, content: string, - # tabs: openArray[string]) = - <head><title>$title</title></head> - <body> - <div id="menu"> - <ul> - #for tab in items(tabs): - #if currentTab == tab: - <li><a id="selected" - #else: - <li><a - #end if - href="${tab}.html" title = "$title - $tab">$tab</a></li> - #end for - </ul> - </div> - <div id="content"> - $content - A dollar: $$. - </div> - </body> +writes the template code directly to a file: + + #? stdtmpl(emit="f.write") | standard + #proc writeHTMLPage(f: File, title, currentTab, content: string, + # tabs: openArray[string]) = + <head><title>$title</title></head> + <body> + <div id="menu"> + <ul> + #for tab in items(tabs): + #if currentTab == tab: + <li><a id="selected" + #else: + <li><a + #end if + href="${tab}.html" title = "$title - $tab">$tab</a></li> + #end for + </ul> + </div> + <div id="content"> + $content + A dollar: $$. + </div> + </body> diff --git a/doc/grammar.txt b/doc/grammar.txt index 0d5eef179..51b3e0053 100644 --- a/doc/grammar.txt +++ b/doc/grammar.txt @@ -1,5 +1,5 @@ # This file is generated by compiler/parser.nim. -module = stmt ^* (';' / IND{=}) +module = complexOrSimpleStmt ^* (';' / IND{=}) comma = ',' COMMENT? semicolon = ';' COMMENT? colon = ':' COMMENT? @@ -7,7 +7,7 @@ colcom = ':' COMMENT? operator = OP0 | OP1 | OP2 | OP3 | OP4 | OP5 | OP6 | OP7 | OP8 | OP9 | 'or' | 'xor' | 'and' | 'is' | 'isnot' | 'in' | 'notin' | 'of' | 'as' | 'from' - | 'div' | 'mod' | 'shl' | 'shr' | 'not' | 'static' | '..' + | 'div' | 'mod' | 'shl' | 'shr' | 'not' | '..' prefixOperator = operator optInd = COMMENT? IND? optPar = (IND{>} | IND{=})? @@ -22,64 +22,70 @@ ampExpr = plusExpr (OP7 optInd plusExpr)* plusExpr = mulExpr (OP8 optInd mulExpr)* mulExpr = dollarExpr (OP9 optInd dollarExpr)* dollarExpr = primary (OP10 optInd primary)* +operatorB = OP0 | OP1 | OP2 | OP3 | OP4 | OP5 | OP6 | OP7 | OP8 | OP9 | + 'div' | 'mod' | 'shl' | 'shr' | 'in' | 'notin' | + 'is' | 'isnot' | 'not' | 'of' | 'as' | 'from' | '..' | 'and' | 'or' | 'xor' symbol = '`' (KEYW|IDENT|literal|(operator|'('|')'|'['|']'|'{'|'}'|'=')+)+ '`' - | IDENT | KEYW -exprColonEqExpr = expr (':'|'=' expr)? + | IDENT | 'addr' | 'type' | 'static' +symbolOrKeyword = symbol | KEYW +exprColonEqExpr = expr ((':'|'=') expr + / doBlock extraPostExprBlock*)? +exprEqExpr = expr ('=' expr + / doBlock extraPostExprBlock*)? exprList = expr ^+ comma +optionalExprList = expr ^* comma exprColonEqExprList = exprColonEqExpr (comma exprColonEqExpr)* (comma)? -dotExpr = expr '.' optInd (symbol | '[:' exprList ']') -explicitGenericInstantiation = '[:' exprList ']' ( '(' exprColonEqExpr ')' )? -qualifiedIdent = symbol ('.' optInd symbol)? +qualifiedIdent = symbol ('.' optInd symbolOrKeyword)? setOrTableConstr = '{' ((exprColonEqExpr comma)* | ':' ) '}' castExpr = 'cast' ('[' optInd typeDesc optPar ']' '(' optInd expr optPar ')') / parKeyw = 'discard' | 'include' | 'if' | 'while' | 'case' | 'try' | 'finally' | 'except' | 'for' | 'block' | 'const' | 'let' | 'when' | 'var' | 'mixin' par = '(' optInd - ( &parKeyw (ifExpr \ complexOrSimpleStmt) ^+ ';' - | ';' (ifExpr \ complexOrSimpleStmt) ^+ ';' + ( &parKeyw (ifExpr / complexOrSimpleStmt) ^+ ';' + | ';' (ifExpr / complexOrSimpleStmt) ^+ ';' | pragmaStmt - | simpleExpr ( ('=' expr (';' (ifExpr \ complexOrSimpleStmt) ^+ ';' )? ) + | simpleExpr ( (doBlock extraPostExprBlock*) + | ('=' expr (';' (ifExpr / complexOrSimpleStmt) ^+ ';' )? ) | (':' expr (',' exprColonEqExpr ^+ ',' )? ) ) ) optPar ')' literal = | INT_LIT | INT8_LIT | INT16_LIT | INT32_LIT | INT64_LIT | UINT_LIT | UINT8_LIT | UINT16_LIT | UINT32_LIT | UINT64_LIT | FLOAT_LIT | FLOAT32_LIT | FLOAT64_LIT | STR_LIT | RSTR_LIT | TRIPLESTR_LIT - | CHAR_LIT + | CHAR_LIT | CUSTOM_NUMERIC_LIT | NIL generalizedLit = GENERALIZED_STR_LIT | GENERALIZED_TRIPLESTR_LIT identOrLiteral = generalizedLit | symbol | literal - | par | arrayConstr | setOrTableConstr + | par | arrayConstr | setOrTableConstr | tupleConstr | castExpr tupleConstr = '(' optInd (exprColonEqExpr comma?)* optPar ')' arrayConstr = '[' optInd (exprColonEqExpr comma?)* optPar ']' primarySuffix = '(' (exprColonEqExpr comma?)* ')' - | '.' optInd symbol generalizedLit? + | '.' optInd symbolOrKeyword ('[:' exprList ']' ( '(' exprColonEqExpr ')' )?)? generalizedLit? + | DOTLIKEOP optInd symbolOrKeyword generalizedLit? | '[' optInd exprColonEqExprList optPar ']' | '{' optInd exprColonEqExprList optPar '}' - | &( '`'|IDENT|literal|'cast'|'addr'|'type') expr # command syntax pragma = '{.' optInd (exprColonEqExpr comma?)* optPar ('.}' | '}') identVis = symbol OPR? # postfix position -identVisDot = symbol '.' optInd symbol OPR? +identVisDot = symbol '.' optInd symbolOrKeyword OPR? identWithPragma = identVis pragma? identWithPragmaDot = identVisDot pragma? declColonEquals = identWithPragma (comma identWithPragma)* comma? - (':' optInd typeDesc)? ('=' optInd expr)? + (':' optInd typeDescExpr)? ('=' optInd expr)? identColonEquals = IDENT (comma IDENT)* comma? - (':' optInd typeDesc)? ('=' optInd expr)?) -inlTupleDecl = 'tuple' - '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']' -extTupleDecl = 'tuple' - COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)? -tupleClass = 'tuple' + (':' optInd typeDescExpr)? ('=' optInd expr)?) +tupleTypeBracket = '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']' +tupleType = 'tuple' tupleTypeBracket +tupleDecl = 'tuple' (tupleTypeBracket / + COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?) paramList = '(' declColonEquals ^* (comma/semicolon) ')' paramListArrow = paramList? ('->' optInd typeDesc)? paramListColon = paramList? (':' optInd typeDesc)? doBlock = 'do' paramListArrow pragma? colcom stmt -procExpr = 'proc' paramListColon pragma? ('=' COMMENT? stmt)? -distinct = 'distinct' optInd typeDesc -forStmt = 'for' (identWithPragma ^+ comma) 'in' expr colcom stmt +routineExpr = ('proc' | 'func' | 'iterator') paramListColon pragma? ('=' COMMENT? stmt)? +routineType = ('proc' | 'iterator') paramListColon pragma? +forStmt = 'for' ((varTuple / identWithPragma) ^+ comma) 'in' expr colcom stmt forExpr = forStmt expr = (blockExpr | ifExpr @@ -88,24 +94,33 @@ expr = (blockExpr | forExpr | tryExpr) / simpleExpr -typeKeyw = 'var' | 'out' | 'ref' | 'ptr' | 'shared' | 'tuple' - | 'proc' | 'iterator' | 'distinct' | 'object' | 'enum' -primary = typeKeyw optInd typeDesc - / prefixOperator* identOrLiteral primarySuffix* - / 'bind' primary -typeDesc = simpleExpr ('not' expr)? -typeDefAux = simpleExpr ('not' expr)? - | 'concept' typeClass -postExprBlocks = ':' stmt? ( IND{=} doBlock - | IND{=} 'of' exprList ':' stmt - | IND{=} 'elif' expr ':' stmt - | IND{=} 'except' exprList ':' stmt - | IND{=} 'else' ':' stmt )* -exprStmt = simpleExpr - (( '=' optInd expr colonBody? ) - / ( expr ^+ comma - postExprBlocks - ))? +simplePrimary = SIGILLIKEOP? identOrLiteral primarySuffix* +commandStart = &('`'|IDENT|literal|'cast'|'addr'|'type'|'var'|'out'| + 'static'|'enum'|'tuple'|'object'|'proc') +primary = simplePrimary (commandStart expr (doBlock extraPostExprBlock*)?)? + / operatorB primary + / routineExpr + / rawTypeDesc + / prefixOperator primary +rawTypeDesc = (tupleType | routineType | 'enum' | 'object' | + ('var' | 'out' | 'ref' | 'ptr' | 'distinct') typeDesc?) + ('not' primary)? +typeDescExpr = (routineType / simpleExpr) ('not' primary)? +typeDesc = rawTypeDesc / typeDescExpr +typeDefValue = ((tupleDecl | enumDecl | objectDecl | conceptDecl | + ('ref' | 'ptr' | 'distinct') (tupleDecl | objectDecl)) + / (simpleExpr (exprEqExpr ^+ comma postExprBlocks?)?)) + ('not' primary)? +extraPostExprBlock = ( IND{=} doBlock + | IND{=} 'of' exprList ':' stmt + | IND{=} 'elif' expr ':' stmt + | IND{=} 'except' optionalExprList ':' stmt + | IND{=} 'finally' ':' stmt + | IND{=} 'else' ':' stmt ) +postExprBlocks = (doBlock / ':' (extraPostExprBlock / stmt)) extraPostExprBlock* +exprStmt = simpleExpr postExprBlocks? + / simplePrimary (exprEqExpr ^+ comma) postExprBlocks? + / simpleExpr '=' optInd (expr postExprBlocks?) importStmt = 'import' optInd expr ((comma expr)* / 'except' optInd (expr ^+ comma)) @@ -125,9 +140,9 @@ condStmt = expr colcom stmt COMMENT? (IND{=} 'else' colcom stmt)? ifStmt = 'if' condStmt whenStmt = 'when' condStmt -condExpr = expr colcom expr optInd - ('elif' expr colcom expr optInd)* - 'else' colcom expr +condExpr = expr colcom stmt optInd + ('elif' expr colcom stmt optInd)* + 'else' colcom stmt ifExpr = 'if' condExpr whenExpr = 'when' condExpr whileStmt = 'while' expr colcom stmt @@ -139,12 +154,11 @@ caseStmt = 'case' expr ':'? COMMENT? (IND{>} ofBranches DED | IND{=} ofBranches) tryStmt = 'try' colcom stmt &(IND{=}? 'except'|'finally') - (IND{=}? 'except' exprList colcom stmt)* + (IND{=}? 'except' optionalExprList colcom stmt)* (IND{=}? 'finally' colcom stmt)? tryExpr = 'try' colcom stmt &(optInd 'except'|'finally') - (optInd 'except' exprList colcom stmt)* + (optInd 'except' optionalExprList colcom stmt)* (optInd 'finally' colcom stmt)? -exceptBlock = 'except' colcom stmt blockStmt = 'block' symbol? colcom stmt blockExpr = 'block' symbol? colcom stmt staticStmt = 'static' colcom stmt @@ -159,7 +173,7 @@ routine = optInd identVis pattern? genericParamList? paramListColon pragma? ('=' COMMENT? stmt)? indAndComment commentStmt = COMMENT section(RULE) = COMMENT? RULE / (IND{>} (RULE / COMMENT)^+IND{=} DED) -enum = 'enum' optInd (symbol pragma? optInd ('=' optInd expr COMMENT?)? comma?)+ +enumDecl = 'enum' optInd (symbol pragma? optInd ('=' optInd expr COMMENT?)? comma?)+ objectWhen = 'when' expr colcom objectPart COMMENT? ('elif' expr colcom objectPart COMMENT?)* ('else' colcom objectPart COMMENT?)? @@ -167,19 +181,19 @@ objectBranch = 'of' exprList colcom objectPart objectBranches = objectBranch (IND{=} objectBranch)* (IND{=} 'elif' expr colcom objectPart)* (IND{=} 'else' colcom objectPart)? -objectCase = 'case' identWithPragma ':' typeDesc ':'? COMMENT? +objectCase = 'case' declColonEquals ':'? COMMENT? (IND{>} objectBranches DED | IND{=} objectBranches) objectPart = IND{>} objectPart^+IND{=} DED / objectWhen / objectCase / 'nil' / 'discard' / declColonEquals -object = 'object' pragma? ('of' typeDesc)? COMMENT? objectPart -typeClassParam = ('var' | 'out')? symbol -typeClass = typeClassParam ^* ',' (pragma)? ('of' typeDesc ^* ',')? +objectDecl = 'object' ('of' typeDesc)? COMMENT? objectPart +conceptParam = ('var' | 'out' | 'ptr' | 'ref' | 'static' | 'type')? symbol +conceptDecl = 'concept' conceptParam ^* ',' (pragma)? ('of' typeDesc ^* ',')? &IND{>} stmt -typeDef = identWithPragmaDot genericParamList? '=' optInd typeDefAux - indAndComment? / identVisDot genericParamList? pragma '=' optInd typeDefAux +typeDef = identVisDot genericParamList? pragma '=' optInd typeDefValue indAndComment? -varTuple = '(' optInd identWithPragma ^+ comma optPar ')' '=' optInd expr +varTupleLhs = '(' optInd (identWithPragma / varTupleLhs) ^+ comma optPar ')' (':' optInd typeDescExpr)? +varTuple = varTupleLhs '=' optInd expr colonBody = colcom stmt postExprBlocks? variable = (varTuple / identColonEquals) colonBody? indAndComment constant = (varTuple / identWithPragma) (colon typeDesc)? '=' optInd expr indAndComment diff --git a/doc/hcr.rst b/doc/hcr.md index 7e9d71da3..285a86282 100644 --- a/doc/hcr.rst +++ b/doc/hcr.md @@ -1,9 +1,10 @@ -.. default-role:: code - =================================== Hot code reloading =================================== +.. default-role:: code +.. include:: rstcommon.rst + The `hotCodeReloading`:idx: option enables special compilation mode where changes in the code can be applied automatically to a running program. The code reloading happens at the granularity of an individual module. @@ -22,11 +23,10 @@ during development resides. In this example, we use SDL2 to create a window and we reload the logic code when `F9` is pressed. The important lines are marked with `#***`. -To install SDL2 you can use `nimble install sdl2`. +To install SDL2 you can use `nimble install sdl2`:cmd:. -.. code-block:: nim - + ```nim # logic.nim import sdl2 @@ -78,14 +78,14 @@ To install SDL2 you can use `nimble install sdl2`. discard renderer.clear() discard renderer.setDrawColor(255, 128, 128, 0) - var rect = Rect(x: posX - 25, y: posY - 25, w: 50.cint, h: 50.cint) + var rect: Rect = (x: posX - 25, y: posY - 25, w: 50.cint, h: 50.cint) discard renderer.fillRect(rect) delay(16) renderer.present() + ``` -.. code-block:: nim - + ```nim # mymain.nim import logic @@ -96,36 +96,44 @@ To install SDL2 you can use `nimble install sdl2`. destroy() main() + ``` -Compile this example via:: +Compile this example via: + ```cmd nim c --hotcodereloading:on mymain.nim + ``` Now start the program and KEEP it running! -:: - + ```cmd # Unix: mymain & # or Windows (click on the .exe) mymain.exe # edit + ``` -For example, change the line:: +For example, change the line: + ```nim discard renderer.setDrawColor(255, 128, 128, 0) + ``` -into:: +into: + ```nim discard renderer.setDrawColor(255, 255, 128, 0) + ``` (This will change the color of the rectangle.) Then recompile the project, but do not restart or quit the mymain.exe program! -:: + ```cmd nim c --hotcodereloading:on mymain.nim + ``` Now give the `mymain` SDL window the focus, press F9, and watch the updated version of the program. @@ -139,7 +147,7 @@ One can use the special event handlers `beforeCodeReload` and `afterCodeReload` to reset the state of a particular variable or to force the execution of certain statements: -.. code-block:: Nim + ```Nim var settings = initTable[string, string]() lastReload: Time @@ -152,6 +160,7 @@ the execution of certain statements: afterCodeReload: lastReload = now() resetProgramState() + ``` On each code reload, Nim will first execute all `beforeCodeReload`:idx: handlers registered in the previous version of the program and then all @@ -160,7 +169,7 @@ that any handlers appearing in modules that weren't reloaded will also be executed. To prevent this behavior, one can guard the code with the `hasModuleChanged()`:idx: API: -.. code-block:: Nim + ```Nim import mydb var myCache = initTable[Key, Value]() @@ -168,6 +177,7 @@ executed. To prevent this behavior, one can guard the code with the afterCodeReload: if hasModuleChanged(mydb): resetCache(myCache) + ``` The hot code reloading is based on dynamic library hot swapping in the native targets and direct manipulation of the global namespace in the JavaScript @@ -188,16 +198,15 @@ Native code targets =================== Native projects using the hot code reloading option will be implicitly -compiled with the `-d:useNimRtl` option and they will depend on both +compiled with the `-d:useNimRtl`:option: option and they will depend on both the `nimrtl` library and the `nimhcr` library which implements the hot code reloading run-time. Both libraries can be found in the `lib` folder of Nim and can be compiled into dynamic libraries to satisfy -runtime demands of the example code above. An example of compiling -`nimhcr.nim` and `nimrtl.nim` when the source dir of Nim is installed +runtime demands of the example code above. An example of compiling +``nimhcr.nim`` and ``nimrtl.nim`` when the source dir of Nim is installed with choosenim follows. -:: - + ```console # Unix/MacOS # Make sure you are in the directory containing your .nim files $ cd your-source-directory @@ -208,6 +217,7 @@ with choosenim follows. # verify that you have two files named libnimhcr and libnimrtl in your # source directory (.dll for Windows, .so for Unix, .dylib for MacOS) + ``` All modules of the project will be compiled to separate dynamic link libraries placed in the `nimcache` directory. Please note that during diff --git a/doc/idetools.rst b/doc/idetools.md index dcafaf45f..0388a76c0 100644 --- a/doc/idetools.rst +++ b/doc/idetools.md @@ -10,24 +10,21 @@ .. contents:: -.. raw:: html - <blockquote><p> - "yes, I'm the creator" -- Araq, 2013-07-26 19:28:32. - </p></blockquote> +> "yes, I'm the creator" -- Araq, 2013-07-26 19:28:32. -Note: this is mostly outdated, see instead `nimsuggest <nimsuggest.html>`_ +Note: this is mostly outdated, see instead [nimsuggest](nimsuggest.html) Nim differs from many other compilers in that it is really fast, and being so fast makes it suited to provide external queries for text editors about the source code being written. Through the -`idetools` command of `the compiler <nimc.html>`_, any IDE +`idetools` command of [the compiler](nimc.html), any IDE can query a `.nim` source file and obtain useful information like definition of symbols or suggestions for completion. This document will guide you through the available options. If you want to look at practical examples of idetools support you can look -at the test files found in the `Test suite`_ or `various editor -integrations <https://github.com/Araq/Nim/wiki/Editor-Support>`_ +at the test files found in the [Test suite] or [various editor +integrations](https://github.com/Araq/Nim/wiki/Editor-Support) already available. @@ -39,39 +36,43 @@ Specifying the location of the query All of the available idetools commands require you to specify a query location through the `--track` or `--trackDirty` switches. -The general idetools invocations are:: +The general idetools invocations are: - nim idetools --track:FILE,LINE,COL <switches> proj.nim + ```cmd + nim idetools --track:FILE,LINE,COL <switches> proj.nim + ``` -Or:: +Or: - nim idetools --trackDirty:DIRTY_FILE,FILE,LINE,COL <switches> proj.nim + ```cmd + nim idetools --trackDirty:DIRTY_FILE,FILE,LINE,COL <switches> proj.nim + ``` `proj.nim` - This is the main *project* filename. Most of the time you will +: This is the main *project* filename. Most of the time you will pass in the same as **FILE**, but for bigger projects this is the file which is used as main entry point for the program, the one which users compile to generate a final binary. `<switches>` - This would be any of the other idetools available options, like +: This would be any of the other idetools available options, like `--def` or `--suggest` explained in the following sections. `COL` - An integer with the column you are going to query. For the +: An integer with the column you are going to query. For the compiler columns start at zero, so the first column will be **0** and the last in an 80 column terminal will be **79**. `LINE` - An integer with the line you are going to query. For the compiler +: An integer with the line you are going to query. For the compiler lines start at **1**. `FILE` - The file you want to perform the query on. Usually you will +: The file you want to perform the query on. Usually you will pass in the same value as **proj.nim**. `DIRTY_FILE` - The **FILE** parameter is enough for static analysis, but IDEs +: The **FILE** parameter is enough for static analysis, but IDEs tend to have *unsaved buffers* where the user may still be in the middle of typing a line. In such situations the IDE can save the current contents to a temporary file and then use the @@ -127,8 +128,8 @@ to the case insensitiveness of the language (plus underscores as separators!). The typical usage scenario for this option is to call it after the -user has typed the dot character for `the object oriented call -syntax <tut2.html#object-oriented-programming-method-call-syntax>`_. +user has typed the dot character for [the object oriented call +syntax](tut2.html#object-oriented-programming-method-call-syntax). Idetools will try to return the suggestions sorted first by scope (from innermost to outermost) and then by item name. @@ -181,14 +182,18 @@ results of the compilation, and subsequent queries should be fast in the millisecond range, thus being responsive enough for IDEs. If you want to start the server using stdin/stdout as communication -you need to type:: +you need to type: - nim serve --server.type:stdin proj.nim + ```cmd + nim serve --server.type:stdin proj.nim + ``` -If you want to start the server using tcp and a port, you need to type:: +If you want to start the server using tcp and a port, you need to type: - nim serve --server.type:tcp --server.port:6000 \ + ```cmd + nim serve --server.type:tcp --server.port:6000 \ --server.address:hostname proj.nim + ``` In both cases the server will start up and await further commands. The syntax of the commands you can now send to the server is @@ -199,7 +204,7 @@ of text it thinks necessary plus an empty line to indicate the end of the answer. You can find examples of client/server communication in the idetools -tests found in the `Test suite`_. +tests found in the [Test suite]. Parsing idetools output @@ -242,45 +247,48 @@ symbol for which idetools returns valid output. skConst ------- -| **Third column**: module + [n scope nesting] + const name. +| **Third column**: module + \[n scope nesting] + const name. | **Fourth column**: the type of the const value. | **Docstring**: always the empty string. -.. code-block:: nim - const SOME_SEQUENCE = @[1, 2] - --> col 2: $MODULE.SOME_SEQUENCE - col 3: seq[int] - col 7: "" + ```nim + const SOME_SEQUENCE = @[1, 2] + --> col 2: $MODULE.SOME_SEQUENCE + col 3: seq[int] + col 7: "" + ``` skEnumField ----------- -| **Third column**: module + [n scope nesting] + enum type + enum field name. +| **Third column**: module + \[n scope nesting] + enum type + enum field name. | **Fourth column**: enum type grouping other enum fields. | **Docstring**: always the empty string. -.. code-block:: nim - Open(filename, fmWrite) - --> col 2: system.FileMode.fmWrite - col 3: FileMode - col 7: "" + ```nim + Open(filename, fmWrite) + --> col 2: system.FileMode.fmWrite + col 3: FileMode + col 7: "" + ``` skForVar -------- -| **Third column**: module + [n scope nesting] + var name. +| **Third column**: module + \[n scope nesting] + var name. | **Fourth column**: type of the var. | **Docstring**: always the empty string. -.. code-block:: nim - proc looper(filename = "tests.nim") = - for letter in filename: - echo letter - --> col 2: $MODULE.looper.letter - col 3: char - col 7: "" + ```nim + proc looper(filename = "tests.nim") = + for letter in filename: + echo letter + --> col 2: $MODULE.looper.letter + col 3: char + col 7: "" + ``` skIterator, skClosureIterator @@ -291,48 +299,51 @@ defined, since at that point in the file the parser hasn't processed the full line yet. The signature will be returned complete in posterior instances of the iterator. -| **Third column**: module + [n scope nesting] + iterator name. +| **Third column**: module + \[n scope nesting] + iterator name. | **Fourth column**: signature of the iterator including return type. | **Docstring**: docstring if available. -.. code-block:: nim - let - text = "some text" - letters = toSeq(runes(text)) - --> col 2: unicode.runes - col 3: iterator (string): Rune - col 7: "iterates over any unicode character of the string `s`." + ```nim + let + text = "some text" + letters = toSeq(runes(text)) + --> col 2: unicode.runes + col 3: iterator (string): Rune + col 7: "iterates over any unicode character of the string `s`." + ``` skLabel ------- -| **Third column**: module + [n scope nesting] + name. +| **Third column**: module + \[n scope nesting] + name. | **Fourth column**: always the empty string. | **Docstring**: always the empty string. -.. code-block:: nim - proc test(text: string) = - var found = -1 - block loops: - --> col 2: $MODULE.test.loops - col 3: "" - col 7: "" + ```nim + proc test(text: string) = + var found = -1 + block loops: + --> col 2: $MODULE.test.loops + col 3: "" + col 7: "" + ``` skLet ----- -| **Third column**: module + [n scope nesting] + let name. +| **Third column**: module + \[n scope nesting] + let name. | **Fourth column**: the type of the let variable. | **Docstring**: always the empty string. -.. code-block:: nim - let - text = "some text" - --> col 2: $MODULE.text - col 3: string - col 7: "" + ```nim + let + text = "some text" + --> col 2: $MODULE.text + col 3: string + col 7: "" + ``` skMacro @@ -343,16 +354,17 @@ defined, since at that point in the file the parser hasn't processed the full line yet. The signature will be returned complete in posterior instances of the macro. -| **Third column**: module + [n scope nesting] + macro name. +| **Third column**: module + \[n scope nesting] + macro name. | **Fourth column**: signature of the macro including return type. | **Docstring**: docstring if available. -.. code-block:: nim - proc testMacro() = - expect(EArithmetic): - --> col 2: idetools_api.expect - col 3: proc (varargs[expr], stmt): stmt - col 7: "" + ```nim + proc testMacro() = + expect(EArithmetic): + --> col 2: idetools_api.expect + col 3: proc (varargs[expr], stmt): stmt + col 7: "" + ``` skMethod @@ -363,8 +375,8 @@ defined, since at that point in the file the parser hasn't processed the full line yet. The signature will be returned complete in posterior instances of the method. -Methods imply `dynamic dispatch -<tut2.html#object-oriented-programming-dynamic-dispatch>`_ and +Methods imply [dynamic dispatch]( +tut2.html#object-oriented-programming-dynamic-dispatch) and idetools performs a static analysis on the code. For this reason idetools may not return the definition of the correct method you are querying because it may be impossible to know until the code @@ -380,33 +392,35 @@ Note that at the moment the word `proc` is returned for the signature of the found method instead of the expected `method`. This may change in the future. -| **Third column**: module + [n scope nesting] + method name. +| **Third column**: module + \[n scope nesting] + method name. | **Fourth column**: signature of the method including return type. | **Docstring**: docstring if available. -.. code-block:: nim - method eval(e: PExpr): int = quit "to override!" - method eval(e: PLiteral): int = e.x - method eval(e: PPlusExpr): int = eval(e.a) + eval(e.b) - echo eval(newPlus(newPlus(newLit(1), newLit(2)), newLit(4))) - --> col 2: $MODULE.eval - col 3: proc (PPlusExpr): int - col 7: "" + ```nim + method eval(e: PExpr): int = quit "to override!" + method eval(e: PLiteral): int = e.x + method eval(e: PPlusExpr): int = eval(e.a) + eval(e.b) + echo eval(newPlus(newPlus(newLit(1), newLit(2)), newLit(4))) + --> col 2: $MODULE.eval + col 3: proc (PPlusExpr): int + col 7: "" + ``` skParam ------- -| **Third column**: module + [n scope nesting] + param name. +| **Third column**: module + \[n scope nesting] + param name. | **Fourth column**: the type of the parameter. | **Docstring**: always the empty string. -.. code-block:: nim - proc reader(filename = "tests.nim") = - let text = readFile(filename) - --> col 2: $MODULE.reader.filename - col 3: string - col 7: "" + ```nim + proc reader(filename = "tests.nim") = + let text = readFile(filename) + --> col 2: $MODULE.reader.filename + col 3: string + col 7: "" + ``` skProc @@ -421,34 +435,36 @@ While at the language level a proc is differentiated from others by the parameters and return value, the signature of the proc returned by idetools returns also the pragmas for the proc. -| **Third column**: module + [n scope nesting] + proc name. +| **Third column**: module + \[n scope nesting] + proc name. | **Fourth column**: signature of the proc including return type. | **Docstring**: docstring if available. -.. code-block:: nim - open(filename, fmWrite) - --> col 2: system.Open - col 3: proc (var File, string, FileMode, int): bool - col 7: - "Opens a file named `filename` with given `mode`. + ```nim + open(filename, fmWrite) + --> col 2: system.Open + col 3: proc (var File, string, FileMode, int): bool + col 7: + "Opens a file named `filename` with given `mode`. - Default mode is readonly. Returns true iff the file could be opened. - This throws no exception if the file could not be opened." + Default mode is readonly. Returns true iff the file could be opened. + This throws no exception if the file could not be opened." + ``` skResult -------- -| **Third column**: module + [n scope nesting] + result. +| **Third column**: module + \[n scope nesting] + result. | **Fourth column**: the type of the result. | **Docstring**: always the empty string. -.. code-block:: nim - proc getRandomValue() : int = - return 4 - --> col 2: $MODULE.getRandomValue.result - col 3: int - col 7: "" + ```nim + proc getRandomValue() : int = + return 4 + --> col 2: $MODULE.getRandomValue.result + col 3: int + col 7: "" + ``` skTemplate @@ -459,11 +475,11 @@ defined, since at that point in the file the parser hasn't processed the full line yet. The signature will be returned complete in posterior instances of the template. -| **Third column**: module + [n scope nesting] + template name. +| **Third column**: module + \[n scope nesting] + template name. | **Fourth column**: signature of the template including return type. | **Docstring**: docstring if available. -.. code-block:: nim + `````nim let text = "some text" letters = toSeq(runes(text)) @@ -474,45 +490,49 @@ posterior instances of the template. Example: - .. code-block:: nim + ```nim let numeric = @[1, 2, 3, 4, 5, 6, 7, 8, 9] odd_numbers = toSeq(filter(numeric) do (x: int) -> bool: if x mod 2 == 1: result = true) assert odd_numbers == @[1, 3, 5, 7, 9]" + ``` + ````` skType ------ -| **Third column**: module + [n scope nesting] + type name. +| **Third column**: module + \[n scope nesting] + type name. | **Fourth column**: the type. | **Docstring**: always the empty string. -.. code-block:: nim - proc writeTempFile() = - var output: File - --> col 2: system.File - col 3: File - col 7: "" + ```nim + proc writeTempFile() = + var output: File + --> col 2: system.File + col 3: File + col 7: "" + ``` skVar ----- -| **Third column**: module + [n scope nesting] + var name. +| **Third column**: module + \[n scope nesting] + var name. | **Fourth column**: the type of the var. | **Docstring**: always the empty string. -.. code-block:: nim - proc writeTempFile() = - var output: File - output.open("/tmp/somefile", fmWrite) - output.write("test") - --> col 2: $MODULE.writeTempFile.output - col 3: File - col 7: "" + ```nim + proc writeTempFile() = + var output: File + output.open("/tmp/somefile", fmWrite) + output.write("test") + --> col 2: $MODULE.writeTempFile.output + col 3: File + col 7: "" + ``` Test suite @@ -530,10 +550,12 @@ Running the test suite At the moment idetools support is still in development so the test suite is not integrated with the main test suite and you have to -run it manually. First you have to compile the tester:: +run it manually. First you have to compile the tester: - $ cd my/nim/checkout/tests - $ nim c testament/caasdriver.nim + ```cmd + $ cd my/nim/checkout/tests + $ nim c testament/caasdriver.nim + ``` Running the `caasdriver` without parameters will attempt to process all the test cases in all three operation modes. If a test succeeds @@ -555,9 +577,11 @@ If you don't want to run all the test case files you can pass any substring as a parameter to `caasdriver`. Only files matching the passed substring will be run. The filtering doesn't use any globbing metacharacters, it's a plain match. For example, to run only -`*-compile*.txt` tests in verbose mode:: +`*-compile*.txt` tests in verbose mode: - ./caasdriver verbose -compile + ```cmd + ./caasdriver verbose -compile + ``` Test case file format @@ -578,7 +602,7 @@ All the `tests/caas/*.txt` files encode a session with the compiler: modes, you can prefix a line with the mode and the line will be processed only in that mode. -* The rest of the line is treated as a `regular expression <re.html>`_, +* The rest of the line is treated as a [regular expression](re.html), so be careful escaping metacharacters like parenthesis. Before the line is processed as a regular expression, some basic diff --git a/doc/intern.md b/doc/intern.md new file mode 100644 index 000000000..6b16bc71f --- /dev/null +++ b/doc/intern.md @@ -0,0 +1,679 @@ +========================================= + Internals of the Nim Compiler +========================================= + + +:Author: Andreas Rumpf +:Version: |nimversion| + +.. default-role:: code +.. include:: rstcommon.rst +.. contents:: + +> "Abstraction is layering ignorance on top of reality." -- Richard Gabriel + + +Directory structure +=================== + +The Nim project's directory structure is: + +============ =================================================== +Path Purpose +============ =================================================== +`bin` generated binary files +`build` generated C code for the installation +`compiler` the Nim compiler itself; note that this + code has been translated from a bootstrapping + version written in Pascal, so the code is **not** + a poster child of good Nim code +`config` configuration files for Nim +`dist` additional packages for the distribution +`doc` the documentation; it is a bunch of + reStructuredText files +`lib` the Nim library +============ =================================================== + + +Bootstrapping the compiler +========================== + +**Note**: Add ``.`` to your PATH so that `koch`:cmd: can be used without the ``./``. + +Compiling the compiler is a simple matter of running: + + ```cmd + nim c koch.nim + koch boot -d:release + ``` + +For a debug version use: + + ```cmd + nim c koch.nim + koch boot + ``` + + +And for a debug version compatible with GDB: + + ```cmd + nim c koch.nim + koch boot --debuginfo --linedir:on + ``` + +The `koch`:cmd: program is Nim's maintenance script. It is a replacement for +make and shell scripting with the advantage that it is much more portable. +More information about its options can be found in the [koch](koch.html) +documentation. + + +Reproducible builds +------------------- + +Set the compilation timestamp with the `SOURCE_DATE_EPOCH` environment variable. + + ```cmd + export SOURCE_DATE_EPOCH=$(git log -n 1 --format=%at) + koch boot # or `./build_all.sh` + ``` + + +Debugging the compiler +====================== + + +Bisecting for regressions +------------------------- + +There are often times when there is a bug that is caused by a regression in the +compiler or stdlib. Bisecting the Nim repo commits is a useful tool to identify +what commit introduced the regression. + +Even if it's not known whether a bug is caused by a regression, bisection can reduce +debugging time by ruling it out. If the bug is found to be a regression, then you +focus on the changes introduced by that one specific commit. + +`koch temp`:cmd: returns 125 as the exit code in case the compiler +compilation fails. This exit code tells `git bisect`:cmd: to skip the +current commit: + + ```cmd + git bisect start bad-commit good-commit + git bisect run ./koch temp -r c test-source.nim + ``` + +You can also bisect using custom options to build the compiler, for example if +you don't need a debug version of the compiler (which runs slower), you can replace +`./koch temp`:cmd: by explicit compilation command, see [Bootstrapping the compiler]. + +See also: + +- Crossplatform C/Cpp/Valgrind/JS Bisect in GitHub: https://github.com/juancarlospaco/nimrun-action#examples + + +Building an instrumented compiler +--------------------------------- + +Considering that a useful method of debugging the compiler is inserting debug +logging, or changing code and then observing the outcome of a testcase, it is +fastest to build a compiler that is instrumented for debugging from an +existing release build. `koch temp`:cmd: provides a convenient method of doing +just that. + +By default, running `koch temp`:cmd: will build a lean version of the compiler +with `-d:debug`:option: enabled. The compiler is written to `bin/nim_temp` by +default. A lean version of the compiler lacks JS and documentation generation. + +`bin/nim_temp` can be directly used to run testcases, or used with testament +with `testament --nim:bin/nim_temp r tests/category/tsometest`:cmd:. + +`koch temp`:cmd: will build the temporary compiler with the `-d:debug`:option: +enabled. Here are compiler options that are of interest when debugging: + +* `-d:debug`:option:\: enables `assert` statements and stacktraces and all + runtime checks +* `--opt:speed`:option:\: build with optimizations enabled +* `--debugger:native`:option:\: enables `--debuginfo --lineDir:on`:option: for using + a native debugger like GDB, LLDB or CDB +* `-d:nimDebug`:option: cause calls to `quit` to raise an assertion exception +* `-d:nimDebugUtils`:option:\: enables various debugging utilities; + see `compiler/debugutils` +* `-d:stacktraceMsgs -d:nimCompilerStacktraceHints`:option:\: adds some additional + stacktrace hints; see https://github.com/nim-lang/Nim/pull/13351 +* `-u:leanCompiler`:option:\: enable JS and doc generation + +Another method to build and run the compiler is directly through `koch`:cmd:\: + + ```cmd + koch temp [options] c test.nim + + # (will build with js support) + koch temp [options] js test.nim + + # (will build with doc support) + koch temp [options] doc test.nim + ``` + +Debug logging +------------- + +"Printf debugging" is still the most appropriate way to debug many problems +arising in compiler development. The typical usage of breakpoints to debug +the code is often less practical, because almost all code paths in the +compiler will be executed hundreds of times before a particular section of the +tested program is reached where the newly developed code must be activated. + +To work around this problem, you'll typically introduce an if statement in the +compiler code detecting more precisely the conditions where the tested feature +is being used. One very common way to achieve this is to use the `mdbg` condition, +which will be true only in contexts, processing expressions and statements from +the currently compiled main module: + + ```nim + # inside some compiler module + if mdbg: + debug someAstNode + ``` + +Using the `isCompilerDebug`:nim: condition along with inserting some statements +into the testcase provides more granular logging: + + ```nim + # compilermodule.nim + if isCompilerDebug(): + debug someAstNode + + # testcase.nim + proc main = + {.define(nimCompilerDebug).} + let a = 2.5 * 3 + {.undef(nimCompilerDebug).} + ``` + +Logging can also be scoped to a specific filename as well. This will of course +match against every module with that name. + + ```nim + if `??`(conf, n.info, "module.nim"): + debug(n) + ``` + +The above examples also makes use of the `debug`:nim: proc, which is able to +print a human-readable form of an arbitrary AST tree. Other common ways to print +information about the internal compiler types include: + + ```nim + # pretty print PNode + + # pretty prints the Nim ast + echo renderTree(someNode) + + # pretty prints the Nim ast, but annotates symbol IDs + echo renderTree(someNode, {renderIds}) + + # pretty print ast as JSON + debug(someNode) + + # print as YAML + echo treeToYaml(config, someNode) + + + # pretty print PType + + # print type name + echo typeToString(someType) + + # pretty print as JSON + debug(someType) + + # print as YAML + echo typeToYaml(config, someType) + + + # pretty print PSym + + # print the symbol's name + echo symbol.name.s + + # pretty print as JSON + debug(symbol) + + # print as YAML + echo symToYaml(config, symbol) + + + # pretty print TLineInfo + lineInfoToStr(lineInfo) + + + # print the structure of any type + repr(someVar) + ``` + +Here are some other helpful utilities: + + ```nim + # how did execution reach this location? + writeStackTrace() + ``` + +These procs may not already be imported by the module you're editing. +You can import them directly for debugging: + + ```nim + from astalgo import debug + from types import typeToString + from renderer import renderTree + from msgs import `??` + ``` + +Native debugging +---------------- + +Stepping through the compiler with a native debugger is a very powerful tool to +both learn and debug it. However, there is still the need to constrain when +breakpoints are triggered. The same methods as in [Debug logging] can be applied +here when combined with calls to the debug helpers `enteringDebugSection()`:nim: +and `exitingDebugSection()`:nim:. + +#. Compile the temp compiler with `--debugger:native -d:nimDebugUtils`:option: +#. Set your desired breakpoints or watchpoints. +#. Configure your debugger: + * GDB: execute `source tools/compiler.gdb` at startup + * LLDB execute `command source tools/compiler.lldb` at startup +#. Use one of the scoping helpers like so: + + ```nim + if isCompilerDebug(): + enteringDebugSection() + else: + exitingDebugSection() + ``` + +A caveat of this method is that all breakpoints and watchpoints are enabled or +disabled. Also, due to a bug, only breakpoints can be constrained for LLDB. + +The compiler's architecture +=========================== + +Nim uses the classic compiler architecture: A lexer/scanner feeds tokens to a +parser. The parser builds a syntax tree that is used by the code generators. +This syntax tree is the interface between the parser and the code generator. +It is essential to understand most of the compiler's code. + +Semantic analysis is separated from parsing. + +.. include:: filelist.txt + + +The syntax tree +--------------- +The syntax tree consists of nodes which may have an arbitrary number of +children. Types and symbols are represented by other nodes, because they +may contain cycles. The AST changes its shape after semantic checking. This +is needed to make life easier for the code generators. See the "ast" module +for the type definitions. The [macros](macros.html) module contains many +examples how the AST represents each syntactic structure. + + +Runtimes +======== + +Nim has two different runtimes, the "old runtime" and the "new runtime". The old +runtime supports the old GCs (markAndSweep, refc, Boehm), the new runtime supports +ARC/ORC. The new runtime is active `when defined(nimV2)`. + + +Coding Guidelines +================= + +* We follow Nim's official style guide, see [NEP1](nep1.html). +* Max line length is 100 characters. +* Provide spaces around binary operators if that enhances readability. +* Use a space after a colon, but not before it. +* (deprecated) Start types with a capital `T`, unless they are + pointers/references which start with `P`. +* Prefer `import package`:nim: over `from package import symbol`:nim:. + +See also the [API naming design](apis.html) document. + + +Porting to new platforms +======================== + +Porting Nim to a new architecture is pretty easy, since C is the most +portable programming language (within certain limits) and Nim generates +C code, porting the code generator is not necessary. + +POSIX-compliant systems on conventional hardware are usually pretty easy to +port: Add the platform to `platform` (if it is not already listed there), +check that the OS, System modules work and recompile Nim. + +The only case where things aren't as easy is when old runtime's garbage +collectors need some assembler tweaking to work. The default +implementation uses C's `setjmp`:c: function to store all registers +on the hardware stack. It may be necessary that the new platform needs to +replace this generic code by some assembler code. + +Files that may need changed for your platform include: + +* `compiler/platform.nim` + Add os/cpu properties. +* `lib/system.nim` + Add os/cpu to the documentation for `system.hostOS` and `system.hostCPU`. +* `compiler/options.nim` + Add special os/cpu property checks in `isDefined`. +* `compiler/installer.ini` + Add os/cpu to `Project.Platforms` field. +* `lib/system/platforms.nim` + Add os/cpu. +* `std/private/osseps.nim` + Add os specializations. +* `lib/pure/distros.nim` + Add os, package handler. +* `tools/niminst/makefile.nimf` + Add os/cpu compiler/linker flags. +* `tools/niminst/buildsh.nimf` + Add os/cpu compiler/linker flags. + +If the `--os` or `--cpu` options aren't passed to the compiler, then Nim will +determine the current host os, cpu and endianness from `system.cpuEndian`, +`system.hostOS` and `system.hostCPU`. Those values are derived from +`compiler/platform.nim`. + +In order for the new platform to be bootstrapped from the `csources`, it must: + +* have `compiler/platform.nim` updated +* have `compiler/installer.ini` updated +* have `tools/niminst/buildsh.nimf` updated +* have `tools/niminst/makefile.nimf` updated +* be backported to the Nim version used by the `csources` +* the new `csources` must be pushed +* the new `csources` revision must be updated in `config/build_config.txt` + + +Runtime type information +======================== + +**Note**: This section describes the "old runtime". + +*Runtime type information* (RTTI) is needed for several aspects of the Nim +programming language: + +Garbage collection +: The old GCs use the RTTI for traversing arbitrary Nim types, but usually + only the `marker` field which contains a proc that does the traversal. + +Complex assignments +: Sequences and strings are implemented as + pointers to resizable buffers, but Nim requires copying for + assignments. Apart from RTTI the compiler also generates copy procedures + as a specialization. + +We already know the type information as a graph in the compiler. +Thus, we need to serialize this graph as RTTI for C code generation. +Look at the file ``lib/system/hti.nim`` for more information. + + +Magics and compilerProcs +======================== + +The `system` module contains the part of the RTL which needs support by +compiler magic. The C code generator generates the C code for it, just like any other +module. However, calls to some procedures like `addInt` are inserted by +the generator. Therefore, there is a table (`compilerprocs`) +with all symbols that are marked as `compilerproc`. `compilerprocs` are +needed by the code generator. A `magic` proc is not the same as a +`compilerproc`: A `magic` is a proc that needs compiler magic for its +semantic checking, a `compilerproc` is a proc that is used by the code +generator. + + +Code generation for closures +============================ + +Code generation for closures is implemented by `lambda lifting`:idx:. + + +Design +------ + +A `closure` proc var can call ordinary procs of the default Nim calling +convention. But not the other way round! A closure is implemented as a +`tuple[prc, env]`. `env` can be nil implying a call without a closure. +This means that a call through a closure generates an `if` but the +interoperability is worth the cost of the `if`. Thunk generation would be +possible too, but it's slightly more effort to implement. + +Tests with GCC on Amd64 showed that it's really beneficial if the +'environment' pointer is passed as the last argument, not as the first argument. + +Proper thunk generation is harder because the proc that is to wrap +could stem from a complex expression: + + ```nim + receivesClosure(returnsDefaultCC[i]) + ``` + +A thunk would need to call `returnsDefaultCC[i]` somehow and that would require +an *additional* closure generation... Ok, not really, but it requires to pass +the function to call. So we'd end up with 2 indirect calls instead of one. +Another much more severe problem with this solution is that it's not GC-safe +to pass a proc pointer around via a generic `ref` type. + + +Example code: + + ```nim + proc add(x: int): proc (y: int): int {.closure.} = + return proc (y: int): int = + return x + y + + var add2 = add(2) + echo add2(5) #OUT 7 + ``` + +This should produce roughly this code: + + ```nim + type + Env = ref object + x: int # data + + proc anon(y: int, c: Env): int = + return y + c.x + + proc add(x: int): tuple[prc, data] = + var env: Env + new env + env.x = x + result = (anon, env) + + var add2 = add(2) + let tmp = if add2.data == nil: add2.prc(5) else: add2.prc(5, add2.data) + echo tmp + ``` + + +Beware of nesting: + + ```nim + proc add(x: int): proc (y: int): proc (z: int): int {.closure.} {.closure.} = + return lambda (y: int): proc (z: int): int {.closure.} = + return lambda (z: int): int = + return x + y + z + + var add24 = add(2)(4) + echo add24(5) #OUT 11 + ``` + +This should produce roughly this code: + + ```nim + type + EnvX = ref object + x: int # data + + EnvY = ref object + y: int + ex: EnvX + + proc lambdaZ(z: int, ey: EnvY): int = + return ey.ex.x + ey.y + z + + proc lambdaY(y: int, ex: EnvX): tuple[prc, data: EnvY] = + var ey: EnvY + new ey + ey.y = y + ey.ex = ex + result = (lambdaZ, ey) + + proc add(x: int): tuple[prc, data: EnvX] = + var ex: EnvX + ex.x = x + result = (lambdaY, ex) + + var tmp = add(2) + var tmp2 = tmp.fn(4, tmp.data) + var add24 = tmp2.fn(4, tmp2.data) + echo add24(5) + ``` + + +We could get rid of nesting environments by always inlining inner anon procs. +More useful is escape analysis and stack allocation of the environment, +however. + + +Accumulator +----------- + + ```nim + proc getAccumulator(start: int): proc (): int {.closure} = + var i = start + return lambda: int = + inc i + return i + + proc p = + var delta = 7 + proc accumulator(start: int): proc(): int = + var x = start-1 + result = proc (): int = + x = x + delta + inc delta + return x + + var a = accumulator(3) + var b = accumulator(4) + echo a() + b() + ``` + + +Internals +--------- + +Lambda lifting is implemented as part of the `transf` pass. The `transf` +pass generates code to set up the environment and to pass it around. However, +this pass does not change the types! So we have some kind of mismatch here; on +the one hand the proc expression becomes an explicit tuple, on the other hand +the tyProc(ccClosure) type is not changed. For C code generation it's also +important the hidden formal param is `void*`:c: and not something more +specialized. However, the more specialized env type needs to passed to the +backend somehow. We deal with this by modifying `s.ast[paramPos]` to contain +the formal hidden parameter, but not `s.typ`! + + +Notes on type and AST representation +==================================== + +To be expanded. + + +Integer literals +---------------- + +In Nim, there is a redundant way to specify the type of an +integer literal. First, it should be unsurprising that every +node has a node kind. The node of an integer literal can be any of the +following values: + + nkIntLit, nkInt8Lit, nkInt16Lit, nkInt32Lit, nkInt64Lit, + nkUIntLit, nkUInt8Lit, nkUInt16Lit, nkUInt32Lit, nkUInt64Lit + +On top of that, there is also the `typ` field for the type. The +kind of the `typ` field can be one of the following ones, and it +should be matching the literal kind: + + tyInt, tyInt8, tyInt16, tyInt32, tyInt64, tyUInt, tyUInt8, + tyUInt16, tyUInt32, tyUInt64 + +Then there is also the integer literal type. This is a specific type +that is implicitly convertible into the requested type if the +requested type can hold the value. For this to work, the type needs to +know the concrete value of the literal. For example an expression +`321` will be of type `int literal(321)`. This type is implicitly +convertible to all integer types and ranges that contain the value +`321`. That would be all builtin integer types except `uint8` and +`int8` where `321` would be out of range. When this literal type is +assigned to a new `var` or `let` variable, it's type will be resolved +to just `int`, not `int literal(321)` unlike constants. A constant +keeps the full `int literal(321)` type. Here is an example where that +difference matters. + + + ```nim + proc foo(arg: int8) = + echo "def" + + const tmp1 = 123 + foo(tmp1) # OK + + let tmp2 = 123 + foo(tmp2) # Error + ``` + +In a context with multiple overloads, the integer literal kind will +always prefer the `int` type over all other types. If none of the +overloads is of type `int`, then there will be an error because of +ambiguity. + + ```nim + proc foo(arg: int) = + echo "abc" + proc foo(arg: int8) = + echo "def" + foo(123) # output: abc + + proc bar(arg: int16) = + echo "abc" + proc bar(arg: int8) = + echo "def" + + bar(123) # Error ambiguous call + ``` + +In the compiler these integer literal types are represented with the +node kind `nkIntLit`, type kind `tyInt` and the member `n` of the type +pointing back to the integer literal node in the ast containing the +integer value. These are the properties that hold true for integer +literal types. + + n.kind == nkIntLit + n.typ.kind == tyInt + n.typ.n == n + +Other literal types, such as `uint literal(123)` that would +automatically convert to other integer types, but prefers to +become a `uint` are not part of the Nim language. + +In an unchecked AST, the `typ` field is nil. The type checker will set +the `typ` field accordingly to the node kind. Nodes of kind `nkIntLit` +will get the integer literal type (e.g. `int literal(123)`). Nodes of +kind `nkUIntLit` will get type `uint` (kind `tyUint`), etc. + +This also means that it is not possible to write a literal in an +unchecked AST that will after sem checking just be of type `int` and +not implicitly convertible to other integer types. This only works for +all integer types that are not `int`. diff --git a/doc/intern.rst b/doc/intern.rst deleted file mode 100644 index 2456b25fd..000000000 --- a/doc/intern.rst +++ /dev/null @@ -1,796 +0,0 @@ -.. default-role:: code - -========================================= - Internals of the Nim Compiler -========================================= - - -:Author: Andreas Rumpf -:Version: |nimversion| - -.. contents:: - - "Abstraction is layering ignorance on top of reality." -- Richard Gabriel - - -Directory structure -=================== - -The Nim project's directory structure is: - -============ =================================================== -Path Purpose -============ =================================================== -`bin` generated binary files -`build` generated C code for the installation -`compiler` the Nim compiler itself; note that this - code has been translated from a bootstrapping - version written in Pascal, so the code is **not** - a poster child of good Nim code -`config` configuration files for Nim -`dist` additional packages for the distribution -`doc` the documentation; it is a bunch of - reStructuredText files -`lib` the Nim library -`web` website of Nim; generated by `nimweb` - from the `*.txt` and `*.nimf` files -============ =================================================== - - -Bootstrapping the compiler -========================== - -Compiling the compiler is a simple matter of running:: - - nim c koch.nim - ./koch boot - -For a release version use:: - - nim c koch.nim - ./koch boot -d:release - -And for a debug version compatible with GDB:: - - nim c koch.nim - ./koch boot --debuginfo --linedir:on - -The `koch` program is Nim's maintenance script. It is a replacement for -make and shell scripting with the advantage that it is much more portable. -More information about its options can be found in the `koch <koch.html>`_ -documentation. - - -Coding Guidelines -================= - -* Use CamelCase, not underscored_identifiers. -* Indent with two spaces. -* Max line length is 80 characters. -* Provide spaces around binary operators if that enhances readability. -* Use a space after a colon, but not before it. -* [deprecated] Start types with a capital `T`, unless they are - pointers/references which start with `P`. - -See also the `API naming design <apis.html>`_ document. - - -Porting to new platforms -======================== - -Porting Nim to a new architecture is pretty easy, since C is the most -portable programming language (within certain limits) and Nim generates -C code, porting the code generator is not necessary. - -POSIX-compliant systems on conventional hardware are usually pretty easy to -port: Add the platform to `platform` (if it is not already listed there), -check that the OS, System modules work and recompile Nim. - -The only case where things aren't as easy is when the garbage -collector needs some assembler tweaking to work. The standard -version of the GC uses C's `setjmp` function to store all registers -on the hardware stack. It may be necessary that the new platform needs to -replace this generic code by some assembler code. - - -Runtime type information -======================== - -*Runtime type information* (RTTI) is needed for several aspects of the Nim -programming language: - -Garbage collection - The most important reason for RTTI. Generating - traversal procedures produces bigger code and is likely to be slower on - modern hardware as dynamic procedure binding is hard to predict. - -Complex assignments - Sequences and strings are implemented as - pointers to resizeable buffers, but Nim requires copying for - assignments. Apart from RTTI the compiler could generate copy procedures - for any type that needs one. However, this would make the code bigger and - the RTTI is likely already there for the GC. - -We already know the type information as a graph in the compiler. -Thus we need to serialize this graph as RTTI for C code generation. -Look at the file `lib/system/hti.nim` for more information. - -Rebuilding the compiler -======================== - -After an initial build via `sh build_all.sh` on posix or `build_all.bat` on windows, -you can rebuild the compiler as follows: - -* `nim c koch` if you need to rebuild koch -* `./koch boot -d:release` this ensures the compiler can rebuild itself - (use `koch` instead of `./koch` on windows), which builds the compiler 3 times. - -A faster approach if you don't need to run the full bootstrapping implied by `koch boot`, -is the following: - -* `pathto/nim c --lib:lib -d:release -o:bin/nim_temp compiler/nim.nim` - -Where `pathto/nim` is any nim binary sufficiently recent (e.g. `bin/nim_cources` -built during bootstrap or `$HOME/.nimble/bin/nim` installed by `choosenim 1.2.0`) - -You can pass any additional options such as `-d:leanCompiler` if you don't need -certain features or `-d:debug --stacktrace:on --excessiveStackTrace --stackTraceMsgs` -for debugging the compiler. See also `Debugging the compiler`_. - -Debugging the compiler -====================== - -You can of course use GDB or Visual Studio to debug the -compiler (via `--debuginfo --lineDir:on`). However, there -are also lots of procs that aid in debugging: - - -.. code-block:: nim - # pretty prints the Nim AST - echo renderTree(someNode) - # outputs some JSON representation - debug(someNode) - # pretty prints some type - echo typeToString(someType) - debug(someType) - echo symbol.name.s - debug(symbol) - # pretty prints the Nim ast, but annotates symbol IDs: - echo renderTree(someNode, {renderIds}) - if `??`(conf, n.info, "temp.nim"): - # only output when it comes from "temp.nim" - echo renderTree(n) - if `??`(conf, n.info, "temp.nim"): - # why does it process temp.nim here? - writeStackTrace() - -These procs may not be imported by a module. You can import them directly for debugging: - -.. code-block:: nim - from astalgo import debug - from types import typeToString - from renderer import renderTree - from msgs import `??` - -To create a new compiler for each run, use `koch temp`:: - - ./koch temp c /tmp/test.nim - -`koch temp` creates a debug build of the compiler, which is useful -to create stacktraces for compiler debugging. See also -`Rebuilding the compiler`_ if you need more control. - -Bisecting for regressions -========================= - -`koch temp` returns 125 as the exit code in case the compiler -compilation fails. This exit code tells `git bisect` to skip the -current commit.:: - - git bisect start bad-commit good-commit - git bisect run ./koch temp -r c test-source.nim - -You can also bisect using custom options to build the compiler, for example if -you don't need a debug version of the compiler (which runs slower), you can replace -`./koch temp` by explicit compilation command, see `Rebuilding the compiler`_. - -The compiler's architecture -=========================== - -Nim uses the classic compiler architecture: A lexer/scanner feds tokens to a -parser. The parser builds a syntax tree that is used by the code generator. -This syntax tree is the interface between the parser and the code generator. -It is essential to understand most of the compiler's code. - -In order to compile Nim correctly, type-checking has to be separated from -parsing. Otherwise generics cannot work. - -.. include:: filelist.txt - - -The syntax tree ---------------- -The syntax tree consists of nodes which may have an arbitrary number of -children. Types and symbols are represented by other nodes, because they -may contain cycles. The AST changes its shape after semantic checking. This -is needed to make life easier for the code generators. See the "ast" module -for the type definitions. The `macros <macros.html>`_ module contains many -examples how the AST represents each syntactic structure. - - -How the RTL is compiled -======================= - -The `system` module contains the part of the RTL which needs support by -compiler magic (and the stuff that needs to be in it because the spec -says so). The C code generator generates the C code for it, just like any other -module. However, calls to some procedures like `addInt` are inserted by -the CCG. Therefore the module `magicsys` contains a table (`compilerprocs`) -with all symbols that are marked as `compilerproc`. `compilerprocs` are -needed by the code generator. A `magic` proc is not the same as a -`compilerproc`: A `magic` is a proc that needs compiler magic for its -semantic checking, a `compilerproc` is a proc that is used by the code -generator. - - -Compilation cache -================= - -The implementation of the compilation cache is tricky: There are lots -of issues to be solved for the front- and backend. - - -General approach: AST replay ----------------------------- - -We store a module's AST of a successful semantic check in a SQLite -database. There are plenty of features that require a sub sequence -to be re-applied, for example: - -.. code-block:: nim - {.compile: "foo.c".} # even if the module is loaded from the DB, - # "foo.c" needs to be compiled/linked. - -The solution is to **re-play** the module's top level statements. -This solves the problem without having to special case the logic -that fills the internal seqs which are affected by the pragmas. - -In fact, this describes how the AST should be stored in the database, -as a "shallow" tree. Let's assume we compile module `m` with the -following contents: - -.. code-block:: nim - import std/strutils - - var x*: int = 90 - {.compile: "foo.c".} - proc p = echo "p" - proc q = echo "q" - static: - echo "static" - -Conceptually this is the AST we store for the module: - -.. code-block:: nim - import std/strutils - - var x* - {.compile: "foo.c".} - proc p - proc q - static: - echo "static" - -The symbol's `ast` field is loaded lazily, on demand. This is where most -savings come from, only the shallow outer AST is reconstructed immediately. - -It is also important that the replay involves the `import` statement so -that dependencies are resolved properly. - - -Shared global compiletime state -------------------------------- - -Nim allows `.global, compiletime` variables that can be filled by macro -invocations across different modules. This feature breaks modularity in a -severe way. Plenty of different solutions have been proposed: - -- Restrict the types of global compiletime variables to `Set[T]` or - similar unordered, only-growable collections so that we can track - the module's write effects to these variables and reapply the changes - in a different order. -- In every module compilation, reset the variable to its default value. -- Provide a restrictive API that can load/save the compiletime state to - a file. - -(These solutions are not mutually exclusive.) - -Since we adopt the "replay the top level statements" idea, the natural -solution to this problem is to emit pseudo top level statements that -reflect the mutations done to the global variable. However, this is -MUCH harder than it sounds, for example `squeaknim` uses this -snippet: - -.. code-block:: nim - apicall.add(") module: '" & dllName & "'>\C" & - "\t^self externalCallFailed\C!\C\C") - stCode.add(st & "\C\t\"Generated by NimSqueak\"\C\t" & apicall) - -We can "replay" `stCode.add` only if the values of `st` -and `apicall` are known. And even then a hash table's `add` with its -hashing mechanism is too hard to replay. - -In practice, things are worse still, consider `someGlobal[i][j].add arg`. -We only know the root is `someGlobal` but the concrete path to the data -is unknown as is the value that is added. We could compute a "diff" between -the global states and use that to compute a symbol patchset, but this is -quite some work, expensive to do at runtime (it would need to run after -every module has been compiled) and would also break for hash tables. - -We need an API that hides the complex aliasing problems by not relying -on Nim's global variables. The obvious solution is to use string keys -instead of global variables: - -.. code-block:: nim - - proc cachePut*(key: string; value: string) - proc cacheGet*(key: string): string - -However, the values being strings/json is quite problematic: Many -lookup tables that are built at compiletime embed *proc vars* and -types which have no obvious string representation... Seems like -AST diffing is still the best idea as it will not require to use -an alien API and works with some existing Nimble packages, at least. - -On the other hand, in Nim's future I would like to replace the VM -by native code. A diff algorithm wouldn't work for that. -Instead the native code would work with an API like `put`, `get`: - -.. code-block:: nim - - proc cachePut*(key: string; value: NimNode) - proc cacheGet*(key: string): NimNode - -The API should embrace the AST diffing notion: See the -module `macrocache` for the final details. - - - -Methods and type converters ---------------------------- - -In the following -sections *global* means *shared between modules* or *property of the whole -program*. - -Nim contains language features that are *global*. The best example for that -are multi methods: Introducing a new method with the same name and some -compatible object parameter means that the method's dispatcher needs to take -the new method into account. So the dispatching logic is only completely known -after the whole program has been translated! - -Other features that are *implicitly* triggered cause problems for modularity -too. Type converters fall into this category: - -.. code-block:: nim - # module A - converter toBool(x: int): bool = - result = x != 0 - -.. code-block:: nim - # module B - import A - - if 1: - echo "ugly, but should work" - -If in the above example module `B` is re-compiled, but `A` is not then -`B` needs to be aware of `toBool` even though `toBool` is not referenced -in `B` *explicitly*. - -Both the multi method and the type converter problems are solved by the -AST replay implementation. - - -Generics -~~~~~~~~ - -We cache generic instantiations and need to ensure this caching works -well with the incremental compilation feature. Since the cache is -attached to the `PSym` datastructure, it should work without any -special logic. - - -Backend issues --------------- - -- Init procs must not be "forgotten" to be called. -- Files must not be "forgotten" to be linked. -- Method dispatchers are global. -- DLL loading via `dlsym` is global. -- Emulated thread vars are global. - -However the biggest problem is that dead code elimination breaks modularity! -To see why, consider this scenario: The module `G` (for example the huge -Gtk2 module...) is compiled with dead code elimination turned on. So none -of `G`'s procs is generated at all. - -Then module `B` is compiled that requires `G.P1`. Ok, no problem, -`G.P1` is loaded from the symbol file and `G.c` now contains `G.P1`. - -Then module `A` (that depends on `B` and `G`) is compiled and `B` -and `G` are left unchanged. `A` requires `G.P2`. - -So now `G.c` MUST contain both `P1` and `P2`, but we haven't even -loaded `P1` from the symbol file, nor do we want to because we then quickly -would restore large parts of the whole program. - - -Solution -~~~~~~~~ - -The backend must have some logic so that if the currently processed module -is from the compilation cache, the `ast` field is not accessed. Instead -the generated C(++) for the symbol's body needs to be cached too and -inserted back into the produced C file. This approach seems to deal with -all the outlined problems above. - - -Debugging Nim's memory management -================================= - -The following paragraphs are mostly a reminder for myself. Things to keep -in mind: - -* If an assertion in Nim's memory manager or GC fails, the stack trace - keeps allocating memory! Thus a stack overflow may happen, hiding the - real issue. -* What seem to be C code generation problems is often a bug resulting from - not producing prototypes, so that some types default to `cint`. Testing - without the `-w` option helps! - - -The Garbage Collector -===================== - -Introduction ------------- - -I use the term *cell* here to refer to everything that is traced -(sequences, refs, strings). -This section describes how the GC works. - -The basic algorithm is *Deferrent Reference Counting* with cycle detection. -References on the stack are not counted for better performance and easier C -code generation. - -Each cell has a header consisting of a RC and a pointer to its type -descriptor. However the program does not know about these, so they are placed at -negative offsets. In the GC code the type `PCell` denotes a pointer -decremented by the right offset, so that the header can be accessed easily. It -is extremely important that `pointer` is not confused with a `PCell` -as this would lead to a memory corruption. - - -The CellSet data structure --------------------------- - -The GC depends on an extremely efficient datastructure for storing a -set of pointers - this is called a `TCellSet` in the source code. -Inserting, deleting and searching are done in constant time. However, -modifying a `TCellSet` during traversal leads to undefined behaviour. - -.. code-block:: Nim - type - TCellSet # hidden - - proc cellSetInit(s: var TCellSet) # initialize a new set - proc cellSetDeinit(s: var TCellSet) # empty the set and free its memory - proc incl(s: var TCellSet, elem: PCell) # include an element - proc excl(s: var TCellSet, elem: PCell) # exclude an element - - proc `in`(elem: PCell, s: TCellSet): bool # tests membership - - iterator elements(s: TCellSet): (elem: PCell) - - -All the operations have to perform efficiently. Because a Cellset can -become huge a hash table alone is not suitable for this. - -We use a mixture of bitset and hash table for this. The hash table maps *pages* -to a page descriptor. The page descriptor contains a bit for any possible cell -address within this page. So including a cell is done as follows: - -- Find the page descriptor for the page the cell belongs to. -- Set the appropriate bit in the page descriptor indicating that the - cell points to the start of a memory block. - -Removing a cell is analogous - the bit has to be set to zero. -Single page descriptors are never deleted from the hash table. This is not -needed as the data structures needs to be rebuilt periodically anyway. - -Complete traversal is done in this way:: - - for each page descriptor d: - for each bit in d: - if bit == 1: - traverse the pointer belonging to this bit - - -Further complications ---------------------- - -In Nim the compiler cannot always know if a reference -is stored on the stack or not. This is caused by var parameters. -Consider this example: - -.. code-block:: Nim - proc setRef(r: var ref TNode) = - new(r) - - proc usage = - var - r: ref TNode - setRef(r) # here we should not update the reference counts, because - # r is on the stack - setRef(r.left) # here we should update the refcounts! - -We have to decide at runtime whether the reference is on the stack or not. -The generated code looks roughly like this: - -.. code-block:: C - void setref(TNode** ref) { - unsureAsgnRef(ref, newObj(TNode_TI, sizeof(TNode))) - } - void usage(void) { - setRef(&r) - setRef(&r->left) - } - -Note that for systems with a continuous stack (which most systems have) -the check whether the ref is on the stack is very cheap (only two -comparisons). - - -Code generation for closures -============================ - -Code generation for closures is implemented by `lambda lifting`:idx:. - - -Design ------- - -A `closure` proc var can call ordinary procs of the default Nim calling -convention. But not the other way round! A closure is implemented as a -`tuple[prc, env]`. `env` can be nil implying a call without a closure. -This means that a call through a closure generates an `if` but the -interoperability is worth the cost of the `if`. Thunk generation would be -possible too, but it's slightly more effort to implement. - -Tests with GCC on Amd64 showed that it's really beneficial if the -'environment' pointer is passed as the last argument, not as the first argument. - -Proper thunk generation is harder because the proc that is to wrap -could stem from a complex expression: - -.. code-block:: nim - receivesClosure(returnsDefaultCC[i]) - -A thunk would need to call 'returnsDefaultCC[i]' somehow and that would require -an *additional* closure generation... Ok, not really, but it requires to pass -the function to call. So we'd end up with 2 indirect calls instead of one. -Another much more severe problem which this solution is that it's not GC-safe -to pass a proc pointer around via a generic `ref` type. - - -Example code: - -.. code-block:: nim - proc add(x: int): proc (y: int): int {.closure.} = - return proc (y: int): int = - return x + y - - var add2 = add(2) - echo add2(5) #OUT 7 - -This should produce roughly this code: - -.. code-block:: nim - type - PEnv = ref object - x: int # data - - proc anon(y: int, c: PEnv): int = - return y + c.x - - proc add(x: int): tuple[prc, data] = - var env: PEnv - new env - env.x = x - result = (anon, env) - - var add2 = add(2) - let tmp = if add2.data == nil: add2.prc(5) else: add2.prc(5, add2.data) - echo tmp - - -Beware of nesting: - -.. code-block:: nim - proc add(x: int): proc (y: int): proc (z: int): int {.closure.} {.closure.} = - return lambda (y: int): proc (z: int): int {.closure.} = - return lambda (z: int): int = - return x + y + z - - var add24 = add(2)(4) - echo add24(5) #OUT 11 - -This should produce roughly this code: - -.. code-block:: nim - type - PEnvX = ref object - x: int # data - - PEnvY = ref object - y: int - ex: PEnvX - - proc lambdaZ(z: int, ey: PEnvY): int = - return ey.ex.x + ey.y + z - - proc lambdaY(y: int, ex: PEnvX): tuple[prc, data: PEnvY] = - var ey: PEnvY - new ey - ey.y = y - ey.ex = ex - result = (lambdaZ, ey) - - proc add(x: int): tuple[prc, data: PEnvX] = - var ex: PEnvX - ex.x = x - result = (labmdaY, ex) - - var tmp = add(2) - var tmp2 = tmp.fn(4, tmp.data) - var add24 = tmp2.fn(4, tmp2.data) - echo add24(5) - - -We could get rid of nesting environments by always inlining inner anon procs. -More useful is escape analysis and stack allocation of the environment, -however. - - -Alternative ------------ - -Process the closure of all inner procs in one pass and accumulate the -environments. This is however not always possible. - - -Accumulator ------------ - -.. code-block:: nim - proc getAccumulator(start: int): proc (): int {.closure} = - var i = start - return lambda: int = - inc i - return i - - proc p = - var delta = 7 - proc accumulator(start: int): proc(): int = - var x = start-1 - result = proc (): int = - x = x + delta - inc delta - return x - - var a = accumulator(3) - var b = accumulator(4) - echo a() + b() - - -Internals ---------- - -Lambda lifting is implemented as part of the `transf` pass. The `transf` -pass generates code to setup the environment and to pass it around. However, -this pass does not change the types! So we have some kind of mismatch here; on -the one hand the proc expression becomes an explicit tuple, on the other hand -the tyProc(ccClosure) type is not changed. For C code generation it's also -important the hidden formal param is `void*` and not something more -specialized. However the more specialized env type needs to passed to the -backend somehow. We deal with this by modifying `s.ast[paramPos]` to contain -the formal hidden parameter, but not `s.typ`! - - -Integer literals: ------------------ - -In Nim, there is a redundant way to specify the type of an -integer literal. First of all, it should be unsurprising that every -node has a node kind. The node of an integer literal can be any of the -following values: - - nkIntLit, nkInt8Lit, nkInt16Lit, nkInt32Lit, nkInt64Lit, - nkUIntLit, nkUInt8Lit, nkUInt16Lit, nkUInt32Lit, nkUInt64Lit - -On top of that, there is also the `typ` field for the type. It the -kind of the `typ` field can be one of the following ones, and it -should be matching the literal kind: - - tyInt, tyInt8, tyInt16, tyInt32, tyInt64, tyUInt, tyUInt8, - tyUInt16, tyUInt32, tyUInt64 - -Then there is also the integer literal type. This is a specific type -that is implicitly convertible into the requested type if the -requested type can hold the value. For this to work, the type needs to -know the concrete value of the literal. For example an expression -`321` will be of type `int literal(321)`. This type is implicitly -convertible to all integer types and ranges that contain the value -`321`. That would be all builtin integer types except `uint8` and -`int8` where `321` would be out of range. When this literal type is -assigned to a new `var` or `let` variable, it's type will be resolved -to just `int`, not `int literal(321)` unlike constants. A constant -keeps the full `int literal(321)` type. Here is an example where that -difference matters. - - -.. code-block:: nim - - proc foo(arg: int8) = - echo "def" - - const tmp1 = 123 - foo(tmp1) # OK - - let tmp2 = 123 - foo(tmp2) # Error - -In a context with multiple overloads, the integer literal kind will -always prefer the `int` type over all other types. If none of the -overloads is of type `int`, then there will be an error because of -ambiguity. - -.. code-block:: nim - - proc foo(arg: int) = - echo "abc" - proc foo(arg: int8) = - echo "def" - foo(123) # output: abc - - proc bar(arg: int16) = - echo "abc" - proc bar(arg: int8) = - echo "def" - - bar(123) # Error ambiguous call - -In the compiler these integer literal types are represented with the -node kind `nkIntLit`, type kind `tyInt` and the member `n` of the type -pointing back to the integer literal node in the ast containing the -integer value. These are the properties that hold true for integer -literal types. - - n.kind == nkIntLit - n.typ.kind == tyInt - n.typ.n == n - -Other literal types, such as `uint literal(123)` that would -automatically convert to other integer types, but prefers to -become a `uint` are not part of the Nim language. - -In an unchecked AST, the `typ` field is nil. The type checker will set -the `typ` field accordingly to the node kind. Nodes of kind `nkIntLit` -will get the integer literal type (e.g. `int literal(123)`). Nodes of -kind `nkUIntLit` will get type `uint` (kind `tyUint`), etc. - -This also means that it is not possible to write a literal in an -unchecked AST that will after sem checking just be of type `int` and -not implicitly convertible to other integer types. This only works for -all integer types that are not `int`. diff --git a/doc/koch.rst b/doc/koch.md index 1eb02d785..8fa19ce44 100644 --- a/doc/koch.rst +++ b/doc/koch.md @@ -1,17 +1,14 @@ -.. default-role:: code - =============================== Nim maintenance script =============================== :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: -.. raw:: html - <blockquote><p> - "A great chef is an artist that I truly respect" -- Robert Stack. - </p></blockquote> +> "A great chef is an artist that I truly respect" -- Robert Stack. Introduction @@ -19,7 +16,7 @@ Introduction The `koch`:idx: program is Nim's maintenance script. It is a replacement for make and shell scripting with the advantage that it is much more portable. -The word *koch* means *cook* in German. `koch` is used mainly to build the +The word *koch* means *cook* in German. `koch`:cmd: is used mainly to build the Nim compiler, but it can also be used for other tasks. This document describes the supported commands and their options. @@ -33,12 +30,14 @@ boot command The `boot`:idx: command bootstraps the compiler, and it accepts different options: --d:release - By default a debug version is created, passing this option will +-d:release By default a debug version is created, passing this option will force a release build, which is much faster and should be preferred unless you are debugging the compiler. --d:nimUseLinenoise - Use the linenoise library for interactive mode (not needed on Windows). +-d:nimUseLinenoise Use the linenoise library for interactive mode + (not needed on Windows). +-d:leanCompiler Produce a compiler without JS codegen or + documentation generator in order to use less RAM + for bootstrapping. After compilation is finished you will hopefully end up with the nim compiler in the `bin` directory. You can add Nim's `bin` directory to @@ -49,23 +48,23 @@ csource command --------------- The `csource`:idx: command builds the C sources for installation. It accepts -the same options as you would pass to the `boot command -<#commands-boot-command>`_. +the same options as you would pass to the [boot command]( +#commands-boot-command). temp command ------------ The temp command builds the Nim compiler but with a different final name -(`nim_temp`), so it doesn't overwrite your normal compiler. You can use -this command to test different options, the same you would issue for the `boot -command <#commands-boot-command>`_. +(`nim_temp`:cmd:), so it doesn't overwrite your normal compiler. You can use +this command to test different options, the same you would issue for the [boot +command](#commands-boot-command). test command ------------ -The `test`:idx: command can also be invoked with the alias `tests`. This -command will compile and run `testament/tester.nim`, which is the main -driver of Nim's test suite. You can pass options to the `test` command, +The `test`:idx: command can also be invoked with the alias `tests`:option:. This +command will compile and run ``testament/tester.nim``, which is the main +driver of Nim's test suite. You can pass options to the `test`:option: command, they will be forwarded to the tester. See its source code for available options. @@ -74,13 +73,13 @@ web command The `web`:idx: command converts the documentation in the `doc` directory from rst to HTML. It also repeats the same operation but places the result in -the `web/upload` which can be used to update the website at +the ``web/upload`` which can be used to update the website at https://nim-lang.org. By default, the documentation will be built in parallel using the number of available CPU cores. If any documentation build sub-commands fail, they will be rerun in serial fashion so that meaningful error output can be gathered for -inspection. The `--parallelBuild:n` switch or configuration option can be +inspection. The `--parallelBuild:n`:option: switch or configuration option can be used to force a specific number of parallel jobs or run everything serially from the start (`n == 1`). @@ -89,4 +88,4 @@ pdf command The `pdf`:idx: command builds PDF versions of Nim documentation: Manual, Tutorial and a few other documents. To run it one needs to -`install Latex/pdflatex <https://www.latex-project.org/get>`_ first. +[install Latex/xelatex](https://www.latex-project.org/get) first. diff --git a/doc/lib.md b/doc/lib.md new file mode 100644 index 000000000..1507bbaac --- /dev/null +++ b/doc/lib.md @@ -0,0 +1,682 @@ +==================== +Nim Standard Library +==================== + +:Author: Andreas Rumpf +:Version: |nimversion| + +.. default-role:: code +.. include:: rstcommon.rst +.. contents:: + +Nim's library is divided into *pure libraries*, *impure libraries*, and *wrappers*. + +Pure libraries do not depend on any external ``*.dll`` or ``lib*.so`` binary +while impure libraries do. A wrapper is an impure library that is a very +low-level interface to a C library. + +Read [this document](apis.html) for a quick overview of the API design. + + +Nimble +====== + +Nim's standard library only covers the basics, check +out https://nimble.directory/ for a list of 3rd party packages. + + +Pure libraries +============== + +Automatic imports +----------------- + +* [system](system.html) + Basic procs and operators that every program needs. It also provides IO + facilities for reading and writing text and binary files. It is imported + implicitly by the compiler. Do not import it directly. It relies on compiler + magic to work. + +Core +---- + +* [atomics](atomics.html) + Types and operations for atomic operations and lockless algorithms. + +* [bitops](bitops.html) + Provides a series of low-level methods for bit manipulation. + +* [compilesettings](compilesettings.html) + Querying the compiler about diverse configuration settings from code. + +* [cpuinfo](cpuinfo.html) + Procs to determine the number of CPUs / cores. + +* [effecttraits](effecttraits.html) + Access to the inferred .raises effects + for Nim's macro system. + +* [endians](endians.html) + Helpers that deal with different byte orders. + +* [locks](locks.html) + Locks and condition variables for Nim. + +* [macrocache](macrocache.html) + Provides an API for macros to collect compile-time information across modules. + +* [macros](macros.html) + Contains the AST API and documentation of Nim for writing macros. + +* [rlocks](rlocks.html) + Reentrant locks for Nim. + +* [typeinfo](typeinfo.html) + Provides (unsafe) access to Nim's run-time type information. + +* [typetraits](typetraits.html) + Compile-time reflection procs for working with types. + +* [volatile](volatile.html) + Code for generating volatile loads and stores, + which are useful in embedded and systems programming. + + +Algorithms +---------- + +* [algorithm](algorithm.html) + Some common generic algorithms like sort or binary search. + +* [enumutils](enumutils.html) + Additional functionality for the built-in `enum` type. + +* [sequtils](sequtils.html) + Operations for the built-in `seq` type + which were inspired by functional programming languages. + +* [setutils](setutils.html) + Additional functionality for the built-in `set` type. + + +Collections +----------- + +* [critbits](critbits.html) + A *crit bit tree* which is an efficient + container for a sorted set of strings, or a sorted mapping of strings. + +* [deques](deques.html) + Implementation of a double-ended queue. + The underlying implementation uses a `seq`. + +* [heapqueue](heapqueue.html) + Implementation of a binary heap data structure that can be used as a priority queue. + +* [intsets](intsets.html) + Efficient implementation of a set of ints as a sparse bit set. + +* [lists](lists.html) + Nim linked list support. Contains singly and doubly linked lists and + circular lists ("rings"). + +* [options](options.html) + The option type encapsulates an optional value. + +* [packedsets](packedsets.html) + Efficient implementation of a set of ordinals as a sparse bit set. + +* [ropes](ropes.html) + A *rope* data type. + Ropes can represent very long strings efficiently; + in particular, concatenation is done in O(1) instead of O(n). + +* [sets](sets.html) + Nim hash set support. + +* [strtabs](strtabs.html) + The `strtabs` module implements an efficient hash table that is a mapping + from strings to strings. Supports a case-sensitive, case-insensitive and + style-insensitive modes. + +* [tables](tables.html) + Nim hash table support. Contains tables, ordered tables, and count tables. + + +String handling +--------------- + +* [cstrutils](cstrutils.html) + Utilities for `cstring` handling. + +* [editdistance](editdistance.html) + An algorithm to compute the edit distance between two + Unicode strings. + +* [encodings](encodings.html) + Converts between different character encodings. On UNIX, this uses + the `iconv` library, on Windows the Windows API. + +* [formatfloat](formatfloat.html) + Formatting floats as strings. + +* [objectdollar](objectdollar.html) + A generic `$` operator to convert objects to strings. + +* [punycode](punycode.html) + Implements a representation of Unicode with the limited ASCII character subset. + +* [strbasics](strbasics.html) + Some high performance string operations. + +* [strformat](strformat.html) + Macro based standard string interpolation/formatting. Inspired by + Python's f-strings.\ + **Note:** if you need templating, consider using Nim + [Source Code Filters (SCF)](filters.html). + +* [strmisc](strmisc.html) + Uncommon string handling operations that do not + fit with the commonly used operations in [strutils](strutils.html). + +* [strscans](strscans.html) + A `scanf` macro for convenient parsing of mini languages. + +* [strutils](strutils.html) + Common string handling operations like changing + case of a string, splitting a string into substrings, searching for + substrings, replacing substrings. + +* [unicode](unicode.html) + Support for handling the Unicode UTF-8 encoding. + +* [unidecode](unidecode.html) + It provides a single proc that does Unicode to ASCII transliterations. + Based on Python's Unidecode module. + +* [widestrs](widestrs.html) + Nim support for C/C++'s wide strings. + +* [wordwrap](wordwrap.html) + An algorithm for word-wrapping Unicode strings. + + +Time handling +------------- + +* [monotimes](monotimes.html) + The `monotimes` module implements monotonic timestamps. + +* [times](times.html) + The `times` module contains support for working with time. + + +Generic Operating System Services +--------------------------------- + +* [appdirs](appdirs.html) + Helpers for determining special directories used by apps. + +* [cmdline](cmdline.html) + System facilities for reading command line parameters. + +* [dirs](dirs.html) + Directory handling. + +* [distros](distros.html) + Basics for OS distribution ("distro") detection + and the OS's native package manager. + Its primary purpose is to produce output for Nimble packages, + but it also contains the widely used **Distribution** enum + that is useful for writing platform-specific code. + See [packaging](packaging.html) for hints on distributing Nim using OS packages. + +* [dynlib](dynlib.html) + Accessing symbols from shared libraries. + +* [envvars](envvars.html) + Environment variable handling. + +* [exitprocs](exitprocs.html) + Adding hooks to program exit. + +* [files](files.html) + File handling. + +* [memfiles](memfiles.html) + Support for memory-mapped files (Posix's `mmap`) + on the different operating systems. + +* [os](os.html) + Basic operating system facilities like retrieving environment variables, + reading command line arguments, working with directories, running shell + commands, etc. + +* [oserrors](oserrors.html) + OS error reporting. + +* [osproc](osproc.html) + Module for process communication beyond `os.execShellCmd`. + +* [paths](paths.html) + Path handling. + +* [reservedmem](reservedmem.html) + Utilities for reserving portions of the + address space of a program without consuming physical memory. + +* [streams](streams.html) + A stream interface and two implementations thereof: + the `FileStream` and the `StringStream` which implement the stream + interface for Nim file objects (`File`) and strings. Other modules + may provide other implementations for this standard stream interface. + +* [symlinks](symlinks.html) + Symlink handling. + +* [syncio](syncio.html) + Various synchronized I/O operations. + +* [terminal](terminal.html) + A module to control the terminal output (also called *console*). + +* [tempfiles](tempfiles.html) + Some utilities for generating temporary path names and + creating temporary files and directories. + + +Math libraries +-------------- + +* [complex](complex.html) + Complex numbers and relevant mathematical operations. + +* [fenv](fenv.html) + Floating-point environment. Handling of floating-point rounding and + exceptions (overflow, zero-divide, etc.). + +* [lenientops](lenientops.html) + Binary operators for mixed integer/float expressions for convenience. + +* [math](math.html) + Mathematical operations like cosine, square root. + +* [random](random.html) + Fast and tiny random number generator. + +* [rationals](rationals.html) + Rational numbers and relevant mathematical operations. + +* [stats](stats.html) + Statistical analysis. + +* [sysrand](sysrand.html) + Cryptographically secure pseudorandom number generator. + + +Internet Protocols and Support +------------------------------ + +* [async](async.html) + Exports `asyncmacro` and `asyncfutures` for native backends, and `asyncjs` on the JS backend. + +* [asyncdispatch](asyncdispatch.html) + An asynchronous dispatcher for IO operations. + +* [asyncfile](asyncfile.html) + An asynchronous file reading and writing using `asyncdispatch`. + +* [asyncftpclient](asyncftpclient.html) + An asynchronous FTP client using the `asyncnet` module. + +* [asynchttpserver](asynchttpserver.html) + An asynchronous HTTP server using the `asyncnet` module. + +* [asyncmacro](asyncmacro.html) + `async` and `multisync` macros for `asyncdispatch`. + +* [asyncnet](asyncnet.html) + Asynchronous sockets based on the `asyncdispatch` module. + +* [asyncstreams](asyncstreams.html) + `FutureStream` - a future that acts as a queue. + +* [cgi](cgi.html) + Helpers for CGI applications. + +* [cookies](cookies.html) + Helper procs for parsing and generating cookies. + +* [httpclient](httpclient.html) + A simple HTTP client with support for both synchronous + and asynchronous retrieval of web pages. + +* [mimetypes](mimetypes.html) + A mimetypes database. + +* [nativesockets](nativesockets.html) + A low-level sockets API. + +* [net](net.html) + A high-level sockets API. + +* [selectors](selectors.html) + A selector API with backends specific to each OS. + Supported OS primitives: `epoll`, `kqueue`, `poll`, and `select` on Windows. + +* [smtp](smtp.html) + A simple SMTP client with support for both synchronous and asynchronous operation. + +* [socketstreams](socketstreams.html) + An implementation of the streams interface for sockets. + +* [uri](uri.html) + Functions for working with URIs and URLs. + + +Threading +--------- + +* [isolation](isolation.html) + The `Isolated[T]` type for + safe construction of isolated subgraphs that can be + passed efficiently to different channels and threads. + +* [tasks](tasks.html) + Basic primitives for creating parallel programs. + +* [threadpool](threadpool.html) + Implements Nim's [spawn](manual_experimental.html#parallel-amp-spawn). + +* [typedthreads](typedthreads.html) + Basic Nim thread support. + + +Parsers +------- + +* [htmlparser](htmlparser.html) + HTML document parser that creates a XML tree representation. + +* [json](json.html) + High-performance JSON parser. + +* [lexbase](lexbase.html) + A low-level module that implements an extremely efficient buffering + scheme for lexers and parsers. This is used by the diverse parsing modules. + +* [parsecfg](parsecfg.html) + The `parsecfg` module implements a high-performance configuration file + parser. The configuration file's syntax is similar to the Windows ``.ini`` + format, but much more powerful, as it is not a line based parser. String + literals, raw string literals, and triple quote string literals are supported + as in the Nim programming language. + +* [parsecsv](parsecsv.html) + The `parsecsv` module implements a simple high-performance CSV parser. + +* [parsejson](parsejson.html) + A JSON parser. It is used and exported by the [json](json.html) module, but can also be used in its own right. + +* [parseopt](parseopt.html) + The `parseopt` module implements a command line option parser. + +* [parsesql](parsesql.html) + The `parsesql` module implements a simple high-performance SQL parser. + +* [parseutils](parseutils.html) + Helpers for parsing tokens, numbers, identifiers, etc. + +* [parsexml](parsexml.html) + The `parsexml` module implements a simple high performance XML/HTML parser. + The only encoding that is supported is UTF-8. The parser has been designed + to be somewhat error-correcting, so that even some "wild HTML" found on the + web can be parsed with it. + +* [pegs](pegs.html) + Procedures and operators for handling PEGs. + + +Docutils +-------- + +* [packages/docutils/highlite](highlite.html) + Source highlighter for programming or markup languages. Currently, + only a few languages are supported, other languages may be added. + The interface supports one language nested in another. + +* [packages/docutils/rst](rst.html) + A reStructuredText parser. A large subset + is implemented. Some features of the markdown wiki syntax are also supported. + +* [packages/docutils/rstast](rstast.html) + An AST for the reStructuredText parser. + +* [packages/docutils/rstgen](rstgen.html) + A generator of HTML/Latex from reStructuredText. + + +XML Processing +-------------- + +* [xmltree](xmltree.html) + A simple XML tree. More efficient and simpler than the DOM. It also + contains a macro for XML/HTML code generation. + +* [xmlparser](xmlparser.html) + XML document parser that creates a XML tree representation. + + +Generators +---------- + +* [genasts](genasts.html) + AST generation using captured variables for macros. + +* [htmlgen](htmlgen.html) + A simple XML and HTML code + generator. Each commonly used HTML tag has a corresponding macro + that generates a string with its HTML representation. + + +Hashing +------- + +* [base64](base64.html) + A Base64 encoder and decoder. + +* [hashes](hashes.html) + Efficient computations of hash values for diverse Nim types. + +* [md5](md5.html) + The MD5 checksum algorithm. + +* [oids](oids.html) + An OID is a global ID that consists of a timestamp, + a unique counter, and a random value. This combination should suffice to + produce a globally distributed unique ID. + +* [sha1](sha1.html) + The SHA-1 checksum algorithm. + + +Serialization +------------- + +* [jsonutils](jsonutils.html) + Hookable (de)serialization for arbitrary types + using JSON. + +* [marshal](marshal.html) + Contains procs for serialization and deserialization of arbitrary Nim + data structures. + + +Miscellaneous +------------- + +* [assertions](assertions.html) + Assertion handling. + +* [browsers](browsers.html) + Procs for opening URLs with the user's default + browser. + +* [colors](colors.html) + Color handling. + +* [coro](coro.html) + Experimental coroutines in Nim. + +* [decls](decls.html) + Syntax sugar for some declarations. + +* [enumerate](enumerate.html) + `enumerate` syntactic sugar based on Nim's macro system. + +* [importutils](importutils.html) + Utilities related to import and symbol resolution. + +* [logging](logging.html) + A simple logger. + +* [segfaults](segfaults.html) + Turns access violations or segfaults into a `NilAccessDefect` exception. + +* [sugar](sugar.html) + Nice syntactic sugar based on Nim's macro system. + +* [unittest](unittest.html) + Implements a Unit testing DSL. + +* [varints](varints.html) + Decode variable-length integers that are compatible with SQLite. + +* [with](with.html) + The `with` macro for easy function chaining. + +* [wrapnils](wrapnils.html) + Allows evaluating expressions safely against nil dereferences. + + +Modules for the JavaScript backend +---------------------------------- + +* [asyncjs](asyncjs.html) + Types and macros for writing asynchronous procedures in JavaScript. + +* [dom](dom.html) + Declaration of the Document Object Model for the JS backend. + +* [jsbigints](jsbigints.html) + Arbitrary precision integers. + +* [jsconsole](jsconsole.html) + Wrapper for the `console` object. + +* [jscore](jscore.html) + The wrapper of core JavaScript functions. For most purposes, you should be using + the `math`, `json`, and `times` stdlib modules instead of this module. + +* [jsfetch](jsfetch.html) + Wrapper for `fetch`. + +* [jsffi](jsffi.html) + Types and macros for easier interaction with JavaScript. + +* [jsre](jsre.html) + Regular Expressions for the JavaScript target. + + +Impure libraries +================ + +Regular expressions +------------------- + +* [re](re.html) + Procedures and operators for handling regular + expressions. The current implementation uses PCRE. + +* [nre](nre.html) + + Many help functions for handling regular expressions. + The current implementation uses PCRE. + +Database support +---------------- + +* [db_mysql](db_mysql.html) + A higher level MySQL database wrapper. The same interface is implemented + for other databases too. + +* [db_odbc](db_odbc.html) + A higher level ODBC database wrapper. The same interface is implemented + for other databases too. + +* [db_postgres](db_postgres.html) + A higher level PostgreSQL database wrapper. The same interface is implemented + for other databases too. + +* [db_sqlite](db_sqlite.html) + A higher level SQLite database wrapper. The same interface is implemented + for other databases too. + + +Generic Operating System Services +--------------------------------- + +* [rdstdin](rdstdin.html) + Code for reading user input from stdin. + + +Wrappers +======== + +The generated HTML for some of these wrappers is so huge that it is +not contained in the distribution. You can then find them on the website. + + +Windows-specific +---------------- + +* [winlean](winlean.html) + Wrapper for a small subset of the Win32 API. +* [registry](registry.html) + Windows registry support. + + +UNIX specific +------------- + +* [posix](posix.html) + Wrapper for the POSIX standard. +* [posix_utils](posix_utils.html) + Contains helpers for the POSIX standard or specialized for Linux and BSDs. + + +Regular expressions +------------------- + +* [pcre](pcre.html) + Wrapper for the PCRE library. + + +Database support +---------------- + +* [mysql](mysql.html) + Wrapper for the mySQL API. +* [odbcsql](odbcsql.html) + interface to the ODBC driver. +* [postgres](postgres.html) + Wrapper for the PostgreSQL API. +* [sqlite3](sqlite3.html) + Wrapper for the SQLite 3 API. + + +Network Programming and Internet Protocols +------------------------------------------ + +* [openssl](openssl.html) + Wrapper for OpenSSL. diff --git a/doc/lib.rst b/doc/lib.rst deleted file mode 100644 index 62e02815c..000000000 --- a/doc/lib.rst +++ /dev/null @@ -1,603 +0,0 @@ -.. default-role:: code - -==================== -Nim Standard Library -==================== - -:Author: Andreas Rumpf -:Version: |nimversion| - -.. contents:: - -Nim's library is divided into *pure libraries*, *impure libraries*, and *wrappers*. - -Pure libraries do not depend on any external `*.dll` or `lib*.so` binary -while impure libraries do. A wrapper is an impure library that is a very -low-level interface to a C library. - -Read this `document <apis.html>`_ for a quick overview of the API design. - - -Nimble -====== - -Nim's standard library only covers the basics, check -out `<https://nimble.directory/>`_ for a list of 3rd party packages. - - -Pure libraries -============== - -Automatic imports ------------------ - -* `system <system.html>`_ - Basic procs and operators that every program needs. It also provides IO - facilities for reading and writing text and binary files. It is imported - implicitly by the compiler. Do not import it directly. It relies on compiler - magic to work. - -* `threads <threads.html>`_ - Basic Nim thread support. **Note:** This is part of the system module. Do not - import it explicitly. Enabled with `--threads:on`. - -* `channels <channels_builtin.html>`_ - Nim message passing support for threads. **Note:** This is part of the - system module. Do not import it explicitly. Enabled with `--threads:on`. - - -Core ----- - -* `bitops <bitops.html>`_ - Provides a series of low-level methods for bit manipulation. - -* `cpuinfo <cpuinfo.html>`_ - This module implements procs to determine the number of CPUs / cores. - -* `endians <endians.html>`_ - This module contains helpers that deal with different byte orders. - -* `lenientops <lenientops.html>`_ - Provides binary operators for mixed integer/float expressions for convenience. - -* `locks <locks.html>`_ - Locks and condition variables for Nim. - -* `macrocache <macrocache.html>`_ - Provides an API for macros to collect compile-time information across modules. - -* `macros <macros.html>`_ - Contains the AST API and documentation of Nim for writing macros. - -* `rlocks <rlocks.html>`_ - Reentrant locks for Nim. - -* `typeinfo <typeinfo.html>`_ - Provides (unsafe) access to Nim's run-time type information. - -* `typetraits <typetraits.html>`_ - This module defines compile-time reflection procs for working with types. - -* `volatile <volatile.html>`_ - This module contains code for generating volatile loads and stores, - which are useful in embedded and systems programming. - - -Algorithms ----------- - -* `algorithm <algorithm.html>`_ - This module implements some common generic algorithms like sort or binary search. - -* `std/enumutils <enumutils.html>`_ - This module adds functionality for the built-in `enum` type. - -* `sequtils <sequtils.html>`_ - This module implements operations for the built-in `seq` type - which were inspired by functional programming languages. - -* `std/setutils <setutils.html>`_ - This module adds functionality for the built-in `set` type. - - -Collections ------------ - -* `critbits <critbits.html>`_ - This module implements a *crit bit tree* which is an efficient - container for a sorted set of strings, or a sorted mapping of strings. - -* `deques <deques.html>`_ - Implementation of a double-ended queue. - The underlying implementation uses a `seq`. - -* `heapqueue <heapqueue.html>`_ - Implementation of a heap data structure that can be used as a priority queue. - -* `intsets <intsets.html>`_ - Efficient implementation of a set of ints as a sparse bit set. - -* `lists <lists.html>`_ - Nim linked list support. Contains singly and doubly linked lists and - circular lists ("rings"). - -* `options <options.html>`_ - The option type encapsulates an optional value. - -* `std/packedsets <packedsets.html>`_ - Efficient implementation of a set of ordinals as a sparse bit set. - -* `sets <sets.html>`_ - Nim hash and bit set support. - -* `sharedlist <sharedlist.html>`_ - Nim shared linked list support. Contains a shared singly-linked list. - -* `sharedtables <sharedtables.html>`_ - Nim shared hash table support. Contains shared tables. - -* `tables <tables.html>`_ - Nim hash table support. Contains tables, ordered tables, and count tables. - - -String handling ---------------- - -* `cstrutils <cstrutils.html>`_ - Utilities for `cstring` handling. - -* `std/editdistance <editdistance.html>`_ - This module contains an algorithm to compute the edit distance between two - Unicode strings. - -* `encodings <encodings.html>`_ - Converts between different character encodings. On UNIX, this uses - the `iconv` library, on Windows the Windows API. - -* `parseutils <parseutils.html>`_ - This module contains helpers for parsing tokens, numbers, identifiers, etc. - -* `pegs <pegs.html>`_ - This module contains procedures and operators for handling PEGs. - -* `punycode <punycode.html>`_ - Implements a representation of Unicode with the limited ASCII character subset. - -* `ropes <ropes.html>`_ - This module contains support for a *rope* data type. - Ropes can represent very long strings efficiently; - especially concatenation is done in O(1) instead of O(n). - -* `strformat <strformat.html>`_ - Macro based standard string interpolation/formatting. Inspired by - Python's `f`-strings. - -* `strmisc <strmisc.html>`_ - This module contains uncommon string handling operations that do not - fit with the commonly used operations in strutils. - -* `strscans <strscans.html>`_ - This module contains a `scanf` macro for convenient parsing of mini languages. - -* `strtabs <strtabs.html>`_ - The `strtabs` module implements an efficient hash table that is a mapping - from strings to strings. Supports a case-sensitive, case-insensitive and - style-insensitive modes. - -* `strutils <strutils.html>`_ - This module contains common string handling operations like changing - case of a string, splitting a string into substrings, searching for - substrings, replacing substrings. - -* `unicode <unicode.html>`_ - This module provides support to handle the Unicode UTF-8 encoding. - -* `unidecode <unidecode.html>`_ - It provides a single proc that does Unicode to ASCII transliterations. - Based on Python's Unidecode module. - -* `std/wordwrap <wordwrap.html>`_ - This module contains an algorithm to wordwrap a Unicode string. - - -Time handling -------------- - -* `std/monotimes <monotimes.html>`_ - The `monotimes` module implements monotonic timestamps. - -* `times <times.html>`_ - The `times` module contains support for working with time. - - -Generic Operating System Services ---------------------------------- - -* `distros <distros.html>`_ - This module implements the basics for OS distribution ("distro") detection - and the OS's native package manager. - Its primary purpose is to produce output for Nimble packages, - but it also contains the widely used **Distribution** enum - that is useful for writing platform-specific code. - See `packaging <packaging.html>`_ for hints on distributing Nim using OS packages. - -* `dynlib <dynlib.html>`_ - This module implements the ability to access symbols from shared libraries. - -* `marshal <marshal.html>`_ - Contains procs for serialization and deserialization of arbitrary Nim - data structures. - -* `memfiles <memfiles.html>`_ - This module provides support for memory-mapped files (Posix's `mmap`) - on the different operating systems. - -* `os <os.html>`_ - Basic operating system facilities like retrieving environment variables, - reading command line arguments, working with directories, running shell - commands, etc. - -* `osproc <osproc.html>`_ - Module for process communication beyond `os.execShellCmd`. - -* `streams <streams.html>`_ - This module provides a stream interface and two implementations thereof: - the `FileStream` and the `StringStream` which implement the stream - interface for Nim file objects (`File`) and strings. Other modules - may provide other implementations for this standard stream interface. - -* `terminal <terminal.html>`_ - This module contains a few procedures to control the *terminal* - (also called *console*). The implementation simply uses ANSI escape - sequences and does not depend on any other module. - - -Math libraries --------------- - -* `complex <complex.html>`_ - This module implements complex numbers and relevant mathematical operations. - -* `fenv <fenv.html>`_ - Floating-point environment. Handling of floating-point rounding and - exceptions (overflow, zero-divide, etc.). - -* `math <math.html>`_ - Mathematical operations like cosine, square root. - -* `mersenne <mersenne.html>`_ - Mersenne twister random number generator. - -* `random <random.html>`_ - Fast and tiny random number generator. - -* `std/sysrand <sysrand.html>`_ - Cryptographically secure pseudorandom number generator. - -* `rationals <rationals.html>`_ - This module implements rational numbers and relevant mathematical operations. - -* `stats <stats.html>`_ - Statistical analysis - -* `std/sums <sums.html>`_ - Accurate summation functions. - - -Internet Protocols and Support ------------------------------- - -* `asyncdispatch <asyncdispatch.html>`_ - This module implements an asynchronous dispatcher for IO operations. - -* `asyncfile <asyncfile.html>`_ - This module implements asynchronous file reading and writing using - `asyncdispatch`. - -* `asyncftpclient <asyncftpclient.html>`_ - This module implements an asynchronous FTP client using the `asyncnet` - module. - -* `asynchttpserver <asynchttpserver.html>`_ - This module implements an asynchronous HTTP server using the `asyncnet` - module. - -* `asyncnet <asyncnet.html>`_ - This module implements asynchronous sockets based on the `asyncdispatch` - module. - -* `asyncstreams <asyncstreams.html>`_ - This module provides `FutureStream` - a future that acts as a queue. - -* `cgi <cgi.html>`_ - This module implements helpers for CGI applications. - -* `cookies <cookies.html>`_ - This module contains helper procs for parsing and generating cookies. - -* `httpclient <httpclient.html>`_ - This module implements a simple HTTP client which supports both synchronous - and asynchronous retrieval of web pages. - -* `mimetypes <mimetypes.html>`_ - This module implements a mimetypes database. - -* `nativesockets <nativesockets.html>`_ - This module implements a low-level sockets API. - -* `net <net.html>`_ - This module implements a high-level sockets API. It replaces the - `sockets` module. - -* `selectors <selectors.html>`_ - This module implements a selector API with backends specific to each OS. - Currently, epoll on Linux and select on other operating systems. - -* `smtp <smtp.html>`_ - This module implements a simple SMTP client. - -* `uri <uri.html>`_ - This module provides functions for working with URIs. - - -Threading ---------- - -* `threadpool <threadpool.html>`_ - Implements Nim's `spawn <manual_experimental.html#parallel-amp-spawn>`_. - - -Parsers -------- - -* `htmlparser <htmlparser.html>`_ - This module parses an HTML document and creates its XML tree representation. - -* `json <json.html>`_ - High-performance JSON parser. - -* `std/jsonutils <jsonutils.html>`_ - This module implements a hookable (de)serialization for arbitrary types. - -* `lexbase <lexbase.html>`_ - This is a low-level module that implements an extremely efficient buffering - scheme for lexers and parsers. This is used by the diverse parsing modules. - -* `parsecfg <parsecfg.html>`_ - The `parsecfg` module implements a high-performance configuration file - parser. The configuration file's syntax is similar to the Windows `.ini` - format, but much more powerful, as it is not a line based parser. String - literals, raw string literals, and triple quote string literals are supported - as in the Nim programming language. - -* `parsecsv <parsecsv.html>`_ - The `parsecsv` module implements a simple high-performance CSV parser. - -* `parsejson <parsejson.html>`_ - This module implements a JSON parser. It is used and exported by the `json <json.html>`_ module, but can also be used in its own right. - -* `parseopt <parseopt.html>`_ - The `parseopt` module implements a command line option parser. - -* `parsesql <parsesql.html>`_ - The `parsesql` module implements a simple high-performance SQL parser. - -* `parsexml <parsexml.html>`_ - The `parsexml` module implements a simple high performance XML/HTML parser. - The only encoding that is supported is UTF-8. The parser has been designed - to be somewhat error-correcting, so that even some "wild HTML" found on the - web can be parsed with it. - - -Docutils --------- - -* `packages/docutils/highlite <highlite.html>`_ - Source highlighter for programming or markup languages. Currently, - only a few languages are supported, other languages may be added. - The interface supports one language nested in another. - -* `packages/docutils/rst <rst.html>`_ - This module implements a reStructuredText parser. A large subset - is implemented. Some features of the markdown wiki syntax are also supported. - -* `packages/docutils/rstast <rstast.html>`_ - This module implements an AST for the reStructuredText parser. - -* `packages/docutils/rstgen <rstgen.html>`_ - This module implements a generator of HTML/Latex from reStructuredText. - - -XML Processing --------------- - -* `xmltree <xmltree.html>`_ - A simple XML tree. More efficient and simpler than the DOM. It also - contains a macro for XML/HTML code generation. - -* `xmlparser <xmlparser.html>`_ - This module parses an XML document and creates its XML tree representation. - - -Generators ----------- - -* `htmlgen <htmlgen.html>`_ - This module implements a simple XML and HTML code - generator. Each commonly used HTML tag has a corresponding macro - that generates a string with its HTML representation. - - -Hashing -------- - -* `base64 <base64.html>`_ - This module implements a base64 encoder and decoder. - -* `hashes <hashes.html>`_ - This module implements efficient computations of hash values for diverse - Nim types. - -* `md5 <md5.html>`_ - This module implements the MD5 checksum algorithm. - -* `oids <oids.html>`_ - An OID is a global ID that consists of a timestamp, - a unique counter, and a random value. This combination should suffice to - produce a globally distributed unique ID. This implementation was extracted - from the Mongodb interface and it thus binary compatible with a Mongo OID. - -* `std/sha1 <sha1.html>`_ - This module implements a sha1 encoder and decoder. - - -Miscellaneous -------------- - -* `browsers <browsers.html>`_ - This module implements procs for opening URLs with the user's default - browser. - -* `colors <colors.html>`_ - This module implements color handling for Nim. - -* `coro <coro.html>`_ - This module implements experimental coroutines in Nim. - -* `std/enumerate <enumerate.html>`_ - This module implements `enumerate` syntactic sugar based on Nim's macro system. - -* `logging <logging.html>`_ - This module implements a simple logger. - -* `segfaults <segfaults.html>`_ - Turns access violations or segfaults into a `NilAccessDefect` exception. - -* `sugar <sugar.html>`_ - This module implements nice syntactic sugar based on Nim's macro system. - -* `unittest <unittest.html>`_ - Implements a Unit testing DSL. - -* `std/varints <varints.html>`_ - Decode variable-length integers that are compatible with SQLite. - -* `std/with <with.html>`_ - This module implements the `with` macro for easy function chaining. - - - -Modules for JS backend ----------------------- - -* `asyncjs <asyncjs.html>`_ - Types and macros for writing asynchronous procedures in JavaScript. - -* `dom <dom.html>`_ - Declaration of the Document Object Model for the JS backend. - -* `jsconsole <jsconsole.html>`_ - Wrapper for the `console` object. - -* `jscore <jscore.html>`_ - The wrapper of core JavaScript functions. For most purposes, you should be using - the `math`, `json`, and `times` stdlib modules instead of this module. - -* `jsffi <jsffi.html>`_ - Types and macros for easier interaction with JavaScript. - - -Impure libraries -================ - -Regular expressions -------------------- - -* `re <re.html>`_ - This module contains procedures and operators for handling regular - expressions. The current implementation uses PCRE. - - - -Database support ----------------- - -* `db_postgres <db_postgres.html>`_ - A higher level PostgreSQL database wrapper. The same interface is implemented - for other databases too. - -* `db_mysql <db_mysql.html>`_ - A higher level MySQL database wrapper. The same interface is implemented - for other databases too. - -* `db_sqlite <db_sqlite.html>`_ - A higher level SQLite database wrapper. The same interface is implemented - for other databases too. - - -Generic Operating System Services ---------------------------------- - -* `rdstdin <rdstdin.html>`_ - This module contains code for reading from stdin. - - -Wrappers -======== - -The generated HTML for some of these wrappers is so huge that it is -not contained in the distribution. You can then find them on the website. - - -Windows-specific ----------------- - -* `winlean <winlean.html>`_ - Contains a wrapper for a small subset of the Win32 API. -* `registry <registry.html>`_ - Windows registry support. - - -UNIX specific -------------- - -* `posix <posix.html>`_ - Contains a wrapper for the POSIX standard. -* `posix_utils <posix_utils.html>`_ - Contains helpers for the POSIX standard or specialized for Linux and BSDs. - - -Regular expressions -------------------- - -* `pcre <pcre.html>`_ - Wrapper for the PCRE library. - - -Database support ----------------- - -* `postgres <postgres.html>`_ - Contains a wrapper for the PostgreSQL API. -* `mysql <mysql.html>`_ - Contains a wrapper for the mySQL API. -* `sqlite3 <sqlite3.html>`_ - Contains a wrapper for SQLite 3 API. -* `odbcsql <odbcsql.html>`_ - interface to the ODBC driver. - - -Network Programming and Internet Protocols ------------------------------------------- - -* `openssl <openssl.html>`_ - Wrapper for OpenSSL. - - - -Unstable -======== - -* `atomics <atomics.html>`_ - Types and operations for atomic operations and lockless algorithms. diff --git a/doc/manual.rst b/doc/manual.md index 1bb47f28b..5c36a0a7b 100644 --- a/doc/manual.rst +++ b/doc/manual.md @@ -1,5 +1,3 @@ -.. default-role:: code - ========== Nim Manual ========== @@ -7,12 +5,14 @@ Nim Manual :Authors: Andreas Rumpf, Zahary Karadjov :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: - "Complexity" seems to be a lot like "energy": you can transfer it from the - end-user to one/some of the other players, but the total amount seems to remain - pretty much constant for a given task. -- Ran +> "Complexity" seems to be a lot like "energy": you can transfer it from the +> end-user to one/some of the other players, but the total amount seems to remain +> pretty much constant for a given task. -- Ran About this document @@ -22,16 +22,16 @@ About this document precise wording. This manual is constantly evolving into a proper specification. **Note**: The experimental features of Nim are -covered `here <manual_experimental.html>`_. +covered [here](manual_experimental.html). **Note**: Assignments, moves, and destruction are specified in -the `destructors <destructors.html>`_ document. +the [destructors](destructors.html) document. This document describes the lexis, the syntax, and the semantics of the Nim language. To learn how to compile Nim programs and generate documentation see -`Compiler User Guide <nimc.html>`_ and `DocGen Tools Guide <docgen.html>`_. +the [Compiler User Guide](nimc.html) and the [DocGen Tools Guide](docgen.html). The language constructs are explained using an extended BNF, in which `(a)*` means 0 or more `a`'s, `a+` means 1 or more `a`'s, and `(a)?` means an @@ -47,16 +47,16 @@ is not ambiguous. Non-terminals start with a lowercase letter, abstract terminal symbols are in UPPERCASE. Verbatim terminal symbols (including keywords) are quoted -with `'`. An example:: +with `'`. An example: - ifStmt = 'if' expr ':' stmts ('elif' expr ':' stmts)* ('else' stmts)? + ifStmt = 'if' expr ':' stmts ('elif' expr ':' stmts)* ('else' stmts)? The binary `^*` operator is used as a shorthand for 0 or more occurrences separated by its second argument; likewise `^+` means 1 or more occurrences: `a ^+ b` is short for `a (b a)*` -and `a ^* b` is short for `(a (b a)*)?`. Example:: +and `a ^* b` is short for `(a (b a)*)?`. Example: - arrayConstructor = '[' expr ^* ',' ']' + arrayConstructor = '[' expr ^* ',' ']' Other parts of Nim, like scoping rules or runtime semantics, are described informally. @@ -91,11 +91,11 @@ The nature of this executable depends on the compiler implementation; it may, for example, be a native binary or JavaScript source code. In a typical Nim program, most of the code is compiled into the executable. -However, some of the code may be executed at +However, some code may be executed at `compile-time`:idx:. This can include constant expressions, macro definitions, and Nim procedures used by macro definitions. Most of the Nim language is -supported at compile-time, but there are some restrictions -- see `Restrictions -on Compile-Time Execution <#restrictions-on-compileminustime-execution>`_ for +supported at compile-time, but there are some restrictions -- see [Restrictions +on Compile-Time Execution] for details. We use the term `runtime`:idx: to cover both compile-time execution and code execution in the executable. @@ -111,25 +111,26 @@ A `panic`:idx: is an error that the implementation detects and reports at runtime. The method for reporting such errors is via *raising exceptions* or *dying with a fatal error*. However, the implementation provides a means to disable these `runtime checks`:idx:. See the section -pragmas_ for details. +[Pragmas] for details. Whether a panic results in an exception or in a fatal error is -implementation specific. Thus the following program is invalid; even though the +implementation specific. Thus, the following program is invalid; even though the code purports to catch the `IndexDefect` from an out-of-bounds array access, the compiler may instead choose to allow the program to die with a fatal error. -.. code-block:: nim + ```nim var a: array[0..1, char] let i = 5 try: a[i] = 'N' except IndexDefect: echo "invalid index" + ``` -The current implementation allows to switch between these different behaviors -via `--panics:on|off`. When panics are turned on, the program dies with a +The current implementation allows switching between these different behaviors +via `--panics:on|off`:option:. When panics are turned on, the program dies with a panic, if they are turned off the runtime errors are turned into -exceptions. The benefit of `--panics:on` is that it produces smaller binary +exceptions. The benefit of `--panics:on`:option: is that it produces smaller binary code and the compiler has more freedom to optimize the code. An `unchecked runtime error`:idx: is an error that is not guaranteed to be @@ -189,16 +190,16 @@ is another pseudo terminal that describes the *action* of popping a value from the stack, `IND{>}` then implies to push onto the stack. With this notation we can now easily define the core of the grammar: A block of -statements (simplified example):: +statements (simplified example): - ifStmt = 'if' expr ':' stmt - (IND{=} 'elif' expr ':' stmt)* - (IND{=} 'else' ':' stmt)? + ifStmt = 'if' expr ':' stmt + (IND{=} 'elif' expr ':' stmt)* + (IND{=} 'else' ':' stmt)? - simpleStmt = ifStmt / ... + simpleStmt = ifStmt / ... - stmt = IND{>} stmt ^+ IND{=} DED # list of statements - / simpleStmt # or a simple statement + stmt = IND{>} stmt ^+ IND{=} DED # list of statements + / simpleStmt # or a simple statement @@ -214,15 +215,16 @@ no other tokens between it and the preceding one, it does not start a new comment: -.. code-block:: nim + ```nim i = 0 # This is a single comment over multiple lines. - # The scanner merges these two pieces. + # The lexer merges these two pieces. # The comment continues here. + ``` `Documentation comments`:idx: are comments that start with two `##`. Documentation comments are tokens; they are only allowed at certain places in -the input file as they belong to the syntax tree! +the input file as they belong to the syntax tree. Multiline comments @@ -231,26 +233,41 @@ Multiline comments Starting with version 0.13.0 of the language Nim supports multiline comments. They look like: -.. code-block:: nim + ```nim #[Comment here. Multiple lines are not a problem.]# + ``` Multiline comments support nesting: -.. code-block:: nim + ```nim #[ #[ Multiline comment in already commented out code. ]# proc p[T](x: T) = discard ]# + ``` Multiline documentation comments also exist and support nesting too: -.. code-block:: nim + ```nim proc foo = ##[Long documentation comment here. ]## + ``` + +You can also use the [discard statement](#statements-and-expressions-discard-statement) together with +[triple quoted string literals](#lexical-analysis-triple-quoted-string-literals) to create multiline comments: + + ```nim + discard """ You can have any Nim code text commented + out inside this with no indentation restrictions. + yes("May I ask a pointless question?") """ + ``` + +This was how multiline comments were done before version 0.13.0, +and it is used to provide specifications to [testament](testament.html#writing-unit-tests) test framework. Identifiers & Keywords @@ -261,11 +278,13 @@ and underscores, with the following restrictions: * begins with a letter * does not end with an underscore `_` -* two immediate following underscores `__` are not allowed:: +* two immediate following underscores `__` are not allowed: + ``` letter ::= 'A'..'Z' | 'a'..'z' | '\x80'..'\xff' digit ::= '0'..'9' IDENTIFIER ::= letter ( ['_'] (letter | digit) )* + ``` Currently, any Unicode character with an ordinal value > 127 (non-ASCII) is classified as a `letter` and may thus be part of an identifier but later @@ -274,8 +293,8 @@ operator characters instead. The following keywords are reserved and cannot be used as identifiers: -.. code-block:: nim - :file: keywords.txt + ```nim file="keywords.txt" + ``` Some keywords are unused; they are reserved for future developments of the language. @@ -286,10 +305,11 @@ Identifier equality Two identifiers are considered equal if the following algorithm returns true: -.. code-block:: nim + ```nim proc sameIdentifier(a, b: string): bool = a[0] == b[0] and a.replace("_", "").toLowerAscii == b.replace("_", "").toLowerAscii + ``` That means only the first letters are compared in a case-sensitive manner. Other letters are compared case-insensitively within the ASCII range and underscores are ignored. @@ -315,23 +335,24 @@ it was not case-sensitive and underscores were ignored and there was not even a distinction between `foo` and `Foo`. -Stropping ---------- +Keywords as identifiers +----------------------- If a keyword is enclosed in backticks it loses its keyword property and becomes an ordinary identifier. Examples -.. code-block:: nim + ```nim var `var` = "Hello Stropping" + ``` -.. code-block:: nim - type Type = object - `int`: int + ```nim + type Obj = object + `type`: int - let `object` = Type(`int`: 9) - assert `object` is Type - assert `object`.`int` == 9 + let `object` = Obj(`type`: 9) + assert `object` is Obj + assert `object`.`type` == 9 var `var` = 42 let `let` = 8 @@ -339,6 +360,7 @@ Examples const `assert` = true assert `assert` + ``` String literals @@ -362,7 +384,7 @@ contain the following `escape sequences`:idx:\ : ``\\`` `backslash`:idx: ``\"`` `quotation mark`:idx: ``\'`` `apostrophe`:idx: - ``\\`` '0'..'9'+ `character with decimal value d`:idx:; + ``\`` '0'..'9'+ `character with decimal value d`:idx:; all decimal digits directly following are used for the character ``\a`` `alert`:idx: @@ -378,7 +400,7 @@ contain the following `escape sequences`:idx:\ : ================== =================================================== -Strings in Nim may contain any 8-bit value, even embedded zeros. However +Strings in Nim may contain any 8-bit value, even embedded zeros. However, some operations may interpret the first binary zero as a terminator. @@ -395,12 +417,13 @@ be whitespace between the opening `"""` and the newline), the newline (and the preceding whitespace) is not included in the string. The ending of the string literal is defined by the pattern `"""[^"]`, so this: -.. code-block:: nim + ```nim """"long string within quotes"""" + ``` -Produces:: +Produces: - "long string within quotes" + "long string within quotes" Raw string literals @@ -413,19 +436,19 @@ letter `r` (or `R`) and are delimited by matching double quotes (just like ordinary string literals) and do not interpret the escape sequences. This is especially convenient for regular expressions or Windows paths: -.. code-block:: nim - + ```nim var f = openFile(r"C:\texts\text.txt") # a raw string, so ``\t`` is no tab + ``` To produce a single `"` within a raw string literal, it has to be doubled: -.. code-block:: nim - + ```nim r"a""b" + ``` -Produces:: +Produces: - a"b + a"b `r""""` is not possible with this notation, because the three leading quotes introduce a triple quoted string literal. `r"""` is the same @@ -442,7 +465,7 @@ Terminal symbols in the grammar: `GENERALIZED_STR_LIT`, The construct `identifier"string literal"` (without whitespace between the identifier and the opening quotation mark) is a generalized raw string literal. It is a shortcut for the construct -`identifier(r"string literal")`, so it denotes a procedure call with a +`identifier(r"string literal")`, so it denotes a routine call with a raw string literal as its only argument. Generalized raw string literals are especially convenient for embedding mini languages directly into Nim (for example regular expressions). @@ -457,8 +480,8 @@ Character literals Character literals are enclosed in single quotes `''` and can contain the same escape sequences as strings - with one exception: the platform dependent `newline`:idx: (``\p``) -is not allowed as it may be wider than one character (often it is the pair -CR/LF for example). Here are the valid `escape sequences`:idx: for character +is not allowed as it may be wider than one character (it can be the pair +CR/LF). Here are the valid `escape sequences`:idx: for character literals: ================== =================================================== @@ -472,7 +495,7 @@ literals: ``\\`` `backslash`:idx: ``\"`` `quotation mark`:idx: ``\'`` `apostrophe`:idx: - ``\\`` '0'..'9'+ `character with decimal value d`:idx:; + ``\`` '0'..'9'+ `character with decimal value d`:idx:; all decimal digits directly following are used for the character ``\a`` `alert`:idx: @@ -482,71 +505,117 @@ literals: exactly two hex digits are allowed ================== =================================================== -A character is not a Unicode character but a single byte. The reason for this -is efficiency: for the overwhelming majority of use-cases, the resulting -programs will still handle UTF-8 properly as UTF-8 was specially designed for -this. Another reason is that Nim can thus support `array[char, int]` or -`set[char]` efficiently as many algorithms rely on this feature. The `Rune` -type is used for Unicode characters, it can represent any Unicode character. -`Rune` is declared in the `unicode module <unicode.html>`_. +A character is not a Unicode character but a single byte. +Rationale: It enables the efficient support of `array[char, int]` or +`set[char]`. + +The `Rune` type can represent any Unicode character. +`Rune` is declared in the [unicode module](unicode.html). + +A character literal that does not end in `'` is interpreted as `'` if there +is a preceding backtick token. There must be no whitespace between the preceding +backtick token and the character literal. This special case ensures that a declaration +like ``proc `'customLiteral`(s: string)`` is valid. ``proc `'customLiteral`(s: string)`` +is the same as ``proc `'\''customLiteral`(s: string)``. + +See also [custom numeric literals]. -Numerical constants -------------------- -Numerical constants are of a single type and have the form:: - - hexdigit = digit | 'A'..'F' | 'a'..'f' - octdigit = '0'..'7' - bindigit = '0'..'1' - HEX_LIT = '0' ('x' | 'X' ) hexdigit ( ['_'] hexdigit )* - DEC_LIT = digit ( ['_'] digit )* - OCT_LIT = '0' 'o' octdigit ( ['_'] octdigit )* - BIN_LIT = '0' ('b' | 'B' ) bindigit ( ['_'] bindigit )* - - INT_LIT = HEX_LIT - | DEC_LIT - | OCT_LIT - | BIN_LIT - - INT8_LIT = INT_LIT ['\''] ('i' | 'I') '8' - INT16_LIT = INT_LIT ['\''] ('i' | 'I') '16' - INT32_LIT = INT_LIT ['\''] ('i' | 'I') '32' - INT64_LIT = INT_LIT ['\''] ('i' | 'I') '64' - - UINT_LIT = INT_LIT ['\''] ('u' | 'U') - UINT8_LIT = INT_LIT ['\''] ('u' | 'U') '8' - UINT16_LIT = INT_LIT ['\''] ('u' | 'U') '16' - UINT32_LIT = INT_LIT ['\''] ('u' | 'U') '32' - UINT64_LIT = INT_LIT ['\''] ('u' | 'U') '64' - - exponent = ('e' | 'E' ) ['+' | '-'] digit ( ['_'] digit )* - FLOAT_LIT = digit (['_'] digit)* (('.' digit (['_'] digit)* [exponent]) |exponent) - FLOAT32_SUFFIX = ('f' | 'F') ['32'] - FLOAT32_LIT = HEX_LIT '\'' FLOAT32_SUFFIX - | (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT32_SUFFIX - FLOAT64_SUFFIX = ( ('f' | 'F') '64' ) | 'd' | 'D' - FLOAT64_LIT = HEX_LIT '\'' FLOAT64_SUFFIX - | (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT64_SUFFIX - - -As can be seen in the productions, numerical constants can contain underscores +Numeric literals +---------------- + +Numeric literals have the form: + + hexdigit = digit | 'A'..'F' | 'a'..'f' + octdigit = '0'..'7' + bindigit = '0'..'1' + unary_minus = '-' # See the section about unary minus + HEX_LIT = unary_minus? '0' ('x' | 'X' ) hexdigit ( ['_'] hexdigit )* + DEC_LIT = unary_minus? digit ( ['_'] digit )* + OCT_LIT = unary_minus? '0' 'o' octdigit ( ['_'] octdigit )* + BIN_LIT = unary_minus? '0' ('b' | 'B' ) bindigit ( ['_'] bindigit )* + + INT_LIT = HEX_LIT + | DEC_LIT + | OCT_LIT + | BIN_LIT + + INT8_LIT = INT_LIT ['\''] ('i' | 'I') '8' + INT16_LIT = INT_LIT ['\''] ('i' | 'I') '16' + INT32_LIT = INT_LIT ['\''] ('i' | 'I') '32' + INT64_LIT = INT_LIT ['\''] ('i' | 'I') '64' + + UINT_LIT = INT_LIT ['\''] ('u' | 'U') + UINT8_LIT = INT_LIT ['\''] ('u' | 'U') '8' + UINT16_LIT = INT_LIT ['\''] ('u' | 'U') '16' + UINT32_LIT = INT_LIT ['\''] ('u' | 'U') '32' + UINT64_LIT = INT_LIT ['\''] ('u' | 'U') '64' + + exponent = ('e' | 'E' ) ['+' | '-'] digit ( ['_'] digit )* + FLOAT_LIT = unary_minus? digit (['_'] digit)* (('.' digit (['_'] digit)* [exponent]) |exponent) + FLOAT32_SUFFIX = ('f' | 'F') ['32'] + FLOAT32_LIT = HEX_LIT '\'' FLOAT32_SUFFIX + | (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT32_SUFFIX + FLOAT64_SUFFIX = ( ('f' | 'F') '64' ) | 'd' | 'D' + FLOAT64_LIT = HEX_LIT '\'' FLOAT64_SUFFIX + | (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT64_SUFFIX + + CUSTOM_NUMERIC_LIT = (FLOAT_LIT | INT_LIT) '\'' CUSTOM_NUMERIC_SUFFIX + + # CUSTOM_NUMERIC_SUFFIX is any Nim identifier that is not + # a pre-defined type suffix. + + +As can be seen in the productions, numeric literals can contain underscores for readability. Integer and floating-point literals may be given in decimal (no prefix), binary (prefix `0b`), octal (prefix `0o`), and hexadecimal (prefix `0x`) notation. -There exists a literal for each numerical type that is -defined. The suffix starting with an apostrophe ('\'') is called a +The fact that the unary minus `-` in a number literal like `-1` is considered +to be part of the literal is a late addition to the language. The rationale is that +an expression `-128'i8` should be valid and without this special case, this would +be impossible -- `128` is not a valid `int8` value, only `-128` is. + +For the `unary_minus` rule there are further restrictions that are not covered +in the formal grammar. For `-` to be part of the number literal the immediately +preceding character has to be in the +set `{' ', '\t', '\n', '\r', ',', ';', '(', '[', '{'}`. This set was designed to +cover most cases in a natural manner. + +In the following examples, `-1` is a single token: + + ```nim + echo -1 + echo(-1) + echo [-1] + echo 3,-1 + + "abc";-1 + ``` + +In the following examples, `-1` is parsed as two separate tokens +(as `-`:tok: `1`:tok:): + + ```nim + echo x-1 + echo (int)-1 + echo [a]-1 + "abc"-1 + ``` + + +The suffix starting with an apostrophe ('\'') is called a `type suffix`:idx:. Literals without a type suffix are of an integer type unless the literal contains a dot or `E|e` in which case it is of type `float`. This integer type is `int` if the literal is in the range -`low(i32)..high(i32)`, otherwise it is `int64`. +`low(int32)..high(int32)`, otherwise it is `int64`. For notational convenience, the apostrophe of a type suffix is optional if it is not ambiguous (only hexadecimal floating-point literals with a type suffix can be ambiguous). -The type suffixes are: +The pre-defined type suffixes are: ================= ========================= Type Suffix Resulting type of literal @@ -571,18 +640,53 @@ notation: `0B0_10001110100_0000101001000111101011101111111011000101001101001001'f64` is approximately 1.72826e35 according to the IEEE floating-point standard. -Literals are bounds checked so that they fit the datatype. Non-base-10 -literals are used mainly for flags and bit pattern representations, therefore -bounds checking is done on bit width, not value range. If the literal fits in -the bit width of the datatype, it is accepted. +Literals must match the datatype, for example, `333'i8` is an invalid literal. +Non-base-10 literals are used mainly for flags and bit pattern representations, +therefore the checking is done on bit width and not on value range. Hence: 0b10000000'u8 == 0x80'u8 == 128, but, 0b10000000'i8 == 0x80'i8 == -1 instead of causing an overflow error. + +### Custom numeric literals + +If the suffix is not predefined, then the suffix is assumed to be a call +to a proc, template, macro or other callable identifier that is passed the +string containing the literal. The callable identifier needs to be declared +with a special ``'`` prefix: + + ```nim + import std/strutils + type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble" + proc `'u4`(n: string): u4 = + # The leading ' is required. + result = (parseInt(n) and 0x0F).u4 + + var x = 5'u4 + ``` + +More formally, a custom numeric literal `123'custom` is transformed +to r"123".`'custom` in the parsing step. There is no AST node kind that +corresponds to this transformation. The transformation naturally handles +the case that additional parameters are passed to the callee: + + ```nim + import std/strutils + type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble" + proc `'u4`(n: string; moreData: int): u4 = + result = (parseInt(n) and 0x0F).u4 + + var x = 5'u4(123) + ``` + +Custom numeric literals are covered by the grammar rule named `CUSTOM_NUMERIC_LIT`. +A custom numeric literal is a single token. + + Operators --------- Nim allows user defined operators. An operator is any combination of the -following characters:: +following characters: = + - * / < > @ $ ~ & % | @@ -594,7 +698,7 @@ defined here.) These keywords are also operators: `and or not xor shl shr div mod in notin is isnot of as from`. -`.`:tok: `=`:tok:, `:`:tok:, `::`:tok: are not available as general operators; they +`.`:tok:, `=`:tok:, `:`:tok:, `::`:tok: are not available as general operators; they are used for other notational purposes. `*:` is as a special case treated as the two tokens `*`:tok: and `:`:tok: @@ -603,26 +707,40 @@ are used for other notational purposes. The `not` keyword is always a unary operator, `a not b` is parsed as `a(not b)`, not as `(a) not (b)`. +Unicode Operators +----------------- + +These Unicode operators are also parsed as operators: + + ∙ ∘ × ★ ⊗ ⊘ ⊙ ⊛ ⊠⊡ ∩ ∧ ⊓ # same priority as * (multiplication) + ± ⊕ ⊖ ⊞ ⊟ ∪ ∨ ⊔ # same priority as + (addition) + + +Unicode operators can be combined with non-Unicode operator +symbols. The usual precedence extensions then apply, for example, `⊠=` is an +assignment like operator just like `*=` is. + +No Unicode normalization step is performed. + Other tokens ------------ -The following strings denote other tokens:: +The following strings denote other tokens: ` ( ) { } [ ] , ; [. .] {. .} (. .) [: The `slice`:idx: operator `..`:tok: takes precedence over other tokens that -contain a dot: `{..}`:tok: are the three tokens `{`:tok:, `..`:tok:, `}`:tok: +contain a dot: `{..}` are the three tokens `{`:tok:, `..`:tok:, `}`:tok: and not the two tokens `{.`:tok:, `.}`:tok:. - Syntax ====== This section lists Nim's standard syntax. How the parser handles -the indentation is already described in the `Lexical Analysis`_ section. +the indentation is already described in the [Lexical Analysis] section. Nim allows user-definable operators. Binary operators have 11 different levels of precedence. @@ -635,12 +753,13 @@ Associativity Binary operators whose first character is `^` are right-associative, all other binary operators are left-associative. -.. code-block:: nim + ```nim proc `^/`(x, y: float): float = # a right-associative division operator result = x / y echo 12 ^/ 4 ^/ 8 # 24.0 (4 / 8 = 0.5, then 12 / 0.5 = 24.0) echo 12 / 4 / 8 # 0.375 (12 / 4 = 3.0, then 3 / 8 = 0.375) + ``` Precedence ---------- @@ -665,11 +784,12 @@ has the second-lowest precedence. Otherwise, precedence is determined by the first character. + ================ ======================================================= ================== =============== Precedence level Operators First character Terminal symbol ================ ======================================================= ================== =============== 10 (highest) `$ ^` OP10 - 9 `* / div mod shl shr %` ``* % \ /`` OP9 + 9 `* / div mod shl shr %` `* % \ /` OP9 8 `+ -` `+ - ~ |` OP8 7 `&` `&` OP7 6 `..` `.` OP6 @@ -685,20 +805,31 @@ Precedence level Operators First Whether an operator is used as a prefix operator is also affected by preceding whitespace (this parsing change was introduced with version 0.13.0): -.. code-block:: nim + ```nim echo $foo # is parsed as echo($foo) + ``` Spacing also determines whether `(a, b)` is parsed as an argument list of a call or whether it is parsed as a tuple constructor: -.. code-block:: nim + ```nim echo(1, 2) # pass 1 and 2 to echo + ``` -.. code-block:: nim + ```nim echo (1, 2) # pass the tuple (1, 2) to echo + ``` + +Dot-like operators +------------------ + +Terminal symbol in the grammar: `DOTLIKEOP`. + +Dot-like operators are operators starting with `.`, but not with `..`, for e.g. `.?`; +they have the same precedence as `.`, so that `a.?b.c` is parsed as `(a.?b).c` instead of `a.?(b.c)`. Grammar @@ -717,9 +848,7 @@ Order of evaluation Order of evaluation is strictly left-to-right, inside-out as it is typical for most others imperative programming languages: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" var s = "" proc p(arg: int): int = @@ -729,14 +858,13 @@ imperative programming languages: discard p(p(1) + p(2)) doAssert s == "123" + ``` Assignments are not special, the left-hand-side expression is evaluated before the right-hand side: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" var v = 0 proc getI(): int = result = v @@ -754,6 +882,7 @@ right-hand side: someCopy(b[getI()], getI()) doAssert b == [1, 0, 0] + ``` Rationale: Consistency with overloaded assignment or assignment-like operations, @@ -764,9 +893,7 @@ However, the concept of "order of evaluation" is only applicable after the code was normalized: The normalization involves template expansions and argument reorderings that have been passed to named parameters: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" var s = "" proc p(): int = @@ -793,6 +920,7 @@ reorderings that have been passed to named parameters: construct(second = q(), first = p()) doAssert s == "qppqpq" + ``` Rationale: This is far easier to implement than hypothetical alternatives. @@ -827,37 +955,37 @@ constant expressions that may be surprising to those coming from other statically typed languages. For example, the following code echoes the beginning of the Fibonacci series **at compile-time**. (This is a demonstration of flexibility in defining constants, not a recommended style for solving this -problem!) +problem.) -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" import std/strformat - var fib_n {.compileTime.}: int - var fib_prev {.compileTime.}: int - var fib_prev_prev {.compileTime.}: int + var fibN {.compileTime.}: int + var fibPrev {.compileTime.}: int + var fibPrevPrev {.compileTime.}: int - proc next_fib(): int = - result = if fib_n < 2: - fib_n + proc nextFib(): int = + result = if fibN < 2: + fibN else: - fib_prev_prev + fib_prev - inc(fib_n) - fib_prev_prev = fib_prev - fib_prev = result + fibPrevPrev + fibPrev + inc(fibN) + fibPrevPrev = fibPrev + fibPrev = result - const f0 = next_fib() - const f1 = next_fib() + const f0 = nextFib() + const f1 = nextFib() - const display_fib = block: - const f2 = next_fib() + const displayFib = block: + const f2 = nextFib() var result = fmt"Fibonacci sequence: {f0}, {f1}, {f2}" for i in 3..12: - add(result, fmt", {next_fib()}") + add(result, fmt", {nextFib()}") result static: - echo display_fib + echo displayFib + ``` Restrictions on Compile-Time Execution @@ -904,10 +1032,12 @@ Ordinal types have the following characteristics: - Ordinal types are countable and ordered. This property allows the operation of functions such as `inc`, `ord`, and `dec` on ordinal types to be defined. -- Ordinal values have the smallest possible value. Trying to count further - down than the smallest value produces a panic or a static error. -- Ordinal values have the largest possible value. Trying to count further - than the largest value produces a panic or a static error. +- Ordinal types have a smallest possible value, accessible with `low(type)`. + Trying to count further down than the smallest value produces a panic or + a static error. +- Ordinal types have a largest possible value, accessible with `high(type)`. + Trying to count further up than the largest value produces a panic or + a static error. Integers, bool, characters, and enumeration types (and subranges of these types) belong to ordinal types. @@ -920,22 +1050,24 @@ Pre-defined integer types These integer types are pre-defined: `int` - the generic signed integer type; its size is platform-dependent and has the +: the generic signed integer type; its size is platform-dependent and has the same size as a pointer. This type should be used in general. An integer literal that has no type suffix is of this type if it is in the range `low(int32)..high(int32)` otherwise the literal's type is `int64`. -intXX - additional signed integer types of XX bits use this naming scheme +`int`\ XX +: additional signed integer types of XX bits use this naming scheme (example: int16 is a 16-bit wide integer). The current implementation supports `int8`, `int16`, `int32`, `int64`. Literals of these types have the suffix 'iXX. `uint` - the generic `unsigned integer`:idx: type; its size is platform-dependent and has the same size as a pointer. An integer literal with the type suffix `'u` is of this type. +: the generic `unsigned integer`:idx: type; its size is platform-dependent and + has the same size as a pointer. An integer literal with the type + suffix `'u` is of this type. -uintXX - additional unsigned integer types of XX bits use this naming scheme +`uint`\ XX +: additional unsigned integer types of XX bits use this naming scheme (example: uint16 is a 16-bit wide unsigned integer). The current implementation supports `uint8`, `uint16`, `uint32`, `uint64`. Literals of these types have the suffix 'uXX. @@ -954,24 +1086,13 @@ the `%` suffix as convention: ====================== ====================================================== operation meaning ====================== ====================================================== -`a +% b` unsigned integer addition -`a -% b` unsigned integer subtraction -`a *% b` unsigned integer multiplication -`a /% b` unsigned integer division -`a %% b` unsigned integer modulo operation -`a <% b` treat `a` and `b` as unsigned and compare -`a <=% b` treat `a` and `b` as unsigned and compare -`ze(a)` extends the bits of `a` with zeros until it has the - width of the `int` type -`toU8(a)` treats `a` as unsigned and converts it to an - unsigned integer of 8 bits (but still the - `int8` type) -`toU16(a)` treats `a` as unsigned and converts it to an - unsigned integer of 16 bits (but still the - `int16` type) -`toU32(a)` treats `a` as unsigned and converts it to an - unsigned integer of 32 bits (but still the - `int32` type) +`a +% b` unsigned integer addition +`a -% b` unsigned integer subtraction +`a *% b` unsigned integer multiplication +`a /% b` unsigned integer division +`a %% b` unsigned integer modulo operation +`a <% b` treat `a` and `b` as unsigned and compare +`a <=% b` treat `a` and `b` as unsigned and compare ====================== ====================================================== `Automatic type conversion`:idx: is performed in expressions where different @@ -982,20 +1103,20 @@ example `int32 -> int16`). A `widening type conversion`:idx: converts a smaller type to a larger type (for example `int16 -> int32`). In Nim only widening type conversions are *implicit*: -.. code-block:: nim + ```nim var myInt16 = 5i16 var myInt: int myInt16 + 34 # of type `int16` myInt16 + myInt # of type `int` myInt16 + 2i32 # of type `int32` + ``` However, `int` literals are implicitly convertible to a smaller integer type if the literal's value fits this smaller type and such a conversion is less expensive than other implicit conversions, so `myInt16 + 34` produces an `int16` result. -For further details, see `Convertible relation -<#type-relations-convertible-relation>`_. +For further details, see [Convertible relation]. Subrange types @@ -1004,10 +1125,12 @@ A subrange type is a range of values from an ordinal or floating-point type (the type). To define a subrange type, one must specify its limiting values -- the lowest and highest value of the type. For example: -.. code-block:: nim + ```nim type Subrange = range[0..5] PositiveFloat = range[0.0..Inf] + Positive* = range[1..high(int)] # as defined in `system` + ``` `Subrange` is a subrange of an integer which can only hold the values 0 @@ -1028,22 +1151,21 @@ Pre-defined floating-point types The following floating-point types are pre-defined: `float` - the generic floating-point type; its size used to be platform-dependent, +: the generic floating-point type; its size used to be platform-dependent, but now it is always mapped to `float64`. This type should be used in general. -floatXX - an implementation may define additional floating-point types of XX bits using - this naming scheme (example: float64 is a 64-bit wide float). The current +`float`\ XX +: an implementation may define additional floating-point types of XX bits using + this naming scheme (example: `float64` is a 64-bit wide float). The current implementation supports `float32` and `float64`. Literals of these types have the suffix 'fXX. -Automatic type conversion in expressions with different kinds -of floating-point types is performed: See `Convertible relation`_ for further -details. Arithmetic performed on floating-point types follows the IEEE -standard. Integer types are not converted to floating-point types automatically -and vice versa. +Automatic type conversion in expressions with different kinds of floating-point +types is performed: See [Convertible relation] for further details. Arithmetic +performed on floating-point types follows the IEEE standard. Integer types are +not converted to floating-point types automatically and vice versa. The IEEE standard defines five types of floating-point exceptions: @@ -1067,12 +1189,13 @@ These exceptions inherit from the `FloatingPointDefect`:idx: base class. Nim provides the pragmas `nanChecks`:idx: and `infChecks`:idx: to control whether the IEEE exceptions are ignored or trap a Nim exception: -.. code-block:: nim + ```nim {.nanChecks: on, infChecks: on.} var a = 1.0 var b = 0.0 echo b / b # raises FloatInvalidOpDefect echo a / b # raises FloatOverflowDefect + ``` In the current implementation `FloatDivByZeroDefect` and `FloatInexactDefect` are never raised. `FloatOverflowDefect` is raised instead of @@ -1085,7 +1208,7 @@ The only operations that are affected by the `floatChecks` pragma are the `+`, `-`, `*`, `/` operators for floating-point types. An implementation should always use the maximum precision available to evaluate -floating pointer values during semantic analysis; this means expressions like +floating-point values during semantic analysis; this means expressions like `0.09'f32 + 0.01'f32 == 0.09'f64 + 0.01'f64` that are evaluating during constant folding are true. @@ -1096,19 +1219,21 @@ The boolean type is named `bool`:idx: in Nim and can be one of the two pre-defined values `true` and `false`. Conditions in `while`, `if`, `elif`, `when`-statements need to be of type `bool`. -This condition holds:: +This condition holds: + ```nim ord(false) == 0 and ord(true) == 1 + ``` The operators `not, and, or, xor, <, <=, >, >=, !=, ==` are defined for the bool type. The `and` and `or` operators perform short-cut evaluation. Example: -.. code-block:: nim - + ```nim while p != nil and p.name != "xyz": # p.name is not evaluated if p == nil p = p.next + ``` The size of the bool type is one byte. @@ -1117,14 +1242,10 @@ The size of the bool type is one byte. Character type -------------- The character type is named `char` in Nim. Its size is one byte. -Thus it cannot represent a UTF-8 character, but a part of it. -The reason for this is efficiency: for the overwhelming majority of use-cases, -the resulting programs will still handle UTF-8 properly as UTF-8 was especially -designed for this. -Another reason is that Nim can support `array[char, int]` or -`set[char]` efficiently as many algorithms rely on this feature. The -`Rune` type is used for Unicode characters, it can represent any Unicode -character. `Rune` is declared in the `unicode module <unicode.html>`_. +Thus, it cannot represent a UTF-8 character, but a part of it. + +The `Rune` type is used for Unicode characters, it can represent any Unicode +character. `Rune` is declared in the [unicode module](unicode.html). @@ -1134,15 +1255,16 @@ Enumeration types Enumeration types define a new type whose values consist of the ones specified. The values are ordered. Example: -.. code-block:: nim - + ```nim type Direction = enum north, east, south, west + ``` -Now the following holds:: +Now the following holds: + ```nim ord(north) == 0 ord(east) == 1 ord(south) == 2 @@ -1150,9 +1272,10 @@ Now the following holds:: # Also allowed: ord(Direction.west) == 3 + ``` -Thus, north < east < south < west. The comparison operators can be used -with enumeration types. Instead of `north` etc, the enum value can also +The implied order is: north < east < south < west. The comparison operators can be used +with enumeration types. Instead of `north` etc., the enum value can also be qualified with the enum type that it resides in, `Direction.north`. For better interfacing to other programming languages, the fields of enum @@ -1162,12 +1285,13 @@ explicitly given is assigned the value of the previous field + 1. An explicit ordered enum can have *holes*: -.. code-block:: nim + ```nim type TokenType = enum a = 2, b = 4, c = 89 # holes are valid + ``` -However, it is then not ordinal anymore, so it is not possible to use these +However, it is then not ordinal anymore, so it is impossible to use these enums as an index type for arrays. The procedures `inc`, `dec`, `succ` and `pred` are not available for them either. @@ -1176,14 +1300,14 @@ The compiler supports the built-in stringify operator `$` for enumerations. The stringify's result can be controlled by explicitly giving the string values to use: -.. code-block:: nim - + ```nim type MyEnum = enum valueA = (0, "my value A"), valueB = "value B", valueC = 2, valueD = (3, "abc") + ``` As can be seen from the example, it is possible to both specify a field's ordinal value and its string value by using a tuple. It is also @@ -1195,8 +1319,7 @@ as the last attempt. Only non-ambiguous symbols are added to this scope. But one can always access these via type qualification written as `MyEnum.value`: -.. code-block:: nim - + ```nim type MyEnum {.pure.} = enum valueA, valueB, valueC, valueD, amb @@ -1208,8 +1331,59 @@ as `MyEnum.value`: echo valueA # MyEnum.valueA echo amb # Error: Unclear whether it's MyEnum.amb or OtherEnum.amb echo MyEnum.amb # OK. + ``` + +Enum value names are overloadable, much like routines. If both of the enums +`T` and `U` have a member named `foo`, then the identifier `foo` corresponds +to a choice between `T.foo` and `U.foo`. During overload resolution, +the correct type of `foo` is decided from the context. If the type of `foo` is +ambiguous, a static error will be produced. + + ```nim test = "nim c $1" + + type + E1 = enum + value1, + value2 + E2 = enum + value1, + value2 = 4 + + const + Lookuptable = [ + E1.value1: "1", + # no need to qualify value2, known to be E1.value2 + value2: "2" + ] + + proc p(e: E1) = + # disambiguation in 'case' statements: + case e + of value1: echo "A" + of value2: echo "B" + + p value2 + ``` + +In some cases, ambiguity of enums is resolved depending on the relation +between the current scope and the scope the enums were defined in. + + ```nim + # a.nim + type Foo* = enum abc + + # b.nim + import a + type Bar = enum abc + echo abc is Bar # true + + block: + type Baz = enum abc + echo abc is Baz # true + ``` + +To implement bit fields with enums see [Bit fields]. -To implement bit fields with enums see `Bit fields <#set-type-bit-fields>`_ String type ----------- @@ -1229,14 +1403,14 @@ Most native Nim types support conversion to strings with the special `$` proc. When calling the `echo` proc, for example, the built-in stringify operation for the parameter is called: -.. code-block:: nim - + ```nim echo 3 # calls `$` for `int` + ``` Whenever a user creates a specialized object, implementation of this procedure provides for `string` representation. -.. code-block:: nim + ```nim type Person = object name: string @@ -1248,6 +1422,7 @@ provides for `string` representation. # is natively an integer to convert it to # a string " years old." + ``` While `$p.name` can also be used, the `$` operation on a string does nothing. Note that we cannot rely on automatic conversion from an `int` to @@ -1257,18 +1432,18 @@ Strings are compared by their lexicographical order. All comparison operators are available. Strings can be indexed like arrays (lower bound is 0). Unlike arrays, they can be used in case statements: -.. code-block:: nim - + ```nim case paramStr(i) of "-v": incl(options, optVerbose) of "-h", "-?": incl(options, optHelp) else: write(stdout, "invalid command line option!\n") + ``` Per convention, all strings are UTF-8 strings, but this is not enforced. For example, when reading strings from binary files, they are merely a sequence of bytes. The index operation `s[i]` means the i-th *char* of `s`, not the -i-th *unichar*. The iterator `runes` from the `unicode module -<unicode.html>`_ can be used for iteration over all Unicode characters. +i-th *unichar*. The iterator `runes` from the [unicode module](unicode.html) +can be used for iteration over all Unicode characters. cstring type @@ -1277,7 +1452,7 @@ cstring type The `cstring` type meaning `compatible string` is the native representation of a string for the compilation backend. For the C backend the `cstring` type represents a pointer to a zero-terminated char array -compatible with the type `char*` in Ansi C. Its primary purpose lies in easy +compatible with the type `char*` in ANSI C. Its primary purpose lies in easy interfacing with C. The index operation `s[i]` means the i-th *char* of `s`; however no bounds checking for `cstring` is performed making the index operation unsafe. @@ -1286,40 +1461,46 @@ A Nim `string` is implicitly convertible to `cstring` for convenience. If a Nim string is passed to a C-style variadic proc, it is implicitly converted to `cstring` too: -.. code-block:: nim + ```nim proc printf(formatstr: cstring) {.importc: "printf", varargs, header: "<stdio.h>".} printf("This works %s", "as expected") + ``` Even though the conversion is implicit, it is not *safe*: The garbage collector does not consider a `cstring` to be a root and may collect the underlying -memory. However, in practice, this almost never happens as the GC considers -stack roots conservatively. One can use the builtin procs `GC_ref` and -`GC_unref` to keep the string data alive for the rare cases where it does -not work. +memory. For this reason, the implicit conversion will be removed in future +releases of the Nim compiler. Certain idioms like conversion of a `const` string +to `cstring` are safe and will remain to be allowed. -A `$` proc is defined for cstrings that returns a string. Thus to get a nim +A `$` proc is defined for cstrings that returns a string. Thus, to get a nim string from a cstring: -.. code-block:: nim + ```nim var str: string = "Hello!" var cstr: cstring = str var newstr: string = $cstr + ``` `cstring` literals shouldn't be modified. -.. code-block:: nim + ```nim var x = cstring"literals" x[1] = 'A' # This is wrong!!! + ``` If the `cstring` originates from a regular memory (not read-only memory), it can be modified: -.. code-block:: nim + ```nim var x = "123456" - var s: cstring = x + prepareMutation(x) # call `prepareMutation` before modifying the strings + var s: cstring = cstring(x) s[0] = 'u' # This is ok + ``` + +`cstring` values may also be used in case statements like strings. Structured types ---------------- @@ -1353,8 +1534,7 @@ A sequence may be passed to a parameter that is of type *open array*. Example: -.. code-block:: nim - + ```nim type IntArray = array[0..5, int] # an array that is indexed with 0..5 IntSeq = seq[int] # a sequence of integers @@ -1365,6 +1545,7 @@ Example: y = @[1, 2, 3, 4, 5, 6] # the @ turns the array into a sequence let z = [1.0, 2, 3, 4] # the type of z is array[0..3, float] + ``` The lower bound of an array or sequence may be received by the built-in proc `low()`, the higher bound by `high()`. The length may be @@ -1378,12 +1559,11 @@ The notation `x[i]` can be used to access the i-th element of `x`. Arrays are always bounds checked (statically or at runtime). These checks can be disabled via pragmas or invoking the compiler with the -`--boundChecks:off` command-line switch. +`--boundChecks:off`:option: command-line switch. An array constructor can have explicit indexes for readability: -.. code-block:: nim - + ```nim type Values = enum valA, valB, valC @@ -1394,12 +1574,12 @@ An array constructor can have explicit indexes for readability: valB: "B", valC: "C" ] + ``` If an index is left out, `succ(lastIndex)` is used as the index value: -.. code-block:: nim - + ```nim type Values = enum valA, valB, valC, valD, valE @@ -1411,6 +1591,7 @@ value: valC: "C", "D", "e" ] + ``` @@ -1419,30 +1600,31 @@ Open arrays Often fixed size arrays turn out to be too inflexible; procedures should be able to deal with arrays of different sizes. The `openarray`:idx: type -allows this; it can only be used for parameters. Openarrays are always +allows this; it can only be used for parameters. Open arrays are always indexed with an `int` starting at position 0. The `len`, `low` and `high` operations are available for open arrays too. Any array with -a compatible base type can be passed to an openarray parameter, the index +a compatible base type can be passed to an open array parameter, the index type does not matter. In addition to arrays, sequences can also be passed to an open array parameter. -The openarray type cannot be nested: multidimensional openarrays are not +The `openarray` type cannot be nested: multidimensional open arrays are not supported because this is seldom needed and cannot be done efficiently. -.. code-block:: nim + ```nim proc testOpenArray(x: openArray[int]) = echo repr(x) testOpenArray([1,2,3]) # array[] testOpenArray(@[1,2,3]) # seq[] + ``` Varargs ------- -A `varargs` parameter is an openarray parameter that additionally -allows to pass a variable number of arguments to a procedure. The compiler +A `varargs` parameter is an open array parameter that additionally +allows a variable number of arguments to be passed to a procedure. The compiler converts the list of arguments to an array implicitly: -.. code-block:: nim + ```nim proc myWriteln(f: File, a: varargs[string]) = for s in items(a): write(f, s) @@ -1451,12 +1633,13 @@ converts the list of arguments to an array implicitly: myWriteln(stdout, "abc", "def", "xyz") # is transformed to: myWriteln(stdout, ["abc", "def", "xyz"]) + ``` -This transformation is only done if the varargs parameter is the +This transformation is only done if the `varargs` parameter is the last parameter in the procedure header. It is also possible to perform type conversions in this context: -.. code-block:: nim + ```nim proc myWriteln(f: File, a: varargs[string, `$`]) = for s in items(a): write(f, s) @@ -1464,7 +1647,8 @@ type conversions in this context: myWriteln(stdout, 123, "abc", 4.0) # is transformed to: - myWriteln(stdout, [$123, $"def", $4.0]) + myWriteln(stdout, [$123, $"abc", $4.0]) + ``` In this example `$` is applied to any argument that is passed to the parameter `a`. (Note that `$` applied to strings is a nop.) @@ -1472,21 +1656,23 @@ parameter `a`. (Note that `$` applied to strings is a nop.) Note that an explicit array constructor passed to a `varargs` parameter is not wrapped in another implicit array construction: -.. code-block:: nim + ```nim proc takeV[T](a: varargs[T]) = discard takeV([123, 2, 1]) # takeV's T is "int", not "array of int" + ``` `varargs[typed]` is treated specially: It matches a variable list of arguments of arbitrary type but *always* constructs an implicit array. This is required so that the builtin `echo` proc does what is expected: -.. code-block:: nim + ```nim proc echo*(x: varargs[typed, `$`]) {...} echo @[1, 2, 3] # prints "@[1, 2, 3]" and not "123" + ``` Unchecked arrays @@ -1496,20 +1682,22 @@ are not checked. This is often useful to implement customized flexibly sized arrays. Additionally, an unchecked array is translated into a C array of undetermined size: -.. code-block:: nim + ```nim type MySeq = object len, cap: int data: UncheckedArray[int] + ``` Produces roughly this C code: -.. code-block:: C + ```C typedef struct { NI len; NI cap; NI data[]; } MySeq; + ``` The base type of the unchecked array may not contain any GC'ed memory but this is currently not checked. @@ -1530,35 +1718,32 @@ heterogeneous storage types with few abstractions. The `()` syntax can be used to construct tuples. The order of the fields in the constructor must match the order of the tuple's definition. Different tuple-types are *equivalent* if they specify the same fields of the same type in the same -order. The *names* of the fields also have to be identical. - -The assignment operator for tuples copies each component. -The default assignment operator for objects copies each component. Overloading -of the assignment operator is described `here -<manual_experimental.html#type-bound-operations>`_. - -.. code-block:: nim +order. The *names* of the fields also have to be the same. + ```nim type Person = tuple[name: string, age: int] # type representing a person: - # a person consists of a name - # and an age - var - person: Person + # it consists of a name and an age. + var person: Person person = (name: "Peter", age: 30) - echo person.name + assert person.name == "Peter" # the same, but less readable: person = ("Peter", 30) - echo person[0] + assert person[0] == "Peter" + assert Person is (string, int) + assert (string, int) is Person + assert Person isnot tuple[other: string, age: int] # `other` is a different identifier + ``` A tuple with one unnamed field can be constructed with the parentheses and a trailing comma: -.. code-block:: nim + ```nim proc echoUnaryTuple(a: (int,)) = echo a[0] echoUnaryTuple (1,) + ``` In fact, a trailing comma is allowed for every tuple construction. @@ -1569,11 +1754,12 @@ is compatible with the way the C compiler does it. For consistency with `object` declarations, tuples in a `type` section can also be defined with indentation instead of `[]`: -.. code-block:: nim + ```nim type Person = tuple # type representing a person name: string # a person consists of a name age: Natural # and an age + ``` Objects provide many features that tuples do not. Objects provide inheritance and the ability to hide fields from other modules. Objects with inheritance @@ -1581,7 +1767,7 @@ enabled have information about their type at runtime so that the `of` operator can be used to determine the object's type. The `of` operator is similar to the `instanceof` operator in Java. -.. code-block:: nim + ```nim type Person = object of RootObj name*: string # the * means that `name` is accessible from other modules @@ -1595,6 +1781,7 @@ the `instanceof` operator in Java. person: Person assert(student of Student) # is true assert(student of Person) # also true + ``` Object fields that should be visible from outside the defining module have to be marked by `*`. In contrast to tuples, different object types are @@ -1603,7 +1790,7 @@ Objects that have no ancestor are implicitly `final` and thus have no hidden type information. One can use the `inheritable` pragma to introduce new object roots apart from `system.RootObj`. -.. code-block:: nim + ```nim type Person = object # example of a final object name*: string @@ -1611,6 +1798,11 @@ introduce new object roots apart from `system.RootObj`. Student = ref object of Person # Error: inheritance only works with non-final objects id: int + ``` + +The assignment operator for tuples and objects copies each component. +The methods to override this copying behavior are described [here][type +bound operators]. Object construction @@ -1620,8 +1812,19 @@ Objects can also be created with an `object construction expression`:idx: that has the syntax `T(fieldA: valueA, fieldB: valueB, ...)` where `T` is an `object` type or a `ref object` type: -.. code-block:: nim - var student = Student(name: "Anton", age: 5, id: 3) + ```nim + type + Student = object + name: string + age: int + PStudent = ref Student + var a1 = Student(name: "Anton", age: 5) + var a2 = PStudent(name: "Anton", age: 5) + # this also works directly: + var a3 = (ref Student)(name: "Anton", age: 5) + # not all fields need to be mentioned, and they can be mentioned out of order: + var a4 = Student(age: 5) + ``` Note that, unlike tuples, objects require the field names along with their values. For a `ref object` type `system.new` is invoked implicitly. @@ -1636,8 +1839,7 @@ enumerated type used for runtime type flexibility, mirroring the concepts of An example: -.. code-block:: nim - + ```nim # This is an example of how an abstract syntax tree could be modelled in Nim type NodeKind = enum # the different node types @@ -1674,6 +1876,7 @@ An example: rightOp: Node(kind: nkInt, intVal: 2)) # valid: does not change the active object branch: x.kind = nkSub + ``` As can be seen from the example, an advantage to an object hierarchy is that no casting between different object types is needed. Yet, access to invalid @@ -1691,12 +1894,12 @@ corresponding discriminator value must be specified as a constant expression. Instead of changing the active object branch, replace the old object in memory with a new one completely: -.. code-block:: nim - + ```nim var x = Node(kind: nkAdd, leftOp: Node(kind: nkInt, intVal: 4), rightOp: Node(kind: nkInt, intVal: 2)) # change the node's contents: x[] = NodeObj(kind: nkString, strVal: "abc") + ``` Starting with version 0.20 `system.reset` cannot be used anymore to support @@ -1713,8 +1916,7 @@ valid for the chosen object branch. A small example: -.. code-block:: nim - + ```nim let unknownKind = nkSub # invalid: unsafe initialization because the kind field is not statically known: @@ -1731,6 +1933,92 @@ A small example: # also valid, since unknownKindBounded can only contain the values nkAdd or nkSub let unknownKindBounded = range[nkAdd..nkSub](unknownKind) z = Node(kind: unknownKindBounded, leftOp: Node(), rightOp: Node()) + ``` + + +cast uncheckedAssign +-------------------- + +Some restrictions for case objects can be disabled via a `{.cast(uncheckedAssign).}` section: + + ```nim test="nim c $1" + type + TokenKind* = enum + strLit, intLit + Token = object + case kind*: TokenKind + of strLit: + s*: string + of intLit: + i*: int64 + + proc passToVar(x: var TokenKind) = discard + + var t = Token(kind: strLit, s: "abc") + + {.cast(uncheckedAssign).}: + # inside the 'cast' section it is allowed to pass 't.kind' to a 'var T' parameter: + passToVar(t.kind) + + # inside the 'cast' section it is allowed to set field 's' even though the + # constructed 'kind' field has an unknown value: + t = Token(kind: t.kind, s: "abc") + + # inside the 'cast' section it is allowed to assign to the 't.kind' field directly: + t.kind = intLit + ``` + +Default values for object fields +-------------------------------- + +Object fields are allowed to have a constant default value. The type of field can be omitted if a default value is given. + +```nim test +type + Foo = object + a: int = 2 + b: float = 3.14 + c = "I can have a default value" + + Bar = ref object + a: int = 2 + b: float = 3.14 + c = "I can have a default value" +``` + +The explicit initialization uses these defaults which includes an `object` created with an object construction expression or the procedure `default`; a `ref object` created with an object construction expression or the procedure `new`; an array or a tuple with a subtype which has a default created with the procedure `default`. + + +```nim test +type + Foo = object + a: int = 2 + b = 3.0 + Bar = ref object + a: int = 2 + b = 3.0 + +block: # created with an object construction expression + let x = Foo() + assert x.a == 2 and x.b == 3.0 + + let y = Bar() + assert y.a == 2 and y.b == 3.0 + +block: # created with an object construction expression + let x = default(Foo) + assert x.a == 2 and x.b == 3.0 + + let y = default(array[1, Foo]) + assert y[0].a == 2 and y[0].b == 3.0 + + let z = default(tuple[x: Foo]) + assert z.x.a == 2 and z.x.b == 3.0 + +block: # created with the procedure `new` + let y = new Bar + assert y.a == 2 and y.b == 3.0 +``` Set type -------- @@ -1746,25 +2034,24 @@ point to and modify the same location in memory (also called `aliasing`:idx:). Nim distinguishes between `traced`:idx: and `untraced`:idx: references. Untraced references are also called *pointers*. Traced references point to objects of a garbage-collected heap, untraced references point to -manually allocated objects or objects somewhere else in memory. Thus +manually allocated objects or objects somewhere else in memory. Thus, untraced references are *unsafe*. However, for certain low-level operations (accessing the hardware) untraced references are unavoidable. Traced references are declared with the **ref** keyword, untraced references -are declared with the **ptr** keyword. In general, a `ptr T` is implicitly +are declared with the **ptr** keyword. In general, a `ptr T` is implicitly convertible to the `pointer` type. An empty subscript `[]` notation can be used to de-refer a reference, the `addr` procedure returns the address of an item. An address is always an untraced reference. -Thus the usage of `addr` is an *unsafe* feature. +Thus, the usage of `addr` is an *unsafe* feature. The `.` (access a tuple/object field operator) and `[]` (array/string/sequence index operator) operators perform implicit dereferencing operations for reference types: -.. code-block:: nim - + ```nim type Node = ref NodeObj NodeObj = object @@ -1776,16 +2063,14 @@ dereferencing operations for reference types: new(n) n.data = 9 # no need to write n[].data; in fact n[].data is highly discouraged! - -Automatic dereferencing can be performed for the first argument of a routine -call, but this is an experimental feature and is described `here -<manual_experimental.html#automatic-dereferencing>`_. + ``` In order to simplify structural type checking, recursive tuples are not valid: -.. code-block:: nim + ```nim # invalid recursion type MyTuple = tuple[a: ref MyTuple] + ``` Likewise `T = ref T` is an invalid type. @@ -1793,17 +2078,17 @@ As a syntactical extension, `object` types can be anonymous if declared in a type section via the `ref object` or `ptr object` notations. This feature is useful if an object should only gain reference semantics: -.. code-block:: nim - + ```nim type Node = ref object le, ri: Node data: int + ``` To allocate a new traced object, the built-in procedure `new` has to be used. To deal with untraced memory, the procedures `alloc`, `dealloc` and -`realloc` can be used. The documentation of the `system <system.html>`_ module +`realloc` can be used. The documentation of the [system](system.html) module contains further information. @@ -1820,20 +2105,20 @@ Dereferencing `nil` is an unrecoverable fatal runtime error (and not a panic). A successful dereferencing operation `p[]` implies that `p` is not nil. This can be exploited by the implementation to optimize code like: -.. code-block:: nim - + ```nim p[].field = 3 if p != nil: # if p were nil, `p[]` would have caused a crash already, # so we know `p` is always not nil here. action() + ``` Into: -.. code-block:: nim - + ```nim p[].field = 3 action() + ``` *Note*: This is not comparable to C's "undefined behavior" for @@ -1848,7 +2133,7 @@ traced references, strings, or sequences: in order to free everything properly, the built-in procedure `reset` has to be called before freeing the untraced memory manually: -.. code-block:: nim + ```nim type Data = tuple[x, y: int, s: string] @@ -1863,9 +2148,11 @@ memory manually: # free the memory: dealloc(d) + ``` Without the `reset` call the memory allocated for the `d.s` string would -never be freed. The example also demonstrates two important features for low-level programming: the `sizeof` proc returns the size of a type or value +never be freed. The example also demonstrates two important features for +low-level programming: the `sizeof` proc returns the size of a type or value in bytes. The `cast` operator can circumvent the type system: the compiler is forced to treat the result of the `alloc0` call (which returns an untyped pointer) as if it would have the type `ptr Data`. Casting should only be @@ -1883,23 +2170,21 @@ details like this when mixing garbage-collected data with unmanaged memory. Procedural type --------------- A procedural type is internally a pointer to a procedure. `nil` is -an allowed value for variables of a procedural type. Nim uses procedural -types to achieve `functional`:idx: programming techniques. +an allowed value for a variable of a procedural type. Examples: -.. code-block:: nim - + ```nim proc printItem(x: int) = ... proc forEach(c: proc (x: int) {.cdecl.}) = ... forEach(printItem) # this will NOT compile because calling conventions differ + ``` -.. code-block:: nim - + ```nim type OnMouseMove = proc (x, y: int) {.closure.} @@ -1912,6 +2197,7 @@ Examples: # ok, 'onMouseMove' has the default calling convention, which is compatible # to 'closure': setOnMouseMove(onMouseMove) + ``` A subtle issue with procedural types is that the calling convention of the @@ -1923,52 +2209,52 @@ that expects a proc of the calling convention `closure`. Nim supports these `calling conventions`:idx:\: `nimcall`:idx: - is the default convention used for a Nim **proc**. It is the +: is the default convention used for a Nim **proc**. It is the same as `fastcall`, but only for C compilers that support `fastcall`. `closure`:idx: - is the default calling convention for a **procedural type** that lacks +: is the default calling convention for a **procedural type** that lacks any pragma annotations. It indicates that the procedure has a hidden implicit parameter (an *environment*). Proc vars that have the calling convention `closure` take up two machine words: One for the proc pointer and another one for the pointer to implicitly passed environment. `stdcall`:idx: - This is the stdcall convention as specified by Microsoft. The generated C +: This is the stdcall convention as specified by Microsoft. The generated C procedure is declared with the `__stdcall` keyword. `cdecl`:idx: - The cdecl convention means that a procedure shall use the same convention +: The cdecl convention means that a procedure shall use the same convention as the C compiler. Under Windows the generated C procedure is declared with the `__cdecl` keyword. `safecall`:idx: - This is the safecall convention as specified by Microsoft. The generated C +: This is the safecall convention as specified by Microsoft. The generated C procedure is declared with the `__safecall` keyword. The word *safe* refers to the fact that all hardware registers shall be pushed to the hardware stack. `inline`:idx: - The inline convention means the caller should not call the procedure, +: The inline convention means the caller should not call the procedure, but inline its code directly. Note that Nim does not inline, but leaves this to the C compiler; it generates `__inline` procedures. This is - only a hint for the compiler: it may completely ignore it and + only a hint for the compiler: it may completely ignore it, and it may inline procedures that are not marked as `inline`. `fastcall`:idx: - Fastcall means different things to different C compilers. One gets whatever +: Fastcall means different things to different C compilers. One gets whatever the C `__fastcall` means. `thiscall`:idx: - This is the thiscall calling convention as specified by Microsoft, used on +: This is the thiscall calling convention as specified by Microsoft, used on C++ class member functions on the x86 architecture. `syscall`:idx: - The syscall convention is the same as `__syscall` in C. It is used for +: The syscall convention is the same as `__syscall`:c: in C. It is used for interrupts. `noconv`:idx: - The generated C code will not have any explicit calling convention and thus +: The generated C code will not have any explicit calling convention and thus use the C compiler's default calling convention. This is needed because Nim's default calling convention for procedures is `fastcall` to improve speed. @@ -1994,8 +2280,7 @@ reverse operation. A distinct type is an ordinal type if its base type is an ordinal type. -Modeling currencies -~~~~~~~~~~~~~~~~~~~~ +### Modeling currencies A distinct type can be used to model different physical `units`:idx: with a numerical base type, for example. The following example models currencies. @@ -2003,7 +2288,7 @@ numerical base type, for example. The following example models currencies. Different currencies should not be mixed in monetary calculations. Distinct types are a perfect tool to model different currencies: -.. code-block:: nim + ```nim type Dollar = distinct int Euro = distinct int @@ -2014,19 +2299,21 @@ types are a perfect tool to model different currencies: echo d + 12 # Error: cannot add a number with no unit and a `Dollar` + ``` Unfortunately, `d + 12.Dollar` is not allowed either, because `+` is defined for `int` (among others), not for `Dollar`. So a `+` for dollars needs to be defined: -.. code-block:: + ```nim proc `+` (x, y: Dollar): Dollar = result = Dollar(int(x) + int(y)) + ``` It does not make sense to multiply a dollar with a dollar, but with a number without unit; and the same holds for division: -.. code-block:: + ```nim proc `*` (x: Dollar, y: int): Dollar = result = Dollar(int(x) * y) @@ -2034,6 +2321,7 @@ number without unit; and the same holds for division: result = Dollar(x * int(y)) proc `div` ... + ``` This quickly gets tedious. The implementations are trivial and the compiler should not generate all this code only to optimize it away later - after all @@ -2041,21 +2329,20 @@ should not generate all this code only to optimize it away later - after all The pragma `borrow`:idx: has been designed to solve this problem; in principle, it generates the above trivial implementations: -.. code-block:: nim + ```nim proc `*` (x: Dollar, y: int): Dollar {.borrow.} proc `*` (x: int, y: Dollar): Dollar {.borrow.} proc `div` (x: Dollar, y: int): Dollar {.borrow.} + ``` The `borrow` pragma makes the compiler use the same implementation as the proc that deals with the distinct type's base type, so no code is generated. But it seems all this boilerplate code needs to be repeated for the `Euro` -currency. This can be solved with templates_. - -.. code-block:: nim - :test: "nim c $1" +currency. This can be solved with [templates]. + ```nim test = "nim c $1" template additive(typ: typedesc) = proc `+` *(x, y: typ): typ {.borrow.} proc `-` *(x, y: typ): typ {.borrow.} @@ -2084,12 +2371,13 @@ currency. This can be solved with templates_. defineCurrency(Dollar, int) defineCurrency(Euro, int) + ``` The borrow pragma can also be used to annotate the distinct type to allow certain builtin operations to be lifted: -.. code-block:: nim + ```nim type Foo = object a, b: int @@ -2102,18 +2390,18 @@ certain builtin operations to be lifted: # field access now valid bb.a = 90 bb.s = "abc" + ``` Currently, only the dot accessor can be borrowed in this way. -Avoiding SQL injection attacks -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +### Avoiding SQL injection attacks An SQL statement that is passed from Nim to an SQL database might be modeled as a string. However, using string templates and filling in the values is vulnerable to the famous `SQL injection attack`:idx:\: -.. code-block:: nim + ```nim import std/strutils proc query(db: DbHandle, statement: string) = ... @@ -2123,12 +2411,13 @@ values is vulnerable to the famous `SQL injection attack`:idx:\: db.query("SELECT FROM users WHERE name = '$1'" % username) # Horrible security hole, but the compiler does not mind! + ``` This can be avoided by distinguishing strings that contain SQL from strings that don't. Distinct types provide a means to introduce a new string type `SQL` that is incompatible with `string`: -.. code-block:: nim + ```nim type SQL = distinct string @@ -2139,13 +2428,14 @@ that don't. Distinct types provide a means to introduce a new string type db.query("SELECT FROM users WHERE name = '$1'" % username) # Static error: `query` expects an SQL string! + ``` It is an essential property of abstract types that they **do not** imply a subtype relation between the abstract type and its base type. Explicit type conversions from `string` to `SQL` are allowed: -.. code-block:: nim + ```nim import std/[strutils, sequtils] proc properQuote(s: string): SQL = @@ -2161,12 +2451,13 @@ conversions from `string` to `SQL` are allowed: result = SQL(string(frmt) % StrSeq(v)) db.query("SELECT FROM users WHERE name = '$1'".SQL % [username]) + ``` -Now we have compile-time checking against SQL injection attacks. Since +Now we have compile-time checking against SQL injection attacks. Since `"".SQL` is transformed to `SQL("")` no new syntax is needed for nice looking `SQL` string literals. The hypothetical `SQL` type actually -exists in the library as the `SqlQuery type <db_common.html#SqlQuery>`_ of -modules like `db_sqlite <db_sqlite.html>`_. +exists in the library as the [SqlQuery type](db_common.html#SqlQuery) of +modules like [db_sqlite](db_sqlite.html). Auto type @@ -2175,18 +2466,21 @@ Auto type The `auto` type can only be used for return types and parameters. For return types it causes the compiler to infer the type from the routine body: -.. code-block:: nim + ```nim proc returnsInt(): auto = 1984 + ``` For parameters it currently creates implicitly generic routines: -.. code-block:: nim + ```nim proc foo(a, b: auto) = discard + ``` Is the same as: -.. code-block:: nim + ```nim proc foo[T1, T2](a: T1, b: T2) = discard + ``` However, later versions of the language might change this to mean "infer the parameters' types from the body". Then the above `foo` would be rejected as @@ -2202,121 +2496,36 @@ describe the type checking done by the compiler. Type equality ------------- -Nim uses structural type equivalence for most types. Only for objects, -enumerations and distinct types name equivalence is used. The following -algorithm, *in pseudo-code*, determines type equality: - -.. code-block:: nim - proc typeEqualsAux(a, b: PType, - s: var HashSet[(PType, PType)]): bool = - if (a,b) in s: return true - incl(s, (a,b)) - if a.kind == b.kind: - case a.kind - of int, intXX, float, floatXX, char, string, cstring, pointer, - bool, nil, void: - # leaf type: kinds identical; nothing more to check - result = true - of ref, ptr, var, set, seq, openarray: - result = typeEqualsAux(a.baseType, b.baseType, s) - of range: - result = typeEqualsAux(a.baseType, b.baseType, s) and - (a.rangeA == b.rangeA) and (a.rangeB == b.rangeB) - of array: - result = typeEqualsAux(a.baseType, b.baseType, s) and - typeEqualsAux(a.indexType, b.indexType, s) - of tuple: - if a.tupleLen == b.tupleLen: - for i in 0..a.tupleLen-1: - if not typeEqualsAux(a[i], b[i], s): return false - result = true - of object, enum, distinct: - result = a == b - of proc: - result = typeEqualsAux(a.parameterTuple, b.parameterTuple, s) and - typeEqualsAux(a.resultType, b.resultType, s) and - a.callingConvention == b.callingConvention - - proc typeEquals(a, b: PType): bool = - var s: HashSet[(PType, PType)] = {} - result = typeEqualsAux(a, b, s) - -Since types are graphs which can have cycles, the above algorithm needs an -auxiliary set `s` to detect this case. - - -Type equality modulo type distinction -------------------------------------- -The following algorithm (in pseudo-code) determines whether two types -are equal with no respect to `distinct` types. For brevity the cycle check -with an auxiliary set `s` is omitted: - -.. code-block:: nim - proc typeEqualsOrDistinct(a, b: PType): bool = - if a.kind == b.kind: - case a.kind - of int, intXX, float, floatXX, char, string, cstring, pointer, - bool, nil, void: - # leaf type: kinds identical; nothing more to check - result = true - of ref, ptr, var, set, seq, openarray: - result = typeEqualsOrDistinct(a.baseType, b.baseType) - of range: - result = typeEqualsOrDistinct(a.baseType, b.baseType) and - (a.rangeA == b.rangeA) and (a.rangeB == b.rangeB) - of array: - result = typeEqualsOrDistinct(a.baseType, b.baseType) and - typeEqualsOrDistinct(a.indexType, b.indexType) - of tuple: - if a.tupleLen == b.tupleLen: - for i in 0..a.tupleLen-1: - if not typeEqualsOrDistinct(a[i], b[i]): return false - result = true - of distinct: - result = typeEqualsOrDistinct(a.baseType, b.baseType) - of object, enum: - result = a == b - of proc: - result = typeEqualsOrDistinct(a.parameterTuple, b.parameterTuple) and - typeEqualsOrDistinct(a.resultType, b.resultType) and - a.callingConvention == b.callingConvention - elif a.kind == distinct: - result = typeEqualsOrDistinct(a.baseType, b) - elif b.kind == distinct: - result = typeEqualsOrDistinct(a, b.baseType) +Nim uses structural type equivalence for most types. Only for objects, +enumerations and distinct types and for generic types name equivalence is used. Subtype relation ---------------- -If object `a` inherits from `b`, `a` is a subtype of `b`. This subtype -relation is extended to the types `var`, `ref`, `ptr`: -.. code-block:: nim - proc isSubtype(a, b: PType): bool = - if a.kind == b.kind: - case a.kind - of object: - var aa = a.baseType - while aa != nil and aa != b: aa = aa.baseType - result = aa == b - of var, ref, ptr: - result = isSubtype(a.baseType, b.baseType) +If object `a` inherits from `b`, `a` is a subtype of `b`. -.. XXX nil is a special value! +This subtype relation is extended to the types `var`, `ref`, `ptr`. +If `A` is a subtype of `B` and `A` and `B` are `object` types then: +- `var A` is a subtype of `var B` +- `ref A` is a subtype of `ref B` +- `ptr A` is a subtype of `ptr B`. +**Note**: One of the above pointer-indirections is required for assignment from +a subtype to its parent type to prevent "object slicing". Convertible relation -------------------- + A type `a` is **implicitly** convertible to type `b` iff the following algorithm returns true: -.. code-block:: nim - + ```nim proc isImplicitlyConvertible(a, b: PType): bool = - if isSubtype(a, b) or isCovariant(a, b): + if isSubtype(a, b): return true if isIntLiteral(a): return b in {int8, int16, int32, int64, int, uint, uint8, uint16, @@ -2342,7 +2551,13 @@ algorithm returns true: result = b == pointer of string: result = b == cstring + of proc: + result = typeEquals(a, b) or compatibleParametersAndEffects(a, b) + ``` +We used the predicate `typeEquals(a, b)` for the "type equality" property +and the predicate `isSubtype(a, b)` for the "subtype relation". +`compatibleParametersAndEffects(a, b)` is currently not specified. Implicit conversions are also performed for Nim's `range` type constructor. @@ -2358,21 +2573,24 @@ are signed integers or if both are unsigned integers. A type `a` is **explicitly** convertible to type `b` iff the following algorithm returns true: -.. code-block:: nim + ```nim proc isIntegralType(t: PType): bool = result = isOrdinal(t) or t.kind in {float, float32, float64} proc isExplicitlyConvertible(a, b: PType): bool = result = false if isImplicitlyConvertible(a, b): return true - if typeEqualsOrDistinct(a, b): return true + if typeEquals(a, b): return true + if a == distinct and typeEquals(a.baseType, b): return true + if b == distinct and typeEquals(b.baseType, a): return true if isIntegralType(a) and isIntegralType(b): return true if isSubtype(a, b) or isSubtype(b, a): return true + ``` The convertible relation can be relaxed by a user-defined type `converter`:idx:. -.. code-block:: nim + ```nim converter toInt(x: char): int = result = ord(x) var @@ -2385,6 +2603,7 @@ The convertible relation can be relaxed by a user-defined type # one can use the explicit form too x = chr.toInt echo x # => 97 + ``` The type conversion `T(a)` is an L-value if `a` is an L-value and `typeEqualsOrDistinct(T, typeof(a))` holds. @@ -2397,16 +2616,23 @@ An expression `b` can be assigned to an expression `a` iff `a` is an `l-value` and `isImplicitlyConvertible(b.typ, a.typ)` holds. -Overloading resolution -====================== +Overload resolution +=================== + +In a call `p(args)` where `p` may refer to more than one +candidate, it is said to be a symbol choice. Overload resolution will attempt to +find the best candidate, thus transforming the symbol choice into a resolved symbol. +The routine `p` that matches best is selected following a series of trials explained below. +In order: Catagory matching, Hierarchical Order Comparison, and finally, Complexity Analysis. + +If multiple candidates match equally well after all trials have been tested, the ambiguity +is reported during semantic analysis. -In a call `p(args)` the routine `p` that matches best is selected. If -multiple routines match equally well, the ambiguity is reported during -semantic analysis. +First Trial: Catagory matching +-------------------------------- -Every arg in args needs to match. There are multiple different categories how an -argument can match. Let `f` be the formal parameter's type and `a` the type -of the argument. +Every arg in `args` needs to match and there are multiple different categories of matches. +Let `f` be the formal parameter's type and `a` the type of the argument. 1. Exact match: `a` and `f` are of the same type. 2. Literal match: `a` is an integer literal of value `v` @@ -2416,7 +2642,7 @@ of the argument. range. 3. Generic match: `f` is a generic type and `a` matches, for instance `a` is `int` and `f` is a generic (constrained) parameter - type (like in `[T]` or `[T: int|char]`. + type (like in `[T]` or `[T: int|char]`). 4. Subrange or subtype match: `a` is a `range[T]` and `T` matches `f` exactly. Or: `a` is a subtype of `f`. 5. Integral conversion match: `a` is convertible to `f` and `f` and `a` @@ -2424,14 +2650,21 @@ of the argument. 6. Conversion match: `a` is convertible to `f`, possibly via a user defined `converter`. -These matching categories have a priority: An exact match is better than a -literal match and that is better than a generic match etc. In the following, -`count(p, m)` counts the number of matches of the matching category `m` -for the routine `p`. +Each operand may fall into one of the categories above; the operand's +highest priority category. The list above is in order or priority. +If a candidate has more priority matches than all other candidates, it is selected as the +resolved symbol. + +For example, if a candidate with one exact match is compared to a candidate with multiple +generic matches and zero exact matches, the candidate with an exact match will win. + +Below is a pseudocode interpretation of category matching, `count(p, m)` counts the number +of matches of the matching category `m` for the routine `p`. A routine `p` matches better than a routine `q` if the following -algorithm returns true:: +algorithm returns true: + ```nim for each matching category m in ["exact match", "literal match", "generic match", "subtype match", "integral match", "conversion match"]: @@ -2441,11 +2674,55 @@ algorithm returns true:: else: return false return "ambiguous" + ``` + +Second Trial: Hierarchical Order Comparison +---------------------------------------------- + +The hierarchical order of a type is analogous to its relative specificity. Consider the type defined: + +```nim +type A[T] = object +``` + +Matching formals for this type include `T`, `object`, `A`, `A[...]` and `A[C]` where `C` is a concrete type, `A[...]` +is a generic typeclass composition and `T` is an unconstrained generic type variable. This list is in order of +specificity with respect to `A` as each subsequent category narrows the set of types that are members of their match set. + +In this trail, the formal parameters of candidates are compared in order (1st parameter, 2nd parameter, etc.) to search for +a candidate that has an unrivaled specificity. If such a formal parameter is found, the candidate it belongs to is chosen +as the resolved symbol. +Third Trial: Complexity Analysis +---------------------------------- + +A slight clarification: While category matching digests all the formal parameters of a candidate at once (order doesn't matter), +specificity comparison and complexity analysis operate on each formal parameter at a time. The following +is the final trial to disambiguate a symbol choice when a pair of formal parameters have the same hierarchical order. + +The complexity of a type is essentially its number of modifiers and depth of shape. The definition with the *highest* +complexity wins. Consider the following types: + +```nim +type + A[T] = object + B[T, H] = object +``` + +Note: The below examples are not exhaustive. -Some examples: +We shall say that: -.. code-block:: nim +1. `A[T]` has a higher complexity than `A` +2. `var A[T]` has a higher complexity than `A[T]` +3. `A[A[T]]` has a higher complexity than `A[T]` +4. `B[T, H]` has a higher complexity than `A[T]` (`A` and `B` are not compatible here, but convoluted versions of this exist) +5. `B[ptr T, H]` has a higher complexity than `B[T, H]` + +Some Examples +--------------- + + ```nim proc takesInt(x: int) = echo "int" proc takesInt[T](x: T) = echo "T" proc takesInt(x: int16) = echo "int16" @@ -2457,14 +2734,14 @@ Some examples: takesInt(y) # "int16" var z: range[0..4] = 0 takesInt(z) # "T" + ``` -If this algorithm returns "ambiguous" further disambiguation is performed: If the argument `a` matches both the parameter type `f` of `p` and `g` of `q` via a subtyping relation, the inheritance depth is taken into account: -.. code-block:: nim + ```nim type A = object of RootObj B = object of A @@ -2486,18 +2763,37 @@ into account: # but this is ambiguous: pp(c, c) + ``` Likewise, for generic matches, the most specialized generic type (that still matches) is preferred: -.. code-block:: nim + ```nim proc gen[T](x: ref ref T) = echo "ref ref T" proc gen[T](x: ref T) = echo "ref T" proc gen[T](x: T) = echo "T" var ri: ref int gen(ri) # "ref T" + ``` + +Type variables match +---------------------- + +When overload resolution is considering candidates, the type variable's definition +is not overlooked as it is used to define the formal parameter's type via variable substitution. + +For example: +```nim +type A +proc p[T: A](param: T) +proc p[T: object](param: T) +``` + +These signatures are not ambiguous for a concrete type of `A` even though the formal parameters match ("T" == "T"). +Instead `T` is treated as a variable in that (`T` ?= `T`) depending on the bound type of `T` at the time of +overload resolution. Overloading based on 'var T' @@ -2508,7 +2804,7 @@ in addition to the ordinary type checking, the argument is checked to be an `l-value`:idx:. `var T` matches better than just `T` then. -.. code-block:: nim + ```nim proc sayHi(x: int): string = # matches a non-var int result = $x @@ -2523,6 +2819,7 @@ the argument is checked to be an `l-value`:idx:. sayHello(3) # 3 # 13 + ``` Lazy type resolution for untyped @@ -2536,10 +2833,11 @@ in overloading resolution, it's essential to have a way to pass unresolved expressions to a template or macro. This is what the meta-type `untyped` accomplishes: -.. code-block:: nim + ```nim template rem(x: untyped) = discard rem unresolvedExpression(undeclaredIdentifier) + ``` A parameter of type `untyped` always matches any argument (as long as there is any argument passed to it). @@ -2547,12 +2845,13 @@ any argument passed to it). But one has to watch out because other overloads might trigger the argument's resolution: -.. code-block:: nim + ```nim template rem(x: untyped) = discard proc rem[T](x: T) = discard # undeclared identifier: 'unresolvedExpression' rem unresolvedExpression(undeclaredIdentifier) + ``` `untyped` and `varargs[untyped]` are the only metatype that are lazy in this sense, the other metatypes `typed` and `typedesc` are not lazy. @@ -2561,7 +2860,68 @@ metatypes `typed` and `typedesc` are not lazy. Varargs matching ---------------- -See `Varargs <#types-varargs>`_. +See [Varargs]. + + +iterable +-------- + +A called `iterator` yielding type `T` can be passed to a template or macro via +a parameter typed as `untyped` (for unresolved expressions) or the type class +`iterable` or `iterable[T]` (after type checking and overload resolution). + + ```nim + iterator iota(n: int): int = + for i in 0..<n: yield i + + template toSeq2[T](a: iterable[T]): seq[T] = + var ret: seq[T] + assert a.typeof is T + for ai in a: ret.add ai + ret + + assert iota(3).toSeq2 == @[0, 1, 2] + assert toSeq2(5..7) == @[5, 6, 7] + assert not compiles(toSeq2(@[1,2])) # seq[int] is not an iterable + assert toSeq2(items(@[1,2])) == @[1, 2] # but items(@[1,2]) is + ``` + + +Overload disambiguation +======================= + +For routine calls "overload resolution" is performed. There is a weaker form of +overload resolution called *overload disambiguation* that is performed when an +overloaded symbol is used in a context where there is additional type information +available. Let `p` be an overloaded symbol. These contexts are: + +- In a function call `q(..., p, ...)` when the corresponding formal parameter + of `q` is a `proc` type. If `q` itself is overloaded then the cartesian product + of every interpretation of `q` and `p` must be considered. +- In an object constructor `Obj(..., field: p, ...)` when `field` is a `proc` + type. Analogous rules exist for array/set/tuple constructors. +- In a declaration like `x: T = p` when `T` is a `proc` type. + +As usual, ambiguous matches produce a compile-time error. + +Named argument overloading +-------------------------- + +Routines with the same type signature can be called individually if +a parameter has different names between them. + + ```Nim + proc foo(x: int) = + echo "Using x: ", x + proc foo(y: int) = + echo "Using y: ", y + + foo(x = 2) # Using x: 2 + foo(y = 2) # Using y: 2 + ``` + +Not supplying the parameter name in such cases results in an +ambiguity error. Statements and expressions @@ -2595,11 +2955,12 @@ Discard statement Example: -.. code-block:: nim + ```nim proc p(x, y: int): int = result = x + y discard p(3, 4) # discard the return value of `p` + ``` The `discard` statement evaluates its expression for side-effects and throws the expression's resulting value away, and should only be used @@ -2611,31 +2972,34 @@ a static error. The return value can be ignored implicitly if the called proc/iterator has been declared with the `discardable`:idx: pragma: -.. code-block:: nim + ```nim proc p(x, y: int): int {.discardable.} = result = x + y p(3, 4) # now valid + ``` however the discardable pragma does not work on templates as templates substitute the AST in place. For example: -.. code-block:: nim + ```nim {.push discardable .} template example(): string = "https://nim-lang.org" {.pop.} example() + ``` This template will resolve into "https://nim-lang.org" which is a string literal and since {.discardable.} doesn't apply to literals, the compiler will error. An empty `discard` statement is often used as a null statement: -.. code-block:: nim + ```nim proc classify(s: string) = case s[0] of SymChars, '_': echo "an identifier" of '0'..'9': echo "a number" else: discard + ``` Void context @@ -2645,15 +3009,17 @@ In a list of statements, every expression except the last one needs to have the type `void`. In addition to this rule an assignment to the builtin `result` symbol also triggers a mandatory `void` context for the subsequent expressions: -.. code-block:: nim + ```nim proc invalid*(): string = result = "foo" "invalid" # Error: value of type 'string' has to be discarded + ``` -.. code-block:: nim + ```nim proc valid*(): string = let x = 317 "valid" + ``` Var statement @@ -2663,11 +3029,11 @@ Var statements declare new local and global variables and initialize them. A comma-separated list of variables can be used to specify variables of the same type: -.. code-block:: nim - + ```nim var a: int = 0 x, y, z: int + ``` If an initializer is given, the type can be omitted: the variable is then of the same type as the initializing expression. Variables are always initialized @@ -2685,26 +3051,28 @@ ref or pointer type nil procedural type nil sequence `@[]` string `""` -tuple[x: A, y: B, ...] (default(A), default(B), ...) +`tuple[x: A, y: B, ...]` (zeroDefault(A), zeroDefault(B), ...) (analogous for objects) -array[0..., T] [default(T), ...] -range[T] default(T); this may be out of the valid range -T = enum cast[T]\(0); this may be an invalid value +`array[0..., T]` `[zeroDefault(T), ...]` +`range[T]` default(T); this may be out of the valid range +T = enum `cast[T](0)`; this may be an invalid value ============================ ============================================== The implicit initialization can be avoided for optimization reasons with the `noinit`:idx: pragma: -.. code-block:: nim + ```nim var - a {.noInit.}: array[0..1023, char] + a {.noinit.}: array[0..1023, char] + ``` If a proc is annotated with the `noinit` pragma, this refers to its implicit `result` variable: -.. code-block:: nim + ```nim proc returnUndefinedValue: int {.noinit.} = discard + ``` The implicit initialization can also be prevented by the `requiresInit`:idx: @@ -2712,9 +3080,9 @@ type pragma. The compiler requires an explicit initialization for the object and all of its fields. However, it does a `control flow analysis`:idx: to prove the variable has been initialized and does not rely on syntactic properties: -.. code-block:: nim + ```nim type - MyObject = object {.requiresInit.} + MyObject {.requiresInit.} = object proc p() = # the following is valid: @@ -2724,37 +3092,46 @@ the variable has been initialized and does not rely on syntactic properties: else: x = a() # use x + ``` -`requiresInit` pragma can also be applyied to `distinct` types. +`requiresInit` pragma can also be applied to `distinct` types. Given the following distinct type definitions: -.. code-block:: nim + ```nim type - DistinctObject {.requiresInit, borrow: `.`.} = distinct MyObject + Foo = object + x: string + + DistinctFoo {.requiresInit, borrow: `.`.} = distinct Foo DistinctString {.requiresInit.} = distinct string + ``` The following code blocks will fail to compile: -.. code-block:: nim + ```nim var foo: DistinctFoo foo.x = "test" doAssert foo.x == "test" + ``` -.. code-block:: nim + ```nim var s: DistinctString s = "test" - doAssert s == "test" + doAssert string(s) == "test" + ``` -But these ones will compile successfully: +But these will compile successfully: -.. code-block:: nim + ```nim let foo = DistinctFoo(Foo(x: "test")) doAssert foo.x == "test" + ``` -.. code-block:: nim - let s = "test" - doAssert s == "test" + ```nim + let s = DistinctString("test") + doAssert string(s) == "test" + ``` Let statement ------------- @@ -2772,18 +3149,61 @@ when they are declared. The only exception to this is if the `{.importc.}` pragma (or any of the other `importX` pragmas) is applied, in this case the value is expected to come from native code, typically a C/C++ `const`. +Special identifier `_` (underscore) +----------------------------------- + +The identifier `_` has a special meaning in declarations. +Any definition with the name `_` will not be added to scope, meaning the +definition is evaluated, but cannot be used. As a result the name `_` can be +indefinitely redefined. + + ```nim + let _ = 123 + echo _ # error + let _ = 456 # compiles + ``` Tuple unpacking --------------- -In a `var` or `let` statement tuple unpacking can be performed. The special -identifier `_` can be used to ignore some parts of the tuple: +In a `var`, `let` or `const` statement tuple unpacking can be performed. +The special identifier `_` can be used to ignore some parts of the tuple: + + ```nim + proc returnsTuple(): (int, int, int) = (4, 2, 3) + + let (x, _, z) = returnsTuple() + ``` + +This is treated as syntax sugar for roughly the following: -.. code-block:: nim - proc returnsTuple(): (int, int, int) = (4, 2, 3) + ```nim + let + tmpTuple = returnsTuple() + x = tmpTuple[0] + z = tmpTuple[2] + ``` - let (x, _, z) = returnsTuple() +For `var` or `let` statements, if the value expression is a tuple literal, +each expression is directly expanded into an assignment without the use of +a temporary variable. + ```nim + let (x, y, z) = (1, 2, 3) + # becomes + let + x = 1 + y = 2 + z = 3 + ``` + +Tuple unpacking can also be nested: + + ```nim + proc returnsNestedTuple(): (int, (int, int), int, int) = (4, (5, 7), 2, 3) + + let (x, (_, y), _, z) = returnsNestedTuple() + ``` Const section @@ -2791,16 +3211,25 @@ Const section A const section declares constants whose values are constant expressions: -.. code-block:: + ```nim import std/[strutils] const roundPi = 3.1415 constEval = contains("abc", 'b') # computed at compile time! + ``` Once declared, a constant's symbol can be used as a constant expression. -See `Constants and Constant Expressions <#constants-and-constant-expressions>`_ -for details. +The value part of a constant declaration opens a new scope for each constant, +so no symbols declared in the constant value are accessible outside of it. + + ```nim + const foo = (var a = 1; a) + const bar = a # error + let baz = a # error + ``` + +See [Constants and Constant Expressions] for details. Static statement/expression --------------------------- @@ -2808,14 +3237,27 @@ Static statement/expression A static statement/expression explicitly requires compile-time execution. Even some code that has side effects is permitted in a static block: -.. code-block:: - + ```nim static: echo "echo at compile time" + ``` + +`static` can also be used like a routine. + + ```nim + proc getNum(a: int): int = a + + # Below calls "echo getNum(123)" at compile time. + static: + echo getNum(123) + + # Below call evaluates the "getNum(123)" at compile time, but its + # result gets used at run time. + echo static(getNum(123)) + ``` There are limitations on what Nim code can be executed at compile time; -see `Restrictions on Compile-Time Execution -<#restrictions-on-compileminustime-execution>`_ for details. +see [Restrictions on Compile-Time Execution] for details. It's a static error if the compiler cannot execute the block at compile time. @@ -2825,8 +3267,7 @@ If statement Example: -.. code-block:: nim - + ```nim var name = readLine(stdin) if name == "Andreas": @@ -2835,10 +3276,11 @@ Example: echo "Don't you have a name?" else: echo "Boring name..." + ``` The `if` statement is a simple way to make a branch in the control flow: The expression after the keyword `if` is evaluated, if it is true -the corresponding statements after the `:` are executed. Otherwise +the corresponding statements after the `:` are executed. Otherwise, the expression after the `elif` is evaluated (if there is an `elif` branch), if it is true the corresponding statements after the `:` are executed. This goes on until the last `elif`. If all @@ -2851,26 +3293,28 @@ corresponding *then* block. For visualization purposes the scopes have been enclosed in `{| |}` in the following example: -.. code-block:: nim + ```nim if {| (let m = input =~ re"(\w+)=\w+"; m.isMatch): echo "key ", m[0], " value ", m[1] |} elif {| (let m = input =~ re""; m.isMatch): echo "new m in this scope" |} else: {| echo "m not declared here" |} + ``` Case statement -------------- Example: -.. code-block:: nim - - case readline(stdin) + ```nim + let line = readline(stdin) + case line of "delete-everything", "restart-computer": echo "permission denied" of "go-for-a-walk": echo "please yourself" - else: echo "unknown command" + elif line.len == 0: echo "empty" # optional, must come after `of` branches + else: echo "unknown command" # ditto # indentation of the branches is also allowed; and so is an optional colon # after the selecting expression: @@ -2879,21 +3323,29 @@ Example: echo "permission denied" of "go-for-a-walk": echo "please yourself" else: echo "unknown command" + ``` -The `case` statement is similar to the if statement, but it represents +The `case` statement is similar to the `if` statement, but it represents a multi-branch selection. The expression after the keyword `case` is evaluated and if its value is in a *slicelist* the corresponding statements (after the `of` keyword) are executed. If the value is not in any -given *slicelist* the `else` part is executed. If there is no `else` -part and not all possible values that `expr` can hold occur in a -*slicelist*, a static error occurs. This holds only for expressions of -ordinal types. "All possible values" of `expr` are determined by `expr`'s -type. To suppress the static error an `else` part with an -empty `discard` statement should be used. +given *slicelist*, trailing `elif` and `else` parts are executed using same +semantics as for `if` statement, and `elif` is handled just like `else: if`. +If there are no `else` or `elif` parts and not +all possible values that `expr` can hold occur in a *slicelist*, a static error occurs. +This holds only for expressions of ordinal types. +"All possible values" of `expr` are determined by `expr`'s type. +To suppress the static error an `else: discard` should be used. + +Only ordinal types, floats, strings and cstrings are allowed as values +in case statements. For non-ordinal types, it is not possible to list every possible value and so these always require an `else` part. +An exception to this rule is for the `string` type, which currently doesn't +require a trailing `else` or `elif` branch; it's unspecified whether this will +keep working in future versions. Because case statements are checked for exhaustiveness during semantic analysis, the value in every `of` branch must be a constant expression. @@ -2903,7 +3355,7 @@ As a special semantic extension, an expression in an `of` branch of a case statement may evaluate to a set or array constructor; the set or array is then expanded into a list of its elements: -.. code-block:: nim + ```nim const SymChars: set[char] = {'a'..'z', 'A'..'Z', '\x80'..'\xFF'} @@ -2919,11 +3371,12 @@ expanded into a list of its elements: of 'a'..'z', 'A'..'Z', '\x80'..'\xFF', '_': echo "an identifier" of '0'..'9': echo "a number" else: echo "other" + ``` The `case` statement doesn't produce an l-value, so the following example won't work: -.. code-block:: nim + ```nim type Foo = ref object x: seq[string] @@ -2938,16 +3391,18 @@ won't work: var foo = Foo(x: @[]) foo.get_x().add("asd") + ``` -This can be fixed by explicitly using `return`: +This can be fixed by explicitly using `result` or `return`: -.. code-block:: nim + ```nim proc get_x(x: Foo): var seq[string] = case true of true: - return x.x + result = x.x else: - return x.x + result = x.x + ``` When statement @@ -2955,8 +3410,7 @@ When statement Example: -.. code-block:: nim - + ```nim when sizeof(int) == 2: echo "running on a 16 bit system!" elif sizeof(int) == 4: @@ -2965,6 +3419,7 @@ Example: echo "running on a 64 bit system!" else: echo "cannot happen!" + ``` The `when` statement is almost identical to the `if` statement with some exceptions: @@ -2989,7 +3444,7 @@ compile-time and the executable. Example: -.. code-block:: nim + ```nim proc someProcThatMayRunInCompileTime(): bool = when nimvm: # This branch is taken at compile time. @@ -3001,6 +3456,7 @@ Example: let rtValue = someProcThatMayRunInCompileTime() assert(ctValue == true) assert(rtValue == false) + ``` A `when nimvm` statement must meet the following requirements: @@ -3017,16 +3473,18 @@ Return statement Example: -.. code-block:: nim - return 40+2 + ```nim + return 40 + 2 + ``` The `return` statement ends the execution of the current procedure. It is only allowed in procedures. If there is an `expr`, this is syntactic sugar for: -.. code-block:: nim + ```nim result = expr return result + ``` `return` without an expression is a short notation for `return result` if @@ -3034,9 +3492,10 @@ the proc has a return type. The `result`:idx: variable is always the return value of the procedure. It is automatically declared by the compiler. As all variables, `result` is initialized to (binary) zero: -.. code-block:: nim + ```nim proc returnZero(): int = # implicitly returns 0 + ``` Yield statement @@ -3044,14 +3503,15 @@ Yield statement Example: -.. code-block:: nim + ```nim yield (1, 2, 3) + ``` The `yield` statement is used instead of the `return` statement in iterators. It is only valid in iterators. Execution is returned to the body of the for loop that called the iterator. Yield does not end the iteration process, but the execution is passed back to the iterator if the next iteration -starts. See the section about iterators (`Iterators and the for statement`_) +starts. See the section about iterators ([Iterators and the for statement]) for further information. @@ -3060,7 +3520,7 @@ Block statement Example: -.. code-block:: nim + ```nim var found = false block myblock: for i in 0..3: @@ -3069,6 +3529,7 @@ Example: found = true break myblock # leave the block, in this case both for-loops echo found + ``` The block statement is a means to group statements to a (named) `block`. Inside the block, the `break` statement is allowed to leave the block @@ -3081,8 +3542,9 @@ Break statement Example: -.. code-block:: nim + ```nim break + ``` The `break` statement is used to leave a block immediately. If `symbol` is given, it is the name of the enclosing block that is to be left. If it is @@ -3094,12 +3556,13 @@ While statement Example: -.. code-block:: nim + ```nim echo "Please tell me your password:" var pw = readLine(stdin) while pw != "12345": echo "Wrong password! Next try:" pw = readLine(stdin) + ``` The `while` statement is executed until the `expr` evaluates to false. @@ -3114,20 +3577,22 @@ A `continue` statement leads to the immediate next iteration of the surrounding loop construct. It is only allowed within a loop. A continue statement is syntactic sugar for a nested block: -.. code-block:: nim + ```nim while expr1: stmt1 continue stmt2 + ``` Is equivalent to: -.. code-block:: nim + ```nim while expr1: block myBlockName: stmt1 break myBlockName stmt2 + ``` Assembler statement @@ -3136,9 +3601,9 @@ Assembler statement The direct embedding of assembler code into Nim code is supported by the unsafe `asm` statement. Identifiers in the assembler code that refer to Nim identifiers shall be enclosed in a special character which can be -specified in the statement's pragmas. The default special character is ``'`'``: +specified in the statement's pragmas. The default special character is `'\`'`: -.. code-block:: nim + ```nim {.push stackTrace:off.} proc addInt(a, b: int): int = # a in eax, and b in edx @@ -3150,10 +3615,11 @@ specified in the statement's pragmas. The default special character is ``'`'``: theEnd: """ {.pop.} + ``` If the GNU assembler is used, quotes and newlines are inserted automatically: -.. code-block:: nim + ```nim proc addInt(a, b: int): int = asm """ addl %%ecx, %%eax @@ -3163,10 +3629,11 @@ If the GNU assembler is used, quotes and newlines are inserted automatically: :"=a"(`result`) :"a"(`a`), "c"(`b`) """ + ``` Instead of: -.. code-block:: nim + ```nim proc addInt(a, b: int): int = asm """ "addl %%ecx, %%eax\n" @@ -3176,23 +3643,25 @@ Instead of: :"=a"(`result`) :"a"(`a`), "c"(`b`) """ + ``` Using statement --------------- -The using statement provides syntactic convenience in modules where +The `using` statement provides syntactic convenience in modules where the same parameter names and types are used over and over. Instead of: -.. code-block:: nim + ```nim proc foo(c: Context; n: Node) = ... proc bar(c: Context; n: Node, counter: int) = ... proc baz(c: Context; n: Node) = ... + ``` One can tell the compiler about the convention that a parameter of name `c` should default to type `Context`, `n` should default to `Node` etc.: -.. code-block:: nim + ```nim using c: Context n: Node @@ -3206,6 +3675,7 @@ name `c` should default to type `Context`, `n` should default to # 'c' is inferred to be of the type 'Context' # 'n' is inferred to be of the type 'Node' # But 'x' and 'y' are of type 'int'. + ``` The `using` section uses the same indentation based grouping syntax as a `var` or `let` section. @@ -3220,27 +3690,28 @@ that are explicitly typed is possible and requires a semicolon between them. If expression ------------- -An `if expression` is almost like an if statement, but it is an expression. -This feature is similar to `ternary operators` in other languages. +An `if` expression is almost like an if statement, but it is an expression. +This feature is similar to *ternary operators* in other languages. Example: -.. code-block:: nim + ```nim var y = if x > 8: 9 else: 10 + ``` -An if expression always results in a value, so the `else` part is -required. `Elif` parts are also allowed. +An `if` expression always results in a value, so the `else` part is +required. `elif` parts are also allowed. When expression --------------- -Just like an `if expression`, but corresponding to the when statement. +Just like an `if` expression, but corresponding to the `when` statement. Case expression --------------- -The `case expression` is again very similar to the case statement: +The `case` expression is again very similar to the case statement: -.. code-block:: nim + ```nim var favoriteFood = case animal of "dog": "bones" of "cat": "mice" @@ -3248,6 +3719,7 @@ The `case expression` is again very similar to the case statement: else: echo "I'm not sure what to serve, but everybody loves ice cream" "ice cream" + ``` As seen in the above example, the case expression can also introduce side effects. When multiple statements are given for a branch, Nim will use @@ -3256,28 +3728,30 @@ the last expression as the result value. Block expression ---------------- -A `block expression` is almost like a block statement, but it is an expression +A `block` expression is almost like a block statement, but it is an expression that uses the last expression under the block as the value. It is similar to the statement list expression, but the statement list expression does not open a new block scope. -.. code-block:: nim + ```nim let a = block: var fib = @[0, 1] for i in 0..10: fib.add fib[^1] + fib[^2] fib + ``` Table constructor ----------------- A table constructor is syntactic sugar for an array constructor: -.. code-block:: nim + ```nim {"key1": "value1", "key2", "key3": "value2"} # is the same as: [("key1", "value1"), ("key2", "value2"), ("key3", "value2")] + ``` The empty table can be written `{:}` (in contrast to the empty set @@ -3310,22 +3784,19 @@ can be used to convert from floating-point to integer or vice versa. Type conversion can also be used to disambiguate overloaded routines: -.. code-block:: nim - + ```nim proc p(x: int) = echo "int" proc p(x: string) = echo "string" let procVar = (proc(x: string))(p) procVar("a") + ``` Since operations on unsigned numbers wrap around and are unchecked so are type conversions to unsigned integers and between unsigned integers. The rationale for this is mostly better interoperability with the C Programming language when algorithms are ported from C to Nim. -Exception: Values that are converted to an unsigned type at compile time -are checked so that code like `byte(-1)` does not compile. - **Note**: Historically the operations were unchecked and the conversions were sometimes checked but starting with the revision 1.0.4 of this document and the language implementation the @@ -3339,21 +3810,26 @@ Type casts as if it would be of another type. Type casts are only needed for low-level programming and are inherently unsafe. -.. code-block:: nim + ```nim cast[int](x) + ``` The target type of a cast must be a concrete type, for instance, a target type that is a type class (which is non-concrete) would be invalid: -.. code-block:: nim + ```nim type Foo = int or float var x = cast[Foo](1) # Error: cannot cast to a non concrete type: 'Foo' + ``` Type casts should not be confused with *type conversions,* as mentioned in the prior section. Unlike type conversions, a type cast cannot change the underlying -bit pattern of the data being casted (aside from that the size of the target type +bit pattern of the data being cast (aside from that the size of the target type may differ from the source type). Casting resembles *type punning* in other -languages or C++'s `reinterpret_cast` and `bit_cast` features. +languages or C++'s `reinterpret_cast`:cpp: and `bit_cast`:cpp: features. + +If the size of the target type is larger than the size of the source type, +the remaining memory is zeroed. The addr operator ----------------- @@ -3362,11 +3838,11 @@ location is `T`, the `addr` operator result is of the type `ptr T`. An address is always an untraced reference. Taking the address of an object that resides on the stack is **unsafe**, as the pointer may live longer than the object on the stack and can thus reference a non-existing object. One can get -the address of variables, but one can't use it on variables declared through -`let` statements: - -.. code-block:: nim +the address of variables. For easier interoperability with other compiled languages +such as C, retrieving the address of a `let` variable, a parameter, +or a `for` loop variable can be accomplished too: + ```nim let t1 = "Hello" var t2 = t1 @@ -3375,23 +3851,19 @@ the address of variables, but one can't use it on variables declared through # --> ref 0x7fff6b71b670 --> 0x10bb81050"Hello" echo cast[ptr string](t3)[] # --> Hello - # The following line doesn't compile: + # The following line also works echo repr(addr(t1)) - # Error: expression has no address - + ``` The unsafeAddr operator ----------------------- -For easier interoperability with other compiled languages such as C, retrieving -the address of a `let` variable, a parameter, or a `for` loop variable can -be accomplished by using the `unsafeAddr` operation: - -.. code-block:: nim +The `unsafeAddr` operator is a deprecated alias for the `addr` operator: + ```nim let myArray = [1, 2, 3] foreignProcThatTakesAnAddr(unsafeAddr myArray) - + ``` Procedures ========== @@ -3406,7 +3878,7 @@ until either the beginning of the parameter list, a semicolon separator, or an already typed parameter, is reached. The semicolon can be used to make separation of types and subsequent identifiers more distinct. -.. code-block:: nim + ```nim # Using only commas proc foo(a, b: int, c, d: bool): int @@ -3415,31 +3887,34 @@ separation of types and subsequent identifiers more distinct. # Will fail: a is untyped since ';' stops type propagation. proc foo(a; b: int; c, d: bool): int + ``` A parameter may be declared with a default value which is used if the caller -does not provide a value for the argument. +does not provide a value for the argument. The value will be reevaluated +every time the function is called. -.. code-block:: nim - # b is optional with 47 as its default value + ```nim + # b is optional with 47 as its default value. proc foo(a: int, b: int = 47): int + ``` Parameters can be declared mutable and so allow the proc to modify those arguments, by using the type modifier `var`. -.. code-block:: nim + ```nim # "returning" a value to the caller through the 2nd argument # Notice that the function uses no actual return value at all (ie void) proc foo(inp: int, outp: var int) = outp = inp + 47 + ``` -If the proc declaration has no body, it is a `forward`:idx: declaration. If the -proc returns a value, the procedure body can access an implicitly declared +If the proc declaration doesn't have a body, it is a `forward`:idx: declaration. +If the proc returns a value, the procedure body can access an implicitly declared variable named `result`:idx: that represents the return value. Procs can be overloaded. The overloading resolution algorithm determines which proc is the best match for the arguments. Example: -.. code-block:: nim - + ```nim proc toLower(c: char): char = # toLower for characters if c in {'A'..'Z'}: result = chr(ord(c) + (ord('a') - ord('A'))) @@ -3450,10 +3925,11 @@ best match for the arguments. Example: result = newString(len(s)) for i in 0..len(s) - 1: result[i] = toLower(s[i]) # calls toLower for characters; no recursion! + ``` -Calling a procedure can be done in many different ways: +Calling a procedure can be done in many ways: -.. code-block:: nim + ```nim proc callme(x, y: int, s: string = "", c: char, b: bool = false) = ... # call with positional arguments # parameter bindings: @@ -3464,16 +3940,18 @@ Calling a procedure can be done in many different ways: callme(c='\t', y=1, x=0) # (x=0, y=1, s="", c='\t', b=false) # call as a command statement: no () needed: callme 0, 1, "abc", '\t' # (x=0, y=1, s="abc", c='\t', b=false) + ``` A procedure may call itself recursively. `Operators`:idx: are procedures with a special operator symbol as identifier: -.. code-block:: nim + ```nim proc `$` (x: int): string = # converts an integer to a string; this is a prefix operator. result = intToStr(x) + ``` Operators with one parameter are prefix operators, operators with two parameters are infix operators. (However, the parser distinguishes these from @@ -3481,15 +3959,16 @@ the operator's position within an expression.) There is no way to declare postfix operators: all postfix operators are built-in and handled by the grammar explicitly. -Any operator can be called like an ordinary proc with the '`opr`' +Any operator can be called like an ordinary proc with the \`opr\` notation. (Thus an operator can have more than two parameters): -.. code-block:: nim + ```nim proc `*+` (a, b, c: int): int = # Multiply and add result = a * b + c assert `*+`(3, 4, 6) == `+`(`*`(a, b), c) + ``` Export marker @@ -3498,8 +3977,7 @@ Export marker If a declared symbol is marked with an `asterisk`:idx: it is exported from the current module: -.. code-block:: nim - + ```nim proc exportedEcho*(s: string) = echo s proc `*`*(a: string; b: int): string = result = newStringOfCap(a.len * b) @@ -3510,24 +3988,25 @@ current module: type ExportedType* = object exportedField*: int + ``` Method call syntax ------------------ -For object-oriented programming, the syntax `obj.method(args)` can be used -instead of `method(obj, args)`. The parentheses can be omitted if there are no -remaining arguments: `obj.len` (instead of `len(obj)`). +For object-oriented programming, the syntax `obj.methodName(args)` can be used +instead of `methodName(obj, args)`. The parentheses can be omitted if +there are no remaining arguments: `obj.len` (instead of `len(obj)`). This method call syntax is not restricted to objects, it can be used to supply any type of first argument for procedures: -.. code-block:: nim - + ```nim echo "abc".len # is the same as echo len "abc" echo "abc".toUpper() echo {'a', 'b', 'c'}.card stdout.writeLine("Hallo") # the same as writeLine(stdout, "Hallo") + ``` Another way to look at the method call syntax is that it provides the missing postfix notation. @@ -3536,8 +4015,7 @@ The method call syntax conflicts with explicit generic instantiations: `p[T](x)` cannot be written as `x.p[T]` because `x.p[T]` is always parsed as `(x.p)[T]`. -See also: `Limitations of the method call syntax -<#templates-limitations-of-the-method-call-syntax>`_. +See also: [Limitations of the method call syntax]. The `[: ]` notation has been designed to mitigate this issue: `x.p[:T]` is rewritten by the parser to `p[T](x)`, `x.p[:T](y)` is rewritten to @@ -3551,7 +4029,7 @@ Nim has no need for *get-properties*: Ordinary get-procedures that are called with the *method call syntax* achieve the same. But setting a value is different; for this, a special setter syntax is needed: -.. code-block:: nim + ```nim # Module asocket type Socket* = ref object of RootObj @@ -3570,24 +4048,26 @@ different; for this, a special setter syntax is needed: ## `host` because the builtin dot access is preferred if it is ## available: s.host + ``` -.. code-block:: nim + ```nim # module B import asocket var s: Socket new s s.host = 34 # same as `host=`(s, 34) + ``` A proc defined as `f=` (with the trailing `=`) is called a `setter`:idx:. A setter can be called explicitly via the common backticks notation: -.. code-block:: nim - + ```nim proc `f=`(x: MyObject; value: string) = discard `f=`(myObject, "value") + ``` `f=` can be called implicitly in the pattern @@ -3608,7 +4088,7 @@ means `echo f 1, f 2` is parsed as `echo(f(1), f(2))` and not as `echo(f(1, f(2)))`. The method call syntax may be used to provide one more argument in this case: -.. code-block:: nim + ```nim proc optarg(x: int, y: int = 0): int = x + y proc singlearg(x: int): int = 20*x @@ -3618,9 +4098,10 @@ more argument in this case: let x = optarg(1, optarg 8) # traditional procedure call with 2 arguments let y = 1.optarg optarg 8 # same thing as above, w/o the parenthesis assert x == y + ``` The command invocation syntax also can't have complex expressions as arguments. -For example: (`anonymous procs <#procedures-anonymous-procs>`_), `if`, +For example: [anonymous procedures], `if`, `case` or `try`. Function calls with no arguments still need () to distinguish between a call and the function itself as a first-class value. @@ -3637,45 +4118,85 @@ the closure and its enclosing scope (i.e. any modifications made to them are visible in both places). The closure environment may be allocated on the heap or on the stack if the compiler determines that this would be safe. -Creating closures in loops -~~~~~~~~~~~~~~~~~~~~~~~~~~ +### Creating closures in loops Since closures capture local variables by reference it is often not wanted -behavior inside loop bodies. See `closureScope -<system.html#closureScope.t,untyped>`_ and `capture -<sugar.html#capture.m,varargs[typed],untyped>`_ for details on how to change this behavior. +behavior inside loop bodies. See [closureScope]( +system.html#closureScope.t,untyped) and [capture]( +sugar.html#capture.m,varargs[typed],untyped) for details on how to change this behavior. -Anonymous Procs ---------------- +Anonymous procedures +-------------------- Unnamed procedures can be used as lambda expressions to pass into other procedures: -.. code-block:: nim + ```nim var cities = @["Frankfurt", "Tokyo", "New York", "Kyiv"] - cities.sort(proc (x,y: string): int = - cmp(x.len, y.len)) + cities.sort(proc (x, y: string): int = + cmp(x.len, y.len)) + ``` Procs as expressions can appear both as nested procs and inside top-level -executable code. The `sugar <sugar.html>`_ module contains the `=>` macro +executable code. The [sugar](sugar.html) module contains the `=>` macro which enables a more succinct syntax for anonymous procedures resembling lambdas as they are in languages like JavaScript, C#, etc. +Do notation +----------- + +As a special convenience notation that keeps most elements of a +regular proc expression, the `do` keyword can be used to pass +anonymous procedures to routines: + + ```nim + var cities = @["Frankfurt", "Tokyo", "New York", "Kyiv"] + + sort(cities) do (x, y: string) -> int: + cmp(x.len, y.len) + + # Less parentheses using the method plus command syntax: + cities = cities.map do (x: string) -> string: + "City of " & x + ``` + +`do` is written after the parentheses enclosing the regular proc parameters. +The proc expression represented by the `do` block is appended to the routine +call as the last argument. In calls using the command syntax, the `do` block +will bind to the immediately preceding expression rather than the command call. + +`do` with a parameter list or pragma list corresponds to an anonymous `proc`, +however `do` without parameters or pragmas is treated as a normal statement +list. This allows macros to receive both indented statement lists as an +argument in inline calls, as well as a direct mirror of Nim's routine syntax. + + ```nim + # Passing a statement list to an inline macro: + macroResults.add quote do: + if not `ex`: + echo `info`, ": Check failed: ", `expString` + + # Processing a routine definition in a macro: + rpc(router, "add") do (a, b: int) -> int: + result = a + b + ``` Func ---- The `func` keyword introduces a shortcut for a `noSideEffect`:idx: proc. -.. code-block:: nim + ```nim func binarySearch[T](a: openArray[T]; elem: T): int + ``` Is short for: -.. code-block:: nim + ```nim proc binarySearch[T](a: openArray[T]; elem: T): int {.noSideEffect.} + ``` @@ -3689,11 +4210,11 @@ Type bound operators A type bound operator is a `proc` or `func` whose name starts with `=` but isn't an operator (i.e. containing only symbols, such as `==`). These are unrelated to setters -(see `properties <manual.html#procedures-properties>`_), which instead end in `=`. +(see [Properties]), which instead end in `=`. A type bound operator declared for a type applies to the type regardless of whether the operator is in scope (including if it is private). -.. code-block:: nim + ```nim # foo.nim: var witness* = 0 type Foo[T] = object @@ -3713,37 +4234,50 @@ the operator is in scope (including if it is private). doAssert witness == 2 # will still be called upon exiting scope doAssert witness == 3 + ``` + +Type bound operators are: +`=destroy`, `=copy`, `=sink`, `=trace`, `=deepcopy`, `=wasMoved`, `=dup`. + +These operations can be *overridden* instead of *overloaded*. This means that +the implementation is automatically lifted to structured types. For instance, +if the type `T` has an overridden assignment operator `=`, this operator is +also used for assignments of the type `seq[T]`. -Type bound operators currently include: -`=destroy`, `=copy`, `=sink`, `=trace`, `=dispose`, `=deepcopy` -(some of which are still implementation defined and not yet documented). +Since these operations are bound to a type, they have to be bound to a +nominal type for reasons of simplicity of implementation; this means an +overridden `deepCopy` for `ref T` is really bound to `T` and not to `ref T`. +This also means that one cannot override `deepCopy` for both `ptr T` and +`ref T` at the same time, instead a distinct or object helper type has to be +used for one pointer type. For more details on some of those procs, see -`lifetimeminustracking-hooks <destructors.html#lifetimeminustracking-hooks>`_. +[Lifetime-tracking hooks](destructors.html#lifetimeminustracking-hooks). Nonoverloadable builtins ------------------------ The following built-in procs cannot be overloaded for reasons of implementation -simplicity (they require specialized semantic checking):: +simplicity (they require specialized semantic checking): - declared, defined, definedInScope, compiles, sizeof, - is, shallowCopy, getAst, astToStr, spawn, procCall + declared, defined, definedInScope, compiles, sizeof, + is, shallowCopy, getAst, astToStr, spawn, procCall -Thus they act more like keywords than like ordinary identifiers; unlike a +Thus, they act more like keywords than like ordinary identifiers; unlike a keyword however, a redefinition may `shadow`:idx: the definition in -the `system` module. From this list the following should not be written in dot +the [system](system.html) module. +From this list the following should not be written in dot notation `x.f` since `x` cannot be type-checked before it gets passed -to `f`:: +to `f`: - declared, defined, definedInScope, compiles, getAst, astToStr + declared, defined, definedInScope, compiles, getAst, astToStr Var parameters -------------- The type of a parameter may be prefixed with the `var` keyword: -.. code-block:: nim + ```nim proc divmod(a, b: int; res, remainder: var int) = res = a div b remainder = a mod b @@ -3754,6 +4288,7 @@ The type of a parameter may be prefixed with the `var` keyword: divmod(8, 5, x, y) # modifies x and y assert x == 1 assert y == 3 + ``` In the example, `res` and `remainder` are `var parameters`. Var parameters can be modified by the procedure and the changes are @@ -3761,7 +4296,7 @@ visible to the caller. The argument passed to a var parameter has to be an l-value. Var parameters are implemented as hidden pointers. The above example is equivalent to: -.. code-block:: nim + ```nim proc divmod(a, b: int; res, remainder: ptr int) = res[] = a div b remainder[] = a mod b @@ -3771,11 +4306,12 @@ above example is equivalent to: divmod(8, 5, addr(x), addr(y)) assert x == 1 assert y == 3 + ``` In the examples, var parameters or pointers are used to provide two return values. This can be done in a cleaner way by returning a tuple: -.. code-block:: nim + ```nim proc divmod(a, b: int): tuple[res, remainder: int] = (a div b, a mod b) @@ -3783,13 +4319,15 @@ return values. This can be done in a cleaner way by returning a tuple: assert t.res == 1 assert t.remainder == 3 + ``` One can use `tuple unpacking`:idx: to access the tuple's fields: -.. code-block:: nim + ```nim var (x, y) = divmod(8, 5) # tuple unpacking assert x == 1 assert y == 3 + ``` **Note**: `var` parameters are never necessary for efficient parameter @@ -3803,7 +4341,7 @@ Var return type A proc, converter, or iterator may return a `var` type which means that the returned value is an l-value and can be modified by the caller: -.. code-block:: nim + ```nim var g = 0 proc writeAccessToG(): var int = @@ -3811,36 +4349,39 @@ returned value is an l-value and can be modified by the caller: writeAccessToG() = 6 assert g == 6 + ``` It is a static error if the implicitly introduced pointer could be used to access a location beyond its lifetime: -.. code-block:: nim + ```nim proc writeAccessToG(): var int = var g = 0 result = g # Error! + ``` For iterators, a component of a tuple return type can have a `var` type too: -.. code-block:: nim + ```nim iterator mpairs(a: var seq[string]): tuple[key: int, val: var string] = for i in 0..a.high: yield (i, a[i]) + ``` In the standard library every name of a routine that returns a `var` type starts with the prefix `m` per convention. -.. include:: manual/var_t_return.rst +.. include:: manual/var_t_return.md -Future directions -~~~~~~~~~~~~~~~~~ +### Future directions Later versions of Nim can be more precise about the borrowing rule with a syntax like: -.. code-block:: nim + ```nim proc foo(other: Y; container: var X): var T from container + ``` Here `var T from container` explicitly exposes that the location is derived from the second parameter (called @@ -3866,7 +4407,7 @@ receives a hidden mutable parameter representing `result`. Informally: -.. code-block:: nim + ```nim proc p(): BigT = ... var x = p() @@ -3878,6 +4419,7 @@ Informally: var x; p(x) p(x) + ``` Let `T`'s be `p`'s return type. NRVO applies for `T` @@ -3887,8 +4429,7 @@ in other words, it applies for "big" structures. If `p` can raise an exception, NRVO applies regardless. This can produce observable differences in behavior: -.. code-block:: nim - + ```nim type BigT = array[16, int] @@ -3905,23 +4446,59 @@ observable differences in behavior: doAssert x == [0, 1, 2, 3, 4, 5, 6, 7, 0, 0, 0, 0, 0, 0, 0, 0] main() + ``` -However, the current implementation produces a warning in these cases. -There are different ways to deal with this warning: - -1. Disable the warning via `{.push warning[ObservableStores]: off.}` ... `{.pop.}`. - Then one may need to ensure that `p` only raises *before* any stores to `result` - happen. - -2. One can use a temporary helper variable, for example instead of `x = p(8)` - use `let tmp = p(8); x = tmp`. +The compiler can produce a warning in these cases, however this behavior is +turned off by default. It can be enabled for a section of code via the +`warning[ObservableStores]` and `push`/`pop` pragmas. Take the above code +as an example: + ```nim + {.push warning[ObservableStores]: on.} + main() + {.pop.} + ``` Overloading of the subscript operator ------------------------------------- -The `[]` subscript operator for arrays/openarrays/sequences can be overloaded. +The `[]` subscript operator for arrays/openarrays/sequences can be overloaded +for any type (with some exceptions) by defining a routine with the name `[]`. + + ```nim + type Foo = object + data: seq[int] + + proc `[]`(foo: Foo, i: int): int = + result = foo.data[i] + + let foo = Foo(data: @[1, 2, 3]) + echo foo[1] # 2 + ``` + +Assignment to subscripts can also be overloaded by naming a routine `[]=`, +which has precedence over assigning to the result of `[]`. + + ```nim + type Foo = object + data: seq[int] + + proc `[]`(foo: Foo, i: int): int = + result = foo.data[i] + proc `[]=`(foo: var Foo, i: int, val: int) = + foo.data[i] = val + + var foo = Foo(data: @[1, 2, 3]) + echo foo[1] # 2 + foo[1] = 5 + echo foo.data # @[1, 5, 3] + echo foo[1] # 5 + ``` + +Overloads of the subscript operator cannot be applied to routine or type +symbols themselves, as this conflicts with the syntax for instantiating +generic parameters, i.e. `foo[int](1, 2, 3)` or `Foo[int]`. Methods @@ -3931,7 +4508,7 @@ Procedures always use static dispatch. Methods use dynamic dispatch. For dynamic dispatch to work on an object it should be a reference type. -.. code-block:: nim + ```nim type Expression = ref object of RootObj ## abstract base class for an expression Literal = ref object of Expression @@ -3959,6 +4536,7 @@ type. result.b = b echo eval(newPlus(newPlus(newLit(1), newLit(2)), newLit(4))) + ``` In the example the constructors `newLit` and `newPlus` are procs because they should use static binding, but `eval` is a method because it @@ -3978,20 +4556,18 @@ Multi-methods -------------- **Note:** Starting from Nim 0.20, to use multi-methods one must explicitly pass -`--multimethods:on` when compiling. +`--multimethods:on`:option: when compiling. In a multi-method, all parameters that have an object type are used for the dispatching: -.. code-block:: nim - :test: "nim c --multiMethods:on $1" - + ```nim test = "nim c --multiMethods:on $1" type Thing = ref object of RootObj Unit = ref object of Thing x: int - method collide(a, b: Thing) {.inline.} = + method collide(a, b: Thing) {.base, inline.} = quit "to override!" method collide(a: Thing, b: Unit) {.inline.} = @@ -4004,6 +4580,7 @@ dispatching: new a new b collide(a, b) # output: 2 + ``` Inhibit dynamic method resolution via procCall ----------------------------------------------- @@ -4012,9 +4589,7 @@ Dynamic method resolution can be inhibited via the builtin `system.procCall`:idx This is somewhat comparable to the `super`:idx: keyword that traditional OOP languages offer. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Thing = ref object of RootObj Unit = ref object of Thing @@ -4027,6 +4602,7 @@ languages offer. # Call the base method: procCall m(Thing(a)) echo "1" + ``` Iterators and the for statement @@ -4049,7 +4625,7 @@ reached, the data is bound to the `for` loop variables and control continues in the body of the `for` loop. The iterator's local variables and execution state are automatically saved between calls. Example: -.. code-block:: nim + ```nim # this definition exists in the system module iterator items*(a: string): char {.inline.} = var i = 0 @@ -4059,15 +4635,17 @@ state are automatically saved between calls. Example: for ch in items("hello world"): # `ch` is an iteration variable echo ch + ``` -The compiler generates code as if the programmer would have written this: +The compiler generates code as if the programmer had written this: -.. code-block:: nim + ```nim var i = 0 while i < len(a): var ch = a[i] echo ch inc(i) + ``` If the iterator yields a tuple, there can be as many iteration variables as there are components in the tuple. The i'th iteration variable's type is @@ -4079,10 +4657,11 @@ Implicit items/pairs invocations If the for loop expression `e` does not denote an iterator and the for loop has exactly 1 variable, the for loop expression is rewritten to `items(e)`; -ie. an `items` iterator is implicitly invoked: +i.e. an `items` iterator is implicitly invoked: -.. code-block:: nim + ```nim for x in [1,2,3]: echo x + ``` If the for loop has exactly 2 variables, a `pairs` iterator is implicitly invoked. @@ -4111,7 +4690,7 @@ templates, macros, and other inline iterators. In contrast to that, a `closure iterator`:idx: can be passed around more freely: -.. code-block:: nim + ```nim iterator count0(): int {.closure.} = yield 0 @@ -4126,18 +4705,15 @@ In contrast to that, a `closure iterator`:idx: can be passed around more freely: invoke(count0) invoke(count2) + ``` Closure iterators and inline iterators have some restrictions: 1. For now, a closure iterator cannot be executed at compile time. 2. `return` is allowed in a closure iterator but not in an inline iterator (but rarely useful) and ends the iteration. -3. Neither inline nor closure iterators can be (directly)* recursive. +3. Inline iterators cannot be recursive. 4. Neither inline nor closure iterators have the special `result` variable. -5. Closure iterators are not supported by the JS backend. - -(*) Closure iterators can be co-recursive with a factory proc which results -in similar syntax to a recursive iterator. More details follow. Iterators that are neither marked `{.closure.}` nor `{.inline.}` explicitly default to being inline, but this may change in future versions of the @@ -4147,7 +4723,7 @@ The `iterator` type is always of the calling convention `closure` implicitly; the following example shows how to use iterators to implement a `collaborative tasking`:idx: system: -.. code-block:: nim + ```nim # simple tasking: type Task = iterator (ticker: int) @@ -4177,15 +4753,16 @@ a `collaborative tasking`:idx: system: inc ticker runTasks(a1, a2) + ``` The builtin `system.finished` can be used to determine if an iterator has finished its operation; no exception is raised on an attempt to invoke an iterator that has already finished its work. -Note that `system.finished` is error prone to use because it only returns +Note that `system.finished` is error-prone to use because it only returns `true` one iteration after the iterator has finished: -.. code-block:: nim + ```nim iterator mycount(a, b: int): int {.closure.} = var x = a while x <= b: @@ -4201,15 +4778,17 @@ Note that `system.finished` is error prone to use because it only returns 2 3 0 + ``` -Instead this code has to be used: +Instead, this code has to be used: -.. code-block:: nim + ```nim var c = mycount # instantiate the iterator while true: let value = c(1, 3) if finished(c): break # and discard 'value'! echo value + ``` It helps to think that the iterator actually returns a pair `(value, done)` and `finished` is used to access the hidden `done` @@ -4220,7 +4799,7 @@ Closure iterators are *resumable functions* and so one has to provide the arguments to every call. To get around this limitation one can capture parameters of an outer factory proc: -.. code-block:: nim + ```nim proc mycount(a, b: int): iterator (): int = result = iterator (): int = var x = a @@ -4232,10 +4811,11 @@ parameters of an outer factory proc: for f in foo(): echo f + ``` The call can be made more like an inline iterator with a for loop macro: -.. code-block:: nim + ```nim import std/macros macro toItr(x: ForLoopStmt): untyped = let expr = x[0] @@ -4249,43 +4829,49 @@ The call can be made more like an inline iterator with a for loop macro: for f in toItr(mycount(1, 4)): # using early `proc mycount` echo f + ``` -Because of full backend function call aparatus involvment, closure iterator -invocation is typically higher cost than inline iterators. Adornment by +Because of full backend function call apparatus involvement, closure iterator +invocation is typically higher cost than inline iterators. Adornment by a macro wrapper at the call site like this is a possibly useful reminder. -The factory `proc`, as an ordinary procedure, can be recursive. The +The factory `proc`, as an ordinary procedure, can be recursive. The above macro allows such recursion to look much like a recursive iterator -would. For example: +would. For example: -.. code-block:: nim + ```nim proc recCountDown(n: int): iterator(): int = result = iterator(): int = if n > 0: yield n for e in toItr(recCountDown(n - 1)): - yield e + yield e for i in toItr(recCountDown(6)): # Emits: 6 5 4 3 2 1 echo i + ``` + + +See also [iterable] for passing iterators to templates and macros. Converters ========== A converter is like an ordinary proc except that it enhances -the "implicitly convertible" type relation (see `Convertible relation`_): +the "implicitly convertible" type relation (see [Convertible relation]): -.. code-block:: nim + ```nim # bad style ahead: Nim is not C. converter toBool(x: int): bool = x != 0 if 4: echo "compiles" + ``` A converter can also be explicitly invoked for improved readability. Note that implicit converter chaining is not supported: If there is a converter from -type A to type B and from type B to type C the implicit conversion from A to C +type A to type B and from type B to type C, the implicit conversion from A to C is not provided. @@ -4294,7 +4880,7 @@ Type sections Example: -.. code-block:: nim + ```nim type # example demonstrating mutually recursive types Node = ref object # an object managed by the garbage collector (ref) le, ri: Node # left and right subtrees @@ -4304,6 +4890,7 @@ Example: name: string # the symbol's name line: int # the line the symbol was declared in code: Node # the symbol's abstract syntax tree + ``` A type section begins with the `type` keyword. It contains multiple type definitions. A type definition binds a type to a name. Type definitions @@ -4321,7 +4908,7 @@ Try statement Example: -.. code-block:: nim + ```nim # read the first two lines of a text file that should contain numbers # and tries to add them var @@ -4333,14 +4920,13 @@ Example: echo "sum: " & $(parseInt(a) + parseInt(b)) except OverflowDefect: echo "overflow!" - except ValueError: - echo "could not convert string to integer" - except IOError: - echo "IO error!" - except: - echo "Unknown exception!" + except ValueError, IOError: + echo "catch multiple exceptions!" + except CatchableError: + echo "Catchable exception!" finally: close(f) + ``` The statements after the `try` are executed in sequential order unless @@ -4349,9 +4935,6 @@ listed in an `except` clause, the corresponding statements are executed. The statements following the `except` clauses are called `exception handlers`:idx:. -The empty `except`:idx: clause is executed if there is an exception that is -not listed otherwise. It is similar to an `else` clause in `if` statements. - If there is a `finally`:idx: clause, it is always executed after the exception handlers. @@ -4369,19 +4952,22 @@ Try can also be used as an expression; the type of the `try` branch then needs to fit the types of `except` branches, but the type of the `finally` branch always has to be `void`: -.. code-block:: nim + ```nim test from std/strutils import parseInt let x = try: parseInt("133a") - except: -1 + except ValueError: -1 finally: echo "hi" + ``` To prevent confusing code there is a parsing limitation; if the `try` follows a `(` it has to be written as a one liner: -.. code-block:: nim - let x = (try: parseInt("133a") except: -1) + ```nim test + from std/strutils import parseInt + let x = (try: parseInt("133a") except ValueError: -1) + ``` Except clauses @@ -4390,59 +4976,65 @@ Except clauses Within an `except` clause it is possible to access the current exception using the following syntax: -.. code-block:: nim + ```nim try: # ... except IOError as e: # Now use "e" echo "I/O error: " & e.msg + ``` Alternatively, it is possible to use `getCurrentException` to retrieve the exception that has been raised: -.. code-block:: nim + ```nim try: # ... except IOError: let e = getCurrentException() # Now use "e" + ``` Note that `getCurrentException` always returns a `ref Exception` type. If a variable of the proper type is needed (in the example above, `IOError`), one must convert it explicitly: -.. code-block:: nim + ```nim try: # ... except IOError: let e = (ref IOError)(getCurrentException()) # "e" is now of the proper type + ``` However, this is seldom needed. The most common case is to extract an error message from `e`, and for such situations, it is enough to use `getCurrentExceptionMsg`: -.. code-block:: nim + ```nim try: # ... - except: + except CatchableError: echo getCurrentExceptionMsg() + ``` Custom exceptions ----------------- It is possible to create custom exceptions. A custom exception is a custom type: -.. code-block:: nim + ```nim type LoadError* = object of Exception + ``` Ending the custom exception's name with `Error` is recommended. Custom exceptions can be raised just like any other exception, e.g.: -.. code-block:: nim + ```nim raise newException(LoadError, "Failed to load data") + ``` Defer statement --------------- @@ -4451,23 +5043,20 @@ Instead of a `try finally` statement a `defer` statement can be used, which avoids lexical nesting and offers more flexibility in terms of scoping as shown below. -Any statements following the `defer` in the current block will be considered -to be in an implicit try block: - -.. code-block:: nim - :test: "nim c $1" +Any statements following the `defer` will be considered +to be in an implicit try block in the current block: + ```nim test = "nim c $1" proc main = var f = open("numbers.txt", fmWrite) defer: close(f) f.write "abc" f.write "def" + ``` Is rewritten to: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" proc main = var f = open("numbers.txt") try: @@ -4475,13 +5064,12 @@ Is rewritten to: f.write "def" finally: close(f) + ``` When `defer` is at the outermost scope of a template/macro, its scope extends -to the block where the template is called from: - -.. code-block:: nim - :test: "nim c $1" +to the block where the template/macro is called from: + ```nim test = "nim c $1" template safeOpenDefer(f, path) = var f = open(path, fmWrite) defer: close(f) @@ -4502,6 +5090,7 @@ to the block where the template is called from: try: f.write "abc" # adds a lexical scope finally: close(f) + ``` Top-level `defer` statements are not supported since it's unclear what such a statement should refer to. @@ -4512,8 +5101,9 @@ Raise statement Example: -.. code-block:: nim + ```nim raise newException(IOError, "IO failed") + ``` Apart from built-in operations like array indexing, memory allocation, etc. the `raise` statement is the only way to raise an exception. @@ -4529,7 +5119,7 @@ exception. Exception hierarchy ------------------- -The exception tree is defined in the `system <system.html>`_ module. +The exception tree is defined in the [system](system.html) module. Every exception inherits from `system.Exception`. Exceptions that indicate programming bugs inherit from `system.Defect` (which is a subtype of `Exception`) and are strictly speaking not catchable as they can also be mapped to an operation @@ -4539,6 +5129,38 @@ exceptions inherit from `Defect`. Exceptions that indicate any other runtime error that can be caught inherit from `system.CatchableError` (which is a subtype of `Exception`). +``` +Exception +|-- CatchableError +| |-- IOError +| | `-- EOFError +| |-- OSError +| |-- ResourceExhaustedError +| `-- ValueError +| `-- KeyError +`-- Defect + |-- AccessViolationDefect + |-- ArithmeticDefect + | |-- DivByZeroDefect + | `-- OverflowDefect + |-- AssertionDefect + |-- DeadThreadDefect + |-- FieldDefect + |-- FloatingPointDefect + | |-- FloatDivByZeroDefect + | |-- FloatInvalidOpDefect + | |-- FloatOverflowDefect + | |-- FloatUnderflowDefect + | `-- InexactDefect + |-- IndexDefect + |-- NilAccessDefect + |-- ObjectAssignmentDefect + |-- ObjectConversionDefect + |-- OutOfMemoryDefect + |-- RangeDefect + |-- ReraiseDefect + `-- StackOverflowDefect +``` Imported exceptions ------------------- @@ -4547,9 +5169,7 @@ It is possible to raise/catch imported C++ exceptions. Types imported using `importcpp` can be raised or caught. Exceptions are raised by value and caught by reference. Example: -.. code-block:: nim - :test: "nim cpp -r $1" - + ```nim test = "nim cpp -r $1" type CStdException {.importcpp: "std::exception", header: "<exception>", inheritable.} = object ## does not inherit from `RootObj`, so we use `inheritable` instead @@ -4580,6 +5200,7 @@ caught by reference. Example: doAssert $b == "foo3" fn() + ``` **Note:** `getCurrentException()` and `getCurrentExceptionMsg()` are not available for imported exceptions from C++. One needs to use the `except ImportedException as x:` syntax @@ -4589,6 +5210,10 @@ and rely on functionality of the `x` object to get exception details. Effect system ============= +**Note**: The rules for effect tracking changed with the release of version +1.6 of the Nim compiler. + + Exception tracking ------------------ @@ -4596,31 +5221,28 @@ Nim supports exception tracking. The `raises`:idx: pragma can be used to explicitly define which exceptions a proc/iterator/method/converter is allowed to raise. The compiler verifies this: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" proc p(what: bool) {.raises: [IOError, OSError].} = if what: raise newException(IOError, "IO") else: raise newException(OSError, "OS") + ``` An empty `raises` list (`raises: []`) means that no exception may be raised: -.. code-block:: nim + ```nim proc p(): bool {.raises: [].} = try: unsafeCall() result = true - except: + except CatchableError: result = false + ``` A `raises` list can also be attached to a proc type. This affects type compatibility: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 type Callback = proc (s: string) {.raises: [IOError].} var @@ -4630,6 +5252,7 @@ compatibility: raise newException(OSError, "OS") c = p # type error + ``` For a routine `p`, the compiler uses inference rules to determine the set of @@ -4638,59 +5261,100 @@ possibly raised exceptions; the algorithm operates on `p`'s call graph: 1. Every indirect call via some proc type `T` is assumed to raise `system.Exception` (the base type of the exception hierarchy) and thus any exception unless `T` has an explicit `raises` list. - However, if the call is of the form `f(...)` where `f` is a parameter of the currently analyzed routine it is ignored. The call is optimistically assumed to have no effect. Rule 2 compensates for this case. -2. Every expression of some proc type within a call that is not a call - itself (and not nil) is assumed to be called indirectly somehow and thus + However, if the call is of the form `f(...)` where `f` is a parameter of + the currently analyzed routine that is marked as `.effectsOf: f`, it is ignored. + The call is optimistically assumed to have no effect. + Rule 2 compensates for this case. +2. Every expression `e` of some proc type within a call that is passed to parameter + marked as `.effectsOf` of proc `p` is assumed to be called indirectly and thus its raises list is added to `p`'s raises list. 3. Every call to a proc `q` which has an unknown body (due to a forward - declaration or an `importc` pragma) is assumed to + declaration) is assumed to raise `system.Exception` unless `q` has an explicit `raises` list. + Procs that are `importc`'ed are assumed to have `.raises: []`, unless explicitly + declared otherwise. 4. Every call to a method `m` is assumed to raise `system.Exception` unless `m` has an explicit `raises` list. 5. For every other call, the analysis can determine an exact `raises` list. 6. For determining a `raises` list, the `raise` and `try` statements of `p` are taken into consideration. -Rules 1-2 ensure the following works: - -.. code-block:: nim - proc noRaise(x: proc()) {.raises: [].} = - # unknown call that might raise anything, but valid: - x() - - proc doRaise() {.raises: [IOError].} = - raise newException(IOError, "IO") - - proc use() {.raises: [].} = - # doesn't compile! Can raise IOError! - noRaise(doRaise) - -So in many cases a callback does not cause the compiler to be overly -conservative in its effect analysis. Exceptions inheriting from `system.Defect` are not tracked with the `.raises: []` exception tracking mechanism. This is more consistent with the built-in operations. The following code is valid: -.. code-block:: nim - + ```nim proc mydiv(a, b): int {.raises: [].} = a div b # can raise an DivByZeroDefect + ``` And so is: -.. code-block:: nim - + ```nim proc mydiv(a, b): int {.raises: [].} = if b == 0: raise newException(DivByZeroDefect, "division by zero") else: result = a div b + ``` The reason for this is that `DivByZeroDefect` inherits from `Defect` and -with `--panics:on` Defects become unrecoverable errors. +with `--panics:on`:option: Defects become unrecoverable errors. (Since version 1.4 of the language.) +EffectsOf annotation +-------------------- + +Rules 1-2 of the exception tracking inference rules (see the previous section) +ensure the following works: + + ```nim + proc weDontRaiseButMaybeTheCallback(callback: proc()) {.raises: [], effectsOf: callback.} = + callback() + + proc doRaise() {.raises: [IOError].} = + raise newException(IOError, "IO") + + proc use() {.raises: [].} = + # doesn't compile! Can raise IOError! + weDontRaiseButMaybeTheCallback(doRaise) + ``` + +As can be seen from the example, a parameter of type `proc (...)` can be +annotated as `.effectsOf`. Such a parameter allows for effect polymorphism: +The proc `weDontRaiseButMaybeTheCallback` raises the exceptions +that `callback` raises. + +So in many cases a callback does not cause the compiler to be overly +conservative in its effect analysis: + + ```nim test = "nim c $1" status = 1 + {.push warningAsError[Effect]: on.} + + import std/algorithm + + type + MyInt = distinct int + + var toSort = @[MyInt 1, MyInt 2, MyInt 3] + + proc cmpN(a, b: MyInt): int = + cmp(a.int, b.int) + + proc harmless {.raises: [].} = + toSort.sort cmpN + + proc cmpE(a, b: MyInt): int {.raises: [Exception].} = + cmp(a.int, b.int) + + proc harmful {.raises: [].} = + # does not compile, `sort` can now raise Exception + toSort.sort cmpE + ``` + + + Tag tracking ------------ @@ -4698,16 +5362,14 @@ Exception tracking is part of Nim's `effect system`:idx:. Raising an exception is an *effect*. Other effects can also be defined. A user defined effect is a means to *tag* a routine and to perform checks against this tag: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c --warningAsError:Effect:on $1" status = 1 type IO = object ## input/output effect proc readLine(): string {.tags: [IO].} = discard - proc no_IO_please() {.tags: [].} = + proc no_effects_please() {.tags: [].} = # the compiler prevents this: let x = readLine() + ``` A tag has to be a type name. A `tags` list - like a `raises` list - can also be attached to a proc type. This affects type compatibility. @@ -4715,6 +5377,126 @@ also be attached to a proc type. This affects type compatibility. The inference for tag tracking is analogous to the inference for exception tracking. +There is also a way which can be used to forbid certain effects: + + ```nim test = "nim c --warningAsError:Effect:on $1" status = 1 + type IO = object ## input/output effect + proc readLine(): string {.tags: [IO].} = discard + proc echoLine(): void = discard + + proc no_IO_please() {.forbids: [IO].} = + # this is OK because it didn't define any tag: + echoLine() + # the compiler prevents this: + let y = readLine() + ``` + +The `forbids` pragma defines a list of illegal effects - if any statement +invokes any of those effects, the compilation will fail. +Procedure types with any disallowed effect are the subtypes of equal +procedure types without such lists: + + ```nim + type MyEffect = object + type ProcType1 = proc (i: int): void {.forbids: [MyEffect].} + type ProcType2 = proc (i: int): void + + proc caller1(p: ProcType1): void = p(1) + proc caller2(p: ProcType2): void = p(1) + + proc effectful(i: int): void {.tags: [MyEffect].} = echo $i + proc effectless(i: int): void {.forbids: [MyEffect].} = echo $i + + proc toBeCalled1(i: int): void = effectful(i) + proc toBeCalled2(i: int): void = effectless(i) + + ## this will fail because toBeCalled1 uses MyEffect which was forbidden by ProcType1: + caller1(toBeCalled1) + ## this is OK because both toBeCalled2 and ProcType1 have the same requirements: + caller1(toBeCalled2) + ## these are OK because ProcType2 doesn't have any effect requirement: + caller2(toBeCalled1) + caller2(toBeCalled2) + ``` + +`ProcType2` is a subtype of `ProcType1`. Unlike with the `tags` pragma, the parent context - the +function which calls other functions with forbidden effects - doesn't inherit the forbidden list of effects. + + +Side effects +------------ + +The `noSideEffect` pragma is used to mark a proc/iterator that can have only +side effects through parameters. This means that the proc/iterator only changes locations that are +reachable from its parameters and the return value only depends on the +parameters. If none of its parameters have the type `var`, `ref`, `ptr`, `cstring`, or `proc`, +then no locations are modified. + +In other words, a routine has no side effects if it does not access a threadlocal +or global variable and it does not call any routine that has a side effect. + +It is a static error to mark a proc/iterator to have no side effect if the compiler cannot verify this. + +As a special semantic rule, the built-in [debugEcho]( +system.html#debugEcho,varargs[typed,]) pretends to be free of side effects +so that it can be used for debugging routines marked as `noSideEffect`. + +`func` is syntactic sugar for a proc with no side effects: + + ```nim + func `+` (x, y: int): int + ``` + + +To override the compiler's side effect analysis a `{.noSideEffect.}` +`cast` pragma block can be used: + + ```nim + func f() = + {.cast(noSideEffect).}: + echo "test" + ``` + +**Side effects are usually inferred. The inference for side effects is +analogous to the inference for exception tracking.** + +When the compiler cannot infer side effects, as is the case for imported +functions, one can annotate them with the `sideEffect` pragma. + +GC safety effect +---------------- + +We call a proc `p` `GC safe`:idx: when it doesn't access any global variable +that contains GC'ed memory (`string`, `seq`, `ref` or a closure) either +directly or indirectly through a call to a GC unsafe proc. + +**The GC safety property is usually inferred. The inference for GC safety is +analogous to the inference for exception tracking.** + +The `gcsafe`:idx: annotation can be used to mark a proc to be gcsafe, +otherwise this property is inferred by the compiler. Note that `noSideEffect` +implies `gcsafe`. + +Routines that are imported from C are always assumed to be `gcsafe`. + +To override the compiler's gcsafety analysis a `{.cast(gcsafe).}` pragma block can +be used: + + ```nim + var + someGlobal: string = "some string here" + perThread {.threadvar.}: string + + proc setPerThread() = + {.cast(gcsafe).}: + deepCopy(perThread, someGlobal) + ``` + + +See also: + +- [Shared heap memory management](mm.html). + Effects pragma @@ -4724,13 +5506,14 @@ The `effects` pragma has been designed to assist the programmer with the effects analysis. It is a statement that makes the compiler output all inferred effects up to the `effects`'s position: -.. code-block:: nim + ```nim proc p(what: bool) = if what: raise newException(IOError, "IO") {.effects.} else: raise newException(OSError, "OS") + ``` The compiler produces a hint message that `IOError` can be raised. `OSError` is not listed as it cannot be raised in the branch the `effects` pragma @@ -4744,14 +5527,13 @@ Generics are Nim's means to parametrize procs, iterators or types with `type parameters`:idx:. Depending on the context, the brackets are used either to introduce type parameters or to instantiate a generic proc, iterator, or type. -The following example shows how a generic binary tree can be modeled: -.. code-block:: nim - :test: "nim c $1" +The following example shows how a generic binary tree can be modeled: + ```nim test = "nim c $1" type BinaryTree*[T] = ref object # BinaryTree is a generic type with - # generic param `T` + # generic parameter `T` le, ri: BinaryTree[T] # left and right subtrees; may be nil data: T # the data stored in a node @@ -4802,10 +5584,54 @@ The following example shows how a generic binary tree can be modeled: add(root, "world") # instantiates the second `add` proc for str in preorder(root): stdout.writeLine(str) + ``` The `T` is called a `generic type parameter`:idx: or a `type variable`:idx:. + +Generic Procs +--------------- + +Let's consider the anatomy of a generic `proc` to agree on defined terminology. + +```nim +p[T: t](arg1: f): y +``` + +- `p`: Callee symbol +- `[...]`: Generic parameters +- `T: t`: Generic constraint +- `T`: Type variable +- `[T: t](arg1: f): y`: Formal signature +- `arg1: f`: Formal parameter +- `f`: Formal parameter type +- `y`: Formal return type + +The use of the word "formal" here is to denote the symbols as they are defined by the programmer, +not as they may be at compile time contextually. Since generics may be instantiated and +types bound, we have more than one entity to think about when generics are involved. + +The usage of a generic will resolve the formally defined expression into an instance of that +expression bound to only concrete types. This process is called "instantiation". + +Brackets at the site of a generic's formal definition specify the "constraints" as in: + +```nim +type Foo[T] = object +proc p[H;T: Foo[H]](param: T): H +``` + +A constraint definition may have more than one symbol defined by separating each definition by +a `;`. Notice how `T` is composed of `H` and the return type of `p` is defined as `H`. When this +generic proc is instantiated `H` will be bound to a concrete type, thus making `T` concrete and +the return type of `p` will be bound to the same concrete type used to define `H`. + +Brackets at the site of usage can be used to supply concrete types to instantiate the generic in the same +order that the symbols are defined in the constraint. Alternatively, type bindings may be inferred by the compiler +in some situations, allowing for cleaner code. + + Is operator ----------- @@ -4813,16 +5639,17 @@ The `is` operator is evaluated during semantic analysis to check for type equivalence. It is therefore very useful for type specialization within generic code: -.. code-block:: nim + ```nim type Table[Key, Value] = object keys: seq[Key] values: seq[Value] when not (Key is string): # empty value for strings used for optimization deletedKeys: seq[bool] + ``` -Type Classes +Type classes ------------ A type class is a special pseudo-type that can be used to match against @@ -4834,9 +5661,9 @@ type class matches ================== =================================================== `object` any object type `tuple` any tuple type - `enum` any enumeration `proc` any proc type +`iterator` any iterator type `ref` any `ref` type `ptr` any `ptr` type `var` any `var` type @@ -4853,20 +5680,22 @@ name that will match any instantiation of the generic type. Type classes can be combined using the standard boolean operators to form more complex type classes: -.. code-block:: nim + ```nim # create a type class that will match all tuple and object types - type RecordType = tuple or object + type RecordType = (tuple or object) proc printFields[T: RecordType](rec: T) = for key, value in fieldPairs(rec): echo key, " = ", value + ``` Type constraints on generic parameters can be grouped with `,` and propagation stops with `;`, similarly to parameters for macros and templates: -.. code-block:: nim + ```nim proc fn1[T; U, V: SomeFloat]() = discard # T is unconstrained template fn2(t; u, v: SomeFloat) = discard # t is unconstrained + ``` Whilst the syntax of type classes appears to resemble that of ADTs/algebraic data types in ML-like languages, it should be understood that type classes are static @@ -4877,20 +5706,33 @@ runtime type dynamism, unlike object variants or methods. As an example, the following would not compile: -.. code-block:: nim + ```nim type TypeClass = int | string var foo: TypeClass = 2 # foo's type is resolved to an int here foo = "this will fail" # error here, because foo is an int + ``` Nim allows for type classes and regular types to be specified as `type constraints`:idx: of the generic type parameter: -.. code-block:: nim + ```nim proc onlyIntOrString[T: int|string](x, y: T) = discard onlyIntOrString(450, 616) # valid onlyIntOrString(5.0, 0.0) # type mismatch onlyIntOrString("xy", 50) # invalid as 'T' cannot be both at the same time + ``` + +`proc` and `iterator` type classes also accept a calling convention pragma +to restrict the calling convention of the matching `proc` or `iterator` type. + + ```nim + proc onlyClosure[T: proc {.closure.}](x: T) = discard + + onlyClosure(proc() = echo "hello") # valid + proc foo() {.nimcall.} = discard + onlyClosure(foo) # type mismatch + ``` Implicit generics @@ -4898,25 +5740,25 @@ Implicit generics A type class can be used directly as the parameter's type. -.. code-block:: nim - + ```nim # create a type class that will match all tuple and object types - type RecordType = tuple or object + type RecordType = (tuple or object) proc printFields(rec: RecordType) = for key, value in fieldPairs(rec): echo key, " = ", value + ``` Procedures utilizing type classes in such a manner are considered to be `implicitly generic`:idx:. They will be instantiated once for each unique -combination of param types used within the program. +combination of parameter types used within the program. By default, during overload resolution, each named type class will bind to exactly one concrete type. We call such type classes `bind once`:idx: types. Here is an example taken directly from the system module to illustrate this: -.. code-block:: nim + ```nim proc `==`*(x, y: tuple): bool = ## requires `x` and `y` to be of the same tuple type ## generic `==` operator for tuples that is lifted from the components @@ -4924,81 +5766,82 @@ Here is an example taken directly from the system module to illustrate this: result = true for a, b in fields(x, y): if a != b: result = false + ``` Alternatively, the `distinct` type modifier can be applied to the type class -to allow each param matching the type class to bind to a different type. Such +to allow each parameter matching the type class to bind to a different type. Such type classes are called `bind many`:idx: types. Procs written with the implicitly generic style will often need to refer to the type parameters of the matched generic type. They can be easily accessed using the dot syntax: -.. code-block:: nim + ```nim type Matrix[T, Rows, Columns] = object ... proc `[]`(m: Matrix, row, col: int): Matrix.T = m.data[col * high(Matrix.Columns) + row] + ``` Here are more examples that illustrate implicit generics: -.. code-block:: nim - + ```nim proc p(t: Table; k: Table.Key): Table.Value # is roughly the same as: proc p[Key, Value](t: Table[Key, Value]; k: Key): Value + ``` -.. code-block:: nim - + ```nim proc p(a: Table, b: Table) # is roughly the same as: proc p[Key, Value](a, b: Table[Key, Value]) + ``` -.. code-block:: nim - + ```nim proc p(a: Table, b: distinct Table) # is roughly the same as: proc p[Key, Value, KeyB, ValueB](a: Table[Key, Value], b: Table[KeyB, ValueB]) + ``` `typedesc` used as a parameter type also introduces an implicit generic. `typedesc` has its own set of rules: -.. code-block:: nim - + ```nim proc p(a: typedesc) # is roughly the same as: proc p[T](a: typedesc[T]) + ``` `typedesc` is a "bind many" type class: -.. code-block:: nim - + ```nim proc p(a, b: typedesc) # is roughly the same as: proc p[T, T2](a: typedesc[T], b: typedesc[T2]) + ``` A parameter of type `typedesc` is itself usable as a type. If it is used -as a type, it's the underlying type. (In other words, one level +as a type, it's the underlying type. In other words, one level of "typedesc"-ness is stripped off: -.. code-block:: nim - + ```nim proc p(a: typedesc; b: a) = discard # is roughly the same as: @@ -5007,6 +5850,7 @@ of "typedesc"-ness is stripped off: # hence this is a valid call: p(int, 4) # as parameter 'a' requires a type, but 'b' requires a value. + ``` Generic inference restrictions @@ -5015,10 +5859,7 @@ Generic inference restrictions The types `var T` and `typedesc[T]` cannot be inferred in a generic instantiation. The following is not allowed: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 proc g[T](f: proc(x: T); x: T) = f(x) @@ -5035,14 +5876,14 @@ instantiation. The following is not allowed: # also not allowed: explicit instantiation via 'var int' g[var int](v, i) + ``` Symbol lookup in generics ------------------------- -Open and Closed symbols -~~~~~~~~~~~~~~~~~~~~~~~ +### Open and Closed symbols The symbol binding rules in generics are slightly subtle: There are "open" and "closed" symbols. A "closed" symbol cannot be re-bound in the instantiation @@ -5052,9 +5893,7 @@ and every other symbol is closed. Open symbols are looked up in two different contexts: Both the context at definition and the context at instantiation are considered: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Index = distinct int @@ -5064,8 +5903,9 @@ at definition and the context at instantiation are considered: var b = (0, 0.Index) echo a == b # works! + ``` -In the example, the generic `==` for tuples (as defined in the system module) +In the example, the [generic `==` for tuples](system.html#%3D%3D%2CT%2CT_2) (as defined in the system module) uses the `==` operators of the tuple's components. However, the `==` for the `Index` type is defined *after* the `==` for tuples; yet the example compiles as the instantiation takes the currently defined symbols into account @@ -5076,15 +5916,14 @@ Mixin statement A symbol can be forced to be open by a `mixin`:idx: declaration: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" proc create*[T](): ref T = # there is no overloaded 'init' here, so we need to state that it's an # open symbol explicitly: mixin init new result init result + ``` `mixin` statements only make sense in templates and generics. @@ -5097,7 +5936,7 @@ can be used to explicitly declare identifiers that should be bound early (i.e. the identifiers should be looked up in the scope of the template/generic definition): -.. code-block:: nim + ```nim # Module A var lastId = 0 @@ -5106,12 +5945,14 @@ definition): bind lastId inc(lastId) lastId + ``` -.. code-block:: nim + ```nim # Module B import A echo genId() + ``` But a `bind` is rarely useful because symbol binding from the definition scope is the default. @@ -5125,16 +5966,15 @@ Delegating bind statements The following example outlines a problem that can arise when generic instantiations cross multiple different modules: -.. code-block:: nim - + ```nim # module A proc genericA*[T](x: T) = mixin init init(x) + ``` -.. code-block:: nim - + ```nim import C # module B @@ -5143,23 +5983,24 @@ instantiations cross multiple different modules: # not available when `genericB` is instantiated: bind init genericA(x) + ``` -.. code-block:: nim - + ```nim # module C type O = object proc init*(x: var O) = discard + ``` -.. code-block:: nim - + ```nim # module main import B, C genericB O() + ``` In module B has an `init` proc from module C in its scope that is not taken into account when `genericB` is instantiated which leads to the -instantiation of `genericA`. The solution is to `forward`:idx these +instantiation of `genericA`. The solution is to `forward`:idx: these symbols by a `bind` statement inside `genericB`. @@ -5174,12 +6015,13 @@ The syntax to *invoke* a template is the same as calling a procedure. Example: -.. code-block:: nim + ```nim template `!=` (a, b: untyped): untyped = - # this definition exists in the System module + # this definition exists in the system module not (a == b) assert(5 != 6) # the compiler rewrites that to: assert(not (5 == 6)) + ``` The `!=`, `>`, `>=`, `in`, `notin`, `isnot` operators are in fact templates: @@ -5201,24 +6043,21 @@ An `untyped` parameter means that symbol lookups and type resolution is not performed before the expression is passed to the template. This means that *undeclared* identifiers, for example, can be passed to the template: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template declareInt(x: untyped) = var x: int declareInt(x) # valid x = 3 + ``` -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 template declareInt(x: typed) = var x: int declareInt(x) # invalid, because x has not been declared and so it has no type + ``` A template where every parameter is `untyped` is called an `immediate`:idx: template. For historical reasons, templates can be explicitly annotated with @@ -5236,9 +6075,7 @@ Passing a code block to a template One can pass a block of statements as the last argument to a template following the special `:` syntax: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template withFile(f, fn, mode, actions: untyped): untyped = var f: File if open(f, fn, mode): @@ -5252,6 +6089,7 @@ following the special `:` syntax: withFile(txt, "ttempl3.txt", fmWrite): # special colon txt.writeLine("line 1") txt.writeLine("line 2") + ``` In the example, the two `writeLine` statements are bound to the `actions` parameter. @@ -5261,10 +6099,7 @@ Usually, to pass a block of code to a template, the parameter that accepts the block needs to be of type `untyped`. Because symbol lookups are then delayed until template instantiation time: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 template t(body: typed) = proc p = echo "hey" block: @@ -5272,6 +6107,7 @@ delayed until template instantiation time: t: p() # fails with 'undeclared identifier: p' + ``` The above code fails with the error message that `p` is not declared. The reason for this is that the `p()` body is type-checked before getting @@ -5279,9 +6115,7 @@ passed to the `body` parameter and type checking in Nim implies symbol lookups. The same code works with `untyped` as the passed body is not required to be type-checked: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template t(body: untyped) = proc p = echo "hey" block: @@ -5289,6 +6123,7 @@ type-checked: t: p() # compiles + ``` Varargs of untyped @@ -5297,12 +6132,11 @@ Varargs of untyped In addition to the `untyped` meta-type that prevents type checking, there is also `varargs[untyped]` so that not even the number of parameters is fixed: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template hideIdentifiers(x: varargs[untyped]) = discard hideIdentifiers(undeclared1, undeclared2) + ``` However, since a template cannot iterate over varargs, this feature is generally much more useful for macros. @@ -5314,7 +6148,7 @@ Symbol binding in templates A template is a `hygienic`:idx: macro and so opens a new scope. Most symbols are bound from the definition scope of the template: -.. code-block:: nim + ```nim # Module A var lastId = 0 @@ -5322,12 +6156,14 @@ bound from the definition scope of the template: template genId*: untyped = inc(lastId) lastId + ``` -.. code-block:: nim + ```nim # Module B import A echo genId() # Works as 'lastId' has been bound in 'genId's defining scope + ``` As in generics, symbol binding can be influenced via `mixin` or `bind` statements. @@ -5339,9 +6175,7 @@ Identifier construction In templates, identifiers can be constructed with the backticks notation: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template typedef(name: untyped, typ: typedesc) = type `T name`* {.inject.} = typ @@ -5349,6 +6183,7 @@ In templates, identifiers can be constructed with the backticks notation: typedef(myint, int) var x: PMyInt + ``` In the example, `name` is instantiated with `myint`, so \`T name\` becomes `Tmyint`. @@ -5361,7 +6196,7 @@ A parameter `p` in a template is even substituted in the expression `x.p`. Thus, template arguments can be used as field names and a global symbol can be shadowed by the same argument name even when fully qualified: -.. code-block:: nim + ```nim # module 'm' type @@ -5375,10 +6210,11 @@ shadowed by the same argument name even when fully qualified: tstLev(levA) # produces: 'levA levA' + ``` But the global symbol can properly be captured by a `bind` statement: -.. code-block:: nim + ```nim # module 'm' type @@ -5393,6 +6229,7 @@ But the global symbol can properly be captured by a `bind` statement: tstLev(levA) # produces: 'levA levB' + ``` Hygiene in templates @@ -5401,9 +6238,7 @@ Hygiene in templates Per default, templates are `hygienic`:idx:\: Local identifiers declared in a template cannot be accessed in the instantiation context: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template newException*(exceptn: typedesc, message: string): untyped = var e: ref exceptn # e is implicitly gensym'ed here @@ -5414,6 +6249,7 @@ template cannot be accessed in the instantiation context: # so this works: let e = "message" raise newException(IoError, e) + ``` Whether a symbol that is declared in a template is exposed to the instantiation @@ -5421,29 +6257,34 @@ scope is controlled by the `inject`:idx: and `gensym`:idx: pragmas: `gensym`'ed symbols are not exposed but `inject`'ed symbols are. The default for symbols of entity `type`, `var`, `let` and `const` -is `gensym` and for `proc`, `iterator`, `converter`, `template`, -`macro` is `inject`. However, if the name of the entity is passed as a -template parameter, it is an `inject`'ed symbol: +is `gensym`. For `proc`, `iterator`, `converter`, `template`, +`macro`, the default is `inject`, but if a `gensym` symbol with the same name +is defined in the same syntax-level scope, it will be `gensym` by default. +This can be overriden by marking the routine as `inject`. -.. code-block:: nim +If the name of the entity is passed as a template parameter, it is an `inject`'ed symbol: + + ```nim template withFile(f, fn, mode: untyped, actions: untyped): untyped = block: - var f: File # since 'f' is a template param, it's injected implicitly + var f: File # since 'f' is a template parameter, it's injected implicitly ... withFile(txt, "ttempl3.txt", fmWrite): txt.writeLine("line 1") txt.writeLine("line 2") + ``` The `inject` and `gensym` pragmas are second class annotations; they have -no semantics outside of a template definition and cannot be abstracted over: +no semantics outside a template definition and cannot be abstracted over: -.. code-block:: nim + ```nim {.pragma myInject: inject.} template t() = var x {.myInject.}: int # does NOT work + ``` To get rid of hygiene in templates, one can use the `dirty`:idx: pragma for @@ -5455,9 +6296,7 @@ and `namedParameterCall(field = value)` syntactic constructs. The reason for this is that code like -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type T = object f: int @@ -5465,6 +6304,7 @@ The reason for this is that code like template tmp(x: T) = let f = 34 echo x.f, T(f: 4) + ``` should work as expected. @@ -5472,10 +6312,7 @@ should work as expected. However, this means that the method call syntax is not available for `gensym`'ed symbols: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 template tmp(x) = type T {.gensym.} = int @@ -5483,11 +6320,7 @@ However, this means that the method call syntax is not available for echo x.T # invalid: instead use: 'echo T(x)'. tmp(12) - - -**Note**: The Nim compiler prior to version 1 was more lenient about this -requirement. Use the `--useVersion:0.19` switch for a transition period. - + ``` Limitations of the method call syntax @@ -5495,53 +6328,31 @@ Limitations of the method call syntax The expression `x` in `x.f` needs to be semantically checked (that means symbol lookup and type checking) before it can be decided that it needs to be -rewritten to `f(x)`. Therefore the dot syntax has some limitations when it +rewritten to `f(x)`. Therefore, the dot syntax has some limitations when it is used to invoke templates/macros: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 template declareVar(name: untyped) = const name {.inject.} = 45 # Doesn't compile: unknownIdentifier.declareVar + ``` -Another common example is this: - -.. code-block:: nim - :test: "nim c $1" - :status: 1 - - from std/sequtils import toSeq - - iterator something: string = - yield "Hello" - yield "World" - - var info = something().toSeq - -The problem here is that the compiler already decided that `something()` as -an iterator is not callable in this context before `toSeq` gets its -chance to convert it into a sequence. - It is also not possible to use fully qualified identifiers with module symbol in method call syntax. The order in which the dot operator binds to symbols prohibits this. -.. code-block:: nim - :test: "nim c $1" - :status: 1 + ```nim test = "nim c $1" status = 1 + import std/sequtils - import std/sequtils - - var myItems = @[1,3,3,7] - let N1 = count(myItems, 3) # OK - let N2 = sequtils.count(myItems, 3) # fully qualified, OK - let N3 = myItems.count(3) # OK - let N4 = myItems.sequtils.count(3) # illegal, `myItems.sequtils` can't be resolved + var myItems = @[1,3,3,7] + let N1 = count(myItems, 3) # OK + let N2 = sequtils.count(myItems, 3) # fully qualified, OK + let N3 = myItems.count(3) # OK + let N4 = myItems.sequtils.count(3) # illegal, `myItems.sequtils` can't be resolved + ``` This means that when for some reason a procedure needs a disambiguation through the module name, the call needs to be @@ -5570,15 +6381,21 @@ twice: While macros enable advanced compile-time code transformations, they cannot change Nim's syntax. -Debug Example +**Style note:** For code readability, it is best to use the least powerful +programming construct that remains expressive. So the "check list" is: + +(1) Use an ordinary proc/iterator, if possible. +(2) Else: Use a generic proc/iterator, if possible. +(3) Else: Use a template, if possible. +(4) Else: Use a macro. + +Debug example ------------- The following example implements a powerful `debug` command that accepts a variable number of arguments: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" # to work with Nim syntax trees, we need an API that is defined in the # `macros` module: import std/macros @@ -5606,10 +6423,11 @@ variable number of arguments: a[1] = 45 debug(a[0], a[1], x) + ``` The macro call expands to: -.. code-block:: nim + ```nim write(stdout, "a[0]") write(stdout, ": ") writeLine(stdout, a[0]) @@ -5621,6 +6439,7 @@ The macro call expands to: write(stdout, "x") write(stdout, ": ") writeLine(stdout, x) + ``` Arguments that are passed to a `varargs` parameter are wrapped in an array @@ -5628,7 +6447,7 @@ constructor expression. This is why `debug` iterates over all of `args`'s children. -BindSym +bindSym ------- The above `debug` macro relies on the fact that `write`, `writeLine` and @@ -5637,9 +6456,7 @@ instantiating context. There is a way to use bound identifiers (aka `symbols`:idx:) instead of using unbound identifiers. The `bindSym` builtin can be used for that: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/macros macro debug(n: varargs[typed]): untyped = @@ -5657,10 +6474,11 @@ builtin can be used for that: a[1] = 45 debug(a[0], a[1], x) + ``` The macro call expands to: -.. code-block:: nim + ```nim write(stdout, "a[0]") write(stdout, ": ") writeLine(stdout, a[0]) @@ -5672,48 +6490,45 @@ The macro call expands to: write(stdout, "x") write(stdout, ": ") writeLine(stdout, x) + ``` -However, the symbols `write`, `writeLine` and `stdout` are already bound -and are not looked up again. As the example shows, `bindSym` does work with -overloaded symbols implicitly. - -Case-Of Macro -------------- - -In Nim, it is possible to have a macro with the syntax of a *case-of* -expression just with the difference that all *of-branches* are passed to -and processed by the macro implementation. It is then up the macro -implementation to transform the *of-branches* into a valid Nim -statement. The following example should show how this feature could be -used for a lexical analyzer. +In this version of `debug`, the symbols `write`, `writeLine` and `stdout` +are already bound and are not looked up again. As the example shows, `bindSym` +does work with overloaded symbols implicitly. -.. code-block:: nim - import std/macros +Note that the symbol names passed to `bindSym` have to be constant. The +experimental feature `dynamicBindSym` ([experimental manual]( +manual_experimental.html#dynamic-arguments-for-bindsym)) +allows this value to be computed dynamically. - macro case_token(args: varargs[untyped]): untyped = - echo args.treeRepr - # creates a lexical analyzer from regular expressions - # ... (implementation is an exercise for the reader ;-) - discard +Post-statement blocks +--------------------- - case_token: # this colon tells the parser it is a macro statement - of r"[A-Za-z_]+[A-Za-z_0-9]*": - return tkIdentifier - of r"0-9+": - return tkInteger - of r"[\+\-\*\?]+": - return tkOperator +Macros can receive `of`, `elif`, `else`, `except`, `finally` and `do` +blocks (including their different forms such as `do` with routine parameters) +as arguments if called in statement form. + + ```nim + macro performWithUndo(task, undo: untyped) = ... + + performWithUndo do: + # multiple-line block of code + # to perform the task + do: + # code to undo it + + let num = 12 + # a single colon may be used if there is no initial block + match (num mod 3, num mod 5): + of (0, 0): + echo "FizzBuzz" + of (0, _): + echo "Fizz" + of (_, 0): + echo "Buzz" else: - return tkUnknown - - -**Style note**: For code readability, it is best to use the least powerful -programming construct that still suffices. So the "check list" is: - -(1) Use an ordinary proc/iterator, if possible. -(2) Else: Use a generic proc/iterator, if possible. -(3) Else: Use a template, if possible. -(4) Else: Use a macro. + echo num + ``` For loop macro @@ -5722,9 +6537,28 @@ For loop macro A macro that takes as its only input parameter an expression of the special type `system.ForLoopStmt` can rewrite the entirety of a `for` loop: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" + import std/macros + + macro example(loop: ForLoopStmt) = + result = newTree(nnkForStmt) # Create a new For loop. + result.add loop[^3] # This is "item". + result.add loop[^2][^1] # This is "[1, 2, 3]". + result.add newCall(bindSym"echo", loop[0]) + + for item in example([1, 2, 3]): discard + ``` + +Expands to: + + ```nim + for item in items([1, 2, 3]): + echo item + ``` +Another example: + + ```nim test = "nim c $1" import std/macros macro enumerate(x: ForLoopStmt): untyped = @@ -5756,19 +6590,60 @@ type `system.ForLoopStmt` can rewrite the entirety of a `for` loop: # names for `a` and `b` here to avoid redefinition errors for a, b in enumerate(10, [1, 2, 3, 5]): echo a, " ", b + ``` + + +Case statement macros +--------------------- + +Macros named `` `case` `` can provide implementations of `case` statements +for certain types. The following is an example of such an implementation +for tuples, leveraging the existing equality operator for tuples +(as provided in `system.==`): + ```nim test = "nim c $1" + import std/macros + + macro `case`(n: tuple): untyped = + result = newTree(nnkIfStmt) + let selector = n[0] + for i in 1 ..< n.len: + let it = n[i] + case it.kind + of nnkElse, nnkElifBranch, nnkElifExpr, nnkElseExpr: + result.add it + of nnkOfBranch: + for j in 0..it.len-2: + let cond = newCall("==", selector, it[j]) + result.add newTree(nnkElifBranch, cond, it[^1]) + else: + error "custom 'case' for tuple cannot handle this node", it + + case ("foo", 78) + of ("foo", 78): echo "yes" + of ("bar", 88): echo "no" + else: discard + ``` + +`case` macros are subject to overload resolution. The type of the +`case` statement's selector expression is matched against the type +of the first argument of the `case` macro. Then the complete `case` +statement is passed in place of the argument and the macro is evaluated. + +In other words, the macro needs to transform the full `case` statement +but only the statement's selector expression is used to determine which +macro to call. Special Types ============= -static[T] ---------- +static\[T] +---------- As their name suggests, static parameters must be constant expressions: -.. code-block:: nim - + ```nim proc precompiledRegex(pattern: static string): RegEx = var res {.global.} = re(pattern) return res @@ -5778,16 +6653,16 @@ As their name suggests, static parameters must be constant expressions: precompiledRegex(paramStr(1)) # Error, command-line options # are not constant expressions + ``` -For the purposes of code generation, all static params are treated as -generic params - the proc will be compiled separately for each unique +For the purposes of code generation, all static parameters are treated as +generic parameters - the proc will be compiled separately for each unique supplied value (or combination of values). -Static params can also appear in the signatures of generic types: - -.. code-block:: nim +Static parameters can also appear in the signatures of generic types: + ```nim type Matrix[M,N: static int; T: Number] = array[0..(M*N - 1), T] # Note how `Number` is just a type constraint here, while @@ -5798,25 +6673,27 @@ Static params can also appear in the signatures of generic types: var m1: AffineTransform3D[float] # OK var m2: AffineTransform2D[string] # Error, `string` is not a `Number` + ``` Please note that `static T` is just a syntactic convenience for the underlying -generic type `static[T]`. The type param can be omitted to obtain the type +generic type `static[T]`. The type parameter can be omitted to obtain the type class of all constant expressions. A more specific type class can be created by instantiating `static` with another type class. One can force an expression to be evaluated at compile time as a constant expression by coercing it to a corresponding `static` type: -.. code-block:: nim + ```nim import std/math echo static(fac(5)), " ", static[bool](16.isPowerOfTwo) + ``` The compiler will report any failure to evaluate the expression or a possible type mismatch error. -typedesc[T] ------------ +typedesc\[T] +------------ In many contexts, Nim treats the names of types as regular values. These values exist only during the compilation phase, but since @@ -5824,48 +6701,46 @@ all values must have a type, `typedesc` is considered their special type. `typedesc` acts as a generic type. For instance, the type of the symbol `int` is `typedesc[int]`. Just like with regular generic types, when the -generic param is omitted, `typedesc` denotes the type class of all types. +generic parameter is omitted, `typedesc` denotes the type class of all types. As a syntactic convenience, one can also use `typedesc` as a modifier. -Procs featuring `typedesc` params are considered implicitly generic. +Procs featuring `typedesc` parameters are considered implicitly generic. They will be instantiated for each unique combination of supplied types, -and within the body of the proc, the name of each param will refer to +and within the body of the proc, the name of each parameter will refer to the bound concrete type: -.. code-block:: nim - + ```nim proc new(T: typedesc): ref T = echo "allocating ", T.name new(result) var n = Node.new var tree = new(BinaryTree[int]) + ``` -When multiple type params are present, they will bind freely to different -types. To force a bind-once behavior, one can use an explicit generic param: +When multiple type parameters are present, they will bind freely to different +types. To force a bind-once behavior, one can use an explicit generic parameter: -.. code-block:: nim + ```nim proc acceptOnlyTypePairs[T, U](A, B: typedesc[T]; C, D: typedesc[U]) + ``` -Once bound, type params can appear in the rest of the proc signature: - -.. code-block:: nim - :test: "nim c $1" +Once bound, type parameters can appear in the rest of the proc signature: + ```nim test = "nim c $1" template declareVariableWithType(T: typedesc, value: T) = var x: T = value declareVariableWithType int, 42 + ``` Overload resolution can be further influenced by constraining the set -of types that will match the type param. This works in practice by +of types that will match the type parameter. This works in practice by attaching attributes to types via templates. The constraint can be a concrete type or a type class. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template maxval(T: typedesc[int]): int = high(int) template maxval(T: typedesc[float]): float = Inf @@ -5881,13 +6756,13 @@ concrete type or a type class. echo "is int a number? ", isNumber(int) echo "is float a number? ", isNumber(float) echo "is RootObj a number? ", isNumber(RootObj) + ``` Passing `typedesc` is almost identical, just with the difference that the macro is not instantiated generically. The type expression is simply passed as a `NimNode` to the macro, like everything else. -.. code-block:: nim - + ```nim import std/macros macro forwardType(arg: typedesc): typedesc = @@ -5896,6 +6771,7 @@ simply passed as a `NimNode` to the macro, like everything else. result = tmp var tmp: forwardType(int) + ``` typeof operator --------------- @@ -5907,10 +6783,10 @@ One can obtain the type of a given expression by constructing a `typeof` value from it (in many other languages this is known as the `typeof`:idx: operator): -.. code-block:: nim - + ```nim var x = 0 var y: typeof(x) # y has type int + ``` If `typeof` is used to determine the result type of a proc/iterator/converter @@ -5919,16 +6795,15 @@ interpretation, where `c` is an iterator, is preferred over the other interpretations, but this behavior can be changed by passing `typeOfProc` as the second argument to `typeof`: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" iterator split(s: string): string = discard proc split(s: string): seq[string] = discard - # since an iterator is the preferred interpretation, `y` has the type `string`: + # since an iterator is the preferred interpretation, this has the type `string`: assert typeof("a b c".split) is string assert typeof("a b c".split, typeOfProc) is seq[string] + ``` @@ -5937,11 +6812,11 @@ Modules Nim supports splitting a program into pieces by a module concept. Each module needs to be in its own file and has its own `namespace`:idx:. Modules enable `information hiding`:idx: and `separate compilation`:idx:. -A module may gain access to symbols of another module by the `import`:idx: +A module may gain access to the symbols of another module by the `import`:idx: statement. `Recursive module dependencies`:idx: are allowed, but are slightly subtle. Only top-level symbols that are marked with an asterisk (`*`) are exported. A valid module name can only be a valid Nim identifier (and thus its -filename is `identifier.nim`). +filename is ``identifier.nim``). The algorithm for compiling modules is: @@ -5952,7 +6827,7 @@ The algorithm for compiling modules is: This is best illustrated by an example: -.. code-block:: nim + ```nim # Module A type T1* = int # Module A exports the type `T1` @@ -5962,9 +6837,10 @@ This is best illustrated by an example: var i = p(3) # works because B has been parsed completely here main() + ``` -.. code-block:: nim + ```nim # Module B import A # A is not parsed here! Only the already known symbols # of A are imported. @@ -5973,31 +6849,32 @@ This is best illustrated by an example: # this works because the compiler has already # added T1 to A's interface symbol table result = x + 1 + ``` Import statement ---------------- -After the `import` statement, a list of module names can follow or a single +After the `import` keyword, a list of module names can follow or a single module name followed by an `except` list to prevent some symbols from being imported: -.. code-block:: nim - :test: "nim c $1" - :status: 1 - + ```nim test = "nim c $1" status = 1 import std/strutils except `%`, toUpperAscii # doesn't work then: echo "$1" % "abc".toUpperAscii + ``` It is not checked that the `except` list is really exported from the module. -This feature allows us to compile against an older version of the module that -does not export these identifiers. +This feature allows us to compile against different versions of the module, +even when one version does not export some of these identifiers. The `import` statement is only allowed at the top level. +String literals can be used for import/include statements. +The compiler performs [path substitution](nimc.html#compiler-usage-commandminusline-switches) when used. Include statement ----------------- @@ -6006,51 +6883,58 @@ The `include` statement does something fundamentally different than importing a module: it merely includes the contents of a file. The `include` statement is useful to split up a large module into several files: -.. code-block:: nim + ```nim include fileA, fileB, fileC + ``` -The `include` statement can be used outside of the top level, as such: +The `include` statement can be used outside the top level, as such: -.. code-block:: nim + ```nim # Module A echo "Hello World!" + ``` -.. code-block:: nim + ```nim # Module B proc main() = include A main() # => Hello World! + ``` Module names in imports ----------------------- -A module alias can be introduced via the `as` keyword: +A module alias can be introduced via the `as` keyword, after which the original module name +is inaccessible: -.. code-block:: nim + ```nim import std/strutils as su, std/sequtils as qu echo su.format("$1", "lalelu") + ``` -The original module name is then not accessible. The notations -`path/to/module` or `"path/to/module"` can be used to refer to a module +The notations `path/to/module` or `"path/to/module"` can be used to refer to a module in subdirectories: -.. code-block:: nim + ```nim import lib/pure/os, "lib/pure/times" + ``` -Note that the module name is still `strutils` and not `lib/pure/strutils` -and so one **cannot** do: +Note that the module name is still `strutils` and not `lib/pure/strutils`, +thus one **cannot** do: -.. code-block:: nim + ```nim import lib/pure/strutils echo lib/pure/strutils.toUpperAscii("abc") + ``` Likewise, the following does not make sense as the name is `strutils` already: -.. code-block:: nim + ```nim import lib/pure/strutils as strutils + ``` Collective imports from a directory @@ -6062,8 +6946,9 @@ from the same directory. Path names are syntactically either Nim identifiers or string literals. If the path name is not a valid Nim identifier it needs to be a string literal: -.. code-block:: nim + ```nim import "gfx/3d/somemodule" # in quotes because '3d' is not a valid Nim identifier + ``` Pseudo import/include paths @@ -6075,29 +6960,29 @@ avoid ambiguity when there are multiple modules with the same path. There are two pseudo directories: 1. `std`: The `std` pseudo directory is the abstract location of Nim's standard -library. For example, the syntax `import std / strutils` is used to unambiguously -refer to the standard library's `strutils` module. + library. For example, the syntax `import std / strutils` is used to unambiguously + refer to the standard library's `strutils` module. 2. `pkg`: The `pkg` pseudo directory is used to unambiguously refer to a Nimble -package. However, for technical details that lie outside the scope of this document, -its semantics are: *Use the search path to look for module name but ignore the standard -library locations*. In other words, it is the opposite of `std`. + package. However, for technical details that lie outside the scope of this document, + its semantics are: *Use the search path to look for module name but ignore the standard + library locations*. In other words, it is the opposite of `std`. +It is recommended and preferred but not currently enforced that all stdlib module imports include the std/ "pseudo directory" as part of the import name. From import statement --------------------- -After the `from` statement, a module name follows followed by +After the `from` keyword, a module name followed by an `import` to list the symbols one likes to use without explicit full qualification: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" from std/strutils import `%` echo "$1" % "abc" # always possible: full qualification: echo strutils.replace("abc", "a", "z") + ``` It's also possible to use `from module import nil` if one wants to import the module but wants to enforce fully qualified access to every symbol @@ -6110,34 +6995,38 @@ Export statement An `export` statement can be used for symbol forwarding so that client modules don't need to import a module's dependencies: -.. code-block:: nim + ```nim # module B type MyObject* = object + ``` -.. code-block:: nim + ```nim # module A import B export B.MyObject proc `$`*(x: MyObject): string = "my object" + ``` -.. code-block:: nim + ```nim # module C import A # B.MyObject has been imported implicitly here: var x: MyObject echo $x + ``` When the exported symbol is another module, all of its definitions will be forwarded. One can use an `except` list to exclude some of the symbols. Notice that when exporting, one needs to specify only the module name: -.. code-block:: nim + ```nim import foo/bar/baz export baz + ``` @@ -6148,8 +7037,8 @@ the block in which the declaration occurred. The range where the identifier is known is the scope of the identifier. The exact scope of an identifier depends on the way it was declared. -Block scope -~~~~~~~~~~~ +### Block scope + The *scope* of a variable declared in the declaration part of a block is valid from the point of declaration until the end of the block. If a block contains a second block, in which the identifier is redeclared, @@ -6159,8 +7048,8 @@ identifier cannot be redefined in the same block, except if valid for procedure or iterator overloading purposes. -Tuple or object scope -~~~~~~~~~~~~~~~~~~~~~ +### Tuple or object scope + The field identifiers inside a tuple or object definition are valid in the following places: @@ -6168,32 +7057,99 @@ following places: * Field designators of a variable of the given tuple/object type. * In all descendant types of the object type. -Module scope -~~~~~~~~~~~~ +### Module scope + All identifiers of a module are valid from the point of declaration until the end of the module. Identifiers from indirectly dependent modules are *not* available. The `system`:idx: module is automatically imported in every module. -If a module imports an identifier by two different modules, each occurrence of -the identifier has to be qualified unless it is an overloaded procedure or -iterator in which case the overloading resolution takes place: +If a module imports the same identifier from two different modules, the +identifier is considered ambiguous, which can be resolved in the following ways: + +* Qualifying the identifier as `module.identifier` resolves ambiguity + between modules. (See below for the case that the module name itself + is ambiguous.) +* Calling the identifier as a routine makes overload resolution take place, + which resolves ambiguity in the case that one overload matches stronger + than the others. +* Using the identifier in a context where the compiler can infer the type + of the identifier resolves ambiguity in the case that one definition + matches the type stronger than the others. -.. code-block:: nim + ```nim # Module A var x*: string + proc foo*(a: string) = + echo "A: ", a + ``` -.. code-block:: nim + ```nim # Module B var x*: int + proc foo*(b: int) = + echo "B: ", b + ``` -.. code-block:: nim + ```nim # Module C import A, B + + foo("abc") # A: abc + foo(123) # B: 123 + let inferred: proc (x: string) = foo + foo("def") # A: def + write(stdout, x) # error: x is ambiguous write(stdout, A.x) # no error: qualifier used + + proc bar(a: int): int = a + 1 + assert bar(x) == x + 1 # no error: only A.x of type int matches var x = 4 write(stdout, x) # not ambiguous: uses the module C's x + ``` +Modules can share their name, however, when trying to qualify an identifier with the module name the compiler will fail with ambiguous identifier error. One can qualify the identifier by aliasing the module. + + +```nim +# Module A/C +proc fb* = echo "fizz" +``` + + +```nim +# Module B/C +proc fb* = echo "buzz" +``` + + +```nim +import A/C +import B/C + +C.fb() # Error: ambiguous identifier: 'C' +``` + + +```nim +import A/C as fizz +import B/C + +fizz.fb() # Works +``` + + +Packages +-------- +A collection of modules in a file tree with an ``identifier.nimble`` file in the +root of the tree is called a Nimble package. A valid package name can only be a +valid Nim identifier and thus its filename is ``identifier.nimble`` where +``identifier`` is the desired package name. A module without a ``.nimble`` file +is assigned the package identifier: `unknown`. + +The distinction between packages allows diagnostic compiler messages to be +scoped to the current project's package vs foreign packages. + Compiler Messages @@ -6221,65 +7177,18 @@ deprecated pragma The deprecated pragma is used to mark a symbol as deprecated: -.. code-block:: nim + ```nim proc p() {.deprecated.} var x {.deprecated.}: char + ``` This pragma can also take in an optional warning string to relay to developers. -.. code-block:: nim + ```nim proc thing(x: bool) {.deprecated: "use thong instead".} + ``` -noSideEffect pragma -------------------- - -The `noSideEffect` pragma is used to mark a proc/iterator that can have only -side effects through parameters. This means that the proc/iterator only changes locations that are -reachable from its parameters and the return value only depends on the -parameters. If none of its parameters have the type `var`, `ref`, `ptr`, `cstring`, or `proc`, -then no locations are modified. - -It is a static error to mark a proc/iterator to have no side effect if the compiler cannot verify this. - -As a special semantic rule, the built-in `debugEcho -<system.html#debugEcho,varargs[typed,]>`_ pretends to be free of side effects -so that it can be used for debugging routines marked as `noSideEffect`. - -`func` is syntactic sugar for a proc with no side effects: - -.. code-block:: nim - func `+` (x, y: int): int - - -To override the compiler's side effect analysis a `{.noSideEffect.}` -`cast` pragma block can be used: - -.. code-block:: nim - - func f() = - {.cast(noSideEffect).}: - echo "test" - -When a `noSideEffect` proc has proc params `bar`, whether it can be used inside a `noSideEffect` context -depends on what the compiler knows about `bar`: - -.. code-block:: nim - :test: "nim c $1" - - func foo(bar: proc(): int): int = bar() - var count = 0 - proc fn1(): int = 1 - proc fn2(): int = (count.inc; count) - func fun1() = discard foo(fn1) # ok because fn1 is inferred as `func` - # func fun2() = discard foo(fn2) # would give: Error: 'fun2' can have side effects - - # with callbacks, the compiler is conservative, ie that bar will have side effects - var foo2: type(foo) = foo - func main() = - discard foo(fn1) # ok - # discard foo2(fn1) # now this errors - compileTime pragma ------------------ @@ -6289,23 +7198,23 @@ procs are useful as helpers for macros. Since version 0.12.0 of the language, a proc that uses `system.NimNode` within its parameter types is implicitly declared `compileTime`: -.. code-block:: nim + ```nim proc astHelper(n: NimNode): NimNode = result = n + ``` Is the same as: -.. code-block:: nim + ```nim proc astHelper(n: NimNode): NimNode {.compileTime.} = result = n + ``` `compileTime` variables are available at runtime too. This simplifies certain idioms where variables are filled at compile-time (for example, lookup tables) but accessed at runtime: -.. code-block:: nim - :test: "nim c -r $1" - + ```nim test = "nim c -r $1" import std/macros var nameToProc {.compileTime.}: seq[(string, proc (): string {.nimcall.})] @@ -6323,9 +7232,10 @@ but accessed at runtime: proc baz: string {.registerProc.} = "baz" doAssert nameToProc[2][1]() == "baz" + ``` -noReturn pragma +noreturn pragma --------------- The `noreturn` pragma is used to mark a proc that never returns. @@ -6336,20 +7246,22 @@ The `acyclic` pragma can be used for object types to mark them as acyclic even though they seem to be cyclic. This is an **optimization** for the garbage collector to not consider objects of this type as part of a cycle: -.. code-block:: nim + ```nim type Node = ref NodeObj NodeObj {.acyclic.} = object left, right: Node data: string + ``` Or if we directly use a ref object: -.. code-block:: nim + ```nim type Node {.acyclic.} = ref object left, right: Node data: string + ``` In the example, a tree structure is declared with the `Node` type. Note that the type definition is recursive and the GC has to assume that objects of @@ -6377,7 +7289,7 @@ because the semantics of Nim require deep copying of sequences and strings. This can be expensive, especially if sequences are used to build a tree structure: -.. code-block:: nim + ```nim type NodeKind = enum nkLeaf, nkInner Node {.shallow.} = object @@ -6386,6 +7298,7 @@ structure: strVal: string of nkInner: children: seq[Node] + ``` pure pragma @@ -6403,7 +7316,7 @@ asmNoStackFrame pragma A proc can be marked with the `asmNoStackFrame` pragma to tell the compiler it should not generate a stack frame for the proc. There are also no exit statements like `return result;` generated and the generated C function is -declared as `__declspec(naked)` or `__attribute__((naked))` (depending on +declared as `__declspec(naked)`:c: or `__attribute__((naked))`:c: (depending on the used C compiler). **Note**: This pragma should only be used by procs which consist solely of @@ -6420,9 +7333,10 @@ annotate a symbol (like an iterator or proc). The *usage* of the symbol then triggers a static error. This is especially useful to rule out that some operation is valid due to overloading and type conversions: -.. code-block:: nim + ```nim ## check that underlying int values are compared and not the pointers: proc `==`(x, y: ptr int): bool {.error.} + ``` fatal pragma @@ -6431,9 +7345,10 @@ The `fatal` pragma is used to make the compiler output an error message with the given content. In contrast to the `error` pragma, the compilation is guaranteed to be aborted by this pragma. Example: -.. code-block:: nim + ```nim when not defined(objc): {.fatal: "Compile this program with the objc command!".} + ``` warning pragma -------------- @@ -6450,17 +7365,17 @@ line pragma The `line` pragma can be used to affect line information of the annotated statement, as seen in stack backtraces: -.. code-block:: nim - + ```nim template myassert*(cond: untyped, msg = "") = if not cond: # change run-time line information of the 'raise' statement: {.line: instantiationInfo().}: - raise newException(EAssertionFailed, msg) + raise newException(AssertionDefect, msg) + ``` -If the `line` pragma is used with a parameter, the parameter needs be a +If the `line` pragma is used with a parameter, the parameter needs to be a `tuple[filename: string, line: int]`. If it is used without a parameter, -`system.InstantiationInfo()` is used. +`system.instantiationInfo()` is used. linearScanEnd pragma @@ -6469,7 +7384,7 @@ The `linearScanEnd` pragma can be used to tell the compiler how to compile a Nim `case`:idx: statement. Syntactically it has to be used as a statement: -.. code-block:: nim + ```nim case myInt of 0: echo "most common case" @@ -6478,9 +7393,10 @@ statement: echo "second most common case" of 2: echo "unlikely: use branch table" else: echo "unlikely too: use branch table for ", myInt + ``` In the example, the case branches `0` and `1` are much more common than -the other cases. Therefore the generated assembler code should test for these +the other cases. Therefore, the generated assembler code should test for these values first so that the CPU's branch predictor has a good chance to succeed (avoiding an expensive CPU pipeline stall). The other cases might be put into a jump table for O(1) overhead but at the cost of a (very likely) pipeline @@ -6497,8 +7413,7 @@ The `computedGoto` pragma can be used to tell the compiler how to compile a Nim `case`:idx: in a `while true` statement. Syntactically it has to be used as a statement inside the loop: -.. code-block:: nim - + ```nim type MyEnum = enum enumA, enumB, enumC, enumD, enumE @@ -6530,6 +7445,7 @@ Syntactically it has to be used as a statement inside the loop: inc(pc) vm() + ``` As the example shows, `computedGoto` is mostly useful for interpreters. If the underlying backend (C compiler) does not support the computed goto @@ -6539,9 +7455,24 @@ extension the pragma is simply ignored. immediate pragma ---------------- -The immediate pragma is obsolete. See `Typed vs untyped parameters -<#templates-typed-vs-untyped-parameters>`_. +The immediate pragma is obsolete. See [Typed vs untyped parameters]. + +redefine pragma +--------------- + +Redefinition of template symbols with the same signature is allowed. +This can be made explicit with the `redefine` pragma: + +```nim +template foo: int = 1 +echo foo() # 1 +template foo: int {.redefine.} = 2 +echo foo() # 2 +# warning: implicit redefinition of template +template foo: int = 3 +``` +This is mostly intended for macro generated code. compilation option pragmas -------------------------- @@ -6579,9 +7510,10 @@ callconv cdecl|... Specifies the default calling convention for Example: -.. code-block:: nim + ```nim {.checks: off, optimization: speed.} # compile without runtime checks and optimize for speed + ``` push and pop pragmas @@ -6589,16 +7521,17 @@ push and pop pragmas The `push/pop`:idx: pragmas are very similar to the option directive, but are used to override the settings temporarily. Example: -.. code-block:: nim + ```nim {.push checks: off.} # compile this section without runtime checks as it is # speed critical # ... some code ... {.pop.} # restore old settings + ``` `push/pop`:idx: can switch on/off some standard library pragmas, example: -.. code-block:: nim + ```nim {.push inline.} proc thisIsInlined(): int = 42 func willBeInlined(): float = 42.0 @@ -6609,9 +7542,10 @@ but are used to override the settings temporarily. Example: template example(): string = "https://nim-lang.org" {.pop.} - {.push deprecated, hint[LineTooLong]: off, used, stackTrace: off.} + {.push deprecated, used, stackTrace: off.} proc sample(): bool = true {.pop.} + ``` For third party pragmas, it depends on its implementation but uses the same syntax. @@ -6633,10 +7567,11 @@ The `global` pragma can be applied to a variable within a proc to instruct the compiler to store it in a global location and initialize it once at program startup. -.. code-block:: nim + ```nim proc isHexNumber(s: string): bool = var pattern {.global.} = re"[0-9a-fA-F]+" result = s.match(pattern) + ``` When used within a generic proc, a separate unique global variable will be created for each instantiation of the proc. The order of initialization of @@ -6646,13 +7581,15 @@ and before any variable in a module that imports it. Disabling certain messages -------------------------- -Nim generates some warnings and hints ("line too long") that may annoy the +Nim generates some warnings and hints that may annoy the user. A mechanism for disabling certain messages is provided: Each hint -and warning message contains a symbol in brackets. This is the message's -identifier that can be used to enable or disable it: +and warning message is associated with a symbol. This is the message's +identifier, which can be used to enable or disable the message by putting it +in brackets following the pragma: -.. code-block:: Nim - {.hint[LineTooLong]: off.} # turn off the hint about too long lines + ```Nim + {.hint[XDeclaredButNotUsed]: off.} # Turn off the hint about declared but not used symbols. + ``` This is often better than disabling all warnings at once. @@ -6664,7 +7601,7 @@ Nim produces a warning for symbols that are not exported and not used either. The `used` pragma can be attached to a symbol to suppress this warning. This is particularly useful when the symbol was generated by a macro: -.. code-block:: nim + ```nim template implementArithOps(T) = proc echoAdd(a, b: T) {.used.} = echo a + b @@ -6674,18 +7611,19 @@ is particularly useful when the symbol was generated by a macro: # no warning produced for the unused 'echoSub' implementArithOps(int) echoAdd 3, 5 + ``` `used` can also be used as a top-level statement to mark a module as "used". This prevents the "Unused import" warning: -.. code-block:: nim - + ```nim # module: debughelper.nim when defined(nimHasUsed): # 'import debughelper' is so useful for debugging # that Nim shouldn't produce a warning for that import, # even if currently unused: {.used.} + ``` experimental pragma @@ -6694,11 +7632,12 @@ experimental pragma The `experimental` pragma enables experimental language features. Depending on the concrete feature, this means that the feature is either considered too unstable for an otherwise stable release or that the future of the feature -is uncertain (it may be removed at any time). +is uncertain (it may be removed at any time). See the +[experimental manual](manual_experimental.html) for more details. Example: -.. code-block:: nim + ```nim import std/threadpool {.experimental: "parallel".} @@ -6711,6 +7650,7 @@ Example: spawn threadedEcho("echo in parallel", i) useParallel() + ``` As a top-level statement, the experimental pragma enables a feature for the @@ -6718,8 +7658,7 @@ rest of the module it's enabled in. This is problematic for macro and generic instantiations that cross a module scope. Currently, these usages have to be put into a `.push/pop` environment: -.. code-block:: nim - + ```nim # client.nim proc useParallel*[T](unused: T) = # use a generic T here to show the problem. @@ -6729,12 +7668,13 @@ put into a `.push/pop` environment: echo "echo in parallel" {.pop.} + ``` -.. code-block:: nim - + ```nim import client useParallel(1) + ``` Implementation Specific Pragmas @@ -6749,17 +7689,50 @@ Bitsize pragma The `bitsize` pragma is for object field members. It declares the field as a bitfield in C/C++. -.. code-block:: Nim + ```Nim type mybitfield = object flag {.bitsize:1.}: cuint + ``` generates: -.. code-block:: C + ```C struct mybitfield { unsigned int flag:1; }; + ``` + + +size pragma +----------- +Nim automatically determines the size of an enum. +But when wrapping a C enum type, it needs to be of a specific size. +The `size pragma` allows specifying the size of the enum type. + + ```Nim + type + EventType* {.size: sizeof(uint32).} = enum + QuitEvent, + AppTerminating, + AppLowMemory + + doAssert sizeof(EventType) == sizeof(uint32) + ``` + +The `size pragma` can also specify the size of an `importc` incomplete object type +so that one can get the size of it at compile time even if it was declared without fields. + + ```Nim + type + AtomicFlag* {.importc: "atomic_flag", header: "<stdatomic.h>", size: 1.} = object + + static: + # if AtomicFlag didn't have the size pragma, this code would result in a compile time error. + echo sizeof(AtomicFlag) + ``` + +The `size pragma` accepts only the values 1, 2, 4 or 8. Align pragma @@ -6772,33 +7745,48 @@ alignments that are weaker than other align pragmas on the same declaration are ignored. Alignments that are weaker than the alignment requirement of the type are ignored. -.. code-block:: Nim - - type - sseType = object - sseData {.align(16).}: array[4, float32] + ```Nim + type + sseType = object + sseData {.align(16).}: array[4, float32] - # every object will be aligned to 128-byte boundary - Data = object - x: char - cacheline {.align(128).}: array[128, char] # over-aligned array of char, + # every object will be aligned to 128-byte boundary + Data = object + x: char + cacheline {.align(128).}: array[128, char] # over-aligned array of char, - proc main() = - echo "sizeof(Data) = ", sizeof(Data), " (1 byte + 127 bytes padding + 128-byte array)" - # output: sizeof(Data) = 256 (1 byte + 127 bytes padding + 128-byte array) - echo "alignment of sseType is ", alignof(sseType) - # output: alignment of sseType is 16 - var d {.align(2048).}: Data # this instance of data is aligned even stricter + proc main() = + echo "sizeof(Data) = ", sizeof(Data), " (1 byte + 127 bytes padding + 128-byte array)" + # output: sizeof(Data) = 256 (1 byte + 127 bytes padding + 128-byte array) + echo "alignment of sseType is ", alignof(sseType) + # output: alignment of sseType is 16 + var d {.align(2048).}: Data # this instance of data is aligned even stricter - main() + main() + ``` This pragma has no effect on the JS backend. +Noalias pragma +-------------- + +Since version 1.4 of the Nim compiler, there is a `.noalias` annotation for variables +and parameters. It is mapped directly to C/C++'s `restrict`:c: keyword and means that +the underlying pointer is pointing to a unique location in memory, no other aliases to +this location exist. It is *unchecked* that this alias restriction is followed. If the +restriction is violated, the backend optimizer is free to miscompile the code. +This is an **unsafe** language feature. + +Ideally in later versions of the language, the restriction will be enforced at +compile time. (This is also why the name `noalias` was chosen instead of a more +verbose name like `unsafeAssumeNoAlias`.) + + Volatile pragma --------------- The `volatile` pragma is for variables only. It declares the variable as -`volatile`, whatever that means in C/C++ (its semantics are not well defined +`volatile`:c:, whatever that means in C/C++ (its semantics are not well-defined in C/C++). **Note**: This pragma will not exist for the LLVM backend. @@ -6811,10 +7799,11 @@ type, etc.) and is sometimes useful for interoperability with C: It tells Nim that it should not generate a declaration for the symbol in the C code. For example: -.. code-block:: Nim + ```Nim var EACCES {.importc, nodecl.}: cint # pretend EACCES was a variable, as # Nim does not know its value + ``` However, the `header` pragma is often the better alternative. @@ -6825,17 +7814,18 @@ Header pragma ------------- The `header` pragma is very similar to the `nodecl` pragma: It can be applied to almost any symbol and specifies that it should not be declared -and instead, the generated code should contain an `#include`: +and instead, the generated code should contain an `#include`:c:\: -.. code-block:: Nim + ```Nim type PFile {.importc: "FILE*", header: "<stdio.h>".} = distinct pointer # import C's FILE* type; Nim will treat it as a new pointer type + ``` The `header` pragma always expects a string constant. The string constant contains the header file: As usual for C, a system header file is enclosed -in angle brackets: `<>`. If no angle brackets are given, Nim -encloses the header file in `""` in the generated C code. +in angle brackets: `<>`:c:. If no angle brackets are given, Nim +encloses the header file in `""`:c: in the generated C code. **Note**: This will not work for the LLVM backend. @@ -6843,12 +7833,13 @@ encloses the header file in `""` in the generated C code. IncompleteStruct pragma ----------------------- The `incompleteStruct` pragma tells the compiler to not use the -underlying C `struct` in a `sizeof` expression: +underlying C `struct`:c: in a `sizeof` expression: -.. code-block:: Nim + ```Nim type DIR* {.importc: "DIR", header: "<dirent.h>", pure, incompleteStruct.} = object + ``` Compile pragma @@ -6856,17 +7847,25 @@ Compile pragma The `compile` pragma can be used to compile and link a C/C++ source file with the project: -.. code-block:: Nim +This pragma can take three forms. The first is a simple file input: + ```Nim {.compile: "myfile.cpp".} + ``` + +The second form is a tuple where the second arg is the output name strutils formatter: + ```Nim + {.compile: ("file.c", "$1.o").} + ``` **Note**: Nim computes a SHA1 checksum and only recompiles the file if it -has changed. One can use the `-f` command-line option to force the recompilation -of the file. +has changed. One can use the `-f`:option: command-line option to force +the recompilation of the file. Since 1.4 the `compile` pragma is also available with this syntax: -.. code-block:: Nim + ```Nim {.compile("myfile.cpp", "--custom flags here").} + ``` As can be seen in the example, this new variant allows for custom flags that are passed to the C compiler when the file is recompiled. @@ -6876,52 +7875,58 @@ Link pragma ----------- The `link` pragma can be used to link an additional file with the project: -.. code-block:: Nim + ```Nim {.link: "myfile.o".} + ``` -PassC pragma +passc pragma ------------ The `passc` pragma can be used to pass additional parameters to the C -compiler like one would using the command-line switch `--passc`: +compiler like one would use the command-line switch `--passc`:option:\: -.. code-block:: Nim + ```Nim {.passc: "-Wall -Werror".} + ``` -Note that one can use `gorge` from the `system module <system.html>`_ to +Note that one can use `gorge` from the [system module](system.html) to embed parameters from an external command that will be executed during semantic analysis: -.. code-block:: Nim + ```Nim {.passc: gorge("pkg-config --cflags sdl").} + ``` -LocalPassc pragma +localPassC pragma ----------------- -The `localPassc` pragma can be used to pass additional parameters to the C +The `localPassC` pragma can be used to pass additional parameters to the C compiler, but only for the C/C++ file that is produced from the Nim module the pragma resides in: -.. code-block:: Nim + ```Nim # Module A.nim # Produces: A.nim.cpp - {.localPassc: "-Wall -Werror".} # Passed when compiling A.nim.cpp + {.localPassC: "-Wall -Werror".} # Passed when compiling A.nim.cpp + ``` -PassL pragma +passl pragma ------------ -The `passL` pragma can be used to pass additional parameters to the linker -like one would be using the command-line switch `--passL`: +The `passl` pragma can be used to pass additional parameters to the linker +like one would be using the command-line switch `--passl`:option:\: -.. code-block:: Nim - {.passL: "-lSDLmain -lSDL".} + ```Nim + {.passl: "-lSDLmain -lSDL".} + ``` -Note that one can use `gorge` from the `system module <system.html>`_ to +Note that one can use `gorge` from the [system module](system.html) to embed parameters from an external command that will be executed during semantic analysis: -.. code-block:: Nim - {.passL: gorge("pkg-config --libs sdl").} + ```Nim + {.passl: gorge("pkg-config --libs sdl").} + ``` Emit pragma @@ -6933,7 +7938,7 @@ extremely useful for interfacing with `C++`:idx: or `Objective C`:idx: code. Example: -.. code-block:: Nim + ```Nim {.emit: """ static int cvariable = 420; """.} @@ -6946,27 +7951,29 @@ Example: {.pop.} embedsC() + ``` -`nimbase.h` defines `NIM_EXTERNC` C macro that can be used for -`extern "C"` code to work with both `nim c` and `nim cpp`, e.g.: +``nimbase.h`` defines `NIM_EXTERNC`:c: C macro that can be used for +`extern "C"`:cpp: code to work with both `nim c`:cmd: and `nim cpp`:cmd:, e.g.: -.. code-block:: Nim + ```Nim proc foobar() {.importc:"$1".} {.emit: """ #include <stdio.h> NIM_EXTERNC void fun(){} """.} + ``` -For backward compatibility, if the argument to the `emit` statement -is a single string literal, Nim symbols can be referred to via backticks. -This usage is however deprecated. +.. note:: For backward compatibility, if the argument to the `emit` statement + is a single string literal, Nim symbols can be referred to via backticks. + This usage is however deprecated. For a top-level emit statement, the section where in the generated C/C++ file -the code should be emitted can be influenced via the -prefixes `/*TYPESECTION*/` or `/*VARSECTION*/` or `/*INCLUDESECTION*/`: +the code should be emitted can be influenced via the prefixes +`/*TYPESECTION*/`:c: or `/*VARSECTION*/`:c: or `/*INCLUDESECTION*/`:c:\: -.. code-block:: Nim + ```Nim {.emit: """/*TYPESECTION*/ struct Vector3 { public: @@ -6980,24 +7987,25 @@ prefixes `/*TYPESECTION*/` or `/*VARSECTION*/` or `/*INCLUDESECTION*/`: x: cfloat proc constructVector3(a: cfloat): Vector3 {.importcpp: "Vector3(@)", nodecl} + ``` ImportCpp pragma ---------------- -**Note**: `c2nim <https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst>`_ can parse a large subset of C++ and knows +**Note**: [c2nim](https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst) +can parse a large subset of C++ and knows about the `importcpp` pragma pattern language. It is not necessary to know all the details described here. -Similar to the `importc pragma for C -<#foreign-function-interface-importc-pragma>`_, the +Similar to the [importc pragma] for C, the `importcpp` pragma can be used to import `C++`:idx: methods or C++ symbols in general. The generated code then uses the C++ method calling -syntax: `obj->method(arg)`. In combination with the `header` and `emit` +syntax: `obj->method(arg)`:cpp:. In combination with the `header` and `emit` pragmas this allows *sloppy* interfacing with libraries written in C++: -.. code-block:: Nim + ```Nim # Horrible example of how to interface with a C++ engine ... ;-) {.link: "/usr/lib/libIrrlicht.so".} @@ -7023,138 +8031,147 @@ pragmas this allows *sloppy* interfacing with libraries written in C++: header: irr, importcpp: "createDevice(@)".} proc run(device: IrrlichtDevice): bool {. header: irr, importcpp: "#.run(@)".} + ``` -The compiler needs to be told to generate C++ (command `cpp`) for +The compiler needs to be told to generate C++ (command `cpp`:option:) for this to work. The conditional symbol `cpp` is defined when the compiler emits C++ code. -Namespaces -~~~~~~~~~~ +### Namespaces -The *sloppy interfacing* example uses `.emit` to produce `using namespace` +The *sloppy interfacing* example uses `.emit` to produce `using namespace`:cpp: declarations. It is usually much better to instead refer to the imported name -via the `namespace::identifier` notation: +via the `namespace::identifier`:cpp: notation: -.. code-block:: nim + ```nim type IrrlichtDeviceObj {.header: irr, importcpp: "irr::IrrlichtDevice".} = object + ``` -Importcpp for enums -~~~~~~~~~~~~~~~~~~~ +### Importcpp for enums When `importcpp` is applied to an enum type the numerical enum values are -annotated with the C++ enum type, like in this example: `((TheCppEnum)(3))`. +annotated with the C++ enum type, like in this example: +`((TheCppEnum)(3))`:cpp:. (This turned out to be the simplest way to implement it.) -Importcpp for procs -~~~~~~~~~~~~~~~~~~~ +### Importcpp for procs Note that the `importcpp` variant for procs uses a somewhat cryptic pattern language for maximum flexibility: -- A hash `#` symbol is replaced by the first or next argument. -- A dot following the hash `#.` indicates that the call should use C++'s dot +- A hash ``#`` symbol is replaced by the first or next argument. +- A dot following the hash ``#.`` indicates that the call should use C++'s dot or arrow notation. -- An at symbol `@` is replaced by the remaining arguments, separated by commas. +- An at symbol ``@`` is replaced by the remaining arguments, + separated by commas. For example: -.. code-block:: nim + ```nim proc cppMethod(this: CppObj, a, b, c: cint) {.importcpp: "#.CppMethod(@)".} var x: ptr CppObj cppMethod(x[], 1, 2, 3) + ``` Produces: -.. code-block:: C + ```C x->CppMethod(1, 2, 3) + ``` As a special rule to keep backward compatibility with older versions of the `importcpp` pragma, if there is no special pattern -character (any of `# ' @`) at all, C++'s +character (any of ``# ' @``) at all, C++'s dot or arrow notation is assumed, so the above example can also be written as: -.. code-block:: nim + ```nim proc cppMethod(this: CppObj, a, b, c: cint) {.importcpp: "CppMethod".} + ``` Note that the pattern language naturally also covers C++'s operator overloading capabilities: -.. code-block:: nim + ```nim proc vectorAddition(a, b: Vec3): Vec3 {.importcpp: "# + #".} proc dictLookup(a: Dict, k: Key): Value {.importcpp: "#[#]".} + ``` -- An apostrophe `'` followed by an integer `i` in the range 0..9 +- An apostrophe ``'`` followed by an integer ``i`` in the range 0..9 is replaced by the i'th parameter *type*. The 0th position is the result type. This can be used to pass types to C++ function templates. Between - the `'` and the digit, an asterisk can be used to get to the base type - of the type. (So it "takes away a star" from the type; `T*` becomes `T`.) + the ``'`` and the digit, an asterisk can be used to get to the base type + of the type. (So it "takes away a star" from the type; `T*`:c: becomes `T`.) Two stars can be used to get to the element type of the element type etc. For example: -.. code-block:: nim - + ```nim type Input {.importcpp: "System::Input".} = object proc getSubsystem*[T](): ptr T {.importcpp: "SystemManager::getSubsystem<'*0>()", nodecl.} let x: ptr Input = getSubsystem[Input]() + ``` Produces: -.. code-block:: C + ```C x = SystemManager::getSubsystem<System::Input>() + ``` -- `#@` is a special case to support a `cnew` operation. It is required so +- ``#@`` is a special case to support a `cnew` operation. It is required so that the call expression is inlined directly, without going through a temporary location. This is only required to circumvent a limitation of the current code generator. -For example C++'s `new` operator can be "imported" like this: +For example C++'s `new`:cpp: operator can be "imported" like this: -.. code-block:: nim + ```nim proc cnew*[T](x: T): ptr T {.importcpp: "(new '*0#@)", nodecl.} # constructor of 'Foo': proc constructFoo(a, b: cint): Foo {.importcpp: "Foo(@)".} let x = cnew constructFoo(3, 4) + ``` Produces: -.. code-block:: C + ```C x = new Foo(3, 4) + ``` -However, depending on the use case `new Foo` can also be wrapped like this +However, depending on the use case `new Foo`:cpp: can also be wrapped like this instead: -.. code-block:: nim + ```nim proc newFoo(a, b: cint): ptr Foo {.importcpp: "new Foo(@)".} let x = newFoo(3, 4) + ``` -Wrapping constructors -~~~~~~~~~~~~~~~~~~~~~ +### Wrapping constructors Sometimes a C++ class has a private copy constructor and so code like -`Class c = Class(1,2);` must not be generated but instead `Class c(1,2);`. +`Class c = Class(1,2);`:cpp: must not be generated but instead +`Class c(1,2);`:cpp:. For this purpose the Nim proc that wraps a C++ constructor needs to be annotated with the `constructor`:idx: pragma. This pragma also helps to generate faster C++ code since construction then doesn't invoke the copy constructor: -.. code-block:: nim + ```nim # a better constructor of 'Foo': proc constructFoo(a, b: cint): Foo {.importcpp: "Foo(@)", constructor.} + ``` -Wrapping destructors -~~~~~~~~~~~~~~~~~~~~ +### Wrapping destructors Since Nim generates C++ directly, any destructor is called implicitly by the C++ compiler at the scope exits. This means that often one can get away with @@ -7162,20 +8179,18 @@ not wrapping the destructor at all! However, when it needs to be invoked explicitly, it needs to be wrapped. The pattern language provides everything that is required: -.. code-block:: nim + ```nim proc destroyFoo(this: var Foo) {.importcpp: "#.~Foo()".} + ``` -Importcpp for objects -~~~~~~~~~~~~~~~~~~~~~ +### Importcpp for objects Generic `importcpp`'ed objects are mapped to C++ templates. This means that one can import C++'s templates rather easily without the need for a pattern language for object types: -.. code-block:: nim - :test: "nim cpp $1" - + ```nim test = "nim cpp $1" type StdMap[K, V] {.importcpp: "std::map", header: "<map>".} = object proc `[]=`[K, V](this: var StdMap[K, V]; key: K; val: V) {. @@ -7183,56 +8198,57 @@ language for object types: var x: StdMap[cint, cdouble] x[6] = 91.4 + ``` Produces: -.. code-block:: C + ```C std::map<int, double> x; x[6] = 91.4; + ``` - If more precise control is needed, the apostrophe `'` can be used in the supplied pattern to denote the concrete type parameters of the generic type. See the usage of the apostrophe operator in proc patterns for more details. -.. code-block:: nim - - type - VectorIterator {.importcpp: "std::vector<'0>::iterator".} [T] = object + ```nim + type + VectorIterator[T] {.importcpp: "std::vector<'0>::iterator".} = object - var x: VectorIterator[cint] + var x: VectorIterator[cint] + ``` + Produces: -Produces: + ```C -.. code-block:: C - - std::vector<int>::iterator x; + std::vector<int>::iterator x; + ``` ImportJs pragma --------------- -Similar to the `importcpp pragma for C++ <#implementation-specific-pragmas-importcpp-pragma>`_, +Similar to the [importcpp pragma] for C++, the `importjs` pragma can be used to import Javascript methods or symbols in general. The generated code then uses the Javascript method -calling syntax: `obj.method(arg)`. +calling syntax: ``obj.method(arg)``. ImportObjC pragma ----------------- -Similar to the `importc pragma for C -<#foreign-function-interface-importc-pragma>`_, the `importobjc` pragma can -be used to import `Objective C`:idx: methods. The generated code then uses the -Objective C method calling syntax: `[obj method param1: arg]`. +Similar to the [importc pragma] for C, the `importobjc` pragma can +be used to import `Objective C`:idx: methods. The generated code then uses the +Objective C method calling syntax: ``[obj method param1: arg]``. In addition with the `header` and `emit` pragmas this allows *sloppy* interfacing with libraries written in Objective C: -.. code-block:: Nim + ```Nim # horrible example of how to interface with GNUStep ... - {.passL: "-lobjc".} + {.passl: "-lobjc".} {.emit: """ #include <objc/Object.h> @interface Greeter:Object @@ -7264,9 +8280,10 @@ allows *sloppy* interfacing with libraries written in Objective C: var g = newGreeter() g.greet(12, 34) g.free() + ``` -The compiler needs to be told to generate Objective C (command `objc`) for -this to work. The conditional symbol `objc` is defined when the compiler +The compiler needs to be told to generate Objective C (command `objc`:option:) for +this to work. The conditional symbol ``objc`` is defined when the compiler emits Objective C code. @@ -7274,68 +8291,87 @@ CodegenDecl pragma ------------------ The `codegenDecl` pragma can be used to directly influence Nim's code -generator. It receives a format string that determines how the variable -or proc is declared in the generated code. +generator. It receives a format string that determines how the variable, +proc or object type is declared in the generated code. -For variables, $1 in the format string represents the type of the variable -and $2 is the name of the variable. +For variables, $1 in the format string represents the type of the variable, +$2 is the name of the variable, and each appearance of $# represents $1/$2 +respectively according to its position. The following Nim code: -.. code-block:: nim + ```nim var a {.codegenDecl: "$# progmem $#".}: int + ``` will generate this C code: -.. code-block:: c + ```c int progmem a + ``` For procedures, $1 is the return type of the procedure, $2 is the name of -the procedure, and $3 is the parameter list. +the procedure, $3 is the parameter list, and each appearance of $# represents +$1/$2/$3 respectively according to its position. The following nim code: -.. code-block:: nim + ```nim proc myinterrupt() {.codegenDecl: "__interrupt $# $#$#".} = echo "realistic interrupt handler" + ``` will generate this code: -.. code-block:: c + ```c __interrupt void myinterrupt() + ``` + +For object types, the $1 represents the name of the object type, $2 is the list of +fields and $3 is the base type. + +```nim + +const strTemplate = """ + struct $1 { + $2 + }; +""" +type Foo {.codegenDecl:strTemplate.} = object + a, b: int +``` + +will generate this code: +```c +struct Foo { + NI a; + NI b; +}; +``` + `cppNonPod` pragma ------------------ -The `.cppNonPod` pragma should be used for non-POD `importcpp` types so that they +The `cppNonPod` pragma should be used for non-POD `importcpp` types so that they work properly (in particular regarding constructor and destructor) for -`.threadvar` variables. This requires `--tlsEmulation:off`. +`threadvar` variables. This requires `--tlsEmulation:off`:option:. -.. code-block:: nim + ```nim type Foo {.cppNonPod, importcpp, header: "funs.h".} = object x: cint proc main()= var a {.threadvar.}: Foo + ``` -InjectStmt pragma ------------------ - -The `injectStmt` pragma can be used to inject a statement before every -other statement in the current module. It is only supposed to be used for -debugging: - -.. code-block:: nim - {.injectStmt: gcInvariants().} - - # ... complex code here that produces crashes ... compile-time define pragmas --------------------------- The pragmas listed here can be used to optionally accept values from -the -d/--define option at compile time. +the `-d/--define`:option: option at compile time. The implementation currently provides the following possible options (various others may be added later). @@ -7348,19 +8384,40 @@ pragma description `booldefine`:idx: Reads in a build-time define as a bool ================= ============================================ -.. code-block:: nim - const FooBar {.intdefine.}: int = 5 - echo FooBar + ```nim + const FooBar {.intdefine.}: int = 5 + echo FooBar + ``` -:: - nim c -d:FooBar=42 foobar.nim + ```cmd + nim c -d:FooBar=42 foobar.nim + ``` -In the above example, providing the -d flag causes the symbol +In the above example, providing the `-d`:option: flag causes the symbol `FooBar` to be overwritten at compile-time, printing out 42. If the -`-d:FooBar=42` were to be omitted, the default value of 5 would be +`-d:FooBar=42`:option: were to be omitted, the default value of 5 would be used. To see if a value was provided, `defined(FooBar)` can be used. -The syntax `-d:flag` is actually just a shortcut for `-d:flag=true`. +The syntax `-d:flag`:option: is actually just a shortcut for +`-d:flag=true`:option:. + +These pragmas also accept an optional string argument for qualified +define names. + + ```nim + const FooBar {.intdefine: "package.FooBar".}: int = 5 + echo FooBar + ``` + + ```cmd + nim c -d:package.FooBar=42 foobar.nim + ``` + +This helps disambiguate define names in different packages. + +See also the [generic `define` pragma](manual_experimental.html#generic-nimdefine-pragma) +for a version of these pragmas that detects the type of the define based on +the constant value. User-defined pragmas ==================== @@ -7370,19 +8427,21 @@ pragma pragma ------------- The `pragma` pragma can be used to declare user-defined pragmas. This is -useful because Nim's templates and macros do not affect pragmas. User-defined pragmas are in a different module-wide scope than all other symbols. +useful because Nim's templates and macros do not affect pragmas. +User-defined pragmas are in a different module-wide scope than all other symbols. They cannot be imported from a module. Example: -.. code-block:: nim + ```nim when appType == "lib": {.pragma: rtl, exportc, dynlib, cdecl.} else: {.pragma: rtl, importc, dynlib: "client.dll", cdecl.} proc p*(a, b: int): int {.rtl.} = - result = a+b + result = a + b + ``` In the example, a new pragma named `rtl` is introduced that either imports a symbol from a dynamic library or exports the symbol for dynamic library @@ -7395,17 +8454,18 @@ It is possible to define custom typed pragmas. Custom pragmas do not affect code generation directly, but their presence can be detected by macros. Custom pragmas are defined using templates annotated with pragma `pragma`: -.. code-block:: nim + ```nim template dbTable(name: string, table_space: string = "") {.pragma.} template dbKey(name: string = "", primary_key: bool = false) {.pragma.} template dbForeignKey(t: typedesc) {.pragma.} template dbIgnore {.pragma.} + ``` Consider this stylized example of a possible Object Relation Mapping (ORM) implementation: -.. code-block:: nim + ```nim const tblspace {.strdefine.} = "dev" # switch for dev, test and prod environments type @@ -7420,7 +8480,8 @@ implementation: user_id {.dbForeignKey: User.}: int read_access: bool write_access: bool - admin_acess: bool + admin_access: bool + ``` In this example, custom pragmas are used to describe how Nim objects are mapped to the schema of the relational database. Custom pragmas can have @@ -7435,7 +8496,7 @@ definitions, statements, etc. The macros module includes helpers which can be used to simplify custom pragma access `hasCustomPragma`, `getCustomPragmaVal`. Please consult the -`macros <macros.html>`_ module documentation for details. These macros are not +[macros](macros.html) module documentation for details. These macros are not magic, everything they do can also be achieved by walking the AST of the object representation. @@ -7443,65 +8504,68 @@ More examples with custom pragmas: - Better serialization/deserialization control: -.. code-block:: nim - type MyObj = object - a {.dontSerialize.}: int - b {.defaultDeserialize: 5.}: int - c {.serializationKey: "_c".}: string + ```nim + type MyObj = object + a {.dontSerialize.}: int + b {.defaultDeserialize: 5.}: int + c {.serializationKey: "_c".}: string + ``` - Adopting type for gui inspector in a game engine: -.. code-block:: nim - type MyComponent = object - position {.editable, animatable.}: Vector3 - alpha {.editRange: [0.0..1.0], animatable.}: float32 + ```nim + type MyComponent = object + position {.editable, animatable.}: Vector3 + alpha {.editRange: [0.0..1.0], animatable.}: float32 + ``` Macro pragmas ------------- -All macros and templates can also be used as pragmas. They can be attached -to routines (procs, iterators, etc), type names, or type expressions. The -compiler will perform the following simple syntactic transformations: +Macros and templates can sometimes be called with the pragma syntax. Cases +where this is possible include when attached to routine (procs, iterators, etc.) +declarations or routine type expressions. The compiler will perform the +following simple syntactic transformations: -.. code-block:: nim + ```nim template command(name: string, def: untyped) = discard proc p() {.command("print").} = discard + ``` This is translated to: -.. code-block:: nim + ```nim command("print"): proc p() = discard + ``` ------ -.. code-block:: nim + ```nim type AsyncEventHandler = proc (x: Event) {.async.} + ``` This is translated to: -.. code-block:: nim + ```nim type AsyncEventHandler = async(proc (x: Event)) + ``` ------ -.. code-block:: nim - type - MyObject {.schema: "schema.protobuf".} = object - -This is translated to a call to the `schema` macro with a `nnkTypeDef` -AST node capturing both the left-hand side and right-hand side of the -definition. The macro can return a potentially modified `nnkTypeDef` tree -which will replace the original row in the type section. - -When multiple macro pragmas are applied to the same definition, the -compiler will apply them consequently from left to right. Each macro -will receive as input the output of the previous one. +When multiple macro pragmas are applied to the same definition, the first one +from left to right will be evaluated. This macro can then choose to keep +the remaining macro pragmas in its output, and those will be evaluated in +the same way. +There are a few more applications of macro pragmas, such as in type, +variable and constant declarations, but this behavior is considered to be +experimental and is documented in the [experimental manual]( +manual_experimental.html#extended-macro-pragmas) instead. Foreign function interface @@ -7519,60 +8583,67 @@ from C. The optional argument is a string containing the C identifier. If the argument is missing, the C name is the Nim identifier *exactly as spelled*: -.. code-block:: + ```nim proc printf(formatstr: cstring) {.header: "<stdio.h>", importc: "printf", varargs.} + ``` When `importc` is applied to a `let` statement it can omit its value which -will then be expected to come from C. This can be used to import a C `const`: +will then be expected to come from C. This can be used to import a C `const`:c:\: -.. code-block:: + ```nim {.emit: "const int cconst = 42;".} let cconst {.importc, nodecl.}: cint assert cconst == 42 + ``` Note that this pragma has been abused in the past to also work in the JS backend for JS objects and functions. Other backends do provide the same feature under the same name. Also, when the target language is not set to C, other pragmas are available: - * `importcpp <manual.html#implementation-specific-pragmas-importcpp-pragma>`_ - * `importobjc <manual.html#implementation-specific-pragmas-importobjc-pragma>`_ - * `importjs <manual.html#implementation-specific-pragmas-importjs-pragma>`_ + * [importcpp][importcpp pragma] + * [importobjc][importobjc pragma] + * [importjs][importjs pragma] -.. code-block:: Nim +The string literal passed to `importc` can be a format string: + + ```Nim proc p(s: cstring) {.importc: "prefix$1".} + ``` -In the example, the external name of `p` is set to `prefixp`. Only `$1` -is available and a literal dollar sign must be written as `$$`. +In the example, the external name of `p` is set to `prefixp`. Only ``$1`` +is available and a literal dollar sign must be written as ``$$``. Exportc pragma -------------- The `exportc` pragma provides a means to export a type, a variable, or a procedure to C. Enums and constants can't be exported. The optional argument -is a string containing the C identifier. If the argument is missing, the C +is a string containing the C identifier. If the argument is missing, the C name is the Nim identifier *exactly as spelled*: -.. code-block:: Nim + ```Nim proc callme(formatstr: cstring) {.exportc: "callMe", varargs.} + ``` Note that this pragma is somewhat of a misnomer: Other backends do provide the same feature under the same name. The string literal passed to `exportc` can be a format string: -.. code-block:: Nim + ```Nim proc p(s: string) {.exportc: "prefix$1".} = echo s + ``` -In the example, the external name of `p` is set to `prefixp`. Only `$1` -is available and a literal dollar sign must be written as `$$`. +In the example, the external name of `p` is set to `prefixp`. Only ``$1`` +is available and a literal dollar sign must be written as ``$$``. If the symbol should also be exported to a dynamic library, the `dynlib` pragma should be used in addition to the `exportc` pragma. See -`Dynlib pragma for export <#foreign-function-interface-dynlib-pragma-for-export>`_. +[Dynlib pragma for export]. Extern pragma @@ -7580,32 +8651,93 @@ Extern pragma Like `exportc` or `importc`, the `extern` pragma affects name mangling. The string literal passed to `extern` can be a format string: -.. code-block:: Nim + ```Nim proc p(s: string) {.extern: "prefix$1".} = echo s + ``` -In the example, the external name of `p` is set to `prefixp`. Only `$1` -is available and a literal dollar sign must be written as `$$`. +In the example, the external name of `p` is set to `prefixp`. Only ``$1`` +is available and a literal dollar sign must be written as ``$$``. Bycopy pragma ------------- -The `bycopy` pragma can be applied to an object or tuple type and -instructs the compiler to pass the type by value to procs: +The `bycopy` pragma can be applied to an object or tuple type or a proc param. It instructs the compiler to pass the type by value to procs: -.. code-block:: nim + ```nim type Vector {.bycopy.} = object x, y, z: float + ``` +The Nim compiler automatically determines whether a parameter is passed by value or +by reference based on the parameter type's size. If a parameter must be passed by value +or by reference, (such as when interfacing with a C library) use the bycopy or byref pragmas. +Notice params marked as `byref` takes precedence over types marked as `bycopy`. Byref pragma ------------ -The `byref` pragma can be applied to an object or tuple type and instructs -the compiler to pass the type by reference (hidden pointer) to procs. +The `byref` pragma can be applied to an object or tuple type or a proc param. +When applied to a type it instructs the compiler to pass the type by reference +(hidden pointer) to procs. When applied to a param it will take precedence, even +if the the type was marked as `bycopy`. When an `importc` type has a `byref` pragma or +parameters are marked as `byref` in an `importc` proc, these params translate to pointers. +When an `importcpp` type has a `byref` pragma, these params translate to +C++ references `&`. + + ```Nim + {.emit: """/*TYPESECTION*/ + typedef struct { + int x; + } CStruct; + """.} + + {.emit: """ + #ifdef __cplusplus + extern "C" + #endif + int takesCStruct(CStruct* x) { + return x->x; + } + """.} + + type + CStruct {.importc, byref.} = object + x: cint + + proc takesCStruct(x: CStruct): cint {.importc.} + ``` + + or + + + ```Nim + type + CStruct {.importc.} = object + x: cint + + proc takesCStruct(x {.byref.}: CStruct): cint {.importc.} + ``` + ```Nim + {.emit: """/*TYPESECTION*/ + struct CppStruct { + int x; + + int takesCppStruct(CppStruct& y) { + return x + y.x; + } + }; + """.} + + type + CppStruct {.importcpp, byref.} = object + x: cint + + proc takesCppStruct(x, y: CppStruct): cint {.importcpp.} + ``` Varargs pragma -------------- @@ -7614,17 +8746,18 @@ types). It tells Nim that the proc can take a variable number of parameters after the last specified parameter. Nim string values will be converted to C strings automatically: -.. code-block:: Nim - proc printf(formatstr: cstring) {.nodecl, varargs.} + ```Nim + proc printf(formatstr: cstring) {.header: "<stdio.h>", varargs.} printf("hallo %s", "world") # "world" will be passed as C string + ``` Union pragma ------------ The `union` pragma can be applied to any `object` type. It means all -of the object's fields are overlaid in memory. This produces a `union` -instead of a `struct` in the generated C/C++ code. The object declaration +of an object's fields are overlaid in memory. This produces a `union`:c: +instead of a `struct`:c: in the generated C/C++ code. The object declaration then must not use inheritance or any GC'ed memory but this is currently not checked. @@ -7647,12 +8780,13 @@ a static error. Usage with inheritance should be defined and documented. Dynlib pragma for import ------------------------ With the `dynlib` pragma, a procedure or a variable can be imported from -a dynamic library (`.dll` files for Windows, `lib*.so` files for UNIX). +a dynamic library (``.dll`` files for Windows, ``lib*.so`` files for UNIX). The non-optional argument has to be the name of the dynamic library: -.. code-block:: Nim + ```Nim proc gtk_image_new(): PGtkWidget {.cdecl, dynlib: "libgtk-x11-2.0.so", importc.} + ``` In general, importing a dynamic library does not require any special linker options or linking with import libraries. This also implies that no *devel* @@ -7660,25 +8794,26 @@ packages need to be installed. The `dynlib` import mechanism supports a versioning scheme: -.. code-block:: nim + ```nim proc Tcl_Eval(interp: pTcl_Interp, script: cstring): int {.cdecl, importc, dynlib: "libtcl(|8.5|8.4|8.3).so.(1|0)".} + ``` -At runtime, the dynamic library is searched for (in this order):: +At runtime, the dynamic library is searched for (in this order): - libtcl.so.1 - libtcl.so.0 - libtcl8.5.so.1 - libtcl8.5.so.0 - libtcl8.4.so.1 - libtcl8.4.so.0 - libtcl8.3.so.1 - libtcl8.3.so.0 + libtcl.so.1 + libtcl.so.0 + libtcl8.5.so.1 + libtcl8.5.so.0 + libtcl8.4.so.1 + libtcl8.4.so.0 + libtcl8.3.so.1 + libtcl8.3.so.0 The `dynlib` pragma supports not only constant strings as an argument but also string expressions in general: -.. code-block:: nim + ```nim import std/os proc getDllName: string = @@ -7689,16 +8824,17 @@ string expressions in general: quit("could not load dynamic library") proc myImport(s: cstring) {.cdecl, importc, dynlib: getDllName().} + ``` -**Note**: Patterns like `libtcl(|8.5|8.4).so` are only supported in constant +**Note**: Patterns like ``libtcl(|8.5|8.4).so`` are only supported in constant strings, because they are precompiled. **Note**: Passing variables to the `dynlib` pragma will fail at runtime because of order of initialization problems. **Note**: A `dynlib` import can be overridden with -the `--dynlibOverride:name` command-line option. The -`Compiler User Guide <nimc.html>`_ contains further information. +the `--dynlibOverride:name`:option: command-line option. The +[Compiler User Guide](nimc.html) contains further information. Dynlib pragma for export @@ -7708,29 +8844,21 @@ With the `dynlib` pragma, a procedure can also be exported to a dynamic library. The pragma then has no argument and has to be used in conjunction with the `exportc` pragma: -.. code-block:: Nim + ```Nim proc exportme(): int {.cdecl, exportc, dynlib.} + ``` This is only useful if the program is compiled as a dynamic library via the -`--app:lib` command-line option. - +`--app:lib`:option: command-line option. Threads ======= -To enable thread support the `--threads:on` command-line switch needs to -be used. The `system` module then contains several threading primitives. -See the `threads <threads.html>`_ and `channels <channels.html>`_ modules -for the low-level thread API. There are also high-level parallelism constructs -available. See `spawn <manual_experimental.html#parallel-amp-spawn>`_ for +The `--threads:on`:option: command-line switch is enabled by default. The [typedthreads module](typedthreads.html) module then contains several threading primitives. See [spawn](manual_experimental.html#parallel-amp-spawn) for further details. -Nim's memory model for threads is quite different than that of other common -programming languages (C, Pascal, Java): Each thread has its own (garbage -collected) heap, and sharing of memory is restricted to global variables. This -helps to prevent race conditions. GC efficiency is improved quite a lot, -because the GC never has to stop other threads and see what they reference. +The only ways to create a thread is via `spawn` or `createThread`. Thread pragma @@ -7742,46 +8870,8 @@ violations of the `no heap sharing restriction`:idx:\: This restriction implies that it is invalid to construct a data structure that consists of memory allocated from different (thread-local) heaps. -A thread proc is passed to `createThread` or `spawn` and invoked -indirectly; so the `thread` pragma implies `procvar`. - - -GC safety ---------- - -We call a proc `p` `GC safe`:idx: when it doesn't access any global variable -that contains GC'ed memory (`string`, `seq`, `ref` or a closure) either -directly or indirectly through a call to a GC unsafe proc. - -The `gcsafe`:idx: annotation can be used to mark a proc to be gcsafe, -otherwise this property is inferred by the compiler. Note that `noSideEffect` -implies `gcsafe`. The only way to create a thread is via `spawn` or -`createThread`. The invoked proc must not use `var` parameters nor must -any of its parameters contain a `ref` or `closure` type. This enforces -the *no heap sharing restriction*. - -Routines that are imported from C are always assumed to be `gcsafe`. -To disable the GC-safety checking the `--threadAnalysis:off` command-line -switch can be used. This is a temporary workaround to ease the porting effort -from old code to the new threading model. - -To override the compiler's gcsafety analysis a `{.cast(gcsafe).}` pragma block can -be used: - -.. code-block:: nim - - var - someGlobal: string = "some string here" - perThread {.threadvar.}: string - - proc setPerThread() = - {.cast(gcsafe).}: - deepCopy(perThread, someGlobal) - - -See also: +A thread proc can be passed to `createThread` or `spawn`. -- `Shared heap memory management <gc.html>`_. Threadvar pragma @@ -7791,8 +8881,9 @@ A variable can be marked with the `threadvar` pragma, which makes it a `thread-local`:idx: variable; Additionally, this implies all the effects of the `global` pragma. -.. code-block:: nim + ```nim var checkpoints* {.threadvar.}: seq[string] + ``` Due to implementation restrictions, thread-local variables cannot be initialized within the `var` section. (Every thread-local variable needs to @@ -7804,4 +8895,139 @@ Threads and exceptions The interaction between threads and exceptions is simple: A *handled* exception in one thread cannot affect any other thread. However, an *unhandled* exception -in one thread terminates the whole *process*! +in one thread terminates the whole *process*. + + +Guards and locks +================ + +Nim provides common low level concurrency mechanisms like locks, atomic +intrinsics or condition variables. + +Nim significantly improves on the safety of these features via additional +pragmas: + +1) A `guard`:idx: annotation is introduced to prevent data races. +2) Every access of a guarded memory location needs to happen in an + appropriate `locks`:idx: statement. + + +Guards and locks sections +------------------------- + +### Protecting global variables + +Object fields and global variables can be annotated via a `guard` pragma: + + ```nim + import std/locks + + var glock: Lock + var gdata {.guard: glock.}: int + ``` + +The compiler then ensures that every access of `gdata` is within a `locks` +section: + + ```nim + proc invalid = + # invalid: unguarded access: + echo gdata + + proc valid = + # valid access: + {.locks: [glock].}: + echo gdata + ``` + +Top level accesses to `gdata` are always allowed so that it can be initialized +conveniently. It is *assumed* (but not enforced) that every top level statement +is executed before any concurrent action happens. + +The `locks` section deliberately looks ugly because it has no runtime +semantics and should not be used directly! It should only be used in templates +that also implement some form of locking at runtime: + + ```nim + template lock(a: Lock; body: untyped) = + pthread_mutex_lock(a) + {.locks: [a].}: + try: + body + finally: + pthread_mutex_unlock(a) + ``` + + +The guard does not need to be of any particular type. It is flexible enough to +model low level lockfree mechanisms: + + ```nim + var dummyLock {.compileTime.}: int + var atomicCounter {.guard: dummyLock.}: int + + template atomicRead(x): untyped = + {.locks: [dummyLock].}: + memoryReadBarrier() + x + + echo atomicRead(atomicCounter) + ``` + + +The `locks` pragma takes a list of lock expressions `locks: [a, b, ...]` +in order to support *multi lock* statements. + + +### Protecting general locations + +The `guard` annotation can also be used to protect fields within an object. +The guard then needs to be another field within the same object or a +global variable. + +Since objects can reside on the heap or on the stack, this greatly enhances +the expressiveness of the language: + + ```nim + import std/locks + + type + ProtectedCounter = object + v {.guard: L.}: int + L: Lock + + proc incCounters(counters: var openArray[ProtectedCounter]) = + for i in 0..counters.high: + lock counters[i].L: + inc counters[i].v + ``` + +The access to field `x.v` is allowed since its guard `x.L` is active. +After template expansion, this amounts to: + + ```nim + proc incCounters(counters: var openArray[ProtectedCounter]) = + for i in 0..counters.high: + pthread_mutex_lock(counters[i].L) + {.locks: [counters[i].L].}: + try: + inc counters[i].v + finally: + pthread_mutex_unlock(counters[i].L) + ``` + +There is an analysis that checks that `counters[i].L` is the lock that +corresponds to the protected location `counters[i].v`. This analysis is called +`path analysis`:idx: because it deals with paths to locations +like `obj.field[i].fieldB[j]`. + +The path analysis is **currently unsound**, but that doesn't make it useless. +Two paths are considered equivalent if they are syntactically the same. + +This means the following compiles (for now) even though it really should not: + + ```nim + {.locks: [a[i].L].}: + inc i + access a[i].v + ``` diff --git a/doc/manual/var_t_return.rst b/doc/manual/var_t_return.md index e34993e3e..15d908c74 100644 --- a/doc/manual/var_t_return.rst +++ b/doc/manual/var_t_return.md @@ -1,11 +1,12 @@ .. default-role:: code +.. include:: ../rstcommon.rst Memory safety for returning by `var T` is ensured by a simple borrowing rule: If `result` does not refer to a location pointing to the heap (that is in `result = X` the `X` involves a `ptr` or `ref` access) then it has to be derived from the routine's first parameter: -.. code-block:: nim + ```nim proc forward[T](x: var T): var T = result = x # ok, derived from the first parameter. @@ -16,6 +17,7 @@ then it has to be derived from the routine's first parameter: result = forward(x) # Error: location is derived from `x` # which is not p's first parameter and lives # on the stack. + ``` In other words, the lifetime of what `result` points to is attached to the lifetime of the first parameter and that is enough knowledge to verify diff --git a/doc/manual_experimental.rst b/doc/manual_experimental.md index cf2e0c247..da51d59ad 100644 --- a/doc/manual_experimental.rst +++ b/doc/manual_experimental.md @@ -1,5 +1,3 @@ -.. default-role:: code - ========================= Nim Experimental Features ========================= @@ -7,6 +5,8 @@ Nim Experimental Features :Authors: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: @@ -15,69 +15,30 @@ About this document This document describes features of Nim that are to be considered experimental. Some of these are not covered by the `.experimental` pragma or -`--experimental` switch because they are already behind a special syntax and +`--experimental`:option: switch because they are already behind a special syntax and one may want to use Nim libraries using these features without using them oneself. -**Note**: Unless otherwise indicated, these features are not to be removed, -but refined and overhauled. - - -Package level objects -===================== - -Every Nim module resides in a (nimble) package. An object type can be attached -to the package it resides in. If that is done, the type can be referenced from -other modules as an `incomplete`:idx: object type. This feature allows to -break up recursive type dependencies across module boundaries. Incomplete -object types are always passed `byref` and can only be used in pointer like -contexts (`var/ref/ptr IncompleteObject`) in general since the compiler does -not yet know the size of the object. To complete an incomplete object -the `package` pragma has to be used. `package` implies `byref`. - -As long as a type `T` is incomplete, neither `sizeof(T)` nor runtime -type information for `T` is available. - - -Example: - -.. code-block:: nim - - # module A (in an arbitrary package) - type - Pack.SomeObject = object ## declare as incomplete object of package 'Pack' - Triple = object - a, b, c: ref SomeObject ## pointers to incomplete objects are allowed - - ## Incomplete objects can be used as parameters: - proc myproc(x: SomeObject) = discard - - -.. code-block:: nim - - # module B (in package "Pack") - type - SomeObject* {.package.} = object ## Use 'package' to complete the object - s, t: string - x, y: int +.. note:: Unless otherwise indicated, these features are not to be removed, + but refined and overhauled. Void type ========= -The `void` type denotes the absence of any type. Parameters of -type `void` are treated as non-existent, `void` as a return type means that -the procedure does not return a value: +The `void` type denotes the absence of any value, i.e. it is the type that contains no values. Consequently, no value can be provided for parameters of +type `void`, and no value can be returned from a function with return type `void`: -.. code-block:: nim + ```nim proc nothing(x, y: void): void = echo "ha" nothing() # writes "ha" to stdout + ``` The `void` type is particularly useful for generic code: -.. code-block:: nim + ```nim proc callProc[T](p: proc (x: T), x: T) = when T is void: p() @@ -89,135 +50,235 @@ The `void` type is particularly useful for generic code: callProc[int](intProc, 12) callProc[void](emptyProc) + ``` However, a `void` type cannot be inferred in generic code: -.. code-block:: nim + ```nim callProc(emptyProc) # Error: type mismatch: got (proc ()) # but expected one of: # callProc(p: proc (T), x: T) + ``` The `void` type is only valid for parameters and return types; other symbols cannot have the type `void`. +Generic `define` pragma +======================= +Aside the [typed define pragmas for constants](manual.html#implementation-specific-pragmas-compileminustime-define-pragmas), +there is a generic `{.define.}` pragma that interprets the value of the define +based on the type of the constant value. -Covariance -========== + ```nim + const foo {.define: "package.foo".} = 123 + const bar {.define: "package.bar".} = false + ``` -Covariance in Nim can be introduced only through pointer-like types such -as `ptr` and `ref`. Sequence, Array and OpenArray types, instantiated -with pointer-like types will be considered covariant if and only if they -are also immutable. The introduction of a `var` modifier or additional -`ptr` or `ref` indirections would result in invariant treatment of -these types. + ```cmd + nim c -d:package.foo=456 -d:package.bar foobar.nim + ``` -`proc` types are currently always invariant, but future versions of Nim -may relax this rule. +The following types are supported: -User-defined generic types may also be covariant with respect to some of -their parameters. By default, all generic params are considered invariant, -but you may choose the apply the prefix modifier `in` to a parameter to -make it contravariant or `out` to make it covariant: +* `string` and `cstring` +* Signed and unsigned integer types +* `bool` +* Enums -.. code-block:: nim - type - AnnotatedPtr[out T] = - metadata: MyTypeInfo - p: ref T +Top-down type inference +======================= - RingBuffer[out T] = - startPos: int - data: seq[T] +In expressions such as: - Action {.importcpp: "std::function<void ('0)>".} [in T] = object +```nim +let a: T = ex +``` -When the designated generic parameter is used to instantiate a pointer-like -type as in the case of `AnnotatedPtr` above, the resulting generic type will -also have pointer-like covariance: +Normally, the compiler type checks the expression `ex` by itself, then +attempts to statically convert the type-checked expression to the given type +`T` as much as it can, while making sure it matches the type. The extent of +this process is limited however due to the expression usually having +an assumed type that might clash with the given type. -.. code-block:: nim - type - GuiWidget = object of RootObj - Button = object of GuiWidget - ComboBox = object of GuiWidget +With top-down type inference, the expression is type checked with the +extra knowledge that it is supposed to be of type `T`. For example, +the following code is does not compile with the former method, but +compiles with top-down type inference: - var - widgetPtr: AnnotatedPtr[GuiWidget] - buttonPtr: AnnotatedPtr[Button] +```nim +let foo: (float, uint8, cstring) = (1, 2, "abc") +``` - ... +The tuple expression has an expected type of `(float, uint8, cstring)`. +Since it is a tuple literal, we can use this information to assume the types +of its elements. The expected types for the expressions `1`, `2` and `"abc"` +are respectively `float`, `uint8`, and `cstring`; and these expressions can be +statically converted to these types. - proc drawWidget[T](x: AnnotatedPtr[GuiWidget]) = ... +Without this information, the type of the tuple expression would have been +assumed to be `(int, int, string)`. Thus the type of the tuple expression +would not match the type of the variable, and an error would be given. - # you can call procs expecting base types by supplying a derived type - drawWidget(buttonPtr) +The extent of this varies, but there are some notable special cases. - # and you can convert more-specific pointer types to more general ones - widgetPtr = buttonPtr -Just like with regular pointers, covariance will be enabled only for immutable -values: +Inferred generic parameters +--------------------------- -.. code-block:: nim - proc makeComboBox[T](x: var AnnotatedPtr[GuiWidget]) = - x.p = new(ComboBox) +In expressions making use of generic procs or templates, the expected +(unbound) types are often able to be inferred based on context. +This feature has to be enabled via `{.experimental: "inferGenericTypes".}` - makeComboBox(buttonPtr) # Error, AnnotatedPtr[Button] cannot be modified - # to point to a ComboBox + ```nim test = "nim c $1" + {.experimental: "inferGenericTypes".} -On the other hand, in the `RingBuffer` example above, the designated generic -param is used to instantiate the non-pointer `seq` type, which means that -the resulting generic type will have covariance that mimics an array or -sequence (i.e. it will be covariant only when instantiated with `ptr` and -`ref` types): + import std/options -.. code-block:: nim + var x = newSeq[int](1) + # Do some work on 'x'... - type - Base = object of RootObj - Derived = object of Base + # Works! + # 'x' is 'seq[int]' so 'newSeq[int]' is implied + x = newSeq(10) - proc consumeBaseValues(b: RingBuffer[Base]) = ... + # Works! + # 'T' of 'none' is bound to the 'T' of 'noneProducer', passing it along. + # Effectively 'none.T = noneProducer.T' + proc noneProducer[T](): Option[T] = none() + let myNone = noneProducer[int]() - var derivedValues: RingBuffer[Derived] + # Also works + # 'myOtherNone' binds its 'T' to 'float' and 'noneProducer' inherits it + # noneProducer.T = myOtherNone.T + let myOtherNone: Option[float] = noneProducer() - consumeBaseValues(derivedValues) # Error, Base and Derived values may differ - # in size + # Works as well + # none.T = myOtherOtherNone.T + let myOtherOtherNone: Option[int] = none() + ``` - proc consumeBasePointers(b: RingBuffer[ptr Base]) = ... +This is achieved by reducing the types on the lhs and rhs until the *lhs* is left with only types such as `T`. +While lhs and rhs are reduced together, this does *not* mean that the *rhs* will also only be left +with a flat type `Z`, it may be of the form `MyType[Z]`. - var derivedPointers: RingBuffer[ptr Derived] +After the types have been reduced, the types `T` are bound to the types that are left on the rhs. - consumeBaseValues(derivedPointers) # This is legal +If bindings *cannot be inferred*, compilation will fail and manual specification is required. -Please note that Nim will treat the user-defined pointer-like types as -proper alternatives to the built-in pointer types. That is, types such -as `seq[AnnotatedPtr[T]]` or `RingBuffer[AnnotatedPtr[T]]` will also be -considered covariant and you can create new pointer-like types by instantiating -other user-defined pointer-like types. +An example for *failing inference* can be found when passing a generic expression +to a function/template call: -The contravariant parameters introduced with the `in` modifier are currently -useful only when interfacing with imported types having such semantics. + ```nim test = "nim c $1" status = 1 + {.experimental: "inferGenericTypes".} + proc myProc[T](a, b: T) = discard -Automatic dereferencing -======================= + # Fails! Unable to infer that 'T' is supposed to be 'int' + myProc(newSeq[int](), newSeq(1)) -Automatic dereferencing is performed for the first argument of a routine call. -This feature has to be enabled via `{.experimental: "implicitDeref".}`: + # Works! Manual specification of 'T' as 'int' necessary + myProc(newSeq[int](), newSeq[int](1)) + ``` -.. code-block:: nim - {.experimental: "implicitDeref".} +Combination of generic inference with the `auto` type is also unsupported: - proc depth(x: NodeObj): int = ... + ```nim test = "nim c $1" status = 1 + {.experimental: "inferGenericTypes".} + + proc produceValue[T]: auto = default(T) + let a: int = produceValue() # 'auto' cannot be inferred here + ``` + +**Note**: The described inference does not permit the creation of overrides based on +the return type of a procedure. It is a mapping mechanism that does not attempt to +perform deeper inference, nor does it modify what is a valid override. + + ```nim test = "nim c $1" status = 1 + # Doesn't affect the following code, it is invalid either way + {.experimental: "inferGenericTypes".} + + proc a: int = 0 + proc a: float = 1.0 # Fails! Invalid code and not recommended + ``` + + +Sequence literals +----------------- + +Top-down type inference applies to sequence literals. + +```nim +let x: seq[seq[float]] = @[@[1, 2, 3], @[4, 5, 6]] +``` + +This behavior is tied to the `@` overloads in the `system` module, +so overloading `@` can disable this behavior. This can be circumvented by +specifying the `` system.`@` `` overload. + +```nim +proc `@`(x: string): string = "@" & x + +# does not compile: +let x: seq[float] = @[1, 2, 3] +# compiles: +let x: seq[float] = system.`@`([1, 2, 3]) +``` + + +Package level objects +===================== + +Every Nim module resides in a (nimble) package. An object type can be attached +to the package it resides in. If that is done, the type can be referenced from +other modules as an `incomplete`:idx: object type. This feature allows to +break up recursive type dependencies across module boundaries. Incomplete +object types are always passed `byref` and can only be used in pointer like +contexts (`var/ref/ptr IncompleteObject`) in general, since the compiler does +not yet know the size of the object. To complete an incomplete object, +the `package` pragma has to be used. `package` implies `byref`. + +As long as a type `T` is incomplete, no runtime type information for `T` is +available. + + +Example: + + ```nim + # module A (in an arbitrary package) + type + Pack.SomeObject = object # declare as incomplete object of package 'Pack' + Triple = object + a, b, c: ref SomeObject # pointers to incomplete objects are allowed + + # Incomplete objects can be used as parameters: + proc myproc(x: SomeObject) = discard + ``` + + + ```nim + # module B (in package "Pack") + type + SomeObject* {.package.} = object # Use 'package' to complete the object + s, t: string + x, y: int + ``` + +This feature will likely be superseded in the future by support for +recursive module dependencies. + + +Importing private symbols +========================= + +In some situations, it may be useful to import all symbols (public or private) +from a module. The syntax `import foo {.all.}` can be used to import all +symbols from the module `foo`. Note that importing private symbols is +generally not recommended. + +See also the experimental [importutils](importutils.html) module. - var - n: Node - new(n) - echo n.depth - # no need to write n[].depth either Code reordering =============== @@ -263,8 +324,7 @@ preface definitions inside a module. Example: -.. code-block:: nim - + ```nim {.experimental: "codeReordering".} proc foo(x: int) = @@ -274,13 +334,14 @@ Example: echo(x) foo(10) + ``` Variables can also be reordered as well. Variables that are *initialized* (i.e. variables that have their declaration and assignment combined in a single statement) can have their entire initialization statement reordered. Be wary of what code is executed at the top level: -.. code-block:: nim + ```nim {.experimental: "codeReordering".} proc a() = @@ -289,6 +350,7 @@ what code is executed at the top level: var foo = 5 a() # outputs: "5" + ``` .. TODO: Let's table this for now. This is an *experimental feature* and so the @@ -299,7 +361,7 @@ what code is executed at the top level: code reordering process, and not after. As an example, the output of this code is the same as it would be with code reordering disabled. - .. code-block:: nim + ```nim {.experimental: "codeReordering".} proc x() = @@ -308,11 +370,12 @@ what code is executed at the top level: var foo = 4 x() # "false" + ``` It is important to note that reordering *only* works for symbols at top level scope. Therefore, the following will *fail to compile:* -.. code-block:: nim + ```nim {.experimental: "codeReordering".} proc a() = @@ -321,64 +384,10 @@ scope. Therefore, the following will *fail to compile:* echo("Hello!") a() + ``` - -Named argument overloading -========================== - -Routines with the same type signature can be called differently if a parameter -has different names. This does not need an `experimental` switch, but is an -unstable feature. - -.. code-block::nim - proc foo(x: int) = - echo "Using x: ", x - proc foo(y: int) = - echo "Using y: ", y - - foo(x = 2) - # Using x: 2 - foo(y = 2) - # Using y: 2 - - -Do notation -=========== - -As a special more convenient notation, proc expressions involved in procedure -calls can use the `do` keyword: - -.. code-block:: nim - sort(cities) do (x,y: string) -> int: - cmp(x.len, y.len) - - # Less parenthesis using the method plus command syntax: - cities = cities.map do (x:string) -> string: - "City of " & x - - # In macros, the do notation is often used for quasi-quoting - macroResults.add quote do: - if not `ex`: - echo `info`, ": Check failed: ", `expString` - -`do` is written after the parentheses enclosing the regular proc params. -The proc expression represented by the do block is appended to them. -In calls using the command syntax, the do block will bind to the immediately -preceding expression, transforming it in a call. - -`do` with parentheses is an anonymous `proc`; however a `do` without -parentheses is just a block of code. The `do` notation can be used to -pass multiple blocks to a macro: - -.. code-block:: nim - macro performWithUndo(task, undo: untyped) = ... - - performWithUndo do: - # multiple-line block of code - # to perform the task - do: - # code to undo it - +This feature will likely be replaced with a better solution to remove +the need for forward declarations. Special Operators ================= @@ -386,8 +395,8 @@ Special Operators dot operators ------------- -**Note**: Dot operators are still experimental and so need to be enabled -via `{.experimental: "dotOperators".}`. +.. note:: Dot operators are still experimental and so need to be enabled + via `{.experimental: "dotOperators".}`. Nim offers a special family of dot operators that can be used to intercept and rewrite proc call and field access attempts, referring @@ -402,19 +411,21 @@ for a dot operator that can be matched against a re-written form of the expression, where the unknown field or proc name is passed to an `untyped` parameter: -.. code-block:: nim + ```nim a.b # becomes `.`(a, b) a.b(c, d) # becomes `.`(a, b, c, d) + ``` The matched dot operators can be symbols of any callable kind (procs, templates and macros), depending on the desired effect: -.. code-block:: nim + ```nim template `.`(js: PJsonNode, field: untyped): JSON = js[astToStr(field)] var js = parseJson("{ x: 1, y: 2}") echo js.x # outputs 1 echo js.y # outputs 2 + ``` The following dot operators are available: @@ -433,8 +444,9 @@ operator `.=` ------------- This operator will be matched against assignments to missing fields. -.. code-block:: nim + ```nim a.b = c # becomes `.=`(a, b, c) + ``` Call operator ------------- @@ -443,7 +455,7 @@ precedence over dot operators, however it does not match missing overloads for existing routines. The experimental `callOperator` switch must be enabled to use this operator. -.. code-block:: nim + ```nim {.experimental: "callOperator".} template `()`(a: int, b: float): untyped = $(a, b) @@ -467,21 +479,95 @@ to use this operator. doAssert not compiles(a.b(c)) # gives a type mismatch error same as b(a, c) doAssert (a.b)(c) == `()`(a.b, c) + ``` + + +Extended macro pragmas +====================== + +Macro pragmas as described in [the manual](manual.html#userminusdefined-pragmas-macro-pragmas) +can also be applied to type, variable and constant declarations. + +For types: + + ```nim + type + MyObject {.schema: "schema.protobuf".} = object + ``` + +This is translated to a call to the `schema` macro with a `nnkTypeDef` +AST node capturing the left-hand side, remaining pragmas and the right-hand +side of the definition. The macro can return either a type section or +another `nnkTypeDef` node, both of which will replace the original row +in the type section. + +In the future, this `nnkTypeDef` argument may be replaced with a unary +type section node containing the type definition, or some other node that may +be more convenient to work with. The ability to return nodes other than type +definitions may also be supported, however currently this is not convenient +when dealing with mutual type recursion. For now, macros can return an unused +type definition where the right-hand node is of kind `nnkStmtListType`. +Declarations in this node will be attached to the same scope as +the parent scope of the type section. + +------ + +For variables and constants, it is largely the same, except a unary node with +the same kind as the section containing a single definition is passed to macros, +and macros can return any expression. + + ```nim + var + a = ... + b {.importc, foo, nodecl.} = ... + c = ... + ``` + +Assuming `foo` is a macro or a template, this is roughly equivalent to: + + ```nim + var a = ... + foo: + var b {.importc, nodecl.} = ... + var c = ... + ``` + + +Symbols as template/macro calls (alias syntax) +============================================== + +Templates and macros that have no generic parameters and no required arguments +can be called as lone symbols, i.e. without parentheses. This is useful for +repeated uses of complex expressions that cannot conveniently be represented +as runtime values. + + ```nim + type Foo = object + bar: int + + var foo = Foo(bar: 10) + template bar: int = foo.bar + assert bar == 10 + bar = 15 + assert bar == 15 + ``` Not nil annotation ================== **Note:** This is an experimental feature. It can be enabled with -`{.experimental: "notnil"}`. +`{.experimental: "notnil".}`. -All types for which `nil` is a valid value can be annotated with the `not -nil` annotation to exclude `nil` as a valid value: +All types for which `nil` is a valid value can be annotated with the +`not nil` annotation to exclude `nil` as a valid value. Note that only local +symbols are checked. -.. code-block:: nim - {.experimental: "notnil"} + ```nim + {.experimental: "notnil".} type + TObj = object PObject = ref TObj not nil TProc = (proc (x, y: int)) not nil @@ -492,14 +578,323 @@ nil` annotation to exclude `nil` as a valid value: p(nil) # and also this: - var x: PObject - p(x) + proc foo = + var x: PObject + p(x) + + foo() + ``` The compiler ensures that every code path initializes variables which contain non-nilable pointers. The details of this analysis are still to be specified here. -.. include:: manual_experimental_strictnotnil.rst +.. include:: manual_experimental_strictnotnil.md + + +Aliasing restrictions in parameter passing +========================================== + +.. note:: The aliasing restrictions are currently not enforced by the + implementation and need to be fleshed out further. + +"Aliasing" here means that the underlying storage locations overlap in memory +at runtime. An "output parameter" is a parameter of type `var T`, +an input parameter is any parameter that is not of type `var`. + +1. Two output parameters should never be aliased. +2. An input and an output parameter should not be aliased. +3. An output parameter should never be aliased with a global or thread local + variable referenced by the called proc. +4. An input parameter should not be aliased with a global or thread local + variable updated by the called proc. + +One problem with rules 3 and 4 is that they affect specific global or thread +local variables, but Nim's effect tracking only tracks "uses no global variable" +via `.noSideEffect`. The rules 3 and 4 can also be approximated by a different rule: + +5. A global or thread local variable (or a location derived from such a location) + can only passed to a parameter of a `.noSideEffect` proc. + + +Strict funcs +============ + +Since version 1.4, a stricter definition of "side effect" is available. +In addition to the existing rule that a side effect is calling a function +with side effects, the following rule is also enforced: + +A store to the heap via a `ref` or `ptr` indirection is not allowed. + +For example: + + ```nim + {.experimental: "strictFuncs".} + + type + Node = ref object + le, ri: Node + data: string + + func len(n: Node): int = + # valid: len does not have side effects + var it = n + while it != nil: + inc result + it = it.ri + + func mut(n: Node) = + var it = n + while it != nil: + it.data = "yeah" # forbidden mutation + it = it.ri + + ``` + + +View types +========== + +.. tip:: `--experimental:views`:option: is more effective + with `--experimental:strictFuncs`:option:. + +A view type is a type that is or contains one of the following types: + +- `lent T` (view into `T`) +- `openArray[T]` (pair of (pointer to array of `T`, size)) + +For example: + + ```nim + type + View1 = openArray[byte] + View2 = lent string + View3 = Table[openArray[char], int] + ``` + + +Exceptions to this rule are types constructed via `ptr` or `proc`. +For example, the following types are **not** view types: + + ```nim + type + NotView1 = proc (x: openArray[int]) + NotView2 = ptr openArray[char] + NotView3 = ptr array[4, lent int] + ``` + + +The mutability aspect of a view type is not part of the type but part +of the locations it's derived from. More on this later. + +A *view* is a symbol (a let, var, const, etc.) that has a view type. + +Since version 1.4, Nim allows view types to be used as local variables. +This feature needs to be enabled via `{.experimental: "views".}`. + +A local variable of a view type *borrows* from the locations and +it is statically enforced that the view does not outlive the location +it was borrowed from. + +For example: + + ```nim + {.experimental: "views".} + + proc take(a: openArray[int]) = + echo a.len + + proc main(s: seq[int]) = + var x: openArray[int] = s # 'x' is a view into 's' + # it is checked that 'x' does not outlive 's' and + # that 's' is not mutated. + for i in 0 .. high(x): + echo x[i] + take(x) + + take(x.toOpenArray(0, 1)) # slicing remains possible + let y = x # create a view from a view + take y + # it is checked that 'y' does not outlive 'x' and + # that 'x' is not mutated as long as 'y' lives. + + + main(@[11, 22, 33]) + ``` + + +A local variable of a view type can borrow from a location +derived from a parameter, another local variable, a global `const` or `let` +symbol or a thread-local `var` or `let`. + +Let `p` the proc that is analysed for the correctness of the borrow operation. + +Let `source` be one of: + +- A formal parameter of `p`. Note that this does not cover parameters of + inner procs. +- The `result` symbol of `p`. +- A local `var` or `let` or `const` of `p`. Note that this does + not cover locals of inner procs. +- A thread-local `var` or `let`. +- A global `let` or `const`. +- A constant array/seq/object/tuple constructor. + + +Path expressions +---------------- + +A location derived from `source` is then defined as a path expression that +has `source` as the owner. A path expression `e` is defined recursively: + +- `source` itself is a path expression. +- Container access like `e[i]` is a path expression. +- Tuple access `e[0]` is a path expression. +- Object field access `e.field` is a path expression. +- `system.toOpenArray(e, ...)` is a path expression. +- Pointer dereference `e[]` is a path expression. +- An address `addr e` is a path expression. +- A type conversion `T(e)` is a path expression. +- A cast expression `cast[T](e)` is a path expression. +- `f(e, ...)` is a path expression if `f`'s return type is a view type. + Because the view can only have been borrowed from `e`, we then know + that the owner of `f(e, ...)` is `e`. + + +If a view type is used as a return type, the location must borrow from a location +that is derived from the first parameter that is passed to the proc. +See [the manual](manual.html#procedures-var-return-type) +for details about how this is done for `var T`. + +A mutable view can borrow from a mutable location, an immutable view can borrow +from both a mutable or an immutable location. + +If a view borrows from a mutable location, the view can be used to update the +location. Otherwise it cannot be used for mutations. + +The *duration* of a borrow is the span of commands beginning from the assignment +to the view and ending with the last usage of the view. + +For the duration of the borrow operation, no mutations to the borrowed locations +may be performed except via the view that borrowed from the +location. The borrowed location is said to be *sealed* during the borrow. + + ```nim + {.experimental: "views".} + + type + Obj = object + field: string + + proc dangerous(s: var seq[Obj]) = + let v: lent Obj = s[0] # seal 's' + s.setLen 0 # prevented at compile-time because 's' is sealed. + echo v.field + ``` + + +The scope of the view does not matter: + + ```nim + proc valid(s: var seq[Obj]) = + let v: lent Obj = s[0] # begin of borrow + echo v.field # end of borrow + s.setLen 0 # valid because 'v' isn't used afterwards + ``` + + +The analysis requires as much precision about mutations as is reasonably obtainable, +so it is more effective with the experimental [strict funcs] +feature. In other words `--experimental:views`:option: works better +with `--experimental:strictFuncs`:option:. + +The analysis is currently control flow insensitive: + + ```nim + proc invalid(s: var seq[Obj]) = + let v: lent Obj = s[0] + if false: + s.setLen 0 + echo v.field + ``` + +In this example, the compiler assumes that `s.setLen 0` invalidates the +borrow operation of `v` even though a human being can easily see that it +will never do that at runtime. + + +Start of a borrow +----------------- + +A borrow starts with one of the following: + +- The assignment of a non-view-type to a view-type. +- The assignment of a location that is derived from a local parameter + to a view-type. + + +End of a borrow +--------------- + +A borrow operation ends with the last usage of the view variable. + + +Reborrows +--------- + +A view `v` can borrow from multiple different locations. However, the borrow +is always the full span of `v`'s lifetime and every location that is borrowed +from is sealed during `v`'s lifetime. + + +Algorithm +--------- + +The following section is an outline of the algorithm that the current implementation +uses. The algorithm performs two traversals over the AST of the procedure or global +section of code that uses a view variable. No fixpoint iterations are performed, the +complexity of the analysis is O(N) where N is the number of nodes of the AST. + +The first pass over the AST computes the lifetime of each local variable based on +a notion of an "abstract time", in the implementation it's a simple integer that is +incremented for every visited node. + +In the second pass, information about the underlying object "graphs" is computed. +Let `v` be a parameter or a local variable. Let `G(v)` be the graph +that `v` belongs to. A graph is defined by the set of variables that belong +to the graph. Initially for all `v`: `G(v) = {v}`. Every variable can only +be part of a single graph. + +Assignments like `a = b` "connect" two variables, both variables end up in the +same graph `{a, b} = G(a) = G(b)`. Unfortunately, the pattern to look for is +much more complex than that and can involve multiple assignment targets +and sources: + + f(x, y) = g(a, b) + +connects `x` and `y` to `a` and `b`: `G(x) = G(y) = G(a) = G(b) = {x, y, a, b}`. +A type based alias analysis rules out some of these combinations, for example +a `string` value cannot possibly be connected to a `seq[int]`. + +A pattern like `v[] = value` or `v.field = value` marks `G(v)` as mutated. +After the second pass a set of disjoint graphs was computed. + +For strict functions it is then enforced that there is no graph that is both mutated +and has an element that is an immutable parameter (that is a parameter that is not +of type `var T`). + +For borrow checking, a different set of checks is performed. Let `v` be the view +and `b` the location that is borrowed from. + +- The lifetime of `v` must not exceed `b`'s lifetime. Note: The lifetime of + a parameter is the complete proc body. +- If `v` is used for a mutation, `b` must be a mutable location too. +- During `v`'s lifetime, `G(b)` can only be modified by `v` (and only if + `v` is a mutable view). +- If `v` is `result` then `b` has to be a location derived from the first + formal parameter or from a constant location. +- A view cannot be used for a read or a write access before it was assigned to. + Concepts ======== @@ -509,7 +904,7 @@ arbitrary set of requirements that the matched type must satisfy. Concepts are written in the following form: -.. code-block:: nim + ```nim type Comparable = concept x, y (x < y) is bool @@ -522,11 +917,13 @@ Concepts are written in the following form: for value in s: value is T + ``` -The concept is a match if: +The concept matches if: -a) all of the expressions within the body can be compiled for the tested type -b) all statically evaluable boolean expressions in the body must be true +a) all expressions within the body can be compiled for the tested type +b) all statically evaluable boolean expressions in the body are true +c) all type modifiers specified match their respective definitions The identifiers following the `concept` keyword represent instances of the currently matched type. You can apply any of the standard type modifiers such @@ -534,26 +931,28 @@ as `var`, `ref`, `ptr` and `static` to denote a more specific type of instance. You can also apply the `type` modifier to create a named instance of the type itself: -.. code-block:: nim + ```nim type MyConcept = concept x, var v, ref r, ptr p, static s, type T ... + ``` Within the concept body, types can appear in positions where ordinary values and parameters are expected. This provides a more convenient way to check for the presence of callable symbols with specific signatures: -.. code-block:: nim + ```nim type OutputStream = concept var s s.write(string) + ``` In order to check for symbols accepting `type` params, you must prefix the type with the explicit `type` modifier. The named instance of the type, following the `concept` keyword is also considered to have the explicit modifier and will be matched only as a type. -.. code-block:: nim + ```nim type # Let's imagine a user-defined casting framework with operators # such as `val.to(string)` and `val.to(JSonValue)`. We can test @@ -571,6 +970,7 @@ explicit modifier and will be matched only as a type. x is AdditiveMonoid -x is T x - y is T + ``` Please note that the `is` operator allows one to easily verify the precise type signatures of the required operations, but since type inference and @@ -590,11 +990,12 @@ When you need to understand why the compiler is not matching a particular concept and, as a result, a wrong overload is selected, you can apply the `explain` pragma to either the concept body or a particular call-site. -.. code-block:: nim + ```nim type MyConcept {.explain.} = concept ... overloadedProc(x, y, z) {.explain.} + ``` This will provide Hints in the compiler output either every time the concept is not matched or only on the particular call-site. @@ -605,7 +1006,7 @@ Generic concepts and type binding rules The concept types can be parametric just like the regular generic types: -.. code-block:: nim + ```nim ### matrixalgo.nim import std/typetraits @@ -665,6 +1066,7 @@ The concept types can be parametric just like the regular generic types: echo m.transposed.determinant setPerspectiveProjection projectionMatrix + ``` When the concept type is matched against a concrete type, the unbound type parameters are inferred from the body of the concept in a way that closely @@ -678,10 +1080,11 @@ and `x.data is seq[T]`. Unbound static params will be inferred from expressions involving the `==` operator and also when types dependent on them are being matched: -.. code-block:: nim + ```nim type MatrixReducer[M, N: static int; T] = concept x x.reduce(SquareMatrix[N, T]) is array[M, int] + ``` The Nim compiler includes a simple linear equation solver, allowing it to infer static params in some situations where integer arithmetic is involved. @@ -692,7 +1095,7 @@ modifier to any of the otherwise inferable types to get a type that will be matched without permanently inferring it. This may be useful when you need to match several procs accepting the same wide class of types: -.. code-block:: nim + ```nim type Enumerable[T] = concept e for v in e: @@ -709,12 +1112,13 @@ to match several procs accepting the same wide class of types: # it's also possible to give an alias name to a `bind many` type class type Enum = distinct Enumerable o.baz is Enum + ``` On the other hand, using `bind once` types allows you to test for equivalent types used in multiple signatures, without actually requiring any concrete types, thus allowing you to encode implementation-defined types: -.. code-block:: nim + ```nim type MyConcept = concept x type T1 = auto @@ -725,6 +1129,7 @@ types, thus allowing you to encode implementation-defined types: x.alpha(T2) x.omega(T2) # both procs must accept the same type # and it must be a numeric sequence + ``` As seen in the previous examples, you can refer to generic concepts such as `Enumerable[T]` just by their short name. Much like the regular generic types, @@ -742,9 +1147,7 @@ in any required way. For example, here is how one might define the classic `Functor` concept from Haskell and then demonstrate that Nim's `Option[T]` type is an instance of it: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/[sugar, typetraits] type @@ -765,6 +1168,7 @@ type is an instance of it: import std/options echo Option[int] is Functor # prints true + ``` Concept derived values @@ -774,7 +1178,7 @@ All top level constants or types appearing within the concept body are accessible through the dot operator in procs where the concept was successfully matched to a concrete type: -.. code-block:: nim + ```nim type DateTime = concept t1, t2, type T const Min = T.MinDate @@ -796,6 +1200,7 @@ matched to a concrete type: deviation: float ... + ``` Concept refinement @@ -808,7 +1213,7 @@ overload resolution, Nim will assign a higher precedence to the most specific one. As an alternative way of defining concept refinements, you can use the object inheritance syntax involving the `of` keyword: -.. code-block:: nim + ```nim type Graph = concept g, type G of EquallyComparable, Copyable type @@ -842,6 +1247,7 @@ object inheritance syntax involving the `of` keyword: proc f(g: IncidendeGraph) proc f(g: BidirectionalGraph) # this one will be preferred if we pass a type # matching the BidirectionalGraph concept + ``` .. Converter type classes @@ -851,7 +1257,7 @@ object inheritance syntax involving the `of` keyword: a small set of simpler types. This is achieved with a `return` statement within the concept body: - .. code-block:: nim + ```nim type Stringable = concept x $x is string @@ -875,6 +1281,7 @@ object inheritance syntax involving the `of` keyword: # the same call at the cost of additional instantiations # the varargs param will be converted to a tuple proc log(format: static string, varargs[distinct StringRef]) + ``` .. @@ -902,7 +1309,7 @@ object inheritance syntax involving the `of` keyword: a converter type class, which converts the regular instances of the matching types to the corresponding VTable type. - .. code-block:: nim + ```nim type IntEnumerable = vtref Enumerable[int] @@ -915,6 +1322,7 @@ object inheritance syntax involving the `of` keyword: proc addStream(o: var MyObject, e: OutputStream.vtref) = o.streams.add e + ``` The procs that will be included in the vtable are derived from the concept body and include all proc calls for which all param types were specified as @@ -934,97 +1342,44 @@ object inheritance syntax involving the `of` keyword: the `vtptr` magic produced types bound to `ptr` types. -Type bound operations -===================== - -There are 4 operations that are bound to a type: - -1. Assignment -2. Moves -3. Destruction -4. Deep copying for communication between threads - -These operations can be *overridden* instead of *overloaded*. This means the -implementation is automatically lifted to structured types. For instance if type -`T` has an overridden assignment operator `=` this operator is also used -for assignments of the type `seq[T]`. Since these operations are bound to a -type they have to be bound to a nominal type for reasons of simplicity of -implementation: This means an overridden `deepCopy` for `ref T` is really -bound to `T` and not to `ref T`. This also means that one cannot override -`deepCopy` for both `ptr T` and `ref T` at the same time; instead a -helper distinct or object type has to be used for one pointer type. - -Assignments, moves and destruction are specified in -the `destructors <destructors.html>`_ document. - - -deepCopy --------- - -`=deepCopy` is a builtin that is invoked whenever data is passed to -a `spawn`'ed proc to ensure memory safety. The programmer can override its -behaviour for a specific `ref` or `ptr` type `T`. (Later versions of the -language may weaken this restriction.) - -The signature has to be: +.. + deepCopy + -------- + `=deepCopy` is a builtin that is invoked whenever data is passed to + a `spawn`'ed proc to ensure memory safety. The programmer can override its + behaviour for a specific `ref` or `ptr` type `T`. (Later versions of the + language may weaken this restriction.) -.. code-block:: nim - proc `=deepCopy`(x: T): T + The signature has to be: -This mechanism will be used by most data structures that support shared memory -like channels to implement thread safe automatic memory management. + ```nim + proc `=deepCopy`(x: T): T + ``` -The builtin `deepCopy` can even clone closures and their environments. See -the documentation of `spawn <#parallel-amp-spawn-spawn-statement>`_ for details. + This mechanism will be used by most data structures that support shared memory, + like channels, to implement thread safe automatic memory management. + The builtin `deepCopy` can even clone closures and their environments. See + the documentation of [spawn][spawn statement] for details. -Case statement macros -===================== -Macros named `case` can rewrite `case` statements for certain types in order to -implement `pattern matching`:idx:. The following example implements a -simplistic form of pattern matching for tuples, leveraging the existing -equality operator for tuples (as provided in `system.==`): +Dynamic arguments for bindSym +============================= -.. code-block:: nim - :test: "nim c $1" +This experimental feature allows the symbol name argument of `macros.bindSym` +to be computed dynamically. - {.experimental: "caseStmtMacros".} + ```nim + {.experimental: "dynamicBindSym".} import std/macros - macro `case`(n: tuple): untyped = - result = newTree(nnkIfStmt) - let selector = n[0] - for i in 1 ..< n.len: - let it = n[i] - case it.kind - of nnkElse, nnkElifBranch, nnkElifExpr, nnkElseExpr: - result.add it - of nnkOfBranch: - for j in 0..it.len-2: - let cond = newCall("==", selector, it[j]) - result.add newTree(nnkElifBranch, cond, it[^1]) - else: - error "custom 'case' for tuple cannot handle this node", it + macro callOp(opName, arg1, arg2): untyped = + result = newCall(bindSym($opName), arg1, arg2) - case ("foo", 78) - of ("foo", 78): echo "yes" - of ("bar", 88): echo "no" - else: discard - - -Currently case statement macros must be enabled explicitly -via `{.experimental: "caseStmtMacros".}`. - -`case` macros are subject to overload resolution. The type of the -`case` statement's selector expression is matched against the type -of the first argument of the `case` macro. Then the complete `case` -statement is passed in place of the argument and the macro is evaluated. - -In other words, the macro needs to transform the full `case` statement -but only the statement's selector expression is used to determine which -macro to call. + echo callOp("+", 1, 2) + echo callOp("-", 5, 4) + ``` Term rewriting macros @@ -1035,48 +1390,51 @@ a *name* but also a *pattern* that is searched for after the semantic checking phase of the compiler: This means they provide an easy way to enhance the compilation pipeline with user defined optimizations: -.. code-block:: nim - template optMul{`*`(a, 2)}(a: int): int = a+a + ```nim + template optMul{`*`(a, 2)}(a: int): int = a + a let x = 3 echo x * 2 + ``` The compiler now rewrites `x * 2` as `x + x`. The code inside the -curlies is the pattern to match against. The operators `*`, `**`, +curly brackets is the pattern to match against. The operators `*`, `**`, `|`, `~` have a special meaning in patterns if they are written in infix notation, so to match verbatim against `*` the ordinary function call syntax needs to be used. -Term rewriting macro are applied recursively, up to a limit. This means that +Term rewriting macros are applied recursively, up to a limit. This means that if the result of a term rewriting macro is eligible for another rewriting, the compiler will try to perform it, and so on, until no more optimizations are applicable. To avoid putting the compiler into an infinite loop, there is a hard limit on how many times a single term rewriting macro can be applied. Once this limit has been passed, the term rewriting macro will be ignored. -Unfortunately optimizations are hard to get right and even the tiny example +Unfortunately optimizations are hard to get right and even this tiny example is **wrong**: -.. code-block:: nim - template optMul{`*`(a, 2)}(a: int): int = a+a + ```nim + template optMul{`*`(a, 2)}(a: int): int = a + a proc f(): int = echo "side effect!" result = 55 echo f() * 2 + ``` We cannot duplicate 'a' if it denotes an expression that has a side effect! Fortunately Nim supports side effect analysis: -.. code-block:: nim - template optMul{`*`(a, 2)}(a: int{noSideEffect}): int = a+a + ```nim + template optMul{`*`(a, 2)}(a: int{noSideEffect}): int = a + a proc f(): int = echo "side effect!" result = 55 echo f() * 2 # not optimized ;-) + ``` You can make one overload matching with a constraint and one without, and the one with a constraint will have precedence, and so you can handle both cases @@ -1086,13 +1444,15 @@ So what about `2 * a`? We should tell the compiler `*` is commutative. We cannot really do that however as the following code only swaps arguments blindly: -.. code-block:: nim - template mulIsCommutative{`*`(a, b)}(a, b: int): int = b*a + ```nim + template mulIsCommutative{`*`(a, b)}(a, b: int): int = b * a + ``` What optimizers really need to do is a *canonicalization*: -.. code-block:: nim - template canonMul{`*`(a, b)}(a: int{lit}, b: int): int = b*a + ```nim + template canonMul{`*`(a, b)}(a: int{lit}, b: int): int = b * a + ``` The `int{lit}` parameter pattern matches against an expression of type `int`, but only if it's a literal. @@ -1109,7 +1469,7 @@ The `parameter constraint`:idx: expression can use the operators `|` (or), Predicate Meaning =================== ===================================================== `atom` The matching node has no children. -`lit` The matching node is a literal like "abc", 12. +`lit` The matching node is a literal like `"abc"`, `12`. `sym` The matching node must be a symbol (a bound identifier). `ident` The matching node must be an identifier (an unbound @@ -1150,11 +1510,31 @@ The `alias` and `noalias` predicates refer not only to the matching AST, but also to every other bound parameter; syntactically they need to occur after the ordinary AST predicates: -.. code-block:: nim + ```nim template ex{a = b + c}(a: int{noalias}, b, c: int) = # this transformation is only valid if 'b' and 'c' do not alias 'a': a = b inc a, c + ``` + +Another example: + + ```nim + proc somefunc(s: string) = assert s == "variable" + proc somefunc(s: string{nkStrLit}) = assert s == "literal" + proc somefunc(s: string{nkRStrLit}) = assert s == r"raw" + proc somefunc(s: string{nkTripleStrLit}) = assert s == """triple""" + proc somefunc(s: static[string]) = assert s == "constant" + + # Use parameter constraints to provide overloads based on both the input parameter type and form. + var variable = "variable" + somefunc(variable) + const constant = "constant" + somefunc(constant) + somefunc("literal") + somefunc(r"raw") + somefunc("""triple""") + ``` Pattern operators @@ -1164,50 +1544,50 @@ The operators `*`, `**`, `|`, `~` have a special meaning in patterns if they are written in infix notation. -The `|` operator -~~~~~~~~~~~~~~~~~~ +### The `|` operator The `|` operator if used as infix operator creates an ordered choice: -.. code-block:: nim + ```nim template t{0|1}(): untyped = 3 let a = 1 # outputs 3: echo a + ``` The matching is performed after the compiler performed some optimizations like constant folding, so the following does not work: -.. code-block:: nim + ```nim template t{0|1}(): untyped = 3 # outputs 1: echo 1 + ``` The reason is that the compiler already transformed the 1 into "1" for the `echo` statement. However, a term rewriting macro should not change the -semantics anyway. In fact they can be deactivated with the `--patterns:off` +semantics anyway. In fact, they can be deactivated with the `--patterns:off`:option: command line option or temporarily with the `patterns` pragma. -The `{}` operator -~~~~~~~~~~~~~~~~~~~ +### The `{}` operator A pattern expression can be bound to a pattern parameter via the `expr{param}` notation: -.. code-block:: nim - template t{(0|1|2){x}}(x: untyped): untyped = x+1 + ```nim + template t{(0|1|2){x}}(x: untyped): untyped = x + 1 let a = 1 # outputs 2: echo a + ``` -The `~` operator -~~~~~~~~~~~~~~~~~~ +### The `~` operator -The `~` operator is the **not** operator in patterns: +The `~` operator is the 'not' operator in patterns: -.. code-block:: nim + ```nim template t{x = (~x){y} and (~x){z}}(x, y, z: bool) = x = y if x: x = z @@ -1218,15 +1598,15 @@ The `~` operator is the **not** operator in patterns: c = false a = b and c echo a + ``` -The `*` operator -~~~~~~~~~~~~~~~~~~ +### The `*` operator The `*` operator can *flatten* a nested binary expression like `a & b & c` to `&(a, b, c)`: -.. code-block:: nim + ```nim var calls = 0 @@ -1242,6 +1622,7 @@ to `&(a, b, c)`: # check that it's been optimized properly: doAssert calls == 1 + ``` The second operator of `*` must be a parameter; it is used to gather all the @@ -1250,17 +1631,17 @@ is passed to `optConc` in `a` as a special list (of kind `nkArgList`) which is flattened into a call expression; thus the invocation of `optConc` produces: -.. code-block:: nim - `&&`("my", space & "awe", "some ", "concat") + ```nim + `&&`("my", space & "awe", "some ", "concat") + ``` -The `**` operator -~~~~~~~~~~~~~~~~~~~ +### The `**` operator The `**` is much like the `*` operator, except that it gathers not only all the arguments, but also the matched operators in reverse polish notation: -.. code-block:: nim + ```nim import std/macros type @@ -1281,35 +1662,58 @@ all the arguments, but also the matched operators in reverse polish notation: var x, y, z: Matrix echo x + y * z - x + ``` This passes the expression `x + y * z - x` to the `optM` macro as -an `nnkArglist` node containing:: +an `nnkArgList` node containing: - Arglist - Sym "x" - Sym "y" - Sym "z" - Sym "*" - Sym "+" - Sym "x" - Sym "-" + Arglist + Sym "x" + Sym "y" + Sym "z" + Sym "*" + Sym "+" + Sym "x" + Sym "-" -(Which is the reverse polish notation of `x + y * z - x`.) +(This is the reverse polish notation of `x + y * z - x`.) Parameters ---------- Parameters in a pattern are type checked in the matching process. If a -parameter is of the type `varargs` it is treated specially and it can match +parameter is of the type `varargs`, it is treated specially and can match 0 or more arguments in the AST to be matched against: -.. code-block:: nim + ```nim template optWrite{ write(f, x) ((write|writeLine){w})(f, y) }(x, y: varargs[untyped], f: File, w: untyped) = w(f, x, y) + ``` + + +noRewrite pragma +---------------- + +Term rewriting macros and templates are currently greedy and +they will rewrite as long as there is a match. +There was no way to ensure some rewrite happens only once, +e.g. when rewriting term to same term plus extra content. + +`noRewrite` pragma can actually prevent further rewriting on marked code, +e.g. with given example `echo("ab")` will be rewritten just once: + + ```nim + template pwnEcho{echo(x)}(x: untyped) = + {.noRewrite.}: echo("pwned!") + + echo "ab" + ``` + +`noRewrite` pragma can be useful to control term-rewriting macros recursion. @@ -1319,12 +1723,13 @@ Example: Partial evaluation The following example shows how some simple partial evaluation can be implemented with term rewriting: -.. code-block:: nim + ```nim proc p(x, y: int; cond: bool): int = result = if cond: x + y else: x - y template optP1{p(x, y, true)}(x, y: untyped): untyped = x + y template optP2{p(x, y, false)}(x, y: untyped): untyped = x - y + ``` Example: Hoisting @@ -1332,7 +1737,7 @@ Example: Hoisting The following example shows how some form of hoisting can be implemented: -.. code-block:: nim + ```nim import std/pegs template optPeg{peg(pattern)}(pattern: string{lit}): Peg = @@ -1342,6 +1747,7 @@ The following example shows how some form of hoisting can be implemented: for i in 0 .. 3: echo match("(a b c)", peg"'(' @ ')'") echo match("W_HI_Le", peg"\y 'while'") + ``` The `optPeg` template optimizes the case of a peg constructor with a string literal, so that the pattern will only be parsed once at program startup and @@ -1353,9 +1759,9 @@ AST based overloading ===================== Parameter constraints can also be used for ordinary routine parameters; these -constraints affect ordinary overloading resolution then: +constraints then affect ordinary overloading resolution: -.. code-block:: nim + ```nim proc optLit(a: string{lit|`const`}) = echo "string literal" proc optLit(a: string) = @@ -1370,6 +1776,7 @@ constraints affect ordinary overloading resolution then: optLit("literal") optLit(constant) optLit(variable) + ``` However, the constraints `alias` and `noalias` are not available in ordinary routines. @@ -1384,29 +1791,32 @@ Nim has two flavors of parallelism: Nim has a builtin thread pool that can be used for CPU intensive tasks. For IO intensive tasks the `async` and `await` features should be -used instead. Both parallel and spawn need the `threadpool <threadpool.html>`_ +used instead. Both parallel and spawn need the [threadpool](threadpool.html) module to work. Somewhat confusingly, `spawn` is also used in the `parallel` statement with slightly different semantics. `spawn` always takes a call expression of -the form `f(a, ...)`. Let `T` be `f`'s return type. If `T` is `void` -then `spawn`'s return type is also `void` otherwise it is `FlowVar[T]`. +the form `f(a, ...)`. Let `T` be `f`'s return type. If `T` is `void`, +then `spawn`'s return type is also `void`, otherwise it is `FlowVar[T]`. -Within a `parallel` section sometimes the `FlowVar[T]` is eliminated +Within a `parallel` section, the `FlowVar[T]` is sometimes eliminated to `T`. This happens when `T` does not contain any GC'ed memory. The compiler can ensure the location in `location = spawn f(...)` is not read prematurely within a `parallel` section and so there is no need for the overhead of an indirection via `FlowVar[T]` to ensure correctness. -**Note**: Currently exceptions are not propagated between `spawn`'ed tasks! +.. note:: Currently exceptions are not propagated between `spawn`'ed tasks! + +This feature is likely to be removed in the future as external packages +can have better solutions. Spawn statement --------------- -`spawn`:idx: can be used to pass a task to the thread pool: +The `spawn`:idx: statement can be used to pass a task to the thread pool: -.. code-block:: nim + ```nim import std/threadpool proc processLine(line: string) = @@ -1415,6 +1825,7 @@ Spawn statement for x in lines("myinput.txt"): spawn processLine(x) sync() + ``` For reasons of type safety and implementation simplicity the expression that `spawn` takes is restricted: @@ -1425,10 +1836,10 @@ that `spawn` takes is restricted: * `f`'s parameters may not be of type `var`. This means one has to use raw `ptr`'s for data passing reminding the programmer to be careful. -* `ref` parameters are deeply copied which is a subtle semantic change and - can cause performance problems but ensures memory safety. This deep copy - is performed via `system.deepCopy` and so can be overridden. -* For *safe* data exchange between `f` and the caller a global `TChannel` +* `ref` parameters are deeply copied, which is a subtle semantic change and + can cause performance problems, but ensures memory safety. This deep copy + is performed via `system.deepCopy`, so it can be overridden. +* For *safe* data exchange between `f` and the caller, a global `Channel` needs to be used. However, since spawn can return a result, often no further communication is required. @@ -1438,7 +1849,7 @@ a `data flow variable`:idx: `FlowVar[T]` that can be read from. The reading with the `^` operator is **blocking**. However, one can use `blockUntilAny` to wait on multiple flow variables at the same time: -.. code-block:: nim + ```nim import std/threadpool, ... # wait until 2 out of 3 servers received the update: @@ -1450,11 +1861,12 @@ wait on multiple flow variables at the same time: assert index >= 0 responses.del(index) discard blockUntilAny(responses) + ``` -Data flow variables ensure that no data races -are possible. Due to technical limitations not every type `T` is possible in -a data flow variable: `T` has to be of the type `ref`, `string`, `seq` -or of a type that doesn't contain a type that is garbage collected. This +Data flow variables ensure that no data races are possible. Due to +technical limitations, not every type `T` can be used in +a data flow variable: `T` has to be a `ref`, `string`, `seq` +or of a type that doesn't contain any GC'd type. This restriction is not hard to work-around in practice. @@ -1464,17 +1876,15 @@ Parallel statement Example: -.. code-block:: nim - :test: "nim c --threads:on $1" - - # Compute PI in an inefficient way + ```nim test = "nim c --threads:on $1" + # Compute pi in an inefficient way import std/[strutils, math, threadpool] {.experimental: "parallel".} proc term(k: float): float = 4 * math.pow(-1, k) / (2*k + 1) proc pi(n: int): float = - var ch = newSeq[float](n+1) + var ch = newSeq[float](n + 1) parallel: for k in 0..ch.high: ch[k] = spawn term(float(k)) @@ -1482,19 +1892,20 @@ Example: result += ch[k] echo formatFloat(pi(5000)) + ``` The parallel statement is the preferred mechanism to introduce parallelism in a -Nim program. A subset of the Nim language is valid within a `parallel` +Nim program. Only a subset of the Nim language is valid within a `parallel` section. This subset is checked during semantic analysis to be free of data races. A sophisticated `disjoint checker`:idx: ensures that no data races are -possible even though shared memory is extensively supported! +possible, even though shared memory is extensively supported! The subset is in fact the full language with the following restrictions / changes: * `spawn` within a `parallel` section has special semantics. -* Every location of the form `a[i]` and `a[i..j]` and `dest` where +* Every location of the form `a[i]`, `a[i..j]` and `dest` where `dest` is part of the pattern `dest = spawn f(...)` has to be provably disjoint. This is called the *disjoint check*. * Every other complex location `loc` that is used in a spawned @@ -1508,569 +1919,751 @@ restrictions / changes: yet performed for ordinary slices outside of a `parallel` section. -Guards and locks -================ +Strict definitions and `out` parameters +======================================= + +With `experimental: "strictDefs"` *every* local variable must be initialized explicitly before it can be used: -Apart from `spawn` and `parallel` Nim also provides all the common low level -concurrency mechanisms like locks, atomic intrinsics or condition variables. + ```nim + {.experimental: "strictDefs".} -Nim significantly improves on the safety of these features via additional -pragmas: + proc test = + var s: seq[string] + s.add "abc" # invalid! -1) A `guard`:idx: annotation is introduced to prevent data races. -2) Every access of a guarded memory location needs to happen in an - appropriate `locks`:idx: statement. -3) Locks and routines can be annotated with `lock levels`:idx: to allow - potential deadlocks to be detected during semantic analysis. + ``` +Needs to be written as: -Guards and the locks section ----------------------------- + ```nim + {.experimental: "strictDefs".} -Protecting global variables -~~~~~~~~~~~~~~~~~~~~~~~~~~~ + proc test = + var s: seq[string] = @[] + s.add "abc" # valid! -Object fields and global variables can be annotated via a `guard` pragma: + ``` -.. code-block:: nim - var glock: TLock - var gdata {.guard: glock.}: int +A control flow analysis is performed in order to prove that a variable has been written to +before it is used. Thus the following is valid: -The compiler then ensures that every access of `gdata` is within a `locks` -section: + ```nim + {.experimental: "strictDefs".} -.. code-block:: nim - proc invalid = - # invalid: unguarded access: - echo gdata + proc test(cond: bool) = + var s: seq[string] + if cond: + s = @["y"] + else: + s = @[] + s.add "abc" # valid! + ``` - proc valid = - # valid access: - {.locks: [glock].}: - echo gdata +In this example every path does set `s` to a value before it is used. -Top level accesses to `gdata` are always allowed so that it can be initialized -conveniently. It is *assumed* (but not enforced) that every top level statement -is executed before any concurrent action happens. + ```nim + {.experimental: "strictDefs".} -The `locks` section deliberately looks ugly because it has no runtime -semantics and should not be used directly! It should only be used in templates -that also implement some form of locking at runtime: + proc test(cond: bool) = + let s: seq[string] + if cond: + s = @["y"] + else: + s = @[] + ``` + +With `experimental: "strictDefs"`, `let` statements are allowed to not have an initial value, but every path should set `s` to a value before it is used. + + +`out` parameters +---------------- -.. code-block:: nim - template lock(a: TLock; body: untyped) = - pthread_mutex_lock(a) - {.locks: [a].}: - try: - body - finally: - pthread_mutex_unlock(a) +An `out` parameter is like a `var` parameter but it must be written to before it can be used: + ```nim + proc myopen(f: out File; name: string): bool = + f = default(File) + result = open(f, name) + ``` -The guard does not need to be of any particular type. It is flexible enough to -model low level lockfree mechanisms: +While it is usually the better style to use the return type in order to return results API and ABI +considerations might make this infeasible. Like for `var T` Nim maps `out T` to a hidden pointer. +For example POSIX's `stat` routine can be wrapped as: -.. code-block:: nim - var dummyLock {.compileTime.}: int - var atomicCounter {.guard: dummyLock.}: int + ```nim + proc stat*(a1: cstring, a2: out Stat): cint {.importc, header: "<sys/stat.h>".} + ``` - template atomicRead(x): untyped = - {.locks: [dummyLock].}: - memoryReadBarrier() - x +When the implementation of a routine with output parameters is analysed, the compiler +checks that every path before the (implicit or explicit) return does set every output +parameter: - echo atomicRead(atomicCounter) + ```nim + proc p(x: out int; y: out string; cond: bool) = + x = 4 + if cond: + y = "abc" + # error: not every path initializes 'y' + ``` -The `locks` pragma takes a list of lock expressions `locks: [a, b, ...]` -in order to support *multi lock* statements. Why these are essential is -explained in the `lock levels <#guards-and-locks-lock-levels>`_ section. +Out parameters and exception handling +------------------------------------- +The analysis should take exceptions into account (but currently does not): -Protecting general locations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ```nim + proc p(x: out int; y: out string; cond: bool) = + x = canRaise(45) + y = "abc" # <-- error: not every path initializes 'y' + ``` -The `guard` annotation can also be used to protect fields within an object. -The guard then needs to be another field within the same object or a -global variable. +Once the implementation takes exceptions into account it is easy enough to +use `outParam = default(typeof(outParam))` in the beginning of the proc body. -Since objects can reside on the heap or on the stack this greatly enhances the -expressivity of the language: +Out parameters and inheritance +------------------------------ -.. code-block:: nim +It is not valid to pass an lvalue of a supertype to an `out T` parameter: + + ```nim type - ProtectedCounter = object - v {.guard: L.}: int - L: TLock - - proc incCounters(counters: var openArray[ProtectedCounter]) = - for i in 0..counters.high: - lock counters[i].L: - inc counters[i].v - -The access to field `x.v` is allowed since its guard `x.L` is active. -After template expansion, this amounts to: - -.. code-block:: nim - proc incCounters(counters: var openArray[ProtectedCounter]) = - for i in 0..counters.high: - pthread_mutex_lock(counters[i].L) - {.locks: [counters[i].L].}: - try: - inc counters[i].v - finally: - pthread_mutex_unlock(counters[i].L) - -There is an analysis that checks that `counters[i].L` is the lock that -corresponds to the protected location `counters[i].v`. This analysis is called -`path analysis`:idx: because it deals with paths to locations -like `obj.field[i].fieldB[j]`. - -The path analysis is **currently unsound**, but that doesn't make it useless. -Two paths are considered equivalent if they are syntactically the same. - -This means the following compiles (for now) even though it really should not: - -.. code-block:: nim - {.locks: [a[i].L].}: - inc i - access a[i].v - - - -Lock levels ------------ - -Lock levels are used to enforce a global locking order in order to detect -potential deadlocks during semantic analysis. A lock level is an constant -integer in the range 0..1_000. Lock level 0 means that no lock is acquired at -all. - -If a section of code holds a lock of level `M` than it can also acquire any -lock of level `N < M`. Another lock of level `M` cannot be acquired. Locks -of the same level can only be acquired *at the same time* within a -single `locks` section: - -.. code-block:: nim - var a, b: TLock[2] - var x: TLock[1] - # invalid locking order: TLock[1] cannot be acquired before TLock[2]: - {.locks: [x].}: - {.locks: [a].}: - ... - # valid locking order: TLock[2] acquired before TLock[1]: - {.locks: [a].}: - {.locks: [x].}: - ... + Superclass = object of RootObj + a: int + Subclass = object of Superclass + s: string - # invalid locking order: TLock[2] acquired before TLock[2]: - {.locks: [a].}: - {.locks: [b].}: - ... + proc init(x: out Superclass) = + x = Superclass(a: 8) - # valid locking order, locks of the same level acquired at the same time: - {.locks: [a, b].}: - ... + var v: Subclass + init v + use v.s # the 's' field was never initialized! + ``` +However, in the future this could be allowed and provide a better way to write object +constructors that take inheritance into account. -Here is how a typical multilock statement can be implemented in Nim. Note how -the runtime check is required to ensure a global ordering for two locks `a` -and `b` of the same lock level: -.. code-block:: nim - template multilock(a, b: ptr TLock; body: untyped) = - if cast[ByteAddress](a) < cast[ByteAddress](b): - pthread_mutex_lock(a) - pthread_mutex_lock(b) - else: - pthread_mutex_lock(b) - pthread_mutex_lock(a) - {.locks: [a, b].}: - try: - body - finally: - pthread_mutex_unlock(a) - pthread_mutex_unlock(b) +**Note**: The implementation of "strict definitions" and "out parameters" is experimental but the concept +is solid and it is expected that eventually this mode becomes the default in later versions. -Whole routines can also be annotated with a `locks` pragma that takes a lock -level. This then means that the routine may acquire locks of up to this level. -This is essential so that procs can be called within a `locks` section: +Strict case objects +=================== -.. code-block:: nim - proc p() {.locks: 3.} = discard +With `experimental: "strictCaseObjects"` *every* field access is checked to be valid at compile-time. +The field is within a `case` section of an `object`. - var a: TLock[4] - {.locks: [a].}: - # p's locklevel (3) is strictly less than a's (4) so the call is allowed: - p() + ```nim + {.experimental: "strictCaseObjects".} + type + Foo = object + case b: bool + of false: + s: string + of true: + x: int -As usual `locks` is an inferred effect and there is a subtype -relation: `proc () {.locks: N.}` is a subtype of `proc () {.locks: M.}` -iff (M <= N). + var x = Foo(b: true, x: 4) + case x.b + of true: + echo x.x # valid + of false: + echo "no" -The `locks` pragma can also take the special value `"unknown"`. This -is useful in the context of dynamic method dispatching. In the following -example, the compiler can infer a lock level of 0 for the `base` case. -However, one of the overloaded methods calls a procvar which is -potentially locking. Thus, the lock level of calling `g.testMethod` -cannot be inferred statically, leading to compiler warnings. By using -`{.locks: "unknown".}`, the base method can be marked explicitly as -having unknown lock level as well: + case x.b + of false: + echo x.x # error: field access outside of valid case branch: x.x + of true: + echo "no" -.. code-block:: nim - type SomeBase* = ref object of RootObj - type SomeDerived* = ref object of SomeBase - memberProc*: proc () + ``` - method testMethod(g: SomeBase) {.base, locks: "unknown".} = discard - method testMethod(g: SomeDerived) = - if g.memberProc != nil: - g.memberProc() +**Note**: The implementation of "strict case objects" is experimental but the concept +is solid and it is expected that eventually this mode becomes the default in later versions. -noRewrite pragma ----------------- +Quirky routines +=============== -Term rewriting macros and templates are currently greedy and -they will rewrite as long as there is a match. -There was no way to ensure some rewrite happens only once, -e.g. when rewriting term to same term plus extra content. +The default code generation strategy of exceptions under the ARC/ORC model is the so called +`--exceptions:goto` implementation. This implementation inserts a check after every call that +can potentially raise an exception. A typical instruction sequence for this on +for a x86 64 bit machine looks like: -`noRewrite` pragma can actually prevent further rewriting on marked code, -e.g. with given example `echo("ab")` will be rewritten just once: + ``` + cmp DWORD PTR [rbx], 0 + je .L1 + ``` -.. code-block:: nim - template pwnEcho{echo(x)}(x: untyped) = - {.noRewrite.}: echo("pwned!") +This is a memory fetch followed by jump. (An ideal implementation would +use the carry flag and a single instruction like ``jc .L1``.) - echo "ab" +This overhead might not be desired and depending on the semantics of the routine may not be required +either. +So it can be disabled via a `.quirky` annotation: -`noRewrite` pragma can be useful to control term-rewriting macros recursion. + ```nim + proc wontRaise(x: int) {.quirky.} = + if x != 0: + # because of `quirky` this will continue even if `write` raised an IO exception: + write x + wontRaise(x-1) + wontRaise 10 -Aliasing restrictions in parameter passing -========================================== + ``` -**Note**: The aliasing restrictions are currently not enforced by the -implementation and need to be fleshed out further. +If the used exception model is not `--exceptions:goto` then the `quirky` pragma has no effect and is +ignored. -"Aliasing" here means that the underlying storage locations overlap in memory -at runtime. An "output parameter" is a parameter of type `var T`, -an input parameter is any parameter that is not of type `var`. +The `quirky` pragma can also be be pushed in order to affect a group of routines and whether +the compiler supports the pragma can be checked with `defined(nimHasQuirky)`: -1. Two output parameters should never be aliased. -2. An input and an output parameter should not be aliased. -3. An output parameter should never be aliased with a global or thread local - variable referenced by the called proc. -4. An input parameter should not be aliased with a global or thread local - variable updated by the called proc. + ```nim + when defined(nimHasQuirky): + {.push quirky: on.} -One problem with rules 3 and 4 is that they affect specific global or thread -local variables, but Nim's effect tracking only tracks "uses no global variable" -via `.noSideEffect`. The rules 3 and 4 can also be approximated by a different rule: + proc doRaise() = raise newException(ValueError, "") -5. A global or thread local variable (or a location derived from such a location) - can only passed to a parameter of a `.noSideEffect` proc. + proc f(): string = "abc" + proc q(cond: bool) = + if cond: + doRaise() + echo f() -Noalias annotation -================== + q(true) -Since version 1.4 of the Nim compiler, there is a `.noalias` annotation for variables -and parameters. It is mapped directly to C/C++'s `restrict` keyword and means that -the underlying pointer is pointing to a unique location in memory, no other aliases to -this location exist. It is *unchecked* that this alias restriction is followed, if the -restriction is violated, the backend optimizer is free to miscompile the code. -This is an **unsafe** language feature. + when defined(nimHasQuirky): + {.pop.} + ``` -Ideally in later versions of the language, the restriction will be enforced at -compile time. (Which is also why the name `noalias` was choosen instead of a more -verbose name like `unsafeAssumeNoAlias`.) +**Warning**: The `quirky` pragma only affects code generation, no check for validity is performed! -Strict funcs -============ +Threading under ARC/ORC +======================= -Since version 1.4 a stricter definition of "side effect" is available. In addition -to the existing rule that a side effect is calling a function with side effects -the following rule is also enforced: +ARC/ORC supports a shared heap out of the box. This means that messages can be sent between +threads without copies. However, without copying the data there is an inherent danger of +data races. Data races are prevented at compile-time if it is enforced that +only **isolated** subgraphs can be sent around. -Any mutation to an object does count as a side effect if that object is reachable -via a parameter that is not declared as a `var` parameter. -For example: +Isolation +--------- -.. code-block:: nim +The standard library module `isolation.nim` provides a generic type `Isolated[T]` that +captures the important notion that nothing else can reference the graph that is wrapped +inside `Isolated[T]`. It is what a channel implementation should use in order to enforce +the freedom of data races: - {.experimental: "strictFuncs".} + ```nim + proc send*[T](c: var Channel[T]; msg: sink Isolated[T]) + proc recv*[T](c: var Channel[T]): T + ## Note: Returns T, not Isolated[T] for convenience. - type - Node = ref object - le, ri: Node - data: string + proc recvIso*[T](c: var Channel[T]): Isolated[T] + ## remembers the data is Isolated[T]. + ``` - func len(n: Node): int = - # valid: len does not have side effects - var it = n - while it != nil: - inc result - it = it.ri +In order to create an `Isolated` graph one has to use either `isolate` or `unsafeIsolate`. +`unsafeIsolate` is as its name says unsafe and no checking is performed. It should be considered +to be as dangerous as a `cast` operation. - func mut(n: Node) = - let m = n # is the statement that connected the mutation to the parameter - m.data = "yeah" # the mutation is here - # Error: 'mut' can have side effects - # an object reachable from 'n' is potentially mutated +Construction must ensure that the invariant holds, namely that the wrapped `T` +is free of external aliases into it. `isolate` ensures this invariant. It is +inspired by Pony's `recover` construct: -The algorithm behind this analysis is described in -the `view types section <#view-types-algorithm>`_. + ```nim + func isolate(x: sink T): Isolated[T] {.magic: "Isolate".} + ``` -View types -========== +As you can see, this is a new builtin because the check it performs on `x` is non-trivial: -**Note**: `--experimental:views` is more effective -with `--experimental:strictFuncs`. +If `T` does not contain a `ref` or `closure` type, it is isolated. Else the syntactic +structure of `x` is analyzed: -A view type is a type that is or contains one of the following types: +- Literals like `nil`, `4`, `"abc"` are isolated. +- A local variable or a routine parameter is isolated if either of these conditions is true: + 1. Its type is annotated with the `.sendable` pragma. Note `Isolated[T]` is annotated as + `.sendable`. + 2. Its type contains the potentially dangerous `ref` and `proc {.closure}` types + only in places that are protected via a `.sendable` container. -- `var T` (mutable view into `T`) -- `lent T` (immutable view into `T`) -- `openArray[T]` (pair of (pointer to array of `T`, size)) +- An array constructor `[x...]` is isolated if every element `x` is isolated. +- An object constructor `Obj(x...)` is isolated if every element `x` is isolated. +- An `if` or `case` expression is isolated if all possible values the expression + may return are isolated. +- A type conversion `C(x)` is isolated if `x` is isolated. Analogous for `cast` + expressions. +- A function call `f(x...)` is isolated if `f` is `.noSideEffect` and for every argument `x`: + - `x` is isolated **or** + - `f`'s return type cannot *alias* `x`'s type. This is checked via a form of alias analysis as explained in the next paragraph. -For example: -.. code-block:: nim - type - View1 = var int - View2 = openArray[byte] - View3 = lent string - View4 = Table[openArray[char], int] +Alias analysis +-------------- + +We start with an important, simple case that must be valid: Sending the result +of `parseJson` to a channel. Since the signature +is `func parseJson(input: string): JsonNode` it is easy to see that JsonNode +can never simply be a view into `input` which is a `string`. + +A different case is the identity function `id`, `send id(myJsonGraph)` must be +invalid because we do not know how many aliases into `myJsonGraph` exist +elsewhere. + +In general type `A` can alias type `T` if: + +- `A` and `T` are the same types. +- `A` is a distinct type derived from `T`. +- `A` is a field inside `T` if `T` is a final object type. +- `T` is an inheritable object type. (An inherited type could always contain + a `field: A`). +- `T` is a closure type. Reason: `T`'s environment can contain a field of + type `A`. +- `A` is the element type of `T` if `T` is an array, sequence or pointer type. -Exceptions to this rule are types constructed via `ptr` or `proc`. -For example, the following types are **not** view types: -.. code-block:: nim +Sendable pragma +--------------- + +A container type can be marked as `.sendable`. `.sendable` declares that the type +encapsulates a `ref` type effectively so that a variable of this container type +can be used in an `isolate` context: + + ```nim type - NotView1 = proc (x: openArray[int]) - NotView2 = ptr openArray[char] - NotView3 = ptr array[4, var int] + Isolated*[T] {.sendable.} = object ## Isolated data can only be moved, not copied. + value: T + proc `=copy`*[T](dest: var Isolated[T]; src: Isolated[T]) {.error.} -A *mutable* view type is a type that is or contains a `var T` type. -An *immutable* view type is a view type that is not a mutable view type. + proc `=sink`*[T](dest: var Isolated[T]; src: Isolated[T]) {.inline.} = + # delegate to value's sink operation + `=sink`(dest.value, src.value) -A *view* is a symbol (a let, var, const, etc.) that has a view type. + proc `=destroy`*[T](dest: var Isolated[T]) {.inline.} = + # delegate to value's destroy operation + `=destroy`(dest.value) + ``` -Since version 1.4 Nim allows view types to be used as local variables. -This feature needs to be enabled via `{.experimental: "views".}`. +The `.sendable` pragma itself is an experimenal, unchecked, unsafe annotation. It is +currently only used by `Isolated[T]`. -A local variable of a view type *borrows* from the locations and -it is statically enforced that the view does not outlive the location -it was borrowed from. +Virtual pragma +============== + +`virtual` is designed to extend or create virtual functions when targeting the cpp backend. When a proc is marked with virtual, it forward declares the proc header within the type's body. + +Here's an example of how to use the virtual pragma: + +```nim +proc newCpp*[T](): ptr T {.importcpp: "new '*0()".} +type + Foo = object of RootObj + FooPtr = ptr Foo + Boo = object of Foo + BooPtr = ptr Boo + +proc salute(self: FooPtr) {.virtual.} = + echo "hello foo" + +proc salute(self: BooPtr) {.virtual.} = + echo "hello boo" + +let foo = newCpp[Foo]() +let boo = newCpp[Boo]() +let booAsFoo = cast[FooPtr](newCpp[Boo]()) + +foo.salute() # prints hello foo +boo.salute() # prints hello boo +booAsFoo.salute() # prints hello boo +``` +In this example, the `salute` function is virtual in both Foo and Boo types. This allows for polymorphism. + +The virtual pragma also supports a special syntax to express Cpp constraints. Here's how it works: + +`$1` refers to the function name +`'idx` refers to the type of the argument at the position idx. Where idx = 1 is the `this` argument. +`#idx` refers to the argument name. + +The return type can be referred to as `-> '0`, but this is optional and often not needed. + + ```nim + {.emit:"""/*TYPESECTION*/ +#include <iostream> + class CppPrinter { + public: + + virtual void printConst(char* message) const { + std::cout << "Const Message: " << message << std::endl; + } + virtual void printConstRef(char* message, const int& flag) const { + std::cout << "Const Ref Message: " << message << std::endl; + } +}; +""".} + +type + CppPrinter {.importcpp, inheritable.} = object + NimPrinter {.exportc.} = object of CppPrinter + +proc printConst(self: CppPrinter; message:cstring) {.importcpp.} +CppPrinter().printConst(message) + +# override is optional. +proc printConst(self: NimPrinter; message: cstring) {.virtual: "$1('2 #2) const override".} = + echo "NimPrinter: " & $message + +proc printConstRef(self: NimPrinter; message: cstring; flag:int32) {.virtual: "$1('2 #2, const '3& #3 ) const override".} = + echo "NimPrinterConstRef: " & $message + +NimPrinter().printConst(message) +var val: int32 = 10 +NimPrinter().printConstRef(message, val) + +``` + +Constructor pragma +================== + +The `constructor` pragma can be used in two ways: in conjunction with `importcpp` to import a C++ constructor, and to declare constructors that operate similarly to `virtual`. + +Consider: + +```nim +type Foo* = object + x: int32 + +proc makeFoo(x: int32): Foo {.constructor.} = + result.x = x +``` + +It forward declares the constructor in the type definition. When the constructor has parameters, it also generates a default constructor. One can avoid this behaviour by using `noDecl` in a default constructor. + +Like `virtual`, `constructor` also supports a syntax that allows to express C++ constraints. For example: -.. code-block:: nim +```nim +{.emit:"""/*TYPESECTION*/ +struct CppClass { + int x; + int y; + CppClass(int inX, int inY) { + this->x = inX; + this->y = inY; + } + //CppClass() = default; +}; +""".} + +type + CppClass* {.importcpp, inheritable.} = object + x: int32 + y: int32 + NimClass* = object of CppClass + +proc makeNimClass(x: int32): NimClass {.constructor:"NimClass('1 #1) : CppClass(0, #1)".} = + result.x = x + +# Optional: define the default constructor explicitly +proc makeCppClass(): NimClass {.constructor: "NimClass() : CppClass(0, 0)".} = + result.x = 1 +``` + +In the example above `CppClass` has a deleted default constructor. Notice how by using the constructor syntax, one can call the appropriate constructor. + +Notice when calling a constructor in the section of a global variable initialization, it will be called before `NimMain` meaning Nim is not fully initialized. + +Constructor Initializer +======================= - {.experimental: "views".} +By default Nim initializes `importcpp` types with `{}`. This can be problematic when importing +types with a deleted default constructor. In order to avoid this, one can specify default values for a constructor by specifying default values for the proc params in the `constructor` proc. - proc take(a: openArray[int]) = - echo a.len +For example: - proc main(s: seq[int]) = - var x: openArray[int] = s # 'x' is a view into 's' - # it is checked that 'x' does not outlive 's' and - # that 's' is not mutated. - for i in 0 .. high(x): - echo x[i] - take(x) +```nim - take(x.toOpenArray(0, 1)) # slicing remains possible - let y = x # create a view from a view - take y - # it is checked that 'y' does not outlive 'x' and - # that 'x' is not mutated as long as 'y' lives. +{.emit: """/*TYPESECTION*/ +struct CppStruct { + CppStruct(int x, char* y): x(x), y(y){} + int x; + char* y; +}; +""".} +type + CppStruct {.importcpp, inheritable.} = object +proc makeCppStruct(a: cint = 5, b:cstring = "hello"): CppStruct {.importcpp: "CppStruct(@)", constructor.} - main(@[11, 22, 33]) +(proc (s: CppStruct) = echo "hello")(makeCppStruct()) +# If one removes a default value from the constructor and passes it to the call explicitly, the C++ compiler will complain. +``` +Skip initializers in fields members +=================================== -A local variable of a view type can borrow from a location -derived from a parameter, another local variable, a global `const` or `let` -symbol or a thread-local `var` or `let`. +By using `noInit` in a type or field declaration, the compiler will skip the initializer. By doing so one can explicitly initialize those values in the constructor of the type owner. -Let `p` the proc that is analysed for the correctness of the borrow operation. +For example: -Let `source` be one of: +```nim -- A formal parameter of `p`. Note that this does not cover parameters of - inner procs. -- The `result` symbol of `p`. -- A local `var` or `let` or `const` of `p`. Note that this does - not cover locals of inner procs. -- A thread-local `var` or `let`. -- A global `let` or `const`. -- A constant array/seq/object/tuple constructor. +{.emit: """/*TYPESECTION*/ + struct Foo { + Foo(int a){}; + }; + struct Boo { + Boo(int a){}; + }; + """.} -Path expressions ----------------- +type + Foo {.importcpp.} = object + Boo {.importcpp, noInit.} = object + Test {.exportc.} = object + foo {.noInit.}: Foo + boo: Boo -A location derived from `source` is then defined as a path expression that -has `source` as the owner. A path expression `e` is defined recursively: +proc makeTest(): Test {.constructor: "Test() : foo(10), boo(1)".} = + discard -- `source` itself is a path expression. -- Container access like `e[i]` is a path expression. -- Tuple access `e[0]` is a path expression. -- Object field access `e.field` is a path expression. -- `system.toOpenArray(e, ...)` is a path expression. -- Pointer dereference `e[]` is a path expression. -- An address `addr e`, `unsafeAddr e` is a path expression. -- A type conversion `T(e)` is a path expression. -- A cast expression `cast[T](e)` is a path expression. -- `f(e, ...)` is a path expression if `f`'s return type is a view type. - Because the view can only have been borrowed from `e`, we then know - that owner of `f(e, ...)` is `e`. +proc main() = + var t = makeTest() +main() -If a view type is used as a return type, the location must borrow from a location -that is derived from the first parameter that is passed to the proc. -See https://nim-lang.org/docs/manual.html#procedures-var-return-type for -details about how this is done for `var T`. +``` -A mutable view can borrow from a mutable location, an immutable view can borrow -from both a mutable or an immutable location. +Will produce: -The *duration* of a borrow is the span of commands beginning from the assignment -to the view and ending with the last usage of the view. +```cpp -For the duration of the borrow operation, no mutations to the borrowed locations -may be performed except via the potentially mutable view that borrowed from the -location. The borrowed location is said to be *sealed* during the borrow. +struct Test { + Foo foo; + Boo boo; + N_LIB_PRIVATE N_NOCONV(, Test)(void); +}; -.. code-block:: nim +``` - {.experimental: "views".} +Notice that without `noInit` it would produce `Foo foo {}` and `Boo boo {}` - type - Obj = object - field: string - proc dangerous(s: var seq[Obj]) = - let v: lent Obj = s[0] # seal 's' - s.setLen 0 # prevented at compile-time because 's' is sealed. - echo v.field +Member pragma +============= +Like the `constructor` and `virtual` pragmas, the `member` pragma can be used to attach a procedure to a C++ type. It's more flexible than the `virtual` pragma in the sense that it accepts not only names but also operators and destructors. -The scope of the view does not matter: +For example: -.. code-block:: nim +```nim +proc print(s: cstring) {.importcpp: "printf(@)", header: "<stdio.h>".} - proc valid(s: var seq[Obj]) = - let v: lent Obj = s[0] # begin of borrow - echo v.field # end of borrow - s.setLen 0 # valid because 'v' isn't used afterwards +type + Doo {.exportc.} = object + test: int +proc memberProc(f: Doo) {.member.} = + echo $f.test -The analysis requires as much precision about mutations as is reasonably obtainable, -so it is more effective with the experimental `strict funcs <#strict-funcs>`_ -feature. In other words `--experimental:views` works better -with `--experimental:strictFuncs`. +proc destructor(f: Doo) {.member: "~'1()", used.} = + print "destructing\n" -The analysis is currently control flow insensitive: +proc `==`(self, other: Doo): bool {.member: "operator==('2 const & #2) const -> '0".} = + self.test == other.test -.. code-block:: nim +let doo = Doo(test: 2) +doo.memberProc() +echo doo == Doo(test: 1) - proc invalid(s: var seq[Obj]) = - let v: lent Obj = s[0] - if false: - s.setLen 0 - echo v.field +``` -In this example, the compiler assumes that `s.setLen 0` invalidates the -borrow operation of `v` even though a human being can easily see that it -will never do that at runtime. +Will print: +``` +2 +false +destructing +destructing +``` +Notice how the C++ destructor is called automatically. Also notice the double implementation of `==` as an operator in Nim but also in C++. This is useful if you need the type to match some C++ `concept` or `trait` when interoping. -Start of a borrow ------------------ +A side effect of being able to declare C++ operators, is that you can now also create a +C++ functor to have seamless interop with C++ lambdas (syntactic sugar for functors). -A borrow starts with one of the following: +For example: -- The assignment of a non-view-type to a view-type. -- The assignment of a location that is derived from a local parameter - to a view-type. +```nim +type + NimFunctor = object + discard +proc invoke(f: NimFunctor; n: int) {.member: "operator ()('2 #2)".} = + echo "FunctorSupport!" + +{.experimental: "callOperator".} +proc `()`(f: NimFunctor; n:int) {.importcpp: "#(@)" .} +NimFunctor()(1) +``` +Notice we use the overload of `()` to have the same semantics in Nim, but on the `importcpp` we import the functor as a function. +This allows to easy interop with functions that accepts for example a `const` operator in its signature. + + +Injected symbols in generic procs and templates +=============================================== + +With the experimental option `openSym`, captured symbols in generic routine and +template bodies may be replaced by symbols injected locally by templates/macros +at instantiation time. `bind` may be used to keep the captured symbols over the +injected ones regardless of enabling the options, but other methods like +renaming the captured symbols should be used instead so that the code is not +affected by context changes. + +Since this change may affect runtime behavior, the experimental switch +`openSym` needs to be enabled; and a warning is given in the case where an +injected symbol would replace a captured symbol not bound by `bind` and +the experimental switch isn't enabled. + +```nim +const value = "captured" +template foo(x: int, body: untyped): untyped = + let value {.inject.} = "injected" + body + +proc old[T](): string = + foo(123): + return value # warning: a new `value` has been injected, use `bind` or turn on `experimental:openSym` +echo old[int]() # "captured" + +template oldTempl(): string = + block: + foo(123): + value # warning: a new `value` has been injected, use `bind` or turn on `experimental:openSym` +echo oldTempl() # "captured" +{.experimental: "openSym".} -End of a borrow ---------------- +proc bar[T](): string = + foo(123): + return value +assert bar[int]() == "injected" # previously it would be "captured" -A borrow operation ends with the last usage of the view variable. +proc baz[T](): string = + bind value + foo(123): + return value +assert baz[int]() == "captured" +template barTempl(): string = + block: + foo(123): + value +assert barTempl() == "injected" # previously it would be "captured" -Reborrows ---------- +template bazTempl(): string = + bind value + block: + foo(123): + value +assert bazTempl() == "captured" +``` + +This option also generates a new node kind `nnkOpenSym` which contains +exactly 1 `nnkSym` node. In the future this might be merged with a slightly +modified `nnkOpenSymChoice` node but macros that want to support the +experimental feature should still handle `nnkOpenSym`, as the node kind would +simply not be generated as opposed to being removed. + +Another experimental switch `genericsOpenSym` exists that enables this behavior +at instantiation time, meaning templates etc can enable it specifically when +they are being called. However this does not generate `nnkOpenSym` nodes +(unless the other switch is enabled) and so doesn't reflect the regular +behavior of the switch. + +```nim +const value = "captured" +template foo(x: int, body: untyped): untyped = + let value {.inject.} = "injected" + {.push experimental: "genericsOpenSym".} + body + {.pop.} + +proc bar[T](): string = + foo(123): + return value +echo bar[int]() # "injected" + +template barTempl(): string = + block: + var res: string + foo(123): + res = value + res +assert barTempl() == "injected" +``` -A view `v` can borrow from multiple different locations. However, the borrow -is always the full span of `v`'s lifetime and every location that is borrowed -from is sealed during `v`'s lifetime. +VTable for methods +================== -Algorithm ---------- +Methods now support implementations based on a VTable by using `--experimental:vtables`. Note that the option needs to enabled +globally. The virtual method table is stored in the type info of +an object, which is an array of function pointers. -The following section is an outline of the algorithm that the current implementation -uses. The algorithm performs two traversals over the AST of the procedure or global -section of code that uses a view variable. No fixpoint iterations are performed, the -complexity of the analysis is O(N) where N is the number of nodes of the AST. +```nim +method foo(x: Base, ...) {.base.} +method foo(x: Derived, ...) {.base.} +``` -The first pass over the AST computes the lifetime of each local variable based on -a notion of an "abstract time", in the implementation it's a simple integer that is -incremented for every visited node. +It roughly generates a dispatcher like -In the second pass information about the underlying object "graphs" is computed. -Let `v` be a parameter or a local variable. Let `G(v)` be the graph -that `v` belongs to. A graph is defined by the set of variables that belong -to the graph. Initially for all `v`: `G(v) = {v}`. Every variable can only -be part of a single graph. +```nim +proc foo_dispatch(x: Base, ...) = + x.typeinfo.vtable[method_index](x, ...) # method_index is the index of the sorted order of a method +``` -Assignments like `a = b` "connect" two variables, both variables end up in the -same graph `{a, b} = G(a) = G(b)`. Unfortunately, the pattern to look for is -much more complex than that and can involve multiple assignment targets -and sources:: +Methods are required to be in the same module where their type has been defined. - f(x, y) = g(a, b) +```nim +# types.nim +type + Base* = ref object +``` -connects `x` and `y` to `a` and `b`: `G(x) = G(y) = G(a) = G(b) = {x, y, a, b}`. -A type based alias analysis rules out some of these combinations, for example -a `string` value cannot possibly be connected to a `seq[int]`. +```nim +import types -A pattern like `v[] = value` or `v.field = value` marks `G(v)` as mutated. -After the second pass a set of disjoint graphs was computed. +method foo(x: Base) {.base.} = discard +``` -For strict functions it is then enforced that there is no graph that is both mutated -and has an element that is an immutable parameter (that is a parameter that is not -of type `var T`). +It gives an error: method `foo` can be defined only in the same module with its type (Base). -For borrow checking a different set of checks is performed. Let `v` be the view -and `b` the location that is borrowed from. -- The lifetime of `v` must not exceed `b`'s lifetime. Note: The lifetime of - a parameter is the complete proc body. -- If `v` is a mutable view and `v` is used to actually mutate the - borrowed location, then `b` has to be a mutable location. - Note: If it is not actually used for mutation, borrowing a mutable view from an - immutable location is allowed! This allows for many important idioms and will be - justified in an upcoming RFC. -- During `v`'s lifetime, `G(b)` can only be modified by `v` (and only if - `v` is a mutable view). -- If `v` is `result` then `b` has to be a location derived from the first - formal parameter or from a constant location. -- A view cannot be used for a read or a write access before it was assigned to. +asmSyntax pragma +================ + +The `asmSyntax` pragma is used to specify target inline assembler syntax in an `asm` statement. + +It prevents compiling code with different of the target CC inline asm syntax, i.e. it will not allow gcc inline asm code to be compiled with vcc. + +```nim +proc nothing() = + asm {.asmSyntax: "gcc".}""" + nop + """ +``` + +The current C(C++) backend implementation cannot generate code for gcc and for vcc at the same time. For example, `{.asmSyntax: "vcc".}` with the ICC compiler will not generate code with intel asm syntax, even though ICC can use both gcc-like and vcc-like asm. diff --git a/doc/manual_experimental_strictnotnil.rst b/doc/manual_experimental_strictnotnil.md index c7c045683..a6fa6cda8 100644 --- a/doc/manual_experimental_strictnotnil.rst +++ b/doc/manual_experimental_strictnotnil.md @@ -1,17 +1,20 @@ -.. default-role:: code - Strict not nil checking ========================= +.. default-role:: code +.. include:: rstcommon.rst + **Note:** This feature is experimental, you need to enable it with -.. code-block:: nim + ```nim {.experimental: "strictNotNil".} + ``` or -.. code-block:: bash + ```cmd nim c --experimental:strictNotNil <program> + ``` In the second case it would check builtin and imported modules as well. @@ -38,7 +41,7 @@ not nil You can annotate a type where nil isn't a valid value with `not nil`. -.. code-block:: nim + ```nim type NilableObject = ref object a: int @@ -56,32 +59,40 @@ You can annotate a type where nil isn't a valid value with `not nil`. p(x) else: p(x) # ok + ``` If a type can include `nil` as a valid value, dereferencing values of the type -is checked by the compiler: if a value which might be nil is derefenced, this produces a warning by default, you can turn this into an error using the compiler options `--warningAsError:strictNotNil` +is checked by the compiler: if a value which might be nil is dereferenced, this +produces a warning by default, you can turn this into an error using +the compiler options `--warningAsError:strictNotNil`:option:. If a type is nilable, you should dereference its values only after a `isNil` or equivalent check. local turn on/off --------------------- -You can still turn off nil checking on function/module level by using a `{.strictNotNil: off}.` pragma. -Note: test that/TODO for code/manual. +You can still turn off nil checking on function/module level by using a `{.strictNotNil: off.}` pragma. + +.. + Note: test that/TODO for code/manual. nilability state ----------------- -Currently a nilable value can be `Safe`, `MaybeNil` or `Nil` : we use internally `Parent` and `Unreachable` but this is an implementation detail(a parent layer has the actual nilability). - -`Safe` means it shouldn't be nil at that point: e.g. after assignment to a non-nil value or `not a.isNil` check -`MaybeNil` means it might be nil, but it might not be nil: e.g. an argument, a call argument or a value after an `if` and `else`. -`Nil` means it should be nil at that point; e.g. after an assignment to `nil` or a `.isNil` check. +Currently, a nilable value can be `Safe`, `MaybeNil` or `Nil` : we use internally `Parent` and `Unreachable` but this is an implementation detail(a parent layer has the actual nilability). -`Unreachable` means it shouldn't be possible to access this in this branch: so we do generate a warning as well. +- `Safe` means it shouldn't be nil at that point: e.g. after assignment to + a non-nil value or `not a.isNil` check +- `MaybeNil` means it might be nil, but it might not be nil: e.g. an argument, + a call argument or a value after an `if` and `else`. +- `Nil` means it should be nil at that point; e.g. after an assignment to + `nil` or a `.isNil` check. +- `Unreachable` means it shouldn't be possible to access this in this branch: + so we do generate a warning as well. -We show an error for each dereference (`[]`, `.field`, `[index]` `()` etc) which is of a tracked expression which is +We show an error for each dereference (`[]`, `.field`, `[index]` `()` etc.) which is of a tracked expression which is in `MaybeNil` or `Nil` state. @@ -92,7 +103,8 @@ Types are either nilable or non-nilable. When you pass a param or a default value, we use the type : for nilable types we return `MaybeNil` and for non-nilable `Safe`. -TODO: fix the manual here. (This is not great, as default values for non-nilables and nilables are usually actually `nil` , so we should think a bit more about this section.) +.. + TODO: fix the manual here. (This is not great, as default values for non-nilables and nilables are usually actually `nil` , so we should think a bit more about this section.) params rules ------------ @@ -112,38 +124,41 @@ call args rules When we call with arguments, we have two cases when we might change the nilability. -.. code-block:: nim + ```nim callByVar(a) + ``` Here `callByVar` can re-assign `a`, so this might change `a`'s nilability, so we change it to `MaybeNil`. This is also a possible aliasing `move out` (moving out of a current alias set). -.. code-block:: nim + ```nim call(a) + ``` -Here `call` can change a field or element of `a`, so if we have a dependant expression of `a` : e.g. `a.field`. Dependats become `MaybeNil`. +Here `call` can change a field or element of `a`, so if we have a dependant expression of `a` : e.g. `a.field`. Dependants become `MaybeNil`. branches rules --------------- Branches are the reason we do nil checking like this: with flow checking. -Sources of brancing are `if`, `while`, `for`, `and`, `or`, `case`, `try` and combinations with `return`, `break`, `continue` and `raise` +Sources of branching are `if`, `while`, `for`, `and`, `or`, `case`, `try` and combinations with `return`, `break`, `continue` and `raise` We create a new layer/"scope" for each branch where we map expressions to nilability. This happens when we "fork": usually on the beginning of a construct. When branches "join" we usually unify their expression maps or/and nilabilities. Merging usually merges maps and alias sets: nilabilities are merged like this: -.. code-block:: nim + ```nim template union(l: Nilability, r: Nilability): Nilability = ## unify two states if l == r: l else: MaybeNil + ``` -Special handling is for `.isNil` and ` == nil`, also for `not`, `and` and `or`. +Special handling is for `.isNil` and `== nil`, also for `not`, `and` and `or`. `not` reverses the nilability, `and` is similar to "forking" : the right expression is checked in the layer resulting from the left one and `or` is similar to "merging": the right and left expression should be both checked in the original layer. @@ -158,7 +173,7 @@ We want to track also field(dot) and index(bracket) expressions. We track some of those compound expressions which might be nilable as dependants of their bases: `a.field` is changed if `a` is moved (re-assigned), similarly `a[index]` is dependent on `a` and `a.field.field` on `a.field`. -When we move the base, we update dependants to `MaybeNil`. Otherwise we usually start with type nilability. +When we move the base, we update dependants to `MaybeNil`. Otherwise, we usually start with type nilability. When we call args, we update the nilability of their dependants to `MaybeNil` as the calls usually can change them. We might need to check for `strictFuncs` pure funcs and not do that then. @@ -166,10 +181,10 @@ We might need to check for `strictFuncs` pure funcs and not do that then. For field expressions `a.field`, we calculate an integer value based on a hash of the tree and just accept equivalent trees as equivalent expressions. For item expression `a[index]`, we also calculate an integer value based on a hash of the tree and accept equivalent trees as equivalent expressions: for static values only. -For now we support only constant indices: we dont track expression with no-const indices. For those we just report a warning even if they are safe for now: one can use a local variable to workaround. For loops this might be annoying: so one should be able to turn off locally the warning using the `{.warning[StrictCheckNotNil]:off}.`. +For now, we support only constant indices: we don't track expression with no-const indices. For those we just report a warning even if they are safe for now: one can use a local variable to workaround. For loops this might be annoying: so one should be able to turn off locally the warning using the `{.warning[StrictNotNil]:off.}`. For bracket expressions, in the future we might count `a[<any>]` as the same general expression. -This means we should should the index but otherwise handle it the same for assign (maybe "aliasing" all the non-static elements) and differentiate only for static: e.g. `a[0]` and `a[1]`. +This means we should the index but otherwise handle it the same for assign (maybe "aliasing" all the non-static elements) and differentiate only for static: e.g. `a[0]` and `a[1]`. element tracking ----------------- @@ -177,8 +192,9 @@ element tracking When we assign an object construction, we should track the fields as well: -.. code-block:: nim + ```nim var a = Nilable(field: Nilable()) # a : Safe, a.field: Safe + ``` Usually we just track the result of an expression: probably this should apply for elements in other cases as well. Also related to tracking initialization of expressions/fields. @@ -187,14 +203,15 @@ unstructured control flow rules ------------------------------- Unstructured control flow keywords as `return`, `break`, `continue`, `raise` mean that we jump from a branch out. -This means that if there is code after the finishing of the branch, it would be ran if one hasn't hit the direct parent branch of those: so it is similar to an `else`. In those cases we should use the reverse nilabilities for the local to the condition expressions. E.g. +This means that if there is code after the finishing of the branch, it would be run if one hasn't hit the direct parent branch of those: so it is similar to an `else`. In those cases we should use the reverse nilabilities for the local to the condition expressions. E.g. -.. code-block:: nim + ```nim for a in c: if not a.isNil: b() break code # here a: Nil , because if not, we would have breaked + ``` aliasing @@ -208,28 +225,30 @@ Assignments and other changes to nilability can move / move out expressions of s `move`: Moving `left` to `right` means we remove `left` from its current set and unify it with the `right`'s set. This means it stops being aliased with its previous aliases. -.. code-block:: nim + ```nim var left = b left = right # moving left to right + ``` `move out`: Moving out `left` might remove it from the current set and ensure that it's in its own set as a single element. e.g. -.. code-block:: nim + ```nim var left = b left = nil # moving out + ``` +.. + initialization of non nilable and nilable values + ------------------------------------------------- -initialization of non nilable and nilable values -------------------------------------------------- - -TODO + TODO warnings and errors --------------------- -We show an error for each dereference (`[]`, `.field`, `[index]` `()` etc) which is of a tracked expression which is +We show an error for each dereference (`[]`, `.field`, `[index]` `()` etc.) which is of a tracked expression which is in `MaybeNil` or `Nil` state. We might also show a history of the transitions and the reasons for them that might change the nilability of the expression. diff --git a/doc/markdown_rst.md b/doc/markdown_rst.md new file mode 100644 index 000000000..c7977f75a --- /dev/null +++ b/doc/markdown_rst.md @@ -0,0 +1,349 @@ +========================================== +Nim-flavored Markdown and reStructuredText +========================================== + +:Author: Andrey Makarov +:Version: |nimversion| + +.. default-role:: code +.. include:: rstcommon.rst +.. contents:: + +.. importdoc:: docgen.md + +Both `Markdown`:idx: (md) and `reStructuredText`:idx: (RST) are markup +languages whose goal is to typeset texts with complex structure, +formatting and references using simple plaintext representation. + +Command line usage +================== + +Usage (to convert Markdown into HTML): + + ```cmd + nim md2html markdown_rst.md + ``` + +Output: + + You're reading it! + +The `md2tex`:option: command is invoked identically to `md2html`:option:, +but outputs a ``.tex`` file instead of ``.html``. + +These tools embedded into Nim compiler; the compiler can output +the result to HTML [^html] or Latex [^latex]. + +[^html]: commands `nim doc`:cmd: for ``*.nim`` files and + `nim rst2html`:cmd: for ``*.rst`` files + +[^latex]: commands `nim doc2tex`:cmd: for ``*.nim`` and + `nim rst2tex`:cmd: for ``*.rst``. + +Full list of supported commands: + +=================== ====================== ============ ============== +command runs on... input format output format +=================== ====================== ============ ============== +`nim md2html`:cmd: standalone md files ``.md`` ``.html`` HTML +`nim md2tex`:cmd: same same ``.tex`` LaTeX +`nim rst2html`:cmd: standalone rst files ``.rst`` ``.html`` HTML +`nim rst2tex`:cmd: same same ``.tex`` LaTeX +`nim doc`:cmd: documentation comments ``.nim`` ``.html`` HTML +`nim doc2tex`:cmd: same same ``.tex`` LaTeX +`nim jsondoc`:cmd: same same ``.json`` JSON +=================== ====================== ============ ============== + + +Basic markup +============ + +If you are new to Markdown/RST please consider reading the following: + +1) [Markdown Basic Syntax] +2) a long specification of Markdown: [CommonMark Spec] +3) a short [quick introduction] to RST +4) an [RST reference]: a comprehensive cheatsheet for RST +5) a more formal 50-page [RST specification]. + +Features +-------- + +A large subset is implemented with some [limitations] and +[additional Nim-specific features]. + +Supported common RST/Markdown features: + +* body elements + + sections + + transitions + + paragraphs + + bullet lists using \+, \*, \- + + enumerated lists using arabic numerals or alphabet + characters: 1. ... 2. ... *or* a. ... b. ... *or* A. ... B. ... + + footnotes (including manually numbered, auto-numbered, auto-numbered + with label, and auto-symbol footnotes) and citations + + field lists + + option lists + + quoted literal blocks + + line blocks + + simple tables + + directives (see official documentation in [RST directives list]): + - ``image``, ``figure`` for including images and videos + - ``code`` + - ``contents`` (table of contents), ``container``, ``raw`` + - ``include`` + - admonitions: "attention", "caution", "danger", "error", "hint", + "important", "note", "tip", "warning", "admonition" + - substitution definitions: `replace` and `image` + + comments +* inline markup + + *emphasis*, **strong emphasis**, + ``inline literals``, hyperlink references (including embedded URI), + substitution references, standalone hyperlinks, + internal links (inline and outline) + + \`interpreted text\` with roles ``:literal:``, ``:strong:``, + ``emphasis``, ``:sub:``/``:subscript:``, ``:sup:``/``:superscript:`` + (see [RST roles list] for description). + + inline internal targets + +RST mode only features +---------------------- + ++ RST syntax for definition lists (that is additional indentation after + a definition line) ++ indented literal blocks starting from ``::`` + +Markdown-specific features +-------------------------- + +* Markdown tables +* Markdown code blocks. For them the same additional arguments as for RST + code blocks can be provided (e.g. `test` or `number-lines`) but with + a one-line syntax like this: + + ```nim test number-lines=10 + echo "ok" + ``` +* Markdown links ``[...](...)`` +* Pandoc syntax for automatic links ``[...]``, see [Referencing] for description +* Pandoc syntax for footnotes, including ``[^10]`` (manually numbered) + and ``[^someName]`` (auto-numbered with a label) +* Markdown literal blocks indented by 4 or more spaces +* Markdown headlines +* Markdown block quotes +* Markdown syntax for definition lists +* using ``1`` as auto-enumerator in enumerated lists like RST ``#`` + (auto-enumerator ``1`` can not be used with ``#`` in the same list) + +Additional Nim-specific features +-------------------------------- + +* referencing to definitions in external files, see + [Markup external referencing] section +* directives: ``code-block`` [^Sphinx], ``title``, + ``index`` [^Sphinx] +* predefined roles + - ``:nim:`` (default), ``:c:`` (C programming language), + ``:python:``, ``:yaml:``, ``:java:``, ``:cpp:`` (C++), ``:csharp`` (C#). + That is every language that [highlite](highlite.html) supports. + They turn on appropriate syntax highlighting in inline code. + + .. Note:: default role for Nim files is ``:nim:``, + for ``*.rst`` it's currently ``:literal:``. + + - generic command line highlighting roles: + - ``:cmd:`` for commands and common shells syntax + - ``:console:`` the same for interactive sessions + (commands should be prepended by ``$``) + - ``:program:`` for executable names [^Sphinx] + (one can just use ``:cmd:`` on single word) + - ``:option:`` for command line options [^Sphinx] + - ``:tok:``, a role for highlighting of programming language tokens +* ***triple emphasis*** (bold and italic) using \*\*\* +* ``:idx:`` role for \`interpreted text\` to include the link to this + text into an index (example: [Nim index]). +* double slash `//` in option lists serves as a prefix for any option that + starts from a word (without any leading symbols like `-`, `--`, `/`): + + //compile compile the project + //doc generate documentation + + Here the dummy `//` will disappear, while options `compile`:option: + and `doc`:option: will be left in the final document. +* emoji / smiley symbols + +[^Sphinx]: similar but different from the directives of + Python [Sphinx directives] and [Sphinx roles] extensions + +.. Note:: By default Nim has ``roSupportMarkdown`` and + ``roSupportRawDirective`` turned **on**. + +.. warning:: Using Nim-specific features can cause other Markdown and + RST implementations to fail on your document. + +Referencing +=========== + +To be able to copy and share links Nim generates anchors for all +main document elements: + +* headlines (including document title) +* footnotes +* explicitly set anchors: RST internal cross-references and + inline internal targets +* Nim symbols (external referencing), see [Nim DocGen Tools Guide] for details. + +But direct use of those anchors have 2 problems: + +1. the anchors are usually mangled (e.g. spaces substituted to minus + signs, etc). +2. manual usage of anchors is not checked, so it's easy to get broken + links inside your project if e.g. spelling has changed for a heading + or you use a wrong relative path to your document. + +That's why Nim implementation has syntax for using +*original* labels for referencing. +Such referencing can be either local/internal or external: + +* Local referencing (inside any given file) is defined by + RST standard or Pandoc Markdown User guide. +* External (cross-document) referencing is a Nim-specific feature, + though it's not really different from local referencing by its syntax. + +Markup local referencing +------------------------ + +There are 2 syntax option available for referencing to objects +inside any given file, e.g. for headlines: + + Markdown RST + + Some headline Some headline + ============= ============= + + Ref. [Some headline] Ref. `Some headline`_ + + +Markup external referencing +--------------------------- + +The syntax is the same as for local referencing, but the anchors are +saved in ``.idx`` files, so one needs to generate them beforehand, +and they should be loaded by an `.. importdoc::` directive. +E.g. if we want to reference section "Some headline" in ``file1.md`` +from ``file2.md``, then ``file2.md`` may look like: + +``` +.. importdoc:: file1.md + +Ref. [Some headline] +``` + +```cmd +nim md2html --index:only file1.md # creates ``htmldocs/file1.idx`` +nim md2html file2.md # creates ``htmldocs/file2.html`` +``` + +To allow cross-references between any files in any order (especially, if +circular references are present), it's strongly recommended +to make a run for creating all the indexes first: + +```cmd +nim md2html --index:only file1.md # creates ``htmldocs/file1.idx`` +nim md2html --index:only file2.md # creates ``htmldocs/file2.idx`` +nim md2html file1.md # creates ``htmldocs/file1.html`` +nim md2html file2.md # creates ``htmldocs/file2.html`` +``` + +and then one can freely reference any objects as if these 2 documents +are actually 1 file. + +Other +===== + +Idiosyncrasies +-------------- + +Currently we do **not** aim at 100% Markdown or RST compatibility in inline +markup recognition rules because that would provide very little user value. +This parser has 2 modes for inline markup: + +1) Markdown-like mode which is enabled by `roPreferMarkdown` option + (turned **on** by default). + + .. Note:: RST features like directives are still turned **on** + +2) Compatibility mode which is RST rules. + +.. Note:: in both modes the parser interpretes text between single + backticks (code) identically: + backslash does not escape; the only exception: ``\`` folowed by ` + does escape so that we can always input a single backtick ` in + inline code. However that makes impossible to input code with + ``\`` at the end in *single* backticks, one must use *double* + backticks: + + `\` -- WRONG + ``\`` -- GOOD + So single backticks can always be input: `\`` will turn to ` code + +.. Attention:: + We don't support some obviously poor design choices of Markdown (or RST). + + - no support for the rule of 2 spaces causing a line break in Markdown + (use RST "line blocks" syntax for making line breaks) + + - interpretation of Markdown block quotes is also slightly different, + e.g. case + + >>> foo + > bar + >>baz + + is a single 3rd-level quote `foo bar baz` in original Markdown, while + in Nim we naturally see it as 3rd-level quote `foo` + 1st level `bar` + + 2nd level `baz`: + + >>> foo + > bar + >>baz + +Limitations +----------- + +* no Unicode support in character width calculations +* body elements + - no roman numerals in enumerated lists + - no doctest blocks + - no grid tables + - some directives are missing (check official [RST directives list]): + ``parsed-literal``, ``sidebar``, ``topic``, ``math``, ``rubric``, + ``epigraph``, ``highlights``, ``pull-quote``, ``compound``, + ``table``, ``csv-table``, ``list-table``, ``section-numbering``, + ``header``, ``footer``, ``meta``, ``class`` + - no ``role`` directives and no custom interpreted text roles + - some standard roles are not supported (check [RST roles list]) + - no generic admonition support +* inline markup + - no simple-inline-markup + - no embedded aliases + +Additional resources +-------------------- + +* See [Nim DocGen Tools Guide](docgen.html) for the details about + `nim doc`:cmd: command and idiosyncrasies of documentation markup in + ``.nim`` files and Nim programming language projects. +* See also documentation for [rst module](rst.html) -- Nim RST/Markdown parser. + +.. _Markdown Basic Syntax: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax +.. _CommonMark Spec: https://spec.commonmark.org/0.30 +.. _quick introduction: https://docutils.sourceforge.io/docs/user/rst/quickstart.html +.. _RST reference: https://docutils.sourceforge.io/docs/user/rst/quickref.html +.. _RST specification: https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html +.. _RST directives list: https://docutils.sourceforge.io/docs/ref/rst/directives.html +.. _RST roles list: https://docutils.sourceforge.io/docs/ref/rst/roles.html +.. _Nim index: https://nim-lang.org/docs/theindex.html +.. _Sphinx directives: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html +.. _Sphinx roles: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html diff --git a/doc/mm.md b/doc/mm.md new file mode 100644 index 000000000..5e0d2f3b9 --- /dev/null +++ b/doc/mm.md @@ -0,0 +1,95 @@ +======================= +Nim's Memory Management +======================= + +.. default-role:: code +.. include:: rstcommon.rst + +:Author: Andreas Rumpf +:Version: |nimversion| + +.. + + +> "The road to hell is paved with good intentions." + + +Multi-paradigm Memory Management Strategies +=========================================== + +.. default-role:: option + +Nim offers multiple different memory management strategies. +To choose the memory management strategy use the `--mm:` switch. + + .. hint:: **The recommended switch for newly written Nim code is `--mm:orc`.** + + +ARC/ORC +------- + +ORC is the default memory management strategy. It is a memory +management mode primarily based on reference counting. Reference cycles are +handled by a cycle collection mechanism based on "trial deletion". +Since algorithms based on "tracing" are not used, the runtime behavior is oblivious to the involved heap and stack sizes. + +The reference counting operations (= "RC ops") do not use atomic instructions and do not have to -- +instead entire subgraphs are *moved* between threads. The Nim compiler also aggressively +optimizes away RC ops and exploits [move semantics](destructors.html#move-semantics). + +Nim performs a fair share of optimizations for ARC/ORC; you can inspect what it did +to your time critical function via `--expandArc:functionName`. Likewise, you can inspect the whole module via `--expandArc:fileName`. + +`--mm:arc` uses the same mechanism as `--mm:orc`, but it leaves out the cycle collector. +Both ARC and ORC offer deterministic performance for `hard realtime`:idx: systems, but +ARC can be easier to reason about for people coming from Ada/C++/C -- roughly speaking +the memory for a variable is freed when it goes "out of scope". + +We generally advise you to use the `acyclic` annotation in order to optimize away the +cycle collector's overhead +but `--mm:orc` also produces more machine code than `--mm:arc`, so if you're on a target +where code size matters and you know that your code does not produce cycles, you can +use `--mm:arc`. Notice that the default `async`:idx: implementation produces cycles +and leaks memory with `--mm:arc`, in other words, for `async` you need to use `--mm:orc`. + + + +Other MM modes +-------------- + +.. note:: The `refc` GC is incremental, thread-local and not "stop-the-world". + +--mm:refc It's a deferred reference counting based garbage collector + with a simple Mark&Sweep backup GC in order to collect cycles. + Heaps are thread-local. [This document](refc.html) contains further information. +--mm:markAndSweep Simple Mark-And-Sweep based garbage collector. + Heaps are thread-local. +--mm:boehm Boehm based garbage collector, it offers a shared heap. +--mm:go Go's garbage collector, useful for interoperability with Go. + Offers a shared heap. + +--mm:none No memory management strategy nor a garbage collector. Allocated memory is + simply never freed. You should use `--mm:arc` instead. + +Here is a comparison of the different memory management modes: + +================== ======== ================= ============== ====== =================== =================== +Memory Management Heap Reference Cycles Stop-The-World Atomic Valgrind compatible Command line switch +================== ======== ================= ============== ====== =================== =================== +ORC Shared Cycle Collector No No Yes `--mm:orc` +ARC Shared Leak No No Yes `--mm:arc` +Atomic ARC Shared Leak No Yes Yes `--mm:atomicArc` +RefC Local Cycle Collector No No No `--mm:refc` +Mark & Sweep Local Cycle Collector No No No `--mm:markAndSweep` +Boehm Shared Cycle Collector Yes No No `--mm:boehm` +Go Shared Cycle Collector Yes No No `--mm:go` +None Manual Manual Manual Manual Manual `--mm:none` +================== ======== ================= ============== ====== =================== =================== + +.. default-role:: code +.. include:: rstcommon.rst + +JavaScript's garbage collector is used for the [JavaScript and NodeJS]( +backends.html#backends-the-javascript-target) compilation targets. +The [NimScript](nims.html) target uses the memory management strategy built into +the Nim compiler. diff --git a/doc/nep1.rst b/doc/nep1.md index 3bae6a00b..3d2a0cef3 100644 --- a/doc/nep1.rst +++ b/doc/nep1.md @@ -1,11 +1,12 @@ -.. default-role:: code +============================ +Standard Library Style Guide +============================ -========================================================== -Nim Enhancement Proposal #1 - Standard Library Style Guide -========================================================== :Author: Clay Sweetser, Dominik Picheta :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: @@ -20,7 +21,7 @@ library should follow. Note that there can be exceptions to these rules. Nim being as flexible as it is, there will be parts of this style guide that don't make sense in certain contexts. Furthermore, just as -`Python's style guide<http://legacy.python.org/dev/peps/pep-0008/>`_ changes +[Python's style guide](http://legacy.python.org/dev/peps/pep-0008/) changes over time, this style guide will too. These rules will only be enforced for contributions to the Nim @@ -48,7 +49,7 @@ Spacing and Whitespace Conventions Not all editors support automatic alignment of code sections, and re-aligning long sections of code by hand can quickly become tedious. - .. code-block:: nim + ```nim # This is bad, as the next time someone comes # to edit this code block, they # must re-align all the assignments again: @@ -59,6 +60,7 @@ Spacing and Whitespace Conventions CalId* = int LongLong* = int64 LongLongPtr* = ptr LongLong + ``` Naming Conventions @@ -68,7 +70,7 @@ Naming Conventions camelCase with the exception of constants which **may** use PascalCase but are not required to. - .. code-block:: nim + ```nim # Constants can start with either a lower case or upper case letter. const aConstant = 42 const FooBar = 4.2 @@ -78,6 +80,7 @@ Naming Conventions # Types must start with an uppercase letter. type FooBar = object + ``` For constants coming from a C/C++ wrapper, ALL_UPPERCASE are allowed, but ugly. (Why shout CONSTANT? Constants do no harm, variables do!) @@ -88,41 +91,45 @@ Naming Conventions that will be used the most, add the suffixes to the pointer variants only. The same applies to C/C++ wrappers. - .. code-block:: nim + ```nim type Handle = object # Will be used most often fd: int64 HandleRef = ref Handle # Will be used less often + ``` - Exception and Error types should have the "Error" or "Defect" suffix. - .. code-block:: nim + ```nim type ValueError = object of CatchableError AssertionDefect = object of Defect Foo = object of Exception # bad style, try to inherit CatchableError or Defect + ``` - Unless marked with the `{.pure.}` pragma, members of enums should have an identifying prefix, such as an abbreviation of the enum's name. - .. code-block:: nim + ```nim type PathComponent = enum pcDir pcLinkToDir pcFile pcLinkToFile + ``` - Non-pure enum values should use camelCase whereas pure enum values should use PascalCase. - .. code-block:: nim + ```nim type PathComponent {.pure.} = enum Dir LinkToDir File LinkToFile + ``` - In the age of HTTP, HTML, FTP, TCP, IP, UTF, WWW it is foolish to pretend these are somewhat special words requiring all uppercase. Instead treat them @@ -139,7 +146,7 @@ Naming Conventions - rotate and rotated - When the 'returns transformed copy' version already exists like `strutils.replace` - an in-place version should get an `-In` suffix (`replaceIn` for this example). + an in-place version should get an ``-In`` suffix (`replaceIn` for this example). - Use `subjectVerb`, not `verbSubject`, e.g.: `fileExists`, not `existsFile`. @@ -152,12 +159,13 @@ The library uses a simple naming scheme that makes use of common abbreviations to keep the names short but meaningful. -------------------- ------------ -------------------------------------- +=================== ============ ====================================== English word To use Notes -------------------- ------------ -------------------------------------- +=================== ============ ====================================== initialize initFoo initializes a value type `Foo` new newFoo initializes a reference type `Foo` - via `new` + via `new` or a value type `Foo` + with reference semantics. this or self self for method like procs, e.g.: `proc fun(self: Foo, a: int)` rationale: `self` is more unique in English @@ -218,22 +226,23 @@ literal lit string str identifier ident indentation indent -------------------- ------------ -------------------------------------- +=================== ============ ====================================== Coding Conventions ------------------ -- The 'return' statement should ideally be used when its control-flow properties - are required. Use a procedure's implicit 'result' variable whenever possible. +- The `return` statement should ideally be used when its control-flow properties + are required. Use a procedure's implicit `result` variable whenever possible. This improves readability. - .. code-block:: nim + ```nim proc repeat(text: string, x: int): string = result = "" for i in 0..x: result.add($i) + ``` - Use a proc when possible, only using the more powerful facilities of macros, templates, iterators, and converters when necessary. @@ -247,32 +256,45 @@ Coding Conventions Conventions for multi-line statements and expressions ----------------------------------------------------- -- Tuples which are longer than one line should indent their parameters to - align with the parameters above it. +- Tuples which are longer than one line should indent their parameters. - .. code-block:: nim + ```nim type - LongTupleA = tuple[wordyTupleMemberOne: int, wordyTupleMemberTwo: string, - wordyTupleMemberThree: float] + LongTupleA = tuple[ + wordyTupleMemberOne: int, wordyTupleMemberTwo: string, + wordyTupleMemberThree: float] + ``` - Similarly, any procedure and procedure type declarations that are longer - than one line should do the same thing. + than one line should do the same thing. Double indent may be used to + distinguish them from the body that follows - this applies to all constructs + with a body (if, while, etc). - .. code-block:: nim + ```nim type - EventCallback = proc (timeReceived: Time, errorCode: int, event: Event, - output: var string) - - proc lotsOfArguments(argOne: string, argTwo: int, argThree: float, - argFour: proc(), argFive: bool): int - {.heyLookALongPragma.} = - -- Multi-line procedure calls should continue on the same column as the opening - parenthesis (like multi-line procedure declarations). - - .. code-block:: nim - startProcess(nimExecutable, currentDirectory, compilerArguments - environment, processOptions) + EventCallback = proc( + timeReceived: Time, errorCode: int, event: Event, + output: var string) + + proc lotsOfArguments( + argOne: string, argTwo: int, argThree: float, + argFour: proc(), argFive: bool, argSix: int + ): GenericType[int, string] {.heyLookALongPragma.} = + discard + ``` + +- Multi-line procedure calls should continue indented (like multi-line procedure + declarations). + + ```nim + startProcess( + nimExecutable, currentDirectory, compilerArguments + environment, processOptions) + ``` + +Previous versions of this guide advocated vertical alignment along the opening +brace / parenthesis - both styles are permissible with a preference for the +current style in new code. Miscellaneous ------------- @@ -288,15 +310,26 @@ Miscellaneous use this: - .. code-block:: nim + ```nim let a = """ foo bar """ + ``` instead of: - .. code-block:: nim + ```nim let a = """foo bar """ + ``` + +- A getter API for a private field `foo` should preferably be named `foo`, not `getFoo`. + A getter-like API should preferably be named `getFoo`, not `foo` if: + * the API has side effects + * or the cost is not `O(1)` + For in between cases, there is no clear guideline. + +- Likewise with a setter API, replacing `foo` with `foo=` and `getFoo` with `setFoo` + in the above text. diff --git a/doc/nimc.rst b/doc/nimc.md index aad889591..38558454b 100644 --- a/doc/nimc.rst +++ b/doc/nimc.md @@ -1,5 +1,3 @@ -.. default-role:: code - =================================== Nim Compiler User Guide =================================== @@ -7,11 +5,15 @@ :Author: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: - "Look at you, hacker. A pathetic creature of meat and bone, panting and - sweating as you run through my corridors. How can you challenge a perfect, - immortal machine?" +.. + +> "Look at you, hacker. A pathetic creature of meat and bone, panting and +> sweating as you run through my corridors. How can you challenge a perfect, +> immortal machine?" Introduction @@ -19,10 +21,10 @@ Introduction This document describes the usage of the *Nim compiler* on the different supported platforms. It is not a definition of the Nim -programming language (which is covered in the `manual <manual.html>`_). +programming language (which is covered in the [manual](manual.html)). Nim is free software; it is licensed under the -`MIT License <http://www.opensource.org/licenses/mit-license.php>`_. +[MIT License](http://www.opensource.org/licenses/mit-license.php). Compiler Usage @@ -30,8 +32,22 @@ Compiler Usage Command-line switches --------------------- +All options that take a `PATH` or `DIR` argument are subject to path substitution: + +- `$nim`: The global nim prefix path +- `$lib`: The stdlib path +- `$home` and `~`: The user's home path +- `$config`: The directory of the module currently being compiled +- `$projectname`: The project file's name without file extension +- `$projectpath` and `$projectdir`: The project file's path +- `$nimcache`: The nimcache path + + Basic command-line switches are: +.. no syntax highlighting in the below included files at the moment +.. default-role:: code + Usage: .. include:: basicopt.txt @@ -43,12 +59,13 @@ Advanced command-line switches are: .. include:: advopt.txt +.. include:: rstcommon.rst List of warnings ---------------- -Each warning can be activated individually with `--warning[NAME]:on|off` or -in a `push` pragma. +Each warning can be activated individually with `--warning:NAME:on|off`:option: or +in a `push` pragma with `{.warning[NAME]:on|off.}`. ========================== ============================================ Name Description @@ -64,6 +81,14 @@ SmallLshouldNotBeUsed The letter 'l' should not be used as an identifier. EachIdentIsTuple The code contains a confusing `var` declaration. +CStringConv Warn about dangerous implicit conversions + to `cstring`. +EnumConv Warn about conversions from enum to enum. +AnyEnumConv Warn about any conversions to an enum type. +HoleEnumConv Warn about conversion to an enum with + holes. These conversions are unsafe. +ResultUsed Warn about the usage of the + built-in `result` variable. User Some user-defined warning. ========================== ============================================ @@ -71,8 +96,8 @@ User Some user-defined warning. List of hints ------------- -Each hint can be activated individually with `--hint[NAME]:on|off` or in a -`push` pragma. +Each hint can be activated individually with `--hint:NAME:on|off`:option: or in a +`push` pragma with `{.hint[NAME]:on|off.}`. ========================== ============================================ Name Description @@ -90,7 +115,6 @@ ExprAlwaysX ExtendedContext GCStats Dumps statistics about the Garbage Collector. GlobalVar Shows global variables declarations. -LineTooLong Line exceeds the maximum length. Link Linking phase. Name Path Search paths modifications. @@ -116,13 +140,13 @@ Level Description ===== ============================================ 0 Minimal output level for the compiler. 1 Displays compilation of all the compiled files, including those imported - by other modules or through the `compile pragma - <manual.html#implementation-specific-pragmas-compile-pragma>`_. + by other modules or through the [compile pragma]( + manual.html#implementation-specific-pragmas-compile-pragma). This is the default level. 2 Displays compilation statistics, enumerates the dynamic libraries that will be loaded by the final binary, and dumps to - standard output the result of applying `a filter to the source code - <filters.html>`_ if any filter was used during compilation. + standard output the result of applying [a filter to the source code]( + filters.html) if any filter was used during compilation. 3 In addition to the previous levels dumps a debug stack trace for compiler developers. ===== ============================================ @@ -131,54 +155,89 @@ Level Description Compile-time symbols -------------------- -Through the `-d:x` or `--define:x` switch you can define compile-time +Through the `-d:x`:option: or `--define:x`:option: switch you can define compile-time symbols for conditional compilation. The defined switches can be checked in -source code with the `when statement -<manual.html#statements-and-expressions-when-statement>`_ and -`defined proc <system.html#defined,untyped>`_. The typical use of this switch is -to enable builds in release mode (`-d:release`) where optimizations are -enabled for better performance. Another common use is the `-d:ssl` switch to +source code with the [when statement]( +manual.html#statements-and-expressions-when-statement) and +[defined proc](system.html#defined,untyped). The typical use of this switch is +to enable builds in release mode (`-d:release`:option:) where optimizations are +enabled for better performance. Another common use is the `-d:ssl`:option: switch to activate SSL sockets. -Additionally, you may pass a value along with the symbol: `-d:x=y` -which may be used in conjunction with the `compile-time define -pragmas<manual.html#implementation-specific-pragmas-compileminustime-define-pragmas>`_ +Additionally, you may pass a value along with the symbol: `-d:x=y`:option: +which may be used in conjunction with the [compile-time define +pragmas](manual.html#implementation-specific-pragmas-compileminustime-define-pragmas) to override symbols during build time. Compile-time symbols are completely **case insensitive** and underscores are -ignored too. `--define:FOO` and `--define:foo` are identical. +ignored too. `--define:FOO`:option: and `--define:foo`:option: are identical. Compile-time symbols starting with the `nim` prefix are reserved for the implementation and should not be used elsewhere. +========================== ============================================ +Name Description +========================== ============================================ +nimStdSetjmp Use the standard `setjmp()/longjmp()` library + functions for setjmp-based exceptions. This is + the default on most platforms. +nimSigSetjmp Use `sigsetjmp()/siglongjmp()` for setjmp-based exceptions. +nimRawSetjmp Use `_setjmp()/_longjmp()` on POSIX and `_setjmp()/longjmp()` + on Windows, for setjmp-based exceptions. It's the default on + BSDs and BSD-like platforms, where it's significantly faster + than the standard functions. +nimBuiltinSetjmp Use `__builtin_setjmp()/__builtin_longjmp()` for setjmp-based + exceptions. This will not work if an exception is being thrown + and caught inside the same procedure. Useful for benchmarking. +========================== ============================================ + Configuration files ------------------- -**Note:** The *project file name* is the name of the `.nim` file that is +**Note:** The *project file name* is the name of the ``.nim`` file that is passed as a command-line argument to the compiler. -The `nim` executable processes configuration files in the following +The `nim`:cmd: executable processes configuration files in the following directories (in this order; later files overwrite previous settings): -1) `$nim/config/nim.cfg`, `/etc/nim/nim.cfg` (UNIX) or ``<Nim's installation directory>\config\nim.cfg`` (Windows). This file can be skipped with the `--skipCfg` command line option. -2) If environment variable `XDG_CONFIG_HOME` is defined, `$XDG_CONFIG_HOME/nim/nim.cfg` or `~/.config/nim/nim.cfg` (POSIX) or `%APPDATA%/nim/nim.cfg` (Windows). This file can be skipped with the `--skipUserCfg` command line option. -3) `$parentDir/nim.cfg` where `$parentDir` stands for any parent directory of the project file's path. These files can be skipped with the `--skipParentCfg` command-line option. -4) `$projectDir/nim.cfg` where `$projectDir` stands for the project file's path. This file can be skipped with the `--skipProjCfg` command-line option. -5) A project can also have a project-specific configuration file named `$project.nim.cfg` that resides in the same directory as `$project.nim`. This file can be skipped with the `--skipProjCfg` command-line option. +1) ``$nim/config/nim.cfg``, ``/etc/nim/nim.cfg`` (UNIX) or + ``<Nim's installation directory>\config\nim.cfg`` (Windows). + This file can be skipped with the `--skipCfg`:option: command line option. +2) If environment variable `XDG_CONFIG_HOME` is defined, + ``$XDG_CONFIG_HOME/nim/nim.cfg`` or ``~/.config/nim/nim.cfg`` (POSIX) or + ``%APPDATA%/nim/nim.cfg`` (Windows). + This file can be skipped with the `--skipUserCfg`:option: command line + option. +3) ``$parentDir/nim.cfg`` where ``$parentDir`` stands for any parent + directory of the project file's path. + These files can be skipped with the `--skipParentCfg`:option: + command-line option. +4) ``$projectDir/nim.cfg`` where ``$projectDir`` stands for the project + file's path. + This file can be skipped with the `--skipProjCfg`:option: + command-line option. +5) A project can also have a project-specific configuration file named + ``$project.nim.cfg`` that resides in the same directory as ``$project.nim``. + This file can be skipped with the `--skipProjCfg`:option: + command-line option. Command-line settings have priority over configuration file settings. The default build of a project is a `debug build`:idx:. To compile a -`release build`:idx: define the `release` symbol:: +`release build`:idx: define the `release` symbol: + ```cmd nim c -d:release myproject.nim + ``` - To compile a `dangerous release build`:idx: define the `danger` symbol:: +To compile a `dangerous release build`:idx: define the `danger` symbol: + ```cmd nim c -d:danger myproject.nim + ``` Search path handling @@ -188,39 +247,39 @@ Nim has the concept of a global search path (PATH) that is queried to determine where to find imported modules or include files. If multiple files are found an ambiguity error is produced. -`nim dump` shows the contents of the PATH. +`nim dump`:cmd: shows the contents of the PATH. However before the PATH is used the current directory is checked for the -file's existence. So if PATH contains `$lib` and `$lib/bar` and the -directory structure looks like this:: +file's existence. So if PATH contains ``$lib`` and ``$lib/bar`` and the +directory structure looks like this: - $lib/x.nim - $lib/bar/x.nim - foo/x.nim - foo/main.nim - other.nim + $lib/x.nim + $lib/bar/x.nim + foo/x.nim + foo/main.nim + other.nim And `main` imports `x`, `foo/x` is imported. If `other` imports `x` -then both `$lib/x.nim` and `$lib/bar/x.nim` match but `$lib/x.nim` is used +then both ``$lib/x.nim`` and ``$lib/bar/x.nim`` match but ``$lib/x.nim`` is used as it is the first match. Generated C code directory -------------------------- The generated files that Nim produces all go into a subdirectory called -`nimcache`. Its full path is +``nimcache``. Its full path is -- `$XDG_CACHE_HOME/nim/$projectname(_r|_d)` or `~/.cache/nim/$projectname(_r|_d)` +- ``$XDG_CACHE_HOME/nim/$projectname(_r|_d)`` or ``~/.cache/nim/$projectname(_r|_d)`` on Posix -- `$HOME/nimcache/$projectname(_r|_d)` on Windows. +- ``$HOME\nimcache\$projectname(_r|_d)`` on Windows. The `_r` suffix is used for release builds, `_d` is for debug builds. This makes it easy to delete all generated files. -The `--nimcache` -`compiler switch <#compiler-usage-commandminusline-switches>`_ can be used to -to change the `nimcache` directory. +The `--nimcache`:option: +[compiler switch][command-line switches] can be used to +to change the ``nimcache`` directory. However, the generated C code is not platform-independent. C code generated for Linux does not compile on Windows, for instance. The comment on top of the @@ -230,60 +289,75 @@ C file lists the OS, CPU, and CC the file has been compiled for. Compiler Selection ================== -To change the compiler from the default compiler (at the command line):: +To change the compiler from the default compiler (at the command line): + ```cmd nim c --cc:llvm_gcc --compile_only myfile.nim + ``` -This uses the configuration defined in ``config\nim.cfg`` for `lvm_gcc`. +This uses the configuration defined in ``config\nim.cfg`` for `llvm_gcc`:cmd:. If nimcache already contains compiled code from a different compiler for the same project, -add the `-f` flag to force all files to be recompiled. +add the `-f`:option: flag to force all files to be recompiled. The default compiler is defined at the top of ``config\nim.cfg``. -Changing this setting affects the compiler used by `koch` to (re)build Nim. +Changing this setting affects the compiler used by `koch`:cmd: to (re)build Nim. -To use the `CC` environment variable, use `nim c --cc:env myfile.nim`. To use the -`CXX` environment variable, use `nim cpp --cc:env myfile.nim`. `--cc:env` is available -since Nim version 1.4. +To use the `CC` environment variable, use `nim c --cc:env myfile.nim`:cmd:. +To use the `CXX` environment variable, use `nim cpp --cc:env myfile.nim`:cmd:. +`--cc:env`:option: is available since Nim version 1.4. Cross-compilation ================= -To cross compile, use for example:: +To cross compile, use for example: + ```cmd nim c --cpu:i386 --os:linux --compileOnly --genScript myproject.nim + ``` -Then move the C code and the compile script `compile_myproject.sh` to your +Then move the C code and the compile script `compile_myproject.sh`:cmd: to your Linux i386 machine and run the script. -Another way is to make Nim invoke a cross compiler toolchain:: +Another way is to make Nim invoke a cross compiler toolchain: + ```cmd nim c --cpu:arm --os:linux myproject.nim + ``` -For cross compilation, the compiler invokes a C compiler named -like `$cpu.$os.$cc` (for example arm.linux.gcc) and the configuration -system is used to provide meaningful defaults. For example for `ARM` your -configuration file should contain something like:: +For cross compilation, the compiler invokes a C compiler named like +`$cpu.$os.$cc` (for example `arm.linux.gcc`) with options defined in +`$cpu.$os.$cc.options.always`. The configuration system is used to provide +meaningful defaults. For example, for Linux on a 32-bit ARM CPU, your +configuration file should contain something like: - arm.linux.gcc.path = "/usr/bin" - arm.linux.gcc.exe = "arm-linux-gcc" - arm.linux.gcc.linkerexe = "arm-linux-gcc" + arm.linux.gcc.path = "/usr/bin" + arm.linux.gcc.exe = "arm-linux-gcc" + arm.linux.gcc.linkerexe = "arm-linux-gcc" + arm.linux.gcc.options.always = "-w -fmax-errors=3" Cross-compilation for Windows ============================= -To cross-compile for Windows from Linux or macOS using the MinGW-w64 toolchain:: +To cross-compile for Windows from Linux or macOS using the MinGW-w64 toolchain: + ```cmd nim c -d:mingw myproject.nim + # `nim r` also works, running the binary via `wine` or `wine64`: + nim r -d:mingw --eval:'import os; echo "a" / "b"' + ``` -Use `--cpu:i386` or `--cpu:amd64` to switch the CPU architecture. +Use `--cpu:i386`:option: or `--cpu:amd64`:option: to switch the CPU architecture. -The MinGW-w64 toolchain can be installed as follows:: +The MinGW-w64 toolchain can be installed as follows: - Ubuntu: apt install mingw-w64 - CentOS: yum install mingw32-gcc | mingw64-gcc - requires EPEL - OSX: brew install mingw-w64 + ```cmd + apt install mingw-w64 # Ubuntu + yum install mingw32-gcc + yum install mingw64-gcc # CentOS - requires EPEL + brew install mingw-w64 # OSX + ``` Cross-compilation for Android @@ -293,36 +367,40 @@ There are two ways to compile for Android: terminal programs (Termux) and with the NDK (Android Native Development Kit). The first one is to treat Android as a simple Linux and use -`Termux <https://wiki.termux.com>`_ to connect and run the Nim compiler +[Termux](https://wiki.termux.com) to connect and run the Nim compiler directly on android as if it was Linux. These programs are console-only programs that can't be distributed in the Play Store. -Use regular `nim c` inside termux to make Android terminal programs. +Use regular `nim c`:cmd: inside termux to make Android terminal programs. Normal Android apps are written in Java, to use Nim inside an Android app you need a small Java stub that calls out to a native library written in -Nim using the `NDK <https://developer.android.com/ndk>`_. You can also use -`native-activity <https://developer.android.com/ndk/samples/sample_na>`_ +Nim using the [NDK](https://developer.android.com/ndk). You can also use +[native-activity](https://developer.android.com/ndk/samples/sample_na) to have the Java stub be auto-generated for you. -Use `nim c -c --cpu:arm --os:android -d:androidNDK --noMain:on` to +Use `nim c -c --cpu:arm --os:android -d:androidNDK --noMain:on`:cmd: to generate the C source files you need to include in your Android Studio project. Add the generated C files to CMake build script in your Android project. Then do the final compile with Android Studio which uses Gradle to call CMake to compile the project. -Because Nim is part of a library it can't have its own c style `main()` -so you would need to define your own `android_main` and init the Java +Because Nim is part of a library it can't have its own C-style `main()`:c: +so you would need to define your own `android_main`:c: and init the Java environment, or use a library like SDL2 or GLFM to do it. After the Android -stuff is done, it's very important to call `NimMain()` in order to +stuff is done, it's very important to call `NimMain()`:c: in order to initialize Nim's garbage collector and to run the top level statements of your program. -.. code-block:: Nim - + ```Nim proc NimMain() {.importc.} proc glfmMain*(display: ptr GLFMDisplay) {.exportc.} = NimMain() # initialize garbage collector memory, types and stack + ``` + + +The name `NimMain` can be influenced via the `--nimMainPrefix:prefix` switch. +Use `--nimMainPrefix:MyLib` and the function to call is named `MyLibNimMain`. Cross-compilation for iOS @@ -333,85 +411,119 @@ Normal languages for iOS development are Swift and Objective C. Both of these use LLVM and can be compiled into object files linked together with C, C++ or Objective C code produced by Nim. -Use `nim c -c --os:ios --noMain:on` to generate C files and include them in +Use `nim c -c --os:ios --noMain:on`:cmd: to generate C files and include them in your XCode project. Then you can use XCode to compile, link, package and sign everything. -Because Nim is part of a library it can't have its own c style `main()` so you +Because Nim is part of a library it can't have its own C-style `main()`:c: so you would need to define `main` that calls `autoreleasepool` and `UIApplicationMain` to do it, or use a library like SDL2 or GLFM. After -the iOS setup is done, it's very important to call `NimMain()` to +the iOS setup is done, it's very important to call `NimMain()`:c: to initialize Nim's garbage collector and to run the top-level statements of your program. -.. code-block:: Nim - + ```Nim proc NimMain() {.importc.} proc glfmMain*(display: ptr GLFMDisplay) {.exportc.} = NimMain() # initialize garbage collector memory, types and stack + ``` Note: XCode's "make clean" gets confused about the generated nim.c files, so you need to clean those files manually to do a clean build. +The name `NimMain` can be influenced via the `--nimMainPrefix:prefix` switch. +Use `--nimMainPrefix:MyLib` and the function to call is named `MyLibNimMain`. + Cross-compilation for Nintendo Switch ===================================== -Simply add --os:nintendoswitch -to your usual `nim c` or `nim cpp` command and set the `passC` -and `passL` command line switches to something like: +Simply add `--os:nintendoswitch`:option: +to your usual `nim c`:cmd: or `nim cpp`:cmd: command and set the `passC`:option: +and `passL`:option: command line switches to something like: -.. code-block:: console - nim c ... --passC="-I$DEVKITPRO/libnx/include" ... + ```cmd + nim c ... --d:nimAllocPagesViaMalloc --mm:orc --passC="-I$DEVKITPRO/libnx/include" ... --passL="-specs=$DEVKITPRO/libnx/switch.specs -L$DEVKITPRO/libnx/lib -lnx" + ``` -or setup a nim.cfg file like so: +or setup a ``nim.cfg`` file like so: -.. code-block:: Nim - #nim.cfg - --passC="-I$DEVKITPRO/libnx/include" - --passL="-specs=$DEVKITPRO/libnx/switch.specs -L$DEVKITPRO/libnx/lib -lnx" + #nim.cfg + --mm:orc + --d:nimAllocPagesViaMalloc + --define:nimInheritHandles + --passC="-I$DEVKITPRO/libnx/include" + --passL="-specs=$DEVKITPRO/libnx/switch.specs -L$DEVKITPRO/libnx/lib -lnx" -The DevkitPro setup must be the same as the default with their new installer -`here for Mac/Linux <https://github.com/devkitPro/pacman/releases>`_ or -`here for Windows <https://github.com/devkitPro/installer/releases>`_. +The devkitPro setup must be the same as the default with their new installer +[here for Mac/Linux](https://github.com/devkitPro/pacman/releases) or +[here for Windows](https://github.com/devkitPro/installer/releases). -For example, with the above-mentioned config:: +For example, with the above-mentioned config: + ```cmd nim c --os:nintendoswitch switchhomebrew.nim + ``` -This will generate a file called `switchhomebrew.elf` which can then be turned into -an nro file with the `elf2nro` tool in the DevkitPro release. Examples can be found at -`the nim-libnx github repo <https://github.com/jyapayne/nim-libnx.git>`_. +This will generate a file called ``switchhomebrew.elf`` which can then be turned into +an nro file with the `elf2nro`:cmd: tool in the devkitPro release. Examples can be found at +[the nim-libnx github repo](https://github.com/jyapayne/nim-libnx.git). -There are a few things that don't work because the DevkitPro libraries don't support them. +There are a few things that don't work because the devkitPro libraries don't support them. They are: 1. Waiting for a subprocess to finish. A subprocess can be started, but right now it can't be waited on, which sort of makes subprocesses a bit hard to use -2. Dynamic calls. DevkitPro libraries have no dlopen/dlclose functions. -3. Command line parameters. It doesn't make sense to have these for a console - anyways, so no big deal here. -4. mqueue. Sadly there are no mqueue headers. -5. ucontext. No headers for these either. No coroutines for now :( -6. nl_types. No headers for this. +2. Dynamic calls. Switch OS (Horizon) doesn't support dynamic libraries, so dlopen/dlclose are not available. +3. mqueue. Sadly there are no mqueue headers. +4. ucontext. No headers for these either. No coroutines for now :( +5. nl_types. No headers for this. +6. As mmap is not supported, the nimAllocPagesViaMalloc option has to be used. + +GPU Compilation +=============== + +Compiling for GPU computation can be achieved with `--cc:nvcc` for CUDA with nvcc, or with `--cc:hipcc` for AMD GPUs with HIP. Both compilers require building for C++ with `nim cpp`. + +Here's a very simple CUDA kernel example using emit, which can be compiled with `nim cpp --cc:nvcc --define:"useMalloc" hello_kernel.nim` assuming you have the CUDA toolkit installed. + +```nim +{.emit: """ +__global__ void add(int a, int b) { + int c; + c = a + b; +} +""".} + +proc main() = + {.emit: """ + add<<<1,1>>>(2,7); + """.} + +main() +``` DLL generation ============== +**Note**: The same rules apply to `lib*.so` shared object files on UNIX. For better +readability only the DLL version is described here. + Nim supports the generation of DLLs. However, there must be only one instance of the GC per process/address space. This instance is contained in -`nimrtl.dll`. This means that every generated Nim DLL depends -on `nimrtl.dll`. To generate the "nimrtl.dll" file, use the command:: +``nimrtl.dll``. This means that every generated Nim DLL depends +on ``nimrtl.dll``. To generate the "nimrtl.dll" file, use the command: + ```cmd nim c -d:release lib/nimrtl.nim + ``` -To link against `nimrtl.dll` use the command:: +To link against ``nimrtl.dll`` use the command: + ```cmd nim c -d:useNimRtl myprog.nim - -**Note**: Currently the creation of `nimrtl.dll` with thread support has -never been tested and is unlikely to work! + ``` Additional compilation switches @@ -426,20 +538,20 @@ Define Effect ====================== ========================================================= `release` Turns on the optimizer. More aggressive optimizations are possible, e.g.: - `--passC:-ffast-math` (but see issue #10305) + `--passC:-ffast-math`:option: (but see issue #10305) `danger` Turns off all runtime checks and turns on the optimizer. -`useFork` Makes `osproc` use `fork` instead of `posix_spawn`. -`useNimRtl` Compile and link against `nimrtl.dll`. +`useFork` Makes `osproc` use `fork`:c: instead of `posix_spawn`:c:. +`useNimRtl` Compile and link against ``nimrtl.dll``. `useMalloc` Makes Nim use C's `malloc`:idx: instead of Nim's own memory manager, albeit prefixing each allocation with its size to support clearing memory on reallocation. - This only works with `gc:none`, `gc:arc` and - `--gc:orc`. + This only works with `--mm:none`:option:, + `--mm:arc`:option: and `--mm:orc`:option:. `useRealtimeGC` Enables support of Nim's GC for *soft* realtime - systems. See the documentation of the `gc <gc.html>`_ + systems. See the documentation of the [refc](refc.html) for further information. `logGC` Enable GC logging to stdout. -`nodejs` The JS target is actually `node.js`. +`nodejs` The JS target is actually ``node.js``. `ssl` Enables OpenSSL support for the sockets module. `memProfiler` Enables memory profiling for the native GC. `uClibc` Use uClibc instead of libc. (Relevant for Unix-like OSes) @@ -447,18 +559,25 @@ Define Effect what's in the Nim file with what's in the C header. This may become enabled by default in the future. `tempDir` This symbol takes a string as its value, like - `--define:tempDir:/some/temp/path` to override the - temporary directory returned by `os.getTempDir()`. + `--define:tempDir:/some/temp/path`:option: to override + the temporary directory returned by `os.getTempDir()`. The value **should** end with a directory separator character. (Relevant for the Android platform) `useShPath` This symbol takes a string as its value, like - `--define:useShPath:/opt/sh/bin/sh` to override the - path for the `sh` binary, in cases where it is not - located in the default location `/bin/sh`. -`noSignalHandler` Disable the crash handler from `system.nim`. -`globalSymbols` Load all `{.dynlib.}` libraries with the `RTLD_GLOBAL` + `--define:useShPath:/opt/sh/bin/sh`:option: to override + the path for the `sh`:cmd: binary, in cases where it is + not located in the default location ``/bin/sh``. +`noSignalHandler` Disable the crash handler from ``system.nim``. +`globalSymbols` Load all `{.dynlib.}` libraries with the `RTLD_GLOBAL`:c: flag on Posix systems to resolve symbols in subsequently loaded libraries. +`lto` Enable link-time optimization in the backend compiler and + linker. +`lto_incremental` Enable link-time optimization and additionally enable + incremental linking for compilers that support it. + Currently only clang and vcc. +`strip` Strip debug symbols added by the backend compiler from + the executable. ====================== ========================================================= @@ -473,20 +592,21 @@ generator and are subject to change. LineDir option -------------- -The `lineDir` option can be turned on or off. If turned on the -generated C code contains `#line` directives. This may be helpful for +The `--lineDir`:option: option can be turned on or off. If turned on the +generated C code contains `#line`:c: directives. This may be helpful for debugging with GDB. StackTrace option ----------------- -If the `stackTrace` option is turned on, the generated C contains code to +If the `--stackTrace`:option: option is turned on, the generated C contains code to ensure that proper stack traces are given if the program crashes or some uncaught exception is raised. LineTrace option ---------------- -The `lineTrace` option implies the `stackTrace` option. If turned on, +The `--lineTrace`:option: option implies the `stackTrace`:option: option. +If turned on, the generated C contains code to ensure that proper stack traces with line number information are given if the program crashes or an uncaught exception is raised. @@ -496,37 +616,39 @@ DynlibOverride ============== By default Nim's `dynlib` pragma causes the compiler to generate -`GetProcAddress` (or their Unix counterparts) -calls to bind to a DLL. With the `dynlibOverride` command line switch this -can be prevented and then via `--passL` the static library can be linked +`GetProcAddress`:cpp: (or their Unix counterparts) +calls to bind to a DLL. With the `dynlibOverride`:option: command line switch this +can be prevented and then via `--passL`:option: the static library can be linked against. For instance, to link statically against Lua this command might work -on Linux:: +on Linux: + ```cmd nim c --dynlibOverride:lua --passL:liblua.lib program.nim + ``` Backend language options ======================== -The typical compiler usage involves using the `compile` or `c` command to -transform a `.nim` file into one or more `.c` files which are then +The typical compiler usage involves using the `compile`:option: or `c`:option: +command to transform a ``.nim`` file into one or more ``.c`` files which are then compiled with the platform's C compiler into a static binary. However, there are other commands to compile to C++, Objective-C, or JavaScript. More details -can be read in the `Nim Backend Integration document <backends.html>`_. +can be read in the [Nim Backend Integration document](backends.html). Nim documentation tools ======================= Nim provides the `doc`:idx: command to generate HTML -documentation from `.nim` source files. Only exported symbols will appear in -the output. For more details `see the docgen documentation <docgen.html>`_. +documentation from ``.nim`` source files. Only exported symbols will appear in +the output. For more details see [the docgen documentation](docgen.html). Nim idetools integration ======================== Nim provides language integration with external IDEs through the -idetools command. See the documentation of `idetools <idetools.html>`_ +idetools command. See the documentation of [idetools](idetools.html) for further information. .. @@ -557,19 +679,21 @@ embedded microprocessors with only a few kilobytes of memory. A good start is to use the `any` operating target together with the `malloc` memory allocator and the `arc` garbage collector. For example: -`nim c --os:any --gc:arc -d:useMalloc [...] x.nim` + ```cmd + nim c --os:any --mm:arc -d:useMalloc [...] x.nim + ``` -- `--gc:arc` will enable the reference counting memory management instead +- `--mm:arc`:option: will enable the reference counting memory management instead of the default garbage collector. This enables Nim to use heap memory which is required for strings and seqs, for example. -- The `--os:any` target makes sure Nim does not depend on any specific +- The `--os:any`:option: target makes sure Nim does not depend on any specific operating system primitives. Your platform should support only some basic ANSI C library `stdlib` and `stdio` functions which should be available on almost any platform. -- The `-d:useMalloc` option configures Nim to use only the standard C memory - manage primitives `malloc()`, `free()`, `realloc()`. +- The `-d:useMalloc`:option: option configures Nim to use only the standard C memory + manage primitives `malloc()`:c:, `free()`:c:, `realloc()`:c:. If your platform does not provide these functions it should be trivial to provide an implementation for them and link these to your program. @@ -579,20 +703,65 @@ additional flags to both the Nim compiler and the C compiler and/or linker to optimize the build for size. For example, the following flags can be used when targeting a gcc compiler: -`--opt:size --passC:-flto --passL:-flto` +`--opt:size -d:lto -d:strip`:option: -The `--opt:size` flag instructs Nim to optimize code generation for small -size (with the help of the C compiler), the `flto` flags enable link-time -optimization in the compiler and linker. +The `--opt:size`:option: flag instructs Nim to optimize code generation for small +size (with the help of the C compiler), the `-d:lto`:option: flags enable link-time +optimization in the compiler and linker, the `-d:strip`:option: strips debug symbols. -Check the `Cross-compilation` section for instructions on how to compile the +Check the [Cross-compilation] section for instructions on how to compile the program for your target. + +nimAllocPagesViaMalloc +---------------------- + +Nim's default allocator is based on TLSF, this algorithm was designed for embedded +devices. This allocator gets blocks/pages of memory via a currently undocumented +`osalloc` API which usually uses POSIX's `mmap` call. On many environments `mmap` +is not available but C's `malloc` is. You can use the `nimAllocPagesViaMalloc` +define to use `malloc` instead of `mmap`. `nimAllocPagesViaMalloc` is currently +only supported with `--mm:arc` or `--mm:orc`. (Since version 1.6) + +nimPage256 / nimPage512 / nimPage1k +=================================== + +Adjust the page size for Nim's GC allocator. This enables using +`nimAllocPagesViaMalloc` on devices with less RAM. The default +page size requires too much RAM to work. + +Recommended settings: + +- < 32 kB of RAM use `nimPage256` + +- < 512 kB of RAM use `nimPage512` + +- < 2 MB of RAM use `nimPage1k` + +Initial testing hasn't shown much difference between 512B or 1kB page sizes +in terms of performance or latency. Using `nimPages256` will limit the +total amount of allocatable RAM. + +nimMemAlignTiny +=============== + +Sets `MemAlign` to `4` bytes which reduces the memory alignment +to better match some embedded devices. + +Thread stack size +================= + +Nim's thread API provides a simple wrapper around more advanced +RTOS task features. Customizing the stack size and stack guard size can +be done by setting `-d:nimThreadStackSize=16384` or `-d:nimThreadStackGuard=32`. + +Currently only Zephyr, NuttX and FreeRTOS support these configurations. + Nim for realtime systems ======================== -See the documentation of Nim's soft realtime `GC <gc.html>`_ for further -information. +See the `--mm:arc` or `--mm:orc` memory management settings in +[MM](mm.html) for further information. Signal handling in Nim @@ -602,7 +771,7 @@ The Nim programming language has no concept of Posix's signal handling mechanisms. However, the standard library offers some rudimentary support for signal handling, in particular, segmentation faults are turned into fatal errors that produce a stack trace. This can be disabled with the -`-d:noSignalHandler` switch. +`-d:noSignalHandler`:option: switch. Optimizing for Nim @@ -635,38 +804,28 @@ passed to subroutines. The compiler does not copy strings that are a result of a procedure call, because the callee returns a new string anyway. Thus it is efficient to do: -.. code-block:: Nim + ```Nim var s = procA() # assignment will not copy the string; procA allocates a new # string already + ``` However, it is not efficient to do: -.. code-block:: Nim + ```Nim var s = varA # assignment has to copy the whole string into a new buffer! + ``` For `let` symbols a copy is not always necessary: -.. code-block:: Nim + ```Nim let s = varA # may only copy a pointer if it safe to do so - - -If you know what you're doing, you can also mark single-string (or sequence) -objects as `shallow`:idx:\: - -.. code-block:: Nim - var s = "abc" - shallow(s) # mark 's' as a shallow string - var x = s # now might not copy the string! - -Usage of `shallow` is always safe once you know the string won't be modified -anymore, similar to Ruby's `freeze`:idx:. - + ``` The compiler optimizes string case statements: A hashing scheme is used for them if several different string constants are used. So code like this is reasonably efficient: -.. code-block:: Nim + ```Nim case normalize(k.key) of "name": c.name = v of "displayname": c.displayName = v @@ -682,3 +841,4 @@ efficient: else: quit(errorStr(p, "expected: console or gui")) of "license": c.license = UnixToNativePath(k.value) else: quit(errorStr(p, "unknown variable: " & k.key)) + ``` diff --git a/doc/nimdoc.cls b/doc/nimdoc.cls new file mode 100644 index 000000000..37039f130 --- /dev/null +++ b/doc/nimdoc.cls @@ -0,0 +1,196 @@ +\ProvidesClass{nimdoc}[2022/04/17, 2018/01/01 LaTeX2e nonstandard class] + +\LoadClass[a4paper,11pt]{article} + +\usepackage[a4paper,xetex,left=3cm,right=3cm,top=1.5cm,bottom=2cm]{geometry} + +% for 2-sided printing with larger inner "binding" margin +%\usepackage[a4paper,xetex,twoside,left=4cm,right=2cm,top=1.5cm,bottom=2cm]{geometry} +% for e-readers with 1.77:1 aspect ratio (e.g. 1920x1080) +%\usepackage[xetex,paperheight=27.6cm,paperwidth=15.5cm,left=3mm,right=3mm,top=3mm,bottom=3mm]{geometry} +% for e-readers with 1.45:1 aspect ratio (e.g. 1200x825) +%\usepackage[xetex,paperheight=22.5cm,paperwidth=15.5cm,left=3mm,right=3mm,top=3mm,bottom=3mm]{geometry} +% for e-readers with 1.33:1 aspect ratio (e.g. 1872x1404) +%\usepackage[xetex,paperheight=20.7cm,paperwidth=15.5cm,left=3mm,right=3mm,top=3mm,bottom=3mm]{geometry} + +\usepackage{fontspec} +% logic to select default font with some fall-back fonts. +\IfFontExistsTF{Times New Roman}{% + \setmainfont{Times New Roman} % the default font + \typeout{========================================= nim: using Times New Roman} +}{ + \IfFontExistsTF{FreeSerif}{% + \setmainfont{FreeSerif} % fallback #1 - official GNU font, resembles Times + \typeout{========================================= nim: using FreeSerif} + }{ + \IfFontExistsTF{DejaVuSerif}{% + \setmainfont{DejaVuSerif} % fallback #2 - very widespread free font + \typeout{========================================= nim: using DejaVuSerif} + }{ + \typeout{!!!!!!!!!!!!!!!!!!! Fonts not found !!!!!!!!!!!!!!!!!!!!!!!} + } + } +} + +% default monospace font for code: +\usepackage{GoMono} +\usepackage{relsize} +% make this monospace font 2 steps smaller to hold 80-character line +\newcommand{\rstverbblockfont}{\smaller[2]} +\newcommand{\rstverbinlinefont}{\smaller} + +\usepackage{parskip} % paragraphs delimited by vertical space, no indent +\usepackage{graphicx} + +\usepackage{makeidx} +\newcommand{\nimindexterm}[2]{#2\index{#2}\label{#1}} +\makeindex + +\usepackage{dingbat} % for \carriagereturn, etc +\usepackage{fvextra} % for code blocks (works better than original fancyvrb) +\fvset{ + breaklines, + breakafter={=}:|\_\{\}[](){,}.;+-*/'", + breaksymbolleft=\color{red}{\ensuremath{\hookrightarrow}}, + breaksymbolright=\color{red}{\small\carriagereturn} +} +\fvinlineset{% + breaklines, + breakafter={=}:|\_\{\}[](){,}.;+-*/'", + % that does not work at all when we underline inline code by ulem :-( + commandchars=\\\{\} +} + +\usepackage{scrextend} % for the `addmargin` environment + +\usepackage[table]{xcolor} +\usepackage[urlbordercolor=blue,linkbordercolor=cyan, + pdfborderstyle={/S/U/W 1}]{hyperref} +\usepackage{enumitem} % for option list, enumList, and rstfootnote + +\usepackage[most]{tcolorbox} % boxes around admonitions, code blocks, doc.item + +\newtcolorbox{rstadmonition}[1][]{blanker, breakable, + left=3mm, right=0mm, top=1mm, bottom=1mm, + before upper=\indent, parbox=false, #1} + +\newtcolorbox{rstquote}[1][]{blanker, breakable, + left=3mm, right=3mm, top=1mm, bottom=1mm, + parbox=false, + borderline west={0.3em}{0pt}{lightgray}, + borderline north={0.05em}{0pt}{lightgray}, + borderline east={0.05em}{0pt}{lightgray}, + borderline south={0.05em}{0pt}{lightgray}} + +\definecolor{rstframecolor}{rgb}{0.85, 0.8, 0.6} + +\usepackage{booktabs} +\belowrulesep=0ex +\aboverulesep=0ex +\renewcommand{\arraystretch}{1.1} + +\newtcolorbox{rstprebox}[1][]{blanker, breakable, + left=3mm, right=3mm, top=1mm, bottom=1mm, + borderline ={0.1em}{0pt}{rstframecolor}, + before upper=\indent, parbox=false, #1} + +\newenvironment{rstpre}{% +\VerbatimEnvironment\begingroup\begin{rstprebox}% +\begin{Verbatim}[fontsize=\rstverbblockfont , commandchars=\\\{\}]}% +{\end{Verbatim}\end{rstprebox}\endgroup} + +\newtcolorbox{rstdocitembox}[1][]{blanker, breakable, + left=3mm, right=3mm, top=1mm, bottom=1mm, + borderline ={1pt}{0pt}{cyan}, + before upper=\indent, parbox=false, #1} + +% Inline code formatting: grey underline, +% use \Verb from fvextras e.g. to display -- correctly as double - +\usepackage[normalem]{ulem} +\newcommand\rstuline{\bgroup\markoverwith{\textcolor{rstframecolor}{\rule[-0.8ex]{2pt}{1.0pt}}}\ULon} + +\newcommand{\rstcode}[1]{% +{\rstverbinlinefont\Verb{\rstuline{#1}}}% +} + +\newcommand{\rstcodeitem}[1]{\Verb{#1}} + +\newenvironment{rstdocitem}{% +\VerbatimEnvironment\begingroup\begin{rstdocitembox}% +\begin{Verbatim}[fontsize=\rstverbblockfont , commandchars=\\\{\}]}% +{\end{Verbatim}\end{rstdocitembox}\endgroup} + + +\newenvironment{rstfootnote}{\begin{description}[labelindent=1em,leftmargin=1em,labelwidth=2.6em]}{\end{description}} +\ifdim\linewidth<30em + \def\rstoptleftmargin{0.4\linewidth} + \def\rstoptlabelwidth{0.35\linewidth} +\else + \def\rstoptleftmargin{12em} + \def\rstoptlabelwidth{10.5em} +\fi +\newenvironment{rstoptlist}{% +\begin{description}[font=\sffamily\bfseries,style=nextline,leftmargin=\rstoptleftmargin,labelwidth=\rstoptlabelwidth]}{\end{description}} + +\usepackage{multirow} +\usepackage{tabulary} % tables with adjustable cell width and no overflow +% make tabulary prevent overflows (https://tex.stackexchange.com/a/195088) +\tymin=60pt +\tymax=\maxdimen +% to pack tabulary into a new environment, special syntax is needed :-( +\newenvironment{rsttab}[1]{\tabulary{\linewidth}{#1}}{\endtabulary} + +\newcommand{\rstsub}[1]{\raisebox{-0.5ex}{\scriptsize{#1}}} +\newcommand{\rstsup}[1]{\raisebox{0.5ex}{\scriptsize{#1}}} + +\newcommand{\rsthA}[2][]{\section[#1]{#2}} +\newcommand{\rsthB}[2][]{\subsection[#1]{#2}} +\newcommand{\rsthC}[2][]{\subsubsection[#1]{#2}} +\newcommand{\rsthD}[2][]{\paragraph[#1]{#2}} +\newcommand{\rsthE}[2][]{\paragraph[#1]{#2}} + +\newcommand{\rstovA}[2][]{\section*[#1]{#2}} +\newcommand{\rstovB}[2][]{\subsection*[#1]{#2}} +\newcommand{\rstovC}[2][]{\subsubsection*[#1]{#2}} +\newcommand{\rstovD}[2][]{\paragraph*[#1]{#2}} +\newcommand{\rstovE}[2][]{\paragraph*[#1]{#2}} + +% Syntax highlighting: +\newcommand{\spanDecNumber}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanBinNumber}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanHexNumber}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanOctNumber}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanFloatNumber}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanIdentifier}[1]{#1} +\newcommand{\spanKeyword}[1]{\textbf{#1}} +\newcommand{\spanStringLit}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanLongStringLit}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanCharLit}[1]{#1} +\newcommand{\spanEscapeSequence}[1]{#1} +\newcommand{\spanOperator}[1]{\textbf{#1}} +\newcommand{\spanPunctuation}[1]{#1} +\newcommand{\spanComment}[1]{\emph{#1}} +\newcommand{\spanLongComment}[1]{\emph{#1}} +\newcommand{\spanRegularExpression}[1]{#1} +\newcommand{\spanTagStart}[1]{#1} +\newcommand{\spanTagEnd}[1]{#1} +\newcommand{\spanKey}[1]{#1} +\newcommand{\spanValue}[1]{#1} +\newcommand{\spanRawData}[1]{\textbf{\textcolor{darkgray}{#1}}} +\newcommand{\spanAssembler}[1]{#1} +\newcommand{\spanPreprocessor}[1]{#1} +\newcommand{\spanDirective}[1]{#1} +\newcommand{\spanCommand}[1]{#1} +\newcommand{\spanRule}[1]{#1} +\newcommand{\spanHyperlink}[1]{#1} +\newcommand{\spanLabel}[1]{#1} +\newcommand{\spanReference}[1]{#1} +\newcommand{\spanOther}[1]{#1} +\newcommand{\spantok}[1]{\fbox{#1}} +\newcommand{\spanPrompt}[1]{\textcolor{red}{\textbf{#1}}} +\newcommand{\spanProgramOutput}[1]{\textcolor{darkgray}{\textbf{#1}}} +\newcommand{\spanprogram}[1]{\textbf{\underline{#1}}} +\newcommand{\spanoption}[1]{\textbf{\textcolor{darkgray}{#1}}} + +% Never allow text overflow to margin: +\setlength\emergencystretch{\hsize}\hbadness=10000 diff --git a/doc/nimdoc.css b/doc/nimdoc.css index db9a7ce97..0c399e4c1 100644 --- a/doc/nimdoc.css +++ b/doc/nimdoc.css @@ -35,7 +35,13 @@ Modified by Boyd Greenfield and narimiran --escapeSequence: #c4891b; --number: #252dbe; --literal: #a4255b; + --program: #6060c0; + --option: #508000; --raw-data: #a4255b; + + --clipboard-image-normal: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: black' fill='none' viewBox='0 0 24 24' stroke='currentColor'%3E %3Cpath stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2' /%3E %3C/svg%3E"); + --clipboard-image-selected: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: black' viewBox='0 0 20 20' fill='currentColor'%3E %3Cpath d='M8 3a1 1 0 011-1h2a1 1 0 110 2H9a1 1 0 01-1-1z' /%3E %3Cpath d='M6 3a2 2 0 00-2 2v11a2 2 0 002 2h8a2 2 0 002-2V5a2 2 0 00-2-2 3 3 0 01-3 3H9a3 3 0 01-3-3z' /%3E %3C/svg%3E"); + --clipboard-image: var(--clipboard-image-normal) } [data-theme="dark"] { @@ -63,66 +69,54 @@ Modified by Boyd Greenfield and narimiran --escapeSequence: #bd93f9; --number: #bd93f9; --literal: #f1fa8c; + --program: #9090c0; + --option: #90b010; --raw-data: #8be9fd; -} -.theme-switch-wrapper { - display: flex; - align-items: center; -} - -.theme-switch-wrapper em { - margin-left: 10px; - font-size: 1rem; -} - -.theme-switch { - display: inline-block; - height: 22px; - position: relative; - width: 50px; -} - -.theme-switch input { - display: none; -} - -.slider { - background-color: #ccc; - bottom: 0; - cursor: pointer; - left: 0; - position: absolute; - right: 0; - top: 0; - transition: .4s; + --clipboard-image-normal: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: lightgray' fill='none' viewBox='0 0 24 24' stroke='currentColor'%3E %3Cpath stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2' /%3E %3C/svg%3E"); + --clipboard-image-selected: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: lightgray' viewBox='0 0 20 20' fill='currentColor'%3E %3Cpath d='M8 3a1 1 0 011-1h2a1 1 0 110 2H9a1 1 0 01-1-1z' /%3E %3Cpath d='M6 3a2 2 0 00-2 2v11a2 2 0 002 2h8a2 2 0 002-2V5a2 2 0 00-2-2 3 3 0 01-3 3H9a3 3 0 01-3-3z' /%3E %3C/svg%3E"); + --clipboard-image: var(--clipboard-image-normal); } -.slider:before { - background-color: #fff; - bottom: 4px; - content: ""; - height: 13px; - left: 4px; - position: absolute; - transition: .4s; - width: 13px; -} - -input:checked + .slider { - background-color: #66bb6a; -} - -input:checked + .slider:before { - transform: translateX(26px); -} - -.slider.round { - border-radius: 17px; +@media (prefers-color-scheme: dark) { + [data-theme="auto"] { + --primary-background: #171921; + --secondary-background: #1e202a; + --third-background: #2b2e3b; + --info-background: #008000; + --warning-background: #807000; + --error-background: #c03000; + --border: #0e1014; + --text: #fff; + --anchor: #8be9fd; + --anchor-focus: #8be9fd; + --input-focus: #8be9fd; + --strong: #bd93f9; + --hint: #7A7C85; + --nim-sprite-base64: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAARMAAABMCAYAAABOBlMuAAAACXBIWXMAAAsTAAALEwEAmpwYAAAFFmlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPD94cGFja2V0IGJlZ2luPSLvu78iIGlkPSJXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQiPz4gPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iQWRvYmUgWE1QIENvcmUgNS42LWMxNDggNzkuMTY0MDM2LCAyMDE5LzA4LzEzLTAxOjA2OjU3ICAgICAgICAiPiA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyIgeG1sbnM6cGhvdG9zaG9wPSJodHRwOi8vbnMuYWRvYmUuY29tL3Bob3Rvc2hvcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RFdnQ9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZUV2ZW50IyIgeG1wOkNyZWF0b3JUb29sPSJBZG9iZSBQaG90b3Nob3AgMjEuMCAoV2luZG93cykiIHhtcDpDcmVhdGVEYXRlPSIyMDE5LTEyLTAzVDAxOjE4OjIyKzAxOjAwIiB4bXA6TW9kaWZ5RGF0ZT0iMjAxOS0xMi0wM1QwMToyMDoxMCswMTowMCIgeG1wOk1ldGFkYXRhRGF0ZT0iMjAxOS0xMi0wM1QwMToyMDoxMCswMTowMCIgZGM6Zm9ybWF0PSJpbWFnZS9wbmciIHBob3Rvc2hvcDpDb2xvck1vZGU9IjMiIHBob3Rvc2hvcDpJQ0NQcm9maWxlPSJzUkdCIElFQzYxOTY2LTIuMSIgeG1wTU06SW5zdGFuY2VJRD0ieG1wLmlpZDplZGViMzU3MC1iNmZjLWQyNDQtYTExZi0yMjc5YmY4NDNhYTAiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6ZWRlYjM1NzAtYjZmYy1kMjQ0LWExMWYtMjI3OWJmODQzYWEwIiB4bXBNTTpPcmlnaW5hbERvY3VtZW50SUQ9InhtcC5kaWQ6ZWRlYjM1NzAtYjZmYy1kMjQ0LWExMWYtMjI3OWJmODQzYWEwIj4gPHhtcE1NOkhpc3Rvcnk+IDxyZGY6U2VxPiA8cmRmOmxpIHN0RXZ0OmFjdGlvbj0iY3JlYXRlZCIgc3RFdnQ6aW5zdGFuY2VJRD0ieG1wLmlpZDplZGViMzU3MC1iNmZjLWQyNDQtYTExZi0yMjc5YmY4NDNhYTAiIHN0RXZ0OndoZW49IjIwMTktMTItMDNUMDE6MTg6MjIrMDE6MDAiIHN0RXZ0OnNvZnR3YXJlQWdlbnQ9IkFkb2JlIFBob3Rvc2hvcCAyMS4wIChXaW5kb3dzKSIvPiA8L3JkZjpTZXE+IDwveG1wTU06SGlzdG9yeT4gPC9yZGY6RGVzY3JpcHRpb24+IDwvcmRmOlJERj4gPC94OnhtcG1ldGE+IDw/eHBhY2tldCBlbmQ9InIiPz4JZNR8AAAfG0lEQVR4nO2deViTZ7r/7yxkJaxJ2MK+GCBAMCwS1kgUFQSKK4XWWqsz1jpjp3b0tDP1V+eqU391fqfT/mpPPd20drTFDS0KFEVWJSGAEgLIZpAICBJACIRs549Rj1WILAkBfD/XlevySp68z/0S3+/7vPdzLyidTgcLkU2bd+z39/f/q1gshsrKSoJELFCa2iaEuU9K6kb+8uXxv54/fzE8L/eswNT2zCfQpjbAGKS8lPFKSEjIXiaTCSEhIeDj4xNnapsQ5j6rktZGp6UlfxIdzQVzCplmanvmG1hTG2BIAtlc26CgoDfT0tL2e3l5AQCAjY0NkMnk/a9s2k6rrKw8UV8n1JjYTIQ5RlAw14KzmL3xze1vfJyUuMJaq9UCFovFm9qu+YbBxcSPFUYkk8l2Q0NDsvo6ocrQx5+I8Ih4bz6f/0l8fHyKlZXV4/dRKBQwmcwwMpn8A4FAoPgHhH9bV1sxa488wZxoaycnJ/a9e/duCa5fkc3WvAiTI4Ib77p+XdqHG9anbfLy8gAAgLGxMdBpF+bjvzExqJj4scKI0dHRnwQHB++orq7+AgDeMuTxJ2Jl4rqU9PT0EwEBAUQCgTDuGAaDAampqYepVKpHUHDk325Ulw0a266YuFW+Gzdu/MDPz29jfn7+XgA4aOw5ESZP6kvpCXv3vnM8NiaSamVl+fj9BepGNDoGFRN7e/slcXFxO1xcXMDJyWnH7j//H/fi4uJdgutXmgw5z5O8smn7X9euXbvf29sbMBjMhONQKBRYWVlBbGzsbjMzM3JoOG+/sKKwy1h2rd/4elpGRsYuLy+vaDweD2w2Oy1h5ZrCvEunEaeeiVnMiabyl/F2/+X9P+8JDPQHHA5napMWBAYTk6DgSNuEhIS9DAYDAP7tq1i6dOkqOp3OWbNu0wens44emeoxA9lcWwKBYEMkEm2JRKIdHo+3QKFQWJ1Op8ZgMER3d/dVq1evTnFycpr0MSkUCsTExGzH4/Gk1LTME/39/TI0Go1FoVCg1WrVY2NjipGRkcGRkRH5dPwrEZHLXMPCwjJSUlIy3dzcfB+97+rqGhYSEpIOAIiYmBguN3zL77dt3uPh4W5qUxYUBhMTb2/vjeHh4cvR6P/dILK0tITIyEg7BweHr363/Z3Ampqaf1Zcu/zMKiVsyVJvMplsRyKR7IhEor2FhYUbhUJhJCYm2pFIJB6JRAIymQx4PB7QaDRoNBowMzMDJycnwOOn7icjEokQGxu7icFgbLp///7jFY1WqwWlUgkjIyOgUCgO7Ni5Rz48PCwfHh7uGRkZeaBQKOSjo6ODCoVCXlNVKn/6uCsT13FXrVr1emho6BYKhfLMnP7+/omrU9LPX8g+UThloxEMxqJFXjxESAyPQcSEExrLWLNmzW57e/txP/fw8ABHR8cdDAaDt3xF2ru9vb03sVgs0cbGxs/FxWVZUlISj0aj+dna2oKtrS1M5PcwJCgUCry8vODRrs84vPfoH6OjoyCXy6Gvr+/R6+CWrX9s7evrk/b19bWr1Wqli4sLZ8OGDe95eXmxUSjUuAd0cHDwjoqK2sYKXFIhvnldYYTTQpgU4/8+jyASCYDGoCd+ZkYYF8OICYezl8PhuOkbQyAQIDo62s/NzS2np6cHbGxsgEajAYFAAAwGA1gsFia6CE0NgUAABwcHsLe3B61WC2q1eo9WqwWNRgNKpRLUajUQiUSgUCh6zwGHwwGTydzo5+eXBQBnZu8MEJ5keHhYPqyYWMtHR0ZBpVIhYj9FUDONgOUvT12+du3avMDAQJjssdRqNWCxCyrEZdLodDoQi8Ulx44de628NL/V1Pa8iERE8l2dHB2CJvpcq9Nqbt1qKURWj1Njxld0ZGTkAW9v70kLCQC8sEIC8O/HKx8fn2gmk8kHgCk7pRFmzrWyAikASE1tx0Jj2uH0EZHL/N7YtuvT4OBgzmz4OBYSeDweIiMjt2S++vtMP1YYEmmJsCCY8mNOIJtr6+zsHBcZGXmIw+G4mZubG8m0hU9HRwcUFxe/KxQKTyDRsQjznSmJCS9+dVRERMTfQ0NDo2xtbfUGiSFMjtHRUaitrc3Jzc09kHvxVLmp7UFAmC6oZQkvrZLL5RJhReHtiQb5scKIXC7371FRUX90dnYGIpE4JR8Jgn40Gg20t7fXFxYWfnr9+vWjz8sdYi+Osh4vzgUBwZSgtu94V+fs7Hx7YGCgra6u7khLS0u2RCwYeTQgKmYFh8fj/f/g4OAldnZ2prR1wdPd3Q1CofBQSUnJkdLi3N8E93FCY6k+Pj48FxcXjlar1ZSWlh65VvYr4kREmDNg79+/D3FxcW5OTk5uXl5evNbW1tL0jK3ZXV1d1ykUintycvInoaGhdkj+gvGxs7MDPp+/m0AgWMQvS/lyeHhYTqPRPJycnIJSU1NZ3t7eW2g0Gly/fv2oWq1Gij0hzClQ/gHhpLS0tEM8Hm/7I8Ho7++HlpYWsLa2Bg8PDxOb+OKhUCigqakJ7t+/D25ubuDu7g4oFAp0Oh08ePAAvv7666TTWUdzTG0nAsKTYMU3ryuSU18+4+bmFrZo0SIOAICVlRUsXrx4zkakLnRIJBI8CgJ8MtdJp9NBZ2enqL29XWRC8xAQxgUNAHD+3L8KGhoaCp78ABES04JCoX4jJAAAAwMDUFtbe96YpRMQEKbL41DU5ubmko6Ojj2PSgggzD36+/vrb9y4cX425zzw93/8EBjon2is44+NjSkePBjqGRwc7G5v7xBV19w8U5B/3qgrr9+/uWtXUuKKD/TZ9MXh/066/OuFmunO8dGBQ98HBbGSp/t9U6LRaDXK0dHBoeFhuVzeL22/0yFqamopufjLqRJ933ssJi0tLSXV1dWHGAzGbuObOzs8ubqa71vZKpUKOjo6blwpOF8zm/Mu5cVkLlkSaswprAHAaVihgK7O7oSGxltvfXLon3nXK4RHT2cdN4pfKDCAlZyUuMJan02nTmczAaBmunPw4qI3cbnh0/36XICq0+lgcPABp7OrK629vUP5z8++LLh2XXD05L++yxrvC4/F5EZ12WBS8saLS5Ys2U2lUufUY45SqQSlUgkqlQrUavXj19jYGGg0GtBoNKDT6UCn05VotVq1TqfToFAojFar1eh0Og0Wi8XhcDgeGo1+/PhgZmYGOBwOsFgsmJmZ/eY1F+nt7YXa2trs2Z73wdCQBgCMHp1IJpHA09MdPD3dLRIS+OtKisvWvbP7vf2lZdePVFwzbHTwyMiI3hidkZFRUKvUYzOZ48HQkBIA5nWqBAqFAktLC7C0tADmIh88Pz4uMSyUk7hn776DV4tKPn/6d/lNxp1MJqsRCASf8vn8XdMpOjRTVCoVjI2NgUqlAq1WCyMjI9DX1wf379+Hvr6+/Q8ePOgdGRmRKxSKx0WLFAqFXKlUKnQ6nUar1arHq47mxwrD4/F4Eg6HI2GxWDwej7cgkUjWFAqFam5uTjU3N6eRyeQPLSwswNraGqysrIBAIDwWFywW+zja11Qi29LSclIikeSZZPJZBovBAI8XA8HBQR9kZZ3lR8cmvFZSlGe00p8IkwONRkNERBj4+i7a4+XpHv307/IbMakWlciXJbx0nMPh7Jqo0JGh0el0MDo6Cl1dXSCVSkEmk7177969W319fe1DQ0M9KpVKoVarlWq1WjndNhUPG3ApAWDcOxLTLwSDwWAOotFoDBaLxRMIBAsrKysne3t7Xzqd7k2n0/c4OzsDlUoFHA4364IyMDAATU1NxdWikhcq6tXKyhJezljPJZKI2eERS5cZeoWCMD2srCwhPX0tVzk2djiCG//GtfLLUoBxShB0dHTU3Lx580sLC4vtJBLJKMZoNBqQSqUglUqPdnR01PT09DT19/fLHjx40DM0NNQ72933GiSVGgB4JFQK+LfoSAGgnL04yppEIh2xtLS0t7GxcaFSqR7Ozs4fMRgMcHR0nJX8pJs3b54Ui8UXjT7RHIRMIkFK8irfwcEHPwQELUmqvYHUGJkLmJubw8YNa/i9vfffY/px3myQiDTPiEl9nVDDX576jaenZ7SnpyfLUJNrNBqQyWRw+/bt4x0dHTdkMlltV1dXw/XygjkdEv4wB0YOAK0AUM70C8HQ6fSzdDrdm0qlejg6OrLc3Ny2MBiMadWjfR4PHjyAmzdvZs/1v5MxoVAokJK8iicWS95k+nH+s0EiQhqpzQGoVFtYk5a87ba0XQAA34xbpagg/5zoT7s/OGNnZ8eaaYkBuVwOnZ2d5VKpVNTS0lLS2NhYWFVZ3Dujg5qQh6uY+ocvCAiKIPn4+Jz19PSMdnV15VCpVL6Dg4NBViw6nQ5EItHRpqamqzM+2DzHzo4O69amftLQeKsAZrDLgmBY/PyYsCIhfs+SiKUFE5Y8EwqFx11cXDihoaFTjjFAoVAwPDwMHR0dourq6jNCofDHhZqUVnvjmgIAcgAgJyg40mLRokX8kJCQjT4+PussLS1n1JPl7t27UFxcfHguB6mNjY2B7G4naNRTWyygUCjAYDGAx+PB0sICSCSi3vFYLBbCwjjA8vddBQtATKb7d3saBwc7IJPJBpsHjUGDGRYLJBIJLK0sAfucmyIGg4FFi3y8AwNZtycUk5KiS02vvf7WWQaDkejg4DApQwAeh3xDaWnpPoFAcPxFqnP6sEvgGf+A8Bx3d/cvIyIiNi1evHjT8wpNj8fAwACUlZW9P9dD5+/ckcFbf9gd2dcnn9LNAovF4inmZHtXNxdOdBR3+/JlS33pdP29wolEInA4weuiYxOy5vvuTkeHDHb+8c8xvb33Z3R9/N+Df+uIjYk02DwkEsna2trS1d/fNyGeF7uTyw1/7g3R3t4O2OxA/TVghULhcQqFQk1JSfmYSNR/5wD4d6EfgUBwvLS09IhUKhW9qAV5H9YjKQwJi6uvrKw8ERoamhkSEpKp7w7yJEqlEiQSyZmysrJv53qjdaVSCZdyTk+3qFMrAJRHRPLPN95qeifj5fU7mYt8JhyMRqMhMJDFdnF25gDAvBYTpXIMWlpay2fq/8m5mDcIABYGnEcGAGI/VlhBZWX1yZdSkz55OX0dV5+7w9bGGvz8mPrFpK62QskJjf2GTqd7x8bGbpnID4BCoUAmk0lLSkqOiESik2UleS/MakQflYKrXQDQxY1a3tTe3i6KiIjY5OXlxX7e9+rr6wsuXbr0t4ffn9OgMWjghMZQRcLp+8GulRVI/QPC37Wxtnal0ajJtjY2E451ZjiBra31vE9lR2PQQKFQaAAwo98Yi8Xq9fpPd56HO6rlvKWJv/PwcK+JilyCmajWMw6HAzs7+rMFpQOCIn6zHywSFvXm5eUdFAqFZ9Rq9bgHa2trq79w4cK+zz49cAARkmcpL81v/a/Dhz49d+7c3qqqqjyVSjXuOJ1OBxKJpDw3N/fA5V+zax6978cKw/sHhM/raMrnUVdboSy4fPWQSFSjd5yFBQWIRNKEd2IEw1J4JUd88WL+R51d3XrHWVDMnxUTa2tr1zXrNiUGsrmPf7DS4tymCxcu7Kuurs55+kKQSqVN586d23vs+8NHDXUCC5Wzp3/Iy8rKeruysvLM2Nhvo7VVKhXU1tYWnj17du/T7UOdnZ2D7OzsfGGB09raVi4S1RzXl0eFw+EAj8chYjKLVFffyOrq1C8mJBLpWTFRKBRyDofzC4vFWvXk+1ev/CLOzs7eKxAIslQqFeh0Oujp6enKzs7em/XTd7OayTqfKb56sT4rK+sPAoHg5KO/o0KhAKFQmHXy5MkdF3/5+TeZmctXpIXZ29v7zqVcKWNRX1epuXu3U/y8pEw0GmndOZt0dnXVDw0P6/W5oNHoZ30mQ0NDPb29vfvj4+Pf3rR5B/7od188XnEUXr4gDgmL+0NfX5/U19d3d3l5+YGfTnyDtLmcIhXXLsu4UcvfR6PRGGtra9eysrIjYrE45+kt4Fheou/69es/unnz5vm7d+/Wmsre2WRkZGTQ1DYg/JYGiUiTm1ugBAC9IfHPiEmDpFITE7fqJI/H27lmzZpDq5LWtz55t6wUXO3ihMYerK+vz2tpaUFaM0yT8tL81ujYle+TSCTrvEunBU9/voTLd92wYcPHVCqV39XVdXCu7+oYCp1O90Kc50Jk3I5+xVcv1jc3N5d4enpSMzIyvkpK3sh78nORsKg3++yPBS/q1q+hKCm61DSekERGJ3ikp6d/ERsbm1xVVXWwtbX1hRFtFAqFPMLMUyZsDyoQCI7LZDKIiIjwzczM/GpV0vro2TTsRSUqZoX3+vXrP1u9enXi0NAQiESirIdRtggIc5oJ40zq6uryGhoa8ry8vBJCQ0O9USjU94mrN7yWc+EnvaXb5gJMvxCMp6cnl0Kh2Le1tZVXXLs8L1LXefGrWRkZGZ/x+XyeUqkEkUh0vqenZ14HZyG8OEwoJjdrygd37NxTEBkZmWBtbQ3BwcEeKBTq+/UbX3/355Pfzlmn66qk9dGbN29+k8PhbCSRSNDZ2Snb9ae/HCkpKTksEhbN2QTD5NSX+Vu3bj0cHBzsjcFg4O7du1BWVvbNwxB9BIQ5j94I2Fu3bhXW19cDl8sFLBYLHA7Hg0wmf/e77e84ffXlPz6fLSMnQ2paZkJ4eHjmtm3b+B4eHvZkMhlQKBTY29s72dvbfxgUFJT8x7ffP1NRUfHjXErnZ/qFYKKjo7dt3rz5g8DAQPtH/XHa2tpqGhsbC55/BASEuYFeMblz505NTU3NgfDw8PcwGAygUCjw9fW1IJPJn/1130Hv0tLSI4WXL4hny9inYS+Osvbz80tgMpn8jIwMPovFch2vpoiDgwM4ODhwfH19OYsWLeJv3/Hu+cbGxquzXZz5aZYlvMRJT0/fFhkZue3JZmfd3d0gEolOIr4ShPmEXjFpkFRqXlrzSnFnZ+d7Tk5OjzNfXVxcICMjY6ezszNnVdL6vU8HWhmbgKAIkrOzMyc1NTXz0YU4maAuOp0OK1as4EVFRfGEQqHg1dfePHzr1q2rs71S8WOF4f38/BLS09M/iIyM5DxdxLq5uVlcVVU1bgVwBIS5il4xAQCQyWRigUBwJikpKe3JVGQcDgdLly7l2tranti0ecf7IpEoy9hbxX6sMDydTvdevXr1ltjY2F3u7u6AxT73FJ7B3Nwc4uLiwthsdphQKCzZkL7l0/r6+oKbNeVG90+EhMXZL1++fFtycvKHrq6uz4igUqmE5ubmEiTHCWG+8dwrUXD9imz9xtd/jIuLS7N5KpsTjUZDUFCQE4PB+F4oFGYmJW888Mv5k4UTHGpGxC9LYaenp78VEhKyxdHRESgUyoyOh0KhwNraGuLi4qIDAgKi6+rqyjekb/mHMSN6N6RvSdu+ffseNpsdZm09ftuW+vp6EIvFSB9hhHnHpG7rUqm0orW1tdXS0tLj6TIEaDQaaDQaxMfH811dXTl/3Xfw+JUrVz411J01cfWG6IiIiC07d+5McHNzs7ewMGyOFw6HAwcHB6BSqVx3d/fwz7/4rkAgEBwXCoUnHpZonDGrU9J5MTEx27du3Zrm4uKC0beaqq6u/ry+vj7XEPMiIMwmkxKTimuXZe/u+fCkp6fnexPdUfF4PPj7+1szGIydLi4unF1/+kvenTt3RG1tbRXTqfma8lIG39/fP/HVV19NZrFYHpMpzjQTzMzMwNPTE+Pp6Zng6emZ4Ofnl5CesfV8bW1tznQe3/wDwvFeXl7Rvr6+Ca+88kpaUFCQh74GXzqdDrq7u6GpqankRQmdR1hYTNrhUFVVlcXj8d6ysrKy0OfstLS0hPj4eC6Xy+U2NzeDRCI5/sa2XeX37t1rGhwc7BoYGJBN1P+FFbiE5OzszGaxWImvvvrqpoCAAKfp+ERmCpPJBCaTmcnhcDJLS0u/TE59+YxUKhXoi/lg+oVgrKysGJaWlna2trYeaWlpXDabvTMgIGDSfp2KiorzbW1tL0zoPMLCYtJX6uVfs2u++PKowMPDgz+ZIslEIhECAgKAxWJlajSazJ6eHmhra4PW1tZvtmz9o6Czs7O+r6+vfWxsbFir1WosLCzsV6xYkcnj8d7z9vaelmPV0Hh5eYGnp+f2mJiY7UVFRZ/HL0v5tru7+5ZGo1FisVg8Docj4fF4CxsbG1c+nx/m7e39sYeHB7i4uIC5ufmU6r4ODQ1BZWXlifkSrYuA8DRTumIrKytPent78728vCb9HRQKBVgsFhwcHIBOpwObzd4yNja2RaVSwdDQEHR1dcHo6CjQaDRwdXWdsWPV0KBQKPDw8AA7O7udERERO2tra2FgYACoVCo4OTkBjUYDMpkMeDz+8WuqaLVaaGxsbL19+/YzSX8ICPOFqYrJidDQ0AwvLy/e80c/CwaDARKJBI86BdJoNHB3dwe1Wj0nViL6IJPJwGQywdnZGZRKJRAIBDBUx8OBgQEoLS39BtkORpjPTJg1PB61N64pmpqarvb39xvUiLkuJE9CJpPBxsbGYEICANDZ2SlHgtQQ5jtTEhMAgLq6ulyJRFJvDGNeREZGRkAikRSUFuci2cEI85opi0l+7hmBWCzOeV6dToTJcfv27cHr168jxbgR5j1TFhMAgObm5hKZDNl0MAQtLS3Xzpw6hkS8Isx7piUmUqlUIBAIJuyjgzA5Ojs7QSKRINGuCAuCaYmJsKKw68qVK59KJJIu5HFneiiVSigqKjouEolOmtoWBARDMC0xAQC4+MvPJadOnXq3ra1N8yL0dDEkOp0OSktLy/Pz8w8+3d4CAWG+Mm0xAQA4fuy/jl+8ePGju3fvGsqeBY9Wq4XKysrWU6dOvX31yi8mKyyFgGBoZiQmAAD/79D+fadPn96PCMrz0el0UFVV1frtt9+mj9fiAgFhPjNjMQEAyMvLO3Ds2LE/tLS0INmuerh27Vr9999//xoiJAgLEYOEntbVVigB4PNNm3cMpqSkfMRms50McdyFgkqlgqKiovJTp069nZ97BhEShAWJQePYj373xdF1GzbLFQrFx6Ghob766ne8KNy7dw+KiopO5ubmfmTK4tsICMbG4EkxWT99d35l4rre/v7+D0NCQvh0Ot3QU8wL1Go1SKVSTX5+/sH8/PyDSP8bhIWOUTLsLuVklQcFR65pbGzcvnLlyvfc3NwsCASCMaaac+h0OhgaGoLq6uqaCxcu/OV01tGcTw7uM7VZCAhGx2jpug/vxAd58atzoqKitq1cuXKnvb29saabE+h0Oqiurpbm5eUdrK6uPlspuDrvY0hmO4YIhUIBGq1/X2CmNqFQKL3/79HomZ/z82xEowyy9zFr80zGDqPn/hdeviBmL47ad+fOnRsRERGbQkNDo62srIw97azT2dkJxcXFx0tKSo7Mdh8hY4LD4TDPH2U4MFjMc6tLmZmZzaj+Aw6H0/t9PB4PGCxmRudNJBL0ngeZTAI0Gj3jv+1szfM88Hic8cUEAKCmqlQOAN/ELU2qkEgkySwWK3HRokVcBoMxG9MbDZ1OB83NzdDU1FRQW1t7XiAQHJ+ovu18pbr6Rg6L5ZtoM0EhcUPT0tJW8tWRb0vQqIkvgKqqmhnVfrl2TfANXo+gjKlUio4OWc1M5sjOzjnQUH8rbqLPu3t6moaGhmfc+3q25tGHUqmECoEIUKbIrVkcEkONiIh4jcvlvu7s7OxLo9GmVe7QVCgUCujq6oKGhoaCioqKo9XV1WeM3YDMVPDik1gpyas+XrVyeaKXl8czjyANjbcgI/MNmkg49Q4ECPOH3NyC4RUr+M8IcHt7B1y9WlKRl3/5kElKnD1sfXEoJCzueEBAQGJYWFgGk8nk2djYAIFAgLm4pTw6Ogqjo6Mgl8vhxo0b50tLS4/U19fnLvS2FIWXfxEDQNLmLW9ueW1TxtchHDaQyWRTm4VgYkZHR6G+vhF+/NfP+y5e+vVjiVgwZpKVydOwF0dZW1lZOTGZTD6bzU4LCAiIptPp8HTDL1MwOjoKLS0tUFdXd1IsFudIpdKKgYGB7tloJTrX4MUnsVJTEj9etzY10dHRAQAAGm81wcsZW5CVyQInL69gNCGBjwcAGBx8ANnncypOnTr3H9nn/reD55wovvrQpyIHAHFUzIocGo3mQaPRfBwdHVlubm7bXF1dgcFgABqNNvruglwuh7t374JMJoOOjo7P79y5I+ru7m7q7e1tXQi7MzOh8PIv4pCw2DdaWtte37Au7aPIyCWAxWABjUbPif9HCMbjURtKiaQBfvr5zH9evlJ0uLQ4r/nJMXNiZTIRrMAlJAcHB18HBweWo6Mjy8rKajeJRAJLS0uwtLQECwsLoFAogMfjAYvFgpmZ2XNXMyqVCoaHh2FoaAiGh4cfvwYGBqCvrw+6u7vfvnfvXlNvb29rT09Pq0QsUM7S6c4rNqS/lrZ5U+YPRBKR9M7u9xwqBUUvtNAudH766XSLE8PR49ixE78/8tVnX403Zk7fUR46NUUAIPIPCMdTKJTdNjY2QKPRgE6nA51OB1tbWyCRSIDD4YBAIAAejwcCgfDYUajVakGlUoFarQadTvfY79HX1wf9/f0gl8tBLpfDvXv3HvXw+dxQPYYXMj+d+P7Mmzv+5OHr6/OJWq1GBHeB09TcUiKuq/coKS3/eqIx/wPkiIXC3w6YjAAAAABJRU5ErkJggg=="); + + --keyword: #ff79c6; + --identifier: #f8f8f2; + --comment: #6272a4; + --operator: #ff79c6; + --punctuation: #f8f8f2; + --other: #f8f8f2; + --escapeSequence: #bd93f9; + --number: #bd93f9; + --literal: #f1fa8c; + --program: #9090c0; + --option: #90b010; + --raw-data: #8be9fd; + + --clipboard-image-normal: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: lightgray' fill='none' viewBox='0 0 24 24' stroke='currentColor'%3E %3Cpath stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2' /%3E %3C/svg%3E"); + --clipboard-image-selected: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' style='color: lightgray' viewBox='0 0 20 20' fill='currentColor'%3E %3Cpath d='M8 3a1 1 0 011-1h2a1 1 0 110 2H9a1 1 0 01-1-1z' /%3E %3Cpath d='M6 3a2 2 0 00-2 2v11a2 2 0 002 2h8a2 2 0 002-2V5a2 2 0 00-2-2 3 3 0 01-3 3H9a3 3 0 01-3-3z' /%3E %3C/svg%3E"); + --clipboard-image: var(--clipboard-image-normal); + } } -.slider.round:before { - border-radius: 50%; +.theme-select-wrapper { + display: flex; + align-items: center; } html { @@ -147,24 +141,39 @@ body { padding: 0; box-sizing: border-box; } -.column, -.columns { +.column, .columns { width: 100%; float: left; box-sizing: border-box; - margin-left: 1%; + margin-left: 1%; } + +@media print { + #global-links, .link-seesrc, .theme-switch-wrapper, #searchInputDiv, .search-groupby { + display:none; + } + .columns { + width:100% !important; + } } -.column:first-child, -.columns:first-child { +.column:first-child, .columns:first-child { margin-left: 0; } +.container .row { + display: flex; } + .three.columns { - width: 22%; + width: 25.0%; + height: 100vh; + position: sticky; + top: 0px; + overflow-y: auto; + padding: 2px; } .nine.columns { - width: 77.0%; } + width: 75.0%; + padding-left: 1.5em; } .twelve.columns { width: 100%; @@ -230,6 +239,12 @@ select:focus { } /* Docgen styles */ + +:target { + border: 2px solid #B5651D; + border-style: dotted; +} + /* Links */ a { color: var(--anchor); @@ -245,27 +260,32 @@ a.reference-toplevel { font-weight: bold; } +a.nimdoc { + word-spacing: 0.3em; +} + a.toc-backref { text-decoration: none; - color: var(--text); } + color: var(--text); +} a.link-seesrc { color: #607c9f; font-size: 0.9em; - font-style: italic; } + font-style: italic; +} -a:hover, -a:focus { +a:hover, a:focus { color: var(--anchor-focus); - text-decoration: underline; } + text-decoration: underline; +} a:hover span.Identifier { color: var(--anchor); } -sub, -sup { +sub, sup { position: relative; font-size: 75%; line-height: 0; @@ -292,8 +312,7 @@ img { background: transparent !important; box-shadow: none !important; } - a, - a:visited { + a, a:visited { text-decoration: underline; } a[href]:after { @@ -307,16 +326,14 @@ img { a[href^="#"]:after { content: ""; } - pre, - blockquote { + pre, blockquote { border: 1px solid #999; page-break-inside: avoid; } thead { display: table-header-group; } - tr, - img { + tr, img { page-break-inside: avoid; } img { @@ -331,22 +348,18 @@ img { h1.title { page-break-before: avoid; } - p, - h2, - h3 { + p, h2, h3 { orphans: 3; widows: 3; } - h2, - h3 { + h2, h3 { page-break-after: avoid; } } p { margin-top: 0.5em; - margin-bottom: 0.5em; -} + margin-bottom: 0.5em; } small { font-size: 85%; } @@ -354,8 +367,7 @@ small { strong { font-weight: 600; font-size: 0.95em; - color: var(--strong); -} + color: var(--strong); } em { font-style: italic; } @@ -376,8 +388,7 @@ h1.title { text-align: center; font-weight: 900; margin-top: 0.75em; - margin-bottom: 0em; -} + margin-bottom: 0em; } h2 { font-size: 1.3em; @@ -404,36 +415,29 @@ h6 { font-size: 1.1em; } -ul, -ol { +ul, ol { padding: 0; margin-top: 0.5em; margin-left: 0.75em; } -ul ul, -ul ol, -ol ol, -ol ul { +ul ul, ul ol, ol ol, ol ul { margin-bottom: 0; margin-left: 1.25em; } ul.simple > li { - list-style-type: circle; -} + list-style-type: circle; } ul.simple-boot li { - list-style-type: none; - margin-left: 0em; - margin-bottom: 0.5em; -} + list-style-type: none; + margin-left: 0em; + margin-bottom: 0.5em; } ol.simple > li, ul.simple > li { margin-bottom: 0.2em; margin-left: 0.4em } ul.simple.simple-toc > li { - margin-top: 1em; -} + margin-top: 1em; } ul.simple-toc { list-style: none; @@ -442,8 +446,7 @@ ul.simple-toc { margin-top: 1em; } ul.simple-toc > li { - list-style-type: none; -} + list-style-type: none; } ul.simple-toc-section { list-style-type: circle; @@ -453,12 +456,10 @@ ul.simple-toc-section { ul.nested-toc-section { list-style-type: circle; margin-left: -0.75em; - color: var(--text); -} + color: var(--text); } ul.nested-toc-section > li { - margin-left: 1.25em; -} + margin-left: 1.25em; } ol.arabic { @@ -505,12 +506,39 @@ hr.footnote { margin-top: 0.15em; } div.footnote-group { - margin-left: 1em; } + margin-left: 1em; +} div.footnote-label { display: inline-block; min-width: 1.7em; } +div.option-list { + border: 0.1em solid var(--border); +} +div.option-list-item { + padding-left: 12em; + padding-right: 0; + padding-bottom: 0.3em; + padding-top: 0.3em; +} +div.odd { + background-color: var(--secondary-background); +} +div.option-list-label { + margin-left: -11.5em; + margin-right: 0em; + min-width: 11.5em; + display: inline-block; + vertical-align: top; +} +div.option-list-description { + width: calc(100% - 1em); + padding-left: 1em; + padding-right: 0; + display: inline-block; +} + blockquote { font-size: 0.9em; font-style: italic; @@ -519,7 +547,12 @@ blockquote { border-left: 5px solid #bbc; } -.pre { +blockquote.markdown-quote { + font-size: 0.9rem; /* use rem to avoid recursion */ + font-style: normal; +} + +.pre, span.tok { font-family: "Source Code Pro", Monaco, Menlo, Consolas, "Courier New", monospace; font-weight: 500; font-size: 0.85em; @@ -530,6 +563,16 @@ blockquote { border-radius: 4px; } +span.tok { + border: 1px solid #808080; + padding-bottom: 0.1em; + margin-right: 0.2em; +} + +.copyToClipBoard { + position: relative; +} + pre { font-family: "Source Code Pro", Monaco, Menlo, Consolas, "Courier New", monospace; color: var(--text); @@ -548,7 +591,25 @@ pre { border: 1px solid var(--border); -webkit-border-radius: 6px; -moz-border-radius: 6px; - border-radius: 6px; } + border-radius: 6px; +} + +.copyToClipBoardBtn { + visibility: hidden; + position: absolute; + width: 24px; + border-radius: 4px; + background-image: var(--clipboard-image); + right: 5px; + top: 13px; + background-color: var(--secondary-background); + padding: 11px; + border: 0; +} + +.copyToClipBoard:hover .copyToClipBoardBtn { + visibility: visible; +} .pre-scrollable { max-height: 340px; @@ -562,8 +623,8 @@ pre { table.line-nums-table { border-radius: 4px; - border: 1px solid #cccccc; - background-color: ghostwhite; + border: 1px solid var(--border); + background-color: var(--secondary-background); border-collapse: separate; margin-top: 15px; margin-bottom: 25px; } @@ -599,6 +660,9 @@ table { border-collapse: collapse; border-color: var(--third-background); border-spacing: 0; +} + +table:not(.line-nums-table) { font-size: 0.9em; } @@ -613,11 +677,11 @@ table th { font-weight: bold; } table th.docinfo-name { - background-color: transparent; - text-align: right; + background-color: transparent; + text-align: right; } -table tr:hover { +table:not(.line-nums-table) tr:hover { background-color: var(--third-background); } @@ -631,31 +695,31 @@ table.borderless td, table.borderless th { padding: 0 0.5em 0 0 !important; } .admonition { - padding: 0.3em; - background-color: var(--secondary-background); - border-left: 0.4em solid #7f7f84; - margin-bottom: 0.5em; - -webkit-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); - -moz-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); - box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); + padding: 0.3em; + background-color: var(--secondary-background); + border-left: 0.4em solid #7f7f84; + margin-bottom: 0.5em; + -webkit-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); + -moz-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); + box-shadow: 0 5px 8px -6px rgba(0,0,0,.2); } .admonition-info { - border-color: var(--info-background); + border-color: var(--info-background); } .admonition-info-text { - color: var(--info-background); + color: var(--info-background); } .admonition-warning { - border-color: var(--warning-background); + border-color: var(--warning-background); } .admonition-warning-text { - color: var(--warning-background); + color: var(--warning-background); } .admonition-error { - border-color: var(--error-background); + border-color: var(--error-background); } .admonition-error-text { - color: var(--error-background); + color: var(--error-background); } .first { @@ -689,8 +753,7 @@ div.footer, div.header { font-size: smaller; } div.footer { - padding-top: 5em; -} + padding-top: 5em; } div.line-block { display: block; @@ -707,19 +770,23 @@ div.topic { div.search_results { background-color: var(--third-background); - margin: 3em; + margin: 3vh 5vw; padding: 1em; border: 1px solid #4d4d4d; -} + position: fixed; + top: 10px; + isolation: isolate; + max-width: calc(100vw - 6em); + z-index: 1; + max-height: calc(100vh - 6em); + overflow-y: scroll;} div#global-links ul { margin-left: 0; - list-style-type: none; -} + list-style-type: none; } div#global-links > simple-boot { - margin-left: 3em; -} + margin-left: 3em; } hr.docutils { width: 75%; } @@ -817,9 +884,6 @@ span.classifier { span.classifier-delimiter { font-weight: bold; } -span.option { - white-space: nowrap; } - span.problematic { color: #b30000; } @@ -899,8 +963,29 @@ span.Preprocessor { span.Directive { color: #252dbe; } -span.Command, span.Rule, span.Hyperlink, span.Label, span.Reference, -span.Other { +span.option { + font-weight: bold; + font-family: "Source Code Pro", Monaco, Menlo, Consolas, "Courier New", monospace; + color: var(--option); } + +span.Prompt { + font-weight: bold; + color: red; } + +span.ProgramOutput { + font-weight: bold; + color: #808080; } + +span.program { + font-weight: bold; + color: var(--program); + text-decoration: underline; + text-decoration-color: var(--hint); + text-decoration-thickness: 0.05em; + text-underline-offset: 0.15em; } + +span.Command, span.Rule, span.Hyperlink, +span.Label, span.Reference, span.Other { color: var(--other); } /* Pop type, const, proc, and iterator defs in nim def blocks */ @@ -938,17 +1023,14 @@ span.pragmadots { border-radius: 4px; margin: 0 2px; cursor: pointer; - font-size: 0.8em; -} + font-size: 0.8em; } span.pragmadots:hover { - background-color: var(--hint); -} + background-color: var(--hint); } + span.pragmawrap { - display: none; -} + display: none; } span.attachedType { display: none; - visibility: hidden; -} + visibility: hidden; } diff --git a/doc/nimfix.rst b/doc/nimfix.rst deleted file mode 100644 index d105346da..000000000 --- a/doc/nimfix.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. default-role:: code - -===================== - Nimfix User Guide -===================== - -:Author: Andreas Rumpf -:Version: |nimversion| - -**WARNING**: Nimfix is currently beta-quality. - -Nimfix is a tool to help you upgrade from Nimrod (<= version 0.9.6) to -Nim (=> version 0.10.0). - -It performs 3 different actions: - -1. It makes your code case consistent. -2. It renames every symbol that has a deprecation rule. So if a module has a - rule `{.deprecated: [TFoo: Foo].}` then `TFoo` is replaced by `Foo`. -3. It can also check that your identifiers adhere to the official style guide - and optionally modify them to do so (via `--styleCheck:auto`). - -Note that `nimfix` defaults to **overwrite** your code unless you -use `--overwriteFiles:off`! But hey, if you do not use a version control -system by this day and age, your project is already in big trouble. - - -Installation ------------- - -Nimfix is part of the compiler distribution. Compile via:: - - nim c compiler/nimfix/nimfix.nim - mv compiler/nimfix/nimfix bin - -Or on windows:: - - nim c compiler\nimfix\nimfix.nim - move compiler\nimfix\nimfix.exe bin - -Usage ------ - -Usage: - nimfix [options] projectfile.nim - -Options: - - --overwriteFiles:on|off overwrite the original nim files. DEFAULT is ON! - --wholeProject overwrite every processed file. - --checkExtern:on|off style check also extern names - --styleCheck:on|off|auto performs style checking for identifiers - and suggests an alternative spelling; - 'auto' corrects the spelling. - -In addition, all command line options of Nim are supported. - - diff --git a/doc/nimgrep.md b/doc/nimgrep.md new file mode 100644 index 000000000..63f760051 --- /dev/null +++ b/doc/nimgrep.md @@ -0,0 +1,128 @@ +========================= + nimgrep User's manual +========================= + +:Author: Andreas Rumpf +:Version: 1.6.0 + +.. default-role:: option +.. contents:: + +Nimgrep is a command line tool for search and replace tasks. It can search for +regex or peg patterns and can search whole directories at once. User +confirmation for every single replace operation can be requested. + +Nimgrep has particularly good support for Nim's +eccentric *style insensitivity* (see option `-y` below). +Apart from that it is a generic text manipulation tool. + + +Installation +============ + +Compile nimgrep with the command: + + ```cmd + nim c -d:release tools/nimgrep.nim + ``` + +And copy the executable somewhere in your ``$PATH``. + + +Command line switches +===================== + +.. include:: nimgrep_cmdline.txt + +Path filter options +------------------- + +Let us assume we have file `dirA/dirB/dirC/file.nim`. +Filesystem path options will match for these parts of the path: + +| option | matches for | +| :------------------ | :-------------------------------- | +| `--[not]extensions` | ``nim`` | +| `--[not]filename` | ``file.nim`` | +| `--[not]dirname` | ``dirA`` and ``dirB`` and ``dirC`` | +| `--[not]dirpath` | ``dirA/dirB/dirC`` | + +Combining multiple filter options together and negating them +------------------------------------------------------------ + +Options for filtering can be provided multiple times so they form a list, +which works as: +* positive filters + `--filename`, `--dirname`, `--dirpath`, `--inContext`, + `--inFile` accept files/matches if *any* pattern from the list is hit +* negative filters + `--notfilename`, `--notdirname`, `--notdirpath`, `--notinContext`, + `--notinFile` accept files/matches if *no* pattern from the list is hit. + +In other words the same filtering option repeated many times means logical OR. + +.. Important:: + Different filtering options are related by logical AND: they all must + be true for a match to be accepted. + E.g. `--filename:F --dirname:D1 --notdirname:D2` means + `filename(F) AND dirname(D1) AND (NOT dirname(D2))`. + +So negative filtering patterns are effectively related by logical OR also: +`(NOT PAT1) AND (NOT PAT2) == NOT (PAT1 OR PAT2)`:literal: in pseudo-code. + +That means you can always use only 1 such an option with logical OR, e.g. +`--notdirname:PAT1 --notdirname:PAT2` is fully equivalent to +`--notdirname:'PAT1|PAT2'`. + +.. Note:: + If you want logical AND on patterns you should compose 1 appropriate pattern, + possibly combined with multi-line mode `(?s)`:literal:. + E.g. to require that multi-line context of matches has occurrences of + **both** PAT1 and PAT2 use positive lookaheads (`(?=PAT)`:literal:): + ```cmd + nimgrep --inContext:'(?s)(?=.*PAT1)(?=.*PAT2)' + ``` + +Meaning of `^`:literal: and `$`:literal: +======================================== + +`nimgrep`:cmd: PCRE engine is run in a single-line mode so +`^`:literal: matches the beginning of whole input *file* and +`$`:literal: matches the end of *file* (or whole input *string* for +options like `--filename`). + +Add the `(?m)`:literal: modifier to the beginning of your pattern for +`^`:literal: and `$`:literal: to match the beginnings and ends of *lines*. + +Examples +======== + +All examples below use default PCRE Regex patterns: + ++ To search recursively in Nim files using style-insensitive identifiers: + + ```cmd + nimgrep --recursive --ext:'nim|nims' --ignoreStyle + # short: -r --ext:'nim|nims' -y + ``` + + .. Note:: we used `'` quotes to avoid special treatment of `|` symbol + for shells like Bash + ++ To exclude version control directories (Git, Mercurial=hg, Subversion=svn) + from the search: + ```cmd + nimgrep --notdirname:'^\.git$' --notdirname:'^\.hg$' --notdirname:'^\.svn$' + # short: --ndi:'^\.git$' --ndi:'^\.hg$' --ndi:'^\.svn$' + ``` ++ To search only in paths containing the `tests`:literal: sub-directory + recursively: + ```cmd + nimgrep --recursive --dirname:'^tests$' + # short: -r --di:'^tests$' + # or using --dirpath: + nimgrep --recursive --dirpath:'(^|/)tests($|/)' + # short: -r --pa:'(^|/)tests($|/)' + ``` ++ Nimgrep can search multi-line, e.g. to find files containing `import`:literal: + and then `strutils`:literal: use pattern `'import(.|\n)*?strutils'`:literal:. diff --git a/doc/nimgrep.rst b/doc/nimgrep.rst deleted file mode 100644 index 5b0fe0dbb..000000000 --- a/doc/nimgrep.rst +++ /dev/null @@ -1,52 +0,0 @@ -.. default-role:: code - -========================= - nimgrep User's manual -========================= - -:Author: Andreas Rumpf -:Version: 0.9 - - -Nimgrep is a command line tool for search&replace tasks. It can search for -regex or peg patterns and can search whole directories at once. User -confirmation for every single replace operation can be requested. - -Nimgrep has particularly good support for Nim's -eccentric *style insensitivity*. Apart from that it is a generic text -manipulation tool. - - -Installation -============ - -Compile nimgrep with the command:: - - nim c -d:release tools/nimgrep.nim - -And copy the executable somewhere in your `$PATH`. - - -Command line switches -===================== - -Usage: - nimgrep [options] [pattern] [replacement] (file/directory)* -Options: - --find, -f find the pattern (default) - --replace, -r replace the pattern - --peg pattern is a peg - --re pattern is a regular expression (default); extended - syntax for the regular expression is always turned on - --recursive process directories recursively - --confirm confirm each occurrence/replacement; there is a chance - to abort any time without touching the file - --stdin read pattern from stdin (to avoid the shell's confusing - quoting rules) - --word, -w the match should have word boundaries (buggy for pegs!) - --ignoreCase, -i be case insensitive - --ignoreStyle, -y be style insensitive - --ext:EX1|EX2|... only search the files with the given extension(s) - --verbose be verbose: list every processed file - --help, -h shows this help - --version, -v shows the version diff --git a/doc/nimgrep_cmdline.txt b/doc/nimgrep_cmdline.txt new file mode 100644 index 000000000..6f6887bc4 --- /dev/null +++ b/doc/nimgrep_cmdline.txt @@ -0,0 +1,136 @@ + +Usage: + +* To search: + + nimgrep [options] PATTERN [(FILE/DIRECTORY)*/-] + +* To replace: + + nimgrep [options] PATTERN --replace REPLACEMENT (FILE/DIRECTORY)*/- + +* To list file names: + + nimgrep [options] --filenames [PATTERN] [(FILE/DIRECTORY)*] + +Positional arguments, from left to right: +1) PATTERN is either Regex (default) or Peg if `--peg` is specified. + PATTERN and REPLACEMENT should be skipped when `--stdin` is specified. +2) REPLACEMENT supports `$1`, `$#` notations for captured groups in PATTERN. + + .. DANGER:: `--replace` mode **DOES NOT** ask confirmation + unless `--confirm` is specified! + +3) Final arguments are a list of paths (FILE/DIRECTORY) or a standalone + minus `-` or not specified (empty): + + * empty, current directory `.` is assumed (not with `--replace`) + + .. Note:: so when no FILE/DIRECTORY/`-` is specified nimgrep + does **not** read the pipe, but searches files in the current + dir instead! + * `-`, read buffer once from stdin: pipe or terminal input; + in `--replace` mode the result is directed to stdout; + it's not compatible with `--stdin`, `--filenames`, or `--confirm` + + + For any given DIRECTORY nimgrep searches only its immediate files without + traversing subdirectories unless `--recursive` is specified. + +In replacement mode we require all 3 positional arguments to avoid damaging. + +Options: +* Mode of operation: + --find, -f find the PATTERN (default) + --replace, -! replace the PATTERN to REPLACEMENT, rewriting the files + --confirm confirm each occurrence/replacement; there is a chance + to abort any time without touching the file + --filenames just list filenames. Provide a PATTERN to find it in + the filenames (not in the contents of a file) or run + with empty pattern to just list all files: + + nimgrep --filenames # In current dir + nimgrep --filenames "" DIRECTORY + # Note empty pattern "", lists all files in DIRECTORY +* Interprete patterns: + --peg PATTERN and PAT are Peg + --re PATTERN and PAT are regular expressions (default) + --rex, -x use the "extended" syntax for the regular expression + so that whitespace is not significant + --word, -w matches should have word boundaries (buggy for pegs!) + --ignoreCase, -i be case-insensitive in PATTERN and PAT + --ignoreStyle, -y be style insensitive in PATTERN and PAT + .. Note:: PATTERN and patterns PAT (see below in other options) are all either + Regex or Peg simultaneously and options `--rex`, `--word`, `--ignoreCase`, + and `--ignoreStyle` are applied to all of them. + +* File system walk: + --recursive, -r process directories recursively + --follow follow all symlinks when processing recursively + --sortTime, -s[:asc|desc] + order files by the last modification time (default: off): + ascending (recent files go last) or descending + +* Filter files (based on filesystem paths): + + .. Hint:: Instead of `not` you can type just `n` for negative options below. + + --ex[tensions]:EX1|EX2|... + only search the files with the given extension(s), + empty one (`--ex`) means files with missing extension + --notex[tensions]:EX1|EX2|... + exclude files having given extension(s), use empty one to + skip files with no extension (like some binary files are) + --fi[lename]:PAT search only files whose name matches pattern PAT + --notfi[lename]:PAT skip files whose name matches pattern PAT + --di[rname]:PAT select files that in their path have a directory name + that matches pattern PAT + --notdi[rname]:PAT do not descend into directories whose name (not path) + matches pattern PAT + --dirp[ath]:PAT select only files whose whole relative directory path + matches pattern PAT + --notdirp[ath]:PAT skip files whose whole relative directory path + matches pattern PAT + +* Filter files (based on file contents): + --inF[ile]:PAT select files containing a (not displayed) match of PAT + --notinF[ile]:PAT skip files containing a match of PAT + --bin:on|off|only process binary files? (detected by \0 in first 1K bytes) + (default: on - binary and text files treated the same way) + --text, -t process only text files, the same as `--bin:off` + +* Filter matches: + --inC[ontext]:PAT select only matches containing a match of PAT in their + surrounding context (multiline with `-c`, `-a`, `-b`) + --notinC[ontext]:PAT + skip matches not containing a match of PAT + in their surrounding context + +* Represent results: + --nocolor output will be given without any colors + --color[:on] force color even if output is redirected (default: auto) + --colorTheme:THEME select color THEME from `simple` (default), + `bnw` (black and white), `ack`, or `gnu` (GNU grep) + --count only print counts of matches for files that matched + --context:N, -c:N print N lines of leading context before every match and + N lines of trailing context after it (default N: 0) + --afterContext:N, -a:N + print N lines of trailing context after every match + --beforeContext:N, -b:N + print N lines of leading context before every match + --group, -g group matches by file + --newLine, -l display every matching line starting from a new line + --cols[:N] limit max displayed columns/width of output lines from + files by N characters, cropping overflows (default: off) + --cols:auto, -% calculate columns from terminal width for every line + --onlyAscii, -@ display only printable ASCII Latin characters 0x20-0x7E + substitutions: 0 -> ^@, 1 -> ^A, ... 0x1F -> ^_, + 0x7F -> '7F, ..., 0xFF -> 'FF + +* Miscellaneous: + --threads:N, -j:N speed up search by N additional workers (default: 0, off) + --stdin read PATTERN from stdin (to avoid the shell's confusing + quoting rules) and, if `--replace` given, REPLACEMENT + --verbose be verbose: list every processed file + --help, -h shows this help + --version, -v shows the version diff --git a/doc/niminst.rst b/doc/niminst.md index 3ccb47cc8..cc399c57a 100644 --- a/doc/niminst.rst +++ b/doc/niminst.md @@ -1,5 +1,3 @@ -.. default-role:: code - ========================= niminst User's manual ========================= @@ -7,6 +5,8 @@ :Author: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: Introduction @@ -14,7 +14,7 @@ Introduction niminst is a tool to generate an installer for a Nim program. Currently it can create an installer for Windows -via `Inno Setup <http://www.jrsoftware.org/isinfo.php>`_ as well as +via [Inno Setup](http://www.jrsoftware.org/isinfo.php) as well as installation/deinstallation scripts for UNIX. Later versions will support Linux' package management systems. @@ -26,7 +26,7 @@ systems. Configuration file ================== -niminst uses the Nim `parsecfg <parsecfg.html>`_ module to parse the +niminst uses the Nim [parsecfg](parsecfg.html) module to parse the configuration file. Here's an example of how the syntax looks like: .. include:: mytest.cfg @@ -34,7 +34,7 @@ configuration file. Here's an example of how the syntax looks like: The value of a key-value pair can reference user-defined variables via the `$variable` notation: They can be defined in the command line with the -`--var:name=value` switch. This is useful to not hard-coding the +`--var:name=value`:option: switch. This is useful to not hard-coding the program's version number into the configuration file, for instance. It follows a description of each possible section and how it affects the @@ -49,20 +49,20 @@ contain the following key-value pairs: ==================== ======================================================= Key description ==================== ======================================================= -`Name` the project's name; this needs to be a single word -`DisplayName` the project's long name; this can contain spaces. If +`Name` the project's name; this needs to be a single word +`DisplayName` the project's long name; this can contain spaces. If not specified, this is the same as `Name`. -`Version` the project's version -`OS` the OSes to generate C code for; for example: +`Version` the project's version +`OS` the OSes to generate C code for; for example: `"windows;linux;macosx"` -`CPU` the CPUs to generate C code for; for example: +`CPU` the CPUs to generate C code for; for example: `"i386;amd64;powerpc"` -`Authors` the project's authors -`Description` the project's description -`App` the application's type: "Console" or "GUI". If +`Authors` the project's authors +`Description` the project's description +`App` the application's type: "Console" or "GUI". If "Console", niminst generates a special batch file for Windows to open up the command-line shell. -`License` the filename of the application's license +`License` the filename of the application's license ==================== ======================================================= @@ -72,11 +72,11 @@ Key description Many sections support the `files` key. Listed filenames can be separated by semicolon or the `files` key can be repeated. Wildcards in filenames are supported. If it is a directory name, all files in the -directory are used:: +directory are used: - [Config] - Files: "configDir" - Files: "otherconfig/*.conf;otherconfig/*.cfg" + [Config] + Files: "configDir" + Files: "otherconfig/*.conf;otherconfig/*.cfg" Config section @@ -91,10 +91,10 @@ Documentation section The `documentation` section supports the `files` key. Listed files will be installed into the OS's native documentation directory -(which might be `$appdir/doc`). +(which might be ``$appdir/doc``). There is a `start` key which determines whether the Windows installer -generates a link to e.g. the `index.html` of your documentation. +generates a link to e.g. the ``index.html`` of your documentation. Other section @@ -125,9 +125,9 @@ Other possible options are: ==================== ======================================================= Key description ==================== ======================================================= -`BinPath` paths to add to the Windows `%PATH%` environment +`BinPath` paths to add to the Windows `%PATH%` environment variable. Example: ``BinPath: r"bin;dist\mingw\bin"`` -`InnoSetup` boolean flag whether an Inno Setup installer should be +`InnoSetup` boolean flag whether an Inno Setup installer should be generated for Windows. Example: `InnoSetup: "Yes"` ==================== ======================================================= @@ -137,7 +137,7 @@ UnixBin section The `UnixBin` section currently only supports the `files` key. Listed files will be installed into the OS's native bin directory -(e.g. `/usr/local/bin`). The exact location depends on the +(e.g. ``/usr/local/bin``). The exact location depends on the installation path the user specifies when running the `install.sh` script. @@ -149,9 +149,9 @@ Possible options are: ==================== ======================================================= Key description ==================== ======================================================= -`InstallScript` boolean flag whether an installation shell script +`InstallScript` boolean flag whether an installation shell script should be generated. Example: `InstallScript: "Yes"` -`UninstallScript` boolean flag whether a de-installation shell script +`UninstallScript` boolean flag whether a de-installation shell script should be generated. Example: `UninstallScript: "Yes"` ==================== ======================================================= @@ -165,9 +165,9 @@ Possible options are: ==================== ======================================================= Key description ==================== ======================================================= -`path` Path to Inno Setup. +`path` Path to Inno Setup. Example: ``path = r"c:\inno setup 5\iscc.exe"`` -`flags` Flags to pass to Inno Setup. +`flags` Flags to pass to Inno Setup. Example: `flags = "/Q"` ==================== ======================================================= @@ -180,8 +180,8 @@ Possible options are: ==================== ======================================================= Key description ==================== ======================================================= -`path` Path to the C compiler. -`flags` Flags to pass to the C Compiler. +`path` Path to the C compiler. +`flags` Flags to pass to the C Compiler. Example: `flags = "-w"` ==================== ======================================================= diff --git a/doc/nims.rst b/doc/nims.md index f81637d73..987cc2096 100644 --- a/doc/nims.rst +++ b/doc/nims.md @@ -1,34 +1,35 @@ -.. default-role:: code - ================================ NimScript ================================ +.. default-role:: code +.. include:: rstcommon.rst + Strictly speaking, `NimScript` is the subset of Nim that can be evaluated by Nim's builtin virtual machine (VM). This VM is used for Nim's compiletime function evaluation features. -The `nim` executable processes the `.nims` configuration files in +The `nim`:cmd: executable processes the ``.nims`` configuration files in the following directories (in this order; later files overwrite previous settings): 1) If environment variable `XDG_CONFIG_HOME` is defined, - `$XDG_CONFIG_HOME/nim/config.nims` or - `~/.config/nim/config.nims` (POSIX) or - `%APPDATA%/nim/config.nims` (Windows). This file can be skipped - with the `--skipUserCfg` command line option. -2) `$parentDir/config.nims` where `$parentDir` stands for any + ``$XDG_CONFIG_HOME/nim/config.nims`` or + ``~/.config/nim/config.nims`` (POSIX) or + ``%APPDATA%/nim/config.nims`` (Windows). This file can be skipped + with the `--skipUserCfg`:option: command line option. +2) ``$parentDir/config.nims`` where ``$parentDir`` stands for any parent directory of the project file's path. These files can be - skipped with the `--skipParentCfg` command line option. -3) `$projectDir/config.nims` where `$projectDir` stands for the - project's path. This file can be skipped with the `--skipProjCfg` + skipped with the `--skipParentCfg`:option: command line option. +3) ``$projectDir/config.nims`` where ``$projectDir`` stands for the + project's path. This file can be skipped with the `--skipProjCfg`:option: command line option. 4) A project can also have a project specific configuration file named - `$project.nims` that resides in the same directory as - `$project.nim`. This file can be skipped with the same - `--skipProjCfg` command line option. + ``$project.nims`` that resides in the same directory as + ``$project.nim``. This file can be skipped with the same + `--skipProjCfg`:option: command line option. -For available procs and implementation details see `nimscript <nimscript.html>`_. +For available procs and implementation details see [nimscript](nimscript.html). Limitations @@ -60,84 +61,88 @@ Standard library modules At least the following standard library modules are available: -* `macros <macros.html>`_ -* `os <os.html>`_ -* `strutils <strutils.html>`_ -* `math <math.html>`_ -* `distros <distros.html>`_ -* `sugar <sugar.html>`_ -* `algorithm <algorithm.html>`_ -* `base64 <base64.html>`_ -* `bitops <bitops.html>`_ -* `chains <chains.html>`_ -* `colors <colors.html>`_ -* `complex <complex.html>`_ -* `htmlgen <htmlgen.html>`_ -* `httpcore <httpcore.html>`_ -* `lenientops <lenientops.html>`_ -* `mersenne <mersenne.html>`_ -* `options <options.html>`_ -* `parseutils <parseutils.html>`_ -* `punycode <punycode.html>`_ -* `random <punycode.html>`_ -* `stats <stats.html>`_ -* `strformat <strformat.html>`_ -* `strmisc <strmisc.html>`_ -* `strscans <strscans.html>`_ -* `unicode <unicode.html>`_ -* `uri <uri.html>`_ -* `std/editdistance <editdistance.html>`_ -* `std/wordwrap <wordwrap.html>`_ -* `std/sums <sums.html>`_ -* `parsecsv <parsecsv.html>`_ -* `parsecfg <parsecfg.html>`_ -* `parsesql <parsesql.html>`_ -* `xmlparser <xmlparser.html>`_ -* `htmlparser <htmlparser.html>`_ -* `ropes <ropes.html>`_ -* `json <json.html>`_ -* `parsejson <parsejson.html>`_ -* `strtabs <strtabs.html>`_ -* `unidecode <unidecode.html>`_ - -In addition to the standard Nim syntax (`system <system.html>`_ module), +* [algorithm](algorithm.html) +* [base64](base64.html) +* [bitops](bitops.html) +* [chains](chains.html) +* [colors](colors.html) +* [complex](complex.html) +* [distros](distros.html) +* [std/editdistance](editdistance.html) +* [htmlgen](htmlgen.html) +* [htmlparser](htmlparser.html) +* [httpcore](httpcore.html) +* [json](json.html) +* [lenientops](lenientops.html) +* [macros](macros.html) +* [math](math.html) +* [options](options.html) +* [os](os.html) +* [parsecfg](parsecfg.html) +* [parsecsv](parsecsv.html) +* [parsejson](parsejson.html) +* [parsesql](parsesql.html) +* [parseutils](parseutils.html) +* [punycode](punycode.html) +* [random](random.html) +* [ropes](ropes.html) +* [std/setutils](setutils.html) +* [stats](stats.html) +* [strformat](strformat.html) +* [strmisc](strmisc.html) +* [strscans](strscans.html) +* [strtabs](strtabs.html) +* [strutils](strutils.html) +* [sugar](sugar.html) +* [unicode](unicode.html) +* [unidecode](unidecode.html) +* [uri](uri.html) +* [std/wordwrap](wordwrap.html) +* [xmlparser](xmlparser.html) + +In addition to the standard Nim syntax ([system](system.html) module), NimScripts support the procs and templates defined in the -`nimscript <nimscript.html>`_ module too. +[nimscript](nimscript.html) module too. See also: -* `Check the tests for more information about modules compatible with NimScript. <https://github.com/nim-lang/Nim/blob/devel/tests/test_nimscript.nims>`_ +* [Check the tests for more information about modules compatible with NimScript]( + https://github.com/nim-lang/Nim/blob/devel/tests/test_nimscript.nims) NimScript as a configuration file ================================= -A command-line switch `--FOO` is written as `switch("FOO")` in -NimScript. Similarly, command-line `--FOO:VAL` translates to +A command-line switch `--FOO`:option: is written as `switch("FOO")` in +NimScript. Similarly, command-line `--FOO:VAL`:option: translates to `switch("FOO", "VAL")`. Here are few examples of using the `switch` proc: -.. code-block:: nim + ```nim # command-line: --opt:size switch("opt", "size") - # command-line: --define:foo or -d:foo - switch("define", "foo") + # command-line: --define:release or -d:release + switch("define", "release") # command-line: --forceBuild switch("forceBuild") + # command-line: --hint[Conf]:off or --hint:Conf:off + switch("hint", "[Conf]:off") + ``` -NimScripts also support `--` templates for convenience, which look +NimScripts also support `--`:option: templates for convenience, which look like command-line switches written as-is in the NimScript file. So the above example can be rewritten as: -.. code-block:: nim + ```nim --opt:size - --define:foo + --define:release --forceBuild + ``` **Note**: In general, the *define* switches can also be set in -NimScripts using `switch` or `--`, as shown in above -examples. Only the `release` define (`-d:release`) cannot be set -in NimScripts. +NimScripts using `switch` or `--`, as shown in above examples. Few +`define` switches such as `-d:strip`:option:, `-d:lto`:option: and +`-d:lto_incremental`:option: cannot be set in NimScripts. NimScript as a build tool @@ -145,11 +150,12 @@ NimScript as a build tool The `task` template that the `system` module defines allows a NimScript file to be used as a build tool. The following example defines a -task `build` that is an alias for the `c` command: +task `build` that is an alias for the `c`:option: command: -.. code-block:: nim + ```nim task build, "builds an example": setCommand "c" + ``` In fact, as a convention the following tasks should be available: @@ -159,20 +165,20 @@ Task Description ========= =================================================== `help` List all the available NimScript tasks along with their docstrings. `build` Build the project with the required - backend (`c`, `cpp` or `js`). + backend (`c`:option:, `cpp`:option: or `js`:option:). `tests` Runs the tests belonging to the project. `bench` Runs benchmarks belonging to the project. ========= =================================================== -Look at the module `distros <distros.html>`_ for some support of the +Look at the module [distros](distros.html) for some support of the OS's native package managers. Nimble integration ================== -See the `Nimble readme <https://github.com/nim-lang/nimble#readme>`_ +See the [Nimble readme](https://github.com/nim-lang/nimble#readme) for more information. @@ -180,11 +186,10 @@ Standalone NimScript ==================== NimScript can also be used directly as a portable replacement for Bash and -Batch files. Use `nim myscript.nims` to run `myscript.nims`. For example, +Batch files. Use `nim myscript.nims`:cmd: to run ``myscript.nims``. For example, installation of Nimble could be accomplished with this simple script: -.. code-block:: nim - + ```nim mode = ScriptMode.Verbose var id = 0 @@ -197,18 +202,19 @@ installation of Nimble could be accomplished with this simple script: exec "nim c nimble" mvFile "nimble" & $id & "/src/nimble".toExe, "bin/nimble".toExe + ``` On Unix, you can also use the shebang `#!/usr/bin/env nim`, as long as your filename -ends with `.nims`: - -.. code-block:: nim +ends with ``.nims``: + ```nim #!/usr/bin/env nim mode = ScriptMode.Silent echo "hello world" + ``` -Use `#!/usr/bin/env -S nim --hints:off` to disable hints. +Use `#!/usr/bin/env -S nim e --hints:off` to disable hints and relax the file extension constraint. Benefits @@ -228,8 +234,7 @@ allowing the same script to support a lot of systems. See the following (incomplete) example: -.. code-block:: nim - + ```nim import std/distros # Architectures. @@ -255,6 +260,7 @@ See the following (incomplete) example: echo "Distro is ArchLinux" elif detectOs(Debian): echo "Distro is Debian" + ``` Uniform Syntax @@ -276,24 +282,24 @@ making it ideal for functional scripting metaprogramming. This is an example of a third party module that uses macros and templates to translate text strings on unmodified NimScript: -.. code-block:: nim - + ```nim import nimterlingua nimterlingua("translations.cfg") echo "cat" # Run with -d:RU becomes "kot", -d:ES becomes "gato", ... + ``` translations.cfg -.. code-block:: none - + ```none [cat] ES = gato IT = gatto RU = kot FR = chat + ``` -* `Nimterlingua <https://nimble.directory/pkg/nimterlingua>`_ +* [Nimterlingua](https://nimble.directory/pkg/nimterlingua) Graceful Fallback @@ -304,8 +310,7 @@ but often a graceful and seamless fallback degradation is used. See the following NimScript: -.. code-block:: nim - + ```nim if likely(true): discard elif unlikely(false): @@ -315,6 +320,7 @@ See the following NimScript: static: echo CompileDate + ``` `likely()`, `unlikely()`, `static:` and `{.compiletime.}` @@ -325,14 +331,15 @@ Evolving Scripting language --------------------------- NimScript evolves together with Nim, -`occasionally new features might become available on NimScript <https://github.com/nim-lang/Nim/pulls?utf8=%E2%9C%93&q=nimscript>`_ , +[occasionally new features might become available on NimScript]( +https://github.com/nim-lang/Nim/pulls?q=nimscript+is%3Amerged), adapted from compiled Nim or added as new features on both. Scripting Language with a Package Manager ----------------------------------------- You can create your own modules to be compatible with NimScript, -and check `Nimble <https://nimble.directory>`_ +and check [Nimble](https://nimble.directory) to search for third party modules that may work on NimScript. DevOps Scripting @@ -341,4 +348,5 @@ DevOps Scripting You can use NimScript to deploy to production, run tests, build projects, do benchmarks, generate documentation, and all kinds of DevOps/SysAdmin specific tasks. -* `An example of a third party NimScript that can be used as a project-agnostic tool. <https://github.com/kaushalmodi/nim_config#list-available-tasks>`_ +* [An example of a third party NimScript that can be used as a project-agnostic + tool.](https://github.com/kaushalmodi/nim_config#list-available-tasks) diff --git a/doc/nimsuggest.rst b/doc/nimsuggest.md index 509f72e8a..3d076a6f5 100644 --- a/doc/nimsuggest.rst +++ b/doc/nimsuggest.md @@ -1,5 +1,3 @@ -.. default-role:: code - ================================ Nim IDE Integration Guide ================================ @@ -7,49 +5,53 @@ :Author: Unknown :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: Nim differs from many other compilers in that it is really fast, and being so fast makes it suited to provide external queries for text editors about the source code being written. Through the -`nimsuggest` tool, any IDE -can query a `.nim` source file and obtain useful information like +`nimsuggest`:cmd: tool, any IDE +can query a ``.nim`` source file and obtain useful information like definition of symbols or suggestions for completion. This document will guide you through the available options. If you want to look at practical examples of nimsuggest support you can look at the -`various editor integrations <https://github.com/Araq/Nim/wiki/Editor-Support>`_ +[various editor integrations](https://github.com/Araq/Nim/wiki/Editor-Support) already available. Installation ============ -Nimsuggest is part of Nim's core. Build it via:: +Nimsuggest is part of Nim's core. Build it via: + ```cmd koch nimsuggest + ``` Nimsuggest invocation ===================== -Run it via `nimsuggest --stdin --debug myproject.nim`. Nimsuggest is a +Run it via `nimsuggest --stdin --debug myproject.nim`:cmd:. Nimsuggest is a server that takes queries that are related to `myproject`. There is some -support so that you can throw random `.nim` files which are not part +support so that you can throw random ``.nim`` files which are not part of `myproject` at Nimsuggest too, but usually the query refer to modules/files that are part of `myproject`. -`--stdin` means that Nimsuggest reads the query from `stdin`. This is great +`--stdin`:option: means that Nimsuggest reads the query from `stdin`. This is great for testing things out and playing with it but for an editor communication via sockets is more reasonable so that is the default. It listens to port 6000 by default. -Nimsuggest is basically a frontend for the nim compiler so `--path` flags and -`config files <https://nim-lang.org/docs/nimc.html#compiler-usage-configuration-files>`_ +Nimsuggest is basically a frontend for the nim compiler so `--path`:option: flags and +[config files](nimc.html#compiler-usage-configuration-files) can be used to specify additional dependencies like -`nimsuggest --stdin --debug --path:"dependencies" myproject.nim`. +`nimsuggest --stdin --debug --path:"dependencies" myproject.nim`:cmd:. Specifying the location of the query @@ -60,26 +62,26 @@ cryptic 3 letter "command" `def` or `con` or `sug` or `use` followed by a location. A query location consists of: -`file.nim` - This is the name of the module or include file the query refers to. +``file.nim`` +: This is the name of the module or include file the query refers to. -`dirtyfile.nim` - This is optional. +``dirtyfile.nim`` +: This is optional. The `file` parameter is enough for static analysis, but IDEs tend to have *unsaved buffers* where the user may still be in the middle of typing a line. In such situations the IDE can save the current contents to a temporary file and then use the - `dirtyfile.nim` option to tell Nimsuggest that `foobar.nim` should - be taken from `temporary/foobar.nim`. + ``dirtyfile.nim`` option to tell Nimsuggest that ``foobar.nim`` should + be taken from ``temporary/foobar.nim``. -`line` - An integer with the line you are going to query. For the compiler +``line`` +: An integer with the line you are going to query. For the compiler lines start at **1**. -`col` - An integer with the column you are going to query. For the +``col`` +: An integer with the column you are going to query. For the compiler columns start at **0**. @@ -111,8 +113,8 @@ The `sug` Nimsuggest command performs a query about possible completion symbols at some point in the file. The typical usage scenario for this option is to call it after the -user has typed the dot character for `the object oriented call -syntax <tut2.html#object-oriented-programming-method-call-syntax>`_. +user has typed the dot character for [the object oriented call +syntax](tut2.html#object-oriented-programming-method-call-syntax). Nimsuggest will try to return the suggestions sorted first by scope (from innermost to outermost) and then by item name. @@ -149,9 +151,9 @@ tab characters (``\t``). The values of each column are: 1. Three characters indicating the type of returned answer (e.g. `def` for definition, `sug` for suggestion, etc). 2. Type of the symbol. This can be `skProc`, `skLet`, and just - about any of the enums defined in the module `compiler/ast.nim`. + about any of the enums defined in the module ``compiler/ast.nim``. 3. Fully qualified path of the symbol. If you are querying a symbol - defined in the `proj.nim` file, this would have the form + defined in the ``proj.nim`` file, this would have the form `proj.symbolName`. 4. Type/signature. For variables and enums this will contain the type of the symbol, for procs, methods and templates this will diff --git a/doc/overview.rst b/doc/overview.md index e01520d7c..b21eb1e68 100644 --- a/doc/overview.rst +++ b/doc/overview.md @@ -5,5 +5,5 @@ Nim Documentation Overview :Author: Andreas Rumpf :Version: |nimversion| -.. include:: docs.rst +.. include:: docs.md diff --git a/doc/packaging.rst b/doc/packaging.md index ecde73f61..b742bef28 100644 --- a/doc/packaging.rst +++ b/doc/packaging.md @@ -4,7 +4,10 @@ Packaging Nim This page provide hints on distributing Nim using OS packages. -See `distros <distros.html>`_ for tools to detect Linux distribution at runtime. +See [distros](distros.html) for tools to detect Linux distribution at runtime. + +See [here](intern.html#bootstrapping-the-compiler-reproducible-builds) for how to +compile reproducible builds. Supported architectures ----------------------- @@ -45,28 +48,33 @@ The Debian package ships bash and ksh completion and manpages that can be reused Hints on the build process: -:: - - # build from C sources and then using koch - ./build.sh --os $os_type --cpu $cpu_arch - ./bin/nim c koch - ./koch boot -d:release + ```cmd + # build from C sources and then using koch + make -j # supports parallel build + # alternatively: ./build.sh --os $os_type --cpu $cpu_arch + ./bin/nim c -d:release koch + ./koch boot -d:release - # optionally generate docs into doc/html - ./koch docs + # optionally generate docs into doc/html + ./koch docs - ./koch tools -d:release + ./koch tools - # extract files to be really installed - ./install.sh <tempdir> + # extract files to be really installed + ./install.sh <tempdir> - # also include the tools - for fn in nimble nimsuggest nimgrep; do cp ./bin/$fn <tempdir>/nim/bin/; done + # also include the tools + for fn in nimble nimsuggest nimgrep; do cp ./bin/$fn <tempdir>/nim/bin/; done + ``` What to install: -- The expected stdlib location is /usr/lib/nim -- Global configuration files under /etc/nim +- The expected stdlib location is `/usr/lib/nim/lib`, previously it was just `/usr/lib/nim` +- `nimdoc.css` and `nimdoc.cls` from the `doc` folder should go into `/usr/lib/nim/doc/` +- `tools/debug/nim-gdb.py` should go into `/usr/lib/nim/tools/` +- `tools/dochack/dochack.js` should be installed to `/usr/lib/nim/tools/dochack/` +- Global configuration files under `/etc/nim` - Optionally: manpages, documentation, shell completion - When installing documentation, .idx files are not required - The "compiler" directory contains compiler sources and should not be part of the compiler binary package + diff --git a/doc/pegdocs.txt b/doc/pegdocs.txt index 4c557aed8..0a8fd8187 100644 --- a/doc/pegdocs.txt +++ b/doc/pegdocs.txt @@ -13,12 +13,12 @@ notation meaning ``A / ... / Z`` Ordered choice: Apply expressions `A`, ..., `Z`, in this order, to the text ahead, until one of them succeeds and possibly consumes some text. Indicate success if one of - expressions succeeded. Otherwise do not consume any text + expressions succeeded. Otherwise, do not consume any text and indicate failure. ``A ... Z`` Sequence: Apply expressions `A`, ..., `Z`, in this order, to consume consecutive portions of the text ahead, as long as they succeed. Indicate success if all succeeded. - Otherwise do not consume any text and indicate failure. + Otherwise, do not consume any text and indicate failure. The sequence's precedence is higher than that of ordered choice: ``A B / C`` means ``(A B) / Z`` and not ``A (B / Z)``. @@ -27,7 +27,10 @@ notation meaning ``{E}`` Capture: Apply expression `E` and store the substring that matched `E` into a *capture* that can be accessed after the matching process. -``$i`` Back reference to the ``i``th capture. ``i`` counts from 1. +``{}`` Empty capture: Delete the last capture. No character + is consumed. +``$i`` Back reference to the ``i``th capture. ``i`` counts forwards + from 1 or backwards (last capture to first) from ^1. ``$`` Anchor: Matches at the end of the input. No character is consumed. Same as ``!.``. ``^`` Anchor: Matches at the start of the input. No character @@ -41,20 +44,20 @@ notation meaning ``E+`` One or more: Apply expression `E` repeatedly to match the text ahead, as long as it succeeds. Consume the matched text (if any) and indicate success if there was at least - one match. Otherwise indicate failure. + one match. Otherwise, indicate failure. ``E*`` Zero or more: Apply expression `E` repeatedly to match the text ahead, as long as it succeeds. Consume the matched text (if any). Always indicate success. ``E?`` Zero or one: If expression `E` matches the text ahead, consume it. Always indicate success. ``[s]`` Character class: If the character ahead appears in the - string `s`, consume it and indicate success. Otherwise + string `s`, consume it and indicate success. Otherwise, indicate failure. ``[a-b]`` Character range: If the character ahead is one from the range `a` through `b`, consume it and indicate success. - Otherwise indicate failure. + Otherwise, indicate failure. ``'s'`` String: If the text ahead is the string `s`, consume it - and indicate success. Otherwise indicate failure. + and indicate success. Otherwise, indicate failure. ``i's'`` String match ignoring case. ``y's'`` String match ignoring style. ``v's'`` Verbatim string match: Use this to override a global @@ -63,15 +66,15 @@ notation meaning ``y$j`` String match ignoring style for back reference. ``v$j`` Verbatim string match for back reference. ``.`` Any character: If there is a character ahead, consume it - and indicate success. Otherwise (that is, at the end of + and indicate success. Otherwise, (that is, at the end of input) indicate failure. -``_`` Any Unicode character: If there is an UTF-8 character - ahead, consume it and indicate success. Otherwise indicate +``_`` Any Unicode character: If there is a UTF-8 character + ahead, consume it and indicate success. Otherwise, indicate failure. ``@E`` Search: Shorthand for ``(!E .)* E``. (Search loop for the pattern `E`.) ``{@} E`` Captured Search: Shorthand for ``{(!E .)*} E``. (Search - loop for the pattern `E`.) Everything until and exluding + loop for the pattern `E`.) Everything until and excluding `E` is captured. ``@@ E`` Same as ``{@} E``. ``A <- E`` Rule: Bind the expression `E` to the *nonterminal symbol* @@ -79,7 +82,7 @@ notation meaning matching engine.** ``\identifier`` Built-in macro for a longer expression. ``\ddd`` Character with decimal code *ddd*. -``\"``, etc Literal ``"``, etc. +``\"``, etc. Literal ``"``, etc. =============== ============================================================ @@ -128,51 +131,53 @@ notation meaning Supported PEG grammar --------------------- -The PEG parser implements this grammar (written in PEG syntax):: +The PEG parser implements this grammar (written in PEG syntax): - # Example grammar of PEG in PEG syntax. - # Comments start with '#'. - # First symbol is the start symbol. + # Example grammar of PEG in PEG syntax. + # Comments start with '#'. + # First symbol is the start symbol. - grammar <- rule* / expr + grammar <- rule* / expr - identifier <- [A-Za-z][A-Za-z0-9_]* - charsetchar <- "\\" . / [^\]] - charset <- "[" "^"? (charsetchar ("-" charsetchar)?)+ "]" - stringlit <- identifier? ("\"" ("\\" . / [^"])* "\"" / - "'" ("\\" . / [^'])* "'") - builtin <- "\\" identifier / [^\13\10] + identifier <- [A-Za-z][A-Za-z0-9_]* + charsetchar <- "\\" . / [^\]] + charset <- "[" "^"? (charsetchar ("-" charsetchar)?)+ "]" + stringlit <- identifier? ("\"" ("\\" . / [^"])* "\"" / + "'" ("\\" . / [^'])* "'") + builtin <- "\\" identifier / [^\13\10] - comment <- '#' @ \n - ig <- (\s / comment)* # things to ignore + comment <- '#' @ \n + ig <- (\s / comment)* # things to ignore - rule <- identifier \s* "<-" expr ig - identNoArrow <- identifier !(\s* "<-") - prefixOpr <- ig '&' / ig '!' / ig '@' / ig '{@}' / ig '@@' - literal <- ig identifier? '$' [0-9]+ / '$' / '^' / - ig identNoArrow / - ig charset / - ig stringlit / - ig builtin / - ig '.' / - ig '_' / - (ig "(" expr ig ")") - postfixOpr <- ig '?' / ig '*' / ig '+' - primary <- prefixOpr* (literal postfixOpr*) + rule <- identifier \s* "<-" expr ig + identNoArrow <- identifier !(\s* "<-") + prefixOpr <- ig '&' / ig '!' / ig '@' / ig '{@}' / ig '@@' + literal <- ig identifier? '$' '^'? [0-9]+ / '$' / '^' / + ig identNoArrow / + ig charset / + ig stringlit / + ig builtin / + ig '.' / + ig '_' / + (ig "(" expr ig ")") / + (ig "{" expr? ig "}") + postfixOpr <- ig '?' / ig '*' / ig '+' + primary <- prefixOpr* (literal postfixOpr*) - # Concatenation has higher priority than choice: - # ``a b / c`` means ``(a b) / c`` + # Concatenation has higher priority than choice: + # ``a b / c`` means ``(a b) / c`` - seqExpr <- primary+ - expr <- seqExpr (ig "/" expr)* + seqExpr <- primary+ + expr <- seqExpr (ig "/" expr)* **Note**: As a special syntactic extension if the whole PEG is only a single expression, identifiers are not interpreted as non-terminals, but are interpreted as verbatim string: -.. code-block:: nim + ```nim abc =~ peg"abc" # is true + ``` So it is not necessary to write ``peg" 'abc' "`` in the above example. @@ -182,22 +187,25 @@ Examples Check if `s` matches Nim's "while" keyword: -.. code-block:: nim + ```nim s =~ peg" y'while'" + ``` Exchange (key, val)-pairs: -.. code-block:: nim + ```nim "key: val; key2: val2".replacef(peg"{\ident} \s* ':' \s* {\ident}", "$2: $1") + ``` Determine the ``#include``'ed files of a C file: -.. code-block:: nim + ```nim for line in lines("myfile.c"): if line =~ peg"""s <- ws '#include' ws '"' {[^"]+} '"' ws comment <- '/*' @ '*/' / '//' .* ws <- (comment / \s+)* """: echo matches[0] + ``` PEG vs regular expression ------------------------- diff --git a/doc/readme.txt b/doc/readme.txt index 6f4cece87..1157752b9 100644 --- a/doc/readme.txt +++ b/doc/readme.txt @@ -1,7 +1,7 @@ ============================ -Nim's documenation system +Nim's documentation system ============================ This folder contains Nim's documentation. The documentation -is written in a format called *reStructuredText*, a markup language that reads +is written in a format called *Markdown*, a markup language that reads like ASCII and can be converted to HTML automatically! diff --git a/doc/gc.rst b/doc/refc.md index 4455afcbe..4023748e6 100644 --- a/doc/gc.rst +++ b/doc/refc.md @@ -1,73 +1,3 @@ -.. default-role:: code - -======================= -Nim's Memory Management -======================= - -:Author: Andreas Rumpf -:Version: |nimversion| - -.. - - - "The road to hell is paved with good intentions." - - -Introduction -============ - -A memory-management algorithm optimal for every use-case cannot exist. -Nim provides multiple paradigms for needs ranging from large multi-threaded -applications, to games, hard-realtime systems and small microcontrollers. - -This document describes how the management strategies work; -How to tune the garbage collectors for your needs, like (soft) `realtime systems`:idx:, -and how the memory management strategies other than garbage collectors work. - -.. note:: the default GC is incremental, thread-local and not "stop-the-world" - -Multi-paradigm Memory Management Strategies -=========================================== - -To choose the memory management strategy use the `--gc:` switch. - -- `--gc:refc`. This is the default GC. It's a - deferred reference counting based garbage collector - with a simple Mark&Sweep backup GC in order to collect cycles. Heaps are thread-local. -- `--gc:markAndSweep`. Simple Mark-And-Sweep based garbage collector. Heaps are thread-local. -- `--gc:boehm`. Boehm based garbage collector, it offers a shared heap. -- `--gc:go`. Go's garbage collector, useful for interoperability with Go. Offers a shared heap. -- `--gc:arc`. Plain reference counting with - `move semantic optimizations <destructors.html#move-semantics>`_, offers a shared heap. - It offers deterministic performance for `hard realtime`:idx: systems. Reference cycles - cause memory leaks, beware. - -- `--gc:orc`. Same as `--gc:arc` but adds a cycle collector based on "trial deletion". - Unfortunately, that makes its performance profile hard to reason about so it is less - useful for hard real-time systems. - -- `--gc:none`. No memory management strategy nor a garbage collector. Allocated memory is - simply never freed. You should use `--gc:arc` instead. - - -================== ======== ================= ============== =================== -Memory Management Heap Reference Cycles Stop-The-World Command line switch -================== ======== ================= ============== =================== -RefC Local Cycle Collector No `--gc:refc` -Mark & Sweep Local Cycle Collector No `--gc:markAndSweep` -ARC Shared Leak No `--gc:arc` -ORC Shared Cycle Collector No `--gc:orc` -Boehm Shared Cycle Collector Yes `--gc:boehm` -Go Shared Cycle Collector Yes `--gc:go` -None Manual Manual Manual `--gc:none` -================== ======== ================= ============== =================== - -JavaScript's garbage collector is used for the `JavaScript and NodeJS -<backends.html#backends-the-javascript-target>`_ compilation targets. -The `NimScript <nims.html>`_ target uses the memory management strategy built into -the Nim compiler. - - Tweaking the refc GC ==================== @@ -82,13 +12,14 @@ Soft real-time support ---------------------- To enable real-time support, the symbol `useRealtimeGC`:idx: needs to be -defined via `--define:useRealtimeGC` (you can put this into your config +defined via `--define:useRealtimeGC`:option: (you can put this into your config file as well). With this switch the garbage collector supports the following operations: -.. code-block:: nim + ```nim proc GC_setMaxPause*(maxPauseInUs: int) proc GC_step*(us: int, strongAdvice = false, stackSize = -1) + ``` The unit of the parameters `maxPauseInUs` and `us` is microseconds. @@ -132,7 +63,7 @@ Time measurement with garbage collectors ---------------------------------------- The garbage collectors' way of measuring time uses -(see `lib/system/timers.nim` for the implementation): +(see ``lib/system/timers.nim`` for the implementation): 1) `QueryPerformanceCounter` and `QueryPerformanceFrequency` on Windows. 2) `mach_absolute_time` on Mac OS X. @@ -156,13 +87,41 @@ that up to 100 objects are traversed and freed before it checks again. Thus highly specialized environments or for older hardware. +Thread coordination +------------------- + +When the `NimMain()` function is called Nim initializes the garbage +collector to the current thread, which is usually the main thread of your +application. If your C code later spawns a different thread and calls Nim +code, the garbage collector will fail to work properly and you will crash. + +As long as you don't use the threadvar emulation Nim uses native thread +variables, of which you get a fresh version whenever you create a thread. You +can then attach a GC to this thread via + + ```nim + system.setupForeignThreadGc() + ``` + +It is **not** safe to disable the garbage collector and enable it after the +call from your background thread even if the code you are calling is short +lived. + +Before the thread exits, you should tear down the thread's GC to prevent memory +leaks by calling + + ```nim + system.tearDownForeignThreadGc() + ``` + + Keeping track of memory ======================= If you need to pass around memory allocated by Nim to C, you can use the procs `GC_ref` and `GC_unref` to mark objects as referenced to avoid them being freed by the garbage collector. -Other useful procs from `system <system.html>`_ you can use to keep track of memory are: +Other useful procs from [system](system.html) you can use to keep track of memory are: * `getTotalMem()` Returns the amount of total memory managed by the garbage collector. * `getOccupiedMem()` Bytes reserved by the garbage collector and used by objects. @@ -170,7 +129,7 @@ Other useful procs from `system <system.html>`_ you can use to keep track of mem * `GC_getStatistics()` Garbage collector statistics as a human-readable string. These numbers are usually only for the running thread, not for the whole heap, -with the exception of `--gc:boehm` and `--gc:go`. +with the exception of `--mm:boehm`:option: and `--mm:go`:option:. In addition to `GC_ref` and `GC_unref` you can avoid the garbage collector by manually allocating memory with procs like `alloc`, `alloc0`, `allocShared`, `allocShared0` or `allocCStringArray`. @@ -179,12 +138,14 @@ The garbage collector won't try to free them, you need to call their respective when you are done with them or they will leak. + Heap dump ========= The heap dump feature is still in its infancy, but it already proved useful for us, so it might be useful for you. To get a heap dump, compile -with `-d:nimTypeNames` and call `dumpNumberOfInstances` at a strategic place in your program. +with `-d:nimTypeNames`:option: and call `dumpNumberOfInstances` +at a strategic place in your program. This produces a list of the used types in your program and for every type the total amount of object instances for this type as well as the total amount of bytes these instances take up. diff --git a/doc/regexprs.txt b/doc/regexprs.txt index b7370d858..fa7f9d24a 100644 --- a/doc/regexprs.txt +++ b/doc/regexprs.txt @@ -47,16 +47,18 @@ full documentation of Perl's regular expressions. Because the backslash ``\`` is a meta character both in the Nim programming language and in regular expressions, it is strongly recommended that one uses the *raw* strings of Nim, so that -backslashes are interpreted by the regular expression engine:: +backslashes are interpreted by the regular expression engine: +```nim r"\S" # matches any character that is not whitespace +``` A regular expression is a pattern that is matched against a subject string from left to right. Most characters stand for themselves in a pattern, and match the corresponding characters in the subject. As a trivial example, -the pattern:: +the pattern: - The quick brown fox + The quick brown fox matches a portion of a subject string that is identical to itself. The power of regular expressions comes from the ability to include @@ -128,7 +130,7 @@ in patterns in a visible manner. There is no restriction on the appearance of non-printing characters, apart from the binary zero that terminates a pattern, but when a pattern is being prepared by text editing, it is usually easier to use one of the following escape sequences than the binary character it -represents:: +represents: ============== ============================================================ character meaning @@ -146,7 +148,7 @@ character meaning After ``\x``, from zero to two hexadecimal digits are read (letters can be in upper or lower case). In UTF-8 mode, any number of hexadecimal digits may appear between ``\x{`` and ``}``, but the value of the character code must be -less than 2**31 (that is, the maximum hexadecimal value is 7FFFFFFF). If +less than 2^31 (that is, the maximum hexadecimal value is 7FFFFFFF). If characters other than hexadecimal digits appear between ``\x{`` and ``}``, or if there is no terminating ``}``, this form of escape is not recognized. Instead, the initial ``\x`` will be interpreted as a basic hexadecimal escape, @@ -244,7 +246,7 @@ The fourth use of backslash is for certain `simple assertions`:idx:. An assertion specifies a condition that has to be met at a particular point in a match, without consuming any characters from the subject string. The use of subpatterns for more complicated assertions is described below. The -backslashed assertions are:: +backslashed assertions are: ============== ============================================================ assertion meaning diff --git a/doc/rstcommon.rst b/doc/rstcommon.rst new file mode 100644 index 000000000..e9cc615db --- /dev/null +++ b/doc/rstcommon.rst @@ -0,0 +1,51 @@ +.. + Usage of this file: + Add this in the beginning of *.rst file: + + .. default-role:: code + .. include:: rstcommon.rst + + It's the current trick for brevity and compatibility with both Github and + rst2html.py, considering that Github cannot highlight Nim in + RST files anyway and it does not include files. + This way interpreted text is displayed with monospaced font in Github + and it's displayed an Nim code in both rst2html.py + (note ".. default-role:: Nim" above) and `nim rst2html`. + + For files that are user manual and consist of stuff like cmdline + option description, use 'code' as a **real** default role: + + .. include:: rstcommon.rst + .. default-role:: code + +.. define language roles explicitly (for compatibility with rst2html.py): + +.. role:: nim(code) + :language: nim + +.. default-role:: nim + +.. role:: c(code) + :language: c + +.. role:: cpp(code) + :language: cpp + +.. role:: yaml(code) + :language: yaml + +.. role:: python(code) + :language: python + +.. role:: java(code) + :language: java + +.. role:: csharp(code) + :language: csharp + +.. role:: cmd(code) + +.. role:: program(code) + +.. role:: option(code) + diff --git a/doc/sets_fragment.txt b/doc/sets_fragment.txt index 8436190a0..35b1fc023 100644 --- a/doc/sets_fragment.txt +++ b/doc/sets_fragment.txt @@ -1,27 +1,35 @@ The set type models the mathematical notion of a set. The set's basetype can only be an ordinal type of a certain size, namely: -* ``int8``-``int16`` -* ``uint8``/``byte``-``uint16`` -* ``char`` -* ``enum`` - -or equivalent. For signed integers the set's base type is defined to be in the -range ``0 .. MaxSetElements-1`` where ``MaxSetElements`` is currently always -2^16. +* `int8`-`int16` +* `uint8`/`byte`-`uint16` +* `char` +* `enum` +* Ordinal subrange types, i.e. `range[-10..10]` + +or equivalent. When constructing a set with signed integer literals, the set's +base type is defined to be in the range `0 .. DefaultSetElements-1` where +`DefaultSetElements` is currently always 2^8. The maximum range length for the +base type of a set is `MaxSetElements` which is currently always 2^16. Types +with a bigger range length are coerced into the range `0 .. MaxSetElements-1`. The reason is that sets are implemented as high performance bit vectors. Attempting to declare a set with a larger type will result in an error: -.. code-block:: nim +```nim + var s: set[int64] # Error: set is too large; use `std/sets` for ordinal types + # with more than 2^16 elements +``` + - var s: set[int64] # Error: set is too large +**Note:** Nim also offers [hash sets](sets.html) (which you need to import +with `import std/sets`), which have no such restrictions. -Sets can be constructed via the set constructor: ``{}`` is the empty set. The +Sets can be constructed via the set constructor: `{}` is the empty set. The empty set is type compatible with any concrete set type. The constructor can also be used to include elements (and ranges of elements): -.. code-block:: nim + ```nim type CharSet = set[char] var @@ -29,37 +37,44 @@ can also be used to include elements (and ranges of elements): x = {'a'..'z', '0'..'9'} # This constructs a set that contains the # letters from 'a' to 'z' and the digits # from '0' to '9' + ``` + +The module [`std/setutils`](setutils.html) provides a way to initialize a set from an iterable: + +```nim +import std/setutils + +let uniqueChars = myString.toSet +``` These operations are supported by sets: ================== ======================================================== operation meaning ================== ======================================================== -``A + B`` union of two sets -``A * B`` intersection of two sets -``A - B`` difference of two sets (A without B's elements) -``A == B`` set equality -``A <= B`` subset relation (A is subset of B or equal to B) -``A < B`` strict subset relation (A is a proper subset of B) -``e in A`` set membership (A contains element e) -``e notin A`` A does not contain element e -``contains(A, e)`` A contains element e -``card(A)`` the cardinality of A (number of elements in A) -``incl(A, elem)`` same as ``A = A + {elem}`` -``excl(A, elem)`` same as ``A = A - {elem}`` +`A + B` union of two sets +`A * B` intersection of two sets +`A - B` difference of two sets (A without B's elements) +`A == B` set equality +`A <= B` subset relation (A is subset of B or equal to B) +`A < B` strict subset relation (A is a proper subset of B) +`e in A` set membership (A contains element e) +`e notin A` A does not contain element e +`contains(A, e)` A contains element e +`card(A)` the cardinality of A (number of elements in A) +`incl(A, elem)` same as `A = A + {elem}` +`excl(A, elem)` same as `A = A - {elem}` ================== ======================================================== -Bit fields -~~~~~~~~~~ +### Bit fields Sets are often used to define a type for the *flags* of a procedure. This is a cleaner (and type safe) solution than defining integer -constants that have to be ``or``'ed together. +constants that have to be `or`'ed together. Enum, sets and casting can be used together as in: -.. code-block:: nim - + ```nim type MyFlag* {.size: sizeof(cint).} = enum A @@ -77,10 +92,11 @@ Enum, sets and casting can be used together as in: assert toNum({A, C}) == 5 assert toFlags(0) == {} assert toFlags(7) == {A, B, C} + ``` Note how the set turns enum values into powers of 2. If using enums and sets with C, use distinct cint. For interoperability with C see also the -`bitsize pragma <manual.html#implementation-specific-pragmas-bitsize-pragma>`_. +[bitsize pragma](manual.html#implementation-specific-pragmas-bitsize-pragma). diff --git a/doc/spawn.txt b/doc/spawn.txt index c437e8aa3..ed25ad5fd 100644 --- a/doc/spawn.txt +++ b/doc/spawn.txt @@ -6,7 +6,7 @@ Nim has two flavors of parallelism: 1) `Structured`:idx parallelism via the ``parallel`` statement. 2) `Unstructured`:idx: parallelism via the standalone ``spawn`` statement. -Both need the `threadpool <threadpool.html>`_ module to work. +Both need the [threadpool](threadpool.html) module to work. Somewhat confusingly, ``spawn`` is also used in the ``parallel`` statement with slightly different semantics. ``spawn`` always takes a call expression of @@ -28,7 +28,7 @@ the passed expression on the thread pool and returns a `data flow variable`:idx: **blocking**. However, one can use ``blockUntilAny`` to wait on multiple flow variables at the same time: -.. code-block:: nim + ```nim import std/threadpool, ... # wait until 2 out of 3 servers received the update: @@ -40,6 +40,7 @@ variables at the same time: assert index >= 0 responses.del(index) discard blockUntilAny(responses) + ``` Data flow variables ensure that no data races are possible. Due to technical limitations not every type ``T`` is possible in @@ -54,7 +55,7 @@ Parallel statement Example: -.. code-block:: nim + ```nim # Compute PI in an inefficient way import std/[strutils, math, threadpool] @@ -69,6 +70,7 @@ Example: result += ch[k] echo formatFloat(pi(5000)) + ``` The parallel statement is the preferred mechanism to introduce parallelism diff --git a/doc/subexes.txt b/doc/subexes.txt index 62ddd1ec8..1bfd60213 100644 --- a/doc/subexes.txt +++ b/doc/subexes.txt @@ -47,8 +47,7 @@ Notation meaning Examples ======== -.. code-block:: nim - + ```nim subex"$1($', '{2..})" % ["f", "a", "b", "c"] == "f(a, b, c)" subex"$1 $[files|file|files]{1} copied" % ["1"] == "1 file copied" @@ -57,5 +56,5 @@ Examples subex("type\n TEnum = enum\n $', '40c'\n '{..}") % [ "fieldNameA", "fieldNameB", "fieldNameC", "fieldNameD"] - + ``` diff --git a/doc/testament.md b/doc/testament.md new file mode 100644 index 000000000..0ff3591ac --- /dev/null +++ b/doc/testament.md @@ -0,0 +1,390 @@ +=========== + Testament +=========== + +.. default-role:: code +.. include:: rstcommon.rst +.. contents:: + +Testament is an advanced automatic unittests runner for Nim tests, is used for the development of Nim itself, +offers process isolation for your tests, it can generate statistics about test cases, +supports multiple targets (C, C++, ObjectiveC, JavaScript, etc.), +simulated [Dry-Runs](https://en.wikipedia.org/wiki/Dry_run_(testing)), +has logging, can generate HTML reports, skip tests from a file, and more, +so can be useful to run your tests, even the most complex ones. + + +Test files location +=================== + +By default, Testament looks for test files on ``"./tests/category/*.nim"``. +You can overwrite this pattern glob using `pattern <glob>`:option:. +The default working directory path can be changed using +`--directory:"folder/subfolder/"`:option:. + +Testament uses the `nim`:cmd: compiler on `PATH`. +You can change that using `--nim:"folder/subfolder/nim"`:option:. +Running JavaScript tests with `--targets:"js"`:option: requires +a working NodeJS on `PATH`. + +Commands +======== + +p|pat|pattern <glob> run all the tests matching the given pattern +all run all tests inside of category folders +c|cat|category <category> run all the tests of a certain category +r|run <test> run single test file +html generate testresults.html from the database + + +Options +======= + +--print print results to the console +--verbose print commands (compiling and running tests) +--simulate see what tests would be run but don't run them (for debugging) +--failing only show failing/ignored tests +--targets:"c cpp js objc" run tests for specified targets (default: c) +--nim:path use a particular nim executable (default: $PATH/nim) +--directory:dir Change to directory dir before reading the tests or doing anything else. +--colors:on|off Turn messages coloring on|off. +--backendLogging:on|off Disable or enable backend logging. By default turned on. +--megatest:on|off Enable or disable megatest. Default is on. +--valgrind:on|off Enable or disable valgrind support. Default is on. +--skipFrom:file Read tests to skip from `file` - one test per line, # comments ignored + + +Running a single test +===================== + +This is a minimal example to understand the basics, +not very useful for production, but easy to understand: + + ```console + $ mkdir -p tests/category + $ echo "assert 42 == 42" > tests/category/test0.nim + $ testament run tests/category/test0.nim + PASS: tests/category/test0.nim c ( 0.2 sec) + $ testament r tests/category/test0 + PASS: tests/category/test0.nim C ( 0.2 sec) + ``` + + +Running all tests from a directory +================================== + + This will run all tests in the top level tests/ directory. NOTE: these + tests are skipped by `testament all`. + + ```console + $ testament pattern "tests/*.nim" + ``` + +To search for tests deeper in a directory, use + + ```console + $ testament pattern "tests/**/*.nim" # one level deeper + $ testament pattern "tests/**/**/*.nim" # two levels deeper + ``` + +HTML Reports +============ + +Generate HTML Reports ``testresults.html`` from unittests, +you have to run at least 1 test *before* generating a report: + + ```console + $ testament html + ``` + + +Writing Unit tests +================== + +Example "template" **to edit** and write a Testament unittest: + + ```nim + discard """ + + # What actions to expect completion on. + # Options: + # "compile": expect successful compilation + # "run": expect successful compilation and execution + # "reject": expect failed compilation. The "reject" action can catch + # {.error.} pragmas but not {.fatal.} pragmas because + # {.error.} calls are expected to originate from the test-file, + # and can explicitly be specified using the "file", "line" and + # "column" options. + # {.fatal.} pragmas guarantee that compilation will be aborted. + action: "run" + + # For testing failed compilations you can specify the expected origin of the + # compilation error. + # With the "file", "line" and "column" options you can define the file, + # line and column that a compilation-error should have originated from. + # Use only with action: "reject" as it expects a failed compilation. + # Requires errormsg or msg to be defined. + # file: "" + # line: "" + # column: "" + + # The exit code that the test is expected to return. Typically, the default + # value of 0 is fine. Note that if the test will be run by valgrind, then + # the test will exit with either a code of 0 on success or 1 on failure. + exitcode: 0 + + # Provide an `output` string to assert that the test prints to standard out + # exactly the expected string. Provide an `outputsub` string to assert that + # the string given here is a substring of the standard out output of the + # test (the output includes both the compiler and test execution output). + output: "" + outputsub: "" + + # Whether to sort the compiler output lines before comparing them to the + # expected output. + sortoutput: true + + # Provide a `nimout` string to assert that the compiler during compilation + # prints the defined lines to the standard out. + # The lines must match in order, but there may be more lines that appear + # before, after, or in between them. + nimout: ''' + a very long, + multi-line + string''' + + # This is the Standard Input the test should take, if any. + input: "" + + # Error message the test should print, if any. + errormsg: "" + + # Can be run in batch mode, or not. + batchable: true + + # Can be run Joined with other tests to run all together, or not. + joinable: true + + # On Linux 64-bit machines, whether to use Valgrind to check for bad memory + # accesses or memory leaks. On other architectures, the test will be run + # as-is, without Valgrind. + # Options: + # true: run the test with Valgrind + # false: run the without Valgrind + # "leaks": run the test with Valgrind, but do not check for memory leaks + valgrind: false # Can use Valgrind to check for memory leaks, or not (Linux 64Bit only). + + # Checks that the specified piece of C-code is within the generated C-code. + ccodecheck: "'Assert error message'" + + # Command the test should use to run. If left out or an empty string is + # provided, the command is taken to be: + # "nim $target --hints:on -d:testing --nimblePath:build/deps/pkgs $options $file" + # Subject to variable interpolation. + cmd: "nim c -r $file" + + # Maximum generated temporary intermediate code file size for the test. + maxcodesize: 666 + + # Timeout seconds to run the test. Fractional values are supported. + timeout: 1.5 + + # Targets to run the test into (c, cpp, objc, js). Defaults to c. + targets: "c js" + + # flags with which to run the test, delimited by `;` + matrix: "; -d:release; -d:caseFoo -d:release" + + # Conditions that will skip this test. Use of multiple "disabled" clauses + # is permitted. + disabled: "bsd" # Can disable OSes... + disabled: "win" + disabled: "32bit" # ...or architectures + disabled: "i386" + disabled: "azure" # ...or pipeline runners + disabled: true # ...or can disable the test entirely + + """ + assert true + assert 42 == 42, "Assert error message" + ``` + + +* As you can see the "Spec" is just a `discard """ """`. +* Spec has sane defaults, so you don't need to provide them all, any simple assert will work just fine. +* This is not the full spec of Testament, check [the Testament Spec on GitHub, + see parseSpec()](https://github.com/nim-lang/Nim/blob/devel/testament/specs.nim#L317). +* Nim itself uses Testament, so [there are plenty of test examples]( + https://github.com/nim-lang/Nim/tree/devel/tests). +* Has some built-in CI compatibility, like Azure Pipelines, etc. + + +Inline hints, warnings and errors (notes) +----------------------------------------- + +Testing the line, column, kind and message of hints, warnings and errors can +be written inline like so: + ```nim + {.warning: "warning!!"} #[tt.Warning + ^ warning!! [User] ]# + ``` + +The opening `#[tt.` marks the message line. +The `^` marks the message column. + +Inline messages can be combined with `nimout` when `nimoutFull` is false (default). +This allows testing for expected messages from other modules: + + ```nim + discard """ + nimout: "config.nims(1, 1) Hint: some hint message [User]" + """ + {.warning: "warning!!"} #[tt.Warning + ^ warning!! [User] ]# + ``` + +Multiple messages for a line can be checked by delimiting messages with ';': + + ```nim + discard """ + matrix: "--errorMax:0 --styleCheck:error" + """ + + proc generic_proc*[T](a_a: int) = #[tt.Error + ^ 'generic_proc' should be: 'genericProc'; tt.Error + ^ 'a_a' should be: 'aA' ]# + discard + ``` + +Use `--errorMax:0` in `matrix`, or `cmd: "nim check $file"` when testing +for multiple 'Error' messages. + +Output message variable interpolation +------------------------------------- + +`errormsg`, `nimout`, and inline messages are subject to these variable interpolations: + +* `${/}` - platform's directory separator +* `$file` - the filename (without directory) of the test + +All other `$` characters need escaped as `$$`. + +Cmd variable interpolation +-------------------------- + +The `cmd` option is subject to these variable interpolations: + +* `$target` - the compilation target, e.g. `c`. +* `$options` - the options for the compiler. +* `$file` - the file path of the test. +* `$filedir` - the directory of the test file. + +.. code-block:: nim + + discard """ + cmd: "nim $target --nimblePath:./nimbleDir/simplePkgs $options $file" + """ + +All other `$` characters need escaped as `$$`. + + +Unit test Examples +================== + +Expected to fail: + + ```nim + discard """ + errormsg: "undeclared identifier: 'not_defined'" + """ + assert not_defined == "not_defined", "not_defined is not defined" + ``` + +Expected to fail with error thrown from another file: +```nim +# test.nim +discard """ + action: "reject" + errorMsg: "I break" + file: "breakPragma.nim" +""" +import ./breakPragma + +proc x() {.justDo.} = discard + +# breakPragma.nim +import std/macros + +macro justDo*(procDef: typed): untyped = + error("I break") + return procDef +``` + +Expecting generated C to contain a given piece of code: + + ```nim + discard """ + # Checks that the string "Assert error message" is in the generated + # C code. + ccodecheck: "'Assert error message'" + """ + assert 42 == 42, "Assert error message" + ``` + + +Non-Zero exit code: + + ```nim + discard """ + exitcode: 1 + """ + quit "Non-Zero exit code", 1 + ``` + +Standard output checking: + + ```nim + discard """ + + output: ''' + 0 + 1 + 2 + 3 + 4 + 5 + ''' + + """ + for i in 0..5: echo i + ``` + +JavaScript tests: + + ```nim + discard """ + targets: "js" + """ + when defined(js): + import std/jsconsole + console.log("My Frontend Project") + ``` + +Compile-time tests: + + ```nim + discard """ + action: "compile" + """ + static: assert 9 == 9, "Compile time assert" + ``` + +Tests without Spec: + + ```nim + assert 1 == 1 + ``` + + +See also: +* [Unittest](unittest.html) diff --git a/doc/testament.rst b/doc/testament.rst deleted file mode 100644 index 04c966ffe..000000000 --- a/doc/testament.rst +++ /dev/null @@ -1,252 +0,0 @@ -.. default-role:: code - -Testament is an advanced automatic unittests runner for Nim tests, is used for the development of Nim itself, -offers process isolation for your tests, it can generate statistics about test cases, -supports multiple targets (C, C++, ObjectiveC, JavaScript, etc), -simulated `Dry-Runs <https://en.wikipedia.org/wiki/Dry_run_(testing)>`_, -has logging, can generate HTML reports, skip tests from a file, and more, -so can be useful to run your tests, even the most complex ones. - - -Test files location -=================== - -By default Testament looks for test files on `"./tests/*.nim"`. -You can overwrite this pattern glob using `pattern <glob>`. -The default working directory path can be changed using -`--directory:"folder/subfolder/"`. - -Testament uses the `nim` compiler on `PATH`. -You can change that using `--nim:"folder/subfolder/nim"`. -Running JavaScript tests with `--targets:"js"` requires a working NodeJS on -`PATH`. - - -Options -======= - -* `--print` Also print results to the console -* `--simulate` See what tests would be run but don't run them (for debugging) -* `--failing` Only show failing/ignored tests -* `--targets:"c cpp js objc"` Run tests for specified targets (default: all) -* `--nim:path` Use a particular nim executable (default: `$PATH/nim`) -* `--directory:dir` Change to directory dir before reading the tests or doing anything else. -* `--colors:on|off` Turn messages coloring on|off. -* `--backendLogging:on|off` Disable or enable backend logging. By default turned on. -* `--skipFrom:file` Read tests to skip from `file` - one test per line, # comments ignored - - -Running a single test -===================== - -This is a minimal example to understand the basics, -not very useful for production, but easy to understand: - -.. code:: - - $ mkdir tests - $ echo "assert 42 == 42" > tests/test0.nim - $ testament run test0.nim - PASS: tests/test0.nim C ( 0.2 sec) - - $ testament r test0 - PASS: tests/test0.nim C ( 0.2 sec) - - -Running all tests from a directory -================================== - -.. code:: - - $ testament pattern "tests/*.nim" - -To search for tests deeper in a directory, use - -.. code:: - - $ testament pattern "tests/**/*.nim" # one level deeper - $ testament pattern "tests/**/**/*.nim" # two levels deeper - -HTML Reports -============ - -Generate HTML Reports `testresults.html` from unittests, -you have to run at least 1 test *before* generating a report: - -.. code:: - - $ testament html - - -Writing Unitests -================ - -Example "template" **to edit** and write a Testament unittest: - -.. code-block:: nim - - discard """ - - # What actions to expect completion on. - # Options: - # "compile": expect successful compilation - # "run": expect successful compilation and execution - # "reject": expect failed compilation. The "reject" action can catch - # {.error.} pragmas but not {.fatal.} pragmas because - # {.fatal.} pragmas guarantee that compilation will be aborted. - action: "run" - - # The exit code that the test is expected to return. Typically, the default - # value of 0 is fine. Note that if the test will be run by valgrind, then - # the test will exit with either a code of 0 on success or 1 on failure. - exitcode: 0 - - # Provide an `output` string to assert that the test prints to standard out - # exatly the expected string. Provide an `outputsub` string to assert that - # the string given here is a substring of the standard out output of the - # test. - output: "" - outputsub: "" - - # Whether to sort the output lines before comparing them to the desired - # output. - sortoutput: true - - # Each line in the string given here appears in the same order in the - # compiler output, but there may be more lines that appear before, after, or - # in between them. - nimout: ''' - a very long, - multi-line - string''' - - # This is the Standard Input the test should take, if any. - input: "" - - # Error message the test should print, if any. - errormsg: "" - - # Can be run in batch mode, or not. - batchable: true - - # Can be run Joined with other tests to run all togheter, or not. - joinable: true - - # On Linux 64-bit machines, whether to use Valgrind to check for bad memory - # accesses or memory leaks. On other architectures, the test will be run - # as-is, without Valgrind. - # Options: - # true: run the test with Valgrind - # false: run the without Valgrind - # "leaks": run the test with Valgrind, but do not check for memory leaks - valgrind: false # Can use Valgrind to check for memory leaks, or not (Linux 64Bit only). - - # Command the test should use to run. If left out or an empty string is - # provided, the command is taken to be: - # "nim $target --hints:on -d:testing --nimblePath:build/deps/pkgs $options $file" - # You can use the $target, $options, and $file placeholders in your own - # command, too. - cmd: "nim c -r $file" - - # Maximum generated temporary intermediate code file size for the test. - maxcodesize: 666 - - # Timeout seconds to run the test. Fractional values are supported. - timeout: 1.5 - - # Targets to run the test into (c, cpp, objc, js). - targets: "c js" - - # flags with which to run the test, delimited by `;` - matrix: "; -d:release; -d:caseFoo -d:release" - - # Conditions that will skip this test. Use of multiple "disabled" clauses - # is permitted. - disabled: "bsd" # Can disable OSes... - disabled: "win" - disabled: "32bit" # ...or architectures - disabled: "i386" - disabled: "azure" # ...or pipeline runners - disabled: true # ...or can disable the test entirely - - """ - assert true - assert 42 == 42, "Assert error message" - - -* As you can see the "Spec" is just a `discard """ """`. -* Spec has sane defaults, so you don't need to provide them all, any simple assert will work just fine. -* `This is not the full spec of Testament, check the Testament Spec on GitHub, see parseSpec(). <https://github.com/nim-lang/Nim/blob/devel/testament/specs.nim#L238>`_ -* `Nim itself uses Testament, so there are plenty of test examples. <https://github.com/nim-lang/Nim/tree/devel/tests>`_ -* Has some built-in CI compatibility, like Azure Pipelines, etc. -* `Testament supports inlined error messages on Unittests, basically comments with the expected error directly on the code. <https://github.com/nim-lang/Nim/blob/9a110047cbe2826b1d4afe63e3a1f5a08422b73f/tests/effects/teffects1.nim>`_ - - -Unitests Examples -================= - -Expected to fail: - -.. code-block:: nim - - discard """ - errormsg: "undeclared identifier: 'not_defined'" - """ - assert not_defined == "not_defined", "not_defined is not defined" - -Non-Zero exit code: - -.. code-block:: nim - - discard """ - exitcode: 1 - """ - quit "Non-Zero exit code", 1 - -Standard output checking: - -.. code-block:: nim - - discard """ - - output: ''' - 0 - 1 - 2 - 3 - 4 - 5 - ''' - - """ - for i in 0..5: echo i - -JavaScript tests: - -.. code-block:: nim - - discard """ - targets: "js" - """ - when defined(js): - import std/jsconsole - console.log("My Frontend Project") - -Compile-time tests: - -.. code-block:: nim - - discard """ - action: "compile" - """ - static: assert 9 == 9, "Compile time assert" - -Tests without Spec: - -.. code-block:: nim - - assert 1 == 1 - - -See also: -* `Unittest <unittest.html>`_ diff --git a/doc/tools.rst b/doc/tools.md index e0044e1ca..baf7ce386 100644 --- a/doc/tools.rst +++ b/doc/tools.md @@ -1,41 +1,45 @@ -.. default-role:: code - ======================== Tools available with Nim ======================== +.. default-role:: code +.. include:: rstcommon.rst + The standard distribution ships with the following tools: -- | `Hot code reloading <hcr.html>`_ +- | [atlas](atlas.html) + | `atlas`:cmd: is a simple package cloner tool. It manages an isolated workspace that + contains projects and dependencies. + +- | [Hot code reloading](hcr.html) | The "Hot code reloading" feature is built into the compiler but has its own document explaining how it works. -- | `Documentation generator <docgen.html>`_ - | The builtin document generator `nim doc` generates HTML documentation - from `.nim` source files. +- | [Documentation generator](docgen.html) + | The builtin document generator `nim doc`:cmd: generates HTML documentation + from ``.nim`` source files. -- | `Nimsuggest for IDE support <nimsuggest.html>`_ - | Through the `nimsuggest` tool, any IDE can query a `.nim` source file +- | [Nimsuggest for IDE support](nimsuggest.html) + | Through the `nimsuggest`:cmd: tool, any IDE can query a ``.nim`` source file and obtain useful information like the definition of symbols or suggestions for completion. -- | `C2nim <https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst>`_ +- | [C2nim](https://github.com/nim-lang/c2nim/blob/master/doc/c2nim.rst) | C to Nim source converter. Translates C header files to Nim. -- | `niminst <niminst.html>`_ +- | [niminst](niminst.html) | niminst is a tool to generate an installer for a Nim program. -- | `nimgrep <nimgrep.html>`_ +- | [nimgrep](nimgrep.html) | Nim search and replace utility. - | nimpretty - | `nimpretty` is a Nim source code beautifier, + | `nimpretty`:cmd: is a Nim source code beautifier, to format code according to the official style guide. -- | `testament <https://nim-lang.github.io/Nim/testament.html>`_ - | `testament` is an advanced automatic *unittests runner* for Nim tests, +- | [testament](testament.html) + | `testament`:cmd: is an advanced automatic *unittests runner* for Nim tests, is used for the development of Nim itself, offers process isolation for your tests, it can generate statistics about test cases, supports multiple targets (C, JS, etc), - `simulated Dry-Runs <https://en.wikipedia.org/wiki/Dry_run_(testing)>`_, has logging, can generate HTML reports, skip tests from a file, and more, so can be useful to run your tests, even the most complex ones. diff --git a/doc/tut1.rst b/doc/tut1.md index d2d6f8fe7..2e83effa3 100644 --- a/doc/tut1.rst +++ b/doc/tut1.md @@ -1,5 +1,3 @@ -.. default-role:: code - ===================== Nim Tutorial (Part I) ===================== @@ -7,28 +5,31 @@ Nim Tutorial (Part I) :Author: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: Introduction ============ -.. raw:: html - <blockquote><p> - "Der Mensch ist doch ein Augentier -- schöne Dinge wünsch ich mir." - </p></blockquote> +> "Der Mensch ist doch ein Augentier -- Schöne Dinge wünsch' ich mir." This document is a tutorial for the programming language *Nim*. This tutorial assumes that you are familiar with basic programming concepts like variables, types, or statements. -If you would like to have a gentle introduction of those concepts, we recommend -`Nim Basics tutorial <https://narimiran.github.io/nim-basics/>`_. -On the other hand, the `manual <manual.html>`_ contains many more examples of -the advanced language features. + +Here are several other resources for learning Nim: + +* [Nim Basics tutorial](https://narimiran.github.io/nim-basics/) - a gentle + introduction of the concepts mentioned above +* [Learn Nim in 5 minutes](https://learnxinyminutes.com/docs/nim/) - quick, + five-minute introduction to Nim +* [The Nim manual](manual.html) - many more examples of the advanced language features All code examples in this tutorial, as well as the ones found in the rest of -Nim's documentation, follow the `Nim style guide <nep1.html>`_. +Nim's documentation, follow the [Nim style guide](nep1.html). The first program @@ -36,36 +37,46 @@ The first program We start the tour with a modified "hello world" program: -.. code-block:: Nim - :test: "nim c $1" + ```Nim test = "nim c $1" # This is a comment echo "What's your name? " var name: string = readLine(stdin) echo "Hi, ", name, "!" + ``` -Save this code to the file "greetings.nim". Now compile and run it:: - +Save this code to the file "greetings.nim". Now compile and run it: + ```cmd nim compile --run greetings.nim + ``` -With the `--run` `switch <nimc.html#compiler-usage-commandminusline-switches>`_ Nim +With the ``--run`` [switch](nimc.html#compiler-usage-commandminusline-switches) Nim executes the file automatically after compilation. You can give your program -command-line arguments by appending them after the filename:: - +command-line arguments by appending them after the filename: + ```cmd nim compile --run greetings.nim arg1 arg2 + ``` -Commonly used commands and switches have abbreviations, so you can also use:: - +Commonly used commands and switches have abbreviations, so you can also use: + ```cmd nim c -r greetings.nim + ``` -To compile a release version use:: - +This is a **debug version**. +To compile a release version use: + ```cmd nim c -d:release greetings.nim + ``` By default, the Nim compiler generates a large number of runtime checks -aiming for your debugging pleasure. With `-d:release` some checks are -`turned off and optimizations are turned on -<nimc.html#compiler-usage-compileminustime-symbols>`_. +aiming for your debugging pleasure. With ``-d:release`` some checks are +[turned off and optimizations are turned on]( +nimc.html#compiler-usage-compileminustime-symbols). + +For benchmarking or production code, use the ``-d:release`` switch. +For comparing the performance with unsafe languages like C, use the ``-d:danger`` switch +in order to get meaningful, comparable results. Otherwise, Nim might be handicapped +by checks that are **not even available** for C. Though it should be pretty obvious what the program does, I will explain the syntax: statements which are not indented are executed when the program @@ -74,21 +85,21 @@ done with spaces only, tabulators are not allowed. String literals are enclosed in double-quotes. The `var` statement declares a new variable named `name` of type `string` with the value that is -returned by the `readLine <io.html#readLine,File>`_ procedure. Since the -compiler knows that `readLine <io.html#readLine,File>`_ returns a string, +returned by the [readLine](syncio.html#readLine,File) procedure. Since the +compiler knows that [readLine](syncio.html#readLine,File) returns a string, you can leave out the type in the declaration (this is called `local type inference`:idx:). So this will work too: -.. code-block:: Nim - :test: "nim c $1" + ```Nim test = "nim c $1" var name = readLine(stdin) + ``` Note that this is basically the only form of type inference that exists in Nim: it is a good compromise between brevity and readability. The "hello world" program contains several identifiers that are already known -to the compiler: `echo`, `readLine <io.html#readLine,File>`_, etc. -These built-ins are declared in the system_ module which is implicitly +to the compiler: `echo`, [readLine](syncio.html#readLine,File), etc. +These built-ins are declared in the [system](system.html) module which is implicitly imported by any other module. @@ -104,17 +115,18 @@ String and character literals ----------------------------- String literals are enclosed in double-quotes; character literals in single -quotes. Special characters are escaped with ``\\``: ``\n`` means newline, ``\t`` +quotes. Special characters are escaped with ``\``: ``\n`` means newline, ``\t`` means tabulator, etc. There are also *raw* string literals: -.. code-block:: Nim + ```Nim r"C:\program files\nim" + ``` In raw literals, the backslash is not an escape character. The third and last way to write string literals is *long-string literals*. They are written with three quotes: `""" ... """`; they can span over -multiple lines and the ``\\`` is not an escape character either. They are very +multiple lines and the ``\`` is not an escape character either. They are very useful for embedding HTML code templates for example. @@ -124,11 +136,11 @@ Comments Comments start anywhere outside a string or character literal with the hash character `#`. Documentation comments start with `##`: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" # A comment. var myVariable: int ## a documentation comment + ``` Documentation comments are tokens; they are only allowed at certain places in @@ -138,8 +150,7 @@ documentation generators. Multiline comments are started with `#[` and terminated with `]#`. Multiline comments can also be nested. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" #[ You can have any Nim code text commented out inside this with no indentation restrictions. @@ -148,6 +159,7 @@ comments can also be nested. Note: these can be nested!! ]# ]# + ``` Numbers @@ -165,49 +177,19 @@ The var statement ================= The var statement declares a new local or global variable: -.. code-block:: + ```nim var x, y: int # declares x and y to have the type `int` + ``` Indentation can be used after the `var` keyword to list a whole section of variables: -.. code-block:: - :test: "nim c $1" + ```nim test = "nim c $1" var x, y: int # a comment can occur here too a, b, c: string - - -The assignment statement -======================== - -The assignment statement assigns a new value to a variable or more generally -to a storage location: - -.. code-block:: - var x = "abc" # introduces a new variable `x` and assigns a value to it - x = "xyz" # assigns a new value to `x` - -`=` is the *assignment operator*. The assignment operator can be -overloaded. You can declare multiple variables with a single assignment -statement and all the variables will have the same value: - -.. code-block:: - :test: "nim c $1" - var x, y = 3 # assigns 3 to the variables `x` and `y` - echo "x ", x # outputs "x 3" - echo "y ", y # outputs "y 3" - x = 42 # changes `x` to 42 without changing `y` - echo "x ", x # outputs "x 42" - echo "y ", y # outputs "y 3" - -Note that declaring multiple variables with a single assignment that calls a -procedure can have unexpected results: the compiler will *unroll* the -assignments and end up calling the procedure several times. If the result of -the procedure depends on side effects, your variables may end up having -different values! For safety use side-effect-free procedures if making multiple -assignments. + ``` Constants @@ -217,20 +199,20 @@ Constants are symbols which are bound to a value. The constant's value cannot change. The compiler must be able to evaluate the expression in a constant declaration at compile time: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" const x = "abc" # the constant x contains the string "abc" + ``` Indentation can be used after the `const` keyword to list a whole section of constants: -.. code-block:: - :test: "nim c $1" + ```nim test = "nim c $1" const x = 1 # a comment can occur here too y = 2 z = y + 5 # computations are possible + ``` The let statement @@ -239,20 +221,47 @@ The `let` statement works like the `var` statement but the declared symbols are *single assignment* variables: After the initialization their value cannot change: -.. code-block:: + ```nim let x = "abc" # introduces a new variable `x` and binds a value to it x = "xyz" # Illegal: assignment to `x` + ``` The difference between `let` and `const` is: `let` introduces a variable that can not be re-assigned, `const` means "enforce compile time evaluation and put it into a data section": -.. code-block:: + ```nim const input = readLine(stdin) # Error: constant expression expected + ``` -.. code-block:: - :test: "nim c $1" + ```nim test = "nim c $1" let input = readLine(stdin) # works + ``` + + +The assignment statement +======================== + +The assignment statement assigns a new value to a variable or more generally +to a storage location: + + ```nim + var x = "abc" # introduces a new variable `x` and assigns a value to it + x = "xyz" # assigns a new value to `x` + ``` + +`=` is the *assignment operator*. The assignment operator can be +overloaded. You can declare multiple variables with a single assignment +statement and all the variables will have the same value: + + ```nim test = "nim c $1" + var x, y = 3 # assigns 3 to the variables `x` and `y` + echo "x ", x # outputs "x 3" + echo "y ", y # outputs "y 3" + x = 42 # changes `x` to 42 without changing `y` + echo "x ", x # outputs "x 42" + echo "y ", y # outputs "y 3" + ``` Control flow statements @@ -268,8 +277,7 @@ If statement The if statement is one way to branch the control flow: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" let name = readLine(stdin) if name == "": echo "Poor soul, you lost your name?" @@ -277,6 +285,7 @@ The if statement is one way to branch the control flow: echo "Very funny, your name is name." else: echo "Hi, ", name, "!" + ``` There can be zero or more `elif` parts, and the `else` part is optional. The keyword `elif` is short for `else if`, and is useful to avoid @@ -287,11 +296,10 @@ characters.) Case statement -------------- -Another way to branch is provided by the case statement. A case statement is -a multi-branch: +Another way to branch is provided by the case statement. A case statement allows +for multiple branches: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" let name = readLine(stdin) case name of "": @@ -302,6 +310,7 @@ a multi-branch: echo "Cool name!" else: echo "Hi, ", name, "!" + ``` As it can be seen, for an `of` branch a comma-separated list of values is also allowed. @@ -310,7 +319,7 @@ The case statement can deal with integers, other ordinal types, and strings. (What an ordinal type is will be explained soon.) For integers or other ordinal types value ranges are also possible: -.. code-block:: nim + ```nim # this statement will be explained later: from std/strutils import parseInt @@ -319,21 +328,23 @@ For integers or other ordinal types value ranges are also possible: case n of 0..2, 4..7: echo "The number is in the set: {0, 1, 2, 4, 5, 6, 7}" of 3, 8: echo "The number is 3 or 8" + ``` -However, the above code does not compile: the reason is that you have to cover +However, the above code **does not compile**: the reason is that you have to cover every value that `n` may contain, but the code only handles the values `0..8`. Since it is not very practical to list every other possible integer (though it is possible thanks to the range notation), we fix this by telling the compiler that for every other value nothing should be done: -.. code-block:: nim + ```nim ... case n of 0..2, 4..7: echo "The number is in the set: {0, 1, 2, 4, 5, 6, 7}" of 3, 8: echo "The number is 3 or 8" else: discard + ``` -The empty `discard statement <#procedures-discard-statement>`_ is a *do +The empty [discard statement] is a *do nothing* statement. The compiler knows that a case statement with an else part cannot fail and thus the error disappears. Note that it is impossible to cover all possible string values: that is why string cases always need an `else` @@ -349,15 +360,13 @@ While statement The while statement is a simple looping construct: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" echo "What's your name? " var name = readLine(stdin) while name == "": echo "Please tell me your name: " - name = readLine(stdin) - # no `var`, because we do not declare a new variable here + name = readLine(stdin) # no `var`, because we do not declare a new variable here + ``` The example uses a while loop to keep asking the users for their name, as long as the user types in nothing (only presses RETURN). @@ -367,112 +376,117 @@ For statement ------------- The `for` statement is a construct to loop over any element an *iterator* -provides. The example uses the built-in `countup -<system.html#countup.i,T,T,Positive>`_ iterator: +provides. The example uses the built-in [countup]( +system.html#countup.i,T,T,Positive) iterator: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" echo "Counting to ten: " for i in countup(1, 10): echo i # --> Outputs 1 2 3 4 5 6 7 8 9 10 on different lines + ``` The variable `i` is implicitly declared by the -`for` loop and has the type `int`, because that is what `countup -<system.html#countup.i,T,T,Positive>`_ returns. `i` runs through the values +`for` loop and has the type `int`, because that is what [countup]( +system.html#countup.i,T,T,Positive) returns. `i` runs through the values 1, 2, .., 10. Each value is `echo`-ed. This code does the same: -.. code-block:: nim + ```nim echo "Counting to 10: " var i = 1 while i <= 10: echo i - inc(i) # increment i by 1 + inc i # increment i by 1 # --> Outputs 1 2 3 4 5 6 7 8 9 10 on different lines + ``` + +Since counting up occurs so often in programs, Nim also has a [..]( +system.html#...i,T,T) iterator that does the same: + + ```nim + for i in 1 .. 10: + ... + ``` Counting down can be achieved as easily (but is less often needed): -.. code-block:: nim + ```nim echo "Counting down from 10 to 1: " for i in countdown(10, 1): echo i # --> Outputs 10 9 8 7 6 5 4 3 2 1 on different lines - -Since counting up occurs so often in programs, Nim also has a `.. -<system.html#...i,T,T>`_ iterator that does the same: - -.. code-block:: nim - for i in 1 .. 10: - ... + ``` Zero-indexed counting has two shortcuts `..<` and `.. ^1` -(`backward index operator <system.html#^.t%2Cint>`_) to simplify +([backward index operator](system.html#^.t%2Cint)) to simplify counting to one less than the higher index: -.. code-block:: nim + ```nim for i in 0 ..< 10: - ... # 0 .. 9 + ... # the same as 0 .. 9 + ``` or -.. code-block:: nim + ```nim var s = "some string" for i in 0 ..< s.len: ... + ``` or -.. code-block:: nim + ```nim var s = "some string" for idx, c in s[0 .. ^1]: - ... + ... # ^1 is the last element, ^2 would be one before it, and so on + ``` Other useful iterators for collections (like arrays and sequences) are * `items` and `mitems`, which provides immutable and mutable elements respectively, and * `pairs` and `mpairs` which provides the element and an index number (immutable and mutable respectively) -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" for index, item in ["a","b"].pairs: echo item, " at index ", index # => a at index 0 # => b at index 1 + ``` Scopes and the block statement ------------------------------ + Control flow statements have a feature not covered yet: they open a new scope. This means that in the following example, `x` is not accessible outside the loop: -.. code-block:: nim - :test: "nim c $1" - :status: 1 + ```nim test = "nim c $1" status = 1 while false: var x = "hi" echo x # does not work + ``` A while (for) statement introduces an implicit block. Identifiers are only visible within the block they have been declared. The `block` statement can be used to open a new block explicitly: -.. code-block:: nim - :test: "nim c $1" - :status: 1 + ```nim test = "nim c $1" status = 1 block myblock: var x = "hi" echo x # does not work either + ``` The block's *label* (`myblock` in the example) is optional. Break statement --------------- + A block can be left prematurely with a `break` statement. The break statement can leave a `while`, `for`, or a `block` statement. It leaves the innermost construct, unless a label of a block is given: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" block myblock: echo "entering block" while true: @@ -488,19 +502,20 @@ innermost construct, unless a label of a block is given: break myblock2 # leaves the block (and the loop) echo "still in block" # it won't be printed echo "outside the block" + ``` Continue statement ------------------ + Like in many other programming languages, a `continue` statement starts the next iteration immediately: -.. code-block:: nim - :test: "nim c $1" - while true: - let x = readLine(stdin) - if x == "": continue - echo x + ```nim test = "nim c $1" + for i in 1 .. 5: + if i <= 3: continue + echo i # will only print 4 and 5 + ``` When statement @@ -508,9 +523,7 @@ When statement Example: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" when system.hostOS == "windows": echo "running on Windows!" elif system.hostOS == "linux": @@ -519,6 +532,7 @@ Example: echo "running on Mac OS X!" else: echo "unknown operating system" + ``` The `when` statement is almost identical to the `if` statement, but with these differences: @@ -530,7 +544,7 @@ differences: that belong to the first condition that evaluates to `true`. The `when` statement is useful for writing platform-specific code, similar to -the `#ifdef` construct in the C programming language. +the `#ifdef`:c: construct in the C programming language. Statements and indentation @@ -546,7 +560,7 @@ statements. *Complex statements* like `if`, `when`, `for`, `while` can contain other statements. To avoid ambiguities, complex statements must always be indented, but single simple statements do not: -.. code-block:: nim + ```nim # no indentation needed for single-assignment statement: if x: x = false @@ -561,18 +575,19 @@ be indented, but single simple statements do not: if x: x = false y = false + ``` *Expressions* are parts of a statement that usually result in a value. The condition in an if statement is an example of an expression. Expressions can contain indentation at certain places for better readability: -.. code-block:: nim - + ```nim if thisIsaLongCondition() and thisIsAnotherLongCondition(1, 2, 3, 4): x = true + ``` As a rule of thumb, indentation within expressions is allowed after operators, an open parenthesis and after commas. @@ -580,22 +595,23 @@ an open parenthesis and after commas. With parenthesis and semicolons `(;)` you can use statements where only an expression is allowed: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" # computes fac(4) at compile time: const fac4 = (var x = 1; for i in 1..4: x *= i; x) + ``` Procedures ========== -To define new commands like `echo <system.html#echo,varargs[typed,]>`_ -and `readLine <io.html#readLine,File>`_ in the examples, the concept of a -`procedure` is needed. (Some languages call them *methods* or *functions*.) -In Nim new procedures are defined with the `proc` keyword: +To define new commands like [echo](system.html#echo,varargs[typed,]) +and [readLine](syncio.html#readLine,File) in the examples, the concept of a +*procedure* is needed. You might be used to them being called *methods* or +*functions* in other languages, but Nim +[differentiates these concepts](tut1.html#procedures-funcs-and-methods). In +Nim, new procedures are defined with the `proc` keyword: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc yes(question: string): bool = echo question, " (y/n)" while true: @@ -608,6 +624,7 @@ In Nim new procedures are defined with the `proc` keyword: echo "I'm sorry Dave, I'm afraid I can't do that." else: echo "I think you know what the problem is just as well as I do." + ``` This example shows a procedure named `yes` that asks the user a `question` and returns true if they answered "yes" (or something similar) and returns @@ -625,23 +642,24 @@ Some terminology: in the example `question` is called a (formal) *parameter*, Result variable --------------- + A procedure that returns a value has an implicit `result` variable declared -that represents the return value. A `return` statement with no expression is +that represents the return value. A `return` statement with no expression is shorthand for `return result`. The `result` value is always returned automatically at the end of a procedure if there is no `return` statement at the exit. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc sumTillNegative(x: varargs[int]): int = for i in x: if i < 0: return result = result + i - echo sumTillNegative() # echos 0 - echo sumTillNegative(3, 4, 5) # echos 12 - echo sumTillNegative(3, 4 , -1 , 6) # echos 7 + echo sumTillNegative() # echoes 0 + echo sumTillNegative(3, 4, 5) # echoes 12 + echo sumTillNegative(3, 4 , -1 , 6) # echoes 7 + ``` The `result` variable is already implicitly declared at the start of the function, so declaring it again with 'var result', for example, would shadow it @@ -654,33 +672,33 @@ A procedure that does not have any `return` statement and does not use the special `result` variable returns the value of its last expression. For example, this procedure -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc helloWorld(): string = - "Hello, World!" + "Hello, World!" + ``` returns the string "Hello, World!". Parameters ---------- + Parameters are immutable in the procedure body. By default, their value cannot be changed because this allows the compiler to implement parameter passing in the most efficient way. If a mutable variable is needed inside the procedure, it has to be declared with `var` in the procedure body. Shadowing the parameter name is possible, and actually an idiom: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc printSeq(s: seq, nprinted: int = -1) = var nprinted = if nprinted == -1: s.len else: min(nprinted, s.len) for i in 0 ..< nprinted: echo s[i] + ``` If the procedure needs to modify the argument for the caller, a `var` parameter can be used: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc divmod(a, b: int; res, remainder: var int) = res = a div b # integer division remainder = a mod b # integer modulo operation @@ -690,6 +708,7 @@ caller, a `var` parameter can be used: divmod(8, 5, x, y) # modifies x and y echo x echo y + ``` In the example, `res` and `remainder` are `var parameters`. Var parameters can be modified by the procedure and the changes are @@ -699,23 +718,25 @@ a tuple as a return value instead of using var parameters. Discard statement ----------------- + To call a procedure that returns a value just for its side effects and ignoring its return value, a `discard` statement **must** be used. Nim does not allow silently throwing away a return value: -.. code-block:: nim + ```nim discard yes("May I ask a pointless question?") + ``` The return value can be ignored implicitly if the called proc/iterator has been declared with the `discardable` pragma: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc p(x, y: int): int {.discardable.} = return x + y p(3, 4) # now valid + ``` Named arguments @@ -723,41 +744,45 @@ Named arguments Often a procedure has many parameters and it is not clear in which order the parameters appear. This is especially true for procedures that construct a -complex data type. Therefore the arguments to a procedure can be named, so +complex data type. Therefore, the arguments to a procedure can be named, so that it is clear which argument belongs to which parameter: -.. code-block:: nim + ```nim proc createWindow(x, y, width, height: int; title: string; show: bool): Window = ... var w = createWindow(show = true, title = "My Application", x = 0, y = 0, height = 600, width = 800) + ``` Now that we use named arguments to call `createWindow` the argument order does not matter anymore. Mixing named arguments with ordered arguments is also possible, but not very readable: -.. code-block:: nim + ```nim var w = createWindow(0, 0, title = "My Application", height = 600, width = 800, true) + ``` The compiler checks that each parameter receives exactly one argument. Default values -------------- + To make the `createWindow` proc easier to use it should provide `default values`; these are values that are used as arguments if the caller does not specify them: -.. code-block:: nim + ```nim proc createWindow(x = 0, y = 0, width = 500, height = 700, title = "unknown", show = true): Window = ... var w = createWindow(title = "My Application", height = 600, width = 800) + ``` Now the call to `createWindow` only needs to set the values that differ from the defaults. @@ -768,9 +793,10 @@ no need to write `title: string = "unknown"`, for example. Overloaded procedures --------------------- + Nim provides the ability to overload procedures similar to C++: -.. code-block:: nim + ```nim proc toString(x: int): string = result = if x < 0: "negative" @@ -784,20 +810,20 @@ Nim provides the ability to overload procedures similar to C++: assert toString(13) == "positive" # calls the toString(x: int) proc assert toString(true) == "yep" # calls the toString(x: bool) proc + ``` -(Note that `toString` is usually the `$ <dollars.html>`_ operator in +(Note that `toString` is usually the [$](dollars.html) operator in Nim.) The compiler chooses the most appropriate proc for the `toString` calls. How this overloading resolution algorithm works exactly is not -discussed here (it will be specified in the manual soon). However, it does -not lead to nasty surprises and is based on a quite simple unification -algorithm. Ambiguous calls are reported as errors. +discussed here -- see the manual for details. Ambiguous calls are reported as errors. Operators --------- -The Nim library makes heavy use of overloading - one reason for this is that + +The Nim standard library makes heavy use of overloading - one reason for this is that each operator like `+` is just an overloaded proc. The parser lets you -use operators in `infix notation` (`a + b`) or `prefix notation` (`+ a`). +use operators in *infix notation* (`a + b`) or *prefix notation* (`+ a`). An infix operator always receives two arguments, a prefix operator always one. (Postfix operators are not possible, because this would be ambiguous: does `a @ @ b` mean `(a) @ (@b)` or `(a@) @ (b)`? It always means @@ -805,27 +831,28 @@ An infix operator always receives two arguments, a prefix operator always one. Apart from a few built-in keyword operators such as `and`, `or`, `not`, operators always consist of these characters: -``+ - * \ / < > = @ $ ~ & % ! ? ^ . |`` +`+ - * \ / < > = @ $ ~ & % ! ? ^ . |` User-defined operators are allowed. Nothing stops you from defining your own `@!?+~` operator, but doing so may reduce readability. The operator's precedence is determined by its first character. The details -can be found in the manual. +can be [found in the manual](manual.html#syntax-precedence). To define a new operator enclose the operator in backticks "`": -.. code-block:: nim + ```nim proc `$` (x: myDataType): string = ... # now the $ operator also works with myDataType, overloading resolution # ensures that $ works for built-in types just like before + ``` The "`" notation can also be used to call an operator just like any other procedure: -.. code-block:: nim - :test: "nim c $1" - if `==`( `+`(3, 4), 7): echo "True" + ```nim test = "nim c $1" + if `==`( `+`(3, 4), 7): echo "true" + ``` Forward declarations @@ -836,11 +863,12 @@ Every variable, procedure, etc. needs to be declared before it can be used. language that supports metaprogramming as extensively as Nim does.) However, this cannot be done for mutually recursive procedures: -.. code-block:: nim + ```nim # forward declaration: proc even(n: int): bool + ``` -.. code-block:: nim + ```nim proc odd(n: int): bool = assert(n >= 0) # makes sure we don't run into negative recursion if n == 0: false @@ -852,12 +880,13 @@ However, this cannot be done for mutually recursive procedures: if n == 1: false else: n == 0 or odd(n-1) + ``` Here `odd` depends on `even` and vice versa. Thus `even` needs to be introduced to the compiler before it is completely defined. The syntax for such a forward declaration is simple: just omit the `=` and the procedure's body. The `assert` just adds border conditions, and will be -covered later in `Modules`_ section. +covered later in [Modules] section. Later versions of the language will weaken the requirements for forward declarations. @@ -866,40 +895,66 @@ The example also shows that a proc's body can consist of a single expression whose value is then returned implicitly. +Funcs and methods +----------------- + +As mentioned in the introduction, Nim differentiates between procedures, +functions, and methods, defined by the `proc`, `func`, and `method` keywords +respectively. In some ways, Nim is a bit more pedantic in its definitions than +other languages. + +Functions are closer to the concept of a pure mathematical +function, which might be familiar to you if you've ever done functional +programming. Essentially they are procedures with additional limitations set on +them: they can't access global state (except `const`) and can't produce +side-effects. The `func` keyword is basically an alias for `proc` tagged +with `{.noSideEffects.}`. Functions can still change their mutable arguments +however, which are those marked as `var`, along with any `ref` objects. + +Unlike procedures, methods are dynamically dispatched. This sounds a bit +complicated, but it is a concept closely related to inheritance and object-oriented +programming. If you overload a procedure (two procedures with the same name but +of different types or with different sets of arguments are said to be overloaded), the procedure to use is determined +at compile-time. Methods, on the other hand, depend on objects that inherit from +the `RootObj`. This is something that is covered in much greater depth in +the [second part of the tutorial](tut2.html#object-oriented-programming-dynamic-dispatch). + + Iterators ========= Let's return to the simple counting example: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" echo "Counting to ten: " for i in countup(1, 10): echo i + ``` -Can a `countup <system.html#countup.i,T,T,Positive>`_ proc be written that -supports this loop? Lets try: +Can a [countup](system.html#countup.i,T,T,Positive) proc be written that +supports this loop? Let's try: -.. code-block:: nim + ```nim proc countup(a, b: int): int = var res = a while res <= b: return res inc(res) + ``` However, this does not work. The problem is that the procedure should not only `return`, but return and **continue** after an iteration has finished. This *return and continue* is called a `yield` statement. Now the only thing left to do is to replace the `proc` keyword by `iterator` -and here it is - our first iterator: +and here it is -- our first iterator: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" iterator countup(a, b: int): int = var res = a while res <= b: yield res inc(res) + ``` Iterators look very similar to procedures, but there are several important differences: @@ -909,16 +964,17 @@ important differences: `yield` statement). * Iterators have no implicit `result` variable. * Iterators do not support recursion. -* Iterators cannot be forward declared, because the compiler must be able to inline an iterator. (This restriction will be gone in a - future version of the compiler.) +* Iterators cannot be forward declared, because the compiler must be able to inline an iterator. + (This restriction will be gone in a future version of the compiler.) -However, you can also use a `closure` iterator to get a different set of -restrictions. See `first-class iterators <manual.html#iterators-and-the-for-statement-firstminusclass-iterators>`_ +However, you can also use a closure iterator to get a different set of +restrictions. See [first-class iterators]( +manual.html#iterators-and-the-for-statement-firstminusclass-iterators) for details. Iterators can have the same name and parameters as a proc since -essentially they have their own namespaces. Therefore it is common practice to +essentially they have their own namespaces. Therefore, it is common to wrap iterators in procs of the same name which accumulate the result of the -iterator and return it as a sequence, like `split` from the `strutils module -<strutils.html>`_. +iterator and return it as a sequence, like `split` from the [strutils module]( +strutils.html). Basic types @@ -931,25 +987,26 @@ Booleans -------- Nim's boolean type is called `bool` and consists of the two -pre-defined values `true` and `false`. Conditions in while, -if, elif, and when statements must be of type bool. +pre-defined values `true` and `false`. Conditions in `while`, +`if`, `elif`, and `when` statements must be of type bool. The operators `not, and, or, xor, <, <=, >, >=, !=, ==` are defined for the bool type. The `and` and `or` operators perform short-circuit evaluation. For example: -.. code-block:: nim - + ```nim while p != nil and p.name != "xyz": # p.name is not evaluated if p == nil p = p.next + ``` Characters ---------- -The `character type` is called `char`. Its size is always one byte, so + +The *character type* is called `char`. Its size is always one byte, so it cannot represent most UTF-8 characters, but it *can* represent one of the bytes -that makes up a multi-byte UTF-8 character. +that makes up a multibyte UTF-8 character. The reason for this is efficiency: for the overwhelming majority of use-cases, the resulting programs will still handle UTF-8 properly as UTF-8 was especially designed for this. @@ -963,6 +1020,7 @@ Converting from an integer to a `char` is done with the `chr` proc. Strings ------- + String variables are **mutable**, so appending to a string is possible, and quite efficient. Strings in Nim are both zero-terminated and have a length field. A string's length can be retrieved with the builtin `len` @@ -984,6 +1042,7 @@ A string variable is initialized with the empty string `""`. Integers -------- + Nim has these integer types built-in: `int int8 int16 int32 int64 uint uint8 uint16 uint32 uint64`. @@ -991,13 +1050,13 @@ The default integer type is `int`. Integer literals can have a *type suffix* to specify a non-default integer type: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" let x = 0 # x is of type `int` y = 0'i8 # y is of type `int8` - z = 0'i64 # z is of type `int64` + z = 0'i32 # z is of type `int32` u = 0'u # u is of type `uint` + ``` Most often integers are used for counting objects that reside in memory, so `int` has the same size as a pointer. @@ -1014,12 +1073,13 @@ errors. Lossless `Automatic type conversion`:idx: is performed in expressions where different kinds of integer types are used. However, if the type conversion -would cause loss of information, the `EOutOfRange`:idx: exception is raised (if the error +would cause loss of information, the `RangeDefect`:idx: is raised (if the error cannot be detected at compile time). Floats ------ + Nim has these floating-point types built-in: `float float32 float64`. The default float type is `float`. In the current implementation, @@ -1028,51 +1088,51 @@ The default float type is `float`. In the current implementation, Float literals can have a *type suffix* to specify a non-default float type: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" var x = 0.0 # x is of type `float` y = 0.0'f32 # y is of type `float32` z = 0.0'f64 # z is of type `float64` + ``` The common operators `+ - * / < <= == != > >=` are defined for floats and follow the IEEE-754 standard. Automatic type conversion in expressions with different kinds of floating-point types is performed: the smaller type is converted to the larger. Integer types are **not** converted to floating-point types automatically, nor vice -versa. Use the `toInt <system.html#toInt,float>`_ and -`toFloat <system.html#toFloat,int>`_ procs for these conversions. +versa. Use the [toInt](system.html#toInt,float) and +[toFloat](system.html#toFloat,int) procs for these conversions. Type Conversion --------------- + Conversion between numerical types is performed by using the type as a function: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" var x: int32 = 1.int32 # same as calling int32(1) y: int8 = int8('a') # 'a' == 97'i8 z: float = 2.5 # int(2.5) rounds down to 2 sum: int = int(x) + int(y) + int(z) # sum == 100 + ``` Internal type representation ============================ -As mentioned earlier, the built-in `$ <dollars.html>`_ (stringify) operator +As mentioned earlier, the built-in [$](dollars.html) (stringify) operator turns any basic type into a string, which you can then print to the console using the `echo` proc. However, advanced types, and your own custom types, won't work with the `$` operator until you define it for them. Sometimes you just want to debug the current value of a complex type without -having to write its `$` operator. You can use then the `repr -<system.html#repr,T>`_ proc which works with any type and even complex data +having to write its `$` operator. You can use then the [repr]( +system.html#repr,T) proc which works with any type and even complex data graphs with cycles. The following example shows that even for basic types there is a difference between the `$` and `repr` outputs: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" var myBool = true myCharacter = 'n' @@ -1088,7 +1148,8 @@ there is a difference between the `$` and `repr` outputs: echo myInteger, ":", repr(myInteger) # --> 42:42 echo myFloat, ":", repr(myFloat) - # --> 3.1400000000000001e+00:3.1400000000000001e+00 + # --> 3.14:3.14 + ``` Advanced types @@ -1096,11 +1157,11 @@ Advanced types In Nim new types can be defined within a `type` statement: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type biggestInt = int64 # biggest integer type that is available biggestFloat = float64 # biggest float type that is available + ``` Enumeration and object types may only be defined within a `type` statement. @@ -1108,20 +1169,20 @@ Enumeration and object types may only be defined within a Enumerations ------------ + A variable of an enumeration type can only be assigned one of the enumeration's specified values. These values are a set of ordered symbols. Each symbol is mapped to an integer value internally. The first symbol is represented at runtime by 0, the second by 1, and so on. For example: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Direction = enum north, east, south, west var x = south # `x` is of type `Direction`; its value is `south` - echo x # writes "south" to `stdout` + echo x # prints "south" + ``` All the comparison operators can be used with enumeration types. @@ -1138,14 +1199,15 @@ must be in ascending order. Ordinal types ------------- + Enumerations, integer types, `char` and `bool` (and subranges) are called ordinal types. Ordinal types have quite a few special operations: ------------------ -------------------------------------------------------- +================= ======================================================== Operation Comment ------------------ -------------------------------------------------------- +================= ======================================================== `ord(x)` returns the integer value that is used to represent `x`'s value `inc(x)` increments `x` by one @@ -1156,24 +1218,25 @@ Operation Comment `succ(x, n)` returns the `n`'th successor of `x` `pred(x)` returns the predecessor of `x` `pred(x, n)` returns the `n`'th predecessor of `x` ------------------ -------------------------------------------------------- +================= ======================================================== -The `inc <system.html#inc,T,int>`_, `dec <system.html#dec,T,int>`_, `succ -<system.html#succ,T,int>`_ and `pred <system.html#pred,T,int>`_ operations can -fail by raising an `EOutOfRange` or `EOverflow` exception. (If the code has been +The [inc](system.html#inc,T,int), [dec](system.html#dec,T,int), [succ]( +system.html#succ,T,int) and [pred](system.html#pred,T,int) operations can +fail by raising an `RangeDefect` or `OverflowDefect`. (If the code has been compiled with the proper runtime checks turned on.) Subranges --------- + A subrange type is a range of values from an integer or enumeration type (the base type). Example: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type MySubrange = range[0..5] + ``` `MySubrange` is a subrange of `int` which can only hold the values 0 @@ -1181,8 +1244,8 @@ to 5. Assigning any other value to a variable of type `MySubrange` is a compile-time or runtime error. Assignments from the base type to one of its subrange types (and vice versa) are allowed. -The `system` module defines the important `Natural <system.html#Natural>`_ -type as `range[0..high(int)]` (`high <system.html#high,typedesc[T]>`_ returns +The `system` module defines the important [Natural](system.html#Natural) +type as `range[0..high(int)]` ([high](system.html#high,typedesc[T]) returns the maximal value). Other programming languages may suggest the use of unsigned integers for natural numbers. This is often **unwise**: you don't want unsigned arithmetic (which wraps around) just because the numbers cannot be negative. @@ -1196,37 +1259,36 @@ Sets Arrays ------ + An array is a simple fixed-length container. Each element in an array has the same type. The array's index type can be any ordinal type. Arrays can be constructed using `[]`: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type IntArray = array[0..5, int] # an array that is indexed with 0..5 var x: IntArray x = [1, 2, 3, 4, 5, 6] - for i in low(x)..high(x): + for i in low(x) .. high(x): echo x[i] + ``` The notation `x[i]` is used to access the i-th element of `x`. Array access is always bounds checked (at compile-time or at runtime). These checks can be disabled via pragmas or invoking the compiler with the -`--bound_checks:off` command line switch. +``--bound_checks:off`` command line switch. Arrays are value types, like any other Nim type. The assignment operator copies the whole array contents. -The built-in `len <system.html#len,TOpenArray>`_ proc returns the array's -length. `low(a) <system.html#low,openArray[T]>`_ returns the lowest valid index -for the array `a` and `high(a) <system.html#high,openArray[T]>`_ the highest +The built-in [len](system.html#len,TOpenArray) proc returns the array's +length. [low(a)](system.html#low,openArray[T]) returns the lowest valid index +for the array `a` and [high(a)](system.html#high,openArray[T]) the highest valid index. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type Direction = enum north, east, south, west @@ -1238,10 +1300,11 @@ valid index. level[north] = on level[south] = slowBlink level[east] = fastBlink - echo repr(level) # --> [on, fastBlink, slowBlink, off] + echo level # --> [on, fastBlink, slowBlink, off] echo low(level) # --> north echo len(level) # --> 4 echo high(level) # --> west + ``` The syntax for nested arrays (multidimensional) in other languages is a matter of appending more brackets because usually each dimension is restricted to the @@ -1251,7 +1314,7 @@ the previous example where a level is defined as an array of enums indexed by yet another enum, we can add the following lines to add a light tower type subdivided into height levels accessed through their integer index: -.. code-block:: nim + ```nim type LightTower = array[1..10, LevelSetting] var @@ -1260,25 +1323,26 @@ subdivided into height levels accessed through their integer index: tower[1][east] = mediumBlink echo len(tower) # --> 10 echo len(tower[1]) # --> 4 - echo repr(tower) # --> [[slowBlink, mediumBlink, ...more output.. + echo tower # --> [[slowBlink, mediumBlink, ...more output.. # The following lines don't compile due to type mismatch errors #tower[north][east] = on #tower[0][1] = on + ``` Note how the built-in `len` proc returns only the array's first dimension length. Another way of defining the `LightTower` to better illustrate its nested nature would be to omit the previous definition of the `LevelSetting` type and instead write it embedded directly as the type of the first dimension: -.. code-block:: nim + ```nim type LightTower = array[1..10, array[north..west, BlinkLights]] + ``` It is quite common to have arrays start at zero, so there's a shortcut syntax to specify a range from zero to the specified index minus one: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type IntArray = array[0..5, int] # an array that is indexed with 0..5 QuickArray = array[6, int] # an array that is indexed with 0..5 @@ -1287,50 +1351,50 @@ to specify a range from zero to the specified index minus one: y: QuickArray x = [1, 2, 3, 4, 5, 6] y = x - for i in low(x)..high(x): + for i in low(x) .. high(x): echo x[i], y[i] + ``` Sequences --------- + Sequences are similar to arrays but of dynamic length which may change during runtime (like strings). Since sequences are resizable they are always allocated on the heap and garbage collected. -Sequences are always indexed with an `int` starting at position 0. The `len -<system.html#len,seq[T]>`_, `low <system.html#low,openArray[T]>`_ and `high -<system.html#high,openArray[T]>`_ operations are available for sequences too. +Sequences are always indexed with an `int` starting at position 0. The [len]( +system.html#len,seq[T]), [low](system.html#low,openArray[T]) and [high]( +system.html#high,openArray[T]) operations are available for sequences too. The notation `x[i]` can be used to access the i-th element of `x`. Sequences can be constructed by the array constructor `[]` in conjunction with the array to sequence operator `@`. Another way to allocate space for -a sequence is to call the built-in `newSeq <system.html#newSeq>`_ procedure. +a sequence is to call the built-in [newSeq](system.html#newSeq) procedure. A sequence may be passed to an openarray parameter. Example: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" var x: seq[int] # a reference to a sequence of integers x = @[1, 2, 3, 4, 5, 6] # the @ turns the array into a sequence allocated on the heap + ``` Sequence variables are initialized with `@[]`. The `for` statement can be used with one or two variables when used with a sequence. When you use the one variable form, the variable will hold the value provided by the sequence. The `for` statement is looping over the results -from the `items() <iterators.html#items.i,seq[T]>`_ iterator from the `system -<system.html>`_ module. But if you use the two-variable form, the first +from the [items()](iterators.html#items.i,seq[T]) iterator from the [system]( +system.html) module. But if you use the two-variable form, the first variable will hold the index position and the second variable will hold the value. Here the `for` statement is looping over the results from the -`pairs() <iterators.html#pairs.i,seq[T]>`_ iterator from the `system -<system.html>`_ module. Examples: +[pairs()](iterators.html#pairs.i,seq[T]) iterator from the [system]( +system.html) module. Examples: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" for value in @[3, 4, 5]: echo value # --> 3 @@ -1342,22 +1406,23 @@ value. Here the `for` statement is looping over the results from the # --> index: 0, value:3 # --> index: 1, value:4 # --> index: 2, value:5 + ``` Open arrays ----------- + **Note**: Openarrays can only be used for parameters. Often fixed-size arrays turn out to be too inflexible; procedures should be able to deal with arrays of different sizes. The `openarray`:idx: type allows this. Openarrays are always indexed with an `int` starting at position 0. -The `len <system.html#len,TOpenArray>`_, `low <system.html#low,openArray[T]>`_ -and `high <system.html#high,openArray[T]>`_ operations are available for open +The [len](system.html#len,TOpenArray), [low](system.html#low,openArray[T]) +and [high](system.html#high,openArray[T]) operations are available for open arrays too. Any array with a compatible base type can be passed to an openarray parameter, the index type does not matter. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" var fruits: seq[string] # reference to a sequence of strings that is initialized with '@[]' capitals: array[3, string] # array of strings with a fixed size @@ -1371,6 +1436,7 @@ openarray parameter, the index type does not matter. assert openArraySize(fruits) == 2 # procedure accepts a sequence as parameter assert openArraySize(capitals) == 3 # but also an array type + ``` The openarray type cannot be nested: multidimensional openarrays are not supported because this is seldom needed and cannot be done efficiently. @@ -1384,8 +1450,7 @@ also a means to implement passing a variable number of arguments to a procedure. The compiler converts the list of arguments to an array automatically: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc myWriteln(f: File, a: varargs[string]) = for s in items(a): write(f, s) @@ -1394,13 +1459,13 @@ to an array automatically: myWriteln(stdout, "abc", "def", "xyz") # is transformed by the compiler to: myWriteln(stdout, ["abc", "def", "xyz"]) + ``` This transformation is only done if the varargs parameter is the last parameter in the procedure header. It is also possible to perform type conversions in this context: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc myWriteln(f: File, a: varargs[string, `$`]) = for s in items(a): write(f, s) @@ -1409,9 +1474,10 @@ type conversions in this context: myWriteln(stdout, 123, "abc", 4.0) # is transformed by the compiler to: myWriteln(stdout, [$123, $"abc", $4.0]) + ``` -In this example `$ <dollars.html>`_ is applied to any argument that is passed -to the parameter `a`. Note that `$ <dollars.html>`_ applied to strings is a +In this example [$](dollars.html) is applied to any argument that is passed +to the parameter `a`. Note that [$](dollars.html) applied to strings is a nop. @@ -1423,9 +1489,7 @@ context. A slice is just an object of type Slice which contains two bounds, `a` and `b`. By itself a slice is not very useful, but other collection types define operators which accept Slice objects to define ranges. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" var a = "Nim is a programming language" b = "Slices are useless." @@ -1433,29 +1497,30 @@ define operators which accept Slice objects to define ranges. echo a[7 .. 12] # --> 'a prog' b[11 .. ^2] = "useful" echo b # --> 'Slices are useful.' + ``` In the previous example slices are used to modify a part of a string. The slice's bounds can hold any value supported by their type, but it is the proc using the slice object which defines what values are accepted. -To understand some of the different ways of specifying the indices of +To understand the different ways of specifying the indices of strings, arrays, sequences, etc., it must be remembered that Nim uses zero-based indices. So the string `b` is of length 19, and two different ways of specifying the indices are -.. code-block:: nim - + ```nim "Slices are useless." | | | 0 11 17 using indices ^19 ^8 ^2 using ^ syntax + ``` where `b[0 .. ^1]` is equivalent to `b[0 .. b.len-1]` and `b[0 ..< b.len]`, and it -can be seen that the `^1` provides a short-hand way of specifying the `b.len-1`. See -the `backwards index operator <system.html#^.t%2Cint>`_. +can be seen that the `^1` provides a shorthand way of specifying the `b.len-1`. See +the [backwards index operator](system.html#^.t%2Cint). In the above example, because the string ends in a period, to get the portion of the string that is "useless" and replace it with "useful". @@ -1466,7 +1531,7 @@ string that is "useless" and replace it with "useful". Note 1: alternate ways of writing this are `b[^8 .. ^2] = "useful"` or as `b[11 .. b.len-2] = "useful"` or as `b[11 ..< b.len-1] = "useful"`. -Note 2: As the `^` template returns a `distinct int <manual.html#types-distinct-type>`_ +Note 2: As the `^` template returns a [distinct int](manual.html#types-distinct-type) of type `BackwardsIndex`, we can have a `lastIndex` constant defined as `const lastIndex = ^1`, and later used as `b[0 .. lastIndex]`. @@ -1482,7 +1547,7 @@ Each object type `Foo` has a constructor `Foo(field: value, ...)` where all of its fields can be initialized. Unspecified fields will get their default value. -.. code-block:: nim + ```nim type Person = object name: string @@ -1509,18 +1574,18 @@ get their default value. # unspecified members will be initialized with their default # values. In this case it is the empty string. doAssert person4.name == "" + ``` Object fields that should be visible from outside the defining module have to be marked with `*`. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Person* = object # the type is visible from other modules name*: string # the field of this type is visible from other modules age*: int + ``` Tuples ------ @@ -1542,8 +1607,7 @@ tuple's field. Another notation that is not available for objects is `t[i]` to access the `i`'th field. Here `i` must be a constant integer. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type # type representing a person: # A person consists of a name and an age. @@ -1592,43 +1656,60 @@ integer. #person = building # --> Error: type mismatch: got (tuple[street: string, number: int]) # but expected 'Person' + ``` Even though you don't need to declare a type for a tuple to use it, tuples created with different field names will be considered different objects despite having the same field types. -Tuples can be *unpacked* during variable assignment (and only then!). This can +Tuples can be *unpacked* during variable assignment. This can be handy to assign directly the fields of the tuples to individually named -variables. An example of this is the `splitFile <os.html#splitFile,string>`_ -proc from the `os module <os.html>`_ which returns the directory, name, and +variables. An example of this is the [splitFile](os.html#splitFile,string) +proc from the [os module](os.html) which returns the directory, name, and extension of a path at the same time. For tuple unpacking to work you must use parentheses around the values you want to assign the unpacking to, otherwise, you will be assigning the same value to all the individual variables! For example: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/os let path = "usr/local/nimc.html" (dir, name, ext) = splitFile(path) baddir, badname, badext = splitFile(path) - echo dir # outputs `usr/local` - echo name # outputs `nimc` - echo ext # outputs `.html` + echo dir # outputs "usr/local" + echo name # outputs "nimc" + echo ext # outputs ".html" # All the following output the same line: - # `(dir: usr/local, name: nimc, ext: .html)` + # "(dir: usr/local, name: nimc, ext: .html)" echo baddir echo badname echo badext + ``` + +Tuple unpacking is also supported in for-loops: + + ```nim test = "nim c $1" + let a = [(10, 'a'), (20, 'b'), (30, 'c')] + + for (x, c) in a: + echo x + # This will output: 10; 20; 30 + + # Accessing the index is also possible: + for i, (x, c) in a: + echo i, c + # This will output: 0a; 1b; 2c + ``` -Fields of tuples are always public, they don't need to be explicity +Fields of tuples are always public, they don't need to be explicitly marked to be exported, unlike for example fields in an object type. + Reference and pointer types --------------------------- + References (similar to pointers in other programming languages) are a way to introduce many-to-one relationships. This means different references can point to and modify the same location in memory. @@ -1636,9 +1717,9 @@ point to and modify the same location in memory. Nim distinguishes between `traced`:idx: and `untraced`:idx: references. Untraced references are also called *pointers*. Traced references point to objects in a garbage-collected heap, untraced references point to -manually allocated objects or objects elsewhere in memory. Thus +manually allocated objects or objects elsewhere in memory. Thus, untraced references are *unsafe*. However, for certain low-level operations -(e.g., accessing the hardware), untraced references are necessary. +(e.g. accessing the hardware), untraced references are necessary. Traced references are declared with the **ref** keyword; untraced references are declared with the **ptr** keyword. @@ -1648,22 +1729,26 @@ meaning to retrieve the item the reference points to. The `.` (access a tuple/object field operator) and `[]` (array/string/sequence index operator) operators perform implicit dereferencing operations for reference types: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Node = ref object le, ri: Node data: int - var - n: Node - new(n) - n.data = 9 + + var n = Node(data: 9) + echo n.data # no need to write n[].data; in fact n[].data is highly discouraged! + ``` + +To allocate a new traced object, the built-in procedure `new` can be used: + + ```nim + var n: Node + new(n) + ``` -To allocate a new traced object, the built-in procedure `new` must be used. To deal with untraced memory, the procedures `alloc`, `dealloc` and -`realloc` can be used. The `system <system.html>`_ +`realloc` can be used. The [system](system.html) module's documentation contains further details. If a reference points to *nothing*, it has the value `nil`. @@ -1671,6 +1756,7 @@ If a reference points to *nothing*, it has the value `nil`. Procedural type --------------- + A procedural type is a (somewhat abstract) pointer to a procedure. `nil` is an allowed value for a variable of a procedural type. Nim uses procedural types to achieve `functional`:idx: programming @@ -1678,41 +1764,45 @@ techniques. Example: -.. code-block:: nim - :test: "nim c $1" - proc echoItem(x: int) = echo x + ```nim test = "nim c $1" + proc greet(name: string): string = + "Hello, " & name & "!" + + proc bye(name: string): string = + "Goodbye, " & name & "." - proc forEach(action: proc (x: int)) = - const - data = [2, 3, 5, 7, 11] - for d in items(data): - action(d) + proc communicate(greeting: proc (x: string): string, name: string) = + echo greeting(name) - forEach(echoItem) + communicate(greet, "John") + communicate(bye, "Mary") + ``` A subtle issue with procedural types is that the calling convention of the procedure influences the type compatibility: procedural types are only compatible if they have the same calling convention. The different calling conventions are -listed in the `manual <manual.html#types-procedural-type>`_. +listed in the [manual](manual.html#types-procedural-type). Distinct type ------------- + A Distinct type allows for the creation of a new type that "does not imply a subtype relationship between it and its base type". You must **explicitly** define all behavior for the distinct type. To help with this, both the distinct type and its base type can cast from one type to the other. -Examples are provided in the `manual <manual.html#types-distinct-type>`_. +Examples are provided in the [manual](manual.html#types-distinct-type). Modules ======= -Nim supports splitting a program into pieces with a module concept. + +Nim supports splitting a program into pieces with a *module* concept. Each module is in its own file. Modules enable `information hiding`:idx: and `separate compilation`:idx:. A module may gain access to the symbols of another module by using the `import`:idx: statement. Only top-level symbols that are marked with an asterisk (`*`) are exported: -.. code-block:: nim + ```nim # Module A var x*, y: int @@ -1721,11 +1811,12 @@ with an asterisk (`*`) are exported: # allocate a new sequence: newSeq(result, len(a)) # multiply two int sequences: - for i in 0..len(a)-1: result[i] = a[i] * b[i] + for i in 0 ..< len(a): result[i] = a[i] * b[i] when isMainModule: # test the new `*` operator for sequences: assert(@[1, 2, 3] * @[1, 2, 3] == @[1, 4, 9]) + ``` The above module exports `x` and `*`, but not `y`. @@ -1741,15 +1832,17 @@ a symbol is ambiguous, it *must* be qualified. A symbol is ambiguous if it is defined in two (or more) different modules and both modules are imported by a third one: -.. code-block:: nim + ```nim # Module A var x*: string + ``` -.. code-block:: nim + ```nim # Module B var x*: int + ``` -.. code-block:: nim + ```nim # Module C import A, B write(stdout, x) # error: x is ambiguous @@ -1757,20 +1850,23 @@ imported by a third one: var x = 4 write(stdout, x) # not ambiguous: uses the module C's x + ``` But this rule does not apply to procedures or iterators. Here the overloading rules apply: -.. code-block:: nim + ```nim # Module A proc x*(a: int): string = $a + ``` -.. code-block:: nim + ```nim # Module B proc x*(a: string): string = $a + ``` -.. code-block:: nim + ```nim # Module C import A, B write(stdout, x(3)) # no error: A.x is called @@ -1778,6 +1874,7 @@ rules apply: proc x*(a: int): string = discard write(stdout, x(3)) # ambiguous: which `x` is to call? + ``` Excluding symbols @@ -1787,8 +1884,9 @@ The normal `import` statement will bring in all exported symbols. These can be limited by naming symbols that should be excluded using the `except` qualifier. -.. code-block:: nim + ```nim import mymodule except y + ``` From statement @@ -1798,42 +1896,48 @@ We have already seen the simple `import` statement that just imports all exported symbols. An alternative that only imports listed symbols is the `from import` statement: -.. code-block:: nim + ```nim from mymodule import x, y, z + ``` The `from` statement can also force namespace qualification on symbols, thereby making symbols available, but needing to be qualified in order to be used. -.. code-block:: nim + ```nim from mymodule import x, y, z x() # use x without any qualification + ``` -.. code-block:: nim + ```nim from mymodule import nil mymodule.x() # must qualify x with the module name as prefix x() # using x here without qualification is a compile error + ``` Since module names are generally long to be descriptive, you can also define a shorter alias to use when qualifying symbols. -.. code-block:: nim + ```nim from mymodule as m import nil m.x() # m is aliasing mymodule + ``` Include statement ----------------- + The `include` statement does something fundamentally different than importing a module: it merely includes the contents of a file. The `include` statement is useful to split up a large module into several files: -.. code-block:: nim + ```nim include fileA, fileB, fileC + ``` @@ -1841,7 +1945,7 @@ Part 2 ====== So, now that we are done with the basics, let's see what Nim offers apart -from a nice syntax for procedural programming: `Part II <tut2.html>`_ +from a nice syntax for procedural programming: [Part II](tut2.html) .. _strutils: strutils.html diff --git a/doc/tut2.rst b/doc/tut2.md index 3aef6bb0f..1b59288d5 100644 --- a/doc/tut2.rst +++ b/doc/tut2.md @@ -1,5 +1,3 @@ -.. default-role:: code - ====================== Nim Tutorial (Part II) ====================== @@ -7,17 +5,19 @@ Nim Tutorial (Part II) :Author: Andreas Rumpf :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: Introduction ============ - "Repetition renders the ridiculous reasonable." -- Norman Wildberger +> "Repetition renders the ridiculous reasonable." -- Norman Wildberger This document is a tutorial for the advanced constructs of the *Nim* programming language. **Note that this document is somewhat obsolete as the** -`manual <manual.html>`_ **contains many more examples of the advanced language +[manual](manual.html) **contains many more examples of the advanced language features.** @@ -27,8 +27,8 @@ Pragmas Pragmas are Nim's method to give the compiler additional information/ commands without introducing a massive number of new keywords. Pragmas are enclosed in the special `{.` and `.}` curly dot brackets. This tutorial -does not cover pragmas. See the `manual <manual.html#pragmas>`_ or `user guide -<nimc.html#additional-features>`_ for a description of the available +does not cover pragmas. See the [manual](manual.html#pragmas) or [user guide]( +nimc.html#additional-features) for a description of the available pragmas. @@ -53,8 +53,7 @@ types with inheritance are also marked as `ref` types even though this isn't strictly enforced. To check at runtime if an object is of a certain type, the `of` operator can be used. -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type Person = ref object of RootObj name*: string # the * means that `name` is accessible from other modules @@ -70,6 +69,7 @@ type, the `of` operator can be used. # object construction: student = Student(name: "Anton", age: 5, id: 2) echo student[] + ``` Inheritance is done with the `object of` syntax. Multiple inheritance is currently not supported. If an object type has no suitable ancestor, `RootObj` @@ -79,7 +79,7 @@ to introduce new object roots apart from `system.RootObj`. (This is used in the GTK wrapper for instance.) Ref objects should be used whenever inheritance is used. It isn't strictly -necessary, but with non-ref objects assignments such as `let person: Person = +necessary, but with non-ref objects, assignments such as `let person: Person = Student(id: 123)` will truncate subclass fields. **Note**: Composition (*has-a* relation) is often preferable to inheritance @@ -97,8 +97,7 @@ would require arbitrary symbol lookahead which slows down compilation.) Example: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type Node = ref object # a reference to an object with the following field: le, ri: Node # left and right subtrees @@ -108,6 +107,7 @@ Example: name: string # the symbol's name line: int # the line the symbol was declared in code: Node # the symbol's abstract syntax tree + ``` Type conversions @@ -124,9 +124,10 @@ raised. The syntax for type conversions is `destination_type(expression_to_convert)` (like an ordinary call): -.. code-block:: nim + ```nim proc getID(x: Person): int = Student(x).id + ``` The `InvalidObjectConversionDefect` exception is raised if `x` is not a `Student`. @@ -139,9 +140,7 @@ variant types are needed. An example: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" # This is an example how an abstract syntax tree could be modelled in Nim type NodeKind = enum # the different node types @@ -165,8 +164,9 @@ An example: # the following statement raises an `FieldDefect` exception, because # n.kind's value does not fit: n.strVal = "" + ``` -As can been seen from the example, an advantage to an object hierarchy is that +As can be seen from the example, an advantage to an object hierarchy is that no conversion between different object types is needed. Yet, access to invalid object fields raises an exception. @@ -175,34 +175,35 @@ Method call syntax ------------------ There is a syntactic sugar for calling routines: -The syntax `obj.method(args)` can be used instead of `method(obj, args)`. +The syntax `obj.methodName(args)` can be used +instead of `methodName(obj, args)`. If there are no remaining arguments, the parentheses can be omitted: `obj.len` (instead of `len(obj)`). This method call syntax is not restricted to objects, it can be used for any type: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" import std/strutils echo "abc".len # is the same as echo len("abc") echo "abc".toUpperAscii() echo({'a', 'b', 'c'}.card) stdout.writeLine("Hallo") # the same as writeLine(stdout, "Hallo") + ``` (Another way to look at the method call syntax is that it provides the missing postfix notation.) So "pure object oriented" code is easy to write: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" import std/[strutils, sequtils] stdout.writeLine("Give a list of numbers (separated by spaces): ") stdout.write(stdin.readLine.splitWhitespace.map(parseInt).max.`$`) stdout.writeLine(" is the maximum!") + ``` Properties @@ -212,9 +213,7 @@ Ordinary get-procedures that are called with the *method call syntax* achieve the same. But setting a value is different; for this a special setter syntax is needed: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" type Socket* = ref object of RootObj h: int # cannot be accessed from the outside of the module due to missing star @@ -230,6 +229,7 @@ is needed: var s: Socket new s s.host = 34 # same as `host=`(s, 34) + ``` (The example also shows `inline` procedures.) @@ -237,8 +237,7 @@ is needed: The `[]` array access operator can be overloaded to provide `array properties`:idx:\ : -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type Vector* = object x, y, z: float @@ -258,6 +257,7 @@ The `[]` array access operator can be overloaded to provide of 1: result = v.y of 2: result = v.z else: assert(false) + ``` The example is silly, since a vector is better modelled by a tuple which already provides `v[]` access. @@ -269,8 +269,7 @@ Dynamic dispatch Procedures always use static dispatch. For dynamic dispatch replace the `proc` keyword by `method`: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type Expression = ref object of RootObj ## abstract base class for an expression Literal = ref object of Expression @@ -290,20 +289,19 @@ Procedures always use static dispatch. For dynamic dispatch replace the proc newPlus(a, b: Expression): PlusExpr = PlusExpr(a: a, b: b) echo eval(newPlus(newPlus(newLit(1), newLit(2)), newLit(4))) + ``` Note that in the example the constructors `newLit` and `newPlus` are procs because it makes more sense for them to use static binding, but `eval` is a method because it requires dynamic binding. **Note:** Starting from Nim 0.20, to use multi-methods one must explicitly pass -`--multimethods:on` when compiling. +``--multimethods:on`` when compiling. In a multi-method all parameters that have an object type are used for the dispatching: -.. code-block:: nim - :test: "nim c --multiMethods:on $1" - + ```nim test = "nim c --multiMethods:on $1" type Thing = ref object of RootObj Unit = ref object of Thing @@ -322,6 +320,7 @@ dispatching: new a new b collide(a, b) # output: 2 + ``` As the example demonstrates, invocation of a multi-method cannot be ambiguous: @@ -338,7 +337,7 @@ Exceptions ========== In Nim exceptions are objects. By convention, exception types are -suffixed with 'Error'. The `system <system.html>`_ module defines an +suffixed with 'Error'. The [system](system.html) module defines an exception hierarchy that you might want to stick to. Exceptions derive from `system.Exception`, which provides the common interface. @@ -354,20 +353,21 @@ Raise statement --------------- Raising an exception is done with the `raise` statement: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" var e: ref OSError new(e) e.msg = "the request to the OS failed" raise e + ``` If the `raise` keyword is not followed by an expression, the last exception is *re-raised*. For the purpose of avoiding repeating this common code pattern, the template `newException` in the `system` module can be used: -.. code-block:: nim + ```nim raise newException(OSError, "the request to the OS failed") + ``` Try statement @@ -375,8 +375,7 @@ Try statement The `try` statement handles exceptions: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" from std/strutils import parseInt # read the first two lines of a text file that should contain numbers @@ -394,12 +393,13 @@ The `try` statement handles exceptions: echo "could not convert string to integer" except IOError: echo "IO error!" - except: + except CatchableError: echo "Unknown exception!" # reraise the unknown exception: raise finally: close(f) + ``` The statements after the `try` are executed unless an exception is raised. Then the appropriate `except` part is executed. @@ -417,19 +417,20 @@ the rest of the procedure - that is not within a `finally` clause - is not executed (if an exception occurs). If you need to *access* the actual exception object or message inside an -`except` branch you can use the `getCurrentException() -<system.html#getCurrentException>`_ and `getCurrentExceptionMsg() -<system.html#getCurrentExceptionMsg>`_ procs from the `system <system.html>`_ +`except` branch you can use the [getCurrentException()]( +system.html#getCurrentException) and [getCurrentExceptionMsg()]( +system.html#getCurrentExceptionMsg) procs from the [system](system.html) module. Example: -.. code-block:: nim + ```nim try: doSomethingHere() - except: + except CatchableError: let e = getCurrentException() msg = getCurrentExceptionMsg() echo "Got exception ", repr(e), " with message ", msg + ``` Annotating procs with raised exceptions @@ -442,12 +443,13 @@ instance, if you specify that a proc raises `IOError`, and at some point it (or one of the procs it calls) starts raising a new exception the compiler will prevent that proc from compiling. Usage example: -.. code-block:: nim + ```nim proc complexProc() {.raises: [IOError, ArithmeticDefect].} = ... proc simpleProc() {.raises: [].} = ... + ``` Once you have code like this in place, if the list of raised exception changes the compiler will stop with an error specifying the line of the proc which @@ -459,10 +461,10 @@ If you want to add the `{.raises.}` pragma to existing code, the compiler can also help you. You can add the `{.effects.}` pragma statement to your proc and the compiler will output all inferred effects up to that point (exception tracking is part of Nim's effect system). Another more roundabout way to -find out the list of exceptions raised by a proc is to use the Nim `doc` +find out the list of exceptions raised by a proc is to use the Nim ``doc`` command which generates documentation for a whole module and decorates all procs with the list of raised exceptions. You can read more about Nim's -`effect system and related pragmas in the manual <manual.html#effect-system>`_. +[effect system and related pragmas in the manual](manual.html#effect-system). Generics @@ -473,8 +475,7 @@ with `type parameters`:idx:. Generic parameters are written within square brackets, for example `Foo[T]`. They are most useful for efficient type safe containers: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" type BinaryTree*[T] = ref object # BinaryTree is a generic type with # generic param `T` @@ -529,6 +530,7 @@ containers: add(root, "world") # instantiates the second `add` proc for str in preorder(root): stdout.writeLine(str) + ``` The example shows a generic binary tree. Depending on context, the brackets are used either to introduce type parameters or to instantiate a generic proc, @@ -538,8 +540,7 @@ is not hidden and is used in the `preorder` iterator. There is a special `[:T]` syntax when using generics with the method call syntax: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" proc foo[T](i: T) = discard @@ -548,6 +549,7 @@ There is a special `[:T]` syntax when using generics with the method call syntax # i.foo[int]() # Error: expression 'foo(i)' has no type (or is ambiguous) i.foo[:int]() # Success + ``` Templates @@ -562,12 +564,13 @@ To *invoke* a template, call it like a procedure. Example: -.. code-block:: nim + ```nim template `!=` (a, b: untyped): untyped = # this definition exists in the System module not (a == b) assert(5 != 6) # the compiler rewrites that to: assert(not (5 == 6)) + ``` The `!=`, `>`, `>=`, `in`, `notin`, `isnot` operators are in fact templates: this has the benefit that if you overload the `==` operator, @@ -581,8 +584,7 @@ for IEEE floating point numbers - NaN breaks basic boolean logic.) Templates are especially useful for lazy evaluation purposes. Consider a simple proc for logging: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" const debug = true @@ -592,6 +594,7 @@ simple proc for logging: var x = 4 log("x has the value: " & $x) + ``` This code has a shortcoming: if `debug` is set to false someday, the quite expensive `$` and `&` operations are still performed! (The argument @@ -599,8 +602,7 @@ evaluation for procedures is *eager*). Turning the `log` proc into a template solves this problem: -.. code-block:: nim - :test: "nim c $1" + ```nim test = "nim c $1" const debug = true @@ -610,6 +612,7 @@ Turning the `log` proc into a template solves this problem: var x = 4 log("x has the value: " & $x) + ``` The parameters' types can be ordinary types or the meta types `untyped`, `typed`, or `type`. `type` suggests that only a type symbol may be given @@ -621,9 +624,7 @@ If the template has no explicit return type, To pass a block of statements to a template, use `untyped` for the last parameter: -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" template withFile(f: untyped, filename: string, mode: FileMode, body: untyped) = let fn = filename @@ -639,6 +640,7 @@ To pass a block of statements to a template, use `untyped` for the last paramete withFile(txt, "ttempl3.txt", fmWrite): txt.writeLine("line 1") txt.writeLine("line 2") + ``` In the example the two `writeLine` statements are bound to the `body` parameter. The `withFile` template contains boilerplate code and helps to @@ -649,8 +651,7 @@ once. Example: Lifting Procs ---------------------- -.. code-block:: nim - :test: "nim c $1" + `````nim test = "nim c $1" import std/math template liftScalarProc(fname) = @@ -659,9 +660,10 @@ Example: Lifting Procs ## to provide templated procs that can handle a single ## parameter of seq[T] or nested seq[seq[]] or the same type ## - ## .. code-block:: Nim - ## liftScalarProc(abs) - ## # now abs(@[@[1,-2], @[-2,-3]]) == @[@[1,2], @[2,3]] + ## ```Nim + ## liftScalarProc(abs) + ## # now abs(@[@[1,-2], @[-2,-3]]) == @[@[1,2], @[2,3]] + ## ``` proc fname[T](x: openarray[T]): auto = var temp: T type outType = typeof(fname(temp)) @@ -671,6 +673,7 @@ Example: Lifting Procs liftScalarProc(sqrt) # make sqrt() work for sequences echo sqrt(@[4.0, 16.0, 25.0, 36.0]) # => @[2.0, 4.0, 5.0, 6.0] + ````` Compilation to JavaScript ========================= @@ -691,4 +694,4 @@ JavaScript-compatible code you should remember the following: Part 3 ====== -The next part is entirely about metaprogramming via macros: `Part III <tut3.html>`_ +The next part is entirely about metaprogramming via macros: [Part III](tut3.html). diff --git a/doc/tut3.rst b/doc/tut3.md index 358a9b45e..3a55d4790 100644 --- a/doc/tut3.rst +++ b/doc/tut3.md @@ -1,5 +1,3 @@ -.. default-role:: code - ======================= Nim Tutorial (Part III) ======================= @@ -7,13 +5,15 @@ Nim Tutorial (Part III) :Author: Arne Döring :Version: |nimversion| +.. default-role:: code +.. include:: rstcommon.rst .. contents:: Introduction ============ - "With Great Power Comes Great Responsibility." -- Spider Man's Uncle +> "With Great Power Comes Great Responsibility." -- Spider Man's Uncle This document is a tutorial about Nim's macro system. A macro is a function that is executed at compile-time and transforms @@ -91,14 +91,14 @@ in the expression `foo(x)`, `x` needs to be an integer constant, but in the macro body `arg` is just like a normal parameter of type `int`. -.. code-block:: nim - + ```nim import std/macros macro myMacro(arg: static[int]): untyped = echo arg # just an int (7), not `NimNode` myMacro(1 + 2 * 3) + ``` Code Blocks as Arguments @@ -108,12 +108,12 @@ It is possible to pass the last argument of a call expression in a separate code block with indentation. For example, the following code example is a valid (but not a recommended) way to call `echo`: -.. code-block:: nim - + ```nim echo "Hello ": let a = "Wor" let b = "ld!" a & b + ``` For macros this way of calling is very useful; syntax trees of arbitrary complexity can be passed to macros with this notation. @@ -125,7 +125,7 @@ The Syntax Tree In order to build a Nim syntax tree one needs to know how Nim source code is represented as a syntax tree, and how such a tree needs to look like so that the Nim compiler will understand it. The nodes of the -Nim syntax tree are documented in the `macros <macros.html>`_ module. +Nim syntax tree are documented in the [macros](macros.html) module. But a more interactive way to explore the Nim syntax tree is with `macros.treeRepr`, it converts a syntax tree into a multi-line string for printing on the console. It can be used @@ -134,8 +134,7 @@ and for debug printing of generated syntax tree. `dumpTree` is a predefined macro that just prints its argument in a tree representation, but does nothing else. Here is an example of such a tree representation: -.. code-block:: nim - + ```nim dumpTree: var mt: MyType = MyType(a:123.456, b:"abcdef") @@ -153,6 +152,7 @@ but does nothing else. Here is an example of such a tree representation: # ExprColonExpr # Ident "b" # StrLit "abcdef" + ``` Custom Semantic Checking @@ -166,10 +166,10 @@ macro evaluation should be caught and create a nice error message. the checks need to be more complex, arbitrary error messages can be created with the `macros.error` proc. -.. code-block:: nim - + ```nim macro myAssert(arg: untyped): untyped = arg.expectKind nnkInfix + ``` Generating Code @@ -187,22 +187,36 @@ tree with calls to `newTree` and `newLit` the macro Backticks are used to insert code from `NimNode` symbols into the generated expression. -.. code-block:: nim - macro a(i) = quote do: let `i` = 0 - a b + ```nim test = "nim c $1" + import std/macros + macro a(i) = quote do: + let `i` = 0 + + a b + doAssert b == 0 + ``` A custom prefix operator can be defined whenever backticks are needed. -.. code-block:: nim - macro a(i) = quote("@") do: assert @i == 0 - let b = 0 - a b + ```nim test = "nim c $1" + import std/macros + macro a(i) = quote("@") do: + assert @i == 0 + + let b = 0 + a b + ``` The injected symbol needs accent quoted when it resolves to a symbol. -.. code-block:: nim - macro a(i) = quote("@") do: let `@i` == 0 - a b + ```nim test = "nim c $1" + import std/macros + macro a(i) = quote("@") do: + let `@i` = 0 + + a b + doAssert b == 0 + ``` Make sure to inject only symbols of type `NimNode` into the generated syntax tree. You can use `newLit` to convert arbitrary values into @@ -210,9 +224,7 @@ expressions trees of type `NimNode` so that it is safe to inject them into the tree. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/macros type @@ -232,26 +244,26 @@ them into the tree. echo `mtLit` myMacro("Hallo") + ``` The call to `myMacro` will generate the following code: -.. code-block:: nim + ```nim echo "Hallo" echo MyType(a: 123.456'f64, b: "abcdef") + ``` Building Your First Macro ------------------------- To give a starting point to writing macros we will show now how to -implement the `myDebug` macro mentioned earlier. The first thing to +implement the `myAssert` macro mentioned earlier. The first thing to do is to build a simple example of the macro usage, and then just print the argument. This way it is possible to get an idea of what a correct argument should look like. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/macros macro myAssert(arg: untyped): untyped = @@ -261,13 +273,14 @@ correct argument should look like. let b = 2 myAssert(a != b) + ``` -.. code-block:: - + ``` Infix Ident "!=" Ident "a" Ident "b" + ``` From the output, it is possible to see that the argument is an infix @@ -275,9 +288,7 @@ operator (node kind is "Infix"), as well as that the two operands are at index 1 and 2. With this information, the actual macro can be written. -.. code-block:: nim - :test: "nim c $1" - + ```nim test = "nim c $1" import std/macros macro myAssert(arg: untyped): untyped = @@ -298,6 +309,7 @@ written. myAssert(a != b) myAssert(a == b) + ``` This is the code that will be generated. To debug what the macro @@ -305,9 +317,40 @@ actually generated, the statement `echo result.repr` can be used, in the last line of the macro. It is also the statement that has been used to get this output. -.. code-block:: nim + ```nim if not (a != b): raise newException(AssertionDefect, $a & " != " & $b) + ``` + + +Going further +------------- + +It is possible to create more complex macros by combining different +`NimNode` symbols with `quote do:` expressions. For example, you may +use `newStmtList` to build your macro iteratively, and `ident` in cases +in which you wish to create an identifier from a string, as shown below. + + ```nim + import std/macros + + macro createProcedures() = + result = newStmtList() + + for i in 0..<10: + let name = ident("myProc" & $i) + let content = newLit("I am procedure number #" & $i) + + result.add quote do: + proc `name`() = + echo `content` + + createProcedures() + myProc7() + ``` + +The call to `myProc7` will echo `I am procedure number #7`. + With Power Comes Responsibility ------------------------------- @@ -353,7 +396,7 @@ recommended way. But still `strformat` is a good example for a practical use case for a macro that is slightly more complex than the `assert` macro. -`Strformat <https://github.com/nim-lang/Nim/blob/5845716df8c96157a047c2bd6bcdd795a7a2b9b1/lib/pure/strformat.nim#L280>`_ +[Strformat](https://github.com/nim-lang/Nim/blob/devel/lib/pure/strformat.nim) Ast Pattern Matching -------------------- @@ -362,7 +405,7 @@ Ast Pattern Matching is a macro library to aid in writing complex macros. This can be seen as a good example of how to repurpose the Nim syntax tree with new semantics. -`Ast Pattern Matching <https://github.com/krux02/ast-pattern-matching>`_ +[Ast Pattern Matching](https://github.com/nim-lang/ast-pattern-matching) OpenGL Sandbox -------------- @@ -371,4 +414,4 @@ This project has a working Nim to GLSL compiler written entirely in macros. It scans recursively through all used function symbols to compile them so that cross library functions can be executed on the GPU. -`OpenGL Sandbox <https://github.com/krux02/opengl-sandbox>`_ +[OpenGL Sandbox](https://github.com/krux02/opengl-sandbox) |