diff options
Diffstat (limited to 'doc')
-rw-r--r-- | doc/advopt.txt | 15 | ||||
-rw-r--r-- | doc/astspec.txt | 48 | ||||
-rw-r--r-- | doc/backends.md | 10 | ||||
-rw-r--r-- | doc/contributing.md | 14 | ||||
-rw-r--r-- | doc/destructors.md | 152 | ||||
-rw-r--r-- | doc/docgen.md | 354 | ||||
-rw-r--r-- | doc/docgen_sample.nim | 2 | ||||
-rw-r--r-- | doc/estp.md | 2 | ||||
-rw-r--r-- | doc/filelist.txt | 6 | ||||
-rw-r--r-- | doc/grammar.txt | 102 | ||||
-rw-r--r-- | doc/intern.md | 4 | ||||
-rw-r--r-- | doc/lib.md | 212 | ||||
-rw-r--r-- | doc/manual.md | 593 | ||||
-rw-r--r-- | doc/manual_experimental.md | 764 | ||||
-rw-r--r-- | doc/manual_experimental_strictnotnil.md | 15 | ||||
-rw-r--r-- | doc/markdown_rst.md | 154 | ||||
-rw-r--r-- | doc/mm.md | 27 | ||||
-rw-r--r-- | doc/nep1.md | 40 | ||||
-rw-r--r-- | doc/nimc.md | 66 | ||||
-rw-r--r-- | doc/nimdoc.css | 29 | ||||
-rw-r--r-- | doc/nimfix.md | 65 | ||||
-rw-r--r-- | doc/nimgrep.md | 2 | ||||
-rw-r--r-- | doc/nims.md | 39 | ||||
-rw-r--r-- | doc/nimsuggest.md | 2 | ||||
-rw-r--r-- | doc/packaging.md | 7 | ||||
-rw-r--r-- | doc/readme.txt | 2 | ||||
-rw-r--r-- | doc/refc.md | 1 | ||||
-rw-r--r-- | doc/sets_fragment.txt | 21 | ||||
-rw-r--r-- | doc/testament.md | 62 | ||||
-rw-r--r-- | doc/tools.md | 7 | ||||
-rw-r--r-- | doc/tut2.md | 6 | ||||
-rw-r--r-- | doc/tut3.md | 32 |
32 files changed, 2282 insertions, 573 deletions
diff --git a/doc/advopt.txt b/doc/advopt.txt index 3f439fdab..e4d11081a 100644 --- a/doc/advopt.txt +++ b/doc/advopt.txt @@ -4,8 +4,11 @@ Advanced commands: //compileToOC, objc compile project to Objective C code //js compile project to Javascript //e run a Nimscript file + //md2html convert a Markdown file to HTML + use `--docCmd:skip` to skip compiling snippets //rst2html convert a reStructuredText file to HTML use `--docCmd:skip` to skip compiling snippets + //md2tex convert a Markdown file to LaTeX //rst2tex convert a reStructuredText file to LaTeX //doc2tex extract the documentation to a LaTeX file //jsondoc extract the documentation to a json file @@ -45,7 +48,7 @@ Advanced options: --unitsep:on|off use the ASCII unit separator (31) between error messages, useful for IDE-like tooling --declaredLocs:on|off show declaration locations in messages - --spellSuggest|:num show at most `num >= 0` spelling suggestions on typos. + --spellSuggest:num show at most `num >= 0` spelling suggestions on typos. if `num` is not specified (or `auto`), return an implementation defined set of suggestions. --hints:on|off|list. `on|off` enables or disables hints. @@ -130,7 +133,9 @@ Advanced options: select which memory management to use; default is 'orc' --exceptions:setjmp|cpp|goto|quirky select the exception handling implementation - --index:on|off turn index file generation on|off + --index:on|off|only docgen: turn index file generation on|off (`only` means + not generate output files like HTML) + --noImportdoc:on|off turn loading documentation ``.idx`` files on|off --putenv:key=value set an environment variable --NimblePath:PATH add a path for Nimble support --noNimblePath deactivate the Nimble path @@ -164,9 +169,11 @@ Advanced options: enable experimental language feature --legacy:$2 enable obsolete/legacy language feature - --useVersion:1.0|1.2|1.6 emulate Nim version X of the Nim compiler, for testing --benchmarkVM:on|off turn benchmarking of VM code with cpuTime() on|off --profileVM:on|off turn compile time VM profiler on|off - --sinkInference:on|off turn sink parameter inference on|off (default: on) --panics:on|off turn panics into process terminations (default: off) --deepcopy:on|off enable 'system.deepCopy' for ``--mm:arc|orc`` + --jsbigint64:on|off toggle the use of BigInt for 64-bit integers for + the JavaScript backend (default: on) + --nimBasePattern:nimbase.h + allows to specify a custom pattern for `nimbase.h` diff --git a/doc/astspec.txt b/doc/astspec.txt index bfaec7155..7a7053a2d 100644 --- a/doc/astspec.txt +++ b/doc/astspec.txt @@ -893,7 +893,7 @@ on what keywords are present. Let's start with the simplest form. Concrete syntax: ```nim - import math + import std/math ``` AST: @@ -907,7 +907,7 @@ With ``except``, we get ``nnkImportExceptStmt``. Concrete syntax: ```nim - import math except pow + import std/math except pow ``` AST: @@ -916,13 +916,13 @@ AST: nnkImportExceptStmt(nnkIdent("math"),nnkIdent("pow")) ``` -Note that ``import math as m`` does not use a different node; rather, +Note that ``import std/math as m`` does not use a different node; rather, we use ``nnkImportStmt`` with ``as`` as an infix operator. Concrete syntax: ```nim - import strutils as su + import std/strutils as su ``` AST: @@ -945,7 +945,7 @@ If we use ``from ... import``, the result is different, too. Concrete syntax: ```nim - from math import pow + from std/math import pow ``` AST: @@ -954,7 +954,7 @@ AST: nnkFromStmt(nnkIdent("math"), nnkIdent("pow")) ``` -Using ``from math as m import pow`` works identically to the ``as`` modifier +Using ``from std/math as m import pow`` works identically to the ``as`` modifier with the ``import`` statement, but wrapped in ``nnkFromStmt``. Export statement @@ -1348,7 +1348,7 @@ Generic parameters are treated in the type, not the ``proc`` itself. Concrete syntax: ```nim - type MyProc[T] = proc(x: T) + type MyProc[T] = proc(x: T) {.nimcall.} ``` AST: @@ -1363,7 +1363,8 @@ AST: nnkProcTy( # behaves like a procedure declaration from here on nnkFormalParams( # ... - ) + ), + nnkPragma(nnkIdent("nimcall")) ) ) ``` @@ -1371,6 +1372,37 @@ AST: The same syntax applies to ``iterator`` (with ``nnkIteratorTy``), but *does not* apply to ``converter`` or ``template``. +Type class versions of these nodes generally share the same node kind but +without any child nodes. The ``tuple`` type class is represented by +``nnkTupleClassTy``, while a ``proc`` or ``iterator`` type class with pragmas +has an ``nnkEmpty`` node in place of the ``nnkFormalParams`` node of a +concrete ``proc`` or ``iterator`` type node. + + ```nim + type TypeClass = proc {.nimcall.} | ref | tuple + ``` + +AST: + + ```nim + nnkTypeDef( + nnkIdent("TypeClass"), + nnkEmpty(), + nnkInfix( + nnkIdent("|"), + nnkProcTy( + nnkEmpty(), + nnkPragma(nnkIdent("nimcall")) + ), + nnkInfix( + nnkIdent("|"), + nnkRefTy(), + nnkTupleClassTy() + ) + ) + ) + ``` + Mixin statement --------------- diff --git a/doc/backends.md b/doc/backends.md index 5258e9b4d..9f0c54835 100644 --- a/doc/backends.md +++ b/doc/backends.md @@ -250,6 +250,8 @@ which will likely make your program crash at runtime. The name `NimMain` can be influenced via the `--nimMainPrefix:prefix` switch. Use `--nimMainPrefix:MyLib` and the function to call is named `MyLibNimMain`. +When compiling to static or dynamic libraries, they don't call destructors of global variables as normal Nim programs would do. A C API `NimDestroyGlobals` is provided to call these global destructors. + ### Nim invocation example from C @@ -300,7 +302,7 @@ Instead of depending on the generation of the individual ``.c`` files you can also ask the Nim compiler to generate a statically linked library: ```cmd - nim c --app:staticLib --noMain fib.nim + nim c --app:staticLib fib.nim gcc -o m -Inimcache -Ipath/to/nim/lib maths.c libfib.nim.a ``` @@ -371,11 +373,7 @@ The manual mentions that [Nim strings are implicitly convertible to cstrings](manual.html#types-cstring-type) which makes interaction usually painless. Most C functions accepting a Nim string converted to a `cstring` will likely not need to keep this string around and by the time -they return the string won't be needed anymore. However, for the rare cases -where a Nim string has to be preserved and made available to the C backend -as a `cstring`, you will need to manually prevent the string data -from being freed with [GC_ref](system.html#GC_ref,string) and [GC_unref]( -system.html#GC_unref,string). +they return the string won't be needed anymore. A similar thing happens with C code invoking Nim code which returns a `cstring`. Consider the following proc: diff --git a/doc/contributing.md b/doc/contributing.md index d331c88c1..420c1438e 100644 --- a/doc/contributing.md +++ b/doc/contributing.md @@ -11,7 +11,7 @@ Contributing Contributing happens via "Pull requests" (PR) on GitHub. Every PR needs to be reviewed before it can be merged and the Continuous Integration should be green. The title of a PR should contain a brief description. If it fixes an issue, -in addition to the number of the issue, the title should also contain a description +in addition to the number of the issue, the title should also contain a description of the issue. The PR has to be approved by two core developers or by Araq. @@ -137,9 +137,9 @@ You can run the tests with which will run a good subset of tests. Some tests may fail. If you only want to see the output of failing tests, go for -```cmd + ```cmd ./koch tests --failing all -``` + ``` You can also run only a single category of tests. A category is a subdirectory in the ``tests/`` directory. There are a couple of special categories; for a @@ -198,7 +198,7 @@ tell you if any new tests passed/failed. Deprecation =========== -Backwards compatibility is important. When renaming types, procedures, etc. the old name +Backwards compatibility is important. When renaming types, procedures, etc. the old name must be marked as deprecated using the `deprecated` pragma: ```nim @@ -336,7 +336,7 @@ To avoid accidental highlighting follow this rule in ``*.nim`` files: .. Note:: ``*.rst`` files have ``:literal:`` as their default role. So for them the rule above is only applicable if the ``:nim:`` role - is set up manually as the default \[*]: + is set up manually as the default [^1]: .. role:: nim(code) :language: nim @@ -345,7 +345,7 @@ To avoid accidental highlighting follow this rule in ``*.nim`` files: The first 2 lines are for other RST implementations, including Github one. - \[*] this is fulfilled when ``doc/rstcommon.rst`` is included. + [^1]: this is fulfilled when ``doc/rstcommon.rst`` is included. Best practices ============== @@ -513,7 +513,7 @@ General commit rules https://github.com/nim-lang/Nim/pull/9356 8. Do not mix pure formatting changes (e.g. whitespace changes, nimpretty) or - automated changes (e.g. nimfix) with other code changes: these should be in + automated changes with other code changes: these should be in separate commits (and the merge on GitHub should not squash these into 1). diff --git a/doc/destructors.md b/doc/destructors.md index a96ac35ef..e192fd362 100644 --- a/doc/destructors.md +++ b/doc/destructors.md @@ -13,12 +13,12 @@ Nim Destructors and Move Semantics About this document =================== -This document describes the upcoming Nim runtime which does +This document describes the ARC/ORC Nim runtime which does not use classical GC algorithms anymore but is based on destructors and -move semantics. The new runtime's advantages are that Nim programs become +move semantics. The advantages are that Nim programs become oblivious to the involved heap sizes and programs are easier to write to make effective use of multi-core machines. As a nice bonus, files and sockets and -the like will not require manual `close` calls anymore. +the like can be written not to require manual `close` calls anymore. This document aims to be a precise specification about how move semantics and destructors work in Nim. @@ -30,17 +30,20 @@ Motivating example With the language mechanisms described here, a custom seq could be written as: - ```nim + ```nim test type myseq*[T] = object len, cap: int data: ptr UncheckedArray[T] - proc `=destroy`*[T](x: var myseq[T]) = + proc `=destroy`*[T](x: myseq[T]) = if x.data != nil: for i in 0..<x.len: `=destroy`(x.data[i]) dealloc(x.data) + proc `=wasMoved`*[T](x: var myseq[T]) = + x.data = nil + proc `=trace`[T](x: var myseq[T]; env: pointer) = # `=trace` allows the cycle collector `--mm:orc` # to understand how to trace the object graph. @@ -51,7 +54,7 @@ written as: # do nothing for self-assignments: if a.data == b.data: return `=destroy`(a) - wasMoved(a) + `=wasMoved`(a) a.len = b.len a.cap = b.cap if b.data != nil: @@ -59,11 +62,19 @@ written as: for i in 0..<a.len: a.data[i] = b.data[i] + proc `=dup`*[T](a: myseq[T]): myseq[T] {.nodestroy.} = + # an optimized version of `=wasMoved(tmp); `=copy(tmp, src)` + # usually present if a custom `=copy` hook is overridden + result = myseq[T](len: a.len, cap: a.cap, data: nil) + if a.data != nil: + result.data = cast[typeof(result.data)](alloc(result.cap * sizeof(T))) + for i in 0..<result.len: + result.data[i] = `=dup`(a.data[i]) + proc `=sink`*[T](a: var myseq[T]; b: myseq[T]) = # move assignment, optional. # Compiler is using `=destroy` and `copyMem` when not provided `=destroy`(a) - wasMoved(a) a.len = b.len a.cap = b.cap a.data = b.data @@ -84,9 +95,10 @@ written as: x.data[i] = y proc createSeq*[T](elems: varargs[T]): myseq[T] = - result.cap = elems.len - result.len = elems.len - result.data = cast[typeof(result.data)](alloc(result.cap * sizeof(T))) + result = myseq[T]( + len: elems.len, + cap: elems.len, + data: cast[typeof(result.data)](alloc(result.cap * sizeof(T)))) for i in 0..<result.len: result.data[i] = elems[i] proc len*[T](x: myseq[T]): int {.inline.} = x.len @@ -101,7 +113,7 @@ well as other standard collections is performed via so-called "Lifetime-tracking hooks", which are particular [type bound operators]( manual.html#procedures-type-bound-operators). -There are 4 different hooks for each (generic or concrete) object type `T` (`T` can also be a +There are 6 different hooks for each (generic or concrete) object type `T` (`T` can also be a `distinct` type) that are called implicitly by the compiler. (Note: The word "hook" here does not imply any kind of dynamic binding @@ -117,21 +129,48 @@ other associated resources. Variables are destroyed via this hook when they go out of scope or when the routine they were declared in is about to return. -The prototype of this hook for a type `T` needs to be: +A `=destroy` hook is allowed to have a parameter of a `var T` or `T` type. Taking a `var T` type is deprecated. The prototype of this hook for a type `T` needs to be: ```nim - proc `=destroy`(x: var T) + proc `=destroy`(x: T) ``` The general pattern in `=destroy` looks like: ```nim - proc `=destroy`(x: var T) = + proc `=destroy`(x: T) = # first check if 'x' was moved to somewhere else: if x.field != nil: freeResource(x.field) ``` +A `=destroy` is implicitly annotated with `.raises: []`; a destructor +should not raise exceptions. For backwards compatibility the compiler +produces a warning for a `=destroy` that does raise. + +A `=destroy` can explicitly list the exceptions it can raise, if any, +but this of little utility as a raising destructor is implementation defined +behavior. Later versions of the language specification might cover this case precisely. + + +`=wasMoved` hook +---------------- + +A `=wasMoved` hook sets the object to a state that signifies to the destructor there is nothing to destroy. + +The prototype of this hook for a type `T` needs to be: + + ```nim + proc `=wasMoved`(x: var T) + ``` + +Usually some pointer field inside the object is set to `nil`: + + ```nim + proc `=wasMoved`(x: var T) = + x.field = nil + ``` + `=sink` hook ------------ @@ -238,10 +277,10 @@ The general pattern in using `=destroy` with `=trace` looks like: Test[T](size: size, arr: cast[ptr UncheckedArray[T]](alloc0(sizeof(T) * size))) - proc `=destroy`[T](dest: var Test[T]) = + proc `=destroy`[T](dest: Test[T]) = if dest.arr != nil: for i in 0 ..< dest.size: dest.arr[i].`=destroy` - dest.arr.dealloc + dealloc dest.arr proc `=trace`[T](dest: var Test[T]; env: pointer) = if dest.arr != nil: @@ -255,6 +294,31 @@ The general pattern in using `=destroy` with `=trace` looks like: than the other hooks. +`=dup` hook +----------- + +A `=dup` hook duplicates an object. `=dup(x)` can be regarded as an optimization replacing a `wasMoved(dest); =copy(dest, x)` operation. + +The prototype of this hook for a type `T` needs to be: + + ```nim + proc `=dup`(x: T): T + ``` + +The general pattern in implementing `=dup` looks like: + + ```nim + type + Ref[T] = object + data: ptr T + rc: ptr int + + proc `=dup`[T](x: Ref[T]): Ref[T] = + result = x + if x.rc != nil: + inc x.rc[] + ``` + Move semantics ============== @@ -264,6 +328,24 @@ document uses the notation `lastReadOf(x)` to describe that `x` is not used afterward. This property is computed by a static control flow analysis but can also be enforced by using `system.move` explicitly. +One can query if the analysis is able to perform a move with `system.ensureMove`. +`move` enforces a move operation and calls `=wasMoved` whereas `ensureMove` is +an annotation that implies no runtime operation. An `ensureMove` annotation leads to a static error +if the compiler cannot prove that a move would be safe. + +For example: + + ```nim + proc main(normalParam: string; sinkParam: sink string) = + var x = "abc" + # valid: + let valid = ensureMove x + # invalid: + let invalid = ensureMove normalParam + # valid: + let alsoValid = ensureMove sinkParam + ``` + Swap ==== @@ -381,7 +463,7 @@ destroyed at the scope exit. x = lastReadOf z ------------------ (move-optimization) `=sink`(x, z) - wasMoved(z) + `=wasMoved`(z) v = v @@ -401,14 +483,14 @@ destroyed at the scope exit. f_sink(notLastReadOf y) -------------------------- (copy-to-sink) - (let tmp; `=copy`(tmp, y); + (let tmp = `=dup`(y); f_sink(tmp)) f_sink(lastReadOf y) ----------------------- (move-to-sink) f_sink(y) - wasMoved(y) + `=wasMoved`(y) Object and array construction @@ -421,7 +503,7 @@ function has `sink` parameters. Destructor removal ================== -`wasMoved(x);` followed by a `=destroy(x)` operation cancel each other +`=wasMoved(x)` followed by a `=destroy(x)` operation cancel each other out. An implementation is encouraged to exploit this in order to improve efficiency and code sizes. The current implementation does perform this optimization. @@ -430,11 +512,11 @@ optimization. Self assignments ================ -`=sink` in combination with `wasMoved` can handle self-assignments but +`=sink` in combination with `=wasMoved` can handle self-assignments but it's subtle. The simple case of `x = x` cannot be turned -into `=sink(x, x); wasMoved(x)` because that would lose `x`'s value. +into `=sink(x, x); =wasMoved(x)` because that would lose `x`'s value. The solution is that simple self-assignments that consist of - Symbols: `x = x` @@ -469,10 +551,10 @@ Is transformed into: try: if cond: `=sink`(result, a) - wasMoved(a) + `=wasMoved`(a) else: `=sink`(result, b) - wasMoved(b) + `=wasMoved`(b) finally: `=destroy`(b) `=destroy`(a) @@ -486,10 +568,10 @@ Is transformed into: `=sink`(y, "xyz") `=sink`(x, select(true, let blitTmp = x - wasMoved(x) + `=wasMoved`(x) blitTmp, let blitTmp = y - wasMoved(y) + `=wasMoved`(y) blitTmp)) echo [x] finally: @@ -517,7 +599,7 @@ that the pointer does not outlive its origin. No destructor call is injected for expressions of type `lent T` or of type `var T`. - ```nim + ```nim test type Tree = object kids: seq[Tree] @@ -525,13 +607,13 @@ for expressions of type `lent T` or of type `var T`. proc construct(kids: sink seq[Tree]): Tree = result = Tree(kids: kids) # converted into: - `=sink`(result.kids, kids); wasMoved(kids) + `=sink`(result.kids, kids); `=wasMoved`(kids) `=destroy`(kids) proc `[]`*(x: Tree; i: int): lent Tree = result = x.kids[i] # borrows from 'x', this is transformed into: - result = addr x.kids[i] + # result = addr x.kids[i] # This means 'lent' is like 'var T' a hidden pointer. # Unlike 'var' this hidden pointer cannot be used to mutate the object. @@ -637,11 +719,11 @@ The ability to override a hook leads to a phase ordering problem: # error: destructor for 'f' called here before # it was seen in this module. - proc `=destroy`[T](f: var Foo[T]) = + proc `=destroy`[T](f: Foo[T]) = discard ``` -The solution is to define ``proc `=destroy`[T](f: var Foo[T])`` before +The solution is to define ``proc `=destroy`[T](f: Foo[T])`` before it is used. The compiler generates implicit hooks for all types in *strategic places* so that an explicitly provided hook that comes too "late" can be detected reliably. These *strategic places* @@ -662,7 +744,7 @@ The experimental `nodestroy`:idx: pragma inhibits hook injections. This can be used to specialize the object traversal in order to avoid deep recursions: - ```nim + ```nim test type Node = ref object x, y: int32 left, right: Node @@ -670,15 +752,15 @@ used to specialize the object traversal in order to avoid deep recursions: type Tree = object root: Node - proc `=destroy`(t: var Tree) {.nodestroy.} = + proc `=destroy`(t: Tree) {.nodestroy.} = # use an explicit stack so that we do not get stack overflows: var s: seq[Node] = @[t.root] while s.len > 0: let x = s.pop if x.left != nil: s.add(x.left) if x.right != nil: s.add(x.right) - # free the memory explicit: - dispose(x) + # free the memory explicitly: + `=dispose`(x) # notice how even the destructor for 's' is not called implicitly # anymore thanks to .nodestroy, so we have to call it on our own: `=destroy`(s) diff --git a/doc/docgen.md b/doc/docgen.md index e4b040064..3cc75fc18 100644 --- a/doc/docgen.md +++ b/doc/docgen.md @@ -9,17 +9,33 @@ .. include:: rstcommon.rst .. contents:: +.. importdoc:: markdown_rst.md, compiler/docgen.nim Introduction ============ This document describes the `documentation generation tools`:idx: built into the [Nim compiler](nimc.html), which can generate HTML, Latex and JSON output -from input ``.nim`` files and projects, as well as HTML and LaTeX from input RST -(reStructuredText) files. The output documentation will include the module +from input ``.nim`` files and projects. +The output documentation will include the module dependencies (`import`), any top-level documentation comments (`##`), and exported symbols (`*`), including procedures, types, and variables. +=================== ============== +command output format +=================== ============== +`nim doc`:cmd: ``.html`` HTML +`nim doc2tex`:cmd: ``.tex`` LaTeX +`nim jsondoc`:cmd: ``.json`` JSON +=================== ============== + +Nim can generate HTML and LaTeX from input Markdown and +RST (reStructuredText) files as well, which is intended for writing +standalone documents like user's guides and technical specifications. +See [Nim-flavored Markdown and reStructuredText] document for the description +of this feature and particularly section [Command line usage] for the full +list of supported commands. + Quick start ----------- @@ -43,6 +59,7 @@ Generate HTML documentation for a whole project: # or `$nimcache/htmldocs` with `--usenimcache` which avoids clobbering your sources; # and likewise without `--project`. # Adding `-r` will open in a browser directly. + # Use `--showNonExports` to show non-exported fields of an exported type. ``` Documentation Comments @@ -103,6 +120,64 @@ won't influence RST formatting. ## Paragraph. ``` +Structuring output directories +------------------------------ + +Basic directory for output is set by `--outdir:OUTDIR`:option: switch, +by default `OUTDIR` is ``htmldocs`` sub-directory in the directory of +the processed file. + +There are 2 basic options as to how generated HTML output files are stored: + +1) complex hierarchy when docgen-compiling with `--project`:option:, + which follows directory structure of the project itself. + So `nim doc`:cmd: replicates project's directory structure + inside `--outdir:OUTDIR`:option: directory. + `--project`:option: is well suited for projects that have 1 main module. + File name clashes are impossible in this case. + +2) flattened structure, where user-provided script goes through all + needed input files and calls commands like `nim doc`:cmd: + with `--outdir:OUTDIR`:option: switch, thus putting all HTML (and + ``.idx``) files into 1 directory. + + .. Important:: Make sure that you don't have files with same base name + like ``x.nim`` and ``x.md`` in the same package, otherwise you'll + have name conflict for ``x.html``. + + .. Tip:: To structure your output directories and avoid file name + clashes you can split your project into + different *packages* -- parts of your repository that are + docgen-compiled with different `--outdir:OUTDIR`:option: options. + + An example of such strategy is Nim repository itself which has: + + * its stdlib ``.nim`` files from different directories and ``.md`` + documentation from ``doc/`` directory are all docgen-compiled + into `--outdir:web/upload/<version>/`:option: directory + * its ``.nim`` files from ``compiler/`` directory are docgen-compiled + into `--outdir:web/upload/<version>/compiler/`:option: directory. + Interestingly, it's compiled with complex hierarchy using + `--project`:option: switch. + + Contents of ``web/upload/<version>`` are then deployed into Nim's + Web server. + + This output directory structure allows to work correctly with files like + ``compiler/docgen.nim`` (implementation) and ``doc/docgen.md`` (user + documentation) in 1 repository. + + +Index files +----------- + +Index (``.idx``) files are used for 2 different purposes: + +1. easy cross-referencing between different ``.nim`` and/or ``.md`` / ``.rst`` + files described in [Nim external referencing] +2. creating a whole-project index for searching of symbols and keywords, + see [Buildindex command]. + Document Types ============== @@ -226,22 +301,62 @@ Note that the `jsondoc`:option: command outputs its JSON without pretty-printing while `jsondoc0`:option: outputs pretty-printed JSON. -Referencing Nim symbols: simple documentation links -=================================================== - -You can reference Nim identifiers from Nim documentation comments, currently -only inside their ``.nim`` file (or inside a ``.rst`` file included from -a ``.nim``). The point is that such links will be resolved automatically -by `nim doc`:cmd: (or `nim jsondoc`:cmd: or `nim doc2tex`:cmd:). +Simple documentation links +========================== + +It's possible to use normal Markdown/RST syntax to *manually* +reference Nim symbols using HTML anchors, however Nim has an *automatic* +facility that makes referencing inside ``.nim`` and ``.md/.rst`` files and +between them easy and seamless. +The point is that such links will be resolved automatically +by `nim doc`:cmd: (or `md2html`:option:, or `jsondoc`:option:, +or `doc2tex`:option:, ...). And, unlike manual links, such automatic +links **check** that their target exists -- a warning is emitted for +any broken link, so you avoid broken links in your project. + +Nim treats both ``.md/.rst`` files and ``.nim`` modules (their doc comment +part) as *documents* uniformly. +Hence all directions of referencing are equally possible having the same syntax: + +1. ``.md/rst`` -> itself (internal). See [Markup local referencing]. +2. ``.md/rst`` -> external ``.md/rst``. See [Markup external referencing]. + To summarize, referencing in `.md`/`.rst` files was already described in + [Nim-flavored Markdown and reStructuredText] + (particularly it described usage of index files for referencing), + while in this document we focus on Nim-specific details. +3. ``.md/rst`` -> external ``.nim``. See [Nim external referencing]. +4. ``.nim`` -> itself (internal). See [Nim local referencing]. +5. ``.nim`` -> external ``.md/rst``. See [Markup external referencing]. +6. ``.nim`` -> external ``.nim``. See [Nim external referencing]. + +To put it shortly, local referencing always works out of the box, +external referencing requires to use ``.. importdoc:: <file>`` +directive to import `file` and to ensure that the corresponding +``.idx`` file was generated. + +Syntax for referencing is basically the same as for normal markup. +Recall from [Referencing] that our parser supports two equivalent syntaxes +for referencing, Markdown and RST one. +So to reference ``proc f`` one should use something like that, +depending on markup type: + + Markdown RST + + Ref. [proc f] or [f] Ref. `proc f`_ or just f_ for a one-word case + +Nim local referencing +--------------------- + +You can reference Nim identifiers from Nim documentation comments +inside their ``.nim`` file (or inside a ``.rst`` file included from +a ``.nim``). This pertains to any exported symbol like `proc`, `const`, `iterator`, etc. -Syntax for referencing is basically a normal RST one: addition of -underscore `_` to a *link text*. -Link text is either one word or a group of words enclosed by backticks `\`` -(for a one word case backticks are usually omitted). +Link text is either one word or a group of words enclosed by delimiters +(brackets ``[...]`` for Markdown or backticks `\`...\`_` for RST). Link text will be displayed *as is* while *link target* will be set to -the anchor \[*] of Nim symbol that corresponds to link text. +the anchor [^1] of Nim symbol that corresponds to link text. -\[*] anchors' format is described in [HTML anchor generation] section below. +[^1] anchors' format is described in [HTML anchor generation] section below. If you have a constant: @@ -405,6 +520,143 @@ recognized fine: ... ## Ref. `CopyFlag enum`_ +Nim external referencing +------------------------ + +Just like for [Markup external referencing], which saves markup anchors, +the Nim symbols are also saved in ``.idx`` files, so one needs +to generate them beforehand, and they should be loaded by +an ``.. importdoc::`` directive. Arguments to ``.. importdoc::`` is a +comma-separated list of Nim modules or Markdown/RST documents. + +`--index:only`:option: tells Nim to only generate ``.idx`` file and +do **not** attempt to generate HTML/LaTeX output. +For ``.nim`` modules there are 2 alternatives to work with ``.idx`` files: + +1. using [Project switch] implies generation of ``.idx`` files, + however, if ``importdoc`` is called on upper modules as its arguments, + their ``.idx`` are not yet created. Thus one should generate **all** + required ``.idx`` first: + ```cmd + nim doc --project --index:only <main>.nim + nim doc --project <main>.nim + ``` +2. or run `nim doc --index:only <module.nim>`:cmd: command for **all** (used) + Nim modules in your project. Then run `nim doc <module.nim>` on them for + output HTML generation. + + .. Warning:: A mere `nim doc --index:on`:cmd: may fail on an attempt to do + ``importdoc`` from another module (for which ``.idx`` was not yet + generated), that's why `--index:only`:option: shall be used instead. + + For ``.md``/``.rst`` markup documents point 2 is the only option. + +Then, you can freely use something like this in ``your_module.nim``: + + ```nim + ## .. importdoc:: user_manual.md, another_module.nim + + ... + ## Ref. [some section from User Manual]. + + ... + ## Ref. [proc f] + ## (assuming you have a proc `f` in ``another_module``). + ``` + +and compile it by `nim doc`:cmd:. Note that link text will +be automatically prefixed by the module name of symbol, +so you will see something like "Ref. [another_module: proc f](#)" +in the generated output. + +It's also possible to reference a whole module by prefixing or +suffixing full canonical module name with "module": + + Ref. [module subdir/name] or [subdir/name module]. + +Markup documents as a whole can be referenced just by their title +(or by their file name if the title was not set) without any prefix. + +.. Tip:: During development process the stage of ``.idx`` files generation + can be done only *once*, after that you use already generated ``.idx`` + files while working with a document *being developed* (unless you do + incompatible changes to *referenced* documents). + +.. Hint:: After changing a *referenced* document file one may need + to regenerate its corresponding ``.idx`` file to get correct results. + Of course, when referencing *internally* inside any given ``.nim`` file, + it's not needed, one can even immediately use any freshly added anchor + (a document's own ``.idx`` file is not used for resolving its internal links). + +If an ``importdoc`` directive fails to find a ``.idx``, then an error +is emitted. + +In case of such compilation failures please note that: + +* **all** relative paths, given to ``importdoc``, relate to insides of + ``OUTDIR``, and **not** project's directory structure. + +* ``importdoc`` searches for ``.idx`` in `--outdir:OUTDIR`:option: directory + (``htmldocs`` by default) and **not** around original modules, so: + + .. Tip:: look into ``OUTDIR`` to understand what's going on. + +* also keep in mind that ``.html`` and ``.idx`` files should always be + output to the same directory, so check this and, if it's not true, check + that both runs *with* and *without* `--index:only`:option: have all + other options the same. + +To summarize, for 2 basic options of [Structuring output directories] +compilation options are different: + +1) complex hierarchy with `--project`:option: switch. + + As the **original** project's directory structure is replicated in + `OUTDIR`, all passed paths are related to this structure also. + + E.g. if a module ``path1/module.nim`` does + ``.. importdoc:: path2/another.nim`` then docgen tries to load file + ``OUTDIR/path1/path2/another.idx``. + + .. Note:: markup documents are just placed into the specified directory + `OUTDIR`:option: by default (i.e. they are **not** affected by + `--project`:option:), so if you have ``PROJECT/doc/manual.md`` + document and want to use complex hierarchy (with ``doc/``), + compile it with `--docroot`:option:\: + ```cmd + # 1st stage + nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ + --index:only PROJECT/doc/manual.md + ... + # 2nd stage + nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ + PROJECT/doc/manual.md + ``` + + Then the output file will be placed as ``OUTDIR/doc/manual.idx``. + So if you have ``PROJECT/path1/module.nim``, then ``manual.md`` can + be referenced as ``../doc/manual.md``. + +2) flattened structure. + + E.g. if a module ``path1/module.nim`` does + ``.. importdoc:: path2/another.nim`` then docgen tries to load + ``OUTDIR/path2/another.idx``, so the path ``path1`` + does not matter and providing ``path2`` can be useful only + in the case it contains another package that was placed there + using `--outdir:OUTDIR/path2`:option:. + + The links' text will be prefixed as ``another: ...`` in both cases. + + .. Warning:: Again, the same `--outdir:OUTDIR`:option: option should + be provided to both `doc --index:only`:option: / + `md2html --index:only`:option: and final generation by + `doc`:option:/`md2html`:option: inside 1 package. + +To temporarily disable ``importdoc``, e.g. if you don't need +correct link resolution at the moment, use a `--noImportdoc`:option: switch +(only warnings about unresolved links will be generated for external references). + Related Options =============== @@ -434,10 +686,21 @@ index file is line-oriented (newlines have to be escaped). Each line represents a tab-separated record of several columns, the first two mandatory, the rest optional. See the [Index (idx) file format] section for details. +.. Note:: `--index`:option: switch only affects creation of ``.idx`` + index files, while user-searchable Index HTML file is created by + `buildIndex`:option: command. + +Buildindex command +------------------ + Once index files have been generated for one or more modules, the Nim -compiler command `buildIndex directory` can be run to go over all the index +compiler command `nim buildIndex directory`:cmd: can be run to go over all the index files in the specified directory to generate a [theindex.html](theindex.html) -file. +file: + + ```cmd + nim buildIndex -o:path/to/htmldocs/theindex.html path/to/htmldocs + ``` See source switch ----------------- @@ -568,10 +831,22 @@ references so they can be later concatenated into a big index file with the file format in detail. Index files are line-oriented and tab-separated (newline and tab characters -have to be escaped). Each line represents a record with at least two fields -but can have up to four (additional columns are ignored). The content of these -columns is: - +have to be escaped). Each line represents a record with 6 fields. +The content of these columns is: + +0. Discriminator tag denoting type of the index entry, allowed values are: + `markupTitle` + : a title for ``.md``/``.rst`` document + `nimTitle` + : a title of ``.nim`` module + `heading` + : heading of sections, can be both in Nim and markup files + `idx` + : terms marked with :idx: role + `nim` + : a Nim symbol + `nimgrp` + : a Nim group for overloadable symbols like `proc`s 1. Mandatory term being indexed. Terms can include quoting according to Nim's rules (e.g. \`^\`). 2. Base filename plus anchor hyperlink (e.g. ``algorithm.html#*,int,SortOrder``). @@ -581,29 +856,20 @@ columns is: not for an API symbol but for a TOC entry. 4. Optional title or description of the hyperlink. Browsers usually display this as a tooltip after hovering a moment over the hyperlink. +5. A line number of file where the entry was defined. + +The index generation tools differentiate between documentation +generated from ``.nim`` files and documentation generated from ``.md`` or +``.rst`` files by tag `nimTitle` or `markupTitle` in the 1st line of +the ``.idx`` file. -The index generation tools try to differentiate between documentation -generated from ``.nim`` files and documentation generated from ``.txt`` or -``.rst`` files. The former are always closely related to source code and -consist mainly of API entries. The latter are generic documents meant for -human reading. - -To differentiate both types (documents and APIs), the index generator will add -to the index of documents an entry with the title of the document. Since the -title is the topmost element, it will be added with a second field containing -just the filename without any HTML anchor. By convention, this entry without -anchor is the *title entry*, and since entries in the index file are added as -they are scanned, the title entry will be the first line. The title for APIs -is not present because it can be generated concatenating the name of the file -to the word **Module**. - -Normal symbols are added to the index with surrounding whitespaces removed. An -exception to this are the table of content (TOC) entries. TOC entries are added to -the index file with their third column having as much prefix spaces as their -level is in the TOC (at least 1 character). The prefix whitespace helps to -filter TOC entries from API or text symbols. This is important because the -amount of spaces is used to replicate the hierarchy for document TOCs in the -final index, and TOC entries found in ``.nim`` files are discarded. +.. TODO Normal symbols are added to the index with surrounding whitespaces removed. An + exception to this are the table of content (TOC) entries. TOC entries are added to + the index file with their third column having as much prefix spaces as their + level is in the TOC (at least 1 character). The prefix whitespace helps to + filter TOC entries from API or text symbols. This is important because the + amount of spaces is used to replicate the hierarchy for document TOCs in the + final index, and TOC entries found in ``.nim`` files are discarded. Additional resources @@ -615,6 +881,8 @@ Additional resources [Markdown and RST markup languages](markdown_rst.html), which also contains the list of implemented features of these markup languages. +* the implementation is in [module compiler/docgen]. + The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and ``config/nimdoc.tex.cfg`` configuration files. You can add and modify these files to your project to change the look of the docgen output. diff --git a/doc/docgen_sample.nim b/doc/docgen_sample.nim index 7a167cb45..06b8d7f8e 100644 --- a/doc/docgen_sample.nim +++ b/doc/docgen_sample.nim @@ -1,6 +1,6 @@ ## This module is a sample. -import strutils +import std/strutils proc helloWorld*(times: int) = ## Takes an integer and outputs diff --git a/doc/estp.md b/doc/estp.md index d0ef4f1ba..8a986bdf3 100644 --- a/doc/estp.md +++ b/doc/estp.md @@ -28,7 +28,7 @@ Otherwise your program is profiled. ```nim when compileOption("profiler"): - import nimprof + import std/nimprof ``` After your program has finished the profiler will create a diff --git a/doc/filelist.txt b/doc/filelist.txt index 74995f154..5522414fe 100644 --- a/doc/filelist.txt +++ b/doc/filelist.txt @@ -31,16 +31,14 @@ semstmts contains the semantic checking phase for statements semtypes contains the semantic checking phase for types seminst instantiation of generic procs and types semfold contains code to deal with constant folding -semthreads deep program analysis for threads -evals contains an AST interpreter for compile time evaluation +sempass2 Second semantic checking pass over the AST +vm contains an AST interpreter for compile time evaluation pragmas semantic checking of pragmas idents implements a general mapping from identifiers to an internal representation (`PIdent`) that is used so that a simple id-comparison suffices to establish whether two Nim identifiers are equivalent -ropes implements long strings represented as trees for - lazy evaluation; used mainly by the code generators transf transformations on the AST that need to be done before code generation diff --git a/doc/grammar.txt b/doc/grammar.txt index 63ce4503c..51b3e0053 100644 --- a/doc/grammar.txt +++ b/doc/grammar.txt @@ -1,5 +1,5 @@ # This file is generated by compiler/parser.nim. -module = stmt ^* (';' / IND{=}) +module = complexOrSimpleStmt ^* (';' / IND{=}) comma = ',' COMMENT? semicolon = ';' COMMENT? colon = ':' COMMENT? @@ -7,7 +7,7 @@ colcom = ':' COMMENT? operator = OP0 | OP1 | OP2 | OP3 | OP4 | OP5 | OP6 | OP7 | OP8 | OP9 | 'or' | 'xor' | 'and' | 'is' | 'isnot' | 'in' | 'notin' | 'of' | 'as' | 'from' - | 'div' | 'mod' | 'shl' | 'shr' | 'not' | 'static' | '..' + | 'div' | 'mod' | 'shl' | 'shr' | 'not' | '..' prefixOperator = operator optInd = COMMENT? IND? optPar = (IND{>} | IND{=})? @@ -26,11 +26,16 @@ operatorB = OP0 | OP1 | OP2 | OP3 | OP4 | OP5 | OP6 | OP7 | OP8 | OP9 | 'div' | 'mod' | 'shl' | 'shr' | 'in' | 'notin' | 'is' | 'isnot' | 'not' | 'of' | 'as' | 'from' | '..' | 'and' | 'or' | 'xor' symbol = '`' (KEYW|IDENT|literal|(operator|'('|')'|'['|']'|'{'|'}'|'=')+)+ '`' - | IDENT | KEYW -exprColonEqExpr = expr (':'|'=' expr)? + | IDENT | 'addr' | 'type' | 'static' +symbolOrKeyword = symbol | KEYW +exprColonEqExpr = expr ((':'|'=') expr + / doBlock extraPostExprBlock*)? +exprEqExpr = expr ('=' expr + / doBlock extraPostExprBlock*)? exprList = expr ^+ comma +optionalExprList = expr ^* comma exprColonEqExprList = exprColonEqExpr (comma exprColonEqExpr)* (comma)? -qualifiedIdent = symbol ('.' optInd symbol)? +qualifiedIdent = symbol ('.' optInd symbolOrKeyword)? setOrTableConstr = '{' ((exprColonEqExpr comma)* | ':' ) '}' castExpr = 'cast' ('[' optInd typeDesc optPar ']' '(' optInd expr optPar ')') / parKeyw = 'discard' | 'include' | 'if' | 'while' | 'case' | 'try' @@ -40,7 +45,8 @@ par = '(' optInd ( &parKeyw (ifExpr / complexOrSimpleStmt) ^+ ';' | ';' (ifExpr / complexOrSimpleStmt) ^+ ';' | pragmaStmt - | simpleExpr ( ('=' expr (';' (ifExpr / complexOrSimpleStmt) ^+ ';' )? ) + | simpleExpr ( (doBlock extraPostExprBlock*) + | ('=' expr (';' (ifExpr / complexOrSimpleStmt) ^+ ';' )? ) | (':' expr (',' exprColonEqExpr ^+ ',' )? ) ) ) optPar ')' literal = | INT_LIT | INT8_LIT | INT16_LIT | INT32_LIT | INT64_LIT @@ -56,29 +62,30 @@ identOrLiteral = generalizedLit | symbol | literal tupleConstr = '(' optInd (exprColonEqExpr comma?)* optPar ')' arrayConstr = '[' optInd (exprColonEqExpr comma?)* optPar ']' primarySuffix = '(' (exprColonEqExpr comma?)* ')' - | '.' optInd symbol ('[:' exprList ']' ( '(' exprColonEqExpr ')' )?)? generalizedLit? - | DOTLIKEOP optInd symbol generalizedLit? + | '.' optInd symbolOrKeyword ('[:' exprList ']' ( '(' exprColonEqExpr ')' )?)? generalizedLit? + | DOTLIKEOP optInd symbolOrKeyword generalizedLit? | '[' optInd exprColonEqExprList optPar ']' | '{' optInd exprColonEqExprList optPar '}' - | &( '`'|IDENT|literal|'cast'|'addr'|'type') expr (comma expr)* # command syntax pragma = '{.' optInd (exprColonEqExpr comma?)* optPar ('.}' | '}') identVis = symbol OPR? # postfix position -identVisDot = symbol '.' optInd symbol OPR? +identVisDot = symbol '.' optInd symbolOrKeyword OPR? identWithPragma = identVis pragma? identWithPragmaDot = identVisDot pragma? declColonEquals = identWithPragma (comma identWithPragma)* comma? - (':' optInd typeDesc)? ('=' optInd expr)? + (':' optInd typeDescExpr)? ('=' optInd expr)? identColonEquals = IDENT (comma IDENT)* comma? - (':' optInd typeDesc)? ('=' optInd expr)?) -tupleDecl = 'tuple' - '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']' | - COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)? + (':' optInd typeDescExpr)? ('=' optInd expr)?) +tupleTypeBracket = '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']' +tupleType = 'tuple' tupleTypeBracket +tupleDecl = 'tuple' (tupleTypeBracket / + COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?) paramList = '(' declColonEquals ^* (comma/semicolon) ')' paramListArrow = paramList? ('->' optInd typeDesc)? paramListColon = paramList? (':' optInd typeDesc)? doBlock = 'do' paramListArrow pragma? colcom stmt routineExpr = ('proc' | 'func' | 'iterator') paramListColon pragma? ('=' COMMENT? stmt)? -forStmt = 'for' (identWithPragma ^+ comma) 'in' expr colcom stmt +routineType = ('proc' | 'iterator') paramListColon pragma? +forStmt = 'for' ((varTuple / identWithPragma) ^+ comma) 'in' expr colcom stmt forExpr = forStmt expr = (blockExpr | ifExpr @@ -87,25 +94,33 @@ expr = (blockExpr | forExpr | tryExpr) / simpleExpr -primary = operatorB primary primarySuffix* | - tupleDecl | routineExpr | enumDecl - objectDecl | conceptDecl | ('bind' primary) - ('var' | 'out' | 'ref' | 'ptr' | 'distinct') primary - / prefixOperator* identOrLiteral primarySuffix* -typeDesc = simpleExpr ('not' expr)? -typeDefAux = simpleExpr ('not' expr - | postExprBlocks)? -postExprBlocks = ':' stmt? ( IND{=} doBlock - | IND{=} 'of' exprList ':' stmt - | IND{=} 'elif' expr ':' stmt - | IND{=} 'except' exprList ':' stmt - | IND{=} 'finally' ':' stmt - | IND{=} 'else' ':' stmt )* -exprStmt = simpleExpr - (( '=' optInd expr colonBody? ) - / ( expr ^+ comma - postExprBlocks - ))? +simplePrimary = SIGILLIKEOP? identOrLiteral primarySuffix* +commandStart = &('`'|IDENT|literal|'cast'|'addr'|'type'|'var'|'out'| + 'static'|'enum'|'tuple'|'object'|'proc') +primary = simplePrimary (commandStart expr (doBlock extraPostExprBlock*)?)? + / operatorB primary + / routineExpr + / rawTypeDesc + / prefixOperator primary +rawTypeDesc = (tupleType | routineType | 'enum' | 'object' | + ('var' | 'out' | 'ref' | 'ptr' | 'distinct') typeDesc?) + ('not' primary)? +typeDescExpr = (routineType / simpleExpr) ('not' primary)? +typeDesc = rawTypeDesc / typeDescExpr +typeDefValue = ((tupleDecl | enumDecl | objectDecl | conceptDecl | + ('ref' | 'ptr' | 'distinct') (tupleDecl | objectDecl)) + / (simpleExpr (exprEqExpr ^+ comma postExprBlocks?)?)) + ('not' primary)? +extraPostExprBlock = ( IND{=} doBlock + | IND{=} 'of' exprList ':' stmt + | IND{=} 'elif' expr ':' stmt + | IND{=} 'except' optionalExprList ':' stmt + | IND{=} 'finally' ':' stmt + | IND{=} 'else' ':' stmt ) +postExprBlocks = (doBlock / ':' (extraPostExprBlock / stmt)) extraPostExprBlock* +exprStmt = simpleExpr postExprBlocks? + / simplePrimary (exprEqExpr ^+ comma) postExprBlocks? + / simpleExpr '=' optInd (expr postExprBlocks?) importStmt = 'import' optInd expr ((comma expr)* / 'except' optInd (expr ^+ comma)) @@ -125,9 +140,9 @@ condStmt = expr colcom stmt COMMENT? (IND{=} 'else' colcom stmt)? ifStmt = 'if' condStmt whenStmt = 'when' condStmt -condExpr = expr colcom expr optInd - ('elif' expr colcom expr optInd)* - 'else' colcom expr +condExpr = expr colcom stmt optInd + ('elif' expr colcom stmt optInd)* + 'else' colcom stmt ifExpr = 'if' condExpr whenExpr = 'when' condExpr whileStmt = 'while' expr colcom stmt @@ -139,10 +154,10 @@ caseStmt = 'case' expr ':'? COMMENT? (IND{>} ofBranches DED | IND{=} ofBranches) tryStmt = 'try' colcom stmt &(IND{=}? 'except'|'finally') - (IND{=}? 'except' exprList colcom stmt)* + (IND{=}? 'except' optionalExprList colcom stmt)* (IND{=}? 'finally' colcom stmt)? tryExpr = 'try' colcom stmt &(optInd 'except'|'finally') - (optInd 'except' exprList colcom stmt)* + (optInd 'except' optionalExprList colcom stmt)* (optInd 'finally' colcom stmt)? blockStmt = 'block' symbol? colcom stmt blockExpr = 'block' symbol? colcom stmt @@ -172,12 +187,13 @@ objectCase = 'case' declColonEquals ':'? COMMENT? objectPart = IND{>} objectPart^+IND{=} DED / objectWhen / objectCase / 'nil' / 'discard' / declColonEquals objectDecl = 'object' ('of' typeDesc)? COMMENT? objectPart -conceptParam = ('var' | 'out')? symbol +conceptParam = ('var' | 'out' | 'ptr' | 'ref' | 'static' | 'type')? symbol conceptDecl = 'concept' conceptParam ^* ',' (pragma)? ('of' typeDesc ^* ',')? &IND{>} stmt -typeDef = identVisDot genericParamList? pragma '=' optInd typeDefAux +typeDef = identVisDot genericParamList? pragma '=' optInd typeDefValue indAndComment? -varTuple = '(' optInd identWithPragma ^+ comma optPar ')' '=' optInd expr +varTupleLhs = '(' optInd (identWithPragma / varTupleLhs) ^+ comma optPar ')' (':' optInd typeDescExpr)? +varTuple = varTupleLhs '=' optInd expr colonBody = colcom stmt postExprBlocks? variable = (varTuple / identColonEquals) colonBody? indAndComment constant = (varTuple / identWithPragma) (colon typeDesc)? '=' optInd expr indAndComment diff --git a/doc/intern.md b/doc/intern.md index e1861ea7d..6b16bc71f 100644 --- a/doc/intern.md +++ b/doc/intern.md @@ -107,6 +107,10 @@ You can also bisect using custom options to build the compiler, for example if you don't need a debug version of the compiler (which runs slower), you can replace `./koch temp`:cmd: by explicit compilation command, see [Bootstrapping the compiler]. +See also: + +- Crossplatform C/Cpp/Valgrind/JS Bisect in GitHub: https://github.com/juancarlospaco/nimrun-action#examples + Building an instrumented compiler --------------------------------- diff --git a/doc/lib.md b/doc/lib.md index 9e79ff169..1507bbaac 100644 --- a/doc/lib.md +++ b/doc/lib.md @@ -47,17 +47,17 @@ Core Provides a series of low-level methods for bit manipulation. * [compilesettings](compilesettings.html) - This module allows querying the compiler about diverse configuration settings. + Querying the compiler about diverse configuration settings from code. * [cpuinfo](cpuinfo.html) - This module implements procs to determine the number of CPUs / cores. + Procs to determine the number of CPUs / cores. * [effecttraits](effecttraits.html) - This module provides access to the inferred .raises effects + Access to the inferred .raises effects for Nim's macro system. * [endians](endians.html) - This module contains helpers that deal with different byte orders. + Helpers that deal with different byte orders. * [locks](locks.html) Locks and condition variables for Nim. @@ -75,10 +75,10 @@ Core Provides (unsafe) access to Nim's run-time type information. * [typetraits](typetraits.html) - This module defines compile-time reflection procs for working with types. + Compile-time reflection procs for working with types. * [volatile](volatile.html) - This module contains code for generating volatile loads and stores, + Code for generating volatile loads and stores, which are useful in embedded and systems programming. @@ -86,24 +86,24 @@ Algorithms ---------- * [algorithm](algorithm.html) - This module implements some common generic algorithms like sort or binary search. + Some common generic algorithms like sort or binary search. * [enumutils](enumutils.html) - This module adds functionality for the built-in `enum` type. + Additional functionality for the built-in `enum` type. * [sequtils](sequtils.html) - This module implements operations for the built-in `seq` type + Operations for the built-in `seq` type which were inspired by functional programming languages. * [setutils](setutils.html) - This module adds functionality for the built-in `set` type. + Additional functionality for the built-in `set` type. Collections ----------- * [critbits](critbits.html) - This module implements a *crit bit tree* which is an efficient + A *crit bit tree* which is an efficient container for a sorted set of strings, or a sorted mapping of strings. * [deques](deques.html) @@ -127,7 +127,7 @@ Collections Efficient implementation of a set of ordinals as a sparse bit set. * [ropes](ropes.html) - This module contains support for a *rope* data type. + A *rope* data type. Ropes can represent very long strings efficiently; in particular, concatenation is done in O(1) instead of O(n). @@ -150,7 +150,7 @@ String handling Utilities for `cstring` handling. * [editdistance](editdistance.html) - This module contains an algorithm to compute the edit distance between two + An algorithm to compute the edit distance between two Unicode strings. * [encodings](encodings.html) @@ -158,35 +158,37 @@ String handling the `iconv` library, on Windows the Windows API. * [formatfloat](formatfloat.html) - This module implements formatting floats as strings. + Formatting floats as strings. * [objectdollar](objectdollar.html) - This module implements a generic `$` operator to convert objects to strings. + A generic `$` operator to convert objects to strings. * [punycode](punycode.html) Implements a representation of Unicode with the limited ASCII character subset. * [strbasics](strbasics.html) - This module provides some high performance string operations. + Some high performance string operations. * [strformat](strformat.html) Macro based standard string interpolation/formatting. Inspired by - Python's f-strings. + Python's f-strings.\ + **Note:** if you need templating, consider using Nim + [Source Code Filters (SCF)](filters.html). * [strmisc](strmisc.html) - This module contains uncommon string handling operations that do not - fit with the commonly used operations in strutils. + Uncommon string handling operations that do not + fit with the commonly used operations in [strutils](strutils.html). * [strscans](strscans.html) - This module contains a `scanf` macro for convenient parsing of mini languages. + A `scanf` macro for convenient parsing of mini languages. * [strutils](strutils.html) - This module contains common string handling operations like changing + Common string handling operations like changing case of a string, splitting a string into substrings, searching for substrings, replacing substrings. * [unicode](unicode.html) - This module provides support to handle the Unicode UTF-8 encoding. + Support for handling the Unicode UTF-8 encoding. * [unidecode](unidecode.html) It provides a single proc that does Unicode to ASCII transliterations. @@ -196,7 +198,7 @@ String handling Nim support for C/C++'s wide strings. * [wordwrap](wordwrap.html) - This module contains an algorithm to wordwrap a Unicode string. + An algorithm for word-wrapping Unicode strings. Time handling @@ -213,17 +215,16 @@ Generic Operating System Services --------------------------------- * [appdirs](appdirs.html) - This module implements helpers for determining special directories used by apps. + Helpers for determining special directories used by apps. * [cmdline](cmdline.html) - This module contains system facilities for reading command - line parameters. + System facilities for reading command line parameters. * [dirs](dirs.html) - This module implements directory handling. + Directory handling. * [distros](distros.html) - This module implements the basics for OS distribution ("distro") detection + Basics for OS distribution ("distro") detection and the OS's native package manager. Its primary purpose is to produce output for Nimble packages, but it also contains the widely used **Distribution** enum @@ -231,19 +232,19 @@ Generic Operating System Services See [packaging](packaging.html) for hints on distributing Nim using OS packages. * [dynlib](dynlib.html) - This module implements the ability to access symbols from shared libraries. + Accessing symbols from shared libraries. * [envvars](envvars.html) - This module implements environment variable handling. + Environment variable handling. * [exitprocs](exitprocs.html) - This module allows adding hooks to program exit. + Adding hooks to program exit. * [files](files.html) - This module implements file handling. + File handling. * [memfiles](memfiles.html) - This module provides support for memory-mapped files (Posix's `mmap`) + Support for memory-mapped files (Posix's `mmap`) on the different operating systems. * [os](os.html) @@ -252,52 +253,50 @@ Generic Operating System Services commands, etc. * [oserrors](oserrors.html) - This module implements OS error reporting. + OS error reporting. * [osproc](osproc.html) Module for process communication beyond `os.execShellCmd`. * [paths](paths.html) - This module implements path handling. + Path handling. * [reservedmem](reservedmem.html) - This module provides utilities for reserving portions of the + Utilities for reserving portions of the address space of a program without consuming physical memory. * [streams](streams.html) - This module provides a stream interface and two implementations thereof: + A stream interface and two implementations thereof: the `FileStream` and the `StringStream` which implement the stream interface for Nim file objects (`File`) and strings. Other modules may provide other implementations for this standard stream interface. * [symlinks](symlinks.html) - This module implements symlink handling. + Symlink handling. * [syncio](syncio.html) - This module implements various synchronized I/O operations. + Various synchronized I/O operations. * [terminal](terminal.html) - This module contains a few procedures to control the *terminal* - (also called *console*). The implementation simply uses ANSI escape - sequences and does not depend on any other module. + A module to control the terminal output (also called *console*). * [tempfiles](tempfiles.html) - This module provides some utils to generate temporary path names and - create temporary files and directories. + Some utilities for generating temporary path names and + creating temporary files and directories. Math libraries -------------- * [complex](complex.html) - This module implements complex numbers and relevant mathematical operations. + Complex numbers and relevant mathematical operations. * [fenv](fenv.html) Floating-point environment. Handling of floating-point rounding and exceptions (overflow, zero-divide, etc.). * [lenientops](lenientops.html) - Provides binary operators for mixed integer/float expressions for convenience. + Binary operators for mixed integer/float expressions for convenience. * [math](math.html) Mathematical operations like cosine, square root. @@ -306,7 +305,7 @@ Math libraries Fast and tiny random number generator. * [rationals](rationals.html) - This module implements rational numbers and relevant mathematical operations. + Rational numbers and relevant mathematical operations. * [stats](stats.html) Statistical analysis. @@ -322,71 +321,69 @@ Internet Protocols and Support Exports `asyncmacro` and `asyncfutures` for native backends, and `asyncjs` on the JS backend. * [asyncdispatch](asyncdispatch.html) - This module implements an asynchronous dispatcher for IO operations. + An asynchronous dispatcher for IO operations. * [asyncfile](asyncfile.html) - This module implements asynchronous file reading and writing using - `asyncdispatch`. + An asynchronous file reading and writing using `asyncdispatch`. * [asyncftpclient](asyncftpclient.html) - This module implements an asynchronous FTP client using the `asyncnet` - module. + An asynchronous FTP client using the `asyncnet` module. * [asynchttpserver](asynchttpserver.html) - This module implements an asynchronous HTTP server using the `asyncnet` - module. + An asynchronous HTTP server using the `asyncnet` module. * [asyncmacro](asyncmacro.html) - Implements the `async` and `multisync` macros for `asyncdispatch`. + `async` and `multisync` macros for `asyncdispatch`. * [asyncnet](asyncnet.html) - This module implements asynchronous sockets based on the `asyncdispatch` - module. + Asynchronous sockets based on the `asyncdispatch` module. * [asyncstreams](asyncstreams.html) - This module provides `FutureStream` - a future that acts as a queue. + `FutureStream` - a future that acts as a queue. * [cgi](cgi.html) - This module implements helpers for CGI applications. + Helpers for CGI applications. * [cookies](cookies.html) - This module contains helper procs for parsing and generating cookies. + Helper procs for parsing and generating cookies. * [httpclient](httpclient.html) - This module implements a simple HTTP client which supports both synchronous + A simple HTTP client with support for both synchronous and asynchronous retrieval of web pages. * [mimetypes](mimetypes.html) - This module implements a mimetypes database. + A mimetypes database. * [nativesockets](nativesockets.html) - This module implements a low-level sockets API. + A low-level sockets API. * [net](net.html) - This module implements a high-level sockets API. It replaces the - `sockets` module. + A high-level sockets API. * [selectors](selectors.html) - This module implements a selector API with backends specific to each OS. - Currently, epoll on Linux and select on other operating systems. + A selector API with backends specific to each OS. + Supported OS primitives: `epoll`, `kqueue`, `poll`, and `select` on Windows. + +* [smtp](smtp.html) + A simple SMTP client with support for both synchronous and asynchronous operation. * [socketstreams](socketstreams.html) - This module provides an implementation of the streams interface for sockets. + An implementation of the streams interface for sockets. * [uri](uri.html) - This module provides functions for working with URIs. + Functions for working with URIs and URLs. Threading --------- * [isolation](isolation.html) - This module implements the `Isolated[T]` type for + The `Isolated[T]` type for safe construction of isolated subgraphs that can be passed efficiently to different channels and threads. * [tasks](tasks.html) - This module provides basic primitives for creating parallel programs. + Basic primitives for creating parallel programs. * [threadpool](threadpool.html) Implements Nim's [spawn](manual_experimental.html#parallel-amp-spawn). @@ -399,13 +396,13 @@ Parsers ------- * [htmlparser](htmlparser.html) - This module parses an HTML document and creates its XML tree representation. + HTML document parser that creates a XML tree representation. * [json](json.html) High-performance JSON parser. * [lexbase](lexbase.html) - This is a low-level module that implements an extremely efficient buffering + A low-level module that implements an extremely efficient buffering scheme for lexers and parsers. This is used by the diverse parsing modules. * [parsecfg](parsecfg.html) @@ -419,7 +416,7 @@ Parsers The `parsecsv` module implements a simple high-performance CSV parser. * [parsejson](parsejson.html) - This module implements a JSON parser. It is used and exported by the [json](json.html) module, but can also be used in its own right. + A JSON parser. It is used and exported by the [json](json.html) module, but can also be used in its own right. * [parseopt](parseopt.html) The `parseopt` module implements a command line option parser. @@ -428,7 +425,7 @@ Parsers The `parsesql` module implements a simple high-performance SQL parser. * [parseutils](parseutils.html) - This module contains helpers for parsing tokens, numbers, identifiers, etc. + Helpers for parsing tokens, numbers, identifiers, etc. * [parsexml](parsexml.html) The `parsexml` module implements a simple high performance XML/HTML parser. @@ -437,7 +434,7 @@ Parsers web can be parsed with it. * [pegs](pegs.html) - This module contains procedures and operators for handling PEGs. + Procedures and operators for handling PEGs. Docutils @@ -449,14 +446,14 @@ Docutils The interface supports one language nested in another. * [packages/docutils/rst](rst.html) - This module implements a reStructuredText parser. A large subset + A reStructuredText parser. A large subset is implemented. Some features of the markdown wiki syntax are also supported. * [packages/docutils/rstast](rstast.html) - This module implements an AST for the reStructuredText parser. + An AST for the reStructuredText parser. * [packages/docutils/rstgen](rstgen.html) - This module implements a generator of HTML/Latex from reStructuredText. + A generator of HTML/Latex from reStructuredText. XML Processing @@ -467,17 +464,17 @@ XML Processing contains a macro for XML/HTML code generation. * [xmlparser](xmlparser.html) - This module parses an XML document and creates its XML tree representation. + XML document parser that creates a XML tree representation. Generators ---------- * [genasts](genasts.html) - This module implements AST generation using captured variables for macros. + AST generation using captured variables for macros. * [htmlgen](htmlgen.html) - This module implements a simple XML and HTML code + A simple XML and HTML code generator. Each commonly used HTML tag has a corresponding macro that generates a string with its HTML representation. @@ -486,14 +483,13 @@ Hashing ------- * [base64](base64.html) - This module implements a Base64 encoder and decoder. + A Base64 encoder and decoder. * [hashes](hashes.html) - This module implements efficient computations of hash values for diverse - Nim types. + Efficient computations of hash values for diverse Nim types. * [md5](md5.html) - This module implements the MD5 checksum algorithm. + The MD5 checksum algorithm. * [oids](oids.html) An OID is a global ID that consists of a timestamp, @@ -501,14 +497,14 @@ Hashing produce a globally distributed unique ID. * [sha1](sha1.html) - This module implements the SHA-1 checksum algorithm. + The SHA-1 checksum algorithm. Serialization ------------- * [jsonutils](jsonutils.html) - This module implements a hookable (de)serialization for arbitrary types + Hookable (de)serialization for arbitrary types using JSON. * [marshal](marshal.html) @@ -520,35 +516,35 @@ Miscellaneous ------------- * [assertions](assertions.html) - This module implements assertion handling. + Assertion handling. * [browsers](browsers.html) - This module implements procs for opening URLs with the user's default + Procs for opening URLs with the user's default browser. * [colors](colors.html) - This module implements color handling for Nim. + Color handling. * [coro](coro.html) - This module implements experimental coroutines in Nim. + Experimental coroutines in Nim. * [decls](decls.html) - This module implements syntax sugar for some declarations. + Syntax sugar for some declarations. * [enumerate](enumerate.html) - This module implements `enumerate` syntactic sugar based on Nim's macro system. + `enumerate` syntactic sugar based on Nim's macro system. * [importutils](importutils.html) Utilities related to import and symbol resolution. * [logging](logging.html) - This module implements a simple logger. + A simple logger. * [segfaults](segfaults.html) Turns access violations or segfaults into a `NilAccessDefect` exception. * [sugar](sugar.html) - This module implements nice syntactic sugar based on Nim's macro system. + Nice syntactic sugar based on Nim's macro system. * [unittest](unittest.html) Implements a Unit testing DSL. @@ -557,10 +553,10 @@ Miscellaneous Decode variable-length integers that are compatible with SQLite. * [with](with.html) - This module implements the `with` macro for easy function chaining. + The `with` macro for easy function chaining. * [wrapnils](wrapnils.html) - This module allows evaluating expressions safely against nil dereferences. + Allows evaluating expressions safely against nil dereferences. Modules for the JavaScript backend @@ -582,7 +578,7 @@ Modules for the JavaScript backend The wrapper of core JavaScript functions. For most purposes, you should be using the `math`, `json`, and `times` stdlib modules instead of this module. -* [jsfetch](jshttp.html) +* [jsfetch](jsfetch.html) Wrapper for `fetch`. * [jsffi](jsffi.html) @@ -599,9 +595,13 @@ Regular expressions ------------------- * [re](re.html) - This module contains procedures and operators for handling regular + Procedures and operators for handling regular expressions. The current implementation uses PCRE. +* [nre](nre.html) + + Many help functions for handling regular expressions. + The current implementation uses PCRE. Database support ---------------- @@ -627,7 +627,7 @@ Generic Operating System Services --------------------------------- * [rdstdin](rdstdin.html) - This module contains code for reading from stdin. + Code for reading user input from stdin. Wrappers @@ -641,7 +641,7 @@ Windows-specific ---------------- * [winlean](winlean.html) - Contains a wrapper for a small subset of the Win32 API. + Wrapper for a small subset of the Win32 API. * [registry](registry.html) Windows registry support. @@ -650,7 +650,7 @@ UNIX specific ------------- * [posix](posix.html) - Contains a wrapper for the POSIX standard. + Wrapper for the POSIX standard. * [posix_utils](posix_utils.html) Contains helpers for the POSIX standard or specialized for Linux and BSDs. @@ -666,13 +666,13 @@ Database support ---------------- * [mysql](mysql.html) - Contains a wrapper for the mySQL API. + Wrapper for the mySQL API. * [odbcsql](odbcsql.html) interface to the ODBC driver. * [postgres](postgres.html) - Contains a wrapper for the PostgreSQL API. + Wrapper for the PostgreSQL API. * [sqlite3](sqlite3.html) - Contains a wrapper for the SQLite 3 API. + Wrapper for the SQLite 3 API. Network Programming and Internet Protocols diff --git a/doc/manual.md b/doc/manual.md index 378aaed90..5c36a0a7b 100644 --- a/doc/manual.md +++ b/doc/manual.md @@ -267,7 +267,7 @@ You can also use the [discard statement](#statements-and-expressions-discard-sta ``` This was how multiline comments were done before version 0.13.0, -and it is used to provide specifications to [testament](testament.html#writing-unitests) test framework. +and it is used to provide specifications to [testament](testament.html#writing-unit-tests) test framework. Identifiers & Keywords @@ -655,7 +655,7 @@ string containing the literal. The callable identifier needs to be declared with a special ``'`` prefix: ```nim - import strutils + import std/strutils type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble" proc `'u4`(n: string): u4 = # The leading ' is required. @@ -670,7 +670,7 @@ corresponds to this transformation. The transformation naturally handles the case that additional parameters are passed to the callee: ```nim - import strutils + import std/strutils type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble" proc `'u4`(n: string; moreData: int): u4 = result = (parseInt(n) and 0x0F).u4 @@ -1495,7 +1495,8 @@ it can be modified: ```nim var x = "123456" - var s: cstring = x + prepareMutation(x) # call `prepareMutation` before modifying the strings + var s: cstring = cstring(x) s[0] = 'u' # This is ok ``` @@ -2618,13 +2619,20 @@ An expression `b` can be assigned to an expression `a` iff `a` is an Overload resolution =================== -In a call `p(args)` the routine `p` that matches best is selected. If -multiple routines match equally well, the ambiguity is reported during -semantic analysis. +In a call `p(args)` where `p` may refer to more than one +candidate, it is said to be a symbol choice. Overload resolution will attempt to +find the best candidate, thus transforming the symbol choice into a resolved symbol. +The routine `p` that matches best is selected following a series of trials explained below. +In order: Catagory matching, Hierarchical Order Comparison, and finally, Complexity Analysis. + +If multiple candidates match equally well after all trials have been tested, the ambiguity +is reported during semantic analysis. -Every arg in args needs to match. There are multiple different categories how an -argument can match. Let `f` be the formal parameter's type and `a` the type -of the argument. +First Trial: Catagory matching +-------------------------------- + +Every arg in `args` needs to match and there are multiple different categories of matches. +Let `f` be the formal parameter's type and `a` the type of the argument. 1. Exact match: `a` and `f` are of the same type. 2. Literal match: `a` is an integer literal of value `v` @@ -2642,10 +2650,16 @@ of the argument. 6. Conversion match: `a` is convertible to `f`, possibly via a user defined `converter`. -These matching categories have a priority: An exact match is better than a -literal match and that is better than a generic match etc. In the following, -`count(p, m)` counts the number of matches of the matching category `m` -for the routine `p`. +Each operand may fall into one of the categories above; the operand's +highest priority category. The list above is in order or priority. +If a candidate has more priority matches than all other candidates, it is selected as the +resolved symbol. + +For example, if a candidate with one exact match is compared to a candidate with multiple +generic matches and zero exact matches, the candidate with an exact match will win. + +Below is a pseudocode interpretation of category matching, `count(p, m)` counts the number +of matches of the matching category `m` for the routine `p`. A routine `p` matches better than a routine `q` if the following algorithm returns true: @@ -2662,8 +2676,51 @@ algorithm returns true: return "ambiguous" ``` +Second Trial: Hierarchical Order Comparison +---------------------------------------------- + +The hierarchical order of a type is analogous to its relative specificity. Consider the type defined: + +```nim +type A[T] = object +``` + +Matching formals for this type include `T`, `object`, `A`, `A[...]` and `A[C]` where `C` is a concrete type, `A[...]` +is a generic typeclass composition and `T` is an unconstrained generic type variable. This list is in order of +specificity with respect to `A` as each subsequent category narrows the set of types that are members of their match set. -Some examples: +In this trail, the formal parameters of candidates are compared in order (1st parameter, 2nd parameter, etc.) to search for +a candidate that has an unrivaled specificity. If such a formal parameter is found, the candidate it belongs to is chosen +as the resolved symbol. + +Third Trial: Complexity Analysis +---------------------------------- + +A slight clarification: While category matching digests all the formal parameters of a candidate at once (order doesn't matter), +specificity comparison and complexity analysis operate on each formal parameter at a time. The following +is the final trial to disambiguate a symbol choice when a pair of formal parameters have the same hierarchical order. + +The complexity of a type is essentially its number of modifiers and depth of shape. The definition with the *highest* +complexity wins. Consider the following types: + +```nim +type + A[T] = object + B[T, H] = object +``` + +Note: The below examples are not exhaustive. + +We shall say that: + +1. `A[T]` has a higher complexity than `A` +2. `var A[T]` has a higher complexity than `A[T]` +3. `A[A[T]]` has a higher complexity than `A[T]` +4. `B[T, H]` has a higher complexity than `A[T]` (`A` and `B` are not compatible here, but convoluted versions of this exist) +5. `B[ptr T, H]` has a higher complexity than `B[T, H]` + +Some Examples +--------------- ```nim proc takesInt(x: int) = echo "int" @@ -2680,7 +2737,6 @@ Some examples: ``` -If this algorithm returns "ambiguous" further disambiguation is performed: If the argument `a` matches both the parameter type `f` of `p` and `g` of `q` via a subtyping relation, the inheritance depth is taken into account: @@ -2722,6 +2778,23 @@ matches) is preferred: gen(ri) # "ref T" ``` +Type variables match +---------------------- + +When overload resolution is considering candidates, the type variable's definition +is not overlooked as it is used to define the formal parameter's type via variable substitution. + +For example: +```nim +type A +proc p[T: A](param: T) +proc p[T: object](param: T) +``` + +These signatures are not ambiguous for a concrete type of `A` even though the formal parameters match ("T" == "T"). +Instead `T` is treated as a variable in that (`T` ?= `T`) depending on the bound type of `T` at the time of +overload resolution. + Overloading based on 'var T' -------------------------------------- @@ -2978,9 +3051,9 @@ ref or pointer type nil procedural type nil sequence `@[]` string `""` -`tuple[x: A, y: B, ...]` (default(A), default(B), ...) +`tuple[x: A, y: B, ...]` (zeroDefault(A), zeroDefault(B), ...) (analogous for objects) -`array[0..., T]` `[default(T), ...]` +`array[0..., T]` `[zeroDefault(T), ...]` `range[T]` default(T); this may be out of the valid range T = enum `cast[T](0)`; this may be an invalid value ============================ ============================================== @@ -3076,12 +3149,25 @@ when they are declared. The only exception to this is if the `{.importc.}` pragma (or any of the other `importX` pragmas) is applied, in this case the value is expected to come from native code, typically a C/C++ `const`. +Special identifier `_` (underscore) +----------------------------------- + +The identifier `_` has a special meaning in declarations. +Any definition with the name `_` will not be added to scope, meaning the +definition is evaluated, but cannot be used. As a result the name `_` can be +indefinitely redefined. + + ```nim + let _ = 123 + echo _ # error + let _ = 456 # compiles + ``` Tuple unpacking --------------- -In a `var` or `let` statement tuple unpacking can be performed. The special -identifier `_` can be used to ignore some parts of the tuple: +In a `var`, `let` or `const` statement tuple unpacking can be performed. +The special identifier `_` can be used to ignore some parts of the tuple: ```nim proc returnsTuple(): (int, int, int) = (4, 2, 3) @@ -3089,6 +3175,35 @@ identifier `_` can be used to ignore some parts of the tuple: let (x, _, z) = returnsTuple() ``` +This is treated as syntax sugar for roughly the following: + + ```nim + let + tmpTuple = returnsTuple() + x = tmpTuple[0] + z = tmpTuple[2] + ``` + +For `var` or `let` statements, if the value expression is a tuple literal, +each expression is directly expanded into an assignment without the use of +a temporary variable. + + ```nim + let (x, y, z) = (1, 2, 3) + # becomes + let + x = 1 + y = 2 + z = 3 + ``` + +Tuple unpacking can also be nested: + + ```nim + proc returnsNestedTuple(): (int, (int, int), int, int) = (4, (5, 7), 2, 3) + + let (x, (_, y), _, z) = returnsNestedTuple() + ``` Const section @@ -3105,6 +3220,15 @@ A const section declares constants whose values are constant expressions: Once declared, a constant's symbol can be used as a constant expression. +The value part of a constant declaration opens a new scope for each constant, +so no symbols declared in the constant value are accessible outside of it. + + ```nim + const foo = (var a = 1; a) + const bar = a # error + let baz = a # error + ``` + See [Constants and Constant Expressions] for details. Static statement/expression @@ -3574,8 +3698,8 @@ Example: var y = if x > 8: 9 else: 10 ``` -An if expression always results in a value, so the `else` part is -required. `Elif` parts are also allowed. +An `if` expression always results in a value, so the `else` part is +required. `elif` parts are also allowed. When expression --------------- @@ -3673,9 +3797,6 @@ type conversions to unsigned integers and between unsigned integers. The rationale for this is mostly better interoperability with the C Programming language when algorithms are ported from C to Nim. -Exception: Values that are converted to an unsigned type at compile time -are checked so that code like `byte(-1)` does not compile. - **Note**: Historically the operations were unchecked and the conversions were sometimes checked but starting with the revision 1.0.4 of this document and the language implementation the @@ -3707,6 +3828,9 @@ bit pattern of the data being cast (aside from that the size of the target type may differ from the source type). Casting resembles *type punning* in other languages or C++'s `reinterpret_cast`:cpp: and `bit_cast`:cpp: features. +If the size of the target type is larger than the size of the source type, +the remaining memory is zeroed. + The addr operator ----------------- The `addr` operator returns the address of an l-value. If the type of the @@ -3774,15 +3898,6 @@ every time the function is called. proc foo(a: int, b: int = 47): int ``` -Just as the comma propagates the types from right to left until the -first parameter or until a semicolon is hit, it also propagates the -default value starting from the parameter declared with it. - - ```nim - # Both a and b are optional with 47 as their default values. - proc foo(a, b: int = 47): int - ``` - Parameters can be declared mutable and so allow the proc to modify those arguments, by using the type modifier `var`. @@ -4122,7 +4237,7 @@ the operator is in scope (including if it is private). ``` Type bound operators are: -`=destroy`, `=copy`, `=sink`, `=trace`, `=deepcopy`. +`=destroy`, `=copy`, `=sink`, `=trace`, `=deepcopy`, `=wasMoved`, `=dup`. These operations can be *overridden* instead of *overloaded*. This means that the implementation is automatically lifted to structured types. For instance, @@ -4348,7 +4463,42 @@ as an example: Overloading of the subscript operator ------------------------------------- -The `[]` subscript operator for arrays/openarrays/sequences can be overloaded. +The `[]` subscript operator for arrays/openarrays/sequences can be overloaded +for any type (with some exceptions) by defining a routine with the name `[]`. + + ```nim + type Foo = object + data: seq[int] + + proc `[]`(foo: Foo, i: int): int = + result = foo.data[i] + + let foo = Foo(data: @[1, 2, 3]) + echo foo[1] # 2 + ``` + +Assignment to subscripts can also be overloaded by naming a routine `[]=`, +which has precedence over assigning to the result of `[]`. + + ```nim + type Foo = object + data: seq[int] + + proc `[]`(foo: Foo, i: int): int = + result = foo.data[i] + proc `[]=`(foo: var Foo, i: int, val: int) = + foo.data[i] = val + + var foo = Foo(data: @[1, 2, 3]) + echo foo[1] # 2 + foo[1] = 5 + echo foo.data # @[1, 5, 3] + echo foo[1] # 5 + ``` + +Overloads of the subscript operator cannot be applied to routine or type +symbols themselves, as this conflicts with the syntax for instantiating +generic parameters, i.e. `foo[int](1, 2, 3)` or `Foo[int]`. Methods @@ -4564,7 +4714,6 @@ Closure iterators and inline iterators have some restrictions: (but rarely useful) and ends the iteration. 3. Inline iterators cannot be recursive. 4. Neither inline nor closure iterators have the special `result` variable. -5. Closure iterators are not supported by the JS backend. Iterators that are neither marked `{.closure.}` nor `{.inline.}` explicitly default to being inline, but this may change in future versions of the @@ -4773,8 +4922,8 @@ Example: echo "overflow!" except ValueError, IOError: echo "catch multiple exceptions!" - except: - echo "Unknown exception!" + except CatchableError: + echo "Catchable exception!" finally: close(f) ``` @@ -4786,9 +4935,6 @@ listed in an `except` clause, the corresponding statements are executed. The statements following the `except` clauses are called `exception handlers`:idx:. -The empty `except`:idx: clause is executed if there is an exception that is -not listed otherwise. It is similar to an `else` clause in `if` statements. - If there is a `finally`:idx: clause, it is always executed after the exception handlers. @@ -4806,11 +4952,11 @@ Try can also be used as an expression; the type of the `try` branch then needs to fit the types of `except` branches, but the type of the `finally` branch always has to be `void`: - ```nim + ```nim test from std/strutils import parseInt let x = try: parseInt("133a") - except: -1 + except ValueError: -1 finally: echo "hi" ``` @@ -4818,8 +4964,9 @@ branch always has to be `void`: To prevent confusing code there is a parsing limitation; if the `try` follows a `(` it has to be written as a one liner: - ```nim - let x = (try: parseInt("133a") except: -1) + ```nim test + from std/strutils import parseInt + let x = (try: parseInt("133a") except ValueError: -1) ``` @@ -4867,7 +5014,7 @@ error message from `e`, and for such situations, it is enough to use ```nim try: # ... - except: + except CatchableError: echo getCurrentExceptionMsg() ``` @@ -4982,6 +5129,38 @@ exceptions inherit from `Defect`. Exceptions that indicate any other runtime error that can be caught inherit from `system.CatchableError` (which is a subtype of `Exception`). +``` +Exception +|-- CatchableError +| |-- IOError +| | `-- EOFError +| |-- OSError +| |-- ResourceExhaustedError +| `-- ValueError +| `-- KeyError +`-- Defect + |-- AccessViolationDefect + |-- ArithmeticDefect + | |-- DivByZeroDefect + | `-- OverflowDefect + |-- AssertionDefect + |-- DeadThreadDefect + |-- FieldDefect + |-- FloatingPointDefect + | |-- FloatDivByZeroDefect + | |-- FloatInvalidOpDefect + | |-- FloatOverflowDefect + | |-- FloatUnderflowDefect + | `-- InexactDefect + |-- IndexDefect + |-- NilAccessDefect + |-- ObjectAssignmentDefect + |-- ObjectConversionDefect + |-- OutOfMemoryDefect + |-- RangeDefect + |-- ReraiseDefect + `-- StackOverflowDefect +``` Imported exceptions ------------------- @@ -5055,7 +5234,7 @@ An empty `raises` list (`raises: []`) means that no exception may be raised: try: unsafeCall() result = true - except: + except CatchableError: result = false ``` @@ -5083,7 +5262,7 @@ possibly raised exceptions; the algorithm operates on `p`'s call graph: raise `system.Exception` (the base type of the exception hierarchy) and thus any exception unless `T` has an explicit `raises` list. However, if the call is of the form `f(...)` where `f` is a parameter of - the currently analyzed routine it is ignored that is marked as `.effectsOf: f`. + the currently analyzed routine that is marked as `.effectsOf: f`, it is ignored. The call is optimistically assumed to have no effect. Rule 2 compensates for this case. 2. Every expression `e` of some proc type within a call that is passed to parameter @@ -5153,7 +5332,7 @@ conservative in its effect analysis: ```nim test = "nim c $1" status = 1 {.push warningAsError[Effect]: on.} - import algorithm + import std/algorithm type MyInt = distinct int @@ -5281,6 +5460,8 @@ To override the compiler's side effect analysis a `{.noSideEffect.}` **Side effects are usually inferred. The inference for side effects is analogous to the inference for exception tracking.** +When the compiler cannot infer side effects, as is the case for imported +functions, one can annotate them with the `sideEffect` pragma. GC safety effect ---------------- @@ -5346,6 +5527,7 @@ Generics are Nim's means to parametrize procs, iterators or types with `type parameters`:idx:. Depending on the context, the brackets are used either to introduce type parameters or to instantiate a generic proc, iterator, or type. + The following example shows how a generic binary tree can be modeled: ```nim test = "nim c $1" @@ -5407,6 +5589,49 @@ The following example shows how a generic binary tree can be modeled: The `T` is called a `generic type parameter`:idx: or a `type variable`:idx:. + +Generic Procs +--------------- + +Let's consider the anatomy of a generic `proc` to agree on defined terminology. + +```nim +p[T: t](arg1: f): y +``` + +- `p`: Callee symbol +- `[...]`: Generic parameters +- `T: t`: Generic constraint +- `T`: Type variable +- `[T: t](arg1: f): y`: Formal signature +- `arg1: f`: Formal parameter +- `f`: Formal parameter type +- `y`: Formal return type + +The use of the word "formal" here is to denote the symbols as they are defined by the programmer, +not as they may be at compile time contextually. Since generics may be instantiated and +types bound, we have more than one entity to think about when generics are involved. + +The usage of a generic will resolve the formally defined expression into an instance of that +expression bound to only concrete types. This process is called "instantiation". + +Brackets at the site of a generic's formal definition specify the "constraints" as in: + +```nim +type Foo[T] = object +proc p[H;T: Foo[H]](param: T): H +``` + +A constraint definition may have more than one symbol defined by separating each definition by +a `;`. Notice how `T` is composed of `H` and the return type of `p` is defined as `H`. When this +generic proc is instantiated `H` will be bound to a concrete type, thus making `T` concrete and +the return type of `p` will be bound to the same concrete type used to define `H`. + +Brackets at the site of usage can be used to supply concrete types to instantiate the generic in the same +order that the symbols are defined in the constraint. Alternatively, type bindings may be inferred by the compiler +in some situations, allowing for cleaner code. + + Is operator ----------- @@ -5436,9 +5661,9 @@ type class matches ================== =================================================== `object` any object type `tuple` any tuple type - `enum` any enumeration `proc` any proc type +`iterator` any iterator type `ref` any `ref` type `ptr` any `ptr` type `var` any `var` type @@ -5457,7 +5682,7 @@ more complex type classes: ```nim # create a type class that will match all tuple and object types - type RecordType = tuple or object + type RecordType = (tuple or object) proc printFields[T: RecordType](rec: T) = for key, value in fieldPairs(rec): @@ -5498,6 +5723,17 @@ as `type constraints`:idx: of the generic type parameter: onlyIntOrString("xy", 50) # invalid as 'T' cannot be both at the same time ``` +`proc` and `iterator` type classes also accept a calling convention pragma +to restrict the calling convention of the matching `proc` or `iterator` type. + + ```nim + proc onlyClosure[T: proc {.closure.}](x: T) = discard + + onlyClosure(proc() = echo "hello") # valid + proc foo() {.nimcall.} = discard + onlyClosure(foo) # type mismatch + ``` + Implicit generics ----------------- @@ -5506,7 +5742,7 @@ A type class can be used directly as the parameter's type. ```nim # create a type class that will match all tuple and object types - type RecordType = tuple or object + type RecordType = (tuple or object) proc printFields(rec: RecordType) = for key, value in fieldPairs(rec): @@ -5669,7 +5905,7 @@ at definition and the context at instantiation are considered: echo a == b # works! ``` -In the example, the generic `==` for tuples (as defined in the system module) +In the example, the [generic `==` for tuples](system.html#%3D%3D%2CT%2CT_2) (as defined in the system module) uses the `==` operators of the tuple's components. However, the `==` for the `Index` type is defined *after* the `==` for tuples; yet the example compiles as the instantiation takes the currently defined symbols into account @@ -6021,9 +6257,12 @@ scope is controlled by the `inject`:idx: and `gensym`:idx: pragmas: `gensym`'ed symbols are not exposed but `inject`'ed symbols are. The default for symbols of entity `type`, `var`, `let` and `const` -is `gensym` and for `proc`, `iterator`, `converter`, `template`, -`macro` is `inject`. However, if the name of the entity is passed as a -template parameter, it is an `inject`'ed symbol: +is `gensym`. For `proc`, `iterator`, `converter`, `template`, +`macro`, the default is `inject`, but if a `gensym` symbol with the same name +is defined in the same syntax-level scope, it will be `gensym` by default. +This can be overriden by marking the routine as `inject`. + +If the name of the entity is passed as a template parameter, it is an `inject`'ed symbol: ```nim template withFile(f, fn, mode: untyped, actions: untyped): untyped = @@ -6634,6 +6873,8 @@ even when one version does not export some of these identifiers. The `import` statement is only allowed at the top level. +String literals can be used for import/include statements. +The compiler performs [path substitution](nimc.html#compiler-usage-commandminusline-switches) when used. Include statement ----------------- @@ -6822,29 +7063,80 @@ All identifiers of a module are valid from the point of declaration until the end of the module. Identifiers from indirectly dependent modules are *not* available. The `system`:idx: module is automatically imported in every module. -If a module imports an identifier by two different modules, each occurrence of -the identifier has to be qualified unless it is an overloaded procedure or -iterator in which case the overloading resolution takes place: +If a module imports the same identifier from two different modules, the +identifier is considered ambiguous, which can be resolved in the following ways: + +* Qualifying the identifier as `module.identifier` resolves ambiguity + between modules. (See below for the case that the module name itself + is ambiguous.) +* Calling the identifier as a routine makes overload resolution take place, + which resolves ambiguity in the case that one overload matches stronger + than the others. +* Using the identifier in a context where the compiler can infer the type + of the identifier resolves ambiguity in the case that one definition + matches the type stronger than the others. ```nim # Module A var x*: string + proc foo*(a: string) = + echo "A: ", a ``` ```nim # Module B var x*: int + proc foo*(b: int) = + echo "B: ", b ``` ```nim # Module C import A, B + + foo("abc") # A: abc + foo(123) # B: 123 + let inferred: proc (x: string) = foo + foo("def") # A: def + write(stdout, x) # error: x is ambiguous write(stdout, A.x) # no error: qualifier used + + proc bar(a: int): int = a + 1 + assert bar(x) == x + 1 # no error: only A.x of type int matches var x = 4 write(stdout, x) # not ambiguous: uses the module C's x ``` +Modules can share their name, however, when trying to qualify an identifier with the module name the compiler will fail with ambiguous identifier error. One can qualify the identifier by aliasing the module. + + +```nim +# Module A/C +proc fb* = echo "fizz" +``` + + +```nim +# Module B/C +proc fb* = echo "buzz" +``` + + +```nim +import A/C +import B/C + +C.fb() # Error: ambiguous identifier: 'C' +``` + + +```nim +import A/C as fizz +import B/C + +fizz.fb() # Works +``` Packages @@ -7180,7 +7472,7 @@ echo foo() # 2 template foo: int = 3 ``` -This is mostly intended for macro generated code. +This is mostly intended for macro generated code. compilation option pragmas -------------------------- @@ -7250,7 +7542,7 @@ but are used to override the settings temporarily. Example: template example(): string = "https://nim-lang.org" {.pop.} - {.push deprecated, hint[LineTooLong]: off, used, stackTrace: off.} + {.push deprecated, used, stackTrace: off.} proc sample(): bool = true {.pop.} ``` @@ -7289,14 +7581,14 @@ and before any variable in a module that imports it. Disabling certain messages -------------------------- -Nim generates some warnings and hints ("line too long") that may annoy the +Nim generates some warnings and hints that may annoy the user. A mechanism for disabling certain messages is provided: Each hint and warning message is associated with a symbol. This is the message's identifier, which can be used to enable or disable the message by putting it in brackets following the pragma: ```Nim - {.hint[LineTooLong]: off.} # turn off the hint about too long lines + {.hint[XDeclaredButNotUsed]: off.} # Turn off the hint about declared but not used symbols. ``` This is often better than disabling all warnings at once. @@ -7412,6 +7704,37 @@ generates: ``` +size pragma +----------- +Nim automatically determines the size of an enum. +But when wrapping a C enum type, it needs to be of a specific size. +The `size pragma` allows specifying the size of the enum type. + + ```Nim + type + EventType* {.size: sizeof(uint32).} = enum + QuitEvent, + AppTerminating, + AppLowMemory + + doAssert sizeof(EventType) == sizeof(uint32) + ``` + +The `size pragma` can also specify the size of an `importc` incomplete object type +so that one can get the size of it at compile time even if it was declared without fields. + + ```Nim + type + AtomicFlag* {.importc: "atomic_flag", header: "<stdatomic.h>", size: 1.} = object + + static: + # if AtomicFlag didn't have the size pragma, this code would result in a compile time error. + echo sizeof(AtomicFlag) + ``` + +The `size pragma` accepts only the values 1, 2, 4 or 8. + + Align pragma ------------ @@ -7524,10 +7847,16 @@ Compile pragma The `compile` pragma can be used to compile and link a C/C++ source file with the project: +This pragma can take three forms. The first is a simple file input: ```Nim {.compile: "myfile.cpp".} ``` +The second form is a tuple where the second arg is the output name strutils formatter: + ```Nim + {.compile: ("file.c", "$1.o").} + ``` + **Note**: Nim computes a SHA1 checksum and only recompiles the file if it has changed. One can use the `-f`:option: command-line option to force the recompilation of the file. @@ -7962,8 +8291,8 @@ CodegenDecl pragma ------------------ The `codegenDecl` pragma can be used to directly influence Nim's code -generator. It receives a format string that determines how the variable -or proc is declared in the generated code. +generator. It receives a format string that determines how the variable, +proc or object type is declared in the generated code. For variables, $1 in the format string represents the type of the variable, $2 is the name of the variable, and each appearance of $# represents $1/$2 @@ -7999,6 +8328,29 @@ will generate this code: __interrupt void myinterrupt() ``` +For object types, the $1 represents the name of the object type, $2 is the list of +fields and $3 is the base type. + +```nim + +const strTemplate = """ + struct $1 { + $2 + }; +""" +type Foo {.codegenDecl:strTemplate.} = object + a, b: int +``` + +will generate this code: + + +```c +struct Foo { + NI a; + NI b; +}; +``` `cppNonPod` pragma ------------------ @@ -8049,6 +8401,24 @@ used. To see if a value was provided, `defined(FooBar)` can be used. The syntax `-d:flag`:option: is actually just a shortcut for `-d:flag=true`:option:. +These pragmas also accept an optional string argument for qualified +define names. + + ```nim + const FooBar {.intdefine: "package.FooBar".}: int = 5 + echo FooBar + ``` + + ```cmd + nim c -d:package.FooBar=42 foobar.nim + ``` + +This helps disambiguate define names in different packages. + +See also the [generic `define` pragma](manual_experimental.html#generic-nimdefine-pragma) +for a version of these pragmas that detects the type of the define based on +the constant value. + User-defined pragmas ==================== @@ -8293,8 +8663,7 @@ is available and a literal dollar sign must be written as ``$$``. Bycopy pragma ------------- -The `bycopy` pragma can be applied to an object or tuple type and -instructs the compiler to pass the type by value to procs: +The `bycopy` pragma can be applied to an object or tuple type or a proc param. It instructs the compiler to pass the type by value to procs: ```nim type @@ -8302,14 +8671,73 @@ instructs the compiler to pass the type by value to procs: x, y, z: float ``` -The Nim compiler automatically determines whether a parameter is passed by value or by reference based on the parameter type's size. If a parameter must be passed by value or by reference, (such as when interfacing with a C library) use the bycopy or byref pragmas. +The Nim compiler automatically determines whether a parameter is passed by value or +by reference based on the parameter type's size. If a parameter must be passed by value +or by reference, (such as when interfacing with a C library) use the bycopy or byref pragmas. +Notice params marked as `byref` takes precedence over types marked as `bycopy`. Byref pragma ------------ -The `byref` pragma can be applied to an object or tuple type and instructs -the compiler to pass the type by reference (hidden pointer) to procs. +The `byref` pragma can be applied to an object or tuple type or a proc param. +When applied to a type it instructs the compiler to pass the type by reference +(hidden pointer) to procs. When applied to a param it will take precedence, even +if the the type was marked as `bycopy`. When an `importc` type has a `byref` pragma or +parameters are marked as `byref` in an `importc` proc, these params translate to pointers. +When an `importcpp` type has a `byref` pragma, these params translate to +C++ references `&`. + ```Nim + {.emit: """/*TYPESECTION*/ + typedef struct { + int x; + } CStruct; + """.} + + {.emit: """ + #ifdef __cplusplus + extern "C" + #endif + int takesCStruct(CStruct* x) { + return x->x; + } + """.} + + type + CStruct {.importc, byref.} = object + x: cint + + proc takesCStruct(x: CStruct): cint {.importc.} + ``` + + or + + + ```Nim + type + CStruct {.importc.} = object + x: cint + + proc takesCStruct(x {.byref.}: CStruct): cint {.importc.} + ``` + + ```Nim + {.emit: """/*TYPESECTION*/ + struct CppStruct { + int x; + + int takesCppStruct(CppStruct& y) { + return x + y.x; + } + }; + """.} + + type + CppStruct {.importcpp, byref.} = object + x: cint + + proc takesCppStruct(x, y: CppStruct): cint {.importcpp.} + ``` Varargs pragma -------------- @@ -8319,7 +8747,7 @@ after the last specified parameter. Nim string values will be converted to C strings automatically: ```Nim - proc printf(formatstr: cstring) {.nodecl, varargs.} + proc printf(formatstr: cstring) {.header: "<stdio.h>", varargs.} printf("hallo %s", "world") # "world" will be passed as C string ``` @@ -8424,27 +8852,14 @@ This is only useful if the program is compiled as a dynamic library via the `--app:lib`:option: command-line option. - Threads ======= -To enable thread support the `--threads:on`:option: command-line switch needs to -be used. The [system module](system.html) module then contains several threading primitives. -See the [channels](channels_builtin.html) modules -for the low-level thread API. There are also high-level parallelism constructs -available. See [spawn](manual_experimental.html#parallel-amp-spawn) for +The `--threads:on`:option: command-line switch is enabled by default. The [typedthreads module](typedthreads.html) module then contains several threading primitives. See [spawn](manual_experimental.html#parallel-amp-spawn) for further details. -Nim's memory model for threads is quite different than that of other common -programming languages (C, Pascal, Java): Each thread has its own (garbage -collected) heap, and sharing of memory is restricted to global variables. This -helps to prevent race conditions. GC efficiency is improved quite a lot, -because the GC never has to stop other threads and see what they reference. +The only ways to create a thread is via `spawn` or `createThread`. -The only way to create a thread is via `spawn` or -`createThread`. The invoked proc must not use `var` parameters nor must -any of its parameters contain a `ref` or `closure` type. This enforces -the *no heap sharing restriction*. Thread pragma ------------- @@ -8561,9 +8976,7 @@ model low level lockfree mechanisms: The `locks` pragma takes a list of lock expressions `locks: [a, b, ...]` -in order to support *multi lock* statements. Why these are essential is -explained in the [lock levels](manual_experimental.md#lock-levels) section -of experimental manual. +in order to support *multi lock* statements. ### Protecting general locations diff --git a/doc/manual_experimental.md b/doc/manual_experimental.md index 0611f55a7..da51d59ad 100644 --- a/doc/manual_experimental.md +++ b/doc/manual_experimental.md @@ -26,9 +26,8 @@ oneself. Void type ========= -The `void` type denotes the absence of any type. Parameters of -type `void` are treated as non-existent, `void` as a return type means that -the procedure does not return a value: +The `void` type denotes the absence of any value, i.e. it is the type that contains no values. Consequently, no value can be provided for parameters of +type `void`, and no value can be returned from a function with return type `void`: ```nim proc nothing(x, y: void): void = @@ -65,6 +64,29 @@ However, a `void` type cannot be inferred in generic code: The `void` type is only valid for parameters and return types; other symbols cannot have the type `void`. +Generic `define` pragma +======================= + +Aside the [typed define pragmas for constants](manual.html#implementation-specific-pragmas-compileminustime-define-pragmas), +there is a generic `{.define.}` pragma that interprets the value of the define +based on the type of the constant value. + + ```nim + const foo {.define: "package.foo".} = 123 + const bar {.define: "package.bar".} = false + ``` + + ```cmd + nim c -d:package.foo=456 -d:package.bar foobar.nim + ``` + +The following types are supported: + +* `string` and `cstring` +* Signed and unsigned integer types +* `bool` +* Enums + Top-down type inference ======================= @@ -101,6 +123,87 @@ would not match the type of the variable, and an error would be given. The extent of this varies, but there are some notable special cases. + +Inferred generic parameters +--------------------------- + +In expressions making use of generic procs or templates, the expected +(unbound) types are often able to be inferred based on context. +This feature has to be enabled via `{.experimental: "inferGenericTypes".}` + + ```nim test = "nim c $1" + {.experimental: "inferGenericTypes".} + + import std/options + + var x = newSeq[int](1) + # Do some work on 'x'... + + # Works! + # 'x' is 'seq[int]' so 'newSeq[int]' is implied + x = newSeq(10) + + # Works! + # 'T' of 'none' is bound to the 'T' of 'noneProducer', passing it along. + # Effectively 'none.T = noneProducer.T' + proc noneProducer[T](): Option[T] = none() + let myNone = noneProducer[int]() + + # Also works + # 'myOtherNone' binds its 'T' to 'float' and 'noneProducer' inherits it + # noneProducer.T = myOtherNone.T + let myOtherNone: Option[float] = noneProducer() + + # Works as well + # none.T = myOtherOtherNone.T + let myOtherOtherNone: Option[int] = none() + ``` + +This is achieved by reducing the types on the lhs and rhs until the *lhs* is left with only types such as `T`. +While lhs and rhs are reduced together, this does *not* mean that the *rhs* will also only be left +with a flat type `Z`, it may be of the form `MyType[Z]`. + +After the types have been reduced, the types `T` are bound to the types that are left on the rhs. + +If bindings *cannot be inferred*, compilation will fail and manual specification is required. + +An example for *failing inference* can be found when passing a generic expression +to a function/template call: + + ```nim test = "nim c $1" status = 1 + {.experimental: "inferGenericTypes".} + + proc myProc[T](a, b: T) = discard + + # Fails! Unable to infer that 'T' is supposed to be 'int' + myProc(newSeq[int](), newSeq(1)) + + # Works! Manual specification of 'T' as 'int' necessary + myProc(newSeq[int](), newSeq[int](1)) + ``` + +Combination of generic inference with the `auto` type is also unsupported: + + ```nim test = "nim c $1" status = 1 + {.experimental: "inferGenericTypes".} + + proc produceValue[T]: auto = default(T) + let a: int = produceValue() # 'auto' cannot be inferred here + ``` + +**Note**: The described inference does not permit the creation of overrides based on +the return type of a procedure. It is a mapping mechanism that does not attempt to +perform deeper inference, nor does it modify what is a valid override. + + ```nim test = "nim c $1" status = 1 + # Doesn't affect the following code, it is invalid either way + {.experimental: "inferGenericTypes".} + + proc a: int = 0 + proc a: float = 1.0 # Fails! Invalid code and not recommended + ``` + + Sequence literals ----------------- @@ -430,28 +533,25 @@ Assuming `foo` is a macro or a template, this is roughly equivalent to: ``` -Symbols as template/macro calls -=============================== +Symbols as template/macro calls (alias syntax) +============================================== -Templates and macros that take no arguments can be called as lone symbols, -i.e. without parentheses. This is useful for repeated uses of complex -expressions that cannot conveniently be represented as runtime values. +Templates and macros that have no generic parameters and no required arguments +can be called as lone symbols, i.e. without parentheses. This is useful for +repeated uses of complex expressions that cannot conveniently be represented +as runtime values. ```nim type Foo = object bar: int var foo = Foo(bar: 10) - template bar: untyped = foo.bar + template bar: int = foo.bar assert bar == 10 bar = 15 assert bar == 15 ``` -In the future, this may require more specific information on template or macro -signatures to be used. Specializations for some applications of this may also -be introduced to guarantee consistency and circumvent bugs. - Not nil annotation ================== @@ -460,12 +560,14 @@ Not nil annotation `{.experimental: "notnil".}`. All types for which `nil` is a valid value can be annotated with the -`not nil` annotation to exclude `nil` as a valid value: +`not nil` annotation to exclude `nil` as a valid value. Note that only local +symbols are checked. ```nim {.experimental: "notnil".} type + TObj = object PObject = ref TObj not nil TProc = (proc (x, y: int)) not nil @@ -476,8 +578,11 @@ All types for which `nil` is a valid value can be annotated with the p(nil) # and also this: - var x: PObject - p(x) + proc foo = + var x: PObject + p(x) + + foo() ``` The compiler ensures that every code path initializes variables which contain @@ -519,8 +624,7 @@ Since version 1.4, a stricter definition of "side effect" is available. In addition to the existing rule that a side effect is calling a function with side effects, the following rule is also enforced: -Any mutation to an object does count as a side effect if that object is reachable -via a parameter that is not declared as a `var` parameter. +A store to the heap via a `ref` or `ptr` indirection is not allowed. For example: @@ -540,15 +644,12 @@ For example: it = it.ri func mut(n: Node) = - let m = n # is the statement that connected the mutation to the parameter - m.data = "yeah" # the mutation is here - # Error: 'mut' can have side effects - # an object reachable from 'n' is potentially mutated - ``` - + var it = n + while it != nil: + it.data = "yeah" # forbidden mutation + it = it.ri -The algorithm behind this analysis is described in -the [view types algorithm][Algorithm]. + ``` View types @@ -822,6 +923,7 @@ The concept matches if: a) all expressions within the body can be compiled for the tested type b) all statically evaluable boolean expressions in the body are true +c) all type modifiers specified match their respective definitions The identifiers following the `concept` keyword represent instances of the currently matched type. You can apply any of the standard type modifiers such @@ -1270,7 +1372,7 @@ to be computed dynamically. ```nim {.experimental: "dynamicBindSym".} - import macros + import std/macros macro callOp(opName, arg1, arg2): untyped = result = newCall(bindSym($opName), arg1, arg2) @@ -1859,17 +1961,29 @@ before it is used. Thus the following is valid: In this example every path does set `s` to a value before it is used. + ```nim + {.experimental: "strictDefs".} + + proc test(cond: bool) = + let s: seq[string] + if cond: + s = @["y"] + else: + s = @[] + ``` + +With `experimental: "strictDefs"`, `let` statements are allowed to not have an initial value, but every path should set `s` to a value before it is used. + + `out` parameters ---------------- An `out` parameter is like a `var` parameter but it must be written to before it can be used: ```nim - proc myopen(f: out File; name: string): bool = f = default(File) result = open(f, name) - ``` While it is usually the better style to use the return type in order to return results API and ABI @@ -1877,9 +1991,7 @@ considerations might make this infeasible. Like for `var T` Nim maps `out T` to For example POSIX's `stat` routine can be wrapped as: ```nim - proc stat*(a1: cstring, a2: out Stat): cint {.importc, header: "<sys/stat.h>".} - ``` When the implementation of a routine with output parameters is analysed, the compiler @@ -1887,13 +1999,11 @@ checks that every path before the (implicit or explicit) return does set every o parameter: ```nim - proc p(x: out int; y: out string; cond: bool) = x = 4 if cond: y = "abc" # error: not every path initializes 'y' - ``` @@ -1903,11 +2013,9 @@ Out parameters and exception handling The analysis should take exceptions into account (but currently does not): ```nim - proc p(x: out int; y: out string; cond: bool) = x = canRaise(45) y = "abc" # <-- error: not every path initializes 'y' - ``` Once the implementation takes exceptions into account it is easy enough to @@ -1919,7 +2027,6 @@ Out parameters and inheritance It is not valid to pass an lvalue of a supertype to an `out T` parameter: ```nim - type Superclass = object of RootObj a: int @@ -1932,7 +2039,6 @@ It is not valid to pass an lvalue of a supertype to an `out T` parameter: var v: Subclass init v use v.s # the 's' field was never initialized! - ``` However, in the future this could be allowed and provide a better way to write object @@ -1977,3 +2083,587 @@ The field is within a `case` section of an `object`. **Note**: The implementation of "strict case objects" is experimental but the concept is solid and it is expected that eventually this mode becomes the default in later versions. + + +Quirky routines +=============== + +The default code generation strategy of exceptions under the ARC/ORC model is the so called +`--exceptions:goto` implementation. This implementation inserts a check after every call that +can potentially raise an exception. A typical instruction sequence for this on +for a x86 64 bit machine looks like: + + ``` + cmp DWORD PTR [rbx], 0 + je .L1 + ``` + +This is a memory fetch followed by jump. (An ideal implementation would +use the carry flag and a single instruction like ``jc .L1``.) + +This overhead might not be desired and depending on the semantics of the routine may not be required +either. +So it can be disabled via a `.quirky` annotation: + + ```nim + proc wontRaise(x: int) {.quirky.} = + if x != 0: + # because of `quirky` this will continue even if `write` raised an IO exception: + write x + wontRaise(x-1) + + wontRaise 10 + + ``` + +If the used exception model is not `--exceptions:goto` then the `quirky` pragma has no effect and is +ignored. + +The `quirky` pragma can also be be pushed in order to affect a group of routines and whether +the compiler supports the pragma can be checked with `defined(nimHasQuirky)`: + + ```nim + when defined(nimHasQuirky): + {.push quirky: on.} + + proc doRaise() = raise newException(ValueError, "") + + proc f(): string = "abc" + + proc q(cond: bool) = + if cond: + doRaise() + echo f() + + q(true) + + when defined(nimHasQuirky): + {.pop.} + ``` + +**Warning**: The `quirky` pragma only affects code generation, no check for validity is performed! + + +Threading under ARC/ORC +======================= + +ARC/ORC supports a shared heap out of the box. This means that messages can be sent between +threads without copies. However, without copying the data there is an inherent danger of +data races. Data races are prevented at compile-time if it is enforced that +only **isolated** subgraphs can be sent around. + + +Isolation +--------- + +The standard library module `isolation.nim` provides a generic type `Isolated[T]` that +captures the important notion that nothing else can reference the graph that is wrapped +inside `Isolated[T]`. It is what a channel implementation should use in order to enforce +the freedom of data races: + + ```nim + proc send*[T](c: var Channel[T]; msg: sink Isolated[T]) + proc recv*[T](c: var Channel[T]): T + ## Note: Returns T, not Isolated[T] for convenience. + + proc recvIso*[T](c: var Channel[T]): Isolated[T] + ## remembers the data is Isolated[T]. + ``` + +In order to create an `Isolated` graph one has to use either `isolate` or `unsafeIsolate`. +`unsafeIsolate` is as its name says unsafe and no checking is performed. It should be considered +to be as dangerous as a `cast` operation. + + +Construction must ensure that the invariant holds, namely that the wrapped `T` +is free of external aliases into it. `isolate` ensures this invariant. It is +inspired by Pony's `recover` construct: + + ```nim + func isolate(x: sink T): Isolated[T] {.magic: "Isolate".} + ``` + + +As you can see, this is a new builtin because the check it performs on `x` is non-trivial: + +If `T` does not contain a `ref` or `closure` type, it is isolated. Else the syntactic +structure of `x` is analyzed: + +- Literals like `nil`, `4`, `"abc"` are isolated. +- A local variable or a routine parameter is isolated if either of these conditions is true: + 1. Its type is annotated with the `.sendable` pragma. Note `Isolated[T]` is annotated as + `.sendable`. + 2. Its type contains the potentially dangerous `ref` and `proc {.closure}` types + only in places that are protected via a `.sendable` container. + +- An array constructor `[x...]` is isolated if every element `x` is isolated. +- An object constructor `Obj(x...)` is isolated if every element `x` is isolated. +- An `if` or `case` expression is isolated if all possible values the expression + may return are isolated. +- A type conversion `C(x)` is isolated if `x` is isolated. Analogous for `cast` + expressions. +- A function call `f(x...)` is isolated if `f` is `.noSideEffect` and for every argument `x`: + - `x` is isolated **or** + - `f`'s return type cannot *alias* `x`'s type. This is checked via a form of alias analysis as explained in the next paragraph. + + + +Alias analysis +-------------- + +We start with an important, simple case that must be valid: Sending the result +of `parseJson` to a channel. Since the signature +is `func parseJson(input: string): JsonNode` it is easy to see that JsonNode +can never simply be a view into `input` which is a `string`. + +A different case is the identity function `id`, `send id(myJsonGraph)` must be +invalid because we do not know how many aliases into `myJsonGraph` exist +elsewhere. + +In general type `A` can alias type `T` if: + +- `A` and `T` are the same types. +- `A` is a distinct type derived from `T`. +- `A` is a field inside `T` if `T` is a final object type. +- `T` is an inheritable object type. (An inherited type could always contain + a `field: A`). +- `T` is a closure type. Reason: `T`'s environment can contain a field of + type `A`. +- `A` is the element type of `T` if `T` is an array, sequence or pointer type. + + + + +Sendable pragma +--------------- + +A container type can be marked as `.sendable`. `.sendable` declares that the type +encapsulates a `ref` type effectively so that a variable of this container type +can be used in an `isolate` context: + + ```nim + type + Isolated*[T] {.sendable.} = object ## Isolated data can only be moved, not copied. + value: T + + proc `=copy`*[T](dest: var Isolated[T]; src: Isolated[T]) {.error.} + + proc `=sink`*[T](dest: var Isolated[T]; src: Isolated[T]) {.inline.} = + # delegate to value's sink operation + `=sink`(dest.value, src.value) + + proc `=destroy`*[T](dest: var Isolated[T]) {.inline.} = + # delegate to value's destroy operation + `=destroy`(dest.value) + ``` + +The `.sendable` pragma itself is an experimenal, unchecked, unsafe annotation. It is +currently only used by `Isolated[T]`. + +Virtual pragma +============== + +`virtual` is designed to extend or create virtual functions when targeting the cpp backend. When a proc is marked with virtual, it forward declares the proc header within the type's body. + +Here's an example of how to use the virtual pragma: + +```nim +proc newCpp*[T](): ptr T {.importcpp: "new '*0()".} +type + Foo = object of RootObj + FooPtr = ptr Foo + Boo = object of Foo + BooPtr = ptr Boo + +proc salute(self: FooPtr) {.virtual.} = + echo "hello foo" + +proc salute(self: BooPtr) {.virtual.} = + echo "hello boo" + +let foo = newCpp[Foo]() +let boo = newCpp[Boo]() +let booAsFoo = cast[FooPtr](newCpp[Boo]()) + +foo.salute() # prints hello foo +boo.salute() # prints hello boo +booAsFoo.salute() # prints hello boo +``` +In this example, the `salute` function is virtual in both Foo and Boo types. This allows for polymorphism. + +The virtual pragma also supports a special syntax to express Cpp constraints. Here's how it works: + +`$1` refers to the function name +`'idx` refers to the type of the argument at the position idx. Where idx = 1 is the `this` argument. +`#idx` refers to the argument name. + +The return type can be referred to as `-> '0`, but this is optional and often not needed. + + ```nim + {.emit:"""/*TYPESECTION*/ +#include <iostream> + class CppPrinter { + public: + + virtual void printConst(char* message) const { + std::cout << "Const Message: " << message << std::endl; + } + virtual void printConstRef(char* message, const int& flag) const { + std::cout << "Const Ref Message: " << message << std::endl; + } +}; +""".} + +type + CppPrinter {.importcpp, inheritable.} = object + NimPrinter {.exportc.} = object of CppPrinter + +proc printConst(self: CppPrinter; message:cstring) {.importcpp.} +CppPrinter().printConst(message) + +# override is optional. +proc printConst(self: NimPrinter; message: cstring) {.virtual: "$1('2 #2) const override".} = + echo "NimPrinter: " & $message + +proc printConstRef(self: NimPrinter; message: cstring; flag:int32) {.virtual: "$1('2 #2, const '3& #3 ) const override".} = + echo "NimPrinterConstRef: " & $message + +NimPrinter().printConst(message) +var val: int32 = 10 +NimPrinter().printConstRef(message, val) + +``` + +Constructor pragma +================== + +The `constructor` pragma can be used in two ways: in conjunction with `importcpp` to import a C++ constructor, and to declare constructors that operate similarly to `virtual`. + +Consider: + +```nim +type Foo* = object + x: int32 + +proc makeFoo(x: int32): Foo {.constructor.} = + result.x = x +``` + +It forward declares the constructor in the type definition. When the constructor has parameters, it also generates a default constructor. One can avoid this behaviour by using `noDecl` in a default constructor. + +Like `virtual`, `constructor` also supports a syntax that allows to express C++ constraints. + +For example: + +```nim +{.emit:"""/*TYPESECTION*/ +struct CppClass { + int x; + int y; + CppClass(int inX, int inY) { + this->x = inX; + this->y = inY; + } + //CppClass() = default; +}; +""".} + +type + CppClass* {.importcpp, inheritable.} = object + x: int32 + y: int32 + NimClass* = object of CppClass + +proc makeNimClass(x: int32): NimClass {.constructor:"NimClass('1 #1) : CppClass(0, #1)".} = + result.x = x + +# Optional: define the default constructor explicitly +proc makeCppClass(): NimClass {.constructor: "NimClass() : CppClass(0, 0)".} = + result.x = 1 +``` + +In the example above `CppClass` has a deleted default constructor. Notice how by using the constructor syntax, one can call the appropriate constructor. + +Notice when calling a constructor in the section of a global variable initialization, it will be called before `NimMain` meaning Nim is not fully initialized. + +Constructor Initializer +======================= + +By default Nim initializes `importcpp` types with `{}`. This can be problematic when importing +types with a deleted default constructor. In order to avoid this, one can specify default values for a constructor by specifying default values for the proc params in the `constructor` proc. + +For example: + +```nim + +{.emit: """/*TYPESECTION*/ +struct CppStruct { + CppStruct(int x, char* y): x(x), y(y){} + int x; + char* y; +}; +""".} +type + CppStruct {.importcpp, inheritable.} = object + +proc makeCppStruct(a: cint = 5, b:cstring = "hello"): CppStruct {.importcpp: "CppStruct(@)", constructor.} + +(proc (s: CppStruct) = echo "hello")(makeCppStruct()) +# If one removes a default value from the constructor and passes it to the call explicitly, the C++ compiler will complain. + +``` +Skip initializers in fields members +=================================== + +By using `noInit` in a type or field declaration, the compiler will skip the initializer. By doing so one can explicitly initialize those values in the constructor of the type owner. + +For example: + +```nim + +{.emit: """/*TYPESECTION*/ + struct Foo { + Foo(int a){}; + }; + struct Boo { + Boo(int a){}; + }; + + """.} + +type + Foo {.importcpp.} = object + Boo {.importcpp, noInit.} = object + Test {.exportc.} = object + foo {.noInit.}: Foo + boo: Boo + +proc makeTest(): Test {.constructor: "Test() : foo(10), boo(1)".} = + discard + +proc main() = + var t = makeTest() + +main() + +``` + +Will produce: + +```cpp + +struct Test { + Foo foo; + Boo boo; + N_LIB_PRIVATE N_NOCONV(, Test)(void); +}; + +``` + +Notice that without `noInit` it would produce `Foo foo {}` and `Boo boo {}` + + +Member pragma +============= + +Like the `constructor` and `virtual` pragmas, the `member` pragma can be used to attach a procedure to a C++ type. It's more flexible than the `virtual` pragma in the sense that it accepts not only names but also operators and destructors. + +For example: + +```nim +proc print(s: cstring) {.importcpp: "printf(@)", header: "<stdio.h>".} + +type + Doo {.exportc.} = object + test: int + +proc memberProc(f: Doo) {.member.} = + echo $f.test + +proc destructor(f: Doo) {.member: "~'1()", used.} = + print "destructing\n" + +proc `==`(self, other: Doo): bool {.member: "operator==('2 const & #2) const -> '0".} = + self.test == other.test + +let doo = Doo(test: 2) +doo.memberProc() +echo doo == Doo(test: 1) + +``` + +Will print: +``` +2 +false +destructing +destructing +``` + +Notice how the C++ destructor is called automatically. Also notice the double implementation of `==` as an operator in Nim but also in C++. This is useful if you need the type to match some C++ `concept` or `trait` when interoping. + +A side effect of being able to declare C++ operators, is that you can now also create a +C++ functor to have seamless interop with C++ lambdas (syntactic sugar for functors). + +For example: + +```nim +type + NimFunctor = object + discard +proc invoke(f: NimFunctor; n: int) {.member: "operator ()('2 #2)".} = + echo "FunctorSupport!" + +{.experimental: "callOperator".} +proc `()`(f: NimFunctor; n:int) {.importcpp: "#(@)" .} +NimFunctor()(1) +``` +Notice we use the overload of `()` to have the same semantics in Nim, but on the `importcpp` we import the functor as a function. +This allows to easy interop with functions that accepts for example a `const` operator in its signature. + + +Injected symbols in generic procs and templates +=============================================== + +With the experimental option `openSym`, captured symbols in generic routine and +template bodies may be replaced by symbols injected locally by templates/macros +at instantiation time. `bind` may be used to keep the captured symbols over the +injected ones regardless of enabling the options, but other methods like +renaming the captured symbols should be used instead so that the code is not +affected by context changes. + +Since this change may affect runtime behavior, the experimental switch +`openSym` needs to be enabled; and a warning is given in the case where an +injected symbol would replace a captured symbol not bound by `bind` and +the experimental switch isn't enabled. + +```nim +const value = "captured" +template foo(x: int, body: untyped): untyped = + let value {.inject.} = "injected" + body + +proc old[T](): string = + foo(123): + return value # warning: a new `value` has been injected, use `bind` or turn on `experimental:openSym` +echo old[int]() # "captured" + +template oldTempl(): string = + block: + foo(123): + value # warning: a new `value` has been injected, use `bind` or turn on `experimental:openSym` +echo oldTempl() # "captured" + +{.experimental: "openSym".} + +proc bar[T](): string = + foo(123): + return value +assert bar[int]() == "injected" # previously it would be "captured" + +proc baz[T](): string = + bind value + foo(123): + return value +assert baz[int]() == "captured" + +template barTempl(): string = + block: + foo(123): + value +assert barTempl() == "injected" # previously it would be "captured" + +template bazTempl(): string = + bind value + block: + foo(123): + value +assert bazTempl() == "captured" +``` + +This option also generates a new node kind `nnkOpenSym` which contains +exactly 1 `nnkSym` node. In the future this might be merged with a slightly +modified `nnkOpenSymChoice` node but macros that want to support the +experimental feature should still handle `nnkOpenSym`, as the node kind would +simply not be generated as opposed to being removed. + +Another experimental switch `genericsOpenSym` exists that enables this behavior +at instantiation time, meaning templates etc can enable it specifically when +they are being called. However this does not generate `nnkOpenSym` nodes +(unless the other switch is enabled) and so doesn't reflect the regular +behavior of the switch. + +```nim +const value = "captured" +template foo(x: int, body: untyped): untyped = + let value {.inject.} = "injected" + {.push experimental: "genericsOpenSym".} + body + {.pop.} + +proc bar[T](): string = + foo(123): + return value +echo bar[int]() # "injected" + +template barTempl(): string = + block: + var res: string + foo(123): + res = value + res +assert barTempl() == "injected" +``` + + +VTable for methods +================== + +Methods now support implementations based on a VTable by using `--experimental:vtables`. Note that the option needs to enabled +globally. The virtual method table is stored in the type info of +an object, which is an array of function pointers. + +```nim +method foo(x: Base, ...) {.base.} +method foo(x: Derived, ...) {.base.} +``` + +It roughly generates a dispatcher like + +```nim +proc foo_dispatch(x: Base, ...) = + x.typeinfo.vtable[method_index](x, ...) # method_index is the index of the sorted order of a method +``` + +Methods are required to be in the same module where their type has been defined. + +```nim +# types.nim +type + Base* = ref object +``` + +```nim +import types + +method foo(x: Base) {.base.} = discard +``` + +It gives an error: method `foo` can be defined only in the same module with its type (Base). + + +asmSyntax pragma +================ + +The `asmSyntax` pragma is used to specify target inline assembler syntax in an `asm` statement. + +It prevents compiling code with different of the target CC inline asm syntax, i.e. it will not allow gcc inline asm code to be compiled with vcc. + +```nim +proc nothing() = + asm {.asmSyntax: "gcc".}""" + nop + """ +``` + +The current C(C++) backend implementation cannot generate code for gcc and for vcc at the same time. For example, `{.asmSyntax: "vcc".}` with the ICC compiler will not generate code with intel asm syntax, even though ICC can use both gcc-like and vcc-like asm. diff --git a/doc/manual_experimental_strictnotnil.md b/doc/manual_experimental_strictnotnil.md index b6d8e796e..a6fa6cda8 100644 --- a/doc/manual_experimental_strictnotnil.md +++ b/doc/manual_experimental_strictnotnil.md @@ -74,7 +74,9 @@ local turn on/off --------------------- You can still turn off nil checking on function/module level by using a `{.strictNotNil: off.}` pragma. -Note: test that/TODO for code/manual. + +.. + Note: test that/TODO for code/manual. nilability state ----------------- @@ -101,7 +103,8 @@ Types are either nilable or non-nilable. When you pass a param or a default value, we use the type : for nilable types we return `MaybeNil` and for non-nilable `Safe`. -TODO: fix the manual here. (This is not great, as default values for non-nilables and nilables are usually actually `nil` , so we should think a bit more about this section.) +.. + TODO: fix the manual here. (This is not great, as default values for non-nilables and nilables are usually actually `nil` , so we should think a bit more about this section.) params rules ------------ @@ -236,11 +239,11 @@ e.g. left = nil # moving out ``` +.. + initialization of non nilable and nilable values + ------------------------------------------------- -initialization of non nilable and nilable values -------------------------------------------------- - -TODO + TODO warnings and errors --------------------- diff --git a/doc/markdown_rst.md b/doc/markdown_rst.md index 5fb3caefc..c7977f75a 100644 --- a/doc/markdown_rst.md +++ b/doc/markdown_rst.md @@ -9,6 +9,8 @@ Nim-flavored Markdown and reStructuredText .. include:: rstcommon.rst .. contents:: +.. importdoc:: docgen.md + Both `Markdown`:idx: (md) and `reStructuredText`:idx: (RST) are markup languages whose goal is to typeset texts with complex structure, formatting and references using simple plaintext representation. @@ -30,12 +32,12 @@ The `md2tex`:option: command is invoked identically to `md2html`:option:, but outputs a ``.tex`` file instead of ``.html``. These tools embedded into Nim compiler; the compiler can output -the result to HTML \[#html] or Latex \[#latex]. +the result to HTML [^html] or Latex [^latex]. -\[#html] commands `nim doc`:cmd: for ``*.nim`` files and +[^html]: commands `nim doc`:cmd: for ``*.nim`` files and `nim rst2html`:cmd: for ``*.rst`` files -\[#latex] commands `nim doc2tex`:cmd: for ``*.nim`` and +[^latex]: commands `nim doc2tex`:cmd: for ``*.nim`` and `nim rst2tex`:cmd: for ``*.rst``. Full list of supported commands: @@ -70,7 +72,7 @@ Features A large subset is implemented with some [limitations] and [additional Nim-specific features]. -Supported standard RST features: +Supported common RST/Markdown features: * body elements + sections @@ -81,10 +83,8 @@ Supported standard RST features: characters: 1. ... 2. ... *or* a. ... b. ... *or* A. ... B. ... + footnotes (including manually numbered, auto-numbered, auto-numbered with label, and auto-symbol footnotes) and citations - + definition lists + field lists + option lists - + indented literal blocks + quoted literal blocks + line blocks + simple tables @@ -107,11 +107,42 @@ Supported standard RST features: (see [RST roles list] for description). + inline internal targets +RST mode only features +---------------------- + ++ RST syntax for definition lists (that is additional indentation after + a definition line) ++ indented literal blocks starting from ``::`` + +Markdown-specific features +-------------------------- + +* Markdown tables +* Markdown code blocks. For them the same additional arguments as for RST + code blocks can be provided (e.g. `test` or `number-lines`) but with + a one-line syntax like this: + + ```nim test number-lines=10 + echo "ok" + ``` +* Markdown links ``[...](...)`` +* Pandoc syntax for automatic links ``[...]``, see [Referencing] for description +* Pandoc syntax for footnotes, including ``[^10]`` (manually numbered) + and ``[^someName]`` (auto-numbered with a label) +* Markdown literal blocks indented by 4 or more spaces +* Markdown headlines +* Markdown block quotes +* Markdown syntax for definition lists +* using ``1`` as auto-enumerator in enumerated lists like RST ``#`` + (auto-enumerator ``1`` can not be used with ``#`` in the same list) + Additional Nim-specific features -------------------------------- -* directives: ``code-block`` \[cmp:Sphinx], ``title``, - ``index`` \[cmp:Sphinx] +* referencing to definitions in external files, see + [Markup external referencing] section +* directives: ``code-block`` [^Sphinx], ``title``, + ``index`` [^Sphinx] * predefined roles - ``:nim:`` (default), ``:c:`` (C programming language), ``:python:``, ``:yaml:``, ``:java:``, ``:cpp:`` (C++), ``:csharp`` (C#). @@ -125,9 +156,9 @@ Additional Nim-specific features - ``:cmd:`` for commands and common shells syntax - ``:console:`` the same for interactive sessions (commands should be prepended by ``$``) - - ``:program:`` for executable names \[cmp:Sphinx] + - ``:program:`` for executable names [^Sphinx] (one can just use ``:cmd:`` on single word) - - ``:option:`` for command line options \[cmp:Sphinx] + - ``:option:`` for command line options [^Sphinx] - ``:tok:``, a role for highlighting of programming language tokens * ***triple emphasis*** (bold and italic) using \*\*\* * ``:idx:`` role for \`interpreted text\` to include the link to this @@ -140,35 +171,96 @@ Additional Nim-specific features Here the dummy `//` will disappear, while options `compile`:option: and `doc`:option: will be left in the final document. +* emoji / smiley symbols -\[cmp:Sphinx] similar but different from the directives of +[^Sphinx]: similar but different from the directives of Python [Sphinx directives] and [Sphinx roles] extensions -Extra features --------------- +.. Note:: By default Nim has ``roSupportMarkdown`` and + ``roSupportRawDirective`` turned **on**. -Optional additional features, by default turned on: +.. warning:: Using Nim-specific features can cause other Markdown and + RST implementations to fail on your document. -* emoji / smiley symbols -* Markdown tables -* Markdown code blocks. For them the same additional arguments as for RST - code blocks can be provided (e.g. `test` or `number-lines`) but with - a one-line syntax like this: +Referencing +=========== - ```nim test number-lines=10 - echo "ok" - ``` -* Markdown links -* Markdown headlines -* Markdown block quotes -* using ``1`` as auto-enumerator in enumerated lists like RST ``#`` - (auto-enumerator ``1`` can not be used with ``#`` in the same list) +To be able to copy and share links Nim generates anchors for all +main document elements: -.. Note:: By default Nim has ``roSupportMarkdown`` and - ``roSupportRawDirective`` turned **on**. +* headlines (including document title) +* footnotes +* explicitly set anchors: RST internal cross-references and + inline internal targets +* Nim symbols (external referencing), see [Nim DocGen Tools Guide] for details. + +But direct use of those anchors have 2 problems: + +1. the anchors are usually mangled (e.g. spaces substituted to minus + signs, etc). +2. manual usage of anchors is not checked, so it's easy to get broken + links inside your project if e.g. spelling has changed for a heading + or you use a wrong relative path to your document. + +That's why Nim implementation has syntax for using +*original* labels for referencing. +Such referencing can be either local/internal or external: + +* Local referencing (inside any given file) is defined by + RST standard or Pandoc Markdown User guide. +* External (cross-document) referencing is a Nim-specific feature, + though it's not really different from local referencing by its syntax. + +Markup local referencing +------------------------ + +There are 2 syntax option available for referencing to objects +inside any given file, e.g. for headlines: + + Markdown RST + + Some headline Some headline + ============= ============= + + Ref. [Some headline] Ref. `Some headline`_ + + +Markup external referencing +--------------------------- + +The syntax is the same as for local referencing, but the anchors are +saved in ``.idx`` files, so one needs to generate them beforehand, +and they should be loaded by an `.. importdoc::` directive. +E.g. if we want to reference section "Some headline" in ``file1.md`` +from ``file2.md``, then ``file2.md`` may look like: + +``` +.. importdoc:: file1.md + +Ref. [Some headline] +``` + +```cmd +nim md2html --index:only file1.md # creates ``htmldocs/file1.idx`` +nim md2html file2.md # creates ``htmldocs/file2.html`` +``` + +To allow cross-references between any files in any order (especially, if +circular references are present), it's strongly recommended +to make a run for creating all the indexes first: + +```cmd +nim md2html --index:only file1.md # creates ``htmldocs/file1.idx`` +nim md2html --index:only file2.md # creates ``htmldocs/file2.idx`` +nim md2html file1.md # creates ``htmldocs/file1.html`` +nim md2html file2.md # creates ``htmldocs/file2.html`` +``` + +and then one can freely reference any objects as if these 2 documents +are actually 1 file. -.. warning:: Using Nim-specific features can cause other RST implementations - to fail on your document. +Other +===== Idiosyncrasies -------------- diff --git a/doc/mm.md b/doc/mm.md index 9d1fc1955..5e0d2f3b9 100644 --- a/doc/mm.md +++ b/doc/mm.md @@ -22,7 +22,7 @@ Multi-paradigm Memory Management Strategies Nim offers multiple different memory management strategies. To choose the memory management strategy use the `--mm:` switch. -**The recommended switch for newly written Nim code is `--mm:orc`.** + .. hint:: **The recommended switch for newly written Nim code is `--mm:orc`.** ARC/ORC @@ -38,7 +38,7 @@ instead entire subgraphs are *moved* between threads. The Nim compiler also aggr optimizes away RC ops and exploits [move semantics](destructors.html#move-semantics). Nim performs a fair share of optimizations for ARC/ORC; you can inspect what it did -to your time critical function via `--expandArc:functionName`. +to your time critical function via `--expandArc:functionName`. Likewise, you can inspect the whole module via `--expandArc:fileName`. `--mm:arc` uses the same mechanism as `--mm:orc`, but it leaves out the cycle collector. Both ARC and ORC offer deterministic performance for `hard realtime`:idx: systems, but @@ -73,17 +73,18 @@ Other MM modes Here is a comparison of the different memory management modes: -================== ======== ================= ============== =================== -Memory Management Heap Reference Cycles Stop-The-World Command line switch -================== ======== ================= ============== =================== -ORC Shared Cycle Collector No `--mm:orc` -ARC Shared Leak No `--mm:arc` -RefC Local Cycle Collector No `--mm:refc` -Mark & Sweep Local Cycle Collector No `--mm:markAndSweep` -Boehm Shared Cycle Collector Yes `--mm:boehm` -Go Shared Cycle Collector Yes `--mm:go` -None Manual Manual Manual `--mm:none` -================== ======== ================= ============== =================== +================== ======== ================= ============== ====== =================== =================== +Memory Management Heap Reference Cycles Stop-The-World Atomic Valgrind compatible Command line switch +================== ======== ================= ============== ====== =================== =================== +ORC Shared Cycle Collector No No Yes `--mm:orc` +ARC Shared Leak No No Yes `--mm:arc` +Atomic ARC Shared Leak No Yes Yes `--mm:atomicArc` +RefC Local Cycle Collector No No No `--mm:refc` +Mark & Sweep Local Cycle Collector No No No `--mm:markAndSweep` +Boehm Shared Cycle Collector Yes No No `--mm:boehm` +Go Shared Cycle Collector Yes No No `--mm:go` +None Manual Manual Manual Manual Manual `--mm:none` +================== ======== ================= ============== ====== =================== =================== .. default-role:: code .. include:: rstcommon.rst diff --git a/doc/nep1.md b/doc/nep1.md index 0c62e3f9c..3d2a0cef3 100644 --- a/doc/nep1.md +++ b/doc/nep1.md @@ -256,36 +256,46 @@ Coding Conventions Conventions for multi-line statements and expressions ----------------------------------------------------- -- Tuples which are longer than one line should indent their parameters to - align with the parameters above it. +- Tuples which are longer than one line should indent their parameters. ```nim type - LongTupleA = tuple[wordyTupleMemberOne: int, wordyTupleMemberTwo: string, - wordyTupleMemberThree: float] + LongTupleA = tuple[ + wordyTupleMemberOne: int, wordyTupleMemberTwo: string, + wordyTupleMemberThree: float] ``` - Similarly, any procedure and procedure type declarations that are longer - than one line should do the same thing. + than one line should do the same thing. Double indent may be used to + distinguish them from the body that follows - this applies to all constructs + with a body (if, while, etc). ```nim type - EventCallback = proc (timeReceived: Time, errorCode: int, event: Event, - output: var string) - - proc lotsOfArguments(argOne: string, argTwo: int, argThree: float, - argFour: proc(), argFive: bool): int - {.heyLookALongPragma.} = + EventCallback = proc( + timeReceived: Time, errorCode: int, event: Event, + output: var string) + + proc lotsOfArguments( + argOne: string, argTwo: int, argThree: float, + argFour: proc(), argFive: bool, argSix: int + ): GenericType[int, string] {.heyLookALongPragma.} = + discard ``` -- Multi-line procedure calls should continue on the same column as the opening - parenthesis (like multi-line procedure declarations). +- Multi-line procedure calls should continue indented (like multi-line procedure + declarations). ```nim - startProcess(nimExecutable, currentDirectory, compilerArguments - environment, processOptions) + startProcess( + nimExecutable, currentDirectory, compilerArguments + environment, processOptions) ``` +Previous versions of this guide advocated vertical alignment along the opening +brace / parenthesis - both styles are permissible with a preference for the +current style in new code. + Miscellaneous ------------- diff --git a/doc/nimc.md b/doc/nimc.md index 81a060d8a..38558454b 100644 --- a/doc/nimc.md +++ b/doc/nimc.md @@ -32,6 +32,17 @@ Compiler Usage Command-line switches --------------------- +All options that take a `PATH` or `DIR` argument are subject to path substitution: + +- `$nim`: The global nim prefix path +- `$lib`: The stdlib path +- `$home` and `~`: The user's home path +- `$config`: The directory of the module currently being compiled +- `$projectname`: The project file's name without file extension +- `$projectpath` and `$projectdir`: The project file's path +- `$nimcache`: The nimcache path + + Basic command-line switches are: .. no syntax highlighting in the below included files at the moment @@ -104,7 +115,6 @@ ExprAlwaysX ExtendedContext GCStats Dumps statistics about the Garbage Collector. GlobalVar Shows global variables declarations. -LineTooLong Line exceeds the maximum length. Link Linking phase. Name Path Search paths modifications. @@ -316,14 +326,16 @@ Another way is to make Nim invoke a cross compiler toolchain: nim c --cpu:arm --os:linux myproject.nim ``` -For cross compilation, the compiler invokes a C compiler named -like `$cpu.$os.$cc` (for example arm.linux.gcc) and the configuration -system is used to provide meaningful defaults. For example for `ARM` your +For cross compilation, the compiler invokes a C compiler named like +`$cpu.$os.$cc` (for example `arm.linux.gcc`) with options defined in +`$cpu.$os.$cc.options.always`. The configuration system is used to provide +meaningful defaults. For example, for Linux on a 32-bit ARM CPU, your configuration file should contain something like: arm.linux.gcc.path = "/usr/bin" arm.linux.gcc.exe = "arm-linux-gcc" arm.linux.gcc.linkerexe = "arm-linux-gcc" + arm.linux.gcc.options.always = "-w -fmax-errors=3" Cross-compilation for Windows ============================= @@ -469,9 +481,35 @@ They are: 5. nl_types. No headers for this. 6. As mmap is not supported, the nimAllocPagesViaMalloc option has to be used. +GPU Compilation +=============== + +Compiling for GPU computation can be achieved with `--cc:nvcc` for CUDA with nvcc, or with `--cc:hipcc` for AMD GPUs with HIP. Both compilers require building for C++ with `nim cpp`. + +Here's a very simple CUDA kernel example using emit, which can be compiled with `nim cpp --cc:nvcc --define:"useMalloc" hello_kernel.nim` assuming you have the CUDA toolkit installed. + +```nim +{.emit: """ +__global__ void add(int a, int b) { + int c; + c = a + b; +} +""".} + +proc main() = + {.emit: """ + add<<<1,1>>>(2,7); + """.} + +main() +``` + DLL generation ============== +**Note**: The same rules apply to `lib*.so` shared object files on UNIX. For better +readability only the DLL version is described here. + Nim supports the generation of DLLs. However, there must be only one instance of the GC per process/address space. This instance is contained in ``nimrtl.dll``. This means that every generated Nim DLL depends @@ -487,9 +525,6 @@ To link against ``nimrtl.dll`` use the command: nim c -d:useNimRtl myprog.nim ``` -**Note**: Currently the creation of ``nimrtl.dll`` with thread support has -never been tested and is unlikely to work! - Additional compilation switches =============================== @@ -513,7 +548,7 @@ Define Effect This only works with `--mm:none`:option:, `--mm:arc`:option: and `--mm:orc`:option:. `useRealtimeGC` Enables support of Nim's GC for *soft* realtime - systems. See the documentation of the [mm](mm.html) + systems. See the documentation of the [refc](refc.html) for further information. `logGC` Enable GC logging to stdout. `nodejs` The JS target is actually ``node.js``. @@ -536,6 +571,13 @@ Define Effect `globalSymbols` Load all `{.dynlib.}` libraries with the `RTLD_GLOBAL`:c: flag on Posix systems to resolve symbols in subsequently loaded libraries. +`lto` Enable link-time optimization in the backend compiler and + linker. +`lto_incremental` Enable link-time optimization and additionally enable + incremental linking for compilers that support it. + Currently only clang and vcc. +`strip` Strip debug symbols added by the backend compiler from + the executable. ====================== ========================================================= @@ -661,11 +703,11 @@ additional flags to both the Nim compiler and the C compiler and/or linker to optimize the build for size. For example, the following flags can be used when targeting a gcc compiler: -`--opt:size --passC:-flto --passL:-flto`:option: +`--opt:size -d:lto -d:strip`:option: The `--opt:size`:option: flag instructs Nim to optimize code generation for small -size (with the help of the C compiler), the `-flto`:option: flags enable link-time -optimization in the compiler and linker. +size (with the help of the C compiler), the `-d:lto`:option: flags enable link-time +optimization in the compiler and linker, the `-d:strip`:option: strips debug symbols. Check the [Cross-compilation] section for instructions on how to compile the program for your target. @@ -713,7 +755,7 @@ Nim's thread API provides a simple wrapper around more advanced RTOS task features. Customizing the stack size and stack guard size can be done by setting `-d:nimThreadStackSize=16384` or `-d:nimThreadStackGuard=32`. -Currently only Zephyr and FreeRTOS support these configurations. +Currently only Zephyr, NuttX and FreeRTOS support these configurations. Nim for realtime systems ======================== diff --git a/doc/nimdoc.css b/doc/nimdoc.css index f924f5a36..0c399e4c1 100644 --- a/doc/nimdoc.css +++ b/doc/nimdoc.css @@ -147,6 +147,15 @@ body { box-sizing: border-box; margin-left: 1%; } +@media print { + #global-links, .link-seesrc, .theme-switch-wrapper, #searchInputDiv, .search-groupby { + display:none; + } + .columns { + width:100% !important; + } +} + .column:first-child, .columns:first-child { margin-left: 0; } @@ -614,8 +623,8 @@ pre { table.line-nums-table { border-radius: 4px; - border: 1px solid #cccccc; - background-color: ghostwhite; + border: 1px solid var(--border); + background-color: var(--secondary-background); border-collapse: separate; margin-top: 15px; margin-bottom: 25px; } @@ -651,6 +660,9 @@ table { border-collapse: collapse; border-color: var(--third-background); border-spacing: 0; +} + +table:not(.line-nums-table) { font-size: 0.9em; } @@ -669,7 +681,7 @@ table th.docinfo-name { text-align: right; } -table tr:hover { +table:not(.line-nums-table) tr:hover { background-color: var(--third-background); } @@ -758,9 +770,16 @@ div.topic { div.search_results { background-color: var(--third-background); - margin: 3em; + margin: 3vh 5vw; padding: 1em; - border: 1px solid #4d4d4d; } + border: 1px solid #4d4d4d; + position: fixed; + top: 10px; + isolation: isolate; + max-width: calc(100vw - 6em); + z-index: 1; + max-height: calc(100vh - 6em); + overflow-y: scroll;} div#global-links ul { margin-left: 0; diff --git a/doc/nimfix.md b/doc/nimfix.md deleted file mode 100644 index 128b25a45..000000000 --- a/doc/nimfix.md +++ /dev/null @@ -1,65 +0,0 @@ -.. default-role:: code - -===================== - Nimfix User Guide -===================== - -:Author: Andreas Rumpf -:Version: |nimversion| - -**WARNING**: Nimfix is currently beta-quality. - -Nimfix is a tool to help you upgrade from Nimrod (<= version 0.9.6) to -Nim (=> version 0.10.0). - -It performs 3 different actions: - -1. It makes your code case consistent. -2. It renames every symbol that has a deprecation rule. So if a module has a - rule `{.deprecated: [TFoo: Foo].}` then `TFoo` is replaced by `Foo`. -3. It can also check that your identifiers adhere to the official style guide - and optionally modify them to do so (via `--styleCheck:auto`). - -Note that `nimfix` defaults to **overwrite** your code unless you -use `--overwriteFiles:off`! But hey, if you do not use a version control -system by this day and age, your project is already in big trouble. - - -Installation ------------- - -Nimfix is part of the compiler distribution. Compile via: - - ```cmd - nim c compiler/nimfix/nimfix.nim - mv compiler/nimfix/nimfix bin - ``` - -Or on windows: - - ```cmd - nim c compiler\nimfix\nimfix.nim - move compiler\nimfix\nimfix.exe bin - ``` - -Usage ------ - -Usage: - - ```cmd - nimfix [options] projectfile.nim - ``` - -Options: - - --overwriteFiles:on|off overwrite the original nim files. DEFAULT is ON! - --wholeProject overwrite every processed file. - --checkExtern:on|off style check also extern names - --styleCheck:on|off|auto performs style checking for identifiers - and suggests an alternative spelling; - 'auto' corrects the spelling. - -In addition, all command line options of Nim are supported. - - diff --git a/doc/nimgrep.md b/doc/nimgrep.md index 8fb86a9d3..63f760051 100644 --- a/doc/nimgrep.md +++ b/doc/nimgrep.md @@ -77,7 +77,7 @@ That means you can always use only 1 such an option with logical OR, e.g. .. Note:: If you want logical AND on patterns you should compose 1 appropriate pattern, possibly combined with multi-line mode `(?s)`:literal:. - E.g. to require that multi-line context of matches has occurences of + E.g. to require that multi-line context of matches has occurrences of **both** PAT1 and PAT2 use positive lookaheads (`(?=PAT)`:literal:): ```cmd nimgrep --inContext:'(?s)(?=.*PAT1)(?=.*PAT2)' diff --git a/doc/nims.md b/doc/nims.md index a69ea1789..987cc2096 100644 --- a/doc/nims.md +++ b/doc/nims.md @@ -61,43 +61,44 @@ Standard library modules At least the following standard library modules are available: -* [macros](macros.html) -* [os](os.html) -* [strutils](strutils.html) -* [math](math.html) -* [distros](distros.html) -* [sugar](sugar.html) * [algorithm](algorithm.html) * [base64](base64.html) * [bitops](bitops.html) * [chains](chains.html) * [colors](colors.html) * [complex](complex.html) +* [distros](distros.html) +* [std/editdistance](editdistance.html) * [htmlgen](htmlgen.html) +* [htmlparser](htmlparser.html) * [httpcore](httpcore.html) +* [json](json.html) * [lenientops](lenientops.html) +* [macros](macros.html) +* [math](math.html) * [options](options.html) +* [os](os.html) +* [parsecfg](parsecfg.html) +* [parsecsv](parsecsv.html) +* [parsejson](parsejson.html) +* [parsesql](parsesql.html) * [parseutils](parseutils.html) * [punycode](punycode.html) * [random](random.html) +* [ropes](ropes.html) +* [std/setutils](setutils.html) * [stats](stats.html) * [strformat](strformat.html) * [strmisc](strmisc.html) * [strscans](strscans.html) +* [strtabs](strtabs.html) +* [strutils](strutils.html) +* [sugar](sugar.html) * [unicode](unicode.html) +* [unidecode](unidecode.html) * [uri](uri.html) -* [std/editdistance](editdistance.html) * [std/wordwrap](wordwrap.html) -* [parsecsv](parsecsv.html) -* [parsecfg](parsecfg.html) -* [parsesql](parsesql.html) * [xmlparser](xmlparser.html) -* [htmlparser](htmlparser.html) -* [ropes](ropes.html) -* [json](json.html) -* [parsejson](parsejson.html) -* [strtabs](strtabs.html) -* [unidecode](unidecode.html) In addition to the standard Nim syntax ([system](system.html) module), NimScripts support the procs and templates defined in the @@ -124,6 +125,8 @@ Here are few examples of using the `switch` proc: switch("define", "release") # command-line: --forceBuild switch("forceBuild") + # command-line: --hint[Conf]:off or --hint:Conf:off + switch("hint", "[Conf]:off") ``` NimScripts also support `--`:option: templates for convenience, which look @@ -211,7 +214,7 @@ ends with ``.nims``: echo "hello world" ``` -Use `#!/usr/bin/env -S nim --hints:off` to disable hints. +Use `#!/usr/bin/env -S nim e --hints:off` to disable hints and relax the file extension constraint. Benefits @@ -329,7 +332,7 @@ Evolving Scripting language NimScript evolves together with Nim, [occasionally new features might become available on NimScript]( -https://github.com/nim-lang/Nim/pulls?utf8=%E2%9C%93&q=nimscript), +https://github.com/nim-lang/Nim/pulls?q=nimscript+is%3Amerged), adapted from compiled Nim or added as new features on both. Scripting Language with a Package Manager diff --git a/doc/nimsuggest.md b/doc/nimsuggest.md index 97cb2b1fa..3d076a6f5 100644 --- a/doc/nimsuggest.md +++ b/doc/nimsuggest.md @@ -49,7 +49,7 @@ via sockets is more reasonable so that is the default. It listens to port 6000 by default. Nimsuggest is basically a frontend for the nim compiler so `--path`:option: flags and -[config files](https://nim-lang.org/docs/nimc.html#compiler-usage-configuration-files) +[config files](nimc.html#compiler-usage-configuration-files) can be used to specify additional dependencies like `nimsuggest --stdin --debug --path:"dependencies" myproject.nim`:cmd:. diff --git a/doc/packaging.md b/doc/packaging.md index 9a1cc0f6e..b742bef28 100644 --- a/doc/packaging.md +++ b/doc/packaging.md @@ -69,8 +69,11 @@ Hints on the build process: What to install: -- The expected stdlib location is /usr/lib/nim -- Global configuration files under /etc/nim +- The expected stdlib location is `/usr/lib/nim/lib`, previously it was just `/usr/lib/nim` +- `nimdoc.css` and `nimdoc.cls` from the `doc` folder should go into `/usr/lib/nim/doc/` +- `tools/debug/nim-gdb.py` should go into `/usr/lib/nim/tools/` +- `tools/dochack/dochack.js` should be installed to `/usr/lib/nim/tools/dochack/` +- Global configuration files under `/etc/nim` - Optionally: manpages, documentation, shell completion - When installing documentation, .idx files are not required - The "compiler" directory contains compiler sources and should not be part of the compiler binary package diff --git a/doc/readme.txt b/doc/readme.txt index 7b1a445b5..1157752b9 100644 --- a/doc/readme.txt +++ b/doc/readme.txt @@ -3,5 +3,5 @@ Nim's documentation system ============================ This folder contains Nim's documentation. The documentation -is written in a format called *reStructuredText*, a markup language that reads +is written in a format called *Markdown*, a markup language that reads like ASCII and can be converted to HTML automatically! diff --git a/doc/refc.md b/doc/refc.md index 6e4672ac5..4023748e6 100644 --- a/doc/refc.md +++ b/doc/refc.md @@ -138,6 +138,7 @@ The garbage collector won't try to free them, you need to call their respective when you are done with them or they will leak. + Heap dump ========= diff --git a/doc/sets_fragment.txt b/doc/sets_fragment.txt index 4aac75191..35b1fc023 100644 --- a/doc/sets_fragment.txt +++ b/doc/sets_fragment.txt @@ -5,24 +5,25 @@ only be an ordinal type of a certain size, namely: * `uint8`/`byte`-`uint16` * `char` * `enum` +* Ordinal subrange types, i.e. `range[-10..10]` -or equivalent. For signed integers the set's base type is defined to be in the -range `0 .. MaxSetElements-1` where `MaxSetElements` is currently always -2^16. +or equivalent. When constructing a set with signed integer literals, the set's +base type is defined to be in the range `0 .. DefaultSetElements-1` where +`DefaultSetElements` is currently always 2^8. The maximum range length for the +base type of a set is `MaxSetElements` which is currently always 2^16. Types +with a bigger range length are coerced into the range `0 .. MaxSetElements-1`. The reason is that sets are implemented as high performance bit vectors. Attempting to declare a set with a larger type will result in an error: ```nim - var s: set[int64] # Error: set is too large; use `std/sets` for ordinal types # with more than 2^16 elements - ``` **Note:** Nim also offers [hash sets](sets.html) (which you need to import -with `import sets`), which have no such restrictions. +with `import std/sets`), which have no such restrictions. Sets can be constructed via the set constructor: `{}` is the empty set. The empty set is type compatible with any concrete set type. The constructor @@ -38,6 +39,14 @@ can also be used to include elements (and ranges of elements): # from '0' to '9' ``` +The module [`std/setutils`](setutils.html) provides a way to initialize a set from an iterable: + +```nim +import std/setutils + +let uniqueChars = myString.toSet +``` + These operations are supported by sets: ================== ======================================================== diff --git a/doc/testament.md b/doc/testament.md index ee57be4f1..0ff3591ac 100644 --- a/doc/testament.md +++ b/doc/testament.md @@ -50,6 +50,7 @@ Options --colors:on|off Turn messages coloring on|off. --backendLogging:on|off Disable or enable backend logging. By default turned on. --megatest:on|off Enable or disable megatest. Default is on. +--valgrind:on|off Enable or disable valgrind support. Default is on. --skipFrom:file Read tests to skip from `file` - one test per line, # comments ignored @@ -111,8 +112,21 @@ Example "template" **to edit** and write a Testament unittest: # "run": expect successful compilation and execution # "reject": expect failed compilation. The "reject" action can catch # {.error.} pragmas but not {.fatal.} pragmas because + # {.error.} calls are expected to originate from the test-file, + # and can explicitly be specified using the "file", "line" and + # "column" options. # {.fatal.} pragmas guarantee that compilation will be aborted. action: "run" + + # For testing failed compilations you can specify the expected origin of the + # compilation error. + # With the "file", "line" and "column" options you can define the file, + # line and column that a compilation-error should have originated from. + # Use only with action: "reject" as it expects a failed compilation. + # Requires errormsg or msg to be defined. + # file: "" + # line: "" + # column: "" # The exit code that the test is expected to return. Typically, the default # value of 0 is fine. Note that if the test will be run by valgrind, then @@ -126,13 +140,14 @@ Example "template" **to edit** and write a Testament unittest: output: "" outputsub: "" - # Whether to sort the output lines before comparing them to the desired - # output. + # Whether to sort the compiler output lines before comparing them to the + # expected output. sortoutput: true - # Each line in the string given here appears in the same order in the - # compiler output, but there may be more lines that appear before, after, or - # in between them. + # Provide a `nimout` string to assert that the compiler during compilation + # prints the defined lines to the standard out. + # The lines must match in order, but there may be more lines that appear + # before, after, or in between them. nimout: ''' a very long, multi-line @@ -159,6 +174,9 @@ Example "template" **to edit** and write a Testament unittest: # "leaks": run the test with Valgrind, but do not check for memory leaks valgrind: false # Can use Valgrind to check for memory leaks, or not (Linux 64Bit only). + # Checks that the specified piece of C-code is within the generated C-code. + ccodecheck: "'Assert error message'" + # Command the test should use to run. If left out or an empty string is # provided, the command is taken to be: # "nim $target --hints:on -d:testing --nimblePath:build/deps/pkgs $options $file" @@ -195,7 +213,7 @@ Example "template" **to edit** and write a Testament unittest: * As you can see the "Spec" is just a `discard """ """`. * Spec has sane defaults, so you don't need to provide them all, any simple assert will work just fine. * This is not the full spec of Testament, check [the Testament Spec on GitHub, - see parseSpec()](https://github.com/nim-lang/Nim/blob/devel/testament/specs.nim#L315). + see parseSpec()](https://github.com/nim-lang/Nim/blob/devel/testament/specs.nim#L317). * Nim itself uses Testament, so [there are plenty of test examples]( https://github.com/nim-lang/Nim/tree/devel/tests). * Has some built-in CI compatibility, like Azure Pipelines, etc. @@ -282,6 +300,38 @@ Expected to fail: assert not_defined == "not_defined", "not_defined is not defined" ``` +Expected to fail with error thrown from another file: +```nim +# test.nim +discard """ + action: "reject" + errorMsg: "I break" + file: "breakPragma.nim" +""" +import ./breakPragma + +proc x() {.justDo.} = discard + +# breakPragma.nim +import std/macros + +macro justDo*(procDef: typed): untyped = + error("I break") + return procDef +``` + +Expecting generated C to contain a given piece of code: + + ```nim + discard """ + # Checks that the string "Assert error message" is in the generated + # C code. + ccodecheck: "'Assert error message'" + """ + assert 42 == 42, "Assert error message" + ``` + + Non-Zero exit code: ```nim diff --git a/doc/tools.md b/doc/tools.md index 6849103f9..baf7ce386 100644 --- a/doc/tools.md +++ b/doc/tools.md @@ -7,6 +7,10 @@ Tools available with Nim The standard distribution ships with the following tools: +- | [atlas](atlas.html) + | `atlas`:cmd: is a simple package cloner tool. It manages an isolated workspace that + contains projects and dependencies. + - | [Hot code reloading](hcr.html) | The "Hot code reloading" feature is built into the compiler but has its own document explaining how it works. @@ -33,10 +37,9 @@ The standard distribution ships with the following tools: | `nimpretty`:cmd: is a Nim source code beautifier, to format code according to the official style guide. -- | [testament](https://nim-lang.github.io/Nim/testament.html) +- | [testament](testament.html) | `testament`:cmd: is an advanced automatic *unittests runner* for Nim tests, is used for the development of Nim itself, offers process isolation for your tests, it can generate statistics about test cases, supports multiple targets (C, JS, etc), - [simulated Dry-Runs](https://en.wikipedia.org/wiki/Dry_run_(testing)), has logging, can generate HTML reports, skip tests from a file, and more, so can be useful to run your tests, even the most complex ones. diff --git a/doc/tut2.md b/doc/tut2.md index 3c858c64e..1b59288d5 100644 --- a/doc/tut2.md +++ b/doc/tut2.md @@ -166,7 +166,7 @@ An example: n.strVal = "" ``` -As can been seen from the example, an advantage to an object hierarchy is that +As can be seen from the example, an advantage to an object hierarchy is that no conversion between different object types is needed. Yet, access to invalid object fields raises an exception. @@ -393,7 +393,7 @@ The `try` statement handles exceptions: echo "could not convert string to integer" except IOError: echo "IO error!" - except: + except CatchableError: echo "Unknown exception!" # reraise the unknown exception: raise @@ -425,7 +425,7 @@ module. Example: ```nim try: doSomethingHere() - except: + except CatchableError: let e = getCurrentException() msg = getCurrentExceptionMsg() diff --git a/doc/tut3.md b/doc/tut3.md index ccb2eb809..3a55d4790 100644 --- a/doc/tut3.md +++ b/doc/tut3.md @@ -322,6 +322,36 @@ used to get this output. raise newException(AssertionDefect, $a & " != " & $b) ``` + +Going further +------------- + +It is possible to create more complex macros by combining different +`NimNode` symbols with `quote do:` expressions. For example, you may +use `newStmtList` to build your macro iteratively, and `ident` in cases +in which you wish to create an identifier from a string, as shown below. + + ```nim + import std/macros + + macro createProcedures() = + result = newStmtList() + + for i in 0..<10: + let name = ident("myProc" & $i) + let content = newLit("I am procedure number #" & $i) + + result.add quote do: + proc `name`() = + echo `content` + + createProcedures() + myProc7() + ``` + +The call to `myProc7` will echo `I am procedure number #7`. + + With Power Comes Responsibility ------------------------------- @@ -366,7 +396,7 @@ recommended way. But still `strformat` is a good example for a practical use case for a macro that is slightly more complex than the `assert` macro. -[Strformat](https://github.com/nim-lang/Nim/blob/5845716df8c96157a047c2bd6bcdd795a7a2b9b1/lib/pure/strformat.nim#L280) +[Strformat](https://github.com/nim-lang/Nim/blob/devel/lib/pure/strformat.nim) Ast Pattern Matching -------------------- |