diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/abstypes.txt | 152 | ||||
| -rw-r--r-- | doc/advopt.txt | 1 | ||||
| -rw-r--r-- | doc/apis.txt | 13 | ||||
| -rw-r--r-- | doc/backends.txt | 409 | ||||
| -rw-r--r-- | doc/c2nim.txt | 6 | ||||
| -rw-r--r-- | doc/docgen.txt | 72 | ||||
| -rw-r--r-- | doc/docs.txt | 8 | ||||
| -rw-r--r-- | doc/endb.txt | 12 | ||||
| -rw-r--r-- | doc/estp.txt | 8 | ||||
| -rw-r--r-- | doc/filters.txt | 8 | ||||
| -rw-r--r-- | doc/grammar.txt | 9 | ||||
| -rw-r--r-- | doc/idetools.txt | 6 | ||||
| -rw-r--r-- | doc/intern.txt | 10 | ||||
| -rw-r--r-- | doc/koch.txt | 144 | ||||
| -rw-r--r-- | doc/lib.txt | 2 | ||||
| -rw-r--r-- | doc/manual.txt | 346 | ||||
| -rw-r--r-- | doc/nimgrep.txt | 6 | ||||
| -rw-r--r-- | doc/niminst.txt | 6 | ||||
| -rw-r--r-- | doc/nimrodc.txt | 123 | ||||
| -rw-r--r-- | doc/spawn.txt | 98 | ||||
| -rw-r--r-- | doc/subexes.txt | 1 | ||||
| -rw-r--r-- | doc/tools.txt | 6 | ||||
| -rw-r--r-- | doc/trmacros.txt | 275 | ||||
| -rw-r--r-- | doc/tut1.txt | 48 | ||||
| -rw-r--r-- | doc/tut2.txt | 18 |
25 files changed, 1069 insertions, 718 deletions
diff --git a/doc/abstypes.txt b/doc/abstypes.txt deleted file mode 100644 index c5827745a..000000000 --- a/doc/abstypes.txt +++ /dev/null @@ -1,152 +0,0 @@ -============== -Abstract types -============== - -.. contents:: - -Abstract types in Nimrod provide a means to model different `units`:idx: of -a `base type`:idx:. - - -Use case 1: SQL strings ------------------------ -An SQL statement that is passed from Nimrod to an SQL database might be -modelled as a string. However, using string templates and filling in the -values is vulnerable to the famous `SQL injection attack`:idx:\: - -.. code-block:: nimrod - proc query(db: TDbHandle, statement: TSQL) = ... - - var - username: string - - db.query("SELECT FROM users WHERE name = '$1'" % username) - # Horrible security hole, but the compiler does not mind! - -This can be avoided by distinguishing strings that contain SQL from strings -that don't. Abstract types provide a means to introduce a new string type -``TSQL`` that is incompatible with ``string``: - -.. code-block:: nimrod - type - TSQL = abstract string - - proc query(db: TDbHandle, statement: TSQL) = ... - - var - username: string - - db.query("SELECT FROM users WHERE name = '$1'" % username) - # Error at compile time: `query` expects an SQL string! - - -It is an essential property of abstract types that they **do not** imply a -subtype relation between the abtract type and its base type. Explict type -conversions from ``string`` to ``TSQL`` are allowed: - -.. code-block:: nimrod - proc properQuote(s: string): TSQL = - # quotes a string properly for an SQL statement - ... - - proc `%` (frmt: TSQL, values: openarray[string]): TSQL = - # quote each argument: - var v = values.each(properQuote) - # we need a temporary type for the type conversion :-( - type TStrSeq = seq[string] - # call strutils.`%`: - result = TSQL(string(frmt) % TStrSeq(v)) - - db.query("SELECT FROM users WHERE name = $1".TSQL % username) - -Now we have compile-time checking against SQL injection attacks. -Since ``"".TSQL`` is transformed to ``TSQL("")`` no new syntax is needed -for nice looking ``TSQL`` string literals. - - - -Use case 2: Money ------------------ -Different currencies should not be mixed in monetary calculations. Abstract -types are a perfect tool to model different currencies: - -.. code-block:: nimrod - type - TDollar = abstract int - TEuro = abstract int - - var - d: TDollar - e: TEuro - - echo d + 12 - # Error: cannot add a number with no unit with a ``TDollar`` - -Unfortunetaly, ``d + 12.TDollar`` is not allowed either, -because ``+`` is defined for ``int`` (among others), not for ``TDollar``. So -we define our own ``+`` for dollars: - -.. code-block:: - proc `+` (x, y: TDollar): TDollar = - result = TDollar(int(x) + int(y)) - -It does not make sense to multiply a dollar with a dollar, but with a -number without unit; and the same holds for division: - -.. code-block:: - proc `*` (x: TDollar, y: int): TDollar = - result = TDollar(int(x) * y) - - proc `*` (x: int, y: TDollar): TDollar = - result = TDollar(x * int(y)) - - proc `div` ... - -This quickly gets tedious. The implementations are trivial and the compiler -should not generate all this code only to optimize it away later - after all -``+`` for dollars should produce the same binary code as ``+`` for ints. -The pragma ``borrow`` has been designed to solve this problem; in principle -it generates the trivial implementation for us: - -.. code-block:: nimrod - proc `*` (x: TDollar, y: int): TDollar {.borrow.} - proc `*` (x: int, y: TDollar): TDollar {.borrow.} - proc `div` (x: TDollar, y: int): TDollar {.borrow.} - -The ``borrow`` pragma makes the compiler to use the same implementation as -the proc that deals with the abstract type's base type, so no code is -generated. - -But it seems we still have to repeat all this boilerplate code for -the ``TEuro`` currency. Fortunately, Nimrod has a template mechanism: - -.. code-block:: nimrod - template Additive(typ: typeDesc): stmt = - proc `+` *(x, y: typ): typ {.borrow.} - proc `-` *(x, y: typ): typ {.borrow.} - - # unary operators: - proc `+` *(x: typ): typ {.borrow.} - proc `-` *(x: typ): typ {.borrow.} - - template Multiplicative(typ, base: typeDesc): stmt = - proc `*` *(x: typ, y: base): typ {.borrow.} - proc `*` *(x: base, y: typ): typ {.borrow.} - proc `div` *(x: typ, y: base): typ {.borrow.} - proc `mod` *(x: typ, y: base): typ {.borrow.} - - template Comparable(typ: typeDesc): stmt = - proc `<` * (x, y: typ): bool {.borrow.} - proc `<=` * (x, y: typ): bool {.borrow.} - proc `==` * (x, y: typ): bool {.borrow.} - - template DefineCurrency(typ, base: expr): stmt = - type - typ* = abstract base - Additive(typ) - Multiplicative(typ, base) - Comparable(typ) - - DefineCurrency(TDollar, int) - DefineCurrency(TEuro, int) - diff --git a/doc/advopt.txt b/doc/advopt.txt index f5ff90791..08465e457 100644 --- a/doc/advopt.txt +++ b/doc/advopt.txt @@ -2,6 +2,7 @@ Advanced commands: //compileToC, cc compile project with C code generator //compileToCpp, cpp compile project to C++ code //compileToOC, objc compile project to Objective C code + //js compile project to Javascript //rst2html convert a reStructuredText file to HTML //rst2tex convert a reStructuredText file to TeX //jsondoc extract the documentation to a json file diff --git a/doc/apis.txt b/doc/apis.txt index d76bd721e..9906f6bbc 100644 --- a/doc/apis.txt +++ b/doc/apis.txt @@ -1,6 +1,6 @@ -========== -API design -========== +================= +API naming design +================= The API is designed to be **easy to use** and consistent. Ease of use is measured by the number of calls to achieve a concrete high level action. @@ -79,3 +79,10 @@ string str identifier ident indentation indent ------------------- ------------ -------------------------------------- + + +Coding Guidelines +================= + +For coding guidelines see the `Internals of the Nimrod Compiler +<intern.html#coding-guidelines>`_ documentation. diff --git a/doc/backends.txt b/doc/backends.txt new file mode 100644 index 000000000..26576e733 --- /dev/null +++ b/doc/backends.txt @@ -0,0 +1,409 @@ +================================ + Nimrod Backend Integration +================================ + +:Author: Puppet Master +:Version: |nimrodversion| + +.. contents:: + "Heresy grows from idleness." -- Unknown. + + +Introduction +============ + +The `Nimrod Compiler User Guide <nimrodc.html>`_ documents the typical +compiler invocation, using the ``compile`` or ``c`` command to transform a +``.nim`` file into one or more ``.c`` files which are then compiled with the +platform's C compiler into a static binary. However there are other commands +to compile to C++, Objective-C or JavaScript. This document tries to +concentrate in a single place all the backend and interfacing options. + +The Nimrod compiler supports mainly two backend families: the C, C++ and +Objective-C targets and the JavaScript target. `The C like targets`_ creates +source files which can be compiled into a library or a final executable. `The +JavaScript target`_ can generate a ``.js`` file which you reference from an +HTML file or create a `standalone nodejs program <http://nodejs.org>`_. + +On top of generating libraries or standalone applications, Nimrod offers +bidirectional interfacing with the backend targets through generic and +specific pragmas. + + +Backends +======== + +The C like targets +------------------ + +The commands to compile to either C, C++ or Objective-C are: + + //compileToC, cc compile project with C code generator + //compileToCpp, cpp compile project to C++ code + //compileToOC, objc compile project to Objective C code + +The most significant difference between these commands is that if you look +into the ``nimcache`` directory you will find ``.c``, ``.cpp`` or ``.m`` +files, other than that all of them will produce a native binary for your +project. This allows you to take the generated code and place it directly +into a project using any of these languages. Here are some typical command +line invocations:: + + $ nimrod c hallo.nim + $ nimrod cpp hallo.nim + $ nimrod objc hallo.nim + +The compiler commands select the target backend, but if needed you can +`specify additional switches for cross compilation +<nimrodc.html#cross-compilation>`_ to select the target CPU, operative system +or compiler/linker commands. + + +The JavaScript target +--------------------- + +Nimrod can also generate `JavaScript`:idx: code through the ``js`` command. +However, the JavaScript code generator is experimental! + +Nimrod targets JavaScript 1.5 which is supported by any widely used browser. +Since JavaScript does not have a portable means to include another module, +Nimrod just generates a long ``.js`` file. + +Features or modules that the JavaScript platform does not support are not +available. This includes: + +* manual memory management (``alloc``, etc.) +* casting and other unsafe operations (``cast`` operator, ``zeroMem``, etc.) +* file management +* most modules of the Standard library +* proper 64 bit integer arithmetic +* unsigned integer arithmetic + +However, the modules `strutils <strutils.html>`_, `math <math.html>`_, and +`times <times.html>`_ are available! To access the DOM, use the `dom +<dom.html>`_ module that is only available for the JavaScript platform. + +To compile a Nimrod module into a ``.js`` file use the ``js`` command; the +default is a ``.js`` file that is supposed to be referenced in an ``.html`` +file. However, you can also run the code with `nodejs`:idx:, a `software +platform for easily building fast, scalable network applications +<http://nodejs.org>`_:: + + nimrod js -d:nodejs -r examples/hallo.nim + + +Interfacing +=========== + +Nimrod offers bidirectional interfacing with the target backend. This means +that you can call backend code from Nimrod and Nimrod code can be called by +the backend code. Usually the direction of which calls which depends on your +software architecture (is Nimrod your main program or is Nimrod providing a +component?). + + +Nimrod code calling the backend +-------------------------------- + +Nimrod code can interface with the backend through the `Foreign function +interface <manual.html#foreign-function-interface>`_ mainly through the +`importc pragma <manual.html#importc-pragma>`_. The ``importc`` pragma is the +*generic* way of making backend symbols available in Nimrod and is available +in all the target backends (JavaScript too). The C++ or Objective-C backends +have their respective `ImportCpp <nimrodc.html#importcpp-pragma>`_ and +`ImportObjC <nimrodc.html#importobjc-pragma>`_ pragmas to call methods from +classes. + +Whenever you use any of these pragmas you need to integrate native code into +your final binary. In the case of JavaScript this is no problem at all, the +same html file which hosts the generated JavaScript will likely provide other +JavaScript functions which you are importing with ``importc``. + +However, for the C like targets you need to link external code either +statically or dynamically. The preferred way of integrating native code is to +use dynamic linking because it allows you to compile Nimrod programs without +the need for having the related development libraries installed. This is done +through the `dynlib pragma for import +<manual.html#dynlib-pragma-for-import>`_, though more specific control can be +gained using the `dynlib module <dynlib.html>`_. + +The `dynlibOverride <nimrodc.html#dynliboverride>`_ command line switch allows +to avoid dynamic linking if you need to statically link something instead. +Nimrod wrappers designed to statically link source files can use the `compile +pragma <nimrodc.html#compile-pragma>`_ if there are few sources or providing +them along the Nimrod code is easier than using a system library. Libraries +installed on the host system can be linked in with the `PassL pragma +<nimrodc.html#passl-pragma>`_. + +To wrap native code, take a look at the `c2nim tool <c2nim.html>`_ which helps +with the process of scanning and transforming header files into a Nimrod +interface. + +C invocation example +~~~~~~~~~~~~~~~~~~~~ + +Create a ``logic.c`` file with the following content: + +.. code-block:: c + int addTwoIntegers(int a, int b) + { + return a + b; + } + +Create a ``calculator.nim`` file with the following content: + +.. code-block:: nimrod + + {.compile: "logic.c".} + proc addTwoIntegers(a, b: cint): cint {.importc.} + + when isMainModule: + echo addTwoIntegers(3, 7) + +With these two files in place, you can run ``nimrod c -r calculator.nim`` and +the Nimrod compiler will compile the ``logic.c`` file in addition to +``calculator.nim`` and link both into an executable, which outputs ``10`` when +run. Another way to link the C file statically and get the same effect would +be remove the line with the ``compile`` pragma and run the following typical +Unix commands:: + + $ gcc -c logic.c + $ ar rvs mylib.a logic.o + $ nimrod c --passL:mylib.a -r calculator.nim + +Just like in this example we pass the path to the ``mylib.a`` library (and we +could as well pass ``logic.o``) we could be passing switches to link any other +static C library. + + +JavaScript invocation example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create a ``host.html`` file with the following content: + +.. code-block:: + + <html><body> + <script type="text/javascript"> + function addTwoIntegers(a, b) + { + return a + b; + } + </script> + <script type="text/javascript" src="calculator.js"></script> + </body></html> + +Create a ``calculator.nim`` file with the following content (or reuse the one +from the previous section): + +.. code-block:: nimrod + + proc addTwoIntegers(a, b: int): int {.importc.} + + when isMainModule: + echo addTwoIntegers(3, 7) + +Compile the Nimrod code to JavaScript with ``nimrod js -o:calculator.js +calculator.nim`` and open ``host.html`` in a browser. If the browser supports +javascript, you should see the value ``10``. In JavaScript the `echo proc +<system.html#echo>`_ will modify the HTML DOM and append the string. Use the +`dom module <dom.html>`_ for specific DOM querying and modification procs. + + +Backend code calling Nimrod +--------------------------- + +Backend code can interface with Nimrod code exposed through the `exportc +pragma <manual.html#exportc-pragma>`_. The ``exportc`` pragma is the *generic* +way of making Nimrod symbols available to the backends. By default the Nimrod +compiler will mangle all the Nimrod symbols to avoid any name collision, so +the most significant thing the ``exportc`` pragma does is maintain the Nimrod +symbol name, or if specified, use an alternative symbol for the backend in +case the symbol rules don't match. + +The JavaScript target doesn't have any further interfacing considerations +since it also has garbage collection, but the C targets require you to +initialize Nimrod's internals, which is done calling a ``NimMain`` function. +Also, C code requires you to specify a forward declaration for functions or +the compiler will asume certain types for the return value and parameters +which will likely make your program crash at runtime. + +The Nimrod compiler can generate a C interface header through the ``--header`` +command line switch. The generated header will contain all the exported +symbols and the ``NimMain`` proc which you need to call before any other +Nimrod code. + + +Nimrod invocation example from C +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create a ``fib.nim`` file with the following content: + +.. code-block:: nimrod + + proc fib(a: cint): cint {.exportc.} = + if a <= 2: + result = 1 + else: + result = fib(a - 1) + fib(a - 2) + +Create a ``maths.c`` file with the following content: + +.. code-block:: c + + #include "fib.h" + #include <stdio.h> + + int main(void) + { + NimMain(); + for (int f = 0; f < 10; f++) + printf("Fib of %d is %d\n", f, fib(f)); + return 0; + } + +Now you can run the following Unix like commands to first generate C sources +form the Nimrod code, then link them into a static binary along your main C +program:: + + $ nimrod c --noMain --noLinking --header:fib.h fib.nim + $ gcc -o m -Inimcache -Ipath/to/nimrod/lib nimcache/*.c maths.c + +The first command runs the Nimrod compiler with three special options to avoid +generating a ``main()`` function in the generated files, avoid linking the +object files into a final binary, and explicitly generate a header file for C +integration. All the generated files are placed into the ``nimcache`` +directory. That's why the next command compiles the ``maths.c`` source plus +all the ``.c`` files form ``nimcache``. In addition to this path, you also +have to tell the C compiler where to find Nimrod's ``nimbase.h`` header file. + +Instead of depending on the generation of the individual ``.c`` files you can +also ask the Nimrod compiler to generate a statically linked library:: + + $ nimrod c --app:staticLib --noMain --header fib.nim + $ gcc -o m -Inimcache -Ipath/to/nimrod/lib libfib.nim.a maths.c + +The Nimrod compiler will handle linking the source files generated in the +``nimcache`` directory into the ``libfib.nim.a`` static library, which you can +then link into your C program. Note that these commands are generic and will +vary for each system. For instance, on Linux systems you will likely need to +use ``-ldl`` too to link in required dlopen functionality. + + +Nimrod invocation example from JavaScript +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create a ``mhost.html`` file with the following content: + +.. code-block:: + + <html><body> + <script type="text/javascript" src="fib.js"></script> + <script type="text/javascript"> + alert("Fib for 9 is " + fib(9)); + </script> + </body></html> + +Create a ``fib.nim`` file with the following content (or reuse the one +from the previous section): + +.. code-block:: nimrod + + proc fib(a: cint): cint {.exportc.} = + if a <= 2: + result = 1 + else: + result = fib(a - 1) + fib(a - 2) + +Compile the Nimrod code to JavaScript with ``nimrod js -o:fib.js fib.nim`` and +open ``mhost.html`` in a browser. If the browser supports javascript, you +should see an alert box displaying the text ``Fib for 9 is 34``. As mentioned +earlier, JavaScript doesn't require an initialisation call to ``NimMain`` or +similar function and you can call the exported Nimrod proc directly. + + +Memory management +================= + +In the previous sections the ``NimMain()`` function reared its head. Since +JavaScript already provides automatic memory management, you can freely pass +objects between the two language without problems. In C and derivate languages +you need to be careful about what you do and how you share memory. The +previous examples only dealt with simple scalar values, but passing a Nimrod +string to C, or reading back a C string in Nimrod already requires you to be +aware of who controls what to avoid crashing. + + +Strings and C strings +--------------------- + +The manual mentions that `Nimrod strings are implicitly convertible to +cstrings <manual.html#cstring-type>`_ which makes interaction usually +painless. Most C functions accepting a Nimrod string converted to a +``cstring`` will likely not need to keep this string around and by the time +they return the string won't be needed any more. However, for the rare cases +where a Nimrod string has to be preserved and made available to the C backend +as a ``cstring``, you will need to manually prevent the string data from being +freed with `GC_ref <system.html#GC_ref>`_ and `GC_unref +<system.html#GC_unref>`_. + +A similar thing happens with C code invoking Nimrod code which returns a +``cstring``. Consider the following proc: + +.. code-block:: nimrod + + proc gimme(): cstring {.exportc.} = + result = "Hey there C code! " & $random(100) + +Since Nimrod's garbage collector is not aware of the C code, once the +``gimme`` proc has finished it can reclaim the memory of the ``cstring``. +However, from a practical standpoint, the C code invoking the ``gimme`` +function directly will be able to use it since Nimrod's garbage collector has +not had a chance to run *yet*. This gives you enough time to make a copy for +the C side of the program, as calling any further Nimrod procs *might* trigger +garbage collection making the previously returned string garbage. Or maybe you +are `triggering yourself the collection <gc.html>`_. + + +Custom data types +----------------- + +Just like strings, custom data types that are to be shared between Nimrod and +the backend will need careful consideration of who controlls who. If you want +to hand a Nimrod reference to C code, you will need to use `GC_ref +<system.html#GC_ref>`_ to mark the reference as used, so it does not get +freed. And for the C backend you will need to expose the `GC_unref +<system.html#GC_unref>`_ proc to clean up this memory when it is not required +any more. + +Again, if you are wrapping a library which *mallocs* and *frees* data +structures, you need to expose the appropriate *free* function to Nimrod so +you can clean it up. And of course, once cleaned you should avoid accessing it +from Nimrod (or C for that matter). Typically C data structures have their own +``malloc_structure`` and ``free_structure`` specific functions, so wrapping +these for the Nimrod side should be enough. + + +Thread coordination +------------------- + +When the ``NimMain()`` function is called Nimrod initializes the garbage +collector to the current thread, which is usually the main thread of your +application. If your C code later spawns a different thread and calls Nimrod +code, the garbage collector will fail to work properly and you will crash. + +As long as you don't use the threadvar emulation Nimrod uses native thread +variables, of which you get a fresh version whenever you create a thread. You +can then attach a GC to this thread via + +.. code-block:: nimrod + + setStackBottom(addr(someLocal)) + initGC() + +At the moment this support is still experimental so you need to expose these +functions yourself or submit patches to request a public API. + +It is **not** safe to disable the garbage collector and enable it after the +call from your background thread even if the code you are calling is short +lived. diff --git a/doc/c2nim.txt b/doc/c2nim.txt index 6788ef569..237af7fb2 100644 --- a/doc/c2nim.txt +++ b/doc/c2nim.txt @@ -1,6 +1,6 @@ -================================= - c2nim User's manual -================================= +======================= + c2nim User's manual +======================= :Author: Andreas Rumpf :Version: |nimrodversion| diff --git a/doc/docgen.txt b/doc/docgen.txt index 520d5e8a0..30d0c6ff2 100644 --- a/doc/docgen.txt +++ b/doc/docgen.txt @@ -164,20 +164,10 @@ Index switch nimrod doc2 --index:on filename.nim This will generate an index of all the exported symbols in the input Nimrod -module, and put it into a neighboring file with the extension of `.idx`. The -index file is line oriented (newlines have to be escaped). Each line represents -a tab separated record of several columns, the first two mandatory, the rest -optional: - -1. Mandatory term being indexed. Terms can include quoting according to - Nimrod's rules (eg. ```^```) -2. Base filename plus anchor hyper link (eg. - ``algorithm.html#*,int,TSortOrder``). -3. Optional human readable string to display as hyper link. If the value is not - present or is the empty string, the hyper link will be rendered using the - term. -4. Optional title or description of the hyper link. Browsers usually display - this as a tooltip after hovering a moment over the hyper link. +module, and put it into a neighboring file with the extension of ``.idx``. The +index file is line oriented (newlines have to be escaped). Each line +represents a tab separated record of several columns, the first two mandatory, +the rest optional. See the `Index (idx) file format`_ section for details. Once index files have been generated for one or more modules, the Nimrod compiler command ``buildIndex directory`` can be run to go over all the index @@ -295,6 +285,60 @@ symbols in the `system module <system.html>`_. <system.html#newException.t,typedesc,string>`_ +Index (idx) file format +======================= + +Files with the ``.idx`` extension are generated when you use the `Index +switch`_ along with commands to generate documentation from source or text +files. You can programatically generate indices with the `setIndexTerm() +<rstgen.html#setIndexTerm>`_ and `writeIndexFile() +<rstgen.html#writeIndexFile>`_ procs. The purpose of ``idx`` files is to hold +the interesting symbols and their HTML references so they can be later +concatenated into a big index file with `mergeIndexes() +<rstgen.html#mergeIndexes>`_. This section documents the file format in +detail. + +Index files are line oriented and tab separated (newline and tab characters +have to be escaped). Each line represents a record with at least two fields, +but can have up to four (additional columns are ignored). The content of these +columns is: + +1. Mandatory term being indexed. Terms can include quoting according to + Nimrod's rules (eg. \`^\` like in `the actors module + <actors.html#^,ptr.TChannel[T]>`_). +2. Base filename plus anchor hyper link (eg. + ``algorithm.html#*,int,TSortOrder``). +3. Optional human readable string to display as hyper link. If the value is not + present or is the empty string, the hyper link will be rendered + using the term. Prefix whitespace indicates that this entry is + not for an API symbol but for a TOC entry. +4. Optional title or description of the hyper link. Browsers usually display + this as a tooltip after hovering a moment over the hyper link. + +The index generation tools try to differentiate between documentation +generated from ``.nim`` files and documentation generated from ``.txt`` or +``.rst`` files. The former are always closely related to source code and +consist mainly of API entries. The latter are generic documents meant for +human reading. + +To differentiate both types (documents and APIs), the index generator will add +to the index of documents an entry with the title of the document. Since the +title is the topmost element, it will be added with a second field containing +just the filename without any HTML anchor. By convention this entry without +anchor is the *title entry*, and since entries in the index file are added as +they are scanned, the title entry will be the first line. The title for APIs +is not present because it can be generated concatenating the name of the file +to the word **Module**. + +Normal symbols are added to the index with surrounding whitespaces removed. An +exception to this are table of content (TOC) entries. TOC entries are added to +the index file with their third column having as much prefix spaces as their +level is in the TOC (at least 1 character). The prefix whitespace helps to +filter TOC entries from API or text symbols. This is important because the +amount of spaces is used to replicate the hiearchy for document TOCs in the +final index, and TOC entries found in ``.nim`` files are discarded. + + Additional resources ==================== diff --git a/doc/docs.txt b/doc/docs.txt index 2162d6564..8126da86c 100644 --- a/doc/docs.txt +++ b/doc/docs.txt @@ -6,19 +6,19 @@ The documentation consists of several documents: - | `Tutorial (part II) <tut2.html>`_ | The Nimrod tutorial part two deals with the advanced language constructs. +- | `Language Manual <manual.html>`_ + | The Nimrod manual is a draft that will evolve into a proper specification. + - | `Library documentation <lib.html>`_ | This document describes Nimrod's standard library. -- | `User guide <nimrodc.html>`_ +- | `Compiler user guide <nimrodc.html>`_ | The user guide lists command line arguments, special features of the compiler, etc. - | `Tools documentation <tools.html>`_ | Description of some tools that come with the standard distribution. -- | `Manual <manual.html>`_ - | The Nimrod manual is a draft that will evolve into a proper specification. - - | `GC <gc.html>`_ | Additional documentation about Nimrod's GC and how to operate it in a | realtime setting. diff --git a/doc/endb.txt b/doc/endb.txt index d4437e5d9..e9a01b583 100644 --- a/doc/endb.txt +++ b/doc/endb.txt @@ -1,6 +1,6 @@ -=========================================== - Embedded Nimrod Debugger User Guide -=========================================== +============================================== + Embedded Nimrod Debugger (ENDB) User Guide +============================================== :Author: Andreas Rumpf :Version: |nimrodversion| @@ -9,7 +9,7 @@ Nimrod comes with a platform independent debugger - -the `Embedded Nimrod Debugger`:idx: (`ENDB`:idx:). The debugger is +the Embedded Nimrod Debugger (ENDB). The debugger is *embedded* into your executable if it has been compiled with the ``--debugger:on`` command line option. This also defines the conditional symbol ``ENDB`` for you. @@ -102,7 +102,7 @@ necessary: A special pragma has been defined for this: The ``breakpoint`` pragma ------------------------- -The `breakpoint`:idx: pragma is syntactically a statement. It can be used +The ``breakpoint`` pragma is syntactically a statement. It can be used to mark the *following line* as a breakpoint: .. code-block:: Nimrod @@ -122,7 +122,7 @@ debugging. The ``watchpoint`` pragma ------------------------- -The `watchpoint`:idx: pragma is syntactically a statement. It can be used +The ``watchpoint`` pragma is syntactically a statement. It can be used to mark a location as a watchpoint: .. code-block:: Nimrod diff --git a/doc/estp.txt b/doc/estp.txt index 353e4576f..704bc33f2 100644 --- a/doc/estp.txt +++ b/doc/estp.txt @@ -1,13 +1,13 @@ -============================================ - Embedded Stack Trace Profiler User Guide -============================================ +=================================================== + Embedded Stack Trace Profiler (ESTP) User Guide +=================================================== :Author: Andreas Rumpf :Version: |nimrodversion| Nimrod comes with a platform independent profiler - -the `Embedded Stack Trace Profiler`:idx: (`ESTP`:idx:). The profiler +the Embedded Stack Trace Profiler (ESTP). The profiler is *embedded* into your executable. To activate the profiler you need to do: * compile your program with the ``--profiler:on --stackTrace:on`` command diff --git a/doc/filters.txt b/doc/filters.txt index 683599a03..961a0ea49 100644 --- a/doc/filters.txt +++ b/doc/filters.txt @@ -26,7 +26,7 @@ available parameters depend on the invoked filter. Pipe operator ============= -Filters can be combined with the ``|`` `pipe operator`:idx:\ :: +Filters can be combined with the ``|`` pipe operator:: #! strip(startswith="<") | stdtmpl #proc generateXML(name, age: string): string = @@ -46,7 +46,7 @@ after each filter application. Replace filter -------------- -The `replace`:idx: filter replaces substrings in each line. +The replace filter replaces substrings in each line. Parameters and their defaults: @@ -60,7 +60,7 @@ Parameters and their defaults: Strip filter ------------ -The `strip`:idx: filter simply removes leading and trailing whitespace from +The strip filter simply removes leading and trailing whitespace from each line. Parameters and their defaults: @@ -79,7 +79,7 @@ Parameters and their defaults: StdTmpl filter -------------- -The `stdtmpl`:idx: filter provides a simple templating engine for Nimrod. The +The stdtmpl filter provides a simple templating engine for Nimrod. The filter uses a line based parser: Lines prefixed with a *meta character* (default: ``#``) contain Nimrod code, other lines are verbatim. Because indentation-based parsing is not suited for a templating engine, control flow diff --git a/doc/grammar.txt b/doc/grammar.txt index 63e898e11..a54428678 100644 --- a/doc/grammar.txt +++ b/doc/grammar.txt @@ -24,7 +24,7 @@ ampExpr = plusExpr (OP6 optInd plusExpr)* plusExpr = mulExpr (OP7 optInd mulExpr)* mulExpr = dollarExpr (OP8 optInd dollarExpr)* dollarExpr = primary (OP9 optInd primary)* -symbol = '`' (KEYW|IDENT|operator|'(' ')'|'[' ']'|'{' '}'|'='|literal)+ '`' +symbol = '`' (KEYW|IDENT|literal|(operator|'('|')'|'['|']'|'{'|'}'|'=')+)+ '`' | IDENT indexExpr = expr indexExprList = indexExpr ^+ comma @@ -82,10 +82,11 @@ paramListColon = paramList? (':' optInd typeDesc)? doBlock = 'do' paramListArrow pragmas? colcom stmt doBlocks = doBlock ^* IND{=} procExpr = 'proc' paramListColon pragmas? ('=' COMMENT? stmt)? +distinct = 'distinct' optInd typeDesc expr = (ifExpr | whenExpr | caseExpr - | tryStmt) + | tryExpr) / simpleExpr typeKeyw = 'var' | 'ref' | 'ptr' | 'shared' | 'type' | 'tuple' | 'proc' | 'iterator' | 'distinct' | 'object' | 'enum' @@ -134,6 +135,9 @@ caseStmt = 'case' expr ':'? COMMENT? tryStmt = 'try' colcom stmt &(IND{=}? 'except'|'finally') (IND{=}? 'except' exprList colcom stmt)* (IND{=}? 'finally' colcom stmt)? +tryExpr = 'try' colcom stmt &(optInd 'except'|'finally') + (optInd 'except' exprList colcom stmt)* + (optInd 'finally' colcom stmt)? exceptBlock = 'except' colcom stmt forStmt = 'for' (identWithPragma ^+ comma) 'in' expr colcom stmt blockStmt = 'block' symbol? colcom stmt @@ -166,7 +170,6 @@ object = 'object' pragma? ('of' typeDesc)? COMMENT? objectPart typeClassParam = ('var')? symbol typeClass = typeClassParam ^* ',' (pragma)? ('of' typeDesc ^* ',')? &IND{>} stmt -distinct = 'distinct' optInd typeDesc typeDef = identWithPragma genericParamList? '=' optInd typeDefAux indAndComment? varTuple = '(' optInd identWithPragma ^+ comma optPar ')' '=' optInd expr diff --git a/doc/idetools.txt b/doc/idetools.txt index d4f0f077d..97e110b1d 100644 --- a/doc/idetools.txt +++ b/doc/idetools.txt @@ -16,7 +16,7 @@ Nimrod differs from many other compilers in that it is really fast, and being so fast makes it suited to provide external queries for text editors about the source code being written. Through the -`idetools`:idx: command of `the compiler <nimrodc.html>`_, any IDE +``idetools`` command of `the compiler <nimrodc.html>`_, any IDE can query a ``.nim`` source file and obtain useful information like definition of symbols or suggestions for completion. @@ -169,7 +169,7 @@ clicks it and after a second or two the IDE displays where that symbol is defined. Such latencies would be terrible for features like symbol suggestion, plus why wait at all if we can avoid it? -The idetools command can be run as a compiler service (`caas`:idx:), +The idetools command can be run as a compiler service (CAAS), where you first launch the compiler and it will stay online as a server, accepting queries in a telnet like fashion. The advantage of staying on is that for many queries the compiler can cache the @@ -528,7 +528,7 @@ suite is not integrated with the main test suite and you have to run it manually. First you have to compile the tester:: $ cd my/nimrod/checkout/tests - $ nimrod c caasdriver.nim + $ nimrod c testament/caasdriver.nim Running the ``caasdriver`` without parameters will attempt to process all the test cases in all three operation modes. If a test succeeds diff --git a/doc/intern.txt b/doc/intern.txt index c602e4933..1dcf27774 100644 --- a/doc/intern.txt +++ b/doc/intern.txt @@ -59,6 +59,8 @@ And for a debug version compatible with GDB:: The ``koch`` program is Nimrod's maintenance script. It is a replacement for make and shell scripting with the advantage that it is much more portable. +More information about its options can be found in the `koch <koch.html>`_ +documentation. Coding Guidelines @@ -69,8 +71,10 @@ Coding Guidelines * Max line length is 80 characters. * Provide spaces around binary operators if that enhances readability. * Use a space after a colon, but not before it. -* Start types with a capital ``T``, unless they are pointers/references which start - with ``P``. +* Start types with a capital ``T``, unless they are pointers/references which + start with ``P``. + +See also the `API naming design <apis.html>`_ document. Porting to new platforms @@ -156,7 +160,7 @@ generator. Compilation cache ================= -The implementation of the `compilation cache`:idx: is tricky: There are lots +The implementation of the compilation cache is tricky: There are lots of issues to be solved for the front- and backend. In the following sections *global* means *shared between modules* or *property of the whole program*. diff --git a/doc/koch.txt b/doc/koch.txt new file mode 100644 index 000000000..032151808 --- /dev/null +++ b/doc/koch.txt @@ -0,0 +1,144 @@ +=============================== + Nimrod maintenance script +=============================== + +:Version: |nimrodversion| + +.. contents:: + +.. raw:: html + <blockquote><p> + "A great chef is an artist that I truly respect" -- Robert Stack. + </p></blockquote> + + +Introduction +============ + +The `koch`:idx: program is Nimrod's maintenance script. It is a replacement +for make and shell scripting with the advantage that it is much more portable. +The word *koch* means *cook* in German. ``koch`` is used mainly to build the +Nimrod compiler, but it can also be used for other tasks. This document +describes the supported commands and their options. + + +Commands +======== + +boot command +------------ + +The `boot`:idx: command bootstraps the compiler, and it accepts different +options: + +-d:release + By default a debug version is created, passing this option will + force a release build, which is much faster and should be preferred + unless you are debugging the compiler. +-d:tinyc + Include the `Tiny C <http://bellard.org/tcc/>`_ backend. This + option is not supported on Windows. +-d:useGnuReadline + Includes the `rdstdin module <rdstdin.html>`_ for `interactive + mode <nimrodc.html#nimrod-interactive-mode>`_ (aka ``nimrod i``). + This is not needed on Windows. On other platforms this may + incorporate the GNU readline library. +-d:nativeStacktrace + Use native stack traces (only for Mac OS X or Linux). +-d:noCaas + Builds Nimrod without compiler as a service (CAAS) support. CAAS + support is required for functionality like Nimrod's `idetool + <idetools.html>`_ command used to integrate the compiler with + `external IDEs <https://github.com/Araq/Nimrod/wiki/Editor-Support>`_. +-d:avoidTimeMachine + Only for Mac OS X, activating this switch will force excluding + the generated ``nimcache`` directories from Time Machine backups. + By default ``nimcache`` directories will be included in backups, + and just for the Nimrod compiler itself it means backing up 20MB + of generated files each time you update the compiler. Using this + option will make the compiler invoke the `tmutil + <https://developer.apple.com/library/mac/documentation/Darwin/Reference/Manpages/man8/tmutil.8.html>`_ + command on all ``nimcache`` directories, setting their backup + exclusion bit. + + You can use the following command to locate all ``nimcache`` + directories and check their backup exclusion bit:: + + $ find . -type d -name nimcache -exec tmutil isexcluded \{\} \; +-d:useFFI + Nimrod code can use the `foreign function interface (FFI) + <manual.html#foreign-function-interface>`_ at runtime, but macros + are limited to pure Nimrod code at compilation time. Enabling + this switch will allow macros to execute non-nimrod code at + compilation time (eg. open a file and write to it). +--gc:refc|v2|markAndSweep|boehm|none + Selects which garbage collection strategy to use for the compiler + and generated code. See the `Nimrod's Garbage Collector <gc.html>`_ + documentation for more information. + +After compilation is finished you will hopefully end up with the nimrod +compiler in the ``bin`` directory. You can add Nimrod's ``bin`` directory to +your ``$PATH`` or use the `install command`_ to place it where it will be +found. + +clean command +------------- + +The `clean`:idx: command removes all generated files. + +csource command +--------------- + +The `csource`:idx: command builds the C sources for installation. It accepts +the same options as you would pass to the `boot command`_. + +inno command +------------ + +The `inno`:idx: command builds the `Inno Setup installer for Windows +<http://www.jrsoftware.org/isinfo.php>`_. + +install command +--------------- + +The `install`:idx: command installs Nimrod to the specified directory, which +is required as a parameter. For example, on Unix platforms you could run:: + + $ ./koch install /usr/local/bin + +temp command +------------ + +The temp command builds the Nimrod compiler but with a different final name +(``nimrod_temp``), so it doesn't overwrite your normal compiler. You can use +this command to test different options, the same you would issue for the `boot +command`_. + +test command +------------ + +The `test`:idx: command can also be invoked with the alias ``tests``. This +command will compile and run ``tests/testament/tester.nim``, which is the main +driver of Nimrod's test suite. You can pass options to the ``test`` command, +they will be forwarded to the tester. See its source code for available +options. + +update command +-------------- + +The `update`:idx: command updates nimrod to the latest version from github. +For this command to work you need to have compiled ``koch`` itself with the +``-d:withUpdate`` switch. + +web command +----------- + +The `web`:idx: command converts the documentation in the ``doc`` directory +from rst to HTML. It also repeats the same operation but places the result in +the ``web/upload`` which can be used to update the website at +http://nimrod-lang.org. + +zip command +----------- + +The `zip`:idx: command builds the installation ZIP package. diff --git a/doc/lib.txt b/doc/lib.txt index 3ca519c9e..2da753007 100644 --- a/doc/lib.txt +++ b/doc/lib.txt @@ -535,7 +535,7 @@ Database support * `odbcsql <odbcsql.html>`_ interface to the ODBC driver. * `sphinx <sphinx.html>`_ - Nimrod wrapper for ``shpinx``. + Nimrod wrapper for ``sphinx``. XML Processing diff --git a/doc/manual.txt b/doc/manual.txt index 98af5aebc..32b0541e9 100644 --- a/doc/manual.txt +++ b/doc/manual.txt @@ -123,7 +123,7 @@ This means that all the control structures are recognized by indentation. Indentation consists only of spaces; tabulators are not allowed. The indentation handling is implemented as follows: The lexer annotates the -following token with the preceeding number of spaces; indentation is not +following token with the preceding number of spaces; indentation is not a separate token. This trick allows parsing of Nimrod with only 1 token of lookahead. @@ -152,7 +152,7 @@ statements (simplified example):: Comments -------- -`Comments`:idx: start anywhere outside a string or character literal with the +Comments start anywhere outside a string or character literal with the hash character ``#``. Comments consist of a concatenation of `comment pieces`:idx:. A comment piece starts with ``#`` and runs until the end of the line. The end of line characters @@ -188,7 +188,7 @@ which code snippet the comment refers to. Identifiers & Keywords ---------------------- -`Identifiers`:idx: in Nimrod can be any string of letters, digits +Identifiers in Nimrod can be any string of letters, digits and underscores, beginning with a letter. Two immediate following underscores ``__`` are not allowed:: @@ -201,7 +201,7 @@ classified as a ``letter`` and may thus be part of an identifier but later versions of the language may assign some Unicode characters to belong to the operator characters instead. -The following `keywords`:idx: are reserved and cannot be used as identifiers: +The following keywords are reserved and cannot be used as identifiers: .. code-block:: nimrod :file: keywords.txt @@ -224,7 +224,7 @@ String literals Terminal symbol in the grammar: ``STR_LIT``. -`String literals`:idx: can be delimited by matching double quotes, and can +String literals can be delimited by matching double quotes, and can contain the following `escape sequences`:idx:\ : ================== =================================================== @@ -263,15 +263,16 @@ String literals can also be delimited by three double quotes ``"""`` ... ``"""``. Literals in this form may run for several lines, may contain ``"`` and do not interpret any escape sequences. -For convenience, when the opening ``"""`` is immediately followed by a newline, -the newline is not included in the string. The ending of the string literal is -defined by the pattern ``"""[^"]``, so this: - +For convenience, when the opening ``"""`` is followed by a newline (there may +be whitespace between the opening ``"""`` and the newline), +the newline (and the preceding whitespace) is not included in the string. The +ending of the string literal is defined by the pattern ``"""[^"]``, so this: + .. code-block:: nimrod """"long string within quotes"""" - + Produces:: - + "long string within quotes" @@ -280,7 +281,7 @@ Raw string literals Terminal symbol in the grammar: ``RSTR_LIT``. -There are also `raw string literals`:idx: that are preceded with the +There are also raw string literals that are preceded with the letter ``r`` (or ``R``) and are delimited by matching double quotes (just like ordinary string literals) and do not interpret the escape sequences. This is especially convenient for regular expressions or Windows paths: @@ -313,7 +314,7 @@ Terminal symbols in the grammar: ``GENERALIZED_STR_LIT``, The construct ``identifier"string literal"`` (without whitespace between the identifier and the opening quotation mark) is a -`generalized raw string literal`:idx:. It is a shortcut for the construct +generalized raw string literal. It is a shortcut for the construct ``identifier(r"string literal")``, so it denotes a procedure call with a raw string literal as its only argument. Generalized raw string literals are especially convenient for embedding mini languages directly into Nimrod @@ -365,7 +366,7 @@ type is used for Unicode characters, it can represent any Unicode character. Numerical constants ------------------- -`Numerical constants`:idx: are of a single type and have the form:: +Numerical constants are of a single type and have the form:: hexdigit = digit | 'A'..'F' | 'a'..'f' octdigit = '0'..'7' @@ -440,7 +441,7 @@ is approximately 1.72826e35 according to the IEEE floating point standard. Operators --------- -In Nimrod one can define his own operators. An `operator`:idx: is any +In Nimrod one can define his own operators. An operator is any combination of the following characters:: = + - * / < > @@ -587,7 +588,7 @@ The grammar's start symbol is ``module``. Types ===== -All expressions have a `type`:idx: which is known at compile time. Nimrod +All expressions have a type which is known at compile time. Nimrod is statically typed. One can declare new types, which is in essence defining an identifier that can be used to denote this custom type. @@ -605,7 +606,7 @@ These are the major type classes: Ordinal types ------------- -`Ordinal types`:idx: have the following characteristics: +Ordinal types have the following characteristics: - Ordinal types are countable and ordered. This property allows the operation of functions as ``Inc``, ``Ord``, ``Dec`` on ordinal types to @@ -617,7 +618,7 @@ Ordinal types Integers, bool, characters and enumeration types (and subranges of these types) belong to ordinal types. For reasons of simplicity of implementation -the types ``uint`` and ``uint64`` are no ordinal types. +the types ``uint`` and ``uint64`` are not ordinal types. Pre-defined integer types @@ -686,7 +687,7 @@ kinds of integer types are used: the smaller type is converted to the larger. A `narrowing type conversion`:idx: converts a larger to a smaller type (for example ``int32 -> int16``. A `widening type conversion`:idx: converts a smaller type to a larger type (for example ``int16 -> int32``). In Nimrod only -widening type conversion are *implicit*: +widening type conversions are *implicit*: .. code-block:: nimrod var myInt16 = 5i16 @@ -705,7 +706,7 @@ For further details, see `Convertible relation`_. Subrange types -------------- -A `subrange`:idx: type is a range of values from an ordinal type (the base +A subrange type is a range of values from an ordinal type (the base type). To define a subrange type, one must specify it's limiting values: the lowest and highest value of the type: @@ -806,7 +807,7 @@ the ``+``, ``-``, ``*``, ``/`` operators for floating point types. Boolean type ------------ -The `boolean`:idx: type is named `bool`:idx: in Nimrod and can be one of the two +The boolean type is named `bool`:idx: in Nimrod and can be one of the two pre-defined values ``true`` and ``false``. Conditions in while, if, elif, when statements need to be of type bool. @@ -830,7 +831,7 @@ The size of the bool type is one byte. Character type -------------- -The `character type`:idx: is named ``char`` in Nimrod. Its size is one byte. +The character type is named ``char`` in Nimrod. Its size is one byte. Thus it cannot represent an UTF-8 character, but a part of it. The reason for this is efficiency: for the overwhelming majority of use-cases, the resulting programs will still handle UTF-8 properly as UTF-8 was specially @@ -845,7 +846,7 @@ character. ``TRune`` is declared in the `unicode module <unicode.html>`_. Enumeration types ----------------- -`Enumeration`:idx: types define a new type whose values consist of the ones +Enumeration types define a new type whose values consist of the ones specified. The values are ordered. Example: .. code-block:: nimrod @@ -915,7 +916,7 @@ via ``TMyEnum.value``: String type ----------- -All string literals are of the type `string`:idx:. A string in Nimrod is very +All string literals are of the type ``string``. A string in Nimrod is very similar to a sequence of characters. However, strings in Nimrod are both zero-terminated and have a length field. One can retrieve the length with the builtin ``len`` procedure; the length never counts the terminating zero. @@ -942,7 +943,7 @@ i-th *unichar*. The iterator ``runes`` from the `unicode module CString type ------------ -The `cstring`:idx: type represents a pointer to a zero-terminated char array +The ``cstring`` type represents a pointer to a zero-terminated char array compatible to the type ``char*`` in Ansi C. Its primary purpose lies in easy interfacing with C. The index operation ``s[i]`` means the i-th *char* of ``s``; however no bounds checking for ``cstring`` is performed making the @@ -965,23 +966,31 @@ stack roots conservatively. One can use the builtin procs ``GC_ref`` and ``GC_unref`` to keep the string data alive for the rare cases where it does not work. +A `$` proc is defined for cstrings that returns a string. Thus to get a nimrod +string from a cstring: + +.. code-block:: nimrod + var str: string = "Hello!" + var cstr: cstring = s + var newstr: string = $cstr + Structured types ---------------- -A variable of a `structured type`:idx: can hold multiple values at the same +A variable of a structured type can hold multiple values at the same time. Structured types can be nested to unlimited levels. Arrays, sequences, tuples, objects and sets belong to the structured types. Array and sequence types ------------------------ -`Arrays`:idx: are a homogeneous type, meaning that each element in the array +Arrays are a homogeneous type, meaning that each element in the array has the same type. Arrays always have a fixed length which is specified at compile time (except for open arrays). They can be indexed by any ordinal type. A parameter ``A`` may be an *open array*, in which case it is indexed by integers from 0 to ``len(A)-1``. An array expression may be constructed by the array constructor ``[]``. -`Sequences`:idx: are similar to arrays but of dynamic length which may change +Sequences are similar to arrays but of dynamic length which may change during runtime (like strings). Sequences are implemented as growable arrays, allocating pieces of memory as items are added. A sequence ``S`` is always indexed by integers from 0 to ``len(S)-1`` and its bounds are checked. @@ -1038,7 +1047,7 @@ supported because this is seldom needed and cannot be done efficiently. Varargs ------- -A `varargs`:idx: parameter is an openarray parameter that additionally +A ``varargs`` parameter is an openarray parameter that additionally allows to pass a variable number of arguments to a procedure. The compiler converts the list of arguments to an array implicitly: @@ -1073,7 +1082,7 @@ parameter ``a``. (Note that ``$`` applied to strings is a nop.) Tuples and object types ----------------------- -A variable of a `tuple`:idx: or `object`:idx: type is a heterogeneous storage +A variable of a tuple or object type is a heterogeneous storage container. A tuple or object defines various named *fields* of a type. A tuple also defines an *order* of the fields. Tuples are meant for heterogeneous storage @@ -1081,7 +1090,8 @@ types with no overhead and few abstraction possibilities. The constructor ``()`` can be used to construct tuples. The order of the fields in the constructor must match the order of the tuple's definition. Different tuple-types are *equivalent* if they specify the same fields of the same type in the same -order. +order. The *names* of the fields also have to be identical but this might +change in a future version of the language. The assignment operator for tuples copies each component. The default assignment operator for objects copies each component. Overloading @@ -1153,7 +1163,7 @@ For a ``ref object`` type ``system.new`` is invoked implicitly. Object variants --------------- Often an object hierarchy is overkill in certain situations where simple -`variant`:idx: types are needed. +variant types are needed. An example: @@ -1211,7 +1221,7 @@ branch switch ``system.reset`` has to be used. Set type -------- -The `set type`:idx: models the mathematical notion of a set. The set's +The set type models the mathematical notion of a set. The set's basetype can only be an ordinal type. The reason is that sets are implemented as high performance bit vectors. @@ -1246,7 +1256,7 @@ operation meaning Reference and pointer types --------------------------- -References (similar to `pointers`:idx: in other programming languages) are a +References (similar to pointers in other programming languages) are a way to introduce many-to-one relationships. This means different references can point to and modify the same location in memory (also called `aliasing`:idx:). @@ -1344,7 +1354,7 @@ Not nil annotation ------------------ All types for that ``nil`` is a valid value can be annotated to -exclude ``nil`` as a valid value with the `not nil`:idx: annotation: +exclude ``nil`` as a valid value with the ``not nil`` annotation: .. code-block:: nimrod type @@ -1369,7 +1379,7 @@ here. Memory regions -------------- -The types ``ref`` and ``ptr`` can get an optional `region`:idx: annotation. +The types ``ref`` and ``ptr`` can get an optional ``region`` annotation. A region has to be an object type. Regions are very useful to separate user space and kernel memory in the @@ -1428,7 +1438,7 @@ Future directions: Procedural type --------------- -A `procedural type`:idx: is internally a pointer to a procedure. ``nil`` is +A procedural type is internally a pointer to a procedure. ``nil`` is an allowed value for variables of a procedural type. Nimrod uses procedural types to achieve `functional`:idx: programming techniques. @@ -1519,7 +1529,7 @@ Most calling conventions exist only for the Windows 32-bit platform. Assigning/passing a procedure to a procedural variable is only allowed if one of the following conditions hold: -1) The procedure that is accessed resists in the current module. +1) The procedure that is accessed resides in the current module. 2) The procedure is marked with the ``procvar`` pragma (see `procvar pragma`_). 3) The procedure has a calling convention that differs from ``nimcall``. 4) The procedure is anonymous. @@ -1527,8 +1537,8 @@ of the following conditions hold: The rules' purpose is to prevent the case that extending a non-``procvar`` procedure with default parameters breaks client code. -The default calling convention is ``nimcall``, unless it is an inner proc ( -a proc inside of a proc). For an inner proc an analysis is performed whether it +The default calling convention is ``nimcall``, unless it is an inner proc (a +proc inside of a proc). For an inner proc an analysis is performed whether it accesses its environment. If it does so, it has the calling convention ``closure``, otherwise it has the calling convention ``nimcall``. @@ -1536,12 +1546,16 @@ accesses its environment. If it does so, it has the calling convention Distinct type ------------- -A `distinct type`:idx: is new type derived from a `base type`:idx: that is +A ``distinct`` type is new type derived from a `base type`:idx: that is incompatible with its base type. In particular, it is an essential property of a distinct type that it **does not** imply a subtype relation between it and its base type. Explicit type conversions from a distinct type to its base type and vice versa are allowed. + +Modelling currencies +~~~~~~~~~~~~~~~~~~~~ + A distinct type can be used to model different physical `units`:idx: with a numerical base type, for example. The following example models currencies. @@ -1649,12 +1663,73 @@ certain builtin operations to be lifted: Currently only the dot accessor can be borrowed in this way. +Avoiding SQL injection attacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An SQL statement that is passed from Nimrod to an SQL database might be +modelled as a string. However, using string templates and filling in the +values is vulnerable to the famous `SQL injection attack`:idx:\: + +.. code-block:: nimrod + import strutils + + proc query(db: TDbHandle, statement: string) = ... + + var + username: string + + db.query("SELECT FROM users WHERE name = '$1'" % username) + # Horrible security hole, but the compiler does not mind! + +This can be avoided by distinguishing strings that contain SQL from strings +that don't. Distinct types provide a means to introduce a new string type +``TSQL`` that is incompatible with ``string``: + +.. code-block:: nimrod + type + TSQL = distinct string + + proc query(db: TDbHandle, statement: TSQL) = ... + + var + username: string + + db.query("SELECT FROM users WHERE name = '$1'" % username) + # Error at compile time: `query` expects an SQL string! + + +It is an essential property of abstract types that they **do not** imply a +subtype relation between the abtract type and its base type. Explict type +conversions from ``string`` to ``TSQL`` are allowed: + +.. code-block:: nimrod + import strutils, sequtils + + proc properQuote(s: string): TSQL = + # quotes a string properly for an SQL statement + return TSQL(s) + + proc `%` (frmt: TSQL, values: openarray[string]): TSQL = + # quote each argument: + let v = values.mapIt(TSQL, properQuote(it)) + # we need a temporary type for the type conversion :-( + type TStrSeq = seq[string] + # call strutils.`%`: + result = TSQL(string(frmt) % TStrSeq(v)) + + db.query("SELECT FROM users WHERE name = '$1'".TSQL % [username]) + +Now we have compile-time checking against SQL injection attacks. Since +``"".TSQL`` is transformed to ``TSQL("")`` no new syntax is needed for nice +looking ``TSQL`` string literals. The hypothetical ``TSQL`` type actually +exists in the library as the `TSqlQuery type <db_sqlite.html#TSqlQuery>`_ of +modules like `db_sqlite <db_sqlite.html>`_. Void type --------- -The `void`:idx: type denotes the absense of any type. Parameters of +The ``void`` type denotes the absense of any type. Parameters of type ``void`` are treated as non-existent, ``void`` as a return type means that the procedure does not return a value: @@ -1887,7 +1962,7 @@ To be written. Statements and expressions ========================== -Nimrod uses the common statement/expression paradigm: `Statements`:idx: do not +Nimrod uses the common statement/expression paradigm: Statements do not produce a value in contrast to expressions. However, some expressions are statements. @@ -1904,7 +1979,7 @@ Statement list expression Statements can also occur in an expression context that looks like ``(stmt1; stmt2; ...; ex)``. This is called -an `statement list expression`:idx: or ``(;)``. The type +an statement list expression or ``(;)``. The type of ``(stmt1; stmt2; ...; ex)`` is the type of ``ex``. All the other statements must be of type ``void``. (One can use ``discard`` to produce a ``void`` type.) ``(;)`` does not introduce a new scope. @@ -1921,7 +1996,7 @@ Example: discard p(3, 4) # discard the return value of `p` -The `discard`:idx: statement evaluates its expression for side-effects and +The ``discard`` statement evaluates its expression for side-effects and throws the expression's resulting value away. Ignoring the return value of a procedure without using a discard statement is @@ -1949,7 +2024,7 @@ An empty ``discard`` statement is often used as a null statement: Var statement ------------- -`Var`:idx: statements declare new local and global variables and +Var statements declare new local and global variables and initialize them. A comma separated list of variables can be used to specify variables of the same type: @@ -2018,7 +2093,7 @@ initialized and does not rely on syntactic properties: let statement ------------- -A `Let`:idx: statement declares new local and global `single assignment`:idx: +A ``let`` statement declares new local and global `single assignment`:idx: variables and binds a value to them. The syntax is the of the ``var`` statement, except that the keyword ``var`` is replaced by the keyword ``let``. Let variables are not l-values and can thus not be passed to ``var`` parameters @@ -2060,7 +2135,7 @@ they contain such a type. Static statement/expression --------------------------- -A `static`:idx: statement/expression can be used to enforce compile +A static statement/expression can be used to enforce compile time evaluation explicitly. Enforced compile time evaluation can even evaluate code that has side effects: @@ -2094,7 +2169,7 @@ Example: else: echo("Boring name...") -The `if`:idx: statement is a simple way to make a branch in the control flow: +The ``if`` statement is a simple way to make a branch in the control flow: The expression after the keyword ``if`` is evaluated, if it is true the corresponding statements after the ``:`` are executed. Otherwise the expression after the ``elif`` is evaluated (if there is an @@ -2140,7 +2215,7 @@ Example: else: echo("unknown command") -The `case`:idx: statement is similar to the if statement, but it represents +The ``case`` statement is similar to the if statement, but it represents a multi-branch selection. The expression after the keyword ``case`` is evaluated and if its value is in a *slicelist* the corresponding statements (after the ``of`` keyword) are executed. If the value is not in any @@ -2153,12 +2228,12 @@ type. If the expression is not of an ordinal type, and no ``else`` part is given, control passes after the ``case`` statement. -To suppress the static error in the ordinal case an ``else`` part with a ``nil`` -statement can be used. +To suppress the static error in the ordinal case an ``else`` part with an +empty ``discard`` statement can be used. As a special semantic extension, an expression in an ``of`` branch of a case -statement may evaluate to a set constructor; the set is then expanded into -a list of its elements: +statement may evaluate to a set or array constructor; the set or array is then +expanded into a list of its elements: .. code-block:: nimrod const @@ -2194,7 +2269,7 @@ Example: else: echo("cannot happen!") -The `when`:idx: statement is almost identical to the ``if`` statement with some +The ``when`` statement is almost identical to the ``if`` statement with some exceptions: * Each condition (``expr``) has to be a constant expression (of type ``bool``). @@ -2216,7 +2291,7 @@ Example: .. code-block:: nimrod return 40+2 -The `return`:idx: statement ends the execution of the current procedure. +The ``return`` statement ends the execution of the current procedure. It is only allowed in procedures. If there is an ``expr``, this is syntactic sugar for: @@ -2243,7 +2318,7 @@ Example: .. code-block:: nimrod yield (1, 2, 3) -The `yield`:idx: statement is used instead of the ``return`` statement in +The ``yield`` statement is used instead of the ``return`` statement in iterators. It is only valid in iterators. Execution is returned to the body of the for loop that called the iterator. Yield does not end the iteration process, but execution is passed back to the iterator if the next iteration @@ -2266,7 +2341,7 @@ Example: break myblock # leave the block, in this case both for-loops echo(found) -The block statement is a means to group statements to a (named) `block`:idx:. +The block statement is a means to group statements to a (named) ``block``. Inside the block, the ``break`` statement is allowed to leave the block immediately. A ``break`` statement can contain a name of a surrounding block to specify which block is to leave. @@ -2280,7 +2355,7 @@ Example: .. code-block:: nimrod break -The `break`:idx: statement is used to leave a block immediately. If ``symbol`` +The ``break`` statement is used to leave a block immediately. If ``symbol`` is given, it is the name of the enclosing block that is to leave. If it is absent, the innermost block is left. @@ -2298,7 +2373,7 @@ Example: pw = readLine(stdin) -The `while`:idx: statement is executed until the ``expr`` evaluates to false. +The ``while`` statement is executed until the ``expr`` evaluates to false. Endless loops are no error. ``while`` statements open an `implicit block`, so that they can be left with a ``break`` statement. @@ -2306,7 +2381,7 @@ so that they can be left with a ``break`` statement. Continue statement ------------------ -A `continue`:idx: statement leads to the immediate next iteration of the +A ``continue`` statement leads to the immediate next iteration of the surrounding loop construct. It is only allowed within a loop. A continue statement is syntactic sugar for a nested block: @@ -2329,13 +2404,14 @@ Is equivalent to: Assembler statement ------------------- -The direct embedding of `assembler`:idx: code into Nimrod code is supported +The direct embedding of assembler code into Nimrod code is supported by the unsafe ``asm`` statement. Identifiers in the assembler code that refer to Nimrod identifiers shall be enclosed in a special character which can be specified in the statement's pragmas. The default special character is ``'`'``: .. code-block:: nimrod - proc addInt(a, b: int): int {.noStackFrame.} = + {.push stackTrace:off.} + proc addInt(a, b: int): int = # a in eax, and b in edx asm """ mov eax, `a` @@ -2344,6 +2420,7 @@ specified in the statement's pragmas. The default special character is ``'`'``: call `raiseOverflow` theEnd: """ + {.pop.} If the GNU assembler is used, quotes and newlines are inserted automatically: @@ -2376,7 +2453,7 @@ Using statement **Warning**: The ``using`` statement is highly experimental! -The `using statement`:idx: provides syntactic convenience for procs that +The using statement provides syntactic convenience for procs that heavily use a single contextual parameter. When applied to a variable or a constant, it will instruct Nimrod to automatically consider the used symbol as a hidden leading parameter for any procedure calls, following the using @@ -2461,7 +2538,7 @@ the last expression as the result value, much like in an `expr` template. Table constructor ----------------- -A `table constructor`:idx: is syntactic sugar for an array constructor: +A table constructor is syntactic sugar for an array constructor: .. code-block:: nimrod {"key1": "value1", "key2", "key3": "value2"} @@ -2508,7 +2585,7 @@ only needed for low-level programming and are inherently unsafe. The addr operator ----------------- -The `addr`:idx: operator returns the address of an l-value. If the type of the +The ``addr`` operator returns the address of an l-value. If the type of the location is ``T``, the `addr` operator result is of the type ``ptr T``. An address is always an untraced reference. Taking the address of an object that resides on the stack is **unsafe**, as the pointer may live longer than the @@ -2609,7 +2686,7 @@ For object oriented programming, the syntax ``obj.method(args)`` can be used instead of ``method(obj, args)``. The parentheses can be omitted if there are no remaining arguments: ``obj.len`` (instead of ``len(obj)``). -This `method call syntax`:idx: is not restricted to objects, it can be used +This method call syntax is not restricted to objects, it can be used to supply any type of first argument for procedures: .. code-block:: nimrod @@ -2654,7 +2731,7 @@ Command invocation syntax ------------------------- Routines can be invoked without the ``()`` if the call is syntatically -a statement. This `command invocation syntax`:idx: also works for +a statement. This command invocation syntax also works for expressions, but then only a single argument may follow. This restriction means ``echo f 1, f 2`` is parsed as ``echo(f(1), f(2))`` and not as ``echo(f(1, f(2)))``. The method call syntax may be used to provide one @@ -2748,7 +2825,7 @@ The following builtin procs cannot be overloaded for reasons of implementation simplicity (they require specialized semantic checking):: defined, definedInScope, compiles, low, high, sizeOf, - is, of, echo, shallowCopy, getAst + is, of, echo, shallowCopy, getAst, spawn Thus they act more like keywords than like ordinary identifiers; unlike a keyword however, a redefinition may `shadow`:idx: the definition in @@ -2851,7 +2928,7 @@ The ``[]`` subscript operator for arrays/openarrays/sequences can be overloaded. Multi-methods ============= -Procedures always use static dispatch. `Multi-methods`:idx: use dynamic +Procedures always use static dispatch. Multi-methods use dynamic dispatch. .. code-block:: nimrod @@ -3104,7 +3181,7 @@ Example: line: int # the line the symbol was declared in code: PNode # the symbol's abstract syntax tree -A `type`:idx: section begins with the ``type`` keyword. It contains multiple +A type section begins with the ``type`` keyword. It contains multiple type definitions. A type definition binds a type to a name. Type definitions can be recursive or even mutually recursive. Mutually recursive types are only possible within a single ``type`` section. Nominal types like ``objects`` @@ -3141,7 +3218,7 @@ Example: close(f) -The statements after the `try`:idx: are executed in sequential order unless +The statements after the ``try`` are executed in sequential order unless an exception ``e`` is raised. If the exception type of ``e`` matches any listed in an ``except`` clause the corresponding statements are executed. The statements following the ``except`` clauses are called @@ -3163,7 +3240,7 @@ is not executed (if an exception occurs). Except and finally statements ----------------------------- -`except`:idx: and `finally`:idx: can also be used as a stand-alone statements. +``except`` and ``finally`` can also be used as a stand-alone statements. Any statements following them in the current block will be considered to be in an implicit try block: @@ -3212,9 +3289,9 @@ exception (unless a raise hook has been provided). OnRaise builtin --------------- -``system.onRaise`` can be used to override the behaviour of ``raise`` for a -single ``try`` statement. `onRaise`:idx: has to be called within the ``try`` -statement that should be affected. +`system.onRaise() <system.html#onRaise>`_ can be used to override the +behaviour of ``raise`` for a single ``try`` statement. ``onRaise`` has to be +called within the ``try`` statement that should be affected. This allows for a Lisp-like `condition system`:idx:\: @@ -3243,7 +3320,7 @@ Effect system Exception tracking ------------------ -Nimrod supports `exception tracking`:idx:. The `raises`:idx: pragma can be used +Nimrod supports exception tracking. The `raises`:idx: pragma can be used to explicitly define which exceptions a proc/iterator/method/converter is allowed to raise. The compiler verifies this: @@ -3351,7 +3428,7 @@ exception tracking. Effects pragma -------------- -The `effects`:idx: pragma has been designed to assist the programmer with the +The ``effects`` pragma has been designed to assist the programmer with the effects analysis. It is a statement that makes the compiler output all inferred effects up to the ``effects``'s position: @@ -3420,7 +3497,7 @@ Example: for str in inorder(root): writeln(stdout, str) -`Generics`:idx: are Nimrod's means to parametrize procs, iterators or types with +Generics are Nimrod's means to parametrize procs, iterators or types with `type parameters`:idx:. Depending on context, the brackets are used either to introduce type parameters or to instantiate a generic proc, iterator or type. @@ -3428,7 +3505,7 @@ introduce type parameters or to instantiate a generic proc, iterator or type. Is operator ----------- -The `is`:idx: operator checks for type equivalence at compile time. It is +The ``is`` operator checks for type equivalence at compile time. It is therefore very useful for type specialization within generic code: .. code-block:: nimrod @@ -3443,7 +3520,7 @@ therefore very useful for type specialization within generic code: Type operator ------------- -The `type`:idx: (in many other languages called `typeof`:idx:) operator can +The ``type`` (in many other languages called `typeof`:idx:) operator can be used to get the type of an expression: .. code-block:: nimrod @@ -3466,7 +3543,7 @@ other interpretations: Type Classes ------------ -A `type class`:idx: is a special pseudo-type that can be used to match against +A type class is a special pseudo-type that can be used to match against types in the context of overload resolution or the ``is`` operator. Nimrod supports the following built-in type classes: @@ -3684,7 +3761,7 @@ A symbol can be forced to be open by a `mixin`:idx: declaration: Bind statement -------------- -The `bind`:idx: statement is the counterpart to the ``mixin`` statement. It +The ``bind`` statement is the counterpart to the ``mixin`` statement. It can be used to explicitly declare identifiers that should be bound early (i.e. the identifiers should be looked up in the scope of the template/generic definition): @@ -3712,7 +3789,7 @@ scope is the default. Templates ========= -A `template`:idx: is a simple form of a macro: It is a simple substitution +A template is a simple form of a macro: It is a simple substitution mechanism that operates on Nimrod's abstract syntax trees. It is processed in the semantic pass of the compiler. @@ -3744,7 +3821,7 @@ expected. Ordinary vs immediate templates ------------------------------- -There are two different kinds of templates: `immediate`:idx: templates and +There are two different kinds of templates: immediate templates and ordinary templates. Ordinary templates take part in overloading resolution. As such their arguments need to be type checked before the template is invoked. So ordinary templates cannot receive undeclared identifiers: @@ -3939,7 +4016,7 @@ a template. ``inject`` and ``gensym`` have no effect in ``dirty`` templates. Macros ====== -A `macro`:idx: is a special kind of low level template. Macros can be used +A macro is a special kind of low level template. Macros can be used to implement `domain specific languages`:idx:. Like templates, macros come in the 2 flavors *immediate* and *ordinary*. @@ -4317,7 +4394,7 @@ This operator will be matched against assignments to missing fields. Term rewriting macros ===================== -`Term rewriting macros`:idx: are macros or templates that have not only +Term rewriting macros are macros or templates that have not only a *name* but also a *pattern* that is searched for after the semantic checking phase of the compiler: This means they provide an easy way to enhance the compilation pipeline with user defined optimizations: @@ -4653,7 +4730,7 @@ ordinary routines. Move optimization ----------------- -The ``call`` constraint is particularly useful to implement a `move`:idx: +The ``call`` constraint is particularly useful to implement a move optimization for types that have copying semantics: .. code-block:: nimrod @@ -4679,7 +4756,7 @@ optimization for types that have copying semantics: Modules ======= -Nimrod supports splitting a program into pieces by a `module`:idx: concept. +Nimrod supports splitting a program into pieces by a module concept. Each module needs to be in its own file and has its own `namespace`:idx:. Modules enable `information hiding`:idx: and `separate compilation`:idx:. A module may gain access to symbols of another module by the `import`:idx: @@ -4722,7 +4799,7 @@ This is best illustrated by an example: Import statement ~~~~~~~~~~~~~~~~ -After the `import`:idx: statement a list of module names can follow or a single +After the ``import`` statement a list of module names can follow or a single module name followed by an ``except`` to prevent some symbols to be imported: .. code-block:: nimrod @@ -4765,7 +4842,7 @@ Likewise the following does not make sense as the name is ``strutils`` already: From import statement ~~~~~~~~~~~~~~~~~~~~~ -After the `from`:idx: statement a module name follows followed by +After the ``from`` statement a module name follows followed by an ``import`` to list the symbols one likes to use without explict full qualification: @@ -4784,7 +4861,7 @@ in ``module``. Export statement ~~~~~~~~~~~~~~~~ -An `export`:idx: statement can be used for symbol fowarding so that client +An ``export`` statement can be used for symbol fowarding so that client modules don't need to import a module's dependencies: .. code-block:: nimrod @@ -4812,7 +4889,7 @@ Scope rules ----------- Identifiers are valid from the point of their declaration until the end of the block in which the declaration occurred. The range where the identifier -is known is the `scope`:idx: of the identifier. The exact scope of an +is known is the scope of the identifier. The exact scope of an identifier depends on the way it was declared. Block scope @@ -4885,7 +4962,7 @@ to access the feature becomes available. noSideEffect pragma ------------------- -The `noSideEffect`:idx: pragma is used to mark a proc/iterator to have no side +The ``noSideEffect`` pragma is used to mark a proc/iterator to have no side effects. This means that the proc/iterator only changes locations that are reachable from its parameters and the return value only depends on the arguments. If none of its parameters have the type ``var T`` @@ -4907,7 +4984,7 @@ proc with no side effects: destructor pragma ----------------- -The `destructor`:idx: pragma is used to mark a proc to act as a type destructor. +The ``destructor`` pragma is used to mark a proc to act as a type destructor. The proc must have a single parameter with a concrete type (the name of a generic type is allowed too). @@ -4967,25 +5044,25 @@ the ``finalizer`` parameter to ``new``. procvar pragma -------------- -The `procvar`:idx: pragma is used to mark a proc that it can be passed to a +The ``procvar`` pragma is used to mark a proc that it can be passed to a procedural variable. compileTime pragma ------------------ -The `compileTime`:idx: pragma is used to mark a proc to be used at compile +The ``compileTime`` pragma is used to mark a proc to be used at compile time only. No code will be generated for it. Compile time procs are useful as helpers for macros. noReturn pragma --------------- -The `noreturn`:idx: pragma is used to mark a proc that never returns. +The ``noreturn`` pragma is used to mark a proc that never returns. Acyclic pragma -------------- -The `acyclic`:idx: pragma can be used for object types to mark them as acyclic +The ``acyclic`` pragma can be used for object types to mark them as acyclic even though they seem to be cyclic. This is an **optimization** for the garbage collector to not consider objects of this type as part of a cycle: @@ -5016,13 +5093,13 @@ memory, but nothing worse happens. Final pragma ------------ -The `final`:idx: pragma can be used for an object type to specify that it +The ``final`` pragma can be used for an object type to specify that it cannot be inherited from. shallow pragma -------------- -The `shallow`:idx: pragma affects the semantics of a type: The compiler is +The ``shallow`` pragma affects the semantics of a type: The compiler is allowed to make a shallow copy. This can cause serious semantic issues and break memory safety! However, it can speed up assignments considerably, because the semantics of Nimrod require deep copying of sequences and strings. @@ -5042,14 +5119,14 @@ structure: Pure pragma ----------- -An object type can be marked with the `pure`:idx: pragma so that its type +An object type can be marked with the ``pure`` pragma so that its type field which is used for runtime type identification is omitted. This is necessary for binary compatibility with other compiled languages. AsmNoStackFrame pragma ---------------------- -A proc can be marked with the `AsmNoStackFrame`:idx: pragma to tell the compiler +A proc can be marked with the ``AsmNoStackFrame`` pragma to tell the compiler it should not generate a stack frame for the proc. There are also no exit statements like ``return result;`` generated and the generated C function is declared as ``__declspec(naked)`` or ``__attribute__((naked))`` (depending on @@ -5060,7 +5137,7 @@ assembler statements. error pragma ------------ -The `error`:idx: pragma is used to make the compiler output an error message +The ``error`` pragma is used to make the compiler output an error message with the given content. Compilation does not necessarily abort after an error though. @@ -5076,7 +5153,7 @@ operation is valid due to overloading and type conversions: fatal pragma ------------ -The `fatal`:idx: pragma is used to make the compiler output an error message +The ``fatal`` pragma is used to make the compiler output an error message with the given content. In contrast to the ``error`` pragma, compilation is guaranteed to be aborted by this pragma. Example: @@ -5086,17 +5163,17 @@ is guaranteed to be aborted by this pragma. Example: warning pragma -------------- -The `warning`:idx: pragma is used to make the compiler output a warning message +The ``warning`` pragma is used to make the compiler output a warning message with the given content. Compilation continues after the warning. hint pragma ----------- -The `hint`:idx: pragma is used to make the compiler output a hint message with +The ``hint`` pragma is used to make the compiler output a hint message with the given content. Compilation continues after the hint. line pragma ----------- -The `line`:idx: pragma can be used to affect line information of the annotated +The ``line`` pragma can be used to affect line information of the annotated statement as seen in stack backtraces: .. code-block:: nimrod @@ -5114,7 +5191,7 @@ If the ``line`` pragma is used with a parameter, the parameter needs be a linearScanEnd pragma -------------------- -The `linearScanEnd`:idx: pragma can be used to tell the compiler how to +The ``linearScanEnd`` pragma can be used to tell the compiler how to compile a Nimrod `case`:idx: statement. Syntactically it has to be used as a statement: @@ -5142,7 +5219,7 @@ whole ``case`` statement, the whole ``case`` statement uses linear scanning. computedGoto pragma ------------------- -The `computedGoto`:idx: pragma can be used to tell the compiler how to +The ``computedGoto`` pragma can be used to tell the compiler how to compile a Nimrod `case`:idx: in a ``while true`` statement. Syntactically it has to be used as a statement inside the loop: @@ -5187,7 +5264,7 @@ extension the pragma is simply ignored. unroll pragma ------------- -The `unroll`:idx: pragma can be used to tell the compiler that it should unroll +The ``unroll`` pragma can be used to tell the compiler that it should unroll a `for`:idx: or `while`:idx: loop for runtime efficiency: .. code-block:: nimrod @@ -5266,7 +5343,7 @@ but are used to override the settings temporarily. Example: register pragma --------------- -The `register`:idx: pragma is for variables only. It declares the variable as +The ``register`` pragma is for variables only. It declares the variable as ``register``, giving the compiler a hint that the variable should be placed in a hardware register for faster access. C compilers usually ignore this though and for good reasons: Often they do a better job without it anyway. @@ -5277,7 +5354,7 @@ example) it may provide benefits, though. global pragma ------------- -The `global`:idx: pragma can be applied to a variable within a proc to instruct +The ``global`` pragma can be applied to a variable within a proc to instruct the compiler to store it in a global location and initialize it once at program startup. @@ -5294,7 +5371,7 @@ and before any variable in a module that imports it. DeadCodeElim pragma ------------------- -The `deadCodeElim`:idx: pragma only applies to whole modules: It tells the +The ``deadCodeElim`` pragma only applies to whole modules: It tells the compiler to activate (or deactivate) dead code elimination for the module the pragma appears in. @@ -5312,7 +5389,7 @@ Example: .. NoForward pragma ---------------- - The `noforward`:idx: pragma can be used to turn on and off a special compilation + The ``noforward`` pragma can be used to turn on and off a special compilation mode that to large extent eliminates the need for forward declarations. In this mode, the proc definitions may appear out of order and the compiler will postpone their semantic analysis and compilation until it actually needs to generate code @@ -5360,7 +5437,7 @@ Example: Pragma pragma ------------- -The `pragma`:idx: pragma can be used to declare user defined pragmas. This is +The ``pragma`` pragma can be used to declare user defined pragmas. This is useful because Nimrod's templates and macros do not affect pragmas. User defined pragmas are in a different module-wide scope than all other symbols. They cannot be imported from a module. @@ -5389,7 +5466,7 @@ and warning message contains a symbol in brackets. This is the message's identifier that can be used to enable or disable it: .. code-block:: Nimrod - {.warning[LineTooLong]: off.} # turn off warning about too long lines + {.hint[LineTooLong]: off.} # turn off the hint about too long lines This is often better than disabling all warnings at once. @@ -5404,21 +5481,24 @@ are documented here. Importc pragma -------------- -The `importc`:idx: pragma provides a means to import a proc or a variable +The ``importc`` pragma provides a means to import a proc or a variable from C. The optional argument is a string containing the C identifier. If the argument is missing, the C name is the Nimrod identifier *exactly as spelled*: .. code-block:: - proc printf(formatstr: cstring) {.importc: "printf", varargs.} + proc printf(formatstr: cstring) {.header: "<stdio.h>", importc: "printf", varargs.} Note that this pragma is somewhat of a misnomer: Other backends will provide -the same feature under the same name. +the same feature under the same name. Also, if you are interfacing with C++ +you can use the `ImportCpp pragma <nimrodc.html#importcpp-pragma>`_ and +interfacing with Objective-C the `ImportObjC pragma +<nimrodc.html#importobjc-pragma>`_. Exportc pragma -------------- -The `exportc`:idx: pragma provides a means to export a type, a variable, or a +The ``exportc`` pragma provides a means to export a type, a variable, or a procedure to C. Enums and constants can't be exported. The optional argument is a string containing the C identifier. If the argument is missing, the C name is the Nimrod identifier *exactly as spelled*: @@ -5432,7 +5512,7 @@ the same feature under the same name. Extern pragma ------------- -Like ``exportc`` or ``importc`` the `extern`:idx: pragma affects name +Like ``exportc`` or ``importc``, the ``extern`` pragma affects name mangling. The string literal passed to ``extern`` can be a format string: .. code-block:: Nimrod @@ -5445,7 +5525,7 @@ In the example the external name of ``p`` is set to ``prefixp``. Bycopy pragma ------------- -The `bycopy`:idx: pragma can be applied to an object or tuple type and +The ``bycopy`` pragma can be applied to an object or tuple type and instructs the compiler to pass the type by value to procs: .. code-block:: nimrod @@ -5457,13 +5537,13 @@ instructs the compiler to pass the type by value to procs: Byref pragma ------------ -The `byref`:idx: pragma can be applied to an object or tuple type and instructs +The ``byref`` pragma can be applied to an object or tuple type and instructs the compiler to pass the type by reference (hidden pointer) to procs. Varargs pragma -------------- -The `varargs`:idx: pragma can be applied to procedures only (and procedure +The ``varargs`` pragma can be applied to procedures only (and procedure types). It tells Nimrod that the proc can take a variable number of parameters after the last specified parameter. Nimrod string values will be converted to C strings automatically: @@ -5476,7 +5556,7 @@ strings automatically: Union pragma ------------ -The `union`:idx: pragma can be applied to any ``object`` type. It means all +The ``union`` pragma can be applied to any ``object`` type. It means all of the object's fields are overlaid in memory. This produces a ``union`` instead of a ``struct`` in the generated C/C++ code. The object declaration then must not use inheritance or any GC'ed memory but this is currently not @@ -5487,7 +5567,7 @@ should scan unions conservatively. Packed pragma ------------- -The `packed`:idx: pragma can be applied to any ``object`` type. It ensures +The ``packed`` pragma can be applied to any ``object`` type. It ensures that the fields of an object are packed back-to-back in memory. It is useful to store packets or messages from/to network or hardware drivers, and for interoperability with C. Combining packed pragma with inheritance is not @@ -5498,7 +5578,7 @@ compile-time error. Usage with inheritance should be defined and documented. Unchecked pragma ---------------- -The `unchecked`:idx: pragma can be used to mark a named array as ``unchecked`` +The ``unchecked`` pragma can be used to mark a named array as ``unchecked`` meaning its bounds are not checked. This is often useful when one wishes to implement his own flexibly sized arrays. Additionally an unchecked array is translated into a C array of undetermined size: @@ -5533,7 +5613,7 @@ runtime size of the array. Dynlib pragma for import ------------------------ -With the `dynlib`:idx: pragma a procedure or a variable can be imported from +With the ``dynlib`` pragma a procedure or a variable can be imported from a dynamic library (``.dll`` files for Windows, ``lib*.so`` files for UNIX). The non-optional argument has to be the name of the dynamic library: @@ -5623,7 +5703,7 @@ Thread pragma ------------- A proc that is executed as a new thread of execution should be marked by the -`thread pragma`:idx:. The compiler checks procedures marked as ``thread`` for +``thread`` pragma. The compiler checks procedures marked as ``thread`` for violations of the `no heap sharing restriction`:idx:\: This restriction implies that it is invalid to construct a data structure that consists of memory allocated from different (thread local) heaps. @@ -5664,7 +5744,7 @@ Future directions: Threadvar pragma ---------------- -A global variable can be marked with the `threadvar`:idx: pragma; it is +A global variable can be marked with the ``threadvar`` pragma; it is a `thread-local`:idx: variable then: .. code-block:: nimrod @@ -5688,7 +5768,7 @@ Spawn Nimrod has a builtin thread pool that can be used for CPU intensive tasks. For IO intensive tasks the upcoming ``async`` and ``await`` features should be -used. `spawn`:idx: is used to pass a task to the thread pool: +used instead. `spawn`:idx: is used to pass a task to the thread pool: .. code-block:: nimrod proc processLine(line: string) = @@ -5716,7 +5796,7 @@ Taint mode ========== The Nimrod compiler and most parts of the standard library support -a `taint mode`:idx:. Input strings are declared with the `TaintedString`:idx: +a taint mode. Input strings are declared with the `TaintedString`:idx: string type declared in the ``system`` module. If the taint mode is turned on (via the ``--taintMode:on`` command line diff --git a/doc/nimgrep.txt b/doc/nimgrep.txt index c231bf31b..67aaa427e 100644 --- a/doc/nimgrep.txt +++ b/doc/nimgrep.txt @@ -1,6 +1,6 @@ -================================= - nimgrep User's manual -================================= +========================= + nimgrep User's manual +========================= :Author: Andreas Rumpf :Version: 0.9 diff --git a/doc/niminst.txt b/doc/niminst.txt index ebed396e4..8d95d01c9 100644 --- a/doc/niminst.txt +++ b/doc/niminst.txt @@ -1,6 +1,6 @@ -================================= - niminst User's manual -================================= +========================= + niminst User's manual +========================= :Author: Andreas Rumpf :Version: |nimrodversion| diff --git a/doc/nimrodc.txt b/doc/nimrodc.txt index 52e0a6eaf..428c42f39 100644 --- a/doc/nimrodc.txt +++ b/doc/nimrodc.txt @@ -87,6 +87,18 @@ Level Description for compiler developers. ===== ============================================ + +Compile time symbols +-------------------- + +Through the ``-d:x`` or ``--define:x`` switch you can define compile time +symbols for conditional compilation. The defined switches can be checked in +source code with the `when statement <manual.html#when-statement>`_ and +`defined proc <system.html#defined>`_. The typical use of this switch is to +enable builds in release mode (``-d:release``) where certain safety checks are +omitted for better performance. Another common use is the ``-d:ssl`` switch to +activate `SSL sockets <sockets.html>`_. + Configuration files ------------------- @@ -116,7 +128,7 @@ The default build of a project is a `debug build`:idx:. To compile a Search path handling -------------------- -Nimrod has the concept of a global `search path`:idx: (PATH) that is queried to +Nimrod has the concept of a global search path (PATH) that is queried to determine where to find imported modules or include files. If multiple files are found an ambiguity error is produced. @@ -167,14 +179,14 @@ might contain some cruft even when dead code elimination is turned on. So the final release build should be done with ``--symbolFiles:off``. Due to the aggregation of C code it is also recommended that each project -resists in its own directory so that the generated ``nimcache`` directory +resides in its own directory so that the generated ``nimcache`` directory is not shared between different projects. Cross compilation ================= -To `cross compile`:idx:, use for example:: +To cross compile, use for example:: nimrod c --cpu:i386 --os:linux --compile_only --gen_script myproject.nim @@ -200,7 +212,7 @@ DLL generation Nimrod supports the generation of DLLs. However, there must be only one instance of the GC per process/address space. This instance is contained in -``nimrtl.dll``. This means that every generated Nimrod `DLL`:idx: depends +``nimrtl.dll``. This means that every generated Nimrod DLL depends on ``nimrtl.dll``. To generate the "nimrtl.dll" file, use the command:: nimrod c -d:release lib/nimrtl.nim @@ -251,7 +263,7 @@ generator and are subject to change. NoDecl pragma ------------- -The `noDecl`:idx: pragma can be applied to almost any symbol (variable, proc, +The ``noDecl`` pragma can be applied to almost any symbol (variable, proc, type, etc.) and is sometimes useful for interoperability with C: It tells Nimrod that it should not generate a declaration for the symbol in the C code. For example: @@ -268,7 +280,7 @@ However, the ``header`` pragma is often the better alternative. Header pragma ------------- -The `header`:idx: pragma is very similar to the ``noDecl`` pragma: It can be +The ``header`` pragma is very similar to the ``noDecl`` pragma: It can be applied to almost any symbol and specifies that it should not be declared and instead the generated code should contain an ``#include``: @@ -287,7 +299,7 @@ encloses the header file in ``""`` in the generated C code. IncompleteStruct pragma ----------------------- -The `incompleteStruct`:idx: pragma tells the compiler to not use the +The ``incompleteStruct`` pragma tells the compiler to not use the underlying C ``struct`` in a ``sizeof`` expression: .. code-block:: Nimrod @@ -298,7 +310,7 @@ underlying C ``struct`` in a ``sizeof`` expression: Compile pragma -------------- -The `compile`:idx: pragma can be used to compile and link a C/C++ source file +The ``compile`` pragma can be used to compile and link a C/C++ source file with the project: .. code-block:: Nimrod @@ -311,7 +323,7 @@ of the file. Link pragma ----------- -The `link`:idx: pragma can be used to link an additional file with the project: +The ``link`` pragma can be used to link an additional file with the project: .. code-block:: Nimrod {.link: "myfile.o".} @@ -319,7 +331,7 @@ The `link`:idx: pragma can be used to link an additional file with the project: PassC pragma ------------ -The `passC`:idx: pragma can be used to pass additional parameters to the C +The ``passC`` pragma can be used to pass additional parameters to the C compiler like you would using the commandline switch ``--passC``: .. code-block:: Nimrod @@ -333,7 +345,7 @@ embed parameters from an external command at compile time: PassL pragma ------------ -The `passL`:idx: pragma can be used to pass additional parameters to the linker +The ``passL`` pragma can be used to pass additional parameters to the linker like you would using the commandline switch ``--passL``: .. code-block:: Nimrod @@ -348,7 +360,7 @@ embed parameters from an external command at compile time: Emit pragma ----------- -The `emit`:idx: pragma can be used to directly affect the output of the +The ``emit`` pragma can be used to directly affect the output of the compiler's code generator. So it makes your code unportable to other code generators/backends. Its usage is highly discouraged! However, it can be extremely useful for interfacing with `C++`:idx: or `Objective C`:idx: code. @@ -360,20 +372,23 @@ Example: static int cvariable = 420; """.} - proc embedsC() {.noStackFrame.} = + {.push stackTrace:off.} + proc embedsC() = var nimrodVar = 89 # use backticks to access Nimrod symbols within an emit section: {.emit: """fprintf(stdout, "%d\n", cvariable + (int)`nimrodVar`);""".} + {.pop.} embedsC() ImportCpp pragma ---------------- -The `importcpp`:idx: pragma can be used to import `C++`:idx: methods. The -generated code then uses the C++ method calling syntax: ``obj->method(arg)``. -In addition with the ``header`` and ``emit`` pragmas this allows *sloppy* -interfacing with libraries written in C++: +Similar to the `importc pragma for C <manual.html#importc-pragma>`_, the +``importcpp`` pragma can be used to import `C++`:idx: methods. The generated +code then uses the C++ method calling syntax: ``obj->method(arg)``. In +addition with the ``header`` and ``emit`` pragmas this allows *sloppy* +interfacing with libraries written in C++: .. code-block:: Nimrod # Horrible example of how to interface with a C++ engine ... ;-) @@ -408,11 +423,11 @@ emits C++ code. ImportObjC pragma ----------------- -The `importobjc`:idx: pragma can be used to import `Objective C`:idx: methods. -The generated code then uses the Objective C method calling -syntax: ``[obj method param1: arg]``. -In addition with the ``header`` and ``emit`` pragmas this allows *sloppy* -interfacing with libraries written in Objective C: +Similar to the `importc pragma for C <manual.html#importc-pragma>`_, the +``importobjc`` pragma can be used to import `Objective C`:idx: methods. The +generated code then uses the Objective C method calling syntax: ``[obj method +param1: arg]``. In addition with the ``header`` and ``emit`` pragmas this +allows *sloppy* interfacing with libraries written in Objective C: .. code-block:: Nimrod # horrible example of how to interface with GNUStep ... @@ -458,7 +473,7 @@ emits Objective C code. CodegenDecl pragma ------------------ -The `codegenDecl`:idx: pragma can be used to directly influence Nimrod's code +The ``codegenDecl`` pragma can be used to directly influence Nimrod's code generator. It receives a format string that determines how the variable or proc is declared in the generated code: @@ -473,7 +488,7 @@ proc is declared in the generated code: InjectStmt pragma ----------------- -The `injectStmt`:idx: pragma can be used to inject a statement before every +The ``injectStmt`` pragma can be used to inject a statement before every other statement in the current module. It is only supposed to be used for debugging: @@ -485,28 +500,28 @@ debugging: LineDir option -------------- -The `lineDir`:idx: option can be turned on or off. If turned on the +The ``lineDir`` option can be turned on or off. If turned on the generated C code contains ``#line`` directives. This may be helpful for debugging with GDB. StackTrace option ----------------- -If the `stackTrace`:idx: option is turned on, the generated C contains code to +If the ``stackTrace`` option is turned on, the generated C contains code to ensure that proper stack traces are given if the program crashes or an uncaught exception is raised. LineTrace option ---------------- -The `lineTrace`:idx: option implies the ``stackTrace`` option. If turned on, +The ``lineTrace`` option implies the ``stackTrace`` option. If turned on, the generated C contains code to ensure that proper stack traces with line number information are given if the program crashes or an uncaught exception is raised. Debugger option --------------- -The `debugger`:idx: option enables or disables the *Embedded Nimrod Debugger*. +The ``debugger`` option enables or disables the *Embedded Nimrod Debugger*. See the documentation of endb_ for further information. @@ -518,7 +533,7 @@ ENDB. See the documentation of `endb <endb.html>`_ for further information. Volatile pragma --------------- -The `volatile`:idx: pragma is for variables only. It declares the variable as +The ``volatile`` pragma is for variables only. It declares the variable as ``volatile``, whatever that means in C/C++ (its semantics are not well defined in C/C++). @@ -530,14 +545,24 @@ DynlibOverride By default Nimrod's ``dynlib`` pragma causes the compiler to generate ``GetProcAddress`` (or their Unix counterparts) -calls to bind to a DLL. With the `dynlibOverride`:idx: command line switch this +calls to bind to a DLL. With the ``dynlibOverride`` command line switch this can be prevented and then via ``--passL`` the static library can be linked against. For instance, to link statically against Lua this command might work on Linux:: nimrod c --dynlibOverride:lua --passL:liblua.lib program.nim - - + + +Backend language options +======================== + +The typical compiler usage involves using the ``compile`` or ``c`` command to +transform a ``.nim`` file into one or more ``.c`` files which are then +compiled with the platform's C compiler into a static binary. However there +are other commands to compile to C++, Objective-C or Javascript. More details +can be read in the `Nimrod Backend Integration document <backends.html>`_. + + Nimrod documentation tools ========================== @@ -556,7 +581,7 @@ for further information. Nimrod interactive mode ======================= -The Nimrod compiler supports an `interactive mode`:idx:. This is also known as +The Nimrod compiler supports an interactive mode. This is also known as a `REPL`:idx: (*read eval print loop*). If Nimrod has been built with the ``-d:useGnuReadline`` switch, it uses the GNU readline library for terminal input management. To start Nimrod in interactive mode use the command @@ -681,35 +706,3 @@ efficient: else: quit(errorStr(p, "expected: console or gui")) of "license": c.license = UnixToNativePath(k.value) else: quit(errorStr(p, "unknown variable: " & k.key)) - - -The JavaScript target -===================== - -Nimrod can also generate `JavaScript`:idx: code. However, the -JavaScript code generator is experimental! - -Nimrod targets JavaScript 1.5 which is supported by any widely used browser. -Since JavaScript does not have a portable means to include another module, -Nimrod just generates a long ``.js`` file. - -Features or modules that the JavaScript platform does not support are not -available. This includes: - -* manual memory management (``alloc``, etc.) -* casting and other unsafe operations (``cast`` operator, ``zeroMem``, etc.) -* file management -* most modules of the Standard library -* proper 64 bit integer arithmetic -* unsigned integer arithmetic - -However, the modules `strutils`:idx:, `math`:idx:, and `times`:idx: are -available! To access the DOM, use the `dom`:idx: module that is only -available for the JavaScript platform. - -To compile a Nimrod module into a ``.js`` file use the ``js`` command; the -default is a ``.js`` file that is supposed to be referenced in an ``.html`` -file. However, you can also run the code with `nodejs`:idx:\: - - nimrod js -d:nodejs -r examples/hallo.nim - diff --git a/doc/spawn.txt b/doc/spawn.txt new file mode 100644 index 000000000..ed500f3a5 --- /dev/null +++ b/doc/spawn.txt @@ -0,0 +1,98 @@ +========================================================== + Parallel & Spawn +========================================================== + +Nimrod has two flavors of parallelism: +1) `Structured`:idx parallelism via the ``parallel`` statement. +2) `Unstructured`:idx: parallelism via the standalone ``spawn`` statement. + +Both need the `threadpool <threadpool.html>`_ module to work. + +Somewhat confusingly, ``spawn`` is also used in the ``parallel`` statement +with slightly different semantics. ``spawn`` always takes a call expression of +the form ``f(a, ...)``. Let ``T`` be ``f``'s return type. If ``T`` is ``void`` +then ``spawn``'s return type is also ``void``. Within a ``parallel`` section +``spawn``'s return type is ``T``, otherwise it is ``FlowVar[T]``. + +The compiler can ensure the location in ``location = spawn f(...)`` is not +read prematurely within a ``parallel`` section and so there is no need for +the overhead of an indirection via ``FlowVar[T]`` to ensure correctness. + + +Parallel statement +================== + +Example: + +.. code-block:: nimrod + # Compute PI in an inefficient way + import strutils, math, threadpool + + proc term(k: float): float = 4 * math.pow(-1, k) / (2*k + 1) + + proc pi(n: int): float = + var ch = newSeq[float](n+1) + parallel: + for k in 0..ch.high: + ch[k] = spawn term(float(k)) + for k in 0..ch.high: + result += ch[k] + + echo formatFloat(pi(5000)) + + +The parallel statement is the preferred mechanism to introduce parallelism +in a Nimrod program. A subset of the Nimrod language is valid within a +``parallel`` section. This subset is checked to be free of data races at +compile time. A sophisticated `disjoint checker`:idx: ensures that no data +races are possible even though shared memory is extensively supported! + +The subset is in fact the full language with the following +restrictions / changes: + +* ``spawn`` within a ``parallel`` section has special semantics. +* Every location of the form ``a[i]`` and ``a[i..j]`` and ``dest`` where + ``dest`` is part of the pattern ``dest = spawn f(...)`` has to be + provable disjoint. This is called the *disjoint check*. +* Every other complex location ``loc`` that is used in a spawned + proc (``spawn f(loc)``) has to be immutable for the duration of + the ``parallel`` section. This is called the *immutability check*. Currently + it is not specified what exactly "complex location" means. We need to make + this an optimization! +* Every array access has to be provable within bounds. This is called + the *bounds check*. +* Slices are optimized so that no copy is performed. This optimization is not + yet performed for ordinary slices outside of a ``parallel`` section. Slices + are also special in that they currently do not support negative indexes! + + + + +Spawn statement +=============== + +A standalone ``spawn`` statement is a simple construct. It executes +the passed expression on the thread pool and returns a `data flow variable`:idx: +``FlowVar[T]`` that can be read from. The reading with the ``^`` operator is +**blocking**. However, one can use ``awaitAny`` to wait on multiple flow +variables at the same time: + +.. code-block:: nimrod + import threadpool, ... + + # wait until 2 out of 3 servers received the update: + proc main = + var responses = newSeq[RawFlowVar](3) + for i in 0..2: + responses[i] = spawn tellServer(Update, "key", "value") + var index = awaitAny(responses) + assert index >= 0 + responses.del(index) + discard awaitAny(responses) + +Like the ``parallel`` statement data flow variables ensure that no data races +are possible. Due to technical limitations not every type ``T`` is possible in +a data flow variable: ``T`` has to be of the type ``ref``, ``string``, ``seq`` +or of a type that doesn't contain a type that is garbage collected. This +restriction will be removed in the future. + diff --git a/doc/subexes.txt b/doc/subexes.txt index 10e0f4cc1..3b688fd0a 100644 --- a/doc/subexes.txt +++ b/doc/subexes.txt @@ -1,4 +1,3 @@ -================================ Substitution Expressions (subex) ================================ diff --git a/doc/tools.txt b/doc/tools.txt index ae739e778..0127bc4a8 100644 --- a/doc/tools.txt +++ b/doc/tools.txt @@ -1,6 +1,6 @@ -===== -Tools -===== +=========================== +Tools available with Nimrod +=========================== The standard distribution ships with the following tools: diff --git a/doc/trmacros.txt b/doc/trmacros.txt deleted file mode 100644 index d5ad74e6e..000000000 --- a/doc/trmacros.txt +++ /dev/null @@ -1,275 +0,0 @@ -========================================================= - Term rewriting macros for Nimrod -========================================================= - -:Author: Andreas Rumpf - -Term rewriting macros are macros or templates that have not only a *name* but -also a *pattern* that is searched for after the semantic checking phase of -the compiler: This means they provide an easy way to enhance the compilation -pipeline with user defined optimizations: - -.. code-block:: nimrod - template optMul{`*`(a, 2)}(a: int): int = a+a - - let x = 3 - echo x * 2 - -The compiler now rewrites ``x * 2`` as ``x + x``. The code inside the -curlies is the pattern to match against. The operators ``*``, ``**``, -``|``, ``~`` have a special meaning in patterns if they are written in infix -notation, so to match verbatim against ``*`` the ordinary function call syntax -needs to be used. - - -Unfortunately optimizations are hard to get right and even the tiny example -is **wrong**: - -.. code-block:: nimrod - template optMul{`*`(a, 2)}(a: int): int = a+a - - proc f(): int = - echo "side effect!" - result = 55 - - echo f() * 2 - -We cannot duplicate 'a' if it denotes an expression that has a side effect! -Fortunately Nimrod supports side effect analysis: - -.. code-block:: nimrod - template optMul{`*`(a, 2)}(a: int{noSideEffect}): int = a+a - - proc f(): int = - echo "side effect!" - result = 55 - - echo f() * 2 # not optimized ;-) - -So what about ``2 * a``? We should tell the compiler ``*`` is commutative. We -cannot really do that however as the following code only swaps arguments -blindly: - -.. code-block:: nimrod - template mulIsCommutative{`*`(a, b)}(a, b: int): int = b*a - -What optimizers really need to do is a *canonicalization*: - -.. code-block:: nimrod - template canonMul{`*`(a, b)}(a: int{lit}, b: int): int = b*a - -The ``int{lit}`` parameter pattern matches against an expression of -type ``int``, but only if it's a literal. - - - -Parameter constraints -===================== - -The parameter constraint expression can use the operators ``|`` (or), -``&`` (and) and ``~`` (not) and the following predicates: - -=================== ===================================================== -Predicate Meaning -=================== ===================================================== -``atom`` The matching node has no children. -``lit`` The matching node is a literal like "abc", 12. -``sym`` The matching node must be a symbol (a bound - identifier). -``ident`` The matching node must be an identifier (an unbound - identifier). -``call`` The matching AST must be a call/apply expression. -``lvalue`` The matching AST must be an lvalue. -``sideeffect`` The matching AST must have a side effect. -``nosideeffect`` The matching AST must have no side effect. -``param`` A symbol which is a parameter. -``genericparam`` A symbol which is a generic parameter. -``module`` A symbol which is a module. -``type`` A symbol which is a type. -``var`` A symbol which is a variable. -``let`` A symbol which is a ``let`` variable. -``const`` A symbol which is a constant. -``result`` The special ``result`` variable. -``proc`` A symbol which is a proc. -``method`` A symbol which is a method. -``iterator`` A symbol which is an iterator. -``converter`` A symbol which is a converter. -``macro`` A symbol which is a macro. -``template`` A symbol which is a template. -``field`` A symbol which is a field in a tuple or an object. -``enumfield`` A symbol which is a field in an enumeration. -``forvar`` A for loop variable. -``label`` A label (used in ``block`` statements). -``nk*`` The matching AST must have the specified kind. - (Example: ``nkIfStmt`` denotes an ``if`` statement.) -``alias`` States that the marked parameter needs to alias - with *some* other parameter. -``noalias`` States that *every* other parameter must not alias - with the marked parameter. -=================== ===================================================== - -The ``alias`` and ``noalias`` predicates refer not only to the matching AST, -but also to every other bound parameter; syntactially they need to occur after -the ordinary AST predicates: - -.. code-block:: nimrod - template ex{a = b + c}(a: int{noalias}, b, c: int) = - # this transformation is only valid if 'b' and 'c' do not alias 'a': - a = b - inc a, b - - -Pattern operators -================= - -The operators ``*``, ``**``, ``|``, ``~`` have a special meaning in patterns -if they are written in infix notation. - - -The ``|`` operator ------------------- - -The ``|`` operator if used as infix operator creates an ordered choice: - -.. code-block:: nimrod - template t{0|1}(): expr = 3 - let a = 1 - # outputs 3: - echo a - -The matching is performed after the compiler performed some optimizations like -constant folding, so the following does not work: - -.. code-block:: nimrod - template t{0|1}(): expr = 3 - # outputs 1: - echo 1 - -The reason is that the compiler already transformed the 1 into "1" for -the ``echo`` statement. However, a term rewriting macro should not change the -semantics anyway. In fact they can be deactived with the ``--patterns:off`` -command line option or temporarily with the ``patterns`` pragma. - - -The ``{}`` operator -------------------- - -A pattern expression can be bound to a pattern parameter via the ``expr{param}`` -notation: - -.. code-block:: nimrod - template t{(0|1|2){x}}(x: expr): expr = x+1 - let a = 1 - # outputs 2: - echo a - - -The ``~`` operator ------------------- - -The ``~`` operator is the **not** operator in patterns: - -.. code-block:: nimrod - template t{x = (~x){y} and (~x){z}}(x, y, z: bool): stmt = - x = y - if x: x = z - - var - a = false - b = true - c = false - a = b and c - echo a - - -The ``*`` operator ------------------- - -The ``*`` operator can *flatten* a nested binary expression like ``a & b & c`` -to ``&(a, b, c)``: - -.. code-block:: nimrod - var - calls = 0 - - proc `&&`(s: varargs[string]): string = - result = s[0] - for i in 1..len(s)-1: result.add s[i] - inc calls - - template optConc{ `&&` * a }(a: string): expr = &&a - - let space = " " - echo "my" && (space & "awe" && "some " ) && "concat" - - # check that it's been optimized properly: - doAssert calls == 1 - - -The second operator of `*` must be a parameter; it is used to gather all the -arguments. The expression ``"my" && (space & "awe" && "some " ) && "concat"`` -is passed to ``optConc`` in ``a`` as a special list (of kind ``nkArgList``) -which is flattened into a call expression; thus the invocation of ``optConc`` -produces: - -.. code-block:: nimrod - `&&`("my", space & "awe", "some ", "concat") - - -The ``**`` operator -------------------- - -The ``**`` is much like the ``*`` operator, except that it gathers not only -all the arguments, but also the matched operators in reverse polish notation: - -.. code-block:: nimrod - import macros - - type - TMatrix = object - dummy: int - - proc `*`(a, b: TMatrix): TMatrix = nil - proc `+`(a, b: TMatrix): TMatrix = nil - proc `-`(a, b: TMatrix): TMatrix = nil - proc `$`(a: TMatrix): string = result = $a.dummy - proc mat21(): TMatrix = - result.dummy = 21 - - macro optM{ (`+`|`-`|`*`) ** a }(a: TMatrix): expr = - echo treeRepr(a) - result = newCall(bindSym"mat21") - - var x, y, z: TMatrix - - echo x + y * z - x - -This passes the expression ``x + y * z - x`` to the ``optM`` macro as -an ``nnkArgList`` node containing:: - - Arglist - Sym "x" - Sym "y" - Sym "z" - Sym "*" - Sym "+" - Sym "x" - Sym "-" - -(Which is the reverse polish notation of ``x + y * z - x``.) - - -Parameters -========== - -Parameters in a pattern are type checked in the matching process. If a -parameter is of the type ``varargs`` it is treated specially and it can match -0 or more arguments in the AST to be matched against: - -.. code-block:: nimrod - template optWrite{ - write(f, x) - ((write|writeln){w})(f, y) - }(x, y: varargs[expr], f: TFile, w: expr) = - w(f, x, y) - diff --git a/doc/tut1.txt b/doc/tut1.txt index 46eda7ae3..a2aa835ee 100644 --- a/doc/tut1.txt +++ b/doc/tut1.txt @@ -1,4 +1,4 @@ -======================== +======================== Nimrod Tutorial (Part I) ======================== @@ -110,7 +110,7 @@ useful for embedding HTML code templates for example. Comments -------- -`Comments`:idx: start anywhere outside a string or character literal with the +Comments start anywhere outside a string or character literal with the hash character ``#``. Documentation comments start with ``##``. Multiline comments need to be aligned at the same column: @@ -224,7 +224,7 @@ different values! For safety use only constant values. Constants ========= -`Constants`:idx: are symbols which are bound to a value. The constant's value +Constants are symbols which are bound to a value. The constant's value cannot change. The compiler must be able to evaluate the expression in a constant declaration at compile time: @@ -369,7 +369,7 @@ he types in nothing (only presses RETURN). For statement ------------- -The `for`:idx: statement is a construct to loop over any element an *iterator* +The ``for`` statement is a construct to loop over any element an *iterator* provides. The example uses the built-in ``countup`` iterator: .. code-block:: nimrod @@ -481,7 +481,7 @@ Example: else: echo("unknown operating system") -The `when`:idx: statement is almost identical to the ``if`` statement with some +The ``when`` statement is almost identical to the ``if`` statement with some differences: * Each condition has to be a constant expression since it is evaluated by the @@ -791,7 +791,7 @@ However, this cannot be done for mutually recursive procedures: Here ``odd`` depends on ``even`` and vice versa. Thus ``even`` needs to be introduced to the compiler before it is completely defined. The syntax for -such a `forward declaration`:idx: is simple: just omit the ``=`` and the +such a forward declaration is simple: just omit the ``=`` and the procedure's body. Later versions of the language may get rid of the need for forward @@ -863,7 +863,7 @@ that are available for them in detail. Booleans -------- -The `boolean`:idx: type is named ``bool`` in Nimrod and consists of the two +The boolean type is named ``bool`` in Nimrod and consists of the two pre-defined values ``true`` and ``false``. Conditions in while, if, elif, when statements need to be of type bool. @@ -1030,7 +1030,7 @@ Enumeration and object types cannot be defined on the fly, but only within a Enumerations ------------ -A variable of an `enumeration`:idx: type can only be assigned a value of a +A variable of an enumeration type can only be assigned a value of a limited set. This set consists of ordered symbols. Each symbol is mapped to an integer value internally. The first symbol is represented at runtime by 0, the second by 1 and so on. Example: @@ -1069,7 +1069,7 @@ An explicit ordered enum can have *holes*: Ordinal types ------------- Enumerations without holes, integer types, ``char`` and ``bool`` (and -subranges) are called `ordinal`:idx: types. Ordinal types have quite +subranges) are called ordinal types. Ordinal types have quite a few special operations: ----------------- -------------------------------------------------------- @@ -1094,7 +1094,7 @@ checks turned on.) Subranges --------- -A `subrange`:idx: type is a range of values from an integer or enumeration type +A subrange type is a range of values from an integer or enumeration type (the base type). Example: .. code-block:: nimrod @@ -1117,7 +1117,7 @@ avoid this common programming error. Sets ---- -The `set type`:idx: models the mathematical notion of a set. The set's +The set type models the mathematical notion of a set. The set's basetype can only be an ordinal type. The reason is that sets are implemented as high performance bit vectors. @@ -1161,7 +1161,7 @@ constants that should be ``or``'ed together. Arrays ------ -An `array`:idx: is a simple fixed length container. Each element in +An array is a simple fixed length container. Each element in the array has the same type. The array's index type can be any ordinal type. Arrays can be constructed via ``[]``: @@ -1253,7 +1253,7 @@ to specify a range from zero to the specified index minus one: Sequences --------- -`Sequences`:idx: are similar to arrays but of dynamic length which may change +Sequences are similar to arrays but of dynamic length which may change during runtime (like strings). Since sequences are resizable they are always allocated on the heap and garbage collected. @@ -1385,8 +1385,8 @@ Tuples A tuple type defines various named *fields* and an *order* of the fields. The constructor ``()`` can be used to construct tuples. The order of the fields in the constructor must match the order in the tuple's definition. -Different tuple-types are *equivalent* if they specify the same fields of -the same type in the same order. +Different tuple-types are *equivalent* if they specify fields of +the same type and of the same name in the same order. The assignment operator for tuples copies each component. The notation ``t.field`` is used to access a tuple's field. Another notation is @@ -1471,7 +1471,7 @@ won't compile: Reference and pointer types --------------------------- -References (similar to `pointers`:idx: in other programming languages) are a +References (similar to pointers in other programming languages) are a way to introduce many-to-one relationships. This means different references can point to and modify the same location in memory. @@ -1513,7 +1513,7 @@ If a reference points to *nothing*, it has the value ``nil``. Procedural type --------------- -A `procedural type`:idx: is a (somewhat abstract) pointer to a procedure. +A procedural type is a (somewhat abstract) pointer to a procedure. ``nil`` is an allowed value for a variable of a procedural type. Nimrod uses procedural types to achieve `functional`:idx: programming techniques. @@ -1521,17 +1521,13 @@ techniques. Example: .. code-block:: nimrod + proc echoItem(x: int) = echo(x) - type - TCallback = proc (x: int) - - proc echoItem(x: Int) = echo(x) - - proc forEach(callback: TCallback) = + proc forEach(action: proc (x: int)) = const data = [2, 3, 5, 7, 11] for d in items(data): - callback(d) + action(d) forEach(echoItem) @@ -1543,7 +1539,7 @@ listed in the `manual <manual.html>`_. Modules ======= -Nimrod supports splitting a program into pieces with a `module`:idx: concept. +Nimrod supports splitting a program into pieces with a module concept. Each module is in its own file. Modules enable `information hiding`:idx: and `separate compilation`:idx:. A module may gain access to symbols of another module by the `import`:idx: statement. Only top-level symbols that are marked @@ -1698,7 +1694,7 @@ define a shorter alias to use when qualifying symbols. Include statement ----------------- -The `include`:idx: statement does something fundamentally different than +The ``include`` statement does something fundamentally different than importing a module: it merely includes the contents of a file. The ``include`` statement is useful to split up a large module into several files: diff --git a/doc/tut2.txt b/doc/tut2.txt index ea6733c07..1e23618e0 100644 --- a/doc/tut2.txt +++ b/doc/tut2.txt @@ -135,7 +135,7 @@ The ``EInvalidObjectConversion`` exception is raised if ``x`` is not a Object variants --------------- Often an object hierarchy is overkill in certain situations where simple -`variant`:idx: types are needed. +variant types are needed. An example: @@ -182,7 +182,7 @@ bound to a class. This has disadvantages: ``join`` a string method or an array method? Nimrod avoids these problems by not assigning methods to a class. All methods -in Nimrod are `multi-methods`:idx:. As we will see later, multi-methods are +in Nimrod are multi-methods. As we will see later, multi-methods are distinguished from procs only for dynamic binding purposes. @@ -194,7 +194,7 @@ The syntax ``obj.method(args)`` can be used instead of ``method(obj, args)``. If there are no remaining arguments, the parentheses can be omitted: ``obj.len`` (instead of ``len(obj)``). -This `method call syntax`:idx: is not restricted to objects, it can be used +This method call syntax is not restricted to objects, it can be used for any type: .. code-block:: nimrod @@ -343,7 +343,7 @@ evaluation or dead code elimination do not work with methods. Exceptions ========== -In Nimrod `exceptions`:idx: are objects. By convention, exception types are +In Nimrod exceptions are objects. By convention, exception types are prefixed with an 'E', not 'T'. The `system <system.html>`_ module defines an exception hierarchy that you might want to stick to. Exceptions derive from E_Base, which provides the common interface. @@ -380,7 +380,7 @@ the template ``newException`` in the ``system`` module can be used: Try statement ------------- -The `try`:idx: statement handles exceptions: +The ``try`` statement handles exceptions: .. code-block:: nimrod # read the first two lines of a text file that should contain numbers @@ -501,7 +501,7 @@ with the file and line where the uncaught exception is being raised, which may help you locate the offending code which has changed. If you want to add the ``{.raises.}`` pragma to existing code, the compiler can -also help you. You can add the ``{.effect.}`` pragma statement to your proc and +also help you. You can add the ``{.effects.}`` pragma statement to your proc and the compiler will output all inferred effects up to that point (exception tracking is part of Nimrod's effect system). Another more roundabout way to find out the list of exceptions raised by a proc is to use the Nimrod ``doc2`` @@ -513,7 +513,7 @@ procs with the list of raised exceptions. You can read more about Nimrod's Generics ======== -`Generics`:idx: are Nimrod's means to parametrize procs, iterators or types +Generics are Nimrod's means to parametrize procs, iterators or types with `type parameters`:idx:. They are most useful for efficient type safe containers: @@ -714,9 +714,9 @@ know how the Nimrod concrete syntax is converted to an abstract syntax tree (AST). The AST is documented in the `macros <macros.html>`_ module. Once your macro is finished, there are two ways to invoke it: -(1) invoking a macro like a procedure call (`expression macros`:idx:) +(1) invoking a macro like a procedure call (expression macros) (2) invoking a macro with the special ``macrostmt`` - syntax (`statement macros`:idx:) + syntax (statement macros) Expression Macros |