manual split up into multiple files; documented the new concurrency system

1 files changed, 647 insertions, 0 deletions

diff --git a/doc/manual/stmts.txt b/doc/manual/stmts.txt
new file mode 100644
index 000000000..b706101e1
--- /dev/null
+++ b/doc/manual/stmts.txt
@@ -0,0 +1,647 @@
+Statements and expressions
+==========================
+
+Nim uses the common statement/expression paradigm: Statements do not
+produce a value in contrast to expressions. However, some expressions are 
+statements.
+
+Statements are separated into `simple statements`:idx: and
+`complex statements`:idx:.
+Simple statements are statements that cannot contain other statements like
+assignments, calls or the ``return`` statement; complex statements can
+contain other statements. To avoid the `dangling else problem`:idx:, complex
+statements always have to be intended. The details can be found in the grammar.
+
+
+Statement list expression
+-------------------------
+
+Statements can also occur in an expression context that looks 
+like ``(stmt1; stmt2; ...; ex)``. This is called
+an statement list expression or ``(;)``. The type 
+of ``(stmt1; stmt2; ...; ex)`` is the type of ``ex``. All the other statements
+must be of type ``void``. (One can use ``discard`` to produce a ``void`` type.)
+``(;)`` does not introduce a new scope.
+
+
+Discard statement
+-----------------
+
+Example:
+
+.. code-block:: nim
+  proc p(x, y: int): int = 
+    result = x + y
+
+  discard p(3, 4) # discard the return value of `p`
+
+The ``discard`` statement evaluates its expression for side-effects and
+throws the expression's resulting value away. 
+
+Ignoring the return value of a procedure without using a discard statement is
+a static error.
+
+The return value can be ignored implicitly if the called proc/iterator has
+been declared with the `discardable`:idx: pragma: 
+
+.. code-block:: nim
+  proc p(x, y: int): int {.discardable.} = 
+    result = x + y
+    
+  p(3, 4) # now valid
+
+An empty ``discard`` statement is often used as a null statement:
+
+.. code-block:: nim
+  proc classify(s: string) =
+    case s[0]
+    of SymChars, '_': echo "an identifier"
+    of '0'..'9': echo "a number"
+    else: discard
+
+
+Var statement
+-------------
+
+Var statements declare new local and global variables and
+initialize them. A comma separated list of variables can be used to specify
+variables of the same type:
+
+.. code-block:: nim
+
+  var
+    a: int = 0
+    x, y, z: int
+
+If an initializer is given the type can be omitted: the variable is then of the
+same type as the initializing expression. Variables are always initialized
+with a default value if there is no initializing expression. The default
+value depends on the type and is always a zero in binary.
+
+============================    ==============================================
+Type                            default value
+============================    ==============================================
+any integer type                0
+any float                       0.0
+char                            '\\0'
+bool                            false
+ref or pointer type             nil
+procedural type                 nil
+sequence                        nil (*not* ``@[]``)
+string                          nil (*not* "")
+tuple[x: A, y: B, ...]          (default(A), default(B), ...)
+                                (analogous for objects)
+array[0..., T]                  [default(T), ...]
+range[T]                        default(T); this may be out of the valid range
+T = enum                        cast[T](0); this may be an invalid value
+============================    ==============================================
+
+
+The implicit initialization can be avoided for optimization reasons with the
+`noinit`:idx: pragma: 
+
+.. code-block:: nim
+  var
+    a {.noInit.}: array [0..1023, char] 
+
+If a proc is annotated with the ``noinit`` pragma this refers to its implicit
+``result`` variable:
+
+.. code-block:: nim
+  proc returnUndefinedValue: int {.noinit.} = discard
+
+
+The implicit initialization can be also prevented by the `requiresInit`:idx:
+type pragma. The compiler requires an explicit initialization then. However
+it does a `control flow analysis`:idx: to prove the variable has been 
+initialized and does not rely on syntactic properties:
+
+.. code-block:: nim
+  type
+    TMyObject = object {.requiresInit.}
+    
+  proc p() =
+    # the following is valid:
+    var x: TMyObject
+    if someCondition():
+      x = a()
+    else:
+      x = a()
+    use x
+
+let statement
+-------------
+
+A ``let`` statement declares new local and global `single assignment`:idx:
+variables and binds a value to them. The syntax is the of the ``var`` 
+statement, except that the keyword ``var`` is replaced by the keyword ``let``.
+Let variables are not l-values and can thus not be passed to ``var`` parameters
+nor can their address be taken. They cannot be assigned new values.
+
+For let variables the same pragmas are available as for ordinary variables.
+
+
+Const section
+-------------
+
+`Constants`:idx: are symbols which are bound to a value. The constant's value
+cannot change. The compiler must be able to evaluate the expression in a
+constant declaration at compile time.
+
+Nim contains a sophisticated compile-time evaluator, so procedures which
+have no side-effect can be used in constant expressions too:
+
+.. code-block:: nim
+  import strutils
+  const
+    constEval = contains("abc", 'b') # computed at compile time!
+
+
+The rules for compile-time computability are: 
+
+1. Literals are compile-time computable.
+2. Type conversions are compile-time computable.
+3. Procedure calls of the form ``p(X)`` are compile-time computable if
+   ``p`` is a proc without side-effects (see the `noSideEffect pragma`_ 
+   for details) and if ``X`` is a (possibly empty) list of compile-time 
+   computable arguments.
+
+
+Constants cannot be of type ``ptr``, ``ref``, ``var`` or ``object``, nor can 
+they contain such a type.
+
+
+Static statement/expression
+---------------------------
+
+A static statement/expression can be used to enforce compile 
+time evaluation explicitly. Enforced compile time evaluation can even evaluate
+code that has side effects: 
+
+.. code-block::
+
+  static:
+    echo "echo at compile time"
+
+It's a static error if the compiler cannot perform the evaluation at compile 
+time.
+
+The current implementation poses some restrictions for compile time
+evaluation: Code which contains ``cast`` or makes use of the foreign function
+interface cannot be evaluated at compile time. Later versions of Nim will
+support the FFI at compile time.
+
+
+If statement
+------------
+
+Example:
+
+.. code-block:: nim
+
+  var name = readLine(stdin)
+
+  if name == "Andreas":
+    echo("What a nice name!")
+  elif name == "":
+    echo("Don't you have a name?")
+  else:
+    echo("Boring name...")
+
+The ``if`` statement is a simple way to make a branch in the control flow:
+The expression after the keyword ``if`` is evaluated, if it is true
+the corresponding statements after the ``:`` are executed. Otherwise
+the expression after the ``elif`` is evaluated (if there is an
+``elif`` branch), if it is true the corresponding statements after
+the ``:`` are executed. This goes on until the last ``elif``. If all
+conditions fail, the ``else`` part is executed. If there is no ``else``
+part, execution continues with the statement after the ``if`` statement.
+
+The scoping for an ``if`` statement is slightly subtle to support an important 
+use case. A new scope starts for the ``if``/``elif`` condition and ends after
+the corresponding *then* block:
+
+.. code-block:: nim
+  if {| (let m = input =~ re"(\w+)=\w+"; m.isMatch):
+    echo "key ", m[0], " value ", m[1]  |}
+  elif {| (let m = input =~ re""; m.isMatch):
+    echo "new m in this scope" |}
+  else:
+    # 'm' not declared here
+
+In the example the scopes have been enclosed in ``{|  |}``. 
+
+
+Case statement
+--------------
+
+Example:
+
+.. code-block:: nim
+
+  case readline(stdin)
+  of "delete-everything", "restart-computer":
+    echo("permission denied")
+  of "go-for-a-walk":     echo("please yourself")
+  else:                   echo("unknown command")
+  
+  # indentation of the branches is also allowed; and so is an optional colon
+  # after the selecting expression:
+  case readline(stdin):
+    of "delete-everything", "restart-computer":
+      echo("permission denied")
+    of "go-for-a-walk":     echo("please yourself")
+    else:                   echo("unknown command")
+  
+
+The ``case`` statement is similar to the if statement, but it represents
+a multi-branch selection. The expression after the keyword ``case`` is
+evaluated and if its value is in a *slicelist* the corresponding statements
+(after the ``of`` keyword) are executed. If the value is not in any
+given *slicelist* the ``else`` part is executed. If there is no ``else``
+part and not all possible values that ``expr`` can hold occur in a 
+``slicelist``, a static error occurs. This holds only for expressions of 
+ordinal types. "All possible values" of ``expr`` are determined by ``expr``'s
+type. 
+
+If the expression is not of an ordinal type, and no ``else`` part is
+given, control passes after the ``case`` statement.
+
+To suppress the static error in the ordinal case an ``else`` part with an
+empty ``discard`` statement can be used.
+
+As a special semantic extension, an expression in an ``of`` branch of a case
+statement may evaluate to a set or array constructor; the set or array is then
+expanded into a list of its elements:
+
+.. code-block:: nim
+  const
+    SymChars: set[char] = {'a'..'z', 'A'..'Z', '\x80'..'\xFF'}
+
+  proc classify(s: string) =
+    case s[0]
+    of SymChars, '_': echo "an identifier"
+    of '0'..'9': echo "a number"
+    else: echo "other"
+  
+  # is equivalent to:
+  proc classify(s: string) =
+    case s[0]
+    of 'a'..'z', 'A'..'Z', '\x80'..'\xFF', '_': echo "an identifier"
+    of '0'..'9': echo "a number"
+    else: echo "other"
+
+
+When statement
+--------------
+
+Example:
+
+.. code-block:: nim
+
+  when sizeof(int) == 2:
+    echo("running on a 16 bit system!")
+  elif sizeof(int) == 4:
+    echo("running on a 32 bit system!")
+  elif sizeof(int) == 8:
+    echo("running on a 64 bit system!")
+  else:
+    echo("cannot happen!")
+
+The ``when`` statement is almost identical to the ``if`` statement with some
+exceptions:
+
+* Each condition (``expr``) has to be a constant expression (of type ``bool``).
+* The statements do not open a new scope.
+* The statements that belong to the expression that evaluated to true are
+  translated by the compiler, the other statements are not checked for
+  semantics! However, each condition is checked for semantics.
+
+The ``when`` statement enables conditional compilation techniques. As
+a special syntactic extension, the ``when`` construct is also available
+within ``object`` definitions.
+
+
+Return statement
+----------------
+
+Example:
+
+.. code-block:: nim
+  return 40+2
+
+The ``return`` statement ends the execution of the current procedure.
+It is only allowed in procedures. If there is an ``expr``, this is syntactic
+sugar for:
+
+.. code-block:: nim
+  result = expr
+  return result
+
+
+``return`` without an expression is a short notation for ``return result`` if
+the proc has a return type. The `result`:idx: variable is always the return
+value of the procedure. It is automatically declared by the compiler. As all
+variables, ``result`` is initialized to (binary) zero:
+
+.. code-block:: nim
+  proc returnZero(): int =
+    # implicitly returns 0
+
+
+Yield statement
+---------------
+
+Example:
+
+.. code-block:: nim
+  yield (1, 2, 3)
+
+The ``yield`` statement is used instead of the ``return`` statement in
+iterators. It is only valid in iterators. Execution is returned to the body
+of the for loop that called the iterator. Yield does not end the iteration
+process, but execution is passed back to the iterator if the next iteration
+starts. See the section about iterators (`Iterators and the for statement`_)
+for further information.
+
+
+Block statement
+---------------
+
+Example:
+
+.. code-block:: nim
+  var found = false
+  block myblock:
+    for i in 0..3:
+      for j in 0..3:
+        if a[j][i] == 7:
+          found = true
+          break myblock # leave the block, in this case both for-loops
+  echo(found)
+
+The block statement is a means to group statements to a (named) ``block``.
+Inside the block, the ``break`` statement is allowed to leave the block
+immediately. A ``break`` statement can contain a name of a surrounding
+block to specify which block is to leave.
+
+
+Break statement
+---------------
+
+Example:
+
+.. code-block:: nim
+  break
+
+The ``break`` statement is used to leave a block immediately. If ``symbol``
+is given, it is the name of the enclosing block that is to leave. If it is
+absent, the innermost block is left.
+
+
+While statement
+---------------
+
+Example:
+
+.. code-block:: nim
+  echo("Please tell me your password: \n")
+  var pw = readLine(stdin)
+  while pw != "12345":
+    echo("Wrong password! Next try: \n")
+    pw = readLine(stdin)
+
+
+The ``while`` statement is executed until the ``expr`` evaluates to false.
+Endless loops are no error. ``while`` statements open an `implicit block`,
+so that they can be left with a ``break`` statement.
+
+
+Continue statement
+------------------
+
+A ``continue`` statement leads to the immediate next iteration of the
+surrounding loop construct. It is only allowed within a loop. A continue
+statement is syntactic sugar for a nested block:
+
+.. code-block:: nim
+  while expr1:
+    stmt1
+    continue
+    stmt2
+
+Is equivalent to:
+
+.. code-block:: nim
+  while expr1:
+    block myBlockName:
+      stmt1
+      break myBlockName
+      stmt2
+
+
+Assembler statement
+-------------------
+
+The direct embedding of assembler code into Nim code is supported
+by the unsafe ``asm`` statement. Identifiers in the assembler code that refer to
+Nim identifiers shall be enclosed in a special character which can be
+specified in the statement's pragmas. The default special character is ``'`'``:
+
+.. code-block:: nim
+  {.push stackTrace:off.}
+  proc addInt(a, b: int): int =
+    # a in eax, and b in edx
+    asm """
+        mov eax, `a`
+        add eax, `b`
+        jno theEnd
+        call `raiseOverflow`
+      theEnd:
+    """
+  {.pop.}
+
+If the GNU assembler is used, quotes and newlines are inserted automatically:
+
+.. code-block:: nim
+  proc addInt(a, b: int): int =
+    asm """
+      addl %%ecx, %%eax
+      jno 1
+      call `raiseOverflow`
+      1:
+      :"=a"(`result`)
+      :"a"(`a`), "c"(`b`)
+    """
+
+Instead of:
+
+.. code-block:: nim
+  proc addInt(a, b: int): int =
+    asm """
+      "addl %%ecx, %%eax\n"
+      "jno 1\n"
+      "call `raiseOverflow`\n"
+      "1: \n"
+      :"=a"(`result`)
+      :"a"(`a`), "c"(`b`)
+    """
+
+Using statement
+---------------
+
+**Warning**: The ``using`` statement is highly experimental!
+
+The using statement provides syntactic convenience for procs that
+heavily use a single contextual parameter. When applied to a variable or a
+constant, it will instruct Nim to automatically consider the used symbol as
+a hidden leading parameter for any procedure calls, following the using
+statement in the current scope. Thus, it behaves much like the hidden `this`
+parameter available in some object-oriented programming languages.
+
+.. code-block:: nim
+
+  var s = socket()
+  using s
+
+  connect(host, port)
+  send(data)
+
+  while true:
+    let line = readLine(timeout)
+    ...
+
+
+When applied to a callable symbol, it brings the designated symbol in the
+current scope. Thus, it can be used to disambiguate between imported symbols
+from different modules having the same name.
+
+.. code-block:: nim
+  import windows, sdl
+  using sdl.SetTimer
+
+Note that ``using`` only *adds* to the current context, it doesn't remove or
+replace, **neither** does it create a new scope. What this means is that if one
+applies this to multiple variables the compiler will find conflicts in what
+variable to use:
+
+.. code-block:: nim
+  var a, b = "kill it"
+  using a
+  add(" with fire")
+  using b
+  add(" with water")
+  echo a
+  echo b
+
+When the compiler reaches the second ``add`` call, both ``a`` and ``b`` could
+be used with the proc, so one gets ``Error: expression '(a|b)' has no type (or
+is ambiguous)``. To solve this one would need to nest ``using`` with a
+``block`` statement so as to control the reach of the ``using`` statement.
+
+If expression
+-------------
+
+An `if expression` is almost like an if statement, but it is an expression.
+Example:
+
+.. code-block:: nim
+  var y = if x > 8: 9 else: 10
+
+An if expression always results in a value, so the ``else`` part is
+required. ``Elif`` parts are also allowed.
+
+When expression
+---------------
+
+Just like an `if expression`, but corresponding to the when statement.
+
+Case expression
+---------------
+
+The `case expression` is again very similar to the case statement:
+
+.. code-block:: nim
+  var favoriteFood = case animal
+    of "dog": "bones"
+    of "cat": "mice"
+    elif animal.endsWith"whale": "plankton"
+    else:
+      echo "I'm not sure what to serve, but everybody loves ice cream"
+      "ice cream"
+
+As seen in the above example, the case expression can also introduce side
+effects. When multiple statements are given for a branch, Nim will use
+the last expression as the result value, much like in an `expr` template.
+
+Table constructor
+-----------------
+
+A table constructor is syntactic sugar for an array constructor:
+
+.. code-block:: nim
+  {"key1": "value1", "key2", "key3": "value2"}
+  
+  # is the same as:
+  [("key1", "value1"), ("key2", "value2"), ("key3", "value2")]
+
+
+The empty table can be written ``{:}`` (in contrast to the empty set 
+which is ``{}``) which is thus another way to write as the empty array
+constructor ``[]``. This slightly unusal way of supporting tables 
+has lots of advantages:
+
+* The order of the (key,value)-pairs is preserved, thus it is easy to
+  support ordered dicts with for example ``{key: val}.newOrderedTable``.
+* A table literal can be put into a ``const`` section and the compiler
+  can easily put it into the executable's data section just like it can
+  for arrays and the generated data section requires a minimal amount
+  of memory.
+* Every table implementation is treated equal syntactically.
+* Apart from the minimal syntactic sugar the language core does not need to
+  know about tables.
+
+
+Type conversions
+----------------
+Syntactically a `type conversion` is like a procedure call, but a
+type name replaces the procedure name. A type conversion is always
+safe in the sense that a failure to convert a type to another
+results in an exception (if it cannot be determined statically).
+
+
+Type casts
+----------
+Example:
+
+.. code-block:: nim
+  cast[int](x)
+
+Type casts are a crude mechanism to interpret the bit pattern of
+an expression as if it would be of another type. Type casts are
+only needed for low-level programming and are inherently unsafe.
+
+
+The addr operator
+-----------------
+The ``addr`` operator returns the address of an l-value. If the type of the
+location is ``T``, the `addr` operator result is of the type ``ptr T``. An
+address is always an untraced reference. Taking the address of an object that
+resides on the stack is **unsafe**, as the pointer may live longer than the
+object on the stack and can thus reference a non-existing object. One can get
+the address of variables, but one can't use it on variables declared through
+``let`` statements:
+
+.. code-block:: nim
+
+  let t1 = "Hello"
+  var
+    t2 = t1
+    t3 : pointer = addr(t2)
+  echo repr(addr(t2))
+  # --> ref 0x7fff6b71b670 --> 0x10bb81050"Hello"
+  echo cast[ptr string](t3)[]
+  # --> Hello
+  # The following line doesn't compile:
+  echo repr(addr(t1))
+  # Error: expression has no address