renamed most remaining .txt documentation files to .rst

1 files changed, 355 insertions, 0 deletions

diff --git a/doc/docgen.rst b/doc/docgen.rst
new file mode 100644
index 000000000..40c464ebd
--- /dev/null
+++ b/doc/docgen.rst
@@ -0,0 +1,355 @@
+===================================
+   Nim DocGen Tools Guide
+===================================
+
+:Author: Erik O'Leary
+:Version: |nimversion|
+
+.. contents::
+
+
+Introduction
+============
+
+This document describes the `documentation generation tools`:idx: built into
+the `Nim compiler <nimc.html>`_, which can generate HTML and JSON output
+from input .nim files and projects, as well as HTML and LaTeX from input RST
+(reStructuredText) files. The output documentation will include module
+dependencies (``import``), any top-level documentation comments (##), and
+exported symbols (*), including procedures, types, and variables.
+
+
+Documentation Comments
+----------------------
+
+Any comments which are preceded by a double-hash (##), are interpreted as
+documentation.  Comments are parsed as RST (see `reference
+<http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_), providing
+Nim module authors the ability to easily generate richly formatted
+documentation with only their well-documented code.
+
+Example:
+
+.. code-block:: nim
+  type Person* = object
+    ## This type contains a description of a person
+    name: string
+    age: int
+
+Outputs::
+  Person* = object
+    name: string
+    age: int
+
+This type contains a description of a person
+
+Field documentation comments can be added to fields like so:
+
+.. code-block:: nim
+  var numValues: int ## \
+    ## `numValues` stores the number of values
+
+Note that without the `*` following the name of the type, the documentation for
+this type would not be generated. Documentation will only be generated for
+*exported* types/procedures/etc.
+
+
+Nim file input
+-----------------
+
+The following examples will generate documentation for the below contrived
+*Nim* module, aptly named 'sample.nim'
+
+sample.nim:
+
+.. code-block:: nim
+  ## This module is a sample.
+
+  import strutils
+
+  proc helloWorld*(times: int) =
+    ## Takes an integer and outputs
+    ## as many "hello world!"s
+
+    for i in 0 .. times-1:
+      echo "hello world!"
+
+  helloWorld(5)
+
+
+Document Types
+==============
+
+
+HTML
+----
+
+Generation of HTML documents is done via both the ``doc`` and ``doc2``
+commands. These command take either a single .nim file, outputting a single
+.html file with the same base filename, or multiple .nim files, outputting
+multiple .html files and, optionally, an index file.
+
+The ``doc`` command::
+  nim doc sample
+
+Partial Output::
+  ...
+  proc helloWorld*(times: int)
+  ...
+
+Output can be viewed in full here: `docgen_sample.html <docgen_sample.html>`_.
+The next command, called ``doc2``, is very similar to the ``doc`` command, but
+will be run after the compiler performs semantic checking on the input nim
+module(s), which allows it to process macros.
+
+The ``doc2`` command::
+  nim doc2 sample
+
+Partial Output::
+  ...
+  proc helloWorld(times: int) {.raises: [], tags: [].}
+  ...
+
+The full output can be seen here: `docgen_sample2.html <docgen_sample2.html>`_.
+As you can see, the tool has extracted additional information provided to it by
+the compiler beyond what the ``doc`` command provides, such as pragmas attached
+implicitly by the compiler. This type of information is not available from
+looking at the AST (Abstract Syntax Tree) prior to semantic checking, as the
+``doc`` command does.
+
+
+JSON
+----
+
+Generation of JSON documents is done via the ``jsondoc`` command. This command
+takes in a .nim file, and outputs a .json file with the same base filename.
+Note that this tool is built off of the ``doc`` command, and therefore is
+performed before semantic checking.
+
+The ``jsondoc`` command::
+  nim jsondoc sample
+
+Output::
+  [
+    {
+      "comment": "This module is a sample."
+    },
+    {
+      "name": "helloWorld",
+      "type": "skProc",
+      "description": "Takes an integer and outputs as many &quot;hello world!&quot;s",
+      "code": "proc helloWorld*(times: int)"
+    }
+  ]
+
+
+Related Options
+===============
+
+Project switch
+--------------
+
+::
+  nim doc2 --project filename.nim
+
+This will recursively generate documentation of all nim modules imported
+into the input module, including system modules. Be careful with this command,
+as it may end up sprinkling html files all over your filesystem!
+
+
+Index switch
+------------
+
+::
+  nim doc2 --index:on filename.nim
+
+This will generate an index of all the exported symbols in the input Nim
+module, and put it into a neighboring file with the extension of ``.idx``. The
+index file is line oriented (newlines have to be escaped). Each line
+represents a tab separated record of several columns, the first two mandatory,
+the rest optional. See the `Index (idx) file format`_ section for details.
+
+Once index files have been generated for one or more modules, the Nim
+compiler command ``buildIndex directory`` can be run to go over all the index
+files in the specified directory to generate a `theindex.html <theindex.html>`_
+file.
+
+See source switch
+-----------------
+
+::
+  nim doc2 --docSeeSrcUrl:txt filename.nim
+
+When you pass the ``docSeeSrcUrl`` switch to docgen, after each documented item
+in your source code the hyper link *See source* will appear pointing to the
+implementation of that item on a GitHub repository. You can click the link to
+see the implementation of the item.
+
+If you want to reuse this feature in your own documentation you will have to
+modify ``config/nimdoc.cfg`` to contain a ``doc.item.seesrc`` value with a
+hyper link to your own code repository. As you will see by the comments in that
+file, the value ``txt`` passed on the command line will be used in the HTML
+template along others like ``$path`` and ``$line``.
+
+In the case of Nim's own documentation, the ``txt`` value is just a commit
+hash to append to a formatted URL to https://github.com/Araq/Nim. The
+``tools/nimweb.nim`` helper queries the current git commit hash during doc
+generation, but since you might be working on an unpublished repository, it
+also allows specifying a ``githash`` value in ``web/website.ini`` to force a
+specific commit in the output.
+
+
+Other Input Formats
+===================
+
+The *Nim compiler* also has support for RST (reStructuredText) files with
+the ``rst2html`` and ``rst2tex`` commands. Documents like this one are
+initially written in a dialect of RST which adds support for nim sourcecode
+highlighting with the ``.. code-block:: nim`` prefix. ``code-block`` also
+supports highlighting of C++ and some other c-like languages.
+
+Usage::
+  nim rst2html docgen.txt
+
+Output::
+  You're reading it!
+
+The input can be viewed here `docgen.txt <docgen.txt>`_. The ``rst2tex``
+command is invoked identically to ``rst2html``, but outputs a .tex file instead
+of .html.
+
+
+HTML anchor generation
+======================
+
+When you run the ``rst2html`` command, all sections in the RST document will
+get an anchor you can hyper link to. Usually you can guess the anchor lower
+casing the section title and replacing spaces with dashes, and in any case you
+can get it from the table of contents. But when you run the ``doc`` or ``doc2``
+commands to generate API documentation, some symbol get one or two anchors at
+the same time: a numerical identifier, or a plain name plus a complex name.
+
+The numerical identifier is just a random number. The number gets assigned
+according to the section and position of the symbol in the file being processed
+and you should not rely on it being constant: if you add or remove a symbol the
+numbers may shuffle around.
+
+The plain name of a symbol is a simplified version of its fully exported
+signature. Variables or constants have the same plain name symbol as their
+complex name. The plain name for procs, templates, and other callable types
+will be their unquoted value after removing parameters, return types and
+pragmas. The plain name allows short and nice linking of symbols which works
+unless you have a module with collisions due to overloading.
+
+If you hyper link a plain name symbol and there are other matches on the same
+HTML file, most browsers will go to the first one. To differentiate the rest,
+you will need to use the complex name. A complex name for a callable type is
+made up from several parts:
+
+    (**plain symbol**)(**.type**),(**first param**)?(**,param type**)\*
+
+The first thing to note is that all callable types have at least a comma, even
+if they don't have any parameters. If there are parameters, they are
+represented by their types and will be comma separated. To the plain symbol a
+suffix may be added depending on the type of the callable:
+
+-------------   --------------
+Callable type   Suffix
+-------------   --------------
+proc            *empty string*
+macro           ``.m``
+method          ``.e``
+iterator        ``.i``
+template        ``.t``
+converter       ``.c``
+-------------   --------------
+
+The relationship of type to suffix is made by the proc ``complexName`` in the
+``compiler/docgen.nim`` file. Here are some examples of complex names for
+symbols in the `system module <system.html>`_.
+
+* ``type SignedInt = int | int8 | int16 | int32 | int64`` **=>**
+  `#SignedInt <system.html#SignedInt>`_
+* ``var globalRaiseHook: proc (e: ref E_Base): bool {.nimcall.}`` **=>**
+  `#globalRaiseHook <system.html#globalRaiseHook>`_
+* ``const NimVersion = "0.0.0"`` **=>**
+  `#NimVersion <system.html#NimVersion>`_
+* ``proc getTotalMem(): int {.rtl, raises: [], tags: [].}`` **=>**
+  `#getTotalMem, <system.html#getTotalMem,>`_
+* ``proc len[T](x: seq[T]): int {.magic: "LengthSeq", noSideEffect.}`` **=>**
+  `#len,seq[T] <system.html#len,seq[T]>`_
+* ``iterator pairs[T](a: seq[T]): tuple[key: int, val: T] {.inline.}`` **=>**
+  `#pairs.i,seq[T] <system.html#pairs.i,seq[T]>`_
+* ``template newException[](exceptn: typedesc; message: string): expr`` **=>**
+  `#newException.t,typedesc,string
+  <system.html#newException.t,typedesc,string>`_
+
+
+Index (idx) file format
+=======================
+
+Files with the ``.idx`` extension are generated when you use the `Index
+switch`_ along with commands to generate documentation from source or text
+files. You can programatically generate indices with the `setIndexTerm()
+<rstgen.html#setIndexTerm>`_ and `writeIndexFile()
+<rstgen.html#writeIndexFile>`_ procs. The purpose of ``idx`` files is to hold
+the interesting symbols and their HTML references so they can be later
+concatenated into a big index file with `mergeIndexes()
+<rstgen.html#mergeIndexes>`_.  This section documents the file format in
+detail.
+
+Index files are line oriented and tab separated (newline and tab characters
+have to be escaped). Each line represents a record with at least two fields,
+but can have up to four (additional columns are ignored). The content of these
+columns is:
+
+1. Mandatory term being indexed. Terms can include quoting according to
+   Nim's rules (eg. \`^\` like in `the actors module
+   <actors.html#^,ptr.TChannel[T]>`_).
+2. Base filename plus anchor hyper link (eg.
+   ``algorithm.html#*,int,SortOrder``).
+3. Optional human readable string to display as hyper link. If the value is not
+   present or is the empty string, the hyper link will be rendered
+   using the term. Prefix whitespace indicates that this entry is
+   not for an API symbol but for a TOC entry.
+4. Optional title or description of the hyper link. Browsers usually display
+   this as a tooltip after hovering a moment over the hyper link.
+
+The index generation tools try to differentiate between documentation
+generated from ``.nim`` files and documentation generated from ``.txt`` or
+``.rst`` files. The former are always closely related to source code and
+consist mainly of API entries. The latter are generic documents meant for
+human reading.
+
+To differentiate both types (documents and APIs), the index generator will add
+to the index of documents an entry with the title of the document. Since the
+title is the topmost element, it will be added with a second field containing
+just the filename without any HTML anchor.  By convention this entry without
+anchor is the *title entry*, and since entries in the index file are added as
+they are scanned, the title entry will be the first line. The title for APIs
+is not present because it can be generated concatenating the name of the file
+to the word **Module**.
+
+Normal symbols are added to the index with surrounding whitespaces removed. An
+exception to this are table of content (TOC) entries. TOC entries are added to
+the index file with their third column having as much prefix spaces as their
+level is in the TOC (at least 1 character). The prefix whitespace helps to
+filter TOC entries from API or text symbols. This is important because the
+amount of spaces is used to replicate the hiearchy for document TOCs in the
+final index, and TOC entries found in ``.nim`` files are discarded.
+
+
+Additional resources
+====================
+
+`Nim Compiler User Guide <nimc.html#command-line-switches>`_
+
+`RST Quick Reference
+<http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
+
+The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and
+``config/nimdoc.tex.cfg`` configuration files. You can add and modify these
+files to your project to change the look of docgen output.
+
+You can import the `packages/docutils/rstgen module <rstgen.html>`_ in your
+programs if you want to reuse the compiler's documentation generation procs.