path: root/doc/lib.txt

=======================
Nimrod Standard Library
=======================

:Author: Andreas Rumpf
:Version: |nimrodversion|

.. contents::

  "The good thing about reinventing the wheel is that you can get a round one."

Though the Nimrod Standard Library is still evolving, it is already quite
usable. It is divided into *pure libraries*, *impure libraries* and *wrappers*.

Pure libraries do not depend on any external ``*.dll`` or ``lib*.so`` binary
while impure libraries do. A wrapper is an impure library that is a very
low-level interface to a C library.

Read this `document <apis.html>`_ for a quick overview of the API design.

The `bottom <#babel>`_ of this page includes a list of 3rd party packages
created by the Nimrod community. These packages are a useful addition to the
modules in the standard library.


Pure libraries
==============

Core
----

* `system <system.html>`_
  Basic procs and operators that every program needs. It also provides IO
  facilities for reading and writing text and binary files. It is imported
  implicitly by the compiler. Do not import it directly. It relies on compiler
  magic to work.

* `unsigned <unsigned.html>`_
  This module implements basic arithmetic operators for unsigned integers.
  To discourage users from using unsigned integers, it's not part 
  of ``system``, but an extra import.

* `threads <threads.html>`_
  Nimrod thread support. **Note**: This is part of the system module. Do not
  import it explicitly.

* `channels <channels.html>`_
  Nimrod message passing support for threads. **Note**: This is part of the 
  system module. Do not import it explicitly.

* `locks <locks.html>`_
  Locks and condition variables for Nimrod.

* `macros <macros.html>`_
  Contains the AST API and documentation of Nimrod for writing macros.

* `typeinfo <typeinfo.html>`_
  Provides (unsafe) access to Nimrod's run time type information. 
  
* `actors <actors.html>`_
  Actor support for Nimrod; implemented as a layer on top of the threads and
  channels modules.


Collections and algorithms
--------------------------

* `algorithm <algorithm.html>`_
  Implements some common generic algorithms like sort or binary search.
* `tables <tables.html>`_
  Nimrod hash table support. Contains tables, ordered tables and count tables.
* `sets <sets.html>`_
  Nimrod hash and bit set support.
* `lists <lists.html>`_
  Nimrod linked list support. Contains singly and doubly linked lists and
  circular lists ("rings").
* `queues <queues.html>`_
  Implementation of a queue. The underlying implementation uses a ``seq``.
* `intsets <intsets.html>`_
  Efficient implementation of a set of ints as a sparse bit set.
* `critbits <critbits.html>`_
  This module implements a *crit bit tree* which is an efficient
  container for a set or a mapping of strings.
* `sequtils <sequtils.html>`_
  This module implements operations for the built-in seq type
  which were inspired by functional programming languages.


String handling
---------------

* `strutils <strutils.html>`_
  This module contains common string handling operations like changing
  case of a string, splitting a string into substrings, searching for
  substrings, replacing substrings.

* `parseutils <parseutils.html>`_
  This module contains helpers for parsing tokens, numbers, identifiers, etc.

* `strtabs <strtabs.html>`_
  The ``strtabs`` module implements an efficient hash table that is a mapping
  from strings to strings. Supports a case-sensitive, case-insensitive and
  style-insensitive mode. An efficient string substitution operator ``%``
  for the string table is also provided.

* `unicode <unicode.html>`_
  This module provides support to handle the Unicode UTF-8 encoding.
  
* `encodings <encodings.html>`_
  Converts between different character encodings. On UNIX, this uses 
  the ``iconv`` library, on Windows the Windows API.

* `pegs <pegs.html>`_
  This module contains procedures and operators for handling PEGs.

* `ropes <ropes.html>`_
  This module contains support for a *rope* data type.
  Ropes can represent very long strings efficiently; especially concatenation
  is done in O(1) instead of O(n).

* `unidecode <unidecode.html>`_
  This module provides Unicode to ASCII transliterations:
  It finds the sequence of ASCII characters that is the closest approximation
  to the Unicode string.

* `matchers <matchers.html>`_
  This module contains various string matchers for email addresses, etc.

* `subexes <subexes.html>`_
  This module implements advanted string substitution operations.


Generic Operating System Services
---------------------------------

* `os <os.html>`_
  Basic operating system facilities like retrieving environment variables,
  reading command line arguments, working with directories, running shell
  commands, etc.

* `osproc <osproc.html>`_
  Module for process communication beyond ``os.execShellCmd``.

* `times <times.html>`_
  The ``times`` module contains basic support for working with time.

* `dynlib <dynlib.html>`_
  This module implements the ability to access symbols from shared libraries.

* `streams <streams.html>`_
  This module provides a stream interface and two implementations thereof:
  the `PFileStream` and the `PStringStream` which implement the stream
  interface for Nimrod file objects (`TFile`) and strings. Other modules
  may provide other implementations for this standard stream interface.

* `marshal <marshal.html>`_
  Contains procs for serialization and deseralization of arbitrary Nimrod 
  data structures.

* `terminal <terminal.html>`_
  This module contains a few procedures to control the *terminal*
  (also called *console*). The implementation simply uses ANSI escape
  sequences and does not depend on any other module.

* `memfiles <memfiles.html>`_
  This module provides support for memory mapped files (Posix's ``mmap``) 
  on the different operating systems.

* `fsmonitor <fsmonitor.html>`_
  This module implements the ability to monitor a directory/file for changes
  using Posix's inotify API.

Math libraries
--------------

* `math <math.html>`_
  Mathematical operations like cosine, square root.

* `complex <complex.html>`_
  This module implements complex numbers and their mathematical operations.



Internet Protocols and Support
------------------------------

* `cgi <cgi.html>`_
  This module implements helpers for CGI applications.

* `scgi <scgi.html>`_
  This module implements helpers for SCGI applications.

* `sockets <sockets.html>`_
  This module implements a simple portable type-safe sockets layer.

* `asyncio <asyncio.html>`_
  This module implements an asynchronous event loop for sockets.

* `browsers <browsers.html>`_
  This module implements procs for opening URLs with the user's default
  browser.

* `httpserver <httpserver.html>`_
  This module implements a simple HTTP server.

* `httpclient <httpclient.html>`_
  This module implements a simple HTTP client.

* `smtp <smtp.html>`_
  This module implement a simple SMTP client. 

* `irc <irc.html>`_
  This module implements an asynchronous IRC client.

* `ftpclient <ftpclient.html>`_
  This module implements an FTP client.

* `cookies <cookies.html>`_
  This module contains helper procs for parsing and generating cookies.

* `mimetypes <mimetypes.html>`_
  This module implements a mimetypes database.

* `uri <uri.html>`_
  This module provides functions for working with URIs.

* `asyncdispatch <asyncdispatch.html>`_
  This module implements an asynchronous dispatcher for IO operations.

  **Note:** This module is still largely experimental.

* `asyncnet <asyncnet.html>`_
  This module implements asynchronous sockets based on the ``asyncdispatch``
  module.

  **Note:** This module is still largely experimental.

* `asynchttpserver <asynchttpserver.html>`_
  This module implements an asynchronous HTTP server using the ``asyncnet``
  module.

  **Note:** This module is still largely experimental.

* `net <net.html>`_
  This module implements a high-level sockets API. It will replace the
  ``sockets`` module in the future.

* `rawsockets <rawsockets.html>`_
  This module implements a low-level sockets API.

* `selectors <selectors.html>`_
  This module implements a selector API with backends specific to each OS.
  Currently epoll on Linux and select on other operating systems.

Parsers
-------

* `parseopt <parseopt.html>`_
  The ``parseopt`` module implements a command line option parser.
  **Deprecated since version 0.9.3:** Use the `parseopt2
  <parseopt2.html>`_ module instead.

* `parseopt2 <parseopt2.html>`_
  The ``parseopt2`` module implements a command line option parser. This
  supports long and short command options with optional values and command line
  arguments.

* `parsecfg <parsecfg.html>`_
  The ``parsecfg`` module implements a high performance configuration file
  parser. The configuration file's syntax is similar to the Windows ``.ini``
  format, but much more powerful, as it is not a line based parser. String
  literals, raw string literals and triple quote string literals are supported
  as in the Nimrod programming language.

* `parsexml <parsexml.html>`_
  The ``parsexml`` module implements a simple high performance XML/HTML parser.
  The only encoding that is supported is UTF-8. The parser has been designed
  to be somewhat error correcting, so that even some "wild HTML" found on the
  Web can be parsed with it.

* `parsecsv <parsecsv.html>`_
  The ``parsecsv`` module implements a simple high performance CSV parser.

* `parsesql <parsesql.html>`_
  The ``parsesql`` module implements a simple high performance SQL parser.

* `json <json.html>`_
  High performance JSON parser.

* `lexbase <lexbase.html>`_
  This is a low level module that implements an extremely efficient buffering
  scheme for lexers and parsers. This is used by the diverse parsing modules.

* `highlite <highlite.html>`_
  Source highlighter for programming or markup languages.  Currently
  only few languages are supported, other languages may be added.
  The interface supports one language nested in another.

* `rst <rst.html>`_
  This module implements a reStructuredText parser. A large subset
  is implemented. Some features of the markdown wiki syntax are
  also supported.

* `rstast <rstast.html>`_
  This module implements an AST for the reStructuredText parser.

* `rstgen <rstgen.html>`_
  This module implements a generator of HTML/Latex from reStructuredText.


XML Processing
--------------

* `xmldom <xmldom.html>`_
  This module implements the XML DOM Level 2.

* `xmldomparser <xmldomparser.html>`_
  This module parses an XML Document into a XML DOM Document representation.

* `xmltree <xmltree.html>`_
  A simple XML tree. More efficient and simpler than the DOM. It also
  contains a macro for XML/HTML code generation.

* `xmlparser <xmlparser.html>`_
  This module parses an XML document and creates its XML tree representation.

* `htmlparser <htmlparser.html>`_
  This module parses an HTML document and creates its XML tree representation.

* `htmlgen <htmlgen.html>`_
  This module implements a simple XML and HTML code 
  generator. Each commonly used HTML tag has a corresponding macro
  that generates a string with its HTML representation.

Cryptography and Hashing
------------------------

* `hashes <hashes.html>`_
  This module implements efficient computations of hash values for diverse
  Nimrod types.

* `md5 <md5.html>`_
  This module implements the MD5 checksum algorithm.

* `base64 <base64.html>`_
  This module implements a base64 encoder and decoder.


Multimedia support
------------------

* `colors <colors.html>`_
  This module implements color handling for Nimrod. It is used by
  the ``graphics`` module.


Miscellaneous
-------------

* `events <events.html>`_
  This module implements an event system that is not dependant on external
  graphical toolkits.

* `oids <oids.html>`_
  An OID is a global ID that consists of a timestamp,
  a unique counter and a random value. This combination should suffice to 
  produce a globally distributed unique ID. This implementation was extracted
  from the Mongodb interface and it thus binary compatible with a Mongo OID.

* `endians <endians.html>`_
  This module contains helpers that deal with different byte orders.

* `logging <logging.html>`_
  This module implements a simple logger.

* `future <future.html>`_
  This module implements new experimental features. Currently the syntax
  sugar for anonymous procedures.

Database support
----------------

* `redis <redis.html>`_
  This module implements a redis client. It allows you to connect to a
  redis-server instance, send commands and receive replies.


Modules for JS backend
---------------------------

* `dom <dom.html>`_
  Declaration of the Document Object Model for the JS backend.


Impure libraries
================

Regular expressions
-------------------

* `re <re.html>`_
  This module contains procedures and operators for handling regular
  expressions. The current implementation uses PCRE.


Database support
----------------

* `db_postgres <db_postgres.html>`_
  A higher level PostgreSQL database wrapper. The same interface is implemented
  for other databases too.

* `db_mysql <db_mysql.html>`_
  A higher level MySQL database wrapper. The same interface is implemented
  for other databases too.

* `db_sqlite <db_sqlite.html>`_
  A higher level SQLite database wrapper. The same interface is implemented
  for other databases too.


Other
-----

* `graphics <graphics.html>`_
  This module implements graphical output for Nimrod; the current
  implementation uses SDL but the interface is meant to support multiple
  backends some day.

* `dialogs <dialogs.html>`_
  This module implements portable dialogs for Nimrod; the implementation
  builds on the GTK interface. On Windows, native dialogs are shown if
  appropriate.

* `zipfiles <zipfiles.html>`_
  This module implements a zip archive creator/reader/modifier.

* `web <web.html>`_
  This module contains simple high-level procedures for dealing with the
  Web like loading the contents of a Web page from an URL.

* `ssl <ssl.html>`_
  This module provides an easy to use sockets-style 
  Nimrod interface to the OpenSSL library.

* `rdstdin <rdstdin.html>`_
  This module contains code for reading from `stdin`:idx:. On UNIX the GNU
  readline library is wrapped and set up.

* `zmq <zmq.html>`_
  Nimrod 0mq wrapper. This file contains the low level C wrappers as well as
  some higher level constructs.


Wrappers
========

The generated HTML for some of these wrappers is so huge that it is
not contained in the distribution. You can then find them on the website.

Windows specific
----------------

* `windows <windows.html>`_
  Contains a wrapper for the Win32 API.
* `ole2 <ole2.html>`_
  Contains GUIDs for OLE2 automation support.
* `shellapi <shellapi.html>`_
  Contains a wrapper for the ``shellapi.h`` header.
* `shfolder <shfolder.html>`_
  Contains a wrapper for the ``shfolder.h`` header.
* `mmsystem <mmsystem.html>`_
  Contains a wrapper for the ``mmsystem.h`` header.
* `nb30 <nb30.html>`_
  This module contains the definitions for portable NetBIOS 3.0 support.


UNIX specific
-------------

* `posix <posix.html>`_
  Contains a wrapper for the POSIX standard.
* `readline <readline.html>`_
  Part of the wrapper for the GNU readline library.
* `history <history.html>`_
  Part of the wrapper for the GNU readline library.
* `rltypedefs <rltypedefs.html>`_
  Part of the wrapper for the GNU readline library.


Regular expressions
-------------------

* `pcre <pcre.html>`_
  Wrapper for the PCRE library.
* `tre <tre.html>`_
  Wrapper for the TRE library.


Graphics libraries
------------------

* `sdl <sdl.html>`_
  Part of the wrapper for SDL.
* `sdl_gfx <sdl_gfx.html>`_
  Part of the wrapper for SDL.
* `sdl_image <sdl_image.html>`_
  Part of the wrapper for SDL.
* `sdl_mixer <sdl_mixer.html>`_
  Part of the wrapper for SDL.
* `sdl_net <sdl_net.html>`_
  Part of the wrapper for SDL.
* `sdl_ttf <sdl_ttf.html>`_
  Part of the wrapper for SDL.
* `smpeg <smpeg.html>`_
  Part of the wrapper for SDL.


GUI libraries
-------------

* `iup <iup.html>`_
  Wrapper of the IUP GUI library.


Database support
----------------

* `postgres <postgres.html>`_
  Contains a wrapper for the PostgreSQL API.
* `mysql <mysql.html>`_
  Contains a wrapper for the mySQL API.
* `sqlite3 <sqlite3.html>`_
  Contains a wrapper for SQLite 3 API.
* `odbcsql <odbcsql.html>`_
  interface to the ODBC driver.
* `sphinx <sphinx.html>`_
  Nimrod wrapper for ``sphinx``.


XML Processing
--------------

* `expat <expat.html>`_
  Wrapper of the expat XML parser.


Network Programming and Internet Protocols
------------------------------------------

* `libuv <libuv.html>`_
  Wrapper for the libuv library used for async I/O programming.

* `joyent_http_parser <joyent_http_parser.html>`_
  Wrapper for the joyent's high-performance HTTP parser.

* `libcurl <libcurl.html>`_
  Wrapper for the libcurl library.

* `openssl <openssl.html>`_
  Wrapper for OpenSSL.


Data Compression and Archiving
------------------------------

* `zlib <zlib.html>`_
  Wrapper for the zlib library.

* `libzip <libzip.html>`_
  Interface to the `lib zip <http://www.nih.at/libzip/index.html>`_ library by
  Dieter Baron and Thomas Klausner.


Scientific computing
--------------------

* `libsvm <libsvm.html>`_ 
  Low level wrapper for `lib svm <http://www.csie.ntu.edu.tw/~cjlin/libsvm/>`_.

Babel
====================

Babel is a package manager for the Nimrod programming language.
For instructions on how to install Babel packages see
`its README <https://github.com/nimrod-code/babel#readme>`_.

Official packages
-----------------

These packages are officially supported and will therefore be continually
maintained to ensure that they work with the latest versions of the Nimrod
compiler.

.. raw:: html

  <div id="officialPkgList"><b>If you are reading this you are missing
  babelpkglist.js or have javascript disabled in your browser.</b></div>

Unofficial packages
-------------------

These packages have been developed by independent Nimrod developers and as
such may not always be up to date with the latest developments in the
Nimrod programming language.

.. raw:: html

  <div id="unofficialPkgList"><b>If you are reading this you are missing
  babelpkglist.js or have javascript disabled in your browser.</b></div>

  <script type="text/javascript" src="babelpkglist.js"></script>
  <script type="text/javascript" src="http://build.nimrod-lang.org/packages?callback=gotPackageList"></script>