path: root/doc/cha-protocols.7

.\" Automatically generated by Pandoc 3.6.4
.\"
.TH "cha\-protocols" "7" "" "" "Protocol support in Chawan"
.SH Protocols
Chawan supports downloading resources from various protocols: HTTP, FTP,
SFTP, Gopher, Gemini, Spartan, and Finger.
Details on these protocols, and information on how users can add support
to their preferred protocols is outlined in this document.
.PP
You can find network adapters in the source distribution\[cq]s
\f[CR]adapter/protocol\f[R] directory.
For protocol\-specific file formats (like gemtext or gopher directories)
you will also find an appropriate HTML converter in
\f[CR]adapter/format\f[R].
.SS HTTP
HTTP/s support is implemented in Nim.
It supports HTTP/1.1 with arbitrary headers and POST data, is able to
use passed userinfo data, and returns all headers and response body it
receives without exception.
.PP
Deflate decompression with gzip and zlib headers is supported.
(Accept\-Encoding: gzip, deflate.)
This is based on a modified version of the public domain tinfl.h
decompressor by Rich Geldreich.
.PP
Brotli decompression (Accept\-Encoding: br) is supported using the
decoder provided by the reference implementation.
.PP
The \f[CR]bonus\f[R] directory contains two alternative HTTP clients:
.IP \[bu] 2
curlhttp; this is the old HTTP client based on libcurl.
It can be built using curl\-impersonate; see README.md in the bonus/
directory for details.
.IP \[bu] 2
libfetch\-http: based on FreeBSD libfetch.
It is mostly a proof of concept, as FreeBSD libfetch HTTP support is
very limited; in particular, it does not support arbitrary HTTP headers,
so e.g.
cookies will not work.
.SS FTP
Chawan supports FTP passive mode browsing and downloads.
.PP
For directory listings, it assumes UNIX output style, and will probably
break horribly on receiving anything else.
Otherwise, the directory listing view is identical to the file://
directory listing.
.SS SFTP
The sftp adapter (\f[CR]adapter/protocol/sftp.nim\f[R]) wraps libssh2.
It works for me, but YMMV.
.PP
Note that if an IdentityFile declaration is found in your ssh config,
then it will prompt for the identity file password, but there is no way
to tell whether it is really asking for that (or just normal password
auth).
Also, settings covered by the Match field are ignored.
.SS Gopher
Chawan supports the Gopher protocol through the gopher CGI program.
Gopher directories are passed as the \f[CR]text/gopher\f[R] type, and
gopher2html takes care of converting this to HTML.
.PP
Gopher selector types are converted to MIME types when possible; note
however, that this is very limited, as most of them (like \f[CR]s\f[R]
sound, or \f[CR]I\f[R] image) cannot be unambiguously converted without
some other sniffing method.
Chawan will fall back to extension\-based detection in these cases, and
in the worst case may end up with \f[CR]application/octet\-stream\f[R].
.SS Gemini
Chawan\[cq]s Gemini adapter has been rewritten as a Nim program.
It still requires OpenSSL to work.
.PP
A limitation that remains is that the Gemini adapter does not support
sites that require private key authentication.
.PP
gmi2html is its companion program to convert the \f[CR]text/gemini\f[R]
file format to HTML.
.SS Finger
Finger is supported through the \f[CR]finger\f[R] shell script.
It is implemented as a shell script because of the protocol\[cq]s
simplicity.
.PP
For portability, \f[CR]finger\f[R] uses Chawan\[cq]s \f[CR]nc\f[R] tool
(a very limited netcat clone) to make requests.
.SS Spartan
Spartan is a protocol similar to Gemini, but without TLS.
It is supported through the \f[CR]spartan\f[R] shell script, and like
Finger, it uses Chawan\[cq]s \f[CR]nc\f[R] to make requests.
.PP
Spartan has the very strange property of extending gemtext with a
protocol\-specific line type.
This is sort of supported through a sed filter for gemtext outputs in
the CGI script (in other words, no modification to gmi2html was done to
support this).
.SS Local schemes: file:, man:
While these are not necessarily \f[I]protocols\f[R], they are
implemented similarly to the protocols listed above (and thus can also
be replaced, if the user wishes; see below).
.PP
\f[CR]file:\f[R] loads a file from the local filesystem.
In case of directories, it shows the directory listing like the FTP
protocol does.
.PP
\f[CR]man:\f[R], \f[CR]man\-k:\f[R] and \f[CR]man\-l:\f[R] are wrappers
around the commands \f[CR]man\f[R], \f[CR]man \-k\f[R] and
\f[CR]man \-l\f[R].
These look up man pages using \f[CR]/usr/bin/man\f[R] and turn on\-page
references into links.
A wrapper command \f[CR]mancha\f[R] also exists; this has an interface
similar to \f[CR]man\f[R].
Note: this used to be based on w3mman2html.cgi, but it has been
rewritten in Nim (and therefore no longer depends on Perl either).
.SS Internal schemes: cgi\-bin:, stream:, cache:, data:, about:
Five internal protocols exist: \f[CR]cgi\-bin:\f[R], \f[CR]stream:\f[R],
\f[CR]cache:\f[R], \f[CR]data:\f[R] and \f[CR]about:\f[R].
These are the basic building blocks for the implementation of every
protocol mentioned above; for this reason, these can \f[I]not\f[R] be
replaced, and are implemented in the main browser binary.
.PP
\f[CR]cgi\-bin:\f[R] executes a local CGI script.
This scheme is used for the actual implementation of the non\-internal
protocols mentioned above.
Local CGI scripts can also be used to implement wrappers of other
programs inside Chawan (e.g.\ dictionaries).
.PP
\f[CR]stream:\f[R] is used for streams returned by external programs.
It differs from \f[CR]cgi\-bin:\f[R] in that it does not cooperate with
the external process, and that the loader does not keep track of where
the stream originally comes from.
Therefore it is suitable for reading in the output of mailcap entries,
or for turning stdin into a URL.
.PP
It is not possible to reload \f[CR]stream:\f[R] URLs.
To support rewinding and \[lq]view source\[rq], the output of
\f[CR]stream:\f[R]\[cq]s is stored in a cache file until the buffer is
discarded.
.PP
\f[CR]cache:\f[R] is not something an end user would normally see;
it\[cq]s used for rewinding or re\-interpreting streams already
downloaded.
.PP
Caching works differently than in most other browsers; files are
deterministically loaded from the cache upon certain actions, and from
the network upon others, but neither is used as a fallback to the other.
.PP
\f[CR]data:\f[R] decodes a data URL as defined in RFC 2397.
This used to be a CGI module, but has been moved back into the loader
process because these URLs can get so long that they no longer fit into
the environment.
.PP
\f[CR]about:\f[R] is inside the loader to allow for an implementation of
the download list panel.
It should be turned into a CGI module once the loader gets RPC
capabilities.
.PP
The following about pages are available: \f[CR]about:chawan\f[R],
\f[CR]about:blank\f[R], \f[CR]about:license\f[R],
\f[CR]about:download\f[R].
.SS Custom protocols
Chawan is protocol\-agnostic.
This means that the \f[CR]cha\f[R] binary itself does not know much
about the protocols listed above; instead, it loads these through a
combination of local CGI, urimethodmap, and if conversion to HTML or
plain text is necessary, mailcap (using x\-htmloutput, x\-ansioutput and
copiousoutput).
.PP
urimethodmap can also be used to override default handlers for the
protocols listed above.
This is similar to how w3m allows you to override the default directory
listing display, but much more powerful; this way, any library or
program that can retrieve and output text through a certain protocol can
be combined with Chawan.
.PP
For example, consider the urimethodmap definition of cha\-finger:
.IP
.EX
finger:     cgi\-bin:cha\-finger
.EE
.PP
This commands Chawan to load the cha\-finger CGI script, setting the
\f[CR]$MAPPED_URI_*\f[R] variables to the target URL\[cq]s parts in the
process.
.PP
Then, cha\-finger uses these passed parts to construct an appropriate
curl command that will retrieve the specified \f[CR]finger:\f[R] URL; it
prints the header `Content\-Type: text/plain' to the output, then an
empty line, then the body of the retrieved resource.
If an error is encountered, it prints a \f[CR]Cha\-Control\f[R] header
with an error code and a specific error message instead.
.SS Adding a new protocol
Here we will add a protocol called \[lq]cowsay\[rq], so that the URL
cowsay:text prints the output of \f[CR]cowsay text\f[R] after a second
of waiting.
.PP
\f[CR]mkdir \-p \[ti]/.chawan/cgi\-bin\f[R], and create a CGI script in
\f[CR]\[ti]/.chawan/cgi\-bin/cowsay.cgi\f[R]:
.IP
.EX
\f[I]#!/bin/sh\f[R]
\f[I]# We are going to wait a second from now, but want Chawan to show\f[R]
\f[I]# \[dq]Downloading...\[dq] instead of \[dq]Connecting...\[dq]. So signal to the browser that the\f[R]
\f[I]# connection has succeeded.\f[R]
printf \[aq]Cha\-Control: Connected\[rs]n\[aq]
sleep 1 \f[I]# sleep\f[R]
\f[I]# Status is a special header that signals the equivalent HTTP status code.\f[R]
printf \[aq]Status: 200\[aq] \f[I]# HTTP OK\f[R]
\f[I]# Tell the browser that no more control headers are to be expected.\f[R]
\f[I]# This is useful when you want to send remotely received headers; then, it would\f[R]
\f[I]# be an attack vector to simply send the headers without ControlDone, as nothing\f[R]
\f[I]# stops the website from sending a Cha\-Control header. With ControlDone sent,\f[R]
\f[I]# even Cha\-Control headers will be interpreted as regular headers.\f[R]
printf \[aq]Cha\-Control: ControlDone\[rs]n\[aq]
\f[I]# As in HTTP, you must send an empty line before the body.\f[R]
printf \[aq]\[rs]n\[aq]
\f[I]# Now, print the body. We take the path passed to the URL; urimethodmap\f[R]
\f[I]# sets this as MAPPED_URI_PATH. This is URI\-encoded, so we also run the urldec\f[R]
\f[I]# utility on it.\f[R]
cowsay \[dq]$(printf \[aq]%s\[rs]n\[aq] \[dq]$MAPPED_URI_PATH\[dq] \f[B]|\f[R] \[dq]$CHA_LIBEXEC_DIR\[dq]/urldec)\[dq]
.EE
.PP
Now, create a \[lq].urimethodmap\[rq] file in your \f[CR]$HOME\f[R]
directory.
.PP
Then, enter into it the following:
.IP
.EX
cowsay:     /cgi\-bin/cowsay.cgi
.EE
.PP
Now try \f[CR]cha cowsay:Hello,%20world.\f[R].
If you did everything correctly, it should wait one second, then print a
cow saying \[lq]Hello, world.\[rq].
.SS See also
\f[B]cha\f[R](1), \f[B]cha\-localcgi\f[R](5),
\f[B]cha\-urimethodmap\f[R](5), \f[B]cha\-mailcap\f[R](5)