aboutsummaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
authorbptato <nincsnevem662@gmail.com>2024-10-01 21:09:22 +0200
committerbptato <nincsnevem662@gmail.com>2024-10-01 21:18:59 +0200
commitb0a511f900a2884c0d1bb55e3991c068ef3e37f2 (patch)
tree2ac5d77d3f62d844335d0dd00e840a8f5b38d726 /doc
parentee113a5643ed32b7f6ed2522aed92ac700021097 (diff)
downloadchawan-b0a511f900a2884c0d1bb55e3991c068ef3e37f2.tar.gz
ftp: remove libcurl dependency
This splits out sftp into a separate binary that *does* depend on libcurl. However, ftp now uses the same socket code as gopher. ftps is dropped, because I've never even tested it. Maybe I'll add it back when we have working OpenSSL bindings. This is still "doing the easy part first", now I have no clue how to handle sftp because my initial plan ("just use the sftp binary") doesn't work - sftp batch mode doesn't accept passwords. libssh2 remains the sole candidate, but that's what libcurl wraps anyway.
Diffstat (limited to 'doc')
-rw-r--r--doc/architecture.md47
-rw-r--r--doc/protocols.md26
2 files changed, 59 insertions, 14 deletions
diff --git a/doc/architecture.md b/doc/architecture.md
index 7ad3a113..c30c991d 100644
--- a/doc/architecture.md
+++ b/doc/architecture.md
@@ -135,9 +135,10 @@ Scenario: the user attempts to navigate to <https://example.org>.
1. pager creates a new container for the target URL.
2. pager sends a request for "https://example.org" to the loader. Then, it
registers the file descriptor in its selector, and does something else until
- select() reports activity on the file descriptor.
+ poll() reports activity on the file descriptor.
3. loader rewrites "https://example.org" into "cgi-bin:http". It then runs the
- http CGI script with the appropriate environment variables set.
+ http CGI script with the appropriate environment variables set to parts of
+ this URL and request headers.
4. The http CGI script opens a connection to example.org. When connected, it
starts printing out headers it receives to stdout.
5. loader parses these headers, and sends them to pager.
@@ -150,10 +151,50 @@ Scenario: the user attempts to navigate to <https://example.org>.
* If Content-Type is text/html, then a new buffer process is created,
which then parses the response body as HTML. If it is any `text/*`
subtype, then the response is simply inserted into a `<plaintext>` tag.
- * If Content-Type is not a `text/*` type, and no mailcap entry for it
+ * If Content-Type is not a `text/*` subtype, and no mailcap entry for it
is found, then the user is prompted about where they wish to save the
file.
+## Cache
+
+Chawan's caching mechanism is largely inspired by that of w3m, which does not
+have a network cache. Instead, it simply saves source files to the disk before
+displaying them, and lets users view/edit the source without another network
+request.
+
+Chawan improves upon this by simultaneously streaming files to the cache and
+buffers:
+
+1. Client (pager or buffer) initiates request by sending a message to loader.
+2. Loader starts CGI script, reads headers, sends a response, and waits.
+3. Client now may send an "addCacheFile" message, which prompts loader to add a
+ cache file for this request.
+4. Client sends resume, now loader will stream the response both to the client
+ and the cache.
+
+Cached items may be shared between clients; this is how rewinding on wrong
+charset guess is implemented. They are also manually reference counted and are
+unlinked when their reference count drops to zero.
+
+The cache is used in the following ways:
+
+* For view source and edit source operations.
+* For rewinding buffers on incorrect charset guess. (In practice, this is almost
+ never used, because the first chunk we read tends to determine the charset
+ unambiguously.)
+* For reading images multiple times after download. (At least two reads are
+ needed, because the first pass only parses the headers.)
+* As a memory buffer for image coding processes to mmap. (For details, see
+ [image.md](image.md).)
+
+Crucially, the cache *does not* understand Cache-Control headers, and will never
+skip a download when requested by a user. Similarly, loading a "cache:" URL
+(e.g. view source) is guaranteed to never make a network request.
+
+Future directions: for non-JS buffers, we could kill idle processes and reload
+them on-demand from the cache. This could solve the problem of spawning too many
+processes that then do nothing.
+
## Parsing HTML
The character decoder and the HTML parser are implementations of the WHATWG
diff --git a/doc/protocols.md b/doc/protocols.md
index 0d670797..7d23d22c 100644
--- a/doc/protocols.md
+++ b/doc/protocols.md
@@ -14,6 +14,7 @@ this document.
* [HTTP](#http)
* [FTP](#ftp)
+* [SFTP](#sftp)
* [Gopher](#gopher)
* [Gemini](#gemini)
* [Finger](#finger)
@@ -34,7 +35,7 @@ to use passed userinfo data (`https://username:password@example.org`), and
returns all headers and response body it receives from libcurl without
exception.
-It is possible to build these adapters using
+It is possible to build the HTTP adapter using
[curl-impersonate](https://github.com/lwthiker/curl-impersonate) by setting
the compile-time variable CURLLIBNAME to `libcurl-impersonate.so`. Note that
for curl-impersonate to work, you must set `network.default-headers = {}`
@@ -49,17 +50,20 @@ not work.
## FTP
-Chawan supports FTP through the `adapter/protocol/ftp.nim` libcurl adapter. For
-directory listings, it assumes UNIX output style, and will probably break
-horribly on receiving anything else. Otherwise, the directory listing view
-is identical to the file:// directory listing.
+Chawan supports FTP through the `adapter/protocol/ftp.nim` adapter. Only
+passive mode is supported.
-SFTP "works" too, but YMMV. Note that if an IdentityFile declaration is found in
-your ssh config, then it will prompt for the identity file password, but there
-is no way to tell whether it is really asking for that. Also, settings covered
-by the Match field are ignored.
+For directory listings, it assumes UNIX output style, and will probably
+break horribly on receiving anything else. Otherwise, the directory
+listing view is identical to the file:// directory listing.
-In theory, FTPS should work too, but it is completely untested.
+## SFTP
+
+The sftp adapter wraps libcurl. It works for me, but YMMV. Note that if
+an IdentityFile declaration is found in your ssh config, then it will
+prompt for the identity file password, but there is no way to tell
+whether it is really asking for that. Also, settings covered by the
+Match field are ignored.
## Gopher
@@ -78,7 +82,7 @@ case may end up with `application/octet-stream`.
Chawan's gemini adapter (in `adapter/protocol/gmifetch.c`) is a C program. It
requires OpenSSL to work.
-Currently, it still has some limitations:
+It still has some limitations:
* It does not support proxies yet.
* It does not support sites that require private key authentication.