about summary refs log tree commit diff stats
path: root/src/io/dynstream.nim
Commit message (Collapse)AuthorAgeFilesLines
* Enable ProveInit warningbptato2025-05-101-1/+5
| | | | It has caught some minor bugs.
* myaddr: backport & switch to newSeqUninitbptato2025-04-301-2/+2
|
* dynstream: close extraneous fds receivedbptato2025-03-051-2/+9
|
* Refactor bufreader, bufwriterbptato2025-03-011-6/+6
| | | | | This adds a runtime check to packet readers to ensure that all fds have been read, and switches to seqs for packet writers.
* dynstream, console: remove DynFileStreambptato2025-02-261-36/+0
| | | | | Conflating buffered streams with non-buffered streams is generally a bad idea.
* dynstream: remove exceptionsbptato2025-02-261-154/+120
| | | | | | | | | | | | | Now we just pass down the value of n and check errno, plus readDataLoop/writeDataLoop returns a bool indicating whether it failed. For now this seems to work OK, but maybe I'll add a better abstraction in the future. EOFError is still used for handling failed packets; this is brittle, and should be replaced once we have a proper buffering mechanism for them. (That will also let us kill BufStream.) Unrelated: this also fixes a bug in buffer with cacheId.
* dynstream: allow reading less fds than specifiedbptato2025-02-101-7/+19
|
* dynstream: use seq instead of manual allocationbptato2025-02-091-12/+4
|
* pager, term: replace execCmd, Filebptato2025-01-221-0/+3
| | | | | | | | | | pager: I want to change mailcap command parsing, so a custom implementation is needed. term: now it uses a custom buffer for the output stream too. I don't think there is much difference in performance, but stdio was in the way. (Also, this way it will be easier to make it async in the future.)
* term: rework input bufferbptato2025-01-171-5/+1
| | | | heh
* dynstream: remove superfluous zeroMem, add sendFds/recvFdsbptato2025-01-121-22/+31
|
* buffer, dynstream: fix clone race, recvmsg/sendmsg castbptato2025-01-121-3/+3
| | | | | | | The fds must be read before the other buffer resumes execution. Also, for some reason, CMSG_SPACE/CMSG_LEN are inconsistent in their size with controllen on BSDs...
* Fix some strict defsbptato2025-01-121-5/+5
|
* bufreader, bufwriter: send all fds in one messagebptato2025-01-111-71/+57
| | | | | | | | Moves sendfd/recvfd out of C, and fixes some of its flaws too. The main one is that now all file descriptors are sent together. Also, I've decided to remove the ServerSocket after all; it's easy to add it back if it's ever needed again.
* buffer: remove server socketbptato2025-01-091-59/+18
| | | | | | | | | | | Now we just pass through a socket created in pager. This removes the need for a socket directory, and strengthens the buffer sandbox slightly. I've kept the ServerSocket code, because I want to add some form of RPC and communication between separate instances in the future. However, I don't expect this to be handled outside the main process, so I've removed the Capsicum-specific connectat/bindat code.
* dynstream: remove safeClose, add moveFdbptato2025-01-091-10/+9
| | | | | | | | | | | | | | | | | | | safeClose was originally added to prevent this bug: let fd = doSomething() # allocates fd on 0, e.g. by opening a socket let ps = newPosixStream(fd) ... discard dup2(ps.fd, STDIN_FILENO) ps.sclose() # stdin is now closed, despite the opposite intention. With safeClose called on fds that could be stdin, the goal was that stdin/stdout/stderr would never be allocated as a different file, but it was still error-prone. Enter moveFd: ps.moveFd(STDIN_FILENO) If ps is already stdin, this does nothing. If not, it dup2's ps to stdin, closes ps.fd, and sets it to stdin.
* bufstream: allow 0-width packetsbptato2025-01-091-5/+5
|
* loader: use per-process control socketsbptato2025-01-081-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | Previously, each message (load, resume, passFd, etc.) would open a new connection to loader's UNIX socket, and assumed the loader was dead when the loader did not respond (ECONNREFUSED). As it turns out, this model was hopelessly broken: POSIX does not specify when a UNIX socket can refuse connections, so while it happened to work on Linux (which just blocks if it can't accept right now), FreeBSD would randomly refuse connections whenever its listen queue was full. Instead, we now take a socketpair() from the loader in addClient, and pass on one half to the client (the other half stays in loader); this is the control stream, and all messages from the specific client are sent through it. If a message includes a new stream (e.g. lcLoad), then it sends a new socketpair through the control stream. Aside from not being completely broken (and being marginally more efficient), this arrangement has several other benefits: it removes the need for authentication, almost removes the need for sockdir (now only buffers use it), and will make it easier to add async message processing in the future.
* dynstream: simplify, fix fd leakbptato2025-01-081-9/+5
|
* Try to set close-on-exec, misc cleanupbptato2025-01-041-14/+13
| | | | | | | | | | FD_CLOEXEC should hopefully get rid of bugs where buffers outlived the main process because of some stray child process keeping the canary pipe alive. It's not perfect because of the runMailcapWriteFile/runMailcapReadFile double-forks. Ideally they should be replaced with an implementation that tracks temporary files in the main process.
* chabookmark fixes & improvementsbptato2024-12-151-0/+2
| | | | | | | | * correct action on M-b * add external.bookmark option * move openFileExpand functionality into unquote * add menu items * update docs
* pager, mailcap: misc fixes, add prompt for global mailcapbptato2024-12-111-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | In the past, Chawan would read global mailcap (/etc/mailcap, ...) too, but every now and then that would run entries that I didn't even know existed and definitely didn't intend to run. So I changed it to only use ~/.mailcap, but this meant users now had to add mailcap entries for every single mime type. At some point I also changed application/octet-stream to always save to disk, which is usually nice except when a text file is misrecognized as binary. Often times I just want to decide myself what to do. So now there are two layers. First, the global mailcap files (path as per RFC) prompt before executing. Then there is ~/.chawan/auto.mailcap (or ~/.config/chawan/auto.mailcap) which runs entries automatically. If you press shift before selecting an option in the prompt, the corresponding entry gets copied to auto.mailcap. It's also possible to type a new entry on the fly. Overall I think it's quite convenient. One unfortunate side effect is that existing users will have to migrate their entries to auto.mailcap, or redefine external.auto-mailcap to e.g. ~/.mailcap, but this seems acceptable.
* config, mailcap: remove std/streams dependency, specialize mmapbptato2024-12-051-27/+43
| | | | | | | | | | * use PosixStream/mmap for mailcap reading too; this finally lets us get rid of std/streams in the entire codebase * split up recvDataLoopOrMmap into 3 functions: one that can fall back to recvAll, one that falls back to recvDataLoop, and one that does not fall back to anything * use MAP_PRIVATE in mmap for read (we don't care if changes are propagated, as we do no changes to cached files)
* dynstream: recvDataLoopOrMmap improvementsbptato2024-12-051-38/+70
| | | | | * fall back to recvAll on ilen = -1 * handle zero-length files
* toml, config: skip copying buf, use PosixStreambptato2024-12-031-1/+5
| | | | | | | one std/streams less I used mmap for reading the user config. It shouldn't matter in any realistically sized config, but who knows.
* dynstream: check lseek return codebptato2024-11-171-0/+1
| | | | just in case
* dynstream: fix crash on *BSDbptato2024-11-041-1/+4
| | | | turns out fchmod on sockets only works on Linux.
* dynstream: restrict permissions to userbptato2024-10-311-0/+1
| | | | | the man page says this isn't really portable, but it's better than nothing
* dynstream: refactorbptato2024-10-201-24/+84
| | | | | | | | | | * consistently use cint instead of FileHandle - this was another remnant of winapi support; on posix, they are the same. * move "blocking" field to PosixStream * recvFileHandle -> recvFd, sendFileHandle -> sendFd * merge serversocket into dynstream * merge auxiliary C functions into dynstream_aux
* dynstream, serversocket: use posix instead of nativesocketsbptato2024-10-201-54/+13
| | | | | | | | | | | nativesockets is a wrapper over posix and winapi, but we don't support winapi, so we can just fall back to PosixStream instead. SocketStream remains as a constraint over PosixStream to allow sendFileHandle/recvFileHandle. As a nice side effect, we can drop some allowed syscalls from the seccomp filter.
* pager: refactor mailcap, console; misc fixesbptato2024-10-141-0/+28
| | | | | | | | | * use more PosixStream (because it has double-close checking) * factor out some common mailcap operations * move console from client to pager * fix case-insensitive mime type matching * replace convoluted fdin/fdout comparison logic (that only accidentally worked) with a boolean flag
* dynstream: fix memory leakbptato2024-10-041-2/+5
| | | | now I know why overloading dealloc felt wrong
* gopher: do not depend on libcurlbptato2024-09-281-1/+7
| | | | | | | | I'm thinking of making libcurl entirely optional; let's start with the easiest part. I've added a SOCKS5 client for ALL_PROXY support; I know curl supported others too, but whatever.
* dynstream: fix mmap bugsbptato2024-09-251-12/+16
| | | | ugh
* Replace std/selectors with pollbptato2024-09-231-0/+9
| | | | | | | | | | | | std/selectors uses OS-specific selector APIs, which sounds good in theory (faster than poll!), but sucks for portability in practice. Sure, you can fix portability bugs, but who knows how many there are on untested platforms... poll is standard, so if it works on one computer it should work on all other ones. (I hope.) As a bonus, I rewrote the timeout API for poll, which incidentally fixes setTimeout across forks. Also, SIGWINCH should now work on all platforms (as we self-pipe instead of signalfd/kqueue magic).
* client, forkserver, dynstream: misc refactorings, fixesbptato2024-09-231-1/+1
| | | | | | * fix broken int conversion in dynstream * fix EPIPE handling in forkserver * merge fdmap and connectingContainers into loader map
* loader: mmap intermediate image files, misc refactoringbptato2024-09-221-0/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | | * refactor parseHeader * optimize response blob() * add direct "to cache" mode for loader requests which sets stdout to a file, and use it for image processing * move image resizing into a separate process * mmap cache files in between processing steps when possible At last, resize is no longer a part of image decoding. Also, it feels much nicer to keep encoded image data in the same cache as everything else. The mmap operations *should* be more efficient than copying the whole RGBA data through a pipe. In practice, it only makes a difference for loading (well, now just mmapping) the encoded image into the pager, where it singlehandedly speeds up image display by 10x on my test image. For the other steps, the unfortunate fact that "tocache" must delay the next fork/exec in the pipeline until the entire image is processed seems to equal out any wins we might have gotten from skipping a single raw RGBA copy. I have tried moving the delay before the exec (it's possible with yet another pipe), but it didn't help much and made the code much uglier. (Not that tocache didn't, but I can live with this...)
* loader: fix some fd leaksbptato2024-09-021-0/+9
| | | | + be a bit more paranoid about double closes
* Fixes for Nim 2.2bptato2024-07-291-0/+269
| | | | | | | | | * xmlhttprequest: fix missing import * painter: generic tuple workaround * dynstream: merge module with implementations (so it will work with vtables) Not enabling vtables yet since it doesn't work with refc.
* js: fix various leaks etc.bptato2024-05-031-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Previously we didn't actually free the main JS runtime, probably because you can't do this without first waiting for JS to unwind the stack. (This has the unfortunate effect that code now *can* run after quit(). TODO: find a fix for this.) This isn't a huge problem per se, we only have one of these and the OS can clean it up. However, it also disabled the JS_FreeRuntime leak check, which resulted in sieve-like behavior (manual refcounting is a pain). So now we choose the other tradeoff: quit no longer runs exitnow, but it waits for the event loop to run to the end and only then exits the browser. Then, before exit we free the JS context & runtime, and also all JS values allocated by config. Fixes: * fix `ad' flag not being set for just one siteconf/omnirule * fix various leaks (since leak check is enabled now) * use ptr UncheckedArray[JSValue] for QJS bindings that take an array * allow JSAtom in jsgetprop etc., also disallow int types other than uint32 * do not set a destructor for globals
* pager: fix broken writeToFilebptato2024-03-291-0/+3
|
* io: derive DynStream from RootObj (not Stream)bptato2024-03-241-44/+26
| | | | | | | | This way they are no longer compatible, but we no longer need them to be compatible anyway. (This also forces us to throw out the old serialize module, and use packet writers everywhere.)
* bufreader: add initPacketReaderbptato2024-03-241-0/+7
|
* io: add bufreaderbptato2024-03-211-0/+7
| | | | analogous to bufwriter
* io: add BuferedWriterbptato2024-03-161-0/+3
| | | | | | | | | | Unsurprisingly enough, calling `write` a million times is never going to be very fast. BufferedWriter basically does the same thing as serialize.swrite did, but queues up writes in batches before sending them. TODO: give sread a similar treatment
* io: add dynstreambptato2024-03-121-0/+82
a new abstraction that we derive posixstream from; hopefully with time we can get rid of std/streams
generated by cgit-pink 1.4.1-2-gfad0 (git 2.36.2.497.gbbea4dcf42) at 2025-07-01 07:42:07 +0000