summary refs log tree commit diff stats
path: root/lib/pure/memfiles.nim
Commit message (Collapse)AuthorAgeFilesLines
* fixes #14760 (#14769)Andreas Rumpf2020-06-231-1/+1
|
* Make file descriptors from stdlib non-inheritable by default (#13201)alaviss2020-04-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * io: make file descriptors non-inheritable by default This prevents file descriptors/handles leakage to child processes that might cause issues like running out of file descriptors, or potential security issues like leaking a file descriptor to a restricted file. While this breaks backward compatibility, I'm rather certain that not many programs (if any) actually make use of this implementation detail. A new API `setInheritable` is provided for the few that actually want to use this functionality. * io: disable inheritance at file creation time for supported platforms Some platforms provide extension to fopen-family of functions to allow for disabling descriptor inheritance atomically during File creation. This guards against possible leaks when a child process is spawned before we managed to disable the file descriptor inheritance (ie. in a multi-threaded program). * net, nativesockets: make sockets non inheritable by default With this commit, sockets will no longer leak to child processes when you don't want it to. Should solves a lot of "address in use" that might occur when your server has just restarted. All APIs that create sockets in these modules now expose a `inheritable` flag that allow users to toggle inheritance for the resulting sockets. An implementation of `setInheritance()` is also provided for SocketHandle. While atomically disabling inheritance at creation time is supported on Windows, it's only implemented by native winsock2, which is too much for now. This support can be implemented in a future patch. * posix: add F_DUPFD_CLOEXEC This command duplicates file descriptor with close-on-exec flag set. Defined in POSIX.1-2008. * ioselectors_kqueue: don't leak file descriptors File descriptors internally used by ioselectors on BSD/OSX are now shielded from leakage. * posix: add O_CLOEXEC This flag allows file descriptors to be open() with close-on-exec flag set atomically. This flag is specified in POSIX.1-2008 * tfdleak: test for selectors leakage Also simplified the test by using handle-type agnostic APIs to test for validity. * ioselectors_epoll: mark all fd created close-on-exec File descriptors from ioselectors should no longer leaks on Linux. * tfdleak: don't check for selector leakage on Windows The getFd proc for ioselectors_select returns a hardcoded -1 * io: add NoInheritFlag at compile time * io: add support for ioctl-based close-on-exec This allows for the flag to be set/unset in one syscall. While the performance gains might be negliable, we have one less failure point to deal with. * tfdleak: add a test for setInheritable * stdlib: add nimInheritHandles to restore old behaviors * memfiles: make file handle not inheritable by default for posix * io: setInheritable now operates on OS file handle On Windows, the native handle is the only thing that's inheritable, thus we can assume that users of this function will already have the handle available to them. This also allows users to pass down file descriptors from memfiles on Windows with ease, should that be desired. With this, nativesockets.setInheritable can be made much simpler. * changelog: clarify * nativesockets: document setInheritable return value * posix_utils: atomically disable fd inheritance for mkstemp
* Fix word wrappingJjp1372019-10-221-4/+4
|
* Fix many broken linksJjp1372019-10-221-4/+4
| | | | | | Note that contrary to what docgen.rst currently says, the ids have to match exactly or else most web browsers will not jump to the intended symbol.
* Revert "Fixes #12187 (#12321)" (#12447)Andreas Rumpf2019-10-181-1/+1
| | | This reverts commit 00c31e87660d9db813871f5aa23661bf6b9bbdcb.
* Fixes #12187 (#12321)Clyybber2019-10-081-1/+1
| | | | | | * Fixes #12187 * Point to fork of compactdict Since the original repo is now archived / read-only
* [backport] run nimpretty on os-related stuffnarimiran2019-09-301-30/+32
|
* Fix spellings (#12277) [backport]Federico Ceratto2019-09-271-1/+1
|
* fixes #12186Araq2019-09-181-1/+1
|
* Make MemSlice stringification both simpler and faster. (#10464)c-blake2019-01-271-3/+2
|
* Allow an escape hatch for platform specific flags/default override (#9968)c-blake2018-12-141-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Allow an escape hatch for platform specific flags (of which there are many, for example MAP_POPULATE itself is a Linux-only thing, not other Unix). Continue with same defaults as before in this commit, but that really should be changed to *not* include MAP_POPULATE. While pre-faulting all the pages can be useful sometimes *if* you know you're going to access all the data, it is highly unlikely to be what users expect the default to be. For some things all that up front work is 1000s of times slower than being lazy/on-demand/only ever faulting parts of the file. Even the MAP_POPULATE fan who originally in 2014 committed to this file defaulted it to off (but turned it always-on as a "temporary" work around for some long since gone/mutated compiler issue). Anyway, at least something like this `mapFlags` gives users the ability to override the poor default choice or activate any other idiosyncratic platform-specific features. * Use simple, efficient default flags, but also accept whatever the open/mapMem caller specifies. Save flags in MemFile so they can be used in `resize`. This field should not need exporting like the others -- callers can always save whatever values they pass -- but we include a cautionary comment in case anyone ever asks for a `*` there, as well as for documentation. Also make documentation for ``mapFlags`` in open more likely to inspire care.
* clean up the implementation of the new memfiles.resizeAraq2018-12-121-27/+27
|
* export every field of MemFile for more low level trickery; refs #9922Araq2018-12-121-4/+5
|
* For now just implement `resize` per https://github.com/nim-lang/Nim/pull/9922Charles Blake2018-12-111-15/+29
| | | | | discussion (with special mremap usage on Linux, but ordinary munmap, mmap on other POSIX). Someone needs to do the when windows branch.
* Address dom96/Araq opinions in https://github.com/nim-lang/Nim/pull/9922Charles Blake2018-12-111-4/+19
| | | | | | Updating accessors are also provided since the idea of this change is to allow "updating" operations external to the module which are by their very nature closely tied to module internals (as well as to OS interface details).
* Let handles be seen outside of `memfiles` module so that "updating"Charles Blake2018-12-101-4/+4
| | | | | | | | operations (like eg., resizing a file and re-mapping) do not need to worry about race conditions of re-opened paths, renamed parent directories and that sort of thing. Operating directly on already open handles is both safer and more efficient than relying upon the stability of filesystem paths.
* stdlib: documenation updates, the exception names have been changedAndreas Rumpf2018-10-251-3/+3
|
* This has been unnecessary as long as rawNewObj has called zeroMem, (#8867)c-blake2018-09-041-1/+0
| | | | and more recently indexing past the Nim-logical end has become illegal making this line cause a crash.
* Don't depend on string.h in codegen (#8299)Yuriy Glukhov2018-07-131-2/+1
|
* make tmemfile2 work againAraq2018-07-061-1/+2
|
* Add MemMapFileStream. Fixes in memFiles. (#7944)Dmitry Atamanov2018-06-141-9/+109
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add MemMapFileStream * Added tests * Fixed bug in memfiles (zero index for string) * Added flush to changelog * Attempt to fix Win's nuances * Fix attempt to fix * Continue... * And again... * Reworked tests (all for win on Win) * Fixes in flush (Win) * Replace fn vars to consts * Added the attempts parameter to the flush * Replace while to for * Move to memfiles * Use Natural instead of uint * Better error messages for append mode. Handle specific cases.
* remove deprecated stuff from the stdlib; introduce better deprecation warningsAraq2018-05-051-2/+0
|
* memfiles: better error checking for Windows; refs #6361Araq2018-01-031-4/+7
|
* Update memfiles.nim (#6328)Denis Rumyantsev2017-09-151-1/+1
| | | FIX error in MemFile fileSize
* Remove expr/stmt (#5857)Arne Döring2017-07-251-3/+3
|
* Fixes incorrect fd==0 test on Unix; Conserves handles by default. (#5512)c-blake2017-03-121-8/+20
| | | | | | | | | | | | | | * Fix 2 problems. First, 0 is a valid fd on Unix (easily gotten if user first closes all fds and then starts using memfiles). Use -1 instead for an invalid fd. Second, it is best practice to conserve open fds on Unix and file handles on Windows. These handles are not needed unless the user wants to remap the memory with ``mapMem`` (or a hypothetical future ``proc resize``). Adding a new bool param ``allowRemap=false`` to ``memfiles.open`` solves this cleanly in a "mostly" backward compatible way. This is only "mostly" because the default ``false`` case does not keep unneeded resources allocated, but that most sensible default means that any ``mapMem`` callers need to fix all their open calls to have allowRemap=true, as this PR also does for tmemfiles2.nim. * Include backwards compatibility note.
* Oops - forgot export marker.Charles Blake2016-12-221-1/+1
|
* Add a proc `==` for MemSlice.Charles Blake2016-12-221-0/+5
|
* Add doco on mapMem(), extend doco on open(), and add extra lines space for ↵James Parkinson2016-08-051-7/+27
| | | | readability
* fix types of ansi_c/sysio to more closely match C ABIJacek Sieka2016-06-051-4/+2
| | | | | also fixes some instances of using C library functions when there are nim alternatives available
* Bug fix: change non-fmRead file mode to set GENERIC_READ or GENERIC_WRITEJamesP2015-09-241-1/+2
| | | | | instead of GENERIC_ALL which on 64bit windows rasied an [OSError] exception
* change type of windows handles to HandleJamesP2015-09-241-3/+3
|
* fix wasOpened reference and set to false on closeJamesP2015-09-231-1/+2
|
* Add wasOpened flag for windows, so not attempt close when unopenedJamesP2015-09-221-1/+4
|
* Undo zero checkJamesP2015-09-221-1/+1
|
* bug fix: close() only unmapViewOfFile() when fHandle is valid.JamesP2015-09-121-1/+1
| | | | | Added extra test on windows close so if already closed it doesn't throw an exception.
* lib: Trim .nim files trailing whitespaceAdam Strzelecki2015-09-041-5/+5
| | | | via OSX: find . -name '*.nim' -exec sed -i '' -E 's/[[:space:]]+$//' {} +
* Add note about what is included.Charles Blake2015-08-041-0/+3
|
* Use hyperlinks to refer to other procs.Charles Blake2015-08-041-6/+8
|
* Expand memSlices doc comment & use better example.Charles Blake2015-08-041-14/+27
| | | | Have lines() iterators docs point back to memSlices() for details.
* Spruce up doc comments as per dom96 suggestions.Charles Blake2015-08-031-7/+12
|
* toString --> `$`.Charles Blake2015-08-031-1/+1
|
* Try to get examples to render right as HTML.Charles Blake2015-08-021-1/+5
|
* Add some example code blocks.Charles Blake2015-08-021-2/+10
|
* Quote the \ in doc comments.Charles Blake2015-08-021-5/+5
|
* Add some little one-line doc comment of MemSlice.Charles Blake2015-08-021-1/+1
|
* Add top of module comment. Re-order definitions toCharles Blake2015-08-021-6/+36
| | | | | | | | | | type, then converter, then 3 iterators lowest- to highest-level (also fastest to slowest) including a new intermediate iterator lines(MemFile, buffer) that is more like readLine(File) in case that helps anyone port code. Add doc comments. Also have toString just use newString+c_memcpy instead of currently fragile toNimStr which Araq wants a separate PR for.
* Nix unchecked which has no semantics for objects.Charles Blake2015-08-021-1/+1
|
* cstring->pointer in MemSlice to make consistent with MemFile andCharles Blake2015-07-291-8/+8
| | | | | also to make more clear the care required to use MemSlice instances. (E.g., memXXX functions rather than strXXX functions).
* Remove NUL termination unneeded post toNimStr fix.Charles Blake2015-07-281-1/+0
|
generated by cgit-pink 1.4.1-2-gfad0 (git 2.36.2.497.gbbea4dcf42) at 2025-02-15 06:42:43 +0000