3:1
CharTerm: Character-cell Terminal Interface in Racket
| (require (planet neil/charterm:3:1)) |
1 Introduction
The CharTerm package provides a Racket interface for character-cell video
display terminals on Unix-like systems – such as for
GNU Screen and
tmux sessions on
cloud servers,
XTerm windows on a workstation desktop, and some older hardware
terminals (even the venerable
DEC VT100). Currently, it implements a subset of features available on most
terminals.
This package could be used to implement a status/management console
for a Racket-based server process (perhaps run in GNU Screen or tmux on a server machine, to be detached and reattached from SSH
sessions), a lightweight user interface for a systems tool, a command-line
REPL, a text editor, creative retro uses of old equipment, and, perhaps most
importantly, a Rogue-like application.
The CharTerm package does not include any native code (such as from
terminfo,
termcap,
curses, or
ncurses) in the Racket process,
such as through the Racket FFI or C extensions, so there is less potential for
a problem involving native code to threaten the reliability or security of a
program. CharTerm is implemented in pure Racket code except for executing
/bin/stty for some purposes. Specifically,
/bin/stty at startup time and shutdown time, to set modes, and (for terminal
types that don’t seem to support a screen size report control sequence) when
getting screen size. Besides security and stability, lower dependence on
native code might also simplify porting to host platforms that don’t have those
native code facilities.
1.1 Demo
For a demonstration, the following command, run from a terminal, should install the CharTerm package (if not already installed), and run the demo:
racket -pm neil/charterm/demo
This demo reports what keys you pressed, while letting you edit a
text field, and while displaying a clock. The clock is updated roughly once
per second, and is not updated during heavy keyboard input, such as when typing
fast. The demo responds to changing terminal sizes, such as when an XTerm is
window is resized. It also displays the determined terminal size, and some
small tests of the #:width argument to charterm-display. Exit the demo by pressing the Esc key.
Note: Although this demo includes an editable text field, as proof
of concept, the current version of CharTerm does not provide editable text fields as reusable functionality.
1.2 Simple Example
Here’s your first CharTerm program:
| #lang racket/base |
| |
| (require (planet neil/charterm)) |
| |
| (with-charterm |
| (charterm-clear-screen) |
| (charterm-cursor 10 5) |
| (charterm-display "Hello, ") |
| (charterm-bold) |
| (charterm-display "you") |
| (charterm-normal) |
| (charterm-display ".") |
| (charterm-cursor 1 1) |
| (charterm-display "Press a key...") |
| (let ((key (charterm-read-key))) |
| (charterm-cursor 1 1) |
| (charterm-clear-line) |
| (printf "You pressed: ~S\r\n" key))) |
Now you’re living the dream of the ’70s.
2 Terminal Diversity
Like people, few terminals are exactly the same.
Some key (ha) terms (ha) used by CharTerm are:
termvar — a string value like from the Unix-like TERM environment variable, used to determine a default protocol and keydec.
protocol — how to control the display, query for information, etc.
keydec — how to decode key encodings of a particular
terminal. A keydec is constructed from one or more keysets, can produce keycodes or keyinfos.
keyset — a specification of encoding some of the keys in a
particular terminal, including keylabels and keycodes.
keylabel — a string for how a key is likely labeled on a
keyboard, such as the DEC VT100 PF1 key would have a keylabel "PF1" for a keycode 'f1.
keycode — a value produced by a decoded key,
such as a character for normal printable keys, like #\a and #\space, a symbol for some recognized unprintable keys, like 'escape and 'f1, or possibly a number for unrecognized keys.
keyinfo — an object that is used like a keycode, except
bundles together a keycode and a keylabel, as well as alternatate keycodes and
information about how the key was decoded (e.g., from which keyset).
These terms are discussed in the following subsections.
CharTerm is developed with help of original documentation such as that
curated by Paul Williams at
vt100.net, various commentary found on the Web, observed behavior with
modern software terminals like XTerm, various emulators for hardware terminals,
and sometimes original hardware terminals. Thanks to Mark Pearrow for
contributing a TeleVideo 950, and Paul McCabe for a Wyse S50 WinTerm.
At time of this writing, the author is looking to acquire a DEC
VT525, circa 1994, for ongoing testing.
The author welcomes feedback on useful improvements to CharTerm’s support for terminal diversity (no pun). If you have a terminal
that is sending an escape sequence not recognized by the demo, you can run the
demo with the -n (aka --no-escape) argument to see the exact byte sequence:
racket -pm- neil/charterm/demo -n
When -n is used, this will be indicated by the bottom-most scrolling line,
rather than saying “To quit, press Esc.” instead will say “There is no escape from this demo.” You will have to kill the process through some other means.
2.1 Protocol
The first concept CharTerm has for distinguishing how to communicate with a terminal is what
is what is called here
protocol, which concerns everything except how keyboard keys are decoded.
The following protocols are currently implemented:
ansi protocol — Terminals approximating [ANSI X3.64], which is most terminals in use today, including software ones
like XTerm. This protocol is the emphasis of this package; the other protocols
are for unusual situations.
wyse-wy50 protocol — Terminals compatible with the Wyse WY-50. This support is
based on [WY-50-QRG], [WY-60-UG], [wy60], and [PowerTerm]. Note that video attributes are not supported, due to the WY-50’s
model of having video attribute changes occupy character cells; you may wish
to run the Wyse terminal in an ANSI or VT100 mode.
televideo-925 protocol — Terminals compatible with the TeleVideo 925. This support is based on [TVI-925-IUG] and behavior of [PowerTerm]. Note that video attributes are not supported, due to the 925’s
model of having video attribute changes occupy character cells; you may wish to
run your TeleVideo terminal in ANSI or VT100 mode, if it has one.
ascii protocol — Terminals that support ASCII but not much else that we know about.
2.2 Key Encoding
While most video display control, they seem to vary more by key
encoding.
The CharTerm author was motivated to increase the sophistication of its
keyboard handling after a series of revelations on the Sunday of the long
weekend in which CharTerm was initially written. The first was discovering that four of the
function keys that had been working fine in
rxvt did not work in XTerm. Dave Gilbert somewhat demystified this by
pointing out that the original VT100 had only four function keys, which set
into motion an unfortunate series of bad decisions by various developers of
terminal software to be needlessly incompatible with each other. After
Googling, a horrifying 2005 Web post by Phil Gregory [
Gregory], which showed that key encoding among XTerm variants was even
worse than one could ever fear. Even if one already knew how much subtleties
of old terminals varied (e.g., auto-newline behavior, whether an attribute
change consumed a space, etc.), this incompatibility in newer software was
surprising. Then, on a hunch, I tried the Linux Console on a Debian Squeeze
machine, which surely is ANSI, and found, however, that it generated
yet different byte sequences, for the first
five (not four) function keys. Then I compared all to the [
ECMA-48] standard, which turns out to be nigh-inscrutable, so which might
help explain why everyone became so anti-social.
CharTerm now provides the abstractions of
keysets and
keydecs to deal with this diversity in a maintainable way.
2.2.1 Keylabel
A keylabel is a Racket string for how a key is likely labeled on a particular terminal’s keyboard. Different keyboards may have different keylabels for the same keycode. For example, a VT100 has a PF1 key (keylabel "PF1", keycode 'f1), while many other keyboards would label the key F1 (keylabel "F1", keycode 'f1). The keylabel currently is most useful for documenting and debugging, although it could later be used when giving instructions to the user, such as knowing whether to tell the user the Return key or the Enter key; the Backspace or the Rubout key; etc.
2.2.2 Keycode
A
keycode is a value representing a key read from a terminal, which can be a Racket character, symbol, or number. Keys corresponding to printable characters have keycodes as Racket characters. Some keys corresponding to special non-printable characters can have keycodes of Racket symbols, such as
'return,
'f1,
'up, etc.
(charterm-keycode? x) → boolean?
|
| x : any/c |
Predicate for whether or not x is a valid keycode.
2.2.3 Keyinfo
A
keyinfo represents a
keycode for a key, a
keylabel, and how it is encoded as bytes. It is represented in Racket as
a
charterm-keyinfo object.
(charterm-keyinfo? x) → boolean?
|
| x : any/c |
Predicate for whether or not x is a charterm-keyinfo object.
(charterm-keyinfo-keyset-id ki) → symbol?
|
| ki : charterm-keyinfo? |
| (charterm-keyinfo-bytelang ki) → string? |
| ki : charterm-keyinfo? |
| (charterm-keyinfo-bytelist ki) → (listof byte?) |
| ki : charterm-keyinfo? |
| (charterm-keyinfo-keylabel ki) → string? |
| ki : charterm-keyinfo? |
| (charterm-keyinfo-keycode ki) → charterm-keycode? |
| ki : charterm-keyinfo? |
| (charterm-keyinfo-all-keycodes ki) → (listof charterm-keycode?) |
| ki : charterm-keyinfo? |
Get information from a charterm-keyinfo object.
2.2.4 Keyset
A
keyset is a specification of keys on a particular keyboard, including their
keylabel, encoding as bytes, and primary and alternate
keycodes.
The means of constructing a keyset is currently internal to this package.
(charterm-keyset? x) → boolean?
|
| x : any/c |
Predicate for whether or not x is a keyset.
(charterm-keyset-id ks) → symbol?
|
| ks : charterm-keyset? |
Get a symbol identifying the keyset.
charterm-ascii-keyset : charterm-keyset?
|
From the old [
ASCII] standard. When defining a
keydec, this is good to have as a final keyset, after the others.
charterm-dec-vt100-keyset : charterm-keyset?
|
From the DEC VT100. This currently defines the four function
keys (labeled on the keyboard,
PF1 through
PF4) as
'f1 through
'f4, and the arrow keys. [
VT100-UG] and [
PowerTerm] were used as references.
charterm-dec-vt220-keyset : charterm-keyset?
|
From the DEC VT220. This currently defines function keys F1 through F20.
charterm-screen-keyset : charterm-keyset?
|
charterm-linux-keyset : charterm-keyset?
|
From the Linux console. Currently defines function keys F1 through F5 only, since the rest will be inherited from other keysets.
charterm-xterm-x11r6-keyset : charterm-keyset?
|
From the XTerm in X11R6, according to [
Gregory].
charterm-xterm-xfree86-keyset : charterm-keyset?
|
From the XFree86 XTerm, according to [
Gregory].
charterm-xterm-new-keyset : charterm-keyset?
|
From the current
xterm-new, often called simply
xterm, as developed by Thomas E. Dickey, and documented in [
XTerm-ctlseqs]. Several also came from decompiling a
terminfo entry. Thanks to Dickey for his emailed help.
charterm-rxvt-keyset : charterm-keyset?
|
From the
rxvt terminal emulator. These come from [
Gregory], and
currently define function keys
'f1 through
'f44.
charterm-wyse-wy50-keyset : charterm-keyset?
|
From the Wyse WY-50, based on [
WY-50-QRG] and looking at photos of WY-50 keyboard, and tested in [
wy60] and [
PowerTerm]. The shifted function keys are provided as both
'shift-f1 through
'shift-16, and
'f17 through
'f31.
charterm-televideo-925-keyset : charterm-keyset?
|
From the TeleVideo 925, based on [
TVI-925-IUG], [
PowerTerm], and from looking at a TeleVideo 950 keyboard.
2.2.5 Keydec
A
keydec object is a key decoder for a specific variety of terminal, such
as for a specific
termvar. A keydec is used to turn received key encodings from a terminal into
keycode or
keyinfo values. A keydec is constructed from a prioritized list of
keyset objects, with earlier-listed keysets taking priority of
later-listed keysets when there is conflict between them as to how to decode a
particular byte sequence.
(charterm-keydec-id kd) → symbol?
|
| kd : charterm-keydec? |
Gets the ID symbol of the
keydec being used.
ANSI Keydecs
charterm-vt100-keydec : charterm-keydec?
|
charterm-vt220-keydec : charterm-keydec?
|
charterm-screen-keydec : charterm-keydec?
|
charterm-linux-keydec : charterm-keydec?
|
charterm-xterm-new-keydec : charterm-keydec?
|
charterm-xterm-keydec : charterm-keydec?
|
Keydec for
termvar "xterm". Currently same as the keydec for
xterm, except for a different ID.
charterm-rxvt-keydec : charterm-keydec?
|
Wyse Keydecs
charterm-wy50-keydec : charterm-keydec?
|
TeleVideo Keydecs
charterm-tvi925-keydec : charterm-keydec?
|
ASCII Keydecs
charterm-ascii-keydec : charterm-keydec?
|
Default Keydecs
charterm-ansi-keydec : charterm-keydec?
|
Keydec for any presumed ANSI-ish terminal, combining many ANSI-ish
keysets.
charterm-insane-keydec : charterm-keydec?
|
Keydec for the uniquely desperate situation of wanting to possibly have
extensive key decoding for a terminal that might not even be ansi, but be
Wyse, TeleVideo, or some other ASCII.
2.3 Termvar
A termvar is what the charterm package calls the value of the Unix-like TERM environment variable. Each termvar has a default protocol and keydec. Note, however, that TERM is not always a precise indicator of the best protocol and keydec,
but by default we work with what we have.
3 charterm Object
The charterm object captures the state of a session with a particular terminal.
A charterm object is also a synchronizable event, so it can be used with
procedures such as sync. As an event, it becomes ready when there is at least one byte
available for reading from the terminal, and its synchronization result is
itself.
(charterm? x) → boolean?
|
| x : any/c |
Predicate for whether or not x is a charterm.
(charterm-termvar ct) → (or/c #f string?)
|
| ct : charterm? |
(charterm-protocol ct) → symbol?
|
| ct : charterm? |
(charterm-keydec ct) → symbol?
|
| ct : charterm? |
(current-charterm) → (or/c #f charterm?)
|
| (current-charterm ct) → void? |
| ct : (or/c #f charterm?) |
This parameter provides the default charterm for most of the other procedures. It is usually set automatically by call-with-charterm, with-charterm, open-charterm, and close-charterm.
| (open-charterm | [ | #:tty tty | | | | | | | | #:current? current?]) | | → | | charterm? |
|
| tty : (or/c #f path-string?) = #f |
| current? : boolean? = #t |
Returns an open charterm object, by opening I/O ports on the terminal device at tty (or, if #f, file "/dev/tty"), and setting raw mode and disabling echo (via "/bin/stty"). If current? is true, the current-charterm parameter is also set to this object.
(close-charterm [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
Closes ct by closing the I/O ports, and undoing open-charterm’s changes via "/bin/stty". If current-charterm is set to ct, then that parameter will be changed to #f for good measure. You might wish to use with-charterm instead of worrying about calling close-charterm directly.
Note: If you exit your Racket process without properly closing the charterm, your terminal may be left in a crazy state. You can fix it with
the command:
stty sane
(with-charterm expr? ...)
|
Opens a charterm and evaluates the body expressions in sequence with current-charterm set appropriately. When control jumps out of the body, in a
manner of speaking, the charterm is closed.
4 Terminal Information
(charterm-screen-size [#:charterm ct])
|
| | → | | | (or/c #f exact-nonnegative-integer?) | | (or/c #f exact-nonnegative-integer?) |
|
|
| ct : charterm? = (current-charterm) |
Attempts to get the screen size, in character columns and rows.
It may do this through a control sequence or through /bin/stty. If unable to get a value, then default of (80,24) is used.
The current behavior in this version of charterm is to adaptively try different methods of getting screen size,
and to remember what worked for the next time this procedure is called for ct. For terminals that are identified as screen by the TERM environment variable (e.g., terminal emulators like GNU Screen
and tmux), the current behavior is to not try the control sequence (which
causes a 1-second delay waiting for a terminal response that never arrives),
and to just use stty. For all other terminals, the control sequence is tried first, before trying stty. If neither the control sequence nor stty work, then neither method is tried again for ct, and instead the procedure always returns (#f, #f). This behavior very well might change in future versions of charterm, and the author welcomes feedback on which methods work with
which terminals.
5 Display Control
5.1 Cursor
(charterm-cursor x y [#:charterm ct]) → void?
|
| x : exact-positive-integer? |
| y : exact-positive-integer? |
| ct : charterm? = (current-charterm) |
Positions the cursor at column x, row y, with the upper-left character cell being (1, 1).
(charterm-newline [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
Sends a newline to the terminal. This is typically a CR-LF
sequence.
5.2 Displaying
| (charterm-display | [ | #:charterm ct | | | | | | | | #:width width | | | | | | | | #:pad pad | | | | | | | | #:truncate truncate] | | | | | | | | arg ...) | | → | | void? |
|
| ct : charterm? = (current-charterm) |
| width : (or/c #f exact-positive-integer?) = #f |
| pad : (or/c 'width boolean?) = 'width |
| truncate : (or/c 'width boolean?) = 'width |
| arg : any/c |
Displays each arg on the terminal, as if formatted by display, with the exception that unprintable or non-ASCII characters
might not be displayed. (The exact behavior of what is permitted is expected
to change in a later version of CharTerm, so avoid trying to send your own control sequences or using
newlines, making assumptions about non-ASCII characters, etc.)
If width is a number, then pad and truncate specify whether or not to pad with spaces or truncate the output, respectively, to width characters. When pad or width is 'width, that is a convenience meaning “true if, and only if, width is not #f.”
5.3 Video Attributes
(charterm-normal [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
| (charterm-inverse [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
| (charterm-underline [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
| (charterm-blink [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
| (charterm-bold [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
Sets the
video attributes for subsequent writes to the terminal. In this version of
charterm, each is mutually-exclusive, so, for example, setting
bold clears
inverse. Note that that video attributes are currently supported only for protocol
'ansi, due to limitations of the TeleVideo and Wyse models for
video attributes.
5.4 Clearing
(charterm-clear-screen [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
Clears the screen, including first setting the video attributes to
normal, and positioning the cursor at (1, 1).
(charterm-clear-line [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
| (charterm-clear-line-left [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
| (charterm-clear-line-right [#:charterm ct]) → void? |
| ct : charterm? = (current-charterm) |
Clears text from the line with the cursor, or part of the line with the cursor.
5.5 Line Insert and Delete
(charterm-insert-line [count #:charterm ct]) → void?
|
| count : exact-positive-integer? = 1 |
| ct : charterm? = (current-charterm) |
Inserts count blank lines at cursor. Note that not all terminals support
this.
(charterm-delete-line [count #:charterm ct]) → void?
|
| count : exact-positive-integer? = 1 |
| ct : charterm? = (current-charterm) |
Deletes count blank lines at cursor. Note that not all terminals support
this.
5.5.1 Misc. Output
(charterm-bell [#:charterm ct]) → void?
|
| ct : charterm? = (current-charterm) |
Rings the terminal bell. This bell ringing might manifest as a
beep, a flash of the screen, or nothing.
6 Keyboard Input
Normally you will get keyboard input using the charterm-read-key procedure.
(charterm-byte-ready? [#:charterm ct]) → boolean?
|
| ct : charterm? = (current-charterm) |
Returns true/false for whether at least one byte is ready for
reading (either in a buffer or on the port) from ct. Note that, since some keys are encoded as multiple bytes, just
because this procedure returns true doesn’t mean that charterm-read-key won’t block temporarily because it sees part of a potential
multiple-byte key encoding.
| (charterm-read-key | [ | #:charterm ct | | | | | | | | #:timeout timeout]) | | → | | (or #f char? symbol?) |
|
| ct : charterm? = (current-charterm) |
| timeout : (or/c #f positive?) = #f |
Reads a key from ct, blocking indefinitely or until sometime after timeout seconds has been reached, if timeout is non-#f. If timeout is reached, #f is returned.
Many keys are returned as characters, especially ones that
correspond to printable characters. For example, the unshifted Q key is returned as character #\q. Some other keys are returned as symbols, such as 'return, 'escape, 'f1, 'shift-f12, 'right, and many others.
Since some keys are sent as ambiguous sequences, charterm-read-key employs separate timeouts internally, such as to disambuate
the Esc key (byte sequence 27) from what on some terminals would be
the F10 key (bytes sequence 27, 91, 50, 49, 126).
| (charterm-read-keyinfo | [ | #:charterm ct | | | | | | | | #:timeout timeout]) | | → | | charterm-keyinfo? |
|
| ct : charterm? = (current-charterm) |
| timeout : (or/c #f positive?) = #f |
Like
charterm-read-keyinfo except instead of returning a
keycode, it returns a
keyinfo.
7 References
[
PowerTerm] Ericom PowerTerm InterConnect 8.2.0.1000 terminal emulator, as run on Wyse S50 WinTerm
[
VT100-WP] Wikipedia,
VT100[
XTerm-FAQ] Thomas E. Dickey, “
XTerm FAQ,” dated 2012
[
XTerm-WP] Wikipedia,
xterm8 Known Issues
Need to support ANSI alternate CSI for 8-bit terminals, even
before supporting 8-bit characters and multibyte.
Only supports ASCII characters. Adding UTF-8 support, for terminal emulators
that support it, would be nice.
Expose the character-decoding mini-language as a configurable
option. Perhaps wait until we implement timeout-based disambiguation at
arbitrary points in the the DFA rather than just at the top. Also, might be
better to resolve multi-byte characters first, in case that affects the
mini-language.
More controls for terminal features can be added.
Currently only implemented to work on Unix-like systems like
GNU/Linux.
Implement text input controls, either as part of this library or
another, using charterm-demo as a starting point.
9 History
PLaneT 3:1 — 2013-05-13
PLaneT 3:0 — 2012-07-13
Changed “ansi-ish” in identifiers to “ansi”, hence the PLaneT major version number change.
Documentation tweaks.
Renamed package from “charterm” to “CharTerm”.
PLaneT 2:5 — 2012-06-28
PLaneT 2:4 — 2012-06-25
PLaneT 2:3 — 2012-06-25
PLaneT 2:2 — 2012-06-25
PLaneT 2:1 — 2012-06-24
PLaneT 2:0 — 2012-06-24
Greatly increased the sophistication of handling of terminal diversity.
Added the wyse-wy50 and televideo-950 [Correction: televideo-925] protocols, for supporting the native modes of Wyse and
TeleVideo terminals, respectively, and compatibles.
More support for different key encodings and termvars.
Demo is now in a separate file, mainly for convenience
in giving command lines that run it. This breaks a command line example
previously documented, so changed PLaneT major version, although the
previously-published example will need to have :1 added to it anyway.
charterm-screen-size now defaults to (80,24) when all else fails.
Documentation changes.
PLaneT 1:1 — 2012-06-17
For screen and tmux, now gets screen size via stty. This resolves the sluggishness reported with screen. [Correction: In version 1:1, this behavior is
adaptive for all terminals, with the shortcut for termvar screen that it doesn’t bother trying the control sequence.]
Documentation tweaks.
PLaneT 1:0 — 2012-06-16
10 Legal
Copyright 2012 – 2013 Neil Van Dyke. This program is Free Software; you
can redistribute it and/or modify it under the terms of the GNU Lesser General
Public License as published by the Free Software Foundation; either version 3
of the License, or (at your option) any later version. This program is
distributed in the hope that it will be useful, but without any warranty;
without even the implied warranty of merchantability or fitness for a
particular purpose. See http://www.gnu.org/licenses/ for details. For other
licenses and consulting, please contact the author.