1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
|
# getwtxt [](https://goreportcard.com/report/github.com/getwtxt/getwtxt) [](https://travis-ci.com/getwtxt/getwtxt)
twtxt registry written in Go!
[twtxt](https://github.com/buckket/twtxt) is a decentralized microblogging platform "for hackers" based
on text files. The user is "followed" and "mentioned" by referencing the URL to
their `twtxt.txt` (or other text) file and a (not necessarily unique) nickname.
Registries are designed to aggregate several users' statuses into a single location,
facilitating the discovery of new users to follow and allowing the search of statuses
for tags and key words.
\[ [Installation](#installation) \] \[ [Configuration](#configuration) \] \[ [Using the Registry](#using-the-registry) \] \[ [Benchmarks](#benchmarks) \] \[ [Notes](#notes) \]
## Features
* Easy to set up and maintain.
* Uses an in-memory cache to serve requests.
* Pushes to `LevelDB` at a configurable interval for data storage.
* Run directly facing the internet or behind `Caddy` / `nginx`.
A public instance is currently available:
* [twtxt.tilde.institute](https://twtxt.tilde.institute)
## Installation
First, fetch the sources using either the `go` tool or using `git`
and jump into the directory.
```
$ go get github.com/getwtxt/getwtxt
...
$ cd $GOPATH/src/github.com/getwtxt/getwtxt
```
```
$ git clone git://github.com/getwtxt/getwtxt.git
...
$ cd getwtxt
```
Optionally, use the `go` tool to test and benchmark it:
```
$ go test -v -bench . -benchmem
```
Use the `go` tool to build:
```
$ go build -v
```
## Configuration
\[ [Proxying](#proxying) \] \[ [Starting getwtxt](#starting-getwtxt) \]
To configure `getwtxt`, you'll first need to open `getwtxt.yml` in your favorite
editor and modify any values necessary. There are comments in the file explaining
each option. Additionally, you may run `getwtxt` with the `-m` flag for a short
configuration manual.
```
$ ./getwtxt -m | less
```
If you desire, you may additionally modify the template in `assets/tmpl/index.html`
to customize the page users will see when they pull up your registry instance in
a web browser. The values in the configuration file under `Instance:` are used
to replace text `{{.Like This}}` in the template.
### Proxying
Though `getwtxt` will run perfectly fine facing the internet directly, it does not
understand virtual hosts, nor does it use TLS (yet). You'll probably want to proxy it behind
`Caddy` or `nginx` for this reason.
`Caddy` is ludicrously easy to set up, and automatically handles `TLS` certificates. Here's the config:
```caddyfile
twtxt.example.com
proxy / example.com:9001
```
If you're using `nginx`, here's a skeleton config to get you started:
```nginx
server {
server_name twtxt.example.com;
listen [::]:443 ssl http2;
listen 0.0.0.0:443 ssl http2;
ssl_certificate /etc/letsencrypt/live/twtxt.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/twtxt.example.com/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass http://127.0.0.1:9010;
}
}
server {
if ($host = twtxt.example.com) {
return 301 https://$host$request_uri;
}
listen 80;
server_name twtxt.example.com;
return 404;
}
```
### Starting `getwtxt`
Once you've customized the configuration, start it within a `tmux` session (or `screen` works) and detach.
If you're using a configuration file not in one of the expected locations or with a non-default name,
start `getwtxt` like this:
```
$ ./getwtxt -c /path/to/configuration/file.yml
```
Otherwise, just:
```
$ ./getwtxt
```
## Using the Registry
The following examples will all apply to using `curl` from a `Linux`, `BSD`, or `macOS` terminal.
All timestamps are in `RFC3339` format, per the twtxt registry specification. Additionally, all
queries support the `?page=N` parameter, where `N` is a positive integer, that will retrieve page
`N` of results in groups of twenty.
### Adding a User
Both nickname and URL are required
```
$ curl -X POST 'https://twtxt.example.com/api/plain/users?url=https://mysite.ext/twtxt.txt&nickname=FooJr'
200 OK
```
### Fetch All Statuses
```
$ curl 'https://twtxt.example.com/api/plain/tweets'
foo_barrington https://foo.bar.ext/twtxt.txt 2019-03-01T09:31:02.000Z Hey! It's my first status!
...
...
```
### Fetch All Users
Timestamp reflects when the user was added to the registry.
```
$ curl 'https://twtxt.example.com/api/plain/users'
foo_barrington https://foo.barrington.ext/twtxt.txt 2017-01-01T09:17:02.000Z
foo_barrington_jr https://example.com/twtxt.txt 2019-03-01T09:31:02.000Z
...
...
```
### Query Users
Can use either keyword or URL.
```
$ curl 'https://twtxt.example.com/api/plain/users?url=https://example.com/twtxt.txt'
foo https://example.com/twtxt.txt 2019-05-09T08:42:23.000Z
$ curl 'https://twtxt.example.com/api/plain/users?q=foo'
foo https://example.com/twtxt.txt 2019-05-09T08:42:23.000Z
foobar https://example2.com/twtxt.txt 2019-03-14T19:23:00.000Z
foo_barrington https://example3.com/twtxt.txt 2019-05-01T15:59:39.000Z
```
### Query by Tag
```
$ curl 'https://twtxt.example.com/api/plain/tags/programming'
foo https://example.com/twtxt.txt 2019-03-01T09:31:02.000Z I love #programming!
```
### Query Tweets by Keyword
```
$ curl 'https://twtxt.example.com/api/plain/tweets?q=getwtxt'
foo_barrington https://example3.com/twtxt.txt 2019-04-30T06:00:09.000Z I just installed getwtxt!
```
## Benchmarks
* [bombardier](https://github.com/codesenberg/bombardier)
```
$ bombardier -c 100 -n 200000 http://localhost:9001/api/plain/tweets
Bombarding http://localhost:9001/api/plain/tweets with 200000 request(s) using 100 connection(s)
200000 / 200000 [=============================================================] 100.00% 15100/s 13s
Done!
Statistics Avg Stdev Max
Reqs/sec 15249.12 3526.87 25047.46
Latency 6.56ms 2.93ms 64.54ms
HTTP codes:
1xx - 0, 2xx - 200000, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 7.83MB/s
```
## <a name="notes"></a>Notes
twtxt Information
* [twtxt.readthedocs.io](https://twtxt.readthedocs.io)
Registry Specification
* [twtxt.readthedocs.io/.../registry.html](https://twtxt.readthedocs.io/en/latest/user/registry.html)
|