e9da83d268
This patch emulates functionality already in cmd_run, in the context of websockets. A client running "[P]SUBSCRIBE" will restrict the commands allowed to be issued in the context used. Therefore, websocket clients making subscriptions get their own context, so as to not pollute the workers' own Redis contexts. Resource/context deallocation should occur upon http client disconnect, just as it is for client-specific contexts from cmd_run. |
13 years ago | |
---|---|---|
formats | 13 years ago | |
hiredis | 13 years ago | |
http-parser | 14 years ago | |
jansson | 14 years ago | |
libb64 | 14 years ago | |
md5 | 14 years ago | |
sha1 | 13 years ago | |
tests | 13 years ago | |
.gitignore | 14 years ago | |
COPYING | 14 years ago | |
Makefile | 13 years ago | |
README.markdown | 13 years ago | |
acl.c | 14 years ago | |
acl.h | 14 years ago | |
client.c | 13 years ago | |
client.h | 13 years ago | |
cmd.c | 13 years ago | |
cmd.h | 13 years ago | |
conf.c | 13 years ago | |
conf.h | 13 years ago | |
http.c | 13 years ago | |
http.h | 13 years ago | |
pool.c | 14 years ago | |
pool.h | 14 years ago | |
server.c | 13 years ago | |
server.h | 14 years ago | |
slog.c | 14 years ago | |
slog.h | 14 years ago | |
webdis.c | 14 years ago | |
webdis.json | 13 years ago | |
webdis.prod.json | 13 years ago | |
websocket.c | 13 years ago | |
websocket.h | 13 years ago | |
worker.c | 13 years ago | |
worker.h | 14 years ago |
README.markdown
About
A very simple web server providing an HTTP interface to Redis. It uses hiredis, jansson, libevent, and http-parser.
Webdis depends on libevent-dev. You can install it on Ubuntu by typing sudo apt-get install libevent-dev
or on OS X by typing brew install libevent
.
make clean all ./webdis & curl http://127.0.0.1:7379/SET/hello/world → {"SET":[true,"OK"]} curl http://127.0.0.1:7379/GET/hello → {"GET":"world"} curl -d "GET/hello" http://127.0.0.1:7379/ → {"GET":"world"}
Features
- GET and POST are supported, as well as
PUT
for file uploads. - JSON output by default, optional JSONP parameter (
?jsonp=myFunction
or?callback=myFunction
). - Raw Redis 2.0 protocol output with
.raw
suffix - BSON support for compact responses and MongoDB compatibility.
- MessagePack output with
.msg
suffix - HTTP 1.1 pipelining (70,000 http requests per second on a desktop Linux machine.)
- Multi-threaded server, configurable number of worker threads.
- WebSocket support (Currently using the “hixie-76” specification).
- Connects to Redis using a TCP or UNIX socket.
- Restricted commands by IP range (CIDR subnet + mask) or HTTP Basic Auth, returning 403 errors.
- Possible Redis authentication in the config file.
- Pub/Sub using
Transfer-Encoding: chunked
, works with JSONP as well. Webdis can be used as a Comet server. - Drop privileges on startup.
- Custom Content-Type using a pre-defined file extension, or with
?type=some/thing
. - URL-encoded parameters for binary data or slashes and question marks. For instance,
%2f
is decoded as/
but not used as a command separator. - Logs, with a configurable verbosity.
- Cross-origin requests, usable with XMLHttpRequest2 (Cross-Origin Resource Sharing - CORS).
- File upload with PUT.
- With the JSON output, the return value of INFO is parsed and transformed into an object.
- Optional daemonize.
- Default root object: Add
"default_root": "/GET/index.html"
in webdis.json to substitute the request to/
with a Redis request. - HTTP request limit with
http_max_request_size
(in bytes, set to 128MB by default).
Ideas, TODO...
- Add better support for PUT, DELETE, HEAD, OPTIONS? How? For which commands?
- This could be done using a “strict mode” with a table of commands and the verbs that can/must be used with each command. Strict mode would be optional, configurable. How would webdis know of new commands remains to be determined.
- MULTI/EXEC/DISCARD/WATCH are disabled at the moment; find a way to use them.
- Support POST of raw Redis protocol data, and execute the whole thing. This could be useful for MULTI/EXEC transactions.
- Enrich config file:
- Provide timeout (maybe for some commands only?). What should the response be? 504 Gateway Timeout? 503 Service Unavailable?
- Multi-server support, using consistent hashing.
- Database selection in the URL? e.g.
/7/GET/key
to run the command on DB 7.- This might not be very useful, databases will be deprecated from Redis at some point.
- SSL?
- Not sure if this is such a good idea.
- SPDY?
- SPDY is mostly useful for parallel fetches. Not sure if it would make sense for Webdis.
- Send your ideas using the github tracker, on twitter @yowgi or by mail to n.favrefelix@gmail.com.
HTTP error codes
- Unknown HTTP verb: 405 Method Not Allowed.
- Redis is unreachable: 503 Service Unavailable.
- Matching ETag sent using
If-None-Match
: 304 Not Modified. - Could also be used:
- Timeout on the redis side: 503 Service Unavailable.
- Missing key: 404 Not Found.
- Unauthorized command (disabled in config file): 403 Forbidden.
Command format
The URI /COMMAND/arg0/arg1/.../argN.ext
executes the command on Redis and returns the response to the client. GET and POST are supported:
GET /COMMAND/arg0/.../argN.ext
POST /
withCOMMAND/arg0/.../argN
in the HTTP body.
.ext
is an optional extension; it is not read as part of the last argument but only represents the output format. Several formats are available (see below).
Special characters: /
and .
have special meanings, /
separates arguments and .
changes the Content-Type. They can be replaced by %2f
and %2e
, respectively.
ACL
Access control is configured in webdis.json
. Each configuration tries to match a client profile according to two criterias:
- CIDR subnet + mask
- HTTP Basic Auth in the format of "user:password".
Each ACL contains two lists of commands, enabled
and disabled
. All commands being enabled by default, it is up to the administrator to disable or re-enable them on a per-profile basis.
Examples:
{ "disabled": ["DEBUG", "FLUSHDB", "FLUSHALL"], }, { "http_basic_auth": "user:password", "disabled": ["DEBUG", "FLUSHDB", "FLUSHALL"], "enabled": ["SET"] }, { "ip": "192.168.10.0/24", "enabled": ["SET"] }, { "http_basic_auth": "user:password", "ip": "192.168.10.0/24", "enabled": ["SET", "DEL"] }
ACLs are interpreted in order, later authorizations superseding earlier ones if a client matches several. The special value "*" matches all commands.
JSON output
JSON is the default output format. Each command returns a JSON object with the command as a key and the result as a value.
Examples:
// string $ curl http://127.0.0.1:7379/GET/y {"GET":"41"} // number $ curl http://127.0.0.1:7379/INCR/y {"INCR":42} // list $ curl http://127.0.0.1:7379/LRANGE/x/0/1 {"LRANGE":["abc","def"]} // status $ curl http://127.0.0.1:7379/TYPE/y {"TYPE":[true,"string"]} // error, which is basically a status $ curl http://127.0.0.1:7379/MAKE-ME-COFFEE {"MAKE-ME-COFFEE":[false,"ERR unknown command 'MAKE-ME-COFFEE'"]} // JSONP callback: $ curl "http://127.0.0.1:7379/TYPE/y?jsonp=myCustomFunction" myCustomFunction({"TYPE":[true,"string"]})
RAW output
This is the raw output of Redis; enable it with the .raw
suffix.
// string $ curl http://127.0.0.1:7379/GET/z.raw $5 hello // number curl http://127.0.0.1:7379/INCR/a.raw :2 // list $ curl http://127.0.0.1:7379/LRANGE/x/0/-1.raw *2 $3 abc $3 def // status $ curl http://127.0.0.1:7379/TYPE/y.raw +zset // error, which is basically a status $ curl http://127.0.0.1:7379/MAKE-ME-COFFEE.raw -ERR unknown command 'MAKE-ME-COFFEE'
Custom content-type
Several content-types are available:
.json
forapplication/json
(this is the default Content-Type)..bson
forapplication/bson
. See http://bsonspec.org/ for the specs..msg
forapplication/x-msgpack
. See http://msgpack.org/ for the specs..txt
fortext/plain
.html
fortext/html
xhtml
forapplication/xhtml+xml
xml
fortext/xml
.png
forimage/png
jpg
orjpeg
forimage/jpeg
- Any other with the
?type=anything/youwant
query string. - Add a custom separator for list responses with
?sep=,
query string.
curl -v "http://127.0.0.1:7379/GET/hello.html" [...] < HTTP/1.1 200 OK < Content-Type: text/html < Date: Mon, 03 Jan 2011 20:43:36 GMT < Content-Length: 137 < <!DOCTYPE html> <html> [...] </html> curl -v "http://127.0.0.1:7379/GET/hello.txt" [...] < HTTP/1.1 200 OK < Content-Type: text/plain < Date: Mon, 03 Jan 2011 20:43:36 GMT < Content-Length: 137 [...] curl -v "http://127.0.0.1:7379/GET/big-file?type=application/pdf" [...] < HTTP/1.1 200 OK < Content-Type: application/pdf < Date: Mon, 03 Jan 2011 20:45:12 GMT [...]
File upload
Webdis supports file upload using HTTP PUT. The command URI is slightly different, as the last argument is taken from the HTTP body.
For example: instead of /SET/key/value
, the URI becomes /SET/key
and the value is the entirety of the body. This works for other commands such as LPUSH, etc.
Uploading a binary file to webdis:
$ file redis-logo.png redis-logo.png: PNG image, 513 x 197, 8-bit/color RGBA, non-interlaced $ wc -c redis-logo.png 16744 redis-logo.png $ curl -v --upload-file redis-logo.png http://127.0.0.1:7379/SET/logo [...] > PUT /SET/logo HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 127.0.0.1:7379 > Accept: */* > Content-Length: 16744 > Expect: 100-continue > < HTTP/1.1 100 Continue < HTTP/1.1 200 OK < Content-Type: application/json < ETag: "0db1124cf79ffeb80aff6d199d5822f8" < Date: Sun, 09 Jan 2011 16:48:19 GMT < Content-Length: 19 < {"SET":[true,"OK"]} $ curl -vs http://127.0.0.1:7379/GET/logo.png -o out.png > GET /GET/logo.png HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 127.0.0.1:7379 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: image/png < ETag: "1991df597267d70bf9066a7d11969da0" < Date: Sun, 09 Jan 2011 16:50:51 GMT < Content-Length: 16744 $ md5sum redis-logo.png out.png 1991df597267d70bf9066a7d11969da0 redis-logo.png 1991df597267d70bf9066a7d11969da0 out.png
The file was uploaded and re-downloaded properly: it has the same hash and the content-type was set properly thanks to the .png
extension.
WebSockets
Webdis supports WebSocket clients implementing dixie-76.
Web Sockets are supported with the following formats, selected by the connection URL:
- JSON (on
/
or/.json
) - Raw Redis wire protocol (on
/.raw
)
Example:
function testJSON() { var jsonSocket = new WebSocket("ws://127.0.0.1:7379/.json"); jsonSocket.onopen = function() { console.log("JSON socket connected!"); jsonSocket.send(JSON.stringify(["SET", "hello", "world"])); jsonSocket.send(JSON.stringify(["GET", "hello"])); }; jsonSocket.onmessage = function(messageEvent) { console.log("JSON received:", messageEvent.data); }; } testJSON();
This produces the following output:
JSON socket connected! JSON received: {"SET":[true,"OK"]} JSON received: {"GET":"world"}