599 Commits (6539ae13f968132a57a2a2922cc92b71b14c0b65)
 

Author SHA1 Message Date
Jessie Murray e26d6358e7
WS: Better reuse of the cmd struct for WS clients
For WS clients, reuse a persistent cmd struct attached to the
http_client object: take the cmd built from the WS frame, and copy it to
the persistent cmd.
3 years ago
Jessie Murray b98116abc8
Persistent cmd for WS, write buffer for responses
1. Only HTTP-based pub-sub clients were re-using a cmd object, but
   WS clients were not. This led to the commands sent by a WS client
   to be processed out of order, just queued to Redis but with no
   guarantee that they would be de-queued from the event loop in the
   same order. This change attaches a permanent cmd object (with its
   associated Redis context) to WS clients just like pub-sub clients
   do.

2. WS responses are also no longer sent out of order, but added to a
   write buffer that is scheduled for writing as long as there is still
   some data left to send. This replaces the use of http_response which
   contained extra fields (headers, HTTP response) that were duplicated
   without ever being sent out.
3 years ago
Jessie Murray 052458e876
Refactor WS code building raw http_response objects
Both ws_handshake_reply and ws_reply build http_response objects without
using the status code or headers, this code can be refactored to use a
single method.
3 years ago
Jessie Murray 6383cd48dd
Add pub-sub test using WebSockets (disabled)
Add new pub-sub test using WebSockets, disabled by default due to
message ordering not matching what is expected.
Enable the test with `PUBSUB=1 ./ws-tests.py`
3 years ago
Jessie Murray f86bad3bc8
Add Python-based WebSockets tests
1. Create WS tests with the same structure as basic.py
2. Add JSON tests
3. Add "raw" tests
3 years ago
Jessie Murray 8d1b6c40f8
Fix instant vs overall rate in websocket C test
The overall rate was incorrect when using non-default intervals. Also
simplify the code using plain nanosecond values, and fix the rate for
the last event which was based on the number of sleeps instead of the
actual time elapsed.
3 years ago
Jessie Murray 27ad2413d4
Immediately close WS connection on error
The implementation was waiting for the client, which leaves some hanging
even after they called close(). This mirrors the behavior for HTTP
connections in client.c where close() is called right before
http_client_free.
3 years ago
Jessie Murray d0acdf030e
Report WS disconnection at DEBUG to avoid log spam 3 years ago
Jessie Murray 974556defb
WS headers: change order, make origin optional
1. Origin and Sec-WebSocket-Origin are now optional: only return a matching
Sec-WebSocket-Origin if one of the two headers was provided.
2. Change order of headers: return Sec-WebSocket-Accept immediately
   after Upgrade and Connection since some clients expect it there.
3 years ago
Nicolas Favre-Felix 26c74f61f6
Clean "*.d" dependency files in "make clean" 3 years ago
Nicolas Favre-Felix 5c223e7869
Merge pull request #198 from jessie-murray/websocket-fixes 3 years ago
Jessie Murray 1204f66ec1
Add progress thread w/ custom interval for WS test
* Start progress thread that reports once a second by default
* Customizable rate with [--interval|-i]
* Portable semaphore tested on Linux & macOS
3 years ago
Jessie Murray 69be409b87
PR review: unused code, comment style 3 years ago
Jessie Murray 03ff07252e
Clean up http_response object when write cannot be scheduled
If the client closes the connection before we can even start to respond,
the event_add will fail and we need to clean up the http_response object
since this would have happened only after we've sent the whole response.
3 years ago
Jessie Murray 273e2a918d
Log on WS client error 3 years ago
Jessie Murray 8f70db179f
Send WS handshake using event loop instead of write(2)
ws_handshake_reply was sending its response using a single write(2) call
without error-checking, which could well send only part of it and leave
the client hanging. This change creates an HTTP response object and
schedules it for writing using the event loop.

Tested with Valgrind, no memory is leaked.
3 years ago
Jessie Murray 4c0d3f4032
Refactor websocket.html to remove duplicate HTML 3 years ago
Jessie Murray 05a90b7b7a
Add "PING" button to HTML demo 3 years ago
Nicolas Favre-Felix 4471b71074
Disable pub/sub tests for now, rename curl-tests function 3 years ago
Jessie Murray e45779b303
Change TRACE log level to a value greater than was ever in webdis.json 3 years ago
Jessie Murray 3984fb968f
Validate Sec-WebSocket-Accept in WS test
1. Generate random WS connection key
2. Read headers returned in WS connection (upgrade) response
3. Validate Sec-WebSocket-Accept return value from webdis

Tested by introducing an artificial error in webdis, reported as
expected.
3 years ago
Jessie Murray d5dc3d3ff0
Add more error-checking to WS handshake code 3 years ago
Jessie Murray 657197f737
websocket test: make WS mask configurable 3 years ago
Jessie Murray fd8f0b16e0
websocket test: remove special single-threaded mode 3 years ago
Jessie Murray 9a758c6bcf
websocket test: make client sockets non-blocking 3 years ago
Jessie Murray 8141c00ab7
websocket test: formatting/indentation 3 years ago
Jessie Murray e97056f4cf
Add DEBUG=1 logic in tests/Makefile 3 years ago
Jessie Murray c4ac66f78b
Remove extra trace logs in WS-related code 3 years ago
Jessie Murray a666d52471
Requested changes: logger
* Better log level check
* Add output letter for WEBDIS_TRACE logs
3 years ago
Nicolas Favre-Felix 7cec889947
Allow manual runs of "build" GitHub Action 3 years ago
Nicolas Favre-Felix cf0f77907e
Clean up curl test 3 years ago
Nicolas Favre-Felix 2b69ead0a3
Add pub-sub test in basic.py 3 years ago
Jessie Murray b5b76c9c8f
Reduce default number of websocket messages to test for
2,500 per client with 4 clients: 10,000 total
3 years ago
Jessie Murray 665fc19126
Add max execution time to tests/websocket.c
* Add --max-time in seconds
* Break client loop when elapsed
3 years ago
Jessie Murray 2576a1606d
Websocket test: clean up usage(), exit code
* Better usage() help page
* Do not submit another frame once we've sent all we needed
3 years ago
Jessie Murray a15fb4e28e
Modernize websocket.html web page
* Use pure.css for a basic grid
* Detect disconnections, update UI accordingly
* Make GET/SET commands configurable and interactive
* Add button to clear logs
* Test with current branch
3 years ago
Jessie Murray b46bb8504b
Mark websocket http responses as keep_alive
The keep_alive flag is needed on http_response for websocket
connections. Without it, the server closes the connection as soon as a
reply to the first frame is sent.
3 years ago
Jessie Murray 71a30e9d3a
Lower default verbosity level for webdis.json
In prod config, level is even lower and does not need to change
3 years ago
Jessie Murray 873f1b5902
Add trace logs to websocket code, event loop functions
Mostly adding trace logs to websocket.c, but also some in http.c and
worker.c for events relating to event loop and client requests and
responses. This is useful for debugging websocket issues.
3 years ago
Jessie Murray 5a53c31a27
Logging improvements: add tracing, level check
* Add WEBDIS_TRACE log level for internal operations
* Warn when verbosity config is invalid
* Add slog_enabled to bypass buffer allocations if the level is filtered
3 years ago
Jessie Murray b0740d0c87
Major update of websocket test
1. Switch to evbuffer for correct handling of partial writes
2. Implement WS state machine in each worker
3. Clean up debug logging
4. Add detailed network log messages to help find WS issues
5. Switch to getopt_long
3 years ago
Jessie Murray 0a27fc7e10
More progress on evbuffer-based WS client test 3 years ago
Jessie Murray c46b85ab7a
First attempt at fixing the websocket test 3 years ago
Nicolas Favre-Felix 80139110c5
Add support for debug builds
`make` will produce a release build
`make DEBUG=1` will produce a build with debug symbols
3 years ago
Nicolas Favre-Felix 6163d3e342
Update docker images referenced in the README to the latest version 3 years ago
Nicolas Favre-Felix a0b19bac33
Version bump after release 3 years ago
Nicolas Favre-Felix 649cadcab7
Release 0.1.16
* Only process `Connection: close` header if full request was read
  (#194). This likely fixes the same issue also reported in #145.
* Fix small memory leak when the `type` query string parameter is
  used; the value was not being freed leading to growing memory usage
  of a few bytes per request. Upgrading is recommended if you use this
  feature.
* Fix invalid call to `ioctl`, which did not seem to affect Linux
  systems but could have had an impact on macOS (found in #197).
3 years ago
Nicolas Favre-Felix 950c5153d7
Fix invalid ioctl call
The `request` parameter is unsigned long, not int. This was invalid on
macOS and caused issues when sockets were considered non-blocking. Also
adds an error log if the call fails.

Thanks @likuilin for opening an issue that led to this discovery.
3 years ago
Nicolas Favre-Felix d254a72ae4
Update outdated docker image versions in README 3 years ago
Nicolas Favre-Felix 66ef564b13
Fix memory leak in `?type=...` feature
Passing `?type=foo/bar` in the query string makes Webdis return the
response with a `Content-Type: foo/bar` header (this is useful to serve
files from Webdis, e.g. web page or their dependencies such as CSS,
images, etc). I discovered with Valgrind that the *value* of this query
string parameter was leaked and never freed, which would likely not
cause a huge issue but would still gradually grow the memory usage.

There were 2 different functions taking care of this parameter, the
first calling strdup(3) on it and the second *transferring* pointer
ownership into it (meaning overwriting the just-strdup'd value).

This is now fixed, Webdis no longer leaks this small string, and an
allocation was avoided.
3 years ago