* Re-implement network stack
This commit does a complete overhaul to the networking subsystem, and this has
been pending for a long time. For whatever reason, when some generic
flexibility was aimed for, we went ahead with the "expensive emulation" of
async traits which has resulted in unnecessary complication with type arguments
(lifetimes, function signatures looking insane et cetera) besides the usual
downsides.
Also, the termination signaling mechanism was slightly excessive because it was
built with the idea of supporting PUB/SUB by default.
Last but not the least, the authentication system used dynamism which had
resulted in unwanted complication to type arguments for functions that accepted
the auth handle. Now, none of those complications exist and we have an almost
glorious life (for now atleast).
* Fix simple query header for Skyhash-1.0
This ensures that we always have an aligned pointer, even
when the slice is zero sized. This is required for enum
optimizations (as noted in from_raw_parts).
* Switch to using `SharedSlice`
Although I'm a huge fan of case-based optimizations, we don't exactly need the
sophistication that `Bytes` gives us. Instead, we switch to a traditional ref
counted structure, which allocates on creation and deallocates when it is the
only owner (bytes also does this albeit using dynamic dispath via a vft).
The same function path is utilized for every operation on the structure. The
difference between heap-allocating the shared state and having it as fields
on the base structure is a micro-optimization that I'm yet to substantiate
with performance gains.
* Add tests for `SharedSlice`
* Fix erroneous removal of auth table during tree cleanup
When it comes to the handling of system tables, the current storage engine has
a little "funk." It is because the limited flexibility of the format prevents
us from directly storing system tables which have a different structure than
ordinary user created tables; this is why when the cleanup is run, previously,
the dummy keyspace created for the PRELOAD was scanned and since it has nothing
any "alien" file in the `ks/system` directory was purged. This commit fixes
that and ensures that we do not clean up the system directory.
This is also another reason why a new storage engine coupled with a new memory
engine are being developed. These shortcomings will be addressed with the new
engines.
* Add test case for issue #276
Add Changelog entry
I initially planned to implement a scanner-less parser, but however, the
complexity was unnecessary and the performance improvements were marginal.
With increasing language complexity, such a pipeline would become even more
complex and hence, I re-implemented the parser with a traditional "proven"
lexer-parser pipeline. More performance improvements are possible, and I
will add them ahead.
In an earlier commit we marked `as_slice` as safe to call, stating
that only construction is unsafe. However, that is incorrect. The
ctor of `UnsafeSlice` does nothing unsafe, unless we make as_slice
safe. However, since the type is not bounded to any lifetime,
making `as_slice` safe to call assumes a very "rarely true"
safety contract: that the pointers are valid throughout the
execution of the program, id est they are static.
However, that assumption is entirely incorrect for our use case,
hence I'm marking this as `unsafe` again.
For some spurious reason, the tree test is failing on the M1
builder; so we'll hack around by ignoring the test when run
on the M1 CI. Note to self: the test runs successfully on
a physical M1 machine so it is likely that this is a problem
with the runner.
I came across an interesting scenario that led to the discovery of this bug:
if a keyspace is created before a snapshot cycle and before termsig/bgsave,
and then the snapshot cycle runs, then the subsequent runs of bgsave/termsig
would fail. The reason behind this interesting bug turns out to be rather
trivial.
Since the `flush_full` routine, regardless of storage targets would untrip
the PRELOAD switch, the snapshot target as expected untrips the switch. Now,
this means that a tree re-init is not run by BGSAVE or save on termsig.
As a consequence, while attempting to run bgsave/save on termsig, the new
keyspace directories are not found, resulting in flush failure. This commit
fixes it my simply checking if the target should untrip the switch or not.
Tests for the same were added
This commit greatly improves the error reporting infrastructure
all across the `skyd` crate. Now, every error has far more
context making it easier to debug and test while also providing
helpful feedback to users. This feedback can be extremely helpful
to find out offending files and fix them; for example, by being
specific about which file is corrupted or has bad metadata.
Sayan Nandan