This hasn't been an easy task, but I'm finally very happy that we've
gotten rid of all usage instances of `RawSlice` within `engine::ql`.
This is a great feat in terms of codebase cleanup, removing unnecessary
usage of `unsafe` code.
However, this had made lifetimes slightly more complex; for example, we
can no longer directly return from methods that use `&[Token]` since all
returned data is now bound to the same lifetime.
Either way, with some care such issues can be avoided.
Along with this, here's a summary of other changes:
- `RawSlice` now uses `NonNull` to help the compiler with size opts
- The lexer now allows case insensitive keyword usage (I still DO NOT
particularly approve of it, because enforcing is important)
Summary of changes:
- Remaining query parse methods that directly used the `Compiler` were
modified to use token streams instead
- `Entity::parse_from_tokens` to fix assignment to counter instead of
increments
- Allow entity in `drop model`
- `alter model` and `alter space` directly handle space/model names
instead of depending on passing value via args
- Tests added for `drop` and `inspect`
Summary of changes:
- The `drop` queries now use the `DropItem` for drop definitions
- `create model` is now entirely handled by the function in `schema`
- `create space` is now entirely handled by the function in `schema`
- Tests were added for drop (they were never present before)
Per our language specification, unsafe literals are by their inherent
name literals, hence keep them under this variant. Also, it simplifies
a bunch of implementations. Although I'm sure we can do something for
performance to simply provide an error case when we encounter a
disallowed literal.
* Re-implement network stack
This commit does a complete overhaul to the networking subsystem, and this has
been pending for a long time. For whatever reason, when some generic
flexibility was aimed for, we went ahead with the "expensive emulation" of
async traits which has resulted in unnecessary complication with type arguments
(lifetimes, function signatures looking insane et cetera) besides the usual
downsides.
Also, the termination signaling mechanism was slightly excessive because it was
built with the idea of supporting PUB/SUB by default.
Last but not the least, the authentication system used dynamism which had
resulted in unwanted complication to type arguments for functions that accepted
the auth handle. Now, none of those complications exist and we have an almost
glorious life (for now atleast).
* Fix simple query header for Skyhash-1.0
This ensures that we always have an aligned pointer, even
when the slice is zero sized. This is required for enum
optimizations (as noted in from_raw_parts).
* Switch to using `SharedSlice`
Although I'm a huge fan of case-based optimizations, we don't exactly need the
sophistication that `Bytes` gives us. Instead, we switch to a traditional ref
counted structure, which allocates on creation and deallocates when it is the
only owner (bytes also does this albeit using dynamic dispath via a vft).
The same function path is utilized for every operation on the structure. The
difference between heap-allocating the shared state and having it as fields
on the base structure is a micro-optimization that I'm yet to substantiate
with performance gains.
* Add tests for `SharedSlice`
* Fix erroneous removal of auth table during tree cleanup
When it comes to the handling of system tables, the current storage engine has
a little "funk." It is because the limited flexibility of the format prevents
us from directly storing system tables which have a different structure than
ordinary user created tables; this is why when the cleanup is run, previously,
the dummy keyspace created for the PRELOAD was scanned and since it has nothing
any "alien" file in the `ks/system` directory was purged. This commit fixes
that and ensures that we do not clean up the system directory.
This is also another reason why a new storage engine coupled with a new memory
engine are being developed. These shortcomings will be addressed with the new
engines.
* Add test case for issue #276
Add Changelog entry
I initially planned to implement a scanner-less parser, but however, the
complexity was unnecessary and the performance improvements were marginal.
With increasing language complexity, such a pipeline would become even more
complex and hence, I re-implemented the parser with a traditional "proven"
lexer-parser pipeline. More performance improvements are possible, and I
will add them ahead.
In an earlier commit we marked `as_slice` as safe to call, stating
that only construction is unsafe. However, that is incorrect. The
ctor of `UnsafeSlice` does nothing unsafe, unless we make as_slice
safe. However, since the type is not bounded to any lifetime,
making `as_slice` safe to call assumes a very "rarely true"
safety contract: that the pointers are valid throughout the
execution of the program, id est they are static.
However, that assumption is entirely incorrect for our use case,
hence I'm marking this as `unsafe` again.
For some spurious reason, the tree test is failing on the M1
builder; so we'll hack around by ignoring the test when run
on the M1 CI. Note to self: the test runs successfully on
a physical M1 machine so it is likely that this is a problem
with the runner.
I came across an interesting scenario that led to the discovery of this bug:
if a keyspace is created before a snapshot cycle and before termsig/bgsave,
and then the snapshot cycle runs, then the subsequent runs of bgsave/termsig
would fail. The reason behind this interesting bug turns out to be rather
trivial.
Since the `flush_full` routine, regardless of storage targets would untrip
the PRELOAD switch, the snapshot target as expected untrips the switch. Now,
this means that a tree re-init is not run by BGSAVE or save on termsig.
As a consequence, while attempting to run bgsave/save on termsig, the new
keyspace directories are not found, resulting in flush failure. This commit
fixes it my simply checking if the target should untrip the switch or not.
Tests for the same were added
This commit greatly improves the error reporting infrastructure
all across the `skyd` crate. Now, every error has far more
context making it easier to debug and test while also providing
helpful feedback to users. This feedback can be extremely helpful
to find out offending files and fix them; for example, by being
specific about which file is corrupted or has bad metadata.
The previous method used was a dirty hack due to limitations with
expressions in constant contexts, but the latest `rustc` lets
us greatly simplify this impl.
A sample space of 40 provides enough randomness so that the
likelihood of two tokens being the same is almost entirely
ruled out. And even if they are, it shouldn't be a problem
since they correspond to different accounts, although the
possibility itself is as good as impossible due to the
possible number of permutations being 8.560685103E+94, so
we can safely ignore this "risk".
40 is a sensible balance between what clients have to send
and the level of security we're expecting to provide, than
64 bytes.
Yeah, signal handling is not one of the best things to do, and it
definitely is one of the messiest things to do. This commit
simplifies the way we handle "termination signals" which can be
SIGTERM or SIGINT on POSIX-compliant systems or can be Ctrl+C
or Ctrl+Break on Windows systems; whenever we receive any of these,
we'll attempt to terminate the database server.
Now, instead of waiting for multiple futures to complete, we create
a different `TerminationSignal` type that is a `Future`, which on
polling checks whether either of the signals have been received
or closed. We return the poll state for either.
The `upgrade` subcommand from `skyd` which was removed in 0.7, but was
erroneously accepted in the CLI parameters has been removed. This
was silently ignored.
This might make one think that we are being outrageously strict,
but at the end of the day, it can help investigate crashes
or inspect logs without artwork all over the place.
Following some discussions, the `user` mode was renamed to `dev`
mode.
This commit also upgrades some deps, other than clap which has
deprecated yaml support (we will continue to use 2.x).
Finally, the CHANGELOG was updated.
* Add debian package generation
* Install cargo-deb on `make deb`
* Reload systemd daemon on postinst
* Add auto upload for Debian packages
* Consider using runner.os for simplicity
All deps except for `clap` has been upgraded. Due to the removal of
the `args` field in `ArgMatches` in v3, and our dependence on the
field, we cannot upgrade to the latest version.
A PR has been created and once it is merged or a workaround
suggested, we can upgrade. (see clap-rs/clap#3265)
* Move macros into module
* Add the `whereami` action to identify the current entity
* Show entity group in the skysh prompt
* Add tests and actiondoc for `whereami`
* Add changelog entry
* Upgrade deps
The pipeline impl had a bug which caused a parse error; this happened
because we directly wrote the length as an integer (with the tsymbol)
when we were supposed to only write the integer in its string form
to the stream. This was fixed.
Also, some preliminary tests were added for pipelines.
The `ParsedConfig` struct was renamed to `ConfigurationSet` because it
is more clear in contexts as parsing can be an ambiguous term in several
places.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
43bef62a incorrectly dismissed the check for host/port config in the
case of a non-TLS setup. This commit fixes that.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
The previous configuration handling was rather messed up,
which however is something that this commit attempts to
simplify.
The check for configuration conflict was resolved with a far
more feasible approach and the handling of CLI/config file
configuration was also simplified greatly.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>