* Fix erroneous removal of auth table during tree cleanup
When it comes to the handling of system tables, the current storage engine has
a little "funk." It is because the limited flexibility of the format prevents
us from directly storing system tables which have a different structure than
ordinary user created tables; this is why when the cleanup is run, previously,
the dummy keyspace created for the PRELOAD was scanned and since it has nothing
any "alien" file in the `ks/system` directory was purged. This commit fixes
that and ensures that we do not clean up the system directory.
This is also another reason why a new storage engine coupled with a new memory
engine are being developed. These shortcomings will be addressed with the new
engines.
* Add test case for issue #276
Add Changelog entry
This commit re-implements the benchmark tool, doing a lot of refactoring and
speeding up things, while reducing memory usage. Some notes:
1. The `testkey` subcommand was removed. This was needed because it makes no
sense to have it with data types like lists. I'll leave support for creating
test data for another PR (if at all needed)
2. The bench tool now uses lesser memory "in a go." Previously, all the get,
set and update packets were generated in one single step. This is no longer
done. Instead, we generate the packets for one type, run its corresponding
tests and completely deallocate and cleanup once we're done with that test.
This helps alleviate memory usage.
3. Instead of going in the sequence (GET, SET, UPDATE, ...) and running that
set some n times, we now run each test n times, and repeat that for the other
tests. The advantage of this is code clarity.
4. Finally, more validation checks were added ensuring that the benchmark tool
very closely resembles a "real-life" scenario wherein a connection is picked up
from a pool by a thread, a task run-on and evaluated.
5. Better logging and error reporting: instead of vaguely crashing, the
benchmark tool provides more clarity on errors. However, panics in worker
threads are yet to be improved in terms of reporting.
I came across an interesting scenario that led to the discovery of this bug:
if a keyspace is created before a snapshot cycle and before termsig/bgsave,
and then the snapshot cycle runs, then the subsequent runs of bgsave/termsig
would fail. The reason behind this interesting bug turns out to be rather
trivial.
Since the `flush_full` routine, regardless of storage targets would untrip
the PRELOAD switch, the snapshot target as expected untrips the switch. Now,
this means that a tree re-init is not run by BGSAVE or save on termsig.
As a consequence, while attempting to run bgsave/save on termsig, the new
keyspace directories are not found, resulting in flush failure. This commit
fixes it my simply checking if the target should untrip the switch or not.
Tests for the same were added
This commit greatly improves the error reporting infrastructure
all across the `skyd` crate. Now, every error has far more
context making it easier to debug and test while also providing
helpful feedback to users. This feedback can be extremely helpful
to find out offending files and fix them; for example, by being
specific about which file is corrupted or has bad metadata.
The `upgrade` subcommand from `skyd` which was removed in 0.7, but was
erroneously accepted in the CLI parameters has been removed. This
was silently ignored.
This might make one think that we are being outrageously strict,
but at the end of the day, it can help investigate crashes
or inspect logs without artwork all over the place.
Following some discussions, the `user` mode was renamed to `dev`
mode.
This commit also upgrades some deps, other than clap which has
deprecated yaml support (we will continue to use 2.x).
Finally, the CHANGELOG was updated.
It's important that we report an error when the sample space is too
low for the given k/v size and key count. Previously, we'd end up
entering an infinite wait time (essentially an infinite loop); this
has been fixed.
* Move macros into module
* Add the `whereami` action to identify the current entity
* Show entity group in the skysh prompt
* Add tests and actiondoc for `whereami`
* Add changelog entry
* Upgrade deps
Sayan