This commit adds a basic SSL/TLS listener using `openssl`.
The `SslListener` object can accept a connection and get a decrypted
stream.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
This makes sure that we can implement serialization for multiple types.
At the same time, we DON'T put in blanket implementations for this trait
as we want to make sure that unintended things don't get flushed.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
We've made the error handling slightly 'less aggressive' than the
previous version which `unwrap`ped here and there.
Also, the documentation for the entire snapstore module was greatly
improved.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
This commit provides an implementation which allows multiple 'named'
namespaces to be serialized and deserialized.
No information about the data needs to be known for deserialization;
To facilitate this, a partition map is implemented which is stored as a
separate file. For now, the data file is called `snapstore.bin` and the
partition metadata file is called partmap.
The partition map (`PartMap`) contains a vector of `Partition` objects.
This object stores 'markers' (`len`) which are ideally byte positions
or offsets that demarcate the locations of the individual namespaces.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Previously we were cloning data before writing to disk which
caused slowdowns and lead to higher memory usage. This is an
attempt to fix this behavior.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
If any user needs to run an action without the REPL, they can now do it
by running `tsh -e 'expr'`
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Also the deps were upgraded (there's no point of dependabot creating
multiple commits for upgrading deps)
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
* Use binary for building docker image
In the previous workflow, we were building and testing twice: once for
the docker image and once for the test step. Now, we'll build a debug
version in the test step, then build a release version and finally copy
that into the docker image. This would heavily reduce our build times.
* Ignore specific files in target to speed up builds
* Build image only when pushed to next or tagged
* Fix build condition
Since both conditions have to evaluate to true, we'll need to use refs
while also matching against both `push` and `tag` events
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
We were previously using the ubuntu image and then manually installing
rust. From now on, we'll use the rust docker image instead to avoid huge
build times
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
We previously made a big mistake: we tried to set poisoned to true, an
operation that requires a write lock, without dropping the read lock.
This commit ensures that we first drop `rlock` before doing anything
else. At the same time, if BGSAVE has failed, we'll just shut down the
service.
The error descriptions were improved for a failed `get_saved`
operation. And finally, a couple of variables in `MKSNAP` were renamed
to make them sound more sensible.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
For this, `Writable` was implemented for `&'static [u8]`.
Although this won't have a very noticeable impact on performance, we
will stick to using references instead of cloning the data and then
referencing it again.
Along with this, the docs for MKSNAP were updated.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
There's no point of running tests on tags since we only create tags
after the build for the corresponding commit has succeeded.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Let's say that BGSAVE fails for some reason or the other. As it is
our responsibility to ensure the integrity of the user's data and
to enforce the reliability of the overall database, we should
immediately stop accepting writes, so that new changes aren't made
since the last time flushing data to the disk succeeded.
The functions under kvengine were modified as a consequence, and they
will return an "Internal Server Error" which is response code 5 if
the database is poisoned.
To put it in one line: If running BGSAVE fails, we'll `poison` the
in-memory table and stop accepting write operations, which ideally is
any operation that attempts to obtain a write lock.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Instead of forcing the user to provide values for the number of
connections, queries and the packet size, we'll default to presets
if they aren't supplied.
Also, we've switched the output from stdout to stderr to facilitate
commands like `tdb-bench --json > report.json`
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
This commit enables JSON output for the benchmark tool.
This is an effort to support automated benchmarking, once we do have
such a facility in the future. This closes#37
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
There is no merit in having a redundant `enabled` key under
`snapshot` in the configuration file; if a `snapshot` key is present,
it is inherently obvious that snapshotting is enabled
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Using the configuration file has been a bad idea and this is why we're
switching to using command line args.
Also links and docs were updated
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Instead of checking to see if the remote snapshots directory exists
whenever MKSNAP is called, we'll create the directory when the server
starts up
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Since creating snapshots is quite an important utility,
there may be scenarios where creating one may be needed,
even if it is disabled on the server side. This commit
enables such snapshots to be created. This is achieved by
enabling MKSNAP to accept two arguments, where a 'named'
snapshot can be created, which is our "special" snapshot.
All these "special" snapshots are stored in a separate
"snapshots/remote" dir that is ignored by the
`SnapshotEngine`.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
From now on, tdb-bench has to be run with
`tdb-bench -c <c> -q <q> -s <s>` instead of just using positional args.
Also, we'll keep `tdb-bench`'s version in
sync with the version of `tdb`, just like the others in the project
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
This commit enables processing for flags and
options for `tsh`. This is a breaking change as it prevents
users from using the previous launch arguments and
instead requires them to use a `-h` and `-p` combination
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Following on from ab9561258e, we'll allow
the user to configure snapshots and BGSAVE via command-line arguments
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Until now, the database server could only be configured via the
configuration file. This commit enables the host, port and noart
options to be configured via command-line arguments.
This is important as there may be scenarios where creating a file
presents a challenge to the user.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
The user can now run `tdb -r <snapshotname>` to restore data from the
snapshot. Also, we'll show a note in the logs when trying to restore from
a snapshot
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
This module isn't doing anything good staying here, as it does nothing!
More work needs to be done on the file storage format internally before
a public rollout.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Since the ports in the range [49670,50293] along with ports {5357,7680}
are blocked by Hyper-V on Windows VMs, we'll exclude them.
Previously, several tests on Windows have failed as a result of
attempting to connect to these reserved ports.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
Upstream changes in tokio have required several changes.
For example, `delay_until` was renamed to `sleep_until`. Similarly,
`notify` was renamed to `notify_one`.
Also, in tdb-macros, the `runtime::Builder::new()` line was changed into
`runtime::Builder::new_multi_thread()` due to changes upstream
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
If the duration for a periodic operation is set to zero in the
config file then it is likely that the thread would crash.
We want to prevent this at all costs, which is why
we're adding this check.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
We don't need tests for MKSNAP when it is enabled as we already have
tests for snapshotting in `diskstore::snapshot`
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
To streamline the actions we'll make sure that mksnap returns code 3
if an incorrect number of arguments are provided
The actiondoc for the `mksnap` action was added and the contributing
guide was also updated
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
In `cli` other errors are now formatted in a `[ERR]` format
Also the documentation across the project was updated
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
`CoreDB` now has a `SnapshotStatus` object for this purpose
This object holds the current state of the service in an `RWLock`
When `mksnap` is called, `SnapEngine` sets `in_progress` to true
When the snapshot is created, `SnapEngine` sets `in_progress` to false
This prevents multiple entities from creating snapshots at the same time
Sayan
dependabot[bot]