473 Commits (39ab1e7683742022444e2926cd3df47327c6eb50)
 

Author SHA1 Message Date
Sayan Nandan f9932a5b97
Provide default params for queries, cons and size
Instead of forcing the user to provide values for the number of
connections, queries and the packet size, we'll default to presets
if they aren't supplied.

Also, we've switched the output from stdout to stderr to facilitate
commands like `tdb-bench --json > report.json`

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan cbebda85b0
Enable JSON output for `tdb-bench`
This commit enables JSON output for the benchmark tool.
This is an effort to support automated benchmarking, once we do have
such a facility in the future. This closes #37

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 824944e30e
Don't use an explicit `enabled` flag in `snapshot`
There is no merit in having a redundant `enabled` key under
`snapshot` in the configuration file; if a `snapshot` key is present,
it is inherently obvious that snapshotting is enabled

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan d23c4ee947
Fix issues with the Dockerfile
Using the configuration file has been a bad idea and this is why we're
switching to using command line args.

Also links and docs were updated

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan b09beefe60
Don't check remote dir every time MKSNAP is called
Instead of checking to see if the remote snapshots directory exists
whenever MKSNAP is called, we'll create the directory when the server
starts up

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan a0fea2ee08
Enable remote snapshots even if disabled
Since creating snapshots is quite an important utility,
there may be scenarios where creating one may be needed,
even if it is disabled on the server side. This commit
enables such snapshots to be created. This is achieved by
enabling MKSNAP to accept two arguments, where a 'named'
snapshot can be created, which is our "special" snapshot.

All these "special" snapshots are stored in a separate
"snapshots/remote" dir that is ignored by the
`SnapshotEngine`.

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 7232fe9189
Enable tdb-bench to read args and opts from shell
From now on, tdb-bench has to be run with
`tdb-bench -c <c> -q <q> -s <s>` instead of just using positional args.
Also, we'll keep `tdb-bench`'s version in
sync with the version of `tdb`, just like the others in the project

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 477acfab4b
Allow tsh to process flags and opts from the shell
This commit enables processing for flags and
options for `tsh`. This is a breaking change as it prevents
users from using the previous launch arguments and
instead requires them to use a `-h` and `-p` combination

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 951880817b
Let snapshots and bgsave opts be set via cmd args
Following on from ab9561258e, we'll allow
the user to configure snapshots and BGSAVE via command-line arguments

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan ab9561258e
Enable command line configuration
Until now, the database server could only be configured via the
configuration file. This commit enables the host, port and noart
options to be configured via command-line arguments.

This is important as there may be scenarios where creating a file
presents a challenge to the user.

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 5c1fbae609
Fix test for the snapshot config file
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan cf3d213b3f
Enable data to be restored from a snapshot
The user can now run `tdb -r <snapshotname>` to restore data from the
snapshot. Also, we'll show a note in the logs when trying to restore from
a snapshot

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan f06f81cd5c
Remove the `CyanSFW` module
This module isn't doing anything good staying here, as it does nothing!
More work needs to be done on the file storage format internally before
a public rollout.

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan ac37769b7a
Rollout v0.4.5
Also, the actiondoc for `MKSNAP` was updated and a changelog entry was
added for this release

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan f142a41ed7
Fix port issues with the `dbtest` macro on Windows
Since the ports in the range [49670,50293] along with ports {5357,7680}
are blocked by Hyper-V on Windows VMs, we'll exclude them.
Previously, several tests on Windows have failed as a result of
attempting to connect to these reserved ports.

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 93a7ea79cc
Improve docs in `tdb-bench`
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan bc1a6fd456
Upgrade tokio to 0.3.2
Upstream changes in tokio have required several changes.
For example, `delay_until` was renamed to `sleep_until`. Similarly,
`notify` was renamed to `notify_one`.
Also, in tdb-macros, the `runtime::Builder::new()` line was changed into
`runtime::Builder::new_multi_thread()` due to changes upstream

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 71e020db59
Ensure that duration values are not zero
If the duration for a periodic operation is set to zero in the
config file then it is likely that the thread would crash.
We want to prevent this at all costs, which is why
we're adding this check.

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 0ea02f846a
Add tests for MKSNAP and bump up version
We don't need tests for MKSNAP when it is enabled as we already have
tests for snapshotting in `diskstore::snapshot`

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 11a9223e6b
Make MKSNAP return code 3 for `howmany != 0`
To streamline the actions we'll make sure that mksnap returns code 3
if an incorrect number of arguments are provided
The actiondoc for the `mksnap` action was added and the contributing
guide was also updated

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan a3c5135022
Provide `asyncdb` for `dbtest` macro functions
In `cli` other errors are now formatted in a `[ERR]` format
Also the documentation across the project was updated

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan f8fea34e18
Fix tests for snapshots not setting `SnapshotStatus`
Also upgrade syn from 1.0.46 to 1.0.48 (#31)
4 years ago
Sayan Nandan 4ce95c6d42
Make sure that only one snapshot is made at a time
`CoreDB` now has a `SnapshotStatus` object for this purpose
This object holds the current state of the service in an `RWLock`
When `mksnap` is called, `SnapEngine` sets `in_progress` to true
When the snapshot is created, `SnapEngine` sets `in_progress` to false
This prevents multiple entities from creating snapshots at the same time
4 years ago
Sayan Nandan 037064e8d9
Improve `Writable` impl for `terrapipe::RespCodes` 4 years ago
Sayan Nandan 6e27bd256f
Exit block if `SnapEngine` failed to initialize 4 years ago
Sayan Nandan 3b3d2c6994
Implement `mksnap` action 4 years ago
Sayan Nandan c31788754c
Enable module-level imports in `dbtest` proc macro
Also, the macro now unpacks the `__private` module into `super`
4 years ago
Sayan Nandan 980d47a3be
Improve `dbtest` macro docs 4 years ago
Sayan Nandan 5d499fa36d
Remove `testsuite` crate and decompose a few tests 4 years ago
Sayan Nandan 425afbe8c7
Fix lower bound for ports in dbtest compiler macro
Redundant test objects were also removed
4 years ago
Sayan Nandan 3d1dd04b0d
Improved testing to use custom compiler macros
Also, multi-threaded tests were re-enabled
4 years ago
Sayan Nandan 64c14f70c0
Improve compiler macro `dbtest`
Excluded some default ports to avoid permission errors
Also enabled skipping of functions in dbtest modules
Fixed runtime creation statement in `tdb-macros`
4 years ago
dependabot[bot] 270dea251f
Bump syn from 1.0.45 to 1.0.46 (#30)
Bumps syn from 1.0.45 to 1.0.46

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
4 years ago
Sayan 8270a5543b
Don't build docker image on PR
There's no point of building docker images when a PR is created, since we rely on the compiler and not docker
4 years ago
Sayan 9b757de028
Don't checkout `next` while setting variables 4 years ago
Sayan c4f00de5b5
Create Dependabot config file (#27)
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
4 years ago
Sayan Nandan 2d04a4f841
Enable build skips on PRs
Iff no source files are changed - skip the compute-intensive build
4 years ago
Sayan Nandan 47831252b3
Fix the `dbtest` macro using a `#[test]` attribute
Also several misc. fixes were made to the `dbtest` compiler macro
The macro now creates a TcpListener and closes the socket once finished
4 years ago
Sayan 44cb12955d
Fix `tokio` version 4 years ago
Sayan Nandan 147f13a020
Rename `tdb-derive` to `tdb-macros`
The proc_macro can now be applied on modules only
All functions within the module will be considered to be a test
This has the advantage of not having to flag every test function
4 years ago
Sayan 16304cc4f7 Use bash as the default shell in `test-push` 4 years ago
Sayan b55894bf7f
Improve docs 4 years ago
Sayan Nandan 07f97c65b3
Optimize CI script 4 years ago
Sayan Nandan 986c5e9784
Don't build docker image if src hasn't changed 4 years ago
Sayan Nandan 2a9fa9ffcf Fix script to skip builds iff md files are changed
In previous commits, we ignored the fact that bools are strings in bash
This caused the workflow to proceed wrongly and this commit fixes that
Also, auto builds for actions docs were added
4 years ago
Sayan Nandan 336115e493
Fix builds on macOS 4 years ago
Sayan Nandan 0468d0d921
Fix environment variables 4 years ago
Sayan d3e37cfa55
Fix CI script 4 years ago
Sayan Nandan 8a8c4450bf
Add outline for `mksnap` action 4 years ago
Sayan Nandan 973a8971c0
Improve docs 4 years ago