Sayan Nandan
e34e5c87ca
Rename `Scanner` to `QueryProcessor`
2 years ago
Sayan Nandan
08a286d543
Ensure whitespace is skipped
2 years ago
Sayan Nandan
2434170ede
Add support for nested type expressions
2 years ago
Sayan Nandan
53fcecbb62
Add support for types
2 years ago
Sayan Nandan
bc3694dd0c
Add punctuation support
2 years ago
Sayan Nandan
979c427d3a
Add support for escaped quoted strings
2 years ago
Sayan Nandan
b14ca8e97c
Add string literal parsing
2 years ago
Sayan Nandan
42d59ee847
Account for numeric lit overflow
2 years ago
Sayan Nandan
bea99c1e25
Add numeric literal parsing
2 years ago
Sayan Nandan
f9c7a09f43
Add minimal lexing
2 years ago
Sayan Nandan
26645c93ab
Add token definitions
2 years ago
Sayan Nandan
3927940afd
Add `Scanner`
2 years ago
Sayan Nandan
a819196dbd
Simplify COW semantics
2 years ago
Sayan Nandan
06764d3462
Make `UnsafeSlice::as_slice` unsafe to call
...
In an earlier commit we marked `as_slice` as safe to call, stating
that only construction is unsafe. However, that is incorrect. The
ctor of `UnsafeSlice` does nothing unsafe, unless we make as_slice
safe. However, since the type is not bounded to any lifetime,
making `as_slice` safe to call assumes a very "rarely true"
safety contract: that the pointers are valid throughout the
execution of the program, id est they are static.
However, that assumption is entirely incorrect for our use case,
hence I'm marking this as `unsafe` again.
2 years ago
Sayan Nandan
e495172f2f
Upgrade deps
2 years ago
Sayan Nandan
b3b236d9aa
Guard against allocation errors in `libstress`
2 years ago
Sayan Nandan
36b3fc0ee8
Add docs for binary crates and upgrade deps
2 years ago
Sayan Nandan
669eca3ff7
Increase shard count and remove redundant impls
2 years ago
Sayan Nandan
d4c10bee80
Upgrade deps and bump version
2 years ago
Sayan Nandan
4a0cc3579e
Upgrade deps
2 years ago
Sayan Nandan
d4dcd7b127
Upgrade deps and ignore tree test on M1 builder
...
For some spurious reason, the tree test is failing on the M1
builder; so we'll hack around by ignoring the test when run
on the M1 CI. Note to self: the test runs successfully on
a physical M1 machine so it is likely that this is a problem
with the runner.
2 years ago
Sayan Nandan
31bdc83108
Make auth errors generic over protocol
2 years ago
Sayan Nandan
89067c1fd5
Revise trait definitions
2 years ago
Sayan Nandan
231dd53341
Reduce code redundancy by using `RawParser` and `RawParserExt`
...
Also added changelog
2 years ago
Sayan Nandan
20f039cb85
Enable setting protocol version in configuration
2 years ago
Sayan Nandan
b5e0f68c88
Add support for Skyhash 1.0
2 years ago
Sayan Nandan
7ec599edcb
Update bench suite for protocol
...
Turns out that the original PR implementing Skyhash 2.0 did not
update the benchmark code.
2 years ago
Sayan Nandan
67b19602b9
Make all responses/groups generic over protocol
2 years ago
Sayan Nandan
b047845cc5
Upgrade all interfaces to be generic over protocol
2 years ago
Sayan Nandan
9c15e100c8
Add metaframe methods to `ProtocolWrite`
2 years ago
Sayan Nandan
2bb7555e4e
Add `ProtocolWrite` trait for writing data according to `ProtocolSpec`
2 years ago
Sayan Nandan
d31fc5855d
Add memory safety assertions
2 years ago
Sayan Nandan
02a3e9b4e9
Make connections generic over protocols
2 years ago
Sayan Nandan
b2e130626f
Simplify types
2 years ago
Sayan Nandan
0e432e8761
Upgrade deps
2 years ago
Sayan Nandan
3b97c9528f
Remove redundant methods
2 years ago
Sayan Nandan
879e20f6ca
Upgrade all interfaces to use the new protocol
2 years ago
Sayan Nandan
a018b76f40
Ensure empty payload is allowed
...
Bump Skytable driver git revision to ensure builds don't fail
(possibly a force push).
2 years ago
Sayan Nandan
ec24b2caad
Fix segfault due to uninitialized elements in heap array
2 years ago
Sayan Nandan
3b918ff828
Add more tests
2 years ago
Sayan Nandan
71dcfb7efc
Fix server not exiting due to infinite `read_buf` loop
2 years ago
Sayan Nandan
c065ed61e3
Add complete query parsing impl
2 years ago
Sayan Nandan
c2ddb9b2db
Add more `read_usize` tests
2 years ago
Sayan Nandan
dec82cd400
Add `read_line_pedantic` for non-empty lines
2 years ago
Sayan Nandan
3731143831
Add tests with excess bytes
2 years ago
Sayan Nandan
7f3d3cb05e
Add `usize` parsing
2 years ago
Sayan Nandan
59c681e787
Fix `read_line` impl and tests
2 years ago
Sayan Nandan
55a01b172c
Add tests for `read_until`
2 years ago
Sayan Nandan
400ca13d10
Add tests for `not_exhausted`
2 years ago
Sayan Nandan
a17d77bf9e
Fix tests
2 years ago
Sayan Nandan
2c2a236501
Add parserv2 basic impl
2 years ago
Sayan Nandan
7ab9bbe8e0
Upgrade deps
2 years ago
Sayan Nandan
c984bd2e1b
Ensure Windows entity names are sanitized
3 years ago
Sayan Nandan
df994cf4f7
Add `sys metric storage` for checking on-disk storage usage
3 years ago
Sayan Nandan
d201c74150
Remove PID file just before exit
...
Also upgrade GHA caching dep and fix imports that rustfmt
messed up.
3 years ago
Sayan Nandan
7a354a9acb
Improve server shutdown heuristic and fix Windows path evaluation
3 years ago
Sayan Nandan
72304195d1
Use proc macro for correct compile-time response generation
3 years ago
Sayan Nandan
2c6768dc12
Ensure snapshot overwrite protection if host time is incorrect
3 years ago
Sayan Nandan
0e10f2cb5f
Fix remote snapshots with same names being overwritten
3 years ago
Sayan Nandan
076282df2b
Add tests for snapshot directory structure
3 years ago
Sayan Nandan
e5909eb1f9
Use separate flag for tree cleanup
3 years ago
Sayan Nandan
9374404551
Add persist tests for keyspace creation/deletion
3 years ago
Sayan Nandan
08cffa1d3f
Fix bug causing bgsave/termsig flush failure if a snapshot flushed it
...
I came across an interesting scenario that led to the discovery of this bug:
if a keyspace is created before a snapshot cycle and before termsig/bgsave,
and then the snapshot cycle runs, then the subsequent runs of bgsave/termsig
would fail. The reason behind this interesting bug turns out to be rather
trivial.
Since the `flush_full` routine, regardless of storage targets would untrip
the PRELOAD switch, the snapshot target as expected untrips the switch. Now,
this means that a tree re-init is not run by BGSAVE or save on termsig.
As a consequence, while attempting to run bgsave/save on termsig, the new
keyspace directories are not found, resulting in flush failure. This commit
fixes it my simply checking if the target should untrip the switch or not.
Tests for the same were added
3 years ago
Sayan Nandan
c4eaf3d182
Bump version [skip ci]
3 years ago
Sayan Nandan
5002fed675
Fix pipeline with pedantic execution for stages
3 years ago
Sayan Nandan
5054adc33c
Add sys action tests
3 years ago
Sayan Nandan
832da79e9d
Add sys actions
3 years ago
Sayan Nandan
0283d91f67
Fix panic resulting from corrupted metadata in `PARTMAP`
3 years ago
Sayan Nandan
0b17f7b0e4
Fix new instance detection
3 years ago
Sayan Nandan
0e7340c965
Fix LF check in protocol impl
3 years ago
Sayan Nandan
d57b0d98cf
Improve error reporting infrastructure
...
This commit greatly improves the error reporting infrastructure
all across the `skyd` crate. Now, every error has far more
context making it easier to debug and test while also providing
helpful feedback to users. This feedback can be extremely helpful
to find out offending files and fix them; for example, by being
specific about which file is corrupted or has bad metadata.
3 years ago
Sayan Nandan
f9b997c1f1
Avoid verbose logging in test harness
3 years ago
Sayan Nandan
cc6a1b7f27
Enable pushing multiple elements into list at once
3 years ago
Sayan Nandan
14d9eb4bac
Clean up code
...
Fixed Windows tests and also applied recent clippy suggestions
3 years ago
Sayan Nandan
cd1accd646
Simplify command assembly
3 years ago
Sayan Nandan
6234018077
Wait for servers to start/shut down instead of waiting blindly
3 years ago
Sayan Nandan
f40c1d00e5
Add `auth whoami`
3 years ago
Sayan Nandan
82420e2013
Enable inspecting the current table
3 years ago
Sayan Nandan
ced5e75182
Enable inspecting current keyspace
3 years ago
Sayan Nandan
b099dd1825
Add missing respcode output impls
3 years ago
Sayan Nandan
b057de94e6
Bump version to 0.7.4
3 years ago
Sayan Nandan
90ef10a56d
Fix snapshot engine init failure
3 years ago
Sayan Nandan
c80d929cf9
Ensure `AuthID`s are ASCII-only and add actiondoc for `listuser`
3 years ago
Sayan Nandan
ad2f87de7a
Add `auth listuser` subaction
3 years ago
Sayan Nandan
37ba222428
Upgrade deps
3 years ago
Sayan Nandan
c7ae4b69e5
Add auth persistence tests
3 years ago
Sayan Nandan
f446feaac6
Add persistence tests for lists
3 years ago
Sayan Nandan
1dbba0c06e
Add persistence suite for keymap string table types
...
Also fixed persistence bug with tables in the `system` keyspace
3 years ago
Sayan Nandan
14966f6972
Fix missing `system` ks on second boot of server
3 years ago
Sayan Nandan
369f7b89ed
Enable table access using partial entity syntax
3 years ago
Sayan Nandan
d30bc38fbe
Enable `auth restore` for anonymous users and add tests
3 years ago
Sayan Nandan
482f4009db
Enable regenerating tokens and fix panic on wrong length `AuthID`
3 years ago
Sayan Nandan
7dedb37a12
Fix auth error respstring lengths
3 years ago
Sayan Nandan
97112c4b51
Add actiondoc for auth
3 years ago
Sayan Nandan
5681e5ec53
Simplify `KVEngine` impl
3 years ago
Sayan Nandan
3cbe7c981c
Ensure TLS-only mode in prod when authn is enabled
...
Also updated configuration template
3 years ago
Sayan Nandan
426948f8d5
Upgrade deps
3 years ago
Sayan Nandan
88955b62c6
Add changelog and remove unused code
3 years ago
Sayan Nandan
f7a4281ed5
Add `auth` syntax checks
3 years ago
Sayan Nandan
58b3f0dda7
Add authn tests
3 years ago
Sayan Nandan
3c14501913
Simplify tests
3 years ago
Sayan Nandan
f67057dfb5
Add authn tests
3 years ago
Sayan Nandan
ade5b2a105
Enable authentication in dbtest tests
3 years ago
Sayan Nandan
4a075422de
Enable using TLS on dbtest tests
3 years ago
Sayan Nandan
e62fb74e81
Simplify dbtest macros
3 years ago
Sayan Nandan
bb978e5a84
Simplify uninit array generation
...
The previous method used was a dirty hack due to limitations with
expressions in constant contexts, but the latest `rustc` lets
us greatly simplify this impl.
3 years ago
Sayan Nandan
68e5e07450
Fix length checks in auth subactions
3 years ago
Sayan Nandan
ff8bb08ab9
Fix `auth login` and `auth claim` impls for unauth connections
3 years ago
Sayan Nandan
6db1667420
Enable auth to be configured
3 years ago
Sayan Nandan
d70a42fd10
Fix tests
3 years ago
Sayan Nandan
d6fed47eae
Implement `login`, `logout`, `claim` and `adduser` for `auth`
3 years ago
Sayan Nandan
473f435638
Enable users to authenticate
3 years ago
Sayan Nandan
2a91eab273
Add `ActionError` impls for `AuthError`
3 years ago
Sayan Nandan
720764f5d2
Move system tables into separate namespace
3 years ago
Sayan Nandan
21d0e0a2b6
Version gate the storage engine
3 years ago
Sayan Nandan
ffce889c00
Improve bytemark save/restore tests
3 years ago
Sayan Nandan
339e43928f
Add table restoration tests using bytemark
3 years ago
Sayan Nandan
2cf4ec5e87
Fix unflush for volatile tables
3 years ago
Sayan Nandan
380ba67cc6
Make unflush routines generic over keyspaces and tables
3 years ago
Sayan Nandan
8a1bf4b400
Optimize table flush/unflush routines
3 years ago
Sayan Nandan
6832c92775
Make flushing of keyspaces and tables generic over types
3 years ago
Sayan Nandan
fa74a149b4
Fix tests
3 years ago
Sayan Nandan
8d7126994d
Fix `LocalSnapshot` and `RemoteSnapshot` implementations
3 years ago
Sayan Nandan
39726177cd
Simplify flush routines using `StorageTarget`
3 years ago
Sayan Nandan
652ae8d2f0
Reduce duplication in strong action test impls
3 years ago
Sayan Nandan
d5e53eede0
Reduce duplication in `DdlError` handling
3 years ago
Sayan Nandan
750d054144
Don't unnecessarily obtain arc clones
3 years ago
Sayan Nandan
5d9ef41c31
Fix length checks
...
Also improve panic messages in `dbtest` tests.
3 years ago
Sayan Nandan
4c46ff3c2b
Simplify action error propagation
3 years ago
Sayan Nandan
b59243eec1
Fix try_from_slice impl
3 years ago
Sayan Nandan
31d4cbb34d
Enable login and add login tests
3 years ago
Sayan Nandan
2fc77e0393
Use smaller size for random bytes
...
A sample space of 40 provides enough randomness so that the
likelihood of two tokens being the same is almost entirely
ruled out. And even if they are, it shouldn't be a problem
since they correspond to different accounts, although the
possibility itself is as good as impossible due to the
possible number of permutations being 8.560685103E+94, so
we can safely ignore this "risk".
40 is a sensible balance between what clients have to send
and the level of security we're expecting to provide, than
64 bytes.
3 years ago
Sayan Nandan
c8ccfca09b
Implement `AuthProvider`
3 years ago
Sayan Nandan
3a6083b3f1
Add `AuthProvider` definition
3 years ago
Sayan Nandan
7285d63908
Simplify `Once` implementation
3 years ago
Sayan Nandan
6ed994fe19
Add `Once` implementation
3 years ago
Sayan Nandan
53d4c2b529
Use `TcpBackoff` in `SslListener` as well
3 years ago
Sayan Nandan
d2e297e367
Use dedicated backoff algorithm for busy loops
...
Also use `util::exit_error` whenever possible to reduce
duplication.
3 years ago
Sayan Nandan
57f05e9788
Simplify TCP backoff impl
3 years ago
Sayan Nandan
96f817d31e
Simplify `resp` impls
...
The `RespCode` impl is no longer used, so it was removed.
3 years ago
Sayan
e9620d0a8c
Release 0.7.3 ( #238 )
...
* Bump version
* Upgrade deps
3 years ago
Sayan Nandan
49fae295d1
Simplify termination signal handling
...
Yeah, signal handling is not one of the best things to do, and it
definitely is one of the messiest things to do. This commit
simplifies the way we handle "termination signals" which can be
SIGTERM or SIGINT on POSIX-compliant systems or can be Ctrl+C
or Ctrl+Break on Windows systems; whenever we receive any of these,
we'll attempt to terminate the database server.
Now, instead of waiting for multiple futures to complete, we create
a different `TerminationSignal` type that is a `Future`, which on
polling checks whether either of the signals have been received
or closed. We return the poll state for either.
3 years ago
Sayan Nandan
ed66e4f291
Decrement threshold on first call
3 years ago
Sayan Nandan
0b87452b97
Listen to SIGTERM while in save-on-termsig retry loop
3 years ago
Sayan Nandan
2d6d776edc
Cleanup code and ensure strong_count of `Corestore` is 1
...
I suppressed some lints, but all of them will be removed very
soon. (use a TODO finder or something to see what all need to
be removed)
3 years ago
Sayan Nandan
33b0693fcb
Fix infinite loop on save failure during termination
3 years ago
Sayan Nandan
9b296cbf9b
Bump version
3 years ago
Sayan Nandan
c961548b01
Upgrade deps and bump version
3 years ago
Sayan Nandan
e9369a40ef
Simplify recipes
3 years ago
Sayan Nandan
0b54ea1cb3
Bump up version
3 years ago
Sayan Nandan
b52f0d42ca
Fix tests
...
Also remove certs that were incorrectly added
3 years ago
Sayan Nandan
2340a0239e
Add tests for list's range
3 years ago
Sayan Nandan
4186eda627
Add range to lists
3 years ago
Sayan Nandan
84134ed240
Remove `upgrade` subcommand and log on restore
...
The `upgrade` subcommand from `skyd` which was removed in 0.7, but was
erroneously accepted in the CLI parameters has been removed. This
was silently ignored.
3 years ago
Sayan Nandan
36b02e9099
Fix backup restoration and simplify `arbiter::run`
3 years ago
Sayan Nandan
d550e0d7a7
Expect `noart` in production mode
...
This might make one think that we are being outrageously strict,
but at the end of the day, it can help investigate crashes
or inspect logs without artwork all over the place.
Following some discussions, the `user` mode was renamed to `dev`
mode.
This commit also upgrades some deps, other than clap which has
deprecated yaml support (we will continue to use 2.x).
Finally, the CHANGELOG was updated.
3 years ago
Sayan Nandan
46c048f855
Fix rlimit check on Windows
...
Windows (obviously) doesn't have libc's rlimit so simply avoid
any references to it
3 years ago
Sayan Nandan
d121fa96fb
Enable config evaluation for prod mode
3 years ago
Sayan Nandan
0d2a143e12
Fix warningstack_fmt test
3 years ago
Sayan Nandan
d2d96e745e
Improve maxcon diagnostic
3 years ago
Sayan Nandan
481509d927
Fix rlimit impl for 32-bit
3 years ago
Sayan Nandan
6b54217fb2
Add production-mode setting evaluation
3 years ago
Sayan Nandan
4325cf9065
Add method to check number of open files on unix-based systems
3 years ago
Sayan Nandan
857e05529a
Add CLI config tests and improve diagnostic tests
3 years ago
Sayan Nandan
59a67ba0c5
Add cfg file tests
3 years ago
Sayan Nandan
53218ee98a
Add tests for types that implement `TryFromConfigSource`
3 years ago
Sayan Nandan
647d6ce05c
Add tests for config impl
3 years ago
Sayan Nandan
ce7d7bef25
Fix config change checks and error messages
3 years ago
Sayan Nandan
86b9ac3dee
Switch to using new config framework
3 years ago
Sayan Nandan
b46a5ac13f
Add method for chaining configuration sets
3 years ago
Sayan Nandan
d820ef910d
Add config file impl using new config impl
3 years ago
Sayan Nandan
afcd8031c2
Fix missing checks for server.noart and server.maxcon for CLI config
3 years ago
Sayan Nandan
a48b3fd423
Rewrite CLI config using new config impl
3 years ago
Sayan Nandan
3861a32c2e
Rewrite `cfgenv` using new config impl
3 years ago
Sayan Nandan
d46d73301b
Add config impls for `bgsave`, `snapshot` and `ssl`
3 years ago
Sayan Nandan
09afcd3e74
Add new config impl for server section
3 years ago
Sayan Nandan
bd56ee2db5
Add `FeedbackStack`, `ErrorStack` and `WarningStack`
3 years ago
Sayan Nandan
0b1ed6af6e
Add `ErrorStack` definition
3 years ago
Sayan Nandan
85e789a1ee
Upgrade deps
3 years ago
Sayan
2cdabfc43e
Automatically package Debian packages on release ( #235 )
...
* Add debian package generation
* Install cargo-deb on `make deb`
* Reload systemd daemon on postinst
* Add auto upload for Debian packages
* Consider using runner.os for simplicity
3 years ago
Sayan Nandan
88a8095f42
Upgrade deps
3 years ago
Sayan Nandan
c68d3ea3f4
Upgrade deps
...
All deps except for `clap` has been upgraded. Due to the removal of
the `args` field in `ArgMatches` in v3, and our dependence on the
field, we cannot upgrade to the latest version.
A PR has been created and once it is merged or a workaround
suggested, we can upgrade. (see clap-rs/clap#3265 )
3 years ago
Sayan Nandan
1c16e43d3d
Upgrade deps
3 years ago
Sayan Nandan
7c9058a02e
Upgrade deps
3 years ago
Sayan Nandan
6800b5eb0a
Use native endian for testing
...
Add changelog
3 years ago
Sayan Nandan
87f260d6b2
Support writing in native endian
3 years ago
Sayan Nandan
c0497e4339
Support reading data from a different endian
3 years ago
Sayan Nandan
50eb183f46
Upgrade deps
3 years ago
Sayan Nandan
93d1004c80
Add changelog and switch to using rustc edition 2021
3 years ago
Sayan Nandan
769378f7ce
Fix tests for `whereami`
3 years ago
Sayan Nandan
1deac63fb9
Use typed non-null array for `whereami`
3 years ago
Sayan
76f493753b
Add the `whereami` action ( #232 )
...
* Move macros into module
* Add the `whereami` action to identify the current entity
* Show entity group in the skysh prompt
* Add tests and actiondoc for `whereami`
* Add changelog entry
* Upgrade deps
3 years ago
Sayan Nandan
ce8b6e2340
Disable caching on ARM64 CI
...
Caching only adds latency to the entire workflow, so we'll disable
it until GitHub fixes the issue with their caching on IPv6.
3 years ago
Sayan Nandan
b63d90fa58
Add benches for skyd::protocol
...
This was added under the 'nightly' feature gate
3 years ago
Sayan Nandan
9656d7d9f6
Upgrade deps
...
Also added changelog entry
3 years ago
Sayan Nandan
cac9a9225f
Add a sanity test for errors in pipelines
3 years ago
Sayan Nandan
048e79df2c
Fix pipeline impl and add basic pipeline tests
...
The pipeline impl had a bug which caused a parse error; this happened
because we directly wrote the length as an integer (with the tsymbol)
when we were supposed to only write the integer in its string form
to the stream. This was fixed.
Also, some preliminary tests were added for pipelines.
3 years ago
Sayan Nandan
12db246725
Implement basic pipelines
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
9b132f4351
Upgrade deps
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
2814bdb151
Bump up version to 0.7.1
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
90276e3ead
Upgrade deps
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
18a0f557a0
Merge branch 'config/env' into next
3 years ago
Sayan Nandan
374ba57133
Add support for environment variable configuration
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
2387103c5c
Fix missing check for durations in config
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
468b1460e4
Fix conflict check tests
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
ef652befc9
Add env var `ConfigError` variant and rename ident
...
The `ParsedConfig` struct was renamed to `ConfigurationSet` because it
is more clear in contexts as parsing can be an ambiguous term in several
places.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
29a154f70e
Add tests for configuration conflict
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
f56610123e
Fix check of host and port values in config file
...
43bef62a
incorrectly dismissed the check for host/port config in the
case of a non-TLS setup. This commit fixes that.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
9a6bf82408
Simplify configuration handling
...
The previous configuration handling was rather messed up,
which however is something that this commit attempts to
simplify.
The check for configuration conflict was resolved with a far
more feasible approach and the handling of CLI/config file
configuration was also simplified greatly.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
43bef62a9e
Simplify config generation from cfg file
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
788b3073c0
Remove unnecessary use of `Box`
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
b5a0e02091
Move config file de into module
...
This really helps us reduce the grand clutter we created earlier.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
a43e1183b3
Bump up version
3 years ago
Sayan Nandan
eb037fcd62
Add basic `env` config module
3 years ago
Sayan Nandan
4dbcce6e04
Upgrade deps
3 years ago
Sayan Nandan
f85effc1c0
Fix empty list tests
3 years ago
Sayan Nandan
2ee227a5dc
Add `lget last` and `lget first` tests
3 years ago
Sayan Nandan
b368db2407
Add methods `FIRST` and `LAST`
3 years ago
Sayan Nandan
b9c60b4cc7
Use ptr offsets for accesses to reduce bloat
3 years ago
Sayan Nandan
ee78f9b1ce
Avoid excessive const inlining
...
Also removed unnecessary assembly bloat resulting from the generic index
accesses through (unck get)
3 years ago
Sayan Nandan
c3d977c703
Simplify `lmod clear` return
3 years ago
Sayan Nandan
a41fd7fa7f
Fix `lmod` bug resulting in return of wrong rcode
3 years ago
Sayan Nandan
c34f241f2e
Fix encoding check in list actions
3 years ago
Sayan Nandan
c5a58dec3d
Fix `is_empty` impl for `AnyArrayIter`
...
The impl was erroneously checking the base boxed slice for its length
when what we actually needed was the iter len (remainder).
3 years ago
Sayan Nandan
e1ee9314f5
Add `is_empty` to `AnyArrayIter` for specificity
3 years ago
Sayan Nandan
b34d059cd5
Add `Sync` trait bounds for ease with manual afns
3 years ago
Sayan Nandan
f6b3f8dc5c
Simplify encoding checks across actions
...
This commit also removes the forceful `__private` module naming for the
`dbtest` proc macro. Supplied modules can now have any name.
3 years ago
Sayan Nandan
c298f55b23
Use primitive casts to reduce jumps
3 years ago
Sayan Nandan
f4fbdcae16
Fix encoding check correctness in actions
3 years ago
Sayan Nandan
6fe3f53aa9
Use ptr offsets instead of index
3 years ago
Sayan
947327f379
Simplify response writing/handling ( #222 )
...
* Use `BoolTable` to simplify resps
* Fix inversion of table
* Use BLUT and NLUT wherever possible
3 years ago
Sayan Nandan
48ff6003af
Remove `keylen` and add `exists` and `del` tests
...
`keylen` checks the length of value for a given key. This cannot work
with listmaps
3 years ago
Sayan Nandan
b930c5d31b
Add listmap compatibility to `keylen` and `lskeys`
3 years ago
Sayan Nandan
68b9c9b81f
Add listmap compatibility with `exists`
3 years ago
Sayan Nandan
8ba3cb8028
Add `del` compatibility with listmaps
3 years ago
Sayan Nandan
ca77ca9f80
Add sanity tests for wrong-model error string
3 years ago
Sayan Nandan
7246b14115
Add `lmod insert` tests
3 years ago
Sayan Nandan
ed01c2fa82
Add `lmod remove` tests and fix `lget` tests
...
The error string length was incorrectly set causing the `lget` test to
fail
3 years ago
Sayan Nandan
d0ee66ac4c
Add remaining `lget` tests and add `lmod` tests
3 years ago
Sayan Nandan
ce99e974b9
Fix array extend impl ptr eq check
...
The end of allocation is reached when curptr > endptr
3 years ago
Sayan Nandan
acef265b04
Fix list tests
3 years ago
Sayan Nandan
8d43a70bd2
Add `lget` and `lset` tests
3 years ago
Sayan Nandan
677c4a844a
Merge branch 'model/impl-lists' into next
3 years ago
Sayan Nandan
9b4b140af7
Add listmap on-disk storage tests
3 years ago
Sayan Nandan
41a49aaf11
Add `POP` subaction to `lmod`
3 years ago
Sayan Nandan
9a2622c2ad
Add `insert` subaction to `lmod`
3 years ago
Sayan Nandan
02628d6ff3
Add `REMOVE` subaction to `lmod`
3 years ago
Sayan Nandan
c66206b9eb
Add `PUSH` subaction to `lmod`
3 years ago
Sayan Nandan
0278f6063d
Add `lmod` and `CLEAR` subaction
3 years ago
Sayan Nandan
5238298ac4
Add `VALUEAT` and `LIMIT` subactions to `lget`
3 years ago
Sayan Nandan
0e8d60df0a
Add `lget` for getting all list elements and len
3 years ago
Sayan Nandan
a04f676aa5
Add `lset` for list creation
3 years ago
Sayan Nandan
26ed51a077
Simplify modelcode computation
3 years ago
Sayan Nandan
3fc9967bf2
Add modelcode tests
3 years ago
Sayan Nandan
7a8a22c04a
Prevent usage of lists as a key type
3 years ago
Sayan Nandan
55457cf097
Add DDL support for listmaps
3 years ago
Sayan Nandan
47ace776a4
Fix tests
3 years ago
Sayan Nandan
4e68c4bd49
Add listmap se/de support in storage engine
3 years ago
Sayan Nandan
0ee5f69d50
Add basic listmap support
3 years ago
Sayan Nandan
cdbe1a81eb
Add more listmap corruption tests
3 years ago
Sayan Nandan
ff59627e95
Add encoding tests for mutating KVE actions
3 years ago
Sayan Nandan
8c95ef0049
Upgrade deps
3 years ago
Sayan Nandan
3e4bc1e6fc
Add remove method
3 years ago
Sayan Nandan
ef3e617147
Add basic listmap definition
3 years ago
Sayan Nandan
2c2e5f0ea6
Add listmap bytemarks
3 years ago
Sayan Nandan
234623a0cc
Add corruption tests
3 years ago
Sayan Nandan
c560cc68d8
Add multi-element listmap tests
3 years ago
Sayan Nandan
b8c8cda1d6
Add tests for list-based maps
3 years ago
Sayan Nandan
21157b2797
Use `RawSliceIter` and `RawSliceIterBorrowed`
...
These iterators (if you'd call them) provide safe abstractions over raw
pointers, enabling us to easily use them in the storage engine, reducing
verbosity and redundancy.
3 years ago
Sayan Nandan
ad403421d4
Fix storage engine bug for zero-sized map entries
...
gt was applicable here instead of ge, since for zero-sized elements, we
may have already reached the end of allocation.
3 years ago
Sayan Nandan
7dc17a0757
Fix de of empty lists or zero-sized elements
3 years ago
Sayan Nandan
6ad2793e76
Deconstruct se/de methods for lists
3 years ago
Sayan Nandan
08b7b7b821
Add list deserialization
3 years ago
Sayan Nandan
2154fba359
Add `raw_serialize_list` for list based maps
3 years ago
Sayan Nandan
a1ea9c3111
Upgrade deps and document parser
3 years ago
Sayan Nandan
1dfaa0e4a0
Switch to using `next_unchecked`
3 years ago
Sayan Nandan
831d84d65e
Fix forwarding of buffer before query execution
...
This has the ability to introduce UB because advancing the cursor might
invalidate the source pointers obtained by `protocol::Parser::parse()`
That's why we only forward the cursor after the query has been
executed. This is absolutely fine because even if another query
packet has been buffered, we will only forward by the amount we read
and not anything more.
3 years ago
Sayan Nandan
f1cf7e8796
Reduce `memcpy`s in Skyhash protocol impl
...
This commit does an extreme amount of `unsafe` work to reduce the amount
of data that is copied around. Previously, we:
1. Copied in from the buffer
2. Copied again when we did a ptr::read
This changes that to:
1. Zero-copies for operations that can operate on borrowed values
(i.e values by ref like GET, KEYLEN, ..)
2. One copy for operations that need owned values
Overall, this may not immediately seem beneficial for small queries
(afterall, computer memory is very fast), but for large keys -- this can
make a very significant difference. However, a big difference can be noticed
in memory usage under heavy load (with as much as a tenfold drop in some
cases).
But to make this possible, we have to break free from Rust's borrowing
rules as we can't normally pass around refs easily. So, we just turn
everything into raw pointers and operate on them directly. This is why
we introduce a significant amount of unsafe code.
3 years ago
Sayan
111d50b75e
Release v0.7.0 ( #210 )
...
* Simplify query gen in sky-bench and bump versions
* Upgrade to the latest driver version
3 years ago
Sayan Nandan
cdc33b349e
Fix wrong arg length being silently ignored
...
Where actions were supposed to report an action error, they silently ran
their significant part, ignoring the rest. This was fixed in:
- `DROP TABLE`
- `INSPECT KEYSPACES`
- `INSPECT KEYSPACE <ksid>`
- `INSPECT TABLE <entity>`
- `USE <entity>`
Tests for the same were added
3 years ago
Sayan Nandan
4ef28ea7e8
Upgrade deps
3 years ago
Sayan Nandan
76e0cf607c
Switch `MPOP` to returning a `Typed Array`
...
This makes its behavior similar to that of the other mutating actions.
3 years ago
Sayan Nandan
d744eaa2f6
Fix `HEYA` impl and add `HEYA` actiondoc
...
Also added heya echo test
3 years ago
Sayan Nandan
c45bfd03c8
Fix critical bug in flush routine
...
When a new instance is created, we need to:
1. Create the tree
This ONLY creates the directories
2. Create the PRELOAD
This is critical because this is our check for a new instance
3. Flush the tables
This is important because we have never flushed the tables/ks before.
If we don't do this -- the server would fail to start with a
`directory not found` error.
3 years ago
Sayan Nandan
3bd34ae5c9
Merge branch 'model/tsymbol' into next
3 years ago
Sayan Nandan
d7ac4ebe56
Fix inspect tests
3 years ago
Sayan Nandan
a24e03da60
Use `TypedArray`s in inspect commands
3 years ago
Sayan
596b593526
Fix detection of invalid container name ( #207 )
...
* Fix parser allowing `$` in container name
* Fix query engine tests
3 years ago
Sayan Nandan
95504a8be6
Fix `FlatArrayWriter` impl and `mpop` tests
...
Update driver version
3 years ago
Sayan Nandan
2ed3335a1e
Upgrade all interfaces to use the latest driver
...
Fix lskeys impl
3 years ago
Sayan Nandan
e389ee7860
Simplify encoding checks across actions
...
Checking all encoding errors beforehand simplifies a lot of things for
us. At the same time, this saves a lot of bandwidth as we don't have to
write encoding errors for each element -- instead we just write one.
3 years ago
Sayan Nandan
3a6c48727b
Upgrade all actions to use typed arrays
3 years ago
Sayan Nandan
f7d64689a6
Use `TypedArrayWriter` for typed Skyhash arrays
3 years ago
Sayan Nandan
0d2cfc661c
Use tsymbol to determine if binary or unicode
...
This enables clients to know whether they should expect binary data or
an unicode string.
3 years ago
Sayan Nandan
dd12ea599e
Take values by ref in `KVEngine` impl methods
3 years ago
Sayan Nandan
ec10d3f962
Fix `pop` and `mpop` tests
3 years ago
Sayan Nandan
fa2a4c2611
Add the `mpop` action and update the `pop` action
3 years ago
Sayan Nandan
1092bdfd65
Merge pull request #198 from skytable/storage/snapshotengine
3 years ago
Sayan Nandan
27a5562651
Don't panic on unimplemented pipeline query type
...
The return for inpsect table queries for Keymap model tables was fixed
3 years ago
Sayan Nandan
4ba70aa19b
Remove ser/de implementations
...
These impls are no longer needed
3 years ago
Sayan Nandan
42f3251d2c
Upgrade deps
...
Also added docs for Skymap
3 years ago
Sayan Nandan
da8462eda3
Switch hasher implementation
3 years ago
Sayan Nandan
153f940ff6
Merge branch 'memory/improve-cmap' into next
3 years ago
Sayan Nandan
c17e44ddf7
Fix Skymap borrowed iter impl
3 years ago
Sayan Nandan
9da0cdafee
Upgrade interfaces to use Skymap
3 years ago
Sayan Nandan
3759992cf8
Add iterators
3 years ago
Sayan Nandan
7498add90c
Add get, get_mut and entry methods to Skymap
3 years ago
Aaron Hill
28133ddc5f
Remove trailing semicolons from macros ( #201 )
...
This fixes warnings on the latest nightly.
See https://github.com/rust-lang/rust/issues/79813
3 years ago
Sayan Nandan
db60133dc3
Add basic Skymap methods
3 years ago
Sayan Nandan
bb19d024ea
Ignore errors in run loop
...
This fixes CVE-2021-37625
3 years ago
Sayan Nandan
8b7de7173e
Use flocks for pid file to enable auto release
...
This can help in situations where the process is forcefully terminated.
3 years ago
Sayan Nandan
891f9a2e06
Remove unused deps
...
Also simplified new instance check
3 years ago
Sayan Nandan
5a0d3017a5
Support non-interactive TLS passphrase input
3 years ago
Sayan Nandan
d54652c21e
Fix config not parsing `false` in ssl.only key
3 years ago
Sayan Nandan
bb14b62805
Reduce disk accesses with a trip switch
3 years ago
Sayan Nandan
0d3bfe486e
Use specialized result type for strong actions
3 years ago
Sayan Nandan
252dd9c08d
Add supdate concurrency tests
...
Also added changelog entry
3 years ago
Sayan Nandan
39856cdfd5
Add sset concurrency tests
3 years ago
Sayan Nandan
a8b716c892
Add sdel concurrency tests
3 years ago
Sayan Nandan
62b58f1a9f
Decompose strong actions' snapshotting core
3 years ago
Sayan Nandan
b47fcc2c88
Use snapshot isolation for strong actions
...
This makes strong actions far more reliable than the previous
implementation.
3 years ago
Sayan Nandan
2d5d32216c
Add some general borrow optimizations
3 years ago
Sayan
b6fcb4c035
Add `drop keyspace <name> force` ( #192 )
...
* Add forceful dropping of keyspaces
This commit also improves the reliability of `drop keyspace` in general
* Add changelog
* Add tests for `force_drop_keyspace`
* Upgrade deps
3 years ago
Sayan Nandan
e32b3e8ea1
Destructure methods in `BorrowedEntityGroup`
3 years ago
Sayan Nandan
69df98c69a
Only copy into `ObjectID` for `create table`
3 years ago
Sayan Nandan
3f3f381c50
Avoid copies into `ObjectID`
3 years ago
Sayan Nandan
133400b846
Only run mutating DDL queries if state is okay
3 years ago
Sayan Nandan
ee14656354
Use volatile tables for tests
3 years ago
Sayan Nandan
49fa843eb2
Parse the `swapks` header if it is provided
3 years ago
Sayan Nandan
f06e9ccdb5
Add FQE tests for flushdb
3 years ago
Sayan Nandan
abcb60463f
Add tests for `inspect`
3 years ago
Sayan Nandan
6924524c4e
Add ddl tests
3 years ago
Sayan Nandan
c5e4de9538
Fix lskeys and add tests for entity based queries
3 years ago
Sayan Nandan
bc9abd7ac3
Enable `lskeys` to accept entities
3 years ago
Sayan Nandan
32fcbc2075
Add changelog and bump up version
3 years ago
Sayan Nandan
c1249ccbf9
Prevent abuse of `system` table
3 years ago
Sayan Nandan
fed4597208
Parse volatile property from DDL queries
3 years ago
Sayan Nandan
c9e55451f3
Add inspection queries
...
This lets the user explore a keyspace/table.
3 years ago
Sayan Nandan
c36cbe69e4
Remove `HTable`
...
Our entire storage infrastructure has changed and this is no longer
needed.
3 years ago
Sayan Nandan
0a670a6555
Remove compat
...
We'll be shipping a migration tool, so this isn't required anymore.
3 years ago
Sayan Nandan
fb07d385fd
Support entity groups in `flushdb` and `dbsize`
3 years ago
Sayan Nandan
1025933cfb
Run tests parallelly in the `testsuite` keyspace
...
This saves us time and simplifies things. A lot.
3 years ago
Sayan Nandan
728c71f84f
Fix SE tests
3 years ago
Sayan Nandan
28f825910e
Fix tests and strong actions
3 years ago
Sayan Nandan
d43c3dc1cf
Fix storage engine tests
3 years ago
Sayan Nandan
a6fc09f990
Enable entity group based table deletion
3 years ago
Sayan Nandan
8e71ef2a01
Fix tests
3 years ago
Sayan Nandan
2f39e6808b
Enable entity group based table creation
3 years ago
Sayan Nandan
5402028b26
Add `use keyspace` and `use keyspace:table`
3 years ago
Sayan Nandan
2f7a2d546b
Add branch hints to parser
3 years ago