Sayan Nandan
ef652befc9
Add env var `ConfigError` variant and rename ident
...
The `ParsedConfig` struct was renamed to `ConfigurationSet` because it
is more clear in contexts as parsing can be an ambiguous term in several
places.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
29a154f70e
Add tests for configuration conflict
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
f56610123e
Fix check of host and port values in config file
...
43bef62a
incorrectly dismissed the check for host/port config in the
case of a non-TLS setup. This commit fixes that.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
9a6bf82408
Simplify configuration handling
...
The previous configuration handling was rather messed up,
which however is something that this commit attempts to
simplify.
The check for configuration conflict was resolved with a far
more feasible approach and the handling of CLI/config file
configuration was also simplified greatly.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
43bef62a9e
Simplify config generation from cfg file
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
788b3073c0
Remove unnecessary use of `Box`
...
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
b5a0e02091
Move config file de into module
...
This really helps us reduce the grand clutter we created earlier.
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
3 years ago
Sayan Nandan
eb037fcd62
Add basic `env` config module
3 years ago
Sayan Nandan
f85effc1c0
Fix empty list tests
3 years ago
Sayan Nandan
2ee227a5dc
Add `lget last` and `lget first` tests
3 years ago
Sayan Nandan
b368db2407
Add methods `FIRST` and `LAST`
3 years ago
Sayan Nandan
b9c60b4cc7
Use ptr offsets for accesses to reduce bloat
3 years ago
Sayan Nandan
ee78f9b1ce
Avoid excessive const inlining
...
Also removed unnecessary assembly bloat resulting from the generic index
accesses through (unck get)
3 years ago
Sayan Nandan
c3d977c703
Simplify `lmod clear` return
3 years ago
Sayan Nandan
a41fd7fa7f
Fix `lmod` bug resulting in return of wrong rcode
3 years ago
Sayan Nandan
c34f241f2e
Fix encoding check in list actions
3 years ago
Sayan Nandan
c5a58dec3d
Fix `is_empty` impl for `AnyArrayIter`
...
The impl was erroneously checking the base boxed slice for its length
when what we actually needed was the iter len (remainder).
3 years ago
Sayan Nandan
e1ee9314f5
Add `is_empty` to `AnyArrayIter` for specificity
3 years ago
Sayan Nandan
b34d059cd5
Add `Sync` trait bounds for ease with manual afns
3 years ago
Sayan Nandan
f6b3f8dc5c
Simplify encoding checks across actions
...
This commit also removes the forceful `__private` module naming for the
`dbtest` proc macro. Supplied modules can now have any name.
3 years ago
Sayan Nandan
c298f55b23
Use primitive casts to reduce jumps
3 years ago
Sayan Nandan
f4fbdcae16
Fix encoding check correctness in actions
3 years ago
Sayan Nandan
6fe3f53aa9
Use ptr offsets instead of index
3 years ago
Sayan
947327f379
Simplify response writing/handling ( #222 )
...
* Use `BoolTable` to simplify resps
* Fix inversion of table
* Use BLUT and NLUT wherever possible
3 years ago
Sayan Nandan
48ff6003af
Remove `keylen` and add `exists` and `del` tests
...
`keylen` checks the length of value for a given key. This cannot work
with listmaps
3 years ago
Sayan Nandan
b930c5d31b
Add listmap compatibility to `keylen` and `lskeys`
3 years ago
Sayan Nandan
68b9c9b81f
Add listmap compatibility with `exists`
3 years ago
Sayan Nandan
8ba3cb8028
Add `del` compatibility with listmaps
3 years ago
Sayan Nandan
ca77ca9f80
Add sanity tests for wrong-model error string
3 years ago
Sayan Nandan
7246b14115
Add `lmod insert` tests
3 years ago
Sayan Nandan
ed01c2fa82
Add `lmod remove` tests and fix `lget` tests
...
The error string length was incorrectly set causing the `lget` test to
fail
3 years ago
Sayan Nandan
d0ee66ac4c
Add remaining `lget` tests and add `lmod` tests
3 years ago
Sayan Nandan
ce99e974b9
Fix array extend impl ptr eq check
...
The end of allocation is reached when curptr > endptr
3 years ago
Sayan Nandan
acef265b04
Fix list tests
3 years ago
Sayan Nandan
8d43a70bd2
Add `lget` and `lset` tests
3 years ago
Sayan Nandan
677c4a844a
Merge branch 'model/impl-lists' into next
3 years ago
Sayan Nandan
9b4b140af7
Add listmap on-disk storage tests
3 years ago
Sayan Nandan
41a49aaf11
Add `POP` subaction to `lmod`
3 years ago
Sayan Nandan
9a2622c2ad
Add `insert` subaction to `lmod`
3 years ago
Sayan Nandan
02628d6ff3
Add `REMOVE` subaction to `lmod`
3 years ago
Sayan Nandan
c66206b9eb
Add `PUSH` subaction to `lmod`
3 years ago
Sayan Nandan
0278f6063d
Add `lmod` and `CLEAR` subaction
3 years ago
Sayan Nandan
5238298ac4
Add `VALUEAT` and `LIMIT` subactions to `lget`
3 years ago
Sayan Nandan
0e8d60df0a
Add `lget` for getting all list elements and len
3 years ago
Sayan Nandan
a04f676aa5
Add `lset` for list creation
3 years ago
Sayan Nandan
26ed51a077
Simplify modelcode computation
3 years ago
Sayan Nandan
3fc9967bf2
Add modelcode tests
3 years ago
Sayan Nandan
7a8a22c04a
Prevent usage of lists as a key type
3 years ago
Sayan Nandan
55457cf097
Add DDL support for listmaps
3 years ago
Sayan Nandan
47ace776a4
Fix tests
3 years ago
Sayan Nandan
4e68c4bd49
Add listmap se/de support in storage engine
3 years ago
Sayan Nandan
0ee5f69d50
Add basic listmap support
3 years ago
Sayan Nandan
cdbe1a81eb
Add more listmap corruption tests
3 years ago
Sayan Nandan
ff59627e95
Add encoding tests for mutating KVE actions
3 years ago
Sayan Nandan
3e4bc1e6fc
Add remove method
3 years ago
Sayan Nandan
ef3e617147
Add basic listmap definition
3 years ago
Sayan Nandan
2c2e5f0ea6
Add listmap bytemarks
3 years ago
Sayan Nandan
234623a0cc
Add corruption tests
3 years ago
Sayan Nandan
c560cc68d8
Add multi-element listmap tests
3 years ago
Sayan Nandan
b8c8cda1d6
Add tests for list-based maps
3 years ago
Sayan Nandan
21157b2797
Use `RawSliceIter` and `RawSliceIterBorrowed`
...
These iterators (if you'd call them) provide safe abstractions over raw
pointers, enabling us to easily use them in the storage engine, reducing
verbosity and redundancy.
3 years ago
Sayan Nandan
ad403421d4
Fix storage engine bug for zero-sized map entries
...
gt was applicable here instead of ge, since for zero-sized elements, we
may have already reached the end of allocation.
3 years ago
Sayan Nandan
7dc17a0757
Fix de of empty lists or zero-sized elements
3 years ago
Sayan Nandan
6ad2793e76
Deconstruct se/de methods for lists
3 years ago
Sayan Nandan
08b7b7b821
Add list deserialization
3 years ago
Sayan Nandan
2154fba359
Add `raw_serialize_list` for list based maps
3 years ago
Sayan Nandan
a1ea9c3111
Upgrade deps and document parser
3 years ago
Sayan Nandan
1dfaa0e4a0
Switch to using `next_unchecked`
3 years ago
Sayan Nandan
831d84d65e
Fix forwarding of buffer before query execution
...
This has the ability to introduce UB because advancing the cursor might
invalidate the source pointers obtained by `protocol::Parser::parse()`
That's why we only forward the cursor after the query has been
executed. This is absolutely fine because even if another query
packet has been buffered, we will only forward by the amount we read
and not anything more.
3 years ago
Sayan Nandan
f1cf7e8796
Reduce `memcpy`s in Skyhash protocol impl
...
This commit does an extreme amount of `unsafe` work to reduce the amount
of data that is copied around. Previously, we:
1. Copied in from the buffer
2. Copied again when we did a ptr::read
This changes that to:
1. Zero-copies for operations that can operate on borrowed values
(i.e values by ref like GET, KEYLEN, ..)
2. One copy for operations that need owned values
Overall, this may not immediately seem beneficial for small queries
(afterall, computer memory is very fast), but for large keys -- this can
make a very significant difference. However, a big difference can be noticed
in memory usage under heavy load (with as much as a tenfold drop in some
cases).
But to make this possible, we have to break free from Rust's borrowing
rules as we can't normally pass around refs easily. So, we just turn
everything into raw pointers and operate on them directly. This is why
we introduce a significant amount of unsafe code.
3 years ago
Sayan
111d50b75e
Release v0.7.0 ( #210 )
...
* Simplify query gen in sky-bench and bump versions
* Upgrade to the latest driver version
3 years ago
Sayan Nandan
cdc33b349e
Fix wrong arg length being silently ignored
...
Where actions were supposed to report an action error, they silently ran
their significant part, ignoring the rest. This was fixed in:
- `DROP TABLE`
- `INSPECT KEYSPACES`
- `INSPECT KEYSPACE <ksid>`
- `INSPECT TABLE <entity>`
- `USE <entity>`
Tests for the same were added
3 years ago
Sayan Nandan
76e0cf607c
Switch `MPOP` to returning a `Typed Array`
...
This makes its behavior similar to that of the other mutating actions.
3 years ago
Sayan Nandan
d744eaa2f6
Fix `HEYA` impl and add `HEYA` actiondoc
...
Also added heya echo test
3 years ago
Sayan Nandan
c45bfd03c8
Fix critical bug in flush routine
...
When a new instance is created, we need to:
1. Create the tree
This ONLY creates the directories
2. Create the PRELOAD
This is critical because this is our check for a new instance
3. Flush the tables
This is important because we have never flushed the tables/ks before.
If we don't do this -- the server would fail to start with a
`directory not found` error.
3 years ago
Sayan Nandan
3bd34ae5c9
Merge branch 'model/tsymbol' into next
3 years ago
Sayan Nandan
d7ac4ebe56
Fix inspect tests
3 years ago
Sayan Nandan
a24e03da60
Use `TypedArray`s in inspect commands
3 years ago
Sayan
596b593526
Fix detection of invalid container name ( #207 )
...
* Fix parser allowing `$` in container name
* Fix query engine tests
3 years ago
Sayan Nandan
95504a8be6
Fix `FlatArrayWriter` impl and `mpop` tests
...
Update driver version
3 years ago
Sayan Nandan
2ed3335a1e
Upgrade all interfaces to use the latest driver
...
Fix lskeys impl
3 years ago
Sayan Nandan
e389ee7860
Simplify encoding checks across actions
...
Checking all encoding errors beforehand simplifies a lot of things for
us. At the same time, this saves a lot of bandwidth as we don't have to
write encoding errors for each element -- instead we just write one.
3 years ago
Sayan Nandan
3a6c48727b
Upgrade all actions to use typed arrays
3 years ago
Sayan Nandan
f7d64689a6
Use `TypedArrayWriter` for typed Skyhash arrays
3 years ago
Sayan Nandan
0d2cfc661c
Use tsymbol to determine if binary or unicode
...
This enables clients to know whether they should expect binary data or
an unicode string.
3 years ago
Sayan Nandan
dd12ea599e
Take values by ref in `KVEngine` impl methods
3 years ago
Sayan Nandan
ec10d3f962
Fix `pop` and `mpop` tests
3 years ago
Sayan Nandan
fa2a4c2611
Add the `mpop` action and update the `pop` action
3 years ago
Sayan Nandan
1092bdfd65
Merge pull request #198 from skytable/storage/snapshotengine
3 years ago
Sayan Nandan
27a5562651
Don't panic on unimplemented pipeline query type
...
The return for inpsect table queries for Keymap model tables was fixed
3 years ago
Sayan Nandan
4ba70aa19b
Remove ser/de implementations
...
These impls are no longer needed
3 years ago
Sayan Nandan
42f3251d2c
Upgrade deps
...
Also added docs for Skymap
3 years ago
Sayan Nandan
da8462eda3
Switch hasher implementation
3 years ago
Sayan Nandan
153f940ff6
Merge branch 'memory/improve-cmap' into next
3 years ago
Sayan Nandan
c17e44ddf7
Fix Skymap borrowed iter impl
3 years ago
Sayan Nandan
9da0cdafee
Upgrade interfaces to use Skymap
3 years ago
Sayan Nandan
3759992cf8
Add iterators
3 years ago
Sayan Nandan
7498add90c
Add get, get_mut and entry methods to Skymap
3 years ago
Aaron Hill
28133ddc5f
Remove trailing semicolons from macros ( #201 )
...
This fixes warnings on the latest nightly.
See https://github.com/rust-lang/rust/issues/79813
3 years ago
Sayan Nandan
db60133dc3
Add basic Skymap methods
3 years ago
Sayan Nandan
bb19d024ea
Ignore errors in run loop
...
This fixes CVE-2021-37625
3 years ago
Sayan Nandan
8b7de7173e
Use flocks for pid file to enable auto release
...
This can help in situations where the process is forcefully terminated.
3 years ago
Sayan Nandan
891f9a2e06
Remove unused deps
...
Also simplified new instance check
3 years ago
Sayan Nandan
5a0d3017a5
Support non-interactive TLS passphrase input
3 years ago
Sayan Nandan
d54652c21e
Fix config not parsing `false` in ssl.only key
3 years ago
Sayan Nandan
bb14b62805
Reduce disk accesses with a trip switch
3 years ago
Sayan Nandan
0d3bfe486e
Use specialized result type for strong actions
3 years ago
Sayan Nandan
252dd9c08d
Add supdate concurrency tests
...
Also added changelog entry
3 years ago
Sayan Nandan
39856cdfd5
Add sset concurrency tests
3 years ago
Sayan Nandan
a8b716c892
Add sdel concurrency tests
3 years ago
Sayan Nandan
62b58f1a9f
Decompose strong actions' snapshotting core
3 years ago
Sayan Nandan
b47fcc2c88
Use snapshot isolation for strong actions
...
This makes strong actions far more reliable than the previous
implementation.
3 years ago
Sayan Nandan
2d5d32216c
Add some general borrow optimizations
3 years ago
Sayan
b6fcb4c035
Add `drop keyspace <name> force` ( #192 )
...
* Add forceful dropping of keyspaces
This commit also improves the reliability of `drop keyspace` in general
* Add changelog
* Add tests for `force_drop_keyspace`
* Upgrade deps
3 years ago
Sayan Nandan
e32b3e8ea1
Destructure methods in `BorrowedEntityGroup`
3 years ago
Sayan Nandan
69df98c69a
Only copy into `ObjectID` for `create table`
3 years ago
Sayan Nandan
3f3f381c50
Avoid copies into `ObjectID`
3 years ago
Sayan Nandan
133400b846
Only run mutating DDL queries if state is okay
3 years ago
Sayan Nandan
ee14656354
Use volatile tables for tests
3 years ago
Sayan Nandan
49fa843eb2
Parse the `swapks` header if it is provided
3 years ago
Sayan Nandan
f06e9ccdb5
Add FQE tests for flushdb
3 years ago
Sayan Nandan
abcb60463f
Add tests for `inspect`
3 years ago
Sayan Nandan
6924524c4e
Add ddl tests
3 years ago
Sayan Nandan
c5e4de9538
Fix lskeys and add tests for entity based queries
3 years ago
Sayan Nandan
bc9abd7ac3
Enable `lskeys` to accept entities
3 years ago
Sayan Nandan
c1249ccbf9
Prevent abuse of `system` table
3 years ago
Sayan Nandan
fed4597208
Parse volatile property from DDL queries
3 years ago
Sayan Nandan
c9e55451f3
Add inspection queries
...
This lets the user explore a keyspace/table.
3 years ago
Sayan Nandan
c36cbe69e4
Remove `HTable`
...
Our entire storage infrastructure has changed and this is no longer
needed.
3 years ago
Sayan Nandan
0a670a6555
Remove compat
...
We'll be shipping a migration tool, so this isn't required anymore.
3 years ago
Sayan Nandan
fb07d385fd
Support entity groups in `flushdb` and `dbsize`
3 years ago
Sayan Nandan
1025933cfb
Run tests parallelly in the `testsuite` keyspace
...
This saves us time and simplifies things. A lot.
3 years ago
Sayan Nandan
728c71f84f
Fix SE tests
3 years ago
Sayan Nandan
28f825910e
Fix tests and strong actions
3 years ago
Sayan Nandan
d43c3dc1cf
Fix storage engine tests
3 years ago
Sayan Nandan
a6fc09f990
Enable entity group based table deletion
3 years ago
Sayan Nandan
8e71ef2a01
Fix tests
3 years ago
Sayan Nandan
2f39e6808b
Enable entity group based table creation
3 years ago
Sayan Nandan
5402028b26
Add `use keyspace` and `use keyspace:table`
3 years ago
Sayan Nandan
2f7a2d546b
Add branch hints to parser
3 years ago
Sayan Nandan
6eaf580ac8
Add entity group parsing
3 years ago
Sayan Nandan
3249fcb347
Add `drop table` and `drop keyspace` queries
3 years ago
Sayan Nandan
d6e8db7d8f
Add `create table` and `create keyspace` queries
3 years ago
Sayan Nandan
c1064a7cd1
Implement and upgrade to `Corestore`
3 years ago
Sayan Nandan
a5f735e977
Add `drop table` and `drop keyspace`
3 years ago
Sayan Nandan
608e008a65
Add `create table` and `create keyspace`
3 years ago
Sayan Nandan
5bab0fb91b
Use global flush lock to coordinate disk access
3 years ago
Sayan Nandan
6e17ef6d5e
Add system keyspace and remove `_system` table
3 years ago
Sayan Nandan
16ac791ff5
Add flush methods for snapshots
3 years ago
Sayan Nandan
03518c22c3
Add corestore impl
3 years ago