62b58f1a9f
Decompose strong actions' snapshotting core
3 years ago
b47fcc2c88
Use snapshot isolation for strong actions
...
This makes strong actions far more reliable than the previous
implementation.
3 years ago
2d5d32216c
Add some general borrow optimizations
3 years ago
b6fcb4c035
Add `drop keyspace <name> force` ( #192 )
...
* Add forceful dropping of keyspaces
This commit also improves the reliability of `drop keyspace` in general
* Add changelog
* Add tests for `force_drop_keyspace`
* Upgrade deps
3 years ago
e32b3e8ea1
Destructure methods in `BorrowedEntityGroup`
3 years ago
69df98c69a
Only copy into `ObjectID` for `create table`
3 years ago
3f3f381c50
Avoid copies into `ObjectID`
3 years ago
133400b846
Only run mutating DDL queries if state is okay
3 years ago
ee14656354
Use volatile tables for tests
3 years ago
49fa843eb2
Parse the `swapks` header if it is provided
3 years ago
f06e9ccdb5
Add FQE tests for flushdb
3 years ago
abcb60463f
Add tests for `inspect`
3 years ago
6924524c4e
Add ddl tests
3 years ago
c5e4de9538
Fix lskeys and add tests for entity based queries
3 years ago
bc9abd7ac3
Enable `lskeys` to accept entities
3 years ago
32fcbc2075
Add changelog and bump up version
3 years ago
c1249ccbf9
Prevent abuse of `system` table
3 years ago
fed4597208
Parse volatile property from DDL queries
3 years ago
c9e55451f3
Add inspection queries
...
This lets the user explore a keyspace/table.
3 years ago
c36cbe69e4
Remove `HTable`
...
Our entire storage infrastructure has changed and this is no longer
needed.
3 years ago
0a670a6555
Remove compat
...
We'll be shipping a migration tool, so this isn't required anymore.
3 years ago
fb07d385fd
Support entity groups in `flushdb` and `dbsize`
3 years ago
1025933cfb
Run tests parallelly in the `testsuite` keyspace
...
This saves us time and simplifies things. A lot.
3 years ago
728c71f84f
Fix SE tests
3 years ago
28f825910e
Fix tests and strong actions
3 years ago
d43c3dc1cf
Fix storage engine tests
3 years ago
a6fc09f990
Enable entity group based table deletion
3 years ago
8e71ef2a01
Fix tests
3 years ago
2f39e6808b
Enable entity group based table creation
3 years ago
5402028b26
Add `use keyspace` and `use keyspace:table`
3 years ago
2f7a2d546b
Add branch hints to parser
3 years ago
6eaf580ac8
Add entity group parsing
3 years ago
3249fcb347
Add `drop table` and `drop keyspace` queries
3 years ago
d6e8db7d8f
Add `create table` and `create keyspace` queries
3 years ago
c1064a7cd1
Implement and upgrade to `Corestore`
3 years ago
a5f735e977
Add `drop table` and `drop keyspace`
3 years ago
608e008a65
Add `create table` and `create keyspace`
3 years ago
5bab0fb91b
Use global flush lock to coordinate disk access
3 years ago
6e17ef6d5e
Add system keyspace and remove `_system` table
3 years ago
16ac791ff5
Add flush methods for snapshots
3 years ago
03518c22c3
Add corestore impl
3 years ago
421ff19405
Use registry for handling global state
3 years ago
0e88d76444
Simplify hints
3 years ago
d1f6916251
Make mksnap use `action!` and simplify `IoResult`
3 years ago
c9545a30f1
Do not panic on unknown data type
3 years ago
327953dce0
Fix qe returning full resp packet instead of group
3 years ago
9535975d03
Create the dir tree if the instance is new
3 years ago
b2c0b9ecf2
Enable fixed len mutable params in `action!` macro
3 years ago
589ef85e2c
Simplify action impls with `action!` macro
3 years ago
66c9822f5d
Add routines to unflush an entire `Memstore`
3 years ago
a919b1c934
Add tests for flush and unflush routines
3 years ago
454704574b
Fix flush routines
3 years ago
2393efb590
Encode model bytemark into `PARTMAP`
3 years ago
610144f78e
Add unflush routines
3 years ago
6baa61176f
Add volatility tests
3 years ago
66b9ac27af
Write storage type into PARTMAP
...
This commit adds a storage_type segment to the PARTMAP disk file. This
contains information about the storage type of the table.
Is it volatile? Is it persistent? 8-bits were added for future
improvements.
3 years ago
1d403c0d1a
Add flush routines
3 years ago
e89417cbc6
Fix preload generation and add preload decoding
3 years ago
87d79650ce
Add preload generation
3 years ago
bca8df5863
Add methods to create directory tree
3 years ago
058b1ef1c6
Replace ns with ks
3 years ago
ad48e5478c
Add `Integer64BufferRaw` and use it in `resp`
3 years ago
71ab845d02
Fix zeroed impl for `Array<T, N>`
3 years ago
48e29b6ec6
Add `Integer32Buffer` for faster encoding/decoding
3 years ago
5790e99a98
Fix filename generator and use LUT for 32-bit ints
3 years ago
ca55694904
And `interface` for fs
3 years ago
b162756f80
Avoid unnecessary referencing
...
Even though the compiler will do immediate derefs, let us be explicit.
3 years ago
03a229104d
Fix alignment and auto static lifetime causing UB
3 years ago
c24e83c0a7
Encode all sizes to little endian
...
This is very convenient for us and we will provide advanced byte
ordering only if our users demand for it.
3 years ago
e1dfa12ba4
Add runtime panic check on 32-bit or lower
3 years ago
5adc269e11
Add endian info and pointer-width check
...
When a file created by a 64-bit system is read on a 16/32 bit system,
there may be a size overflow. If so, we should do a runtime panic.
3 years ago
74a0592fbc
Simplify transmutation
...
Also use smaller vectors for faster tests to avoid problems with
thread stack sizes on Windows.
3 years ago
c20302ef75
Add storage module for custom encoding/decoding
3 years ago
7c835d03b3
Add NS/KS swap headers in protocol
3 years ago
17d1c472b6
Use proc macro for uninit array magic
...
The proc macro does some magic to give us a const array with the full
size without having to manually write it. Magic!
3 years ago
fd3e06beda
Fix array length check assertion
3 years ago
e5b0588cca
Fix UB due to use of from_const_array
3 years ago
f4379d5688
Use Array[64] for NS/KS names
...
We limit the sizes of keyspaces/namespaces because very long names may
cause fs errors on some file systems.
3 years ago
8faf653d2e
Fix ser/de for `Coremap<Array, Array>`
3 years ago
bd679f9b79
Document `Array` and `IArray`
3 years ago
b997afe89a
Fix push function in `Array`
3 years ago
4a27f83e6b
Impl `Send` and `Sync` for `Array`
3 years ago
0067b7d1b7
Implement `Array` type
...
This allows us to have fixed size arrays right on the stack
3 years ago
5ff045bd93
Impl deserialize for `IArray`
3 years ago
e25d13afff
Add BP optimizations
3 years ago
036b507de8
Impl Serialize for IArray
3 years ago
4fe7aa7050
Add manip methods to `IArray`
3 years ago
6c00ffae3b
Add methods to add items to `IArray`
3 years ago
3739aa54fd
Add basic `IArray` impl
3 years ago
e3d749ac20
Add methods to drop keyspaces and tables
3 years ago
845ef82060
Use static slices for responses
3 years ago
26a22c3102
Add dtor tests for Lazy
3 years ago
a7e11cc281
Add methods to create ns, ks and tables
3 years ago
bbcbe0756b
Add basic methods to memstore
3 years ago
7ea890765d
Correct kvengine defs and add custom `Lazy` type
3 years ago
74126ec7cb
Add variable width characters and failure test
...
This is just for sanity
3 years ago
013be0058b
Add emoji tests
3 years ago
8749cfa134
Add memstore
3 years ago
c953b88695
Add test with bincode
3 years ago
b1383bf8c9
Add encoding evaluation in `KVEngine`
3 years ago
a276091726
Account for perl errors
3 years ago
5f75df7109
Add some optimizations for unicode checks
3 years ago
ead3f62ded
Add failure cases for unicode
3 years ago
bdfaf6ec4d
Ensure that perl doesn't error
3 years ago
5e4cd5be4b
Use deterministic finite automaton for validation
...
A dual stream approach provides even more speed improvements
3 years ago
8cfab3f7d3
Add convenience macros
3 years ago
efec980fa6
Add basic `BufferBlockReader` definition
3 years ago
bdc4b3483f
Add basic `KVEngine` definition
3 years ago
552d454940
Enable TLS port to be configured via CLI args ( #186 )
...
* Enable TLS port to be configured via CLI
* Add changelog entry
3 years ago
141e39eee1
Use env var to determine TLS cert location
3 years ago
d43e6f41da
Use `QuickLock` instead of mutex ( #185 )
...
* Use our own lock instead of parking_lot::Mutex
* Account for spurious failures in cmpxchg weak
* Ignore send error because parent may have panicked
The parent thread may have already panicked, dropping the rx.
3 years ago
3a363d18cb
Simplify artwork [skip ci]
3 years ago
bae2b8354c
Release v0.6.3 ( #184 )
...
* Bump up version to 0.6.3
* Update changelog entry
3 years ago
2d7b9d7667
Add automated tests for TLS ( #183 )
...
* Auto start with TLS
* Add automated tests for SSL
* Add cert generation script
* Use script to generate SSL cert
3 years ago
864c6d461f
Enable maximum connections to be configured manually ( #182 )
...
* Enable maximum connections to be configured
* Add arbiter for handling server startup
* Add handling of maxcon for command-line args
* Add changelog entry
3 years ago
79f657b462
Add more LLVM specific optimizations
...
Just to reduce LLVM bloat
3 years ago
e30d51a599
Release v0.6.2 ( #180 )
...
* Bump up version and add changelog entry
* Use separate cache object for release workflows
3 years ago
0f06f7b26f
Add some LLVM specific optims for O1 builds
3 years ago
d53a0cb505
Fix handling of SIGTERM on *nix ( #178 )
...
* Fix handling of SIGTERM on *nix
This is just for future extensibility
* Fix error codes
I have been silly enough to break error codes
3 years ago
66e5d41302
Simplify overall locks ( #176 )
...
* Remove the need for TableLockStateGuard
The htable impl uses locks under the hood making external locks
redundant.
* Use atomics instead of rwlock for poisoned state
* Simplify snapshot locking
3 years ago
2a2addfa6d
Upgrade deps and add changelog entries
3 years ago
0f1264d312
Decompose linearity tests and utils into modules
...
Also fixed license headers
3 years ago
a87478dcba
Optimize dependencies
3 years ago
76acde2f4f
Fix missing action argument in setkeys macro
3 years ago
26775924ac
Add tests for pop
3 years ago
57c957d4e7
Add `pop` action
3 years ago
cdae667cb0
Fix pid file creation ( #170 )
...
* Remove the pid file if runtime errors occur
* Clean up error handling and fix pid file creation
The pid file was being created before evaluating the args, now it may
happen that incorrect args or --help was passed: in that event, the pid
file remains created. This was also fixed, besides some refactoring.
3 years ago
ca9e482f47
Deter other processes from using the same data dir ( #169 )
...
* Deter other processes from using the same data dir
For more information, see #167
* Don't lock `pid_file`
Windows has mandatory locking so second instance won't be able to read
the PID of the other process. We'll just keep the file descriptor/handle
open
3 years ago
7349e461e6
Try to auto recover the save operation on termination ( #166 )
...
This is very useful because it removes the need for user intervention in
the event save on termination fails. Say the save operation fails due to
'some bad daemon' changing the directory's perms. Now skyd reports this
error while trying to save upon termination. Our sysadmin now fixes the
perms issue. The previous design would force the sysadmin to _somehow_
foreground skyd and hit enter. That is silly. The new design just
attempts to do a save operation every 10 seconds. So in case the issue
is fixed, the save operation will recover on its own.
Why not exponential backoff?
That's because the issue can be fixed some long time later and we may
have reached a large backoff value so the save that could have succeeded
would have to wait for a long duration before it can do anything
meaningful.
This also fixes a bug that caused BGSAVE errors to be reported as info
class log entries.
3 years ago
8df9901740
Upgrade deps and actiondoc ( #165 )
3 years ago
e553c5172b
Release v0.6.1 ( #164 )
...
* Explicitly fsync and relax CPU on snap busy-loop
This commit also switches to using global `VERSION` and `URL` statics
than defining it per-crate.
* Add changelog entry and bump up version
* Optimize `dbtest` macro and rm redundant allocs
* Upgrade deps
3 years ago
0c33395a09
Add some general optimizations
3 years ago
1bde8b197d
Fix file-locking on solaris ( #162 )
3 years ago
72d871ed3f
Upgrade deps
3 years ago
1bec90baac
Optimize `sset` and `sdel` implementations
3 years ago
2eedb041bb
Make `gen_constants_and_matches!` macro logical
...
This commit imporves the overall 'look' of the macro and makes it appear
more logical
3 years ago
58830edc80
Use iterators for actions
3 years ago
a839137643
Move actions into an `actions` module
...
These are actions and shouldn't be called the `kvengine`.
3 years ago
f8ea4c33de
ucase for ASCII only
...
This is a silly optimization but can be significantly faster for larger
action names. Also, since there are (should be) no UTF-8 characters in
the first argument, this is absolutely fine and sensible.
3 years ago
a68c42f720
Document missing instances of `unsafe`
3 years ago
952c5caa86
Poison the database by default if snapshotting fails ( #160 )
...
* Stop accepting writes if snapshotting fails
This is an important consideration: if BGSAVE fails and poisons the
database, snapshotting can and should too. But this is debatable in some
parts. For example, users may configure snapshots to be on a network
file system (symlinked maybe) and this can fail.
Now in some cases, this failure 'may be acceptable'. This commit adds a
way to customize this behavior through the `failsafe` key in the
snapshots section of the cfg file and through the --stop-write-on-fail
option passed to `skyd` on startup. However, BGSAVE remains unchanged:
it will always poison the database if it fails. If the user doesn't want
this, they can simply disable BGSAVE.
* Add changelog
3 years ago
ac336ff821
Add missing changes in changelog
...
Use `pat` token instead of `path` in query engine
3 years ago
eea0f86c97
Upgrade skytable client driver version
3 years ago
6c9a36d397
Improve compat to use storage formats for upgrades ( #158 )
3 years ago
260b336bf6
Add tests for races and synchronized unlocks
3 years ago
a1320da52b
Migrate to using `Coremap` ( #156 )
...
* Upgrade all interfaces to use new HTable
* Document `HTable`
3 years ago
966c2594bf
Fix `lskeys` tests
3 years ago
b77b783064
Add tests for lskeys
3 years ago
ea7891fba7
Implement lskeys
3 years ago
449da56308
Upgrade all interfaces to use new in-memory table
3 years ago
Sayan Nandan