16 KiB
Changelog
All changes in this project will be noted in this file.
Unreleased
Fixes
- Fixed infinite wait (loop) when sample space for key generation is not large enough
- Removed
upgrade
subcommand fromskyd
which was removed in 0.7, but was erroneously accepted in the CLI parameters - Restored ability to use
--restore <backupdir>
to restore data from previous snapshots which was silently ignored
Additions
- Added
dev/prod
mode for making sure that the recommended production settings are used - Added support for system native endian storage (backward compatible)
Version 0.7.2
Additions
skyd
:- Pipelined queries are now supported:
- Saves bandwidth
- Reduces latency
- The new non-null typed array type was added
- The
whereami
action was added which lets one find out which entity the current connection is connected to
- Pipelined queries are now supported:
skysh
:- now shows the current entity in the prompt
- handles special character strings more reliably
- now supports splitting a command across multiple lines
- now supports multi-line pastes
- now supports comments (beginning with
#
)
Version 0.7.1
Additions
- Added the list data type
- Added actions for lists:
LSET <list> <values> ...
LGET
LGET <list>
LGET <list> LEN
LGET <list> VALUEAT <idx>
LGET <list> LIMIT <limit>
LGET <list> FIRST
LGET <list> LAST
LMOD
LMOD <list> push <value>
LMOD <list> insert <index> <value>
LMOD <list> pop <optional index>
LMOD <list> remove <index>
LMOD <list> clear
- Added creation of lists:
CREATE TABLE <entity> keymap(type,list<type>)
- Added compatibility of
DEL
,EXISTS
andLSKEYS
with lists - Added support for configuration via environment variables
Improvements
- Reduced memory usage in
sky-bench
- Reduced allocations in Skyhash (
skyd
) protocol implementation - Misc. fixes in internal structures (
skyd
) - Improvements in printing of binary strings in
skysh
Fixes
- Fixed unexpected removal of single and double quotes from input strings in
skysh
- Fixed incorrect removal of quotes from input strings in
skysh
Version 0.7.0
Additions
-
Multiple keyspaces:
- Keyspaces hold multiple tables and determine the replication strategy
- Keyspaces can be created with:
CREATE KEYSPACE <name>
- The
system
keyspace is reserved for the system while thedefault
keyspace is the one available by default. Thesystem
ordefault
keyspace cannot be removed - Keyspaces can be deleted with:
DROP KEYSPACE <name>
- If a keyspace still has tables, it cannot be dropped by using
DROP KEYSPACE <name>
, instead one needs to useDROP KEYSPACE <name> force
to force drop the keyspace
-
Multiple tables:
- Tables hold the actual data. When you connect to the server, you are connected to the
default
table in thedefault
keyspace. This table is non-removable - Tables can be created with:
CREATE TABLE <entity> <model>(modelargs) <properties>
- Tables can be dropped with:
DROP TABLE <entity>
- Tables hold the actual data. When you connect to the server, you are connected to the
-
Entity groups: While using DDL queries and inspection queries, we can use the Fully Qualified Entity (FQE) syntax instead of the table name. For example, to
inspect
thecyan
table in keyspacesupercyan
, one can simply run:INSPECT TABLE supercyan:cyan
The syntax is:
keyspace:table
Note: Both keyspaces and tables are entities. The names of entities must:
- Begin with an underscore (_) or an ASCII alphabet
- Not begin with a number (0-9)
- Must have lesser than 64 characters
-
Keymap data model:
- To create a keymap table, run:
CREATE TABLE <entity> keymap(<type>,<type>)
- The following types were introduced:
str
: A valid unicode stringbinstr
: A binary string
- To create a keymap table, run:
-
Volatile table property:
- To create a volatile table, irrespective of the data model, run:
CREATE TABLE <entity> <model>(modelargs) volatile
- Volatile tables always exist, but the data in them does not persist between restarts. This makes them extremely useful for caches
- To create a volatile table, irrespective of the data model, run:
-
Inspection:
- Keyspaces can be inspected with:
This will list all the tables in the keyspaceINSPECT KEYSPACE <name>
- Tables can be inspected with:
This will give information about the data model and other properties of the tableINSPECT TABLE <entity>
- To list all keyspaces, this can be run:
INSPECT KEYSPACES
- Keyspaces can be inspected with:
-
Cyanstore 1A disk storage format: Cyanstore (v1A) was a new storage format built for the multi-keyspace-table world. It efficiently stores and retrieves records, tables and keyspaces
-
Realtime keyspace/table switch
- To switch to a new keyspace in real-time, one needs to run:
USE keyspace
- To switch to a new table in real-time, one needs to run:
USE keyspace:table
- To switch to a new keyspace in real-time, one needs to run:
-
Entity respecting actions:
FLUSHDB
: To flush all the data in a specific table, run:FLUSHDB <entity>
DBSIZE
: To see the number of entries in a specific table, run:DBSIZE <entity>
LSKEYS
:LSKEYS
will return keys from the current tableLSKEYS <count>
will return count keys from the current tableLSKEYS <entity>
will return keys from the given tableLSKEYS <entity> <count>
will return count keys from the given table
-
Snapshot isolation for strong actions: This makes strong actions extremely reliable when compared to the earlier releases
-
Non-interactive TLS private key passphrase input: Just save your password in some file and then pass
--tlspassin /path/to/passfile.txt
. You can do the same by using thetlspassin
key under SSL in the configuration file -
MPOP
now replacesPOP
to accept multiple keys whilePOP
will accept a single key to follow theMGET
/GET
naming convention -
TLS port can now be set to a custom port via CLI arguments
-
sky-bench
can now run multiple times to get average values through the--runs
option -
HEYA
now does an echo with the second argument
Fixes
- Zero length argument causing runtime panic in
skysh
HEYA!
not reporting errors on incorrect number of arguments- Panic on incorrect data type in
skyd
sky-bench
no longer affects your personal data because it creates a random temporary table under thedefault
keyspacesky-bench
'stestkey
subcommand causing key collisions- Fix log output in
sky-bench
even if the--json
flag was passed - Use flocks to enable auto release of pid file, even if process is forcefully terminated
- Fixes CVE-2021-37625
Breaking
- All actions now accept the
AnyArray
type introduced in Skyhash 1.1 POP
now accepts one key whileMPOP
accepts multiple keys- Disk storage format has changed
- The
upgrade
subcommand has been removed
Version 0.6.4 [2021-08-05]
Fixes
- Fixes CVE-2021-37625 (backport)
Version 0.6.3 [2021-06-27]
Additions
- The maximum number of clients can now be manually configured [see #182]
Fixes
- Ability to connect to the server over TLS has been restored in
skysh
[see #181]
Version 0.6.2 [2021-06-24]
Fixes
- The save operation now automatically attempts to recover on failure during termination [see #166]
- More than one process can no longer concurrently use the same data directory, preventing any possible data corruption [see #169, #167]
- Fixed longstanding error in
sky-bench
component that caused key collisions - Fixed bug in
sky-bench
that allowed passing 0 for the inputs - Fixed handling of SIGTERM in
skyd
[see #178] - Fixed incorrect termination codes [see #178]
Additions
Workflow
Version 0.6.1 [2021-06-07]
No breaking changes
- Snapshotting failure will now poison the database (customizable through CLI options or the configuration file) [see #160]
- Added file-locking on Solaris [see #162]
- Fixed missing explicit
fsync
orFlushFileBuffers
after writing data - Optimized wait on snapshot busy-loop using
_mm_pause
(on x86/x86_64) or__yield
(on aarch64/arm)
Version 0.6.0 [2021-05-27]
Breaking changes!
- ⚠ Dropped support for Terrapipe 1.0 (reached EOL)
- ⚠ New disk storage format with much faster reads and/or writes
- Added support for Skyhash 1.0 (see #147)
- Fixed persistence bugs (see #151)
- Fixed bugs in background services (see #152)
- Make BGSAVE recoverable (see #153)
- Added
lskeys
action (see #155) - Added
compat
module (see #158) - Added backward compatibility for storage formats all the way upto 0.3.0. See this wiki article for more information
Upgrading existing clients
As Terrapipe 1.0 has reached EOL, all clients have to be upgraded to use the Skyhash 1.0 Protocol.
Upgrading existing datasets
Please refer to this wiki article.
Improvements in the new protocol (Skyhash)
- Upto 40% lower bandwidth requirements
- Upto 30% faster queries
- Support for recursive arrays
- More robust and well tested than Terrapipe
Internal codebase improvements
- BGSAVE is now tested by the test suite
skysh
andsky-bench
both use the Rust client driver for Skytable.
Version 0.5.3 [2021-05-13]
No breaking changes
Fix persistence (see #150)
Version 0.5.2 [2021-05-07]
No breaking changes
sky-bench
is less agressive and runs sanity test before benchmarkingskyd
now locks the data file (thedata.bin
file)- The data directory structure has been changed (see #144) (all files are now stored in ./data/*)
- Fixed 'Connection Forcibly Closed' errors on Windows (see #110)
- Add support for line-editing and keyboard shortcuts on
skysh
(see #142) - Fixed problems while parsing snapshots in the snapshot directory (see #144)
- The old data directory structure has been deprecated (see #144)
- Official support for 32-bit platforms (see #139)
- Tier-2 Support for Linux musl (x86_64) (see #136, #135)
Version 0.5.1 [2021-03-17]
No breaking changes
- Built-in TLS/SSL support
- Custom host/port settings in
sky-bench
- Mock keys can be created with
sky-bench
- Security patch for VE/S/00001 (CVE-2021-32814)
- Escaping for spaces in
skysh
tdb
is now calledskyd
(short for 'Skytable Daemon')
Version 0.5.0 [2020-11-19]
This release introduces breaking changes!
- Command line configuration added to
tdb
- ⚠ Positional arguments in
tsh
andtdb-bench
have been removed MKSNAP
now has an enhanced version which enables snapshots to be created even if they're disabled on the server side- If
BGSAVE
fails, no more writes will be accepted on the server side. All commands that try to modify data will return an internal server error tdb-bench
now provides JSON output with the--json
flag- The
Dockerfile
was fixed to use command line arguments instead of the configuration file which caused problems - The
enabled
key under thesnapshots
key in the configuration file has been removed
Upgrading
- Users who ran
tsh
liketsh 172.17.0.1 2003
will now need to run:
tsh -h 172.17.0.1 -p 2003
- Users who ran
tdb-bench
liketdb-bench 10 100000 4
will now need to run:
tdb-bench -c 10 -q 100000 -s 4
- To enable snapshots, you just have to add the key: there is no need for the
enabled
key. To disable snapshots, you just have to omit thesnapshot
key (block) from your configuration file
Version 0.4.5 [2020-10-29]
No breaking changes
This release adds support for automated snapshots, while also adding the MKSNAP
action for doing the same remotely.
Version 0.4.4 [2020-10-03]
No breaking changes
This release adds the following actions: KEYLEN
and USET
Version 0.4.3 [2020-09-25]
No breaking changes
This release adds the following actions:
SSET
, SUPDATE
, SDEL
, DBSIZE
and FLUSHDB
Version 0.4.2 [2020-09-19]
No breaking changes
This release adds BGSAVE
for automated background saving (see #11)
Version 0.4.1 [2020-09-06]
No breaking changes
This release adds support for configuration files
Version 0.4.0 [2020-08-30]
This release introduces breaking changes
Changes:
- Actions added:
MSET
,MGET
,MUPDATE
- Terrapipe 1.0
- Improved terminal output
Fixes:
- Explicit handling for incomplete responses in
tsh
Migrating existing clients
The Terrapipe protocol was revised and promoted to 1.0. This will cause all existing client implementations to break, since the protocol has changed fundamentally. The clients have to implement the latest spec.
Version 0.3.2 [2020-08-07]
No breaking changes
The tsh
component printed the wrong version number. This has been fixed.
Version 0.3.1 [2020-08-05]
This release introduces breaking changes
This release fixes #7, #8. It also adds several under-the-hood optimizations greatly improving query performance.
Migrating existing clients
The only significant change in the protocol is the new metalayout format: #a#b#c
instead of the previously proposed a#b#c#
.
Disk storage format
The disk storage format was changed rendering existing binary data files incompatible. However, if you have any existing data - which is important, open an issue - because we'll be able to provide a tool that can help you easily migrate your existing datasets - with a one line command - so - no worries!
Version 0.3.0 [2020-07-28] (⚠ EOL)
No breaking changes.
This version enables persistence for stored data
Version 0.2.0 [2020-07-27] (⚠ EOL)
This release introduces breaking changes
This release implements the latest version of the Terrapipe protocol.
Migrating existing clients
All clients have to reimplement the Terrapipe protocol to match the latest spec.
Version 0.1.0 [2020-07-17] (⚠ EOL)
This release provides an experimental client and server implementation.