21 Commits (4fcb0ee3c9653d0df57433a3dbd1af0afd01f76a)

Author SHA1 Message Date
Sayan Nandan d4dcd7b127
Upgrade deps and ignore tree test on M1 builder
For some spurious reason, the tree test is failing on the M1
builder; so we'll hack around by ignoring the test when run
on the M1 CI. Note to self: the test runs successfully on
a physical M1 machine so it is likely that this is a problem
with the runner.
2 years ago
Sayan Nandan 0ee25814ea Enable colored output on CI
This greatly improves readability of the otherwise dull logs
3 years ago
Sayan 8fb8d60b4f
Remove caching until ICC is fixed (#196)
* Remove caching until ICC is fixed

* Fix default toolchain unset error
3 years ago
Sayan Nandan 581e8e2c78 Fix bundling for `CARGO_TARGET_DIR` setups 3 years ago
Sayan Nandan 6283cc9b37 Add auto-builds on M1
Support `CARGO_TARGET_DIR` in Makefile
3 years ago
Sayan Nandan 9ff5ece6c4 Fix CI script (M1) 3 years ago
Sayan Nandan 20383c6e16 Terminate skyd instantly [skip ci] 3 years ago
Sayan 790558d2c7
Improve reliability, simplicity and recoverability of BGSAVE (#153)
* Create a new file on writing to flock-ed file

This fix is a very important one in two ways. Say we have an user A.
They go ahead and launch skyd. skyd creates a data.bin file. Now A just
deletes the data.bin file for fun. Funny enough, this never causes flock
to error!
Why? Well because the descriptor/handle is still valid and was just
unlinked from the current directory. But this might seem silly since
the user exits with a 'successfully saved notice' only to find that the
file never existed and all of their data was lost. That's bad.
There's a hidden problem in our current approach too, apart from this.
Our writing process begins by truncating the old file and then writing
to it by placing the cursor at 0. Nice, but what if this operation just
crashes. So we lost the current data AND the old data. Not good.

This commit does a better thing: it creates a new temporary file, locks
it before writing and then flushes the current data to the temporary
file. Once that succeeds, it replaces the old data.bin file with the
newly created file.

This solves both the problems mentioned here for us:
1. No more of the silly error
2. If BGSAVE crashes in between, we can be sure that at least the last
data.bin file is in proper shape and not half truncated or so.

This commit further moves the background services into their
own module(s) for easy management.

* Fix CI scripts

Fixes:
1. Our custom runner (drone/.ci.yml) was modified to kill the skyd
process once done since this pipeline is not ephemeral.
2. GHA for some reason ignores any error in the test step and proceeds
to kill the skyd process without erroring. Since GHA runners are
ephemeral, we don't need to do this manually.
3 years ago
Sayan Nandan 758baf1eb8 Use single thread for tests (drone) [skip ci] 3 years ago
Sayan Nandan 57bfeab4fa Fix testing on M1 [skip ci] 3 years ago
Sayan Nandan 7dadf4411f Persist artifacts across runs 3 years ago
Sayan Nandan 70d049e3f0 Enable rustup toolchain updates 3 years ago
Sayan Nandan 337ffbda81 Add testing for aarch64-apple-darwin 3 years ago
Sayan Nandan 43779eb08e
And now we're Skytable
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan a3398f1d07
Rebrand from TerrabaseDB to Skybase (#90)
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan 21905b582d
Use parallel jobs for faster builds 4 years ago
Sayan 5d7826976c
Use arch for build cache filename
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan 9c7f4f04af
Utilize caching for builds (#44)
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan 597a82ddf0
Use binary for building docker image (#40)
* Use binary for building docker image

In the previous workflow, we were building and testing twice: once for
the docker image and once for the test step. Now, we'll build a debug
version in the test step, then build a release version and finally copy
that into the docker image. This would heavily reduce our build times.

* Ignore specific files in target to speed up builds

* Build image only when pushed to next or tagged

* Fix build condition

Since both conditions have to evaluate to true, we'll need to use refs
while also matching against both `push` and `tag` events

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan c7b4a8396b
Stop using ubuntu image to reduce build times
We were previously using the ubuntu image and then manually installing
rust. From now on, we'll use the rust docker image instead to avoid huge
build times

Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago
Sayan Nandan 03965d6b5b
Add drone CI
Signed-off-by: Sayan Nandan <nandansayan@outlook.com>
4 years ago