59e6a23b02
Clean-up logging
1 year ago
2fff800273
Substantially shrink Test_MultiNodeClusterLargeQueuedWrites
1 year ago
378dd4b2a1
Simpler logging of first application
1 year ago
5ab7069aa0
Update store.go
1 year ago
a4f7431108
Log first time a log is applied
1 year ago
ad86a2989b
Use non-racy function for Leader ID and address
1 year ago
225b5346f5
Add basic stats for Snapshot store
1 year ago
fedbdb5478
Enable GZIP snapshot compression
...
Tests are failing.
1 year ago
d2c5acc447
Wrap transport as passthru
1 year ago
47e9bbde49
Fix single-node upgrade test
1 year ago
dda86be0d3
Better Store open-and-close logging
1 year ago
c44d020c11
Only attempt to checkpoint once
1 year ago
d6b9c5b39e
Close streams
1 year ago
ee8b23ba94
More logging
1 year ago
e8e0381622
Log when Store closes
1 year ago
a6134b02bc
Instrument the FSM snapshot process
1 year ago
76867a682f
Time FSM snapshot creation
1 year ago
ab6ca783e1
Checkpoint database during Recovery
1 year ago
a9e895bee2
Some cleanups
1 year ago
47476d5b58
Integrate new snapshoting with Store
...
- Snapshot
- Restore
- RecoverNode
1 year ago
5eb4e38cbc
Add comments and errors
1 year ago
e4d5ffb8a9
Flesh our Store check()
1 year ago
07d49961d8
Remove unused functions
1 year ago
9e31a054aa
RecoverNode now uses on-disk database
1 year ago
88ce311b37
Remove all WAL-related files when loading new DB
1 year ago
b931e387be
First pass at removing in-memory support
1 year ago
f06e547ab5
Less disk IO during load
1 year ago
f5517c0f66
Correct comment
1 year ago
9bdf048975
Better naming of temp backup file
1 year ago
b5cb889e1e
Auto-restore doesn't require ready() check
1 year ago
b7633143cc
Restore from file uses chunking
1 year ago
5ca5ce342e
Support internode LoadChunk()
1 year ago
1c8e52c5f7
Test_SingleNodeLoadBinaryFromReader passes
...
That's good, but much more extensive testing needed, and this needs to
replace the existing load path -- and then we move to 8.0.
1 year ago
93a752ae0e
Better variable names
1 year ago
1c6f691ff8
Integrate chunking - WIP
...
I need more sophisticated dechunker management.
1 year ago
13d47b24d8
Store can load from an io.Reader
...
This may be helpful for chunked-loads.
1 year ago
a40c43725f
More logic reversal
1 year ago
f81aad8fd3
Revert logic for clarity
1 year ago
21dff4a68c
Remove on-disk-startup control
...
With the move to WAL and "synchronous mode" to OFF, on-disk startup
times are very close to in-memory. There is no need for this control
anymore and it complicates the start-up code.
1 year ago
f7ced46377
Count number of DB stats errors
1 year ago
36fa7f2afd
Detect DB stats error and log
1 year ago
7ce2c6b99a
Fix comment
1 year ago
7432bceca3
Since we're passed a ReadCloser, close it
1 year ago
4342650c33
Fix RecoverNode
1 year ago
ad73779285
Periodically record actually applied index
...
This ensures the system doesn't apply uncommitted log entries to the
FSM, if it runs the in-memory startup. There have been no reports of
this in the field -- it is an edge case -- but this removes any chance
of it.
1 year ago
abfe056b4c
Remove any -wal and -shm files on shutdown
...
If in WAL mode, open-and-close again to remove the -wal file. This is not strictly necessary, since any on-disk database files will be removed when rqlite next starts, but it leaves the directory containing the database file in a cleaner state.
1 year ago
b2f28d1181
Better WAL control
1 year ago
2f7672d897
Enable WAL mode
1 year ago
585f9c9c05
Use Snapshot decoder in Store
1 year ago
c87931c45c
Factor Snapshot creation into own module
1 year ago
3f462eecfa
Remove all on-disk SQLite files on startup
1 year ago
7e1b8d12d7
Add full WAL support to DB layer
1 year ago
eed215c664
Only 1 Snapshot needs to be retained
1 year ago
b9253e6b41
Improve logging in Store
1 year ago
4a7ac2850f
rqlite doesn't (yet) support WAL mode SQLite data
1 year ago
92bc513242
Move FSMSnapshot to own source file
1 year ago
7badba4a7a
Try fast paths
1 year ago
76ba50ca86
Log last (if any) WaitForLeader error
1 year ago
ef403cdcc6
Fix up WaitForLeader
...
Add dedicated error codes too.
1 year ago
c05e091cd1
Add WaitForRemoval
1 year ago
c23d3e60b1
Reuse existing consistency level
1 year ago
18b76ebd4f
go fmt
1 year ago
2f8d4e95e0
Update store.go
1 year ago
eea101a1d5
Fix duplicate import
1 year ago
f7cdbdea68
Handle Command_COMMAND_TYPE_EXECUTE_QUERY in Store
...
Works! Store.Request() needs unit testing however.
1 year ago
b2b0b188e9
Factor out functions
1 year ago
89f472743b
Start adding Store.Request()
1 year ago
7680483793
Tighten up remove
1 year ago
e64d78b809
Support removing self from cluster on shutdown
1 year ago
02b6f7df61
selfLeaderChange is not just about restores
1 year ago
159b127101
Clean up comments
1 year ago
0c6be22478
Close auto-restore file
1 year ago
1fa7cf73a7
Log any failure to delete auto-restore file
1 year ago
1941c947c7
Fix comment
1 year ago
466c757dcf
Add long-missing init call to Store stats
1 year ago
ee55a35c03
SetRestorePath unit testing
1 year ago
e7919a906f
Avoid the race during testing
1 year ago
31467482a3
Store-level multi-node auto-restore testing
...
This test ensures that auto-restoring works as expected when
Bootstrapping is enabled.
1 year ago
bf1e1cb1cd
Add first tested version of Store auto-restore
1 year ago
5c2e8e7ad6
Add channel which gets signal on becoming Leader
1 year ago
833d9d3a6f
Perform older checks first
1 year ago
5e3f294dfa
Only check for a ready store if making changes
1 year ago
2ce3c5f34c
Check Store is ready in more places
1 year ago
57ca03014b
More integration and testing of Store readiness
1 year ago
716f84ff28
Store supports registering "ready" channels
1 year ago
372e4dd4a9
Store implements simpler Provide() interface
1 year ago
5fcc222f73
Better Store comment
1 year ago
ae97060f2c
Implement the DataProvider interface on the Store
1 year ago
519df067cc
Check for more possible errors with peers
...
https://github.com/rqlite/rqlite/issues/1217
1 year ago
35df643a0b
Remove unneeded type assertions
1 year ago
1794724ef0
Remove unused code and fix error handling
1 year ago
93b3308502
Skip any files that disappear during walking
1 year ago
ff001bc5a8
Migrate to a Protos data model for Notify Requests
2 years ago
a782513a11
Migrate to a join-request proto
2 years ago
fdf5fce053
Update store.go
2 years ago
2730fa06e5
Empty config means not a voter
2 years ago
5636febf9a
Add IsVoter() and add to status/ output
2 years ago
defde21281
Merge remote-tracking branch 'origin' into ignore-join-explicit-if-state
2 years ago
90b4dc0745
Silently ignore unchanged node self-joins
2 years ago
54d61c7866
Update store.go
2 years ago
a2899a1cdd
Support stepping down before shutting down
2 years ago
0702e39ace
Add ResetStats
2 years ago
4da5b6322f
Log message could be confusing with remote remove
2 years ago
98575d727b
Start adding remote remove node
...
More testing, including unit testing, required.
2 years ago
bfc35eaf7c
Simplify reap command line flags ( #1118 )
...
Simplify reap command line flags
2 years ago
220424c7d1
Simple refactor, better Go style
2 years ago
07620cab41
Reap nodes ( #1114 )
...
Support automatic reaping of nodes
2 years ago
e158ae8a72
Start HTTP server ASAP
2 years ago
0e8e266f96
Merge pull request #1090 from rqlite/fix-remote-load-errors
...
Fix remote-load error handling
2 years ago
893fbb0249
More store-level load operation logging
2 years ago
65cd556549
Fix remote-load error handling
...
Error from remote node was being dropped.
2 years ago
5f33bffa0f
More logging and resolution checks
2 years ago
b0f17b86c7
Notified node checks address resolution
...
If a non-resolvable address is passed to the Store, and then Store then
adds that to the config, it can result in a cluster that seems to have
insufficient members (since the remote node can't be contacted). This
may trigger a new leader election.
2 years ago
47d79c859a
Log database backup time
2 years ago
9fc6ea16e9
Even better defer sequence
2 years ago
555883d1a2
Correct order of deferred calls
2 years ago
54aec1ca33
Tighten up backup code
2 years ago
31bba4c577
Move to Backup proto model
2 years ago
dc2b9f5b6f
Ignore freshness when serving queries on Leader
...
It doesn't make sense to check the last leader-contact time when the
node itself is the leader. Any read, when served by the leader, is going
to be within the freshness bound.
Fixes https://github.com/rqlite/rqlite/issues/1048
2 years ago
89b132ac26
Fix Store-level unit tests
2 years ago
b03ed99eda
Update comment
2 years ago
304124952c
Add Store level changes
...
Needs unit testing, and HTTP testing won't compile.
2 years ago
2729a17279
Move some functions to standalone form
2 years ago
9783ef24b1
Switch to error on self-join attempt
3 years ago
a524ed4979
Ignore join request if node ID is same
3 years ago
e4fed0cee3
Add bootstrap-expect ( #974 )
...
Add cluster-boostrap
3 years ago
ad7986630d
Better command-line flags
3 years ago
ce099087ee
Control whether bbolt sync freelist to disk
...
When true, skips syncing freelist to disk. This improves the database write performance under normal operation, but requires a full database re-sync during recovery.# Please enter the commit message for your changes.
3 years ago
490f654f36
Move to etcd's fork of BoltDB
...
bboltdb is the version the Consul now runs.
3 years ago
4aea326959
Consul and etcd autoclustering ( #957 )
...
This PR introduces new node-discovery integration with Consul and etcd. By using one of those systems with rqlite, automatic clustering of rqlite is much easier.
3 years ago
ca556044e3
Merge branch 'master' of github.com:rqlite/rqlite into refactor-main-bootstrap
3 years ago
2ffb29394f
Fix typo
3 years ago
f5c03d4b60
Try simple refactor of bootstrap
3 years ago
769557db7e
Better variable name
3 years ago
3bfcbffc37
Fix typos and Grammar issues
3 years ago
33bee34f89
More Store restart testing
3 years ago
d74c05c2db
Support disabling in-memory init of on-disk
3 years ago
003301aee6
Minor fixes to log messages ( #920 )
...
Minor fixes to log messages
3 years ago
0b621cbec8
More logging tweaks
3 years ago
5ce51bc2ba
Better comments
3 years ago
05534cc94c
Don't use Windows file paths as IDs
3 years ago
f9f35bf95c
Dump file
3 years ago
7093cb25d3
Fix race in test
3 years ago
9fb114c07f
Recover test after snapshot
3 years ago
519a91e635
Actual recovery node test passes on Linux
3 years ago
1b158d09fb
Simple unit test of node-recovery
3 years ago
1abd3007ca
Rename peers.json after successful recovery
3 years ago
cc1de72796
Ensure raft/ exists inside data directory
3 years ago
abccc6ced3
Better function name
3 years ago
d9ad3c080a
Add untested recover-node functionality
3 years ago
32239b5053
Move new function to earlier per convention
3 years ago
4ab72a8194
Refactor to make recover-cluster code easier
3 years ago
5dd4aeed1e
More Snapshot metrics
3 years ago
38eef8ee82
Merge pull request #890 from rqlite/more-abl
...
ABL
3 years ago
dd71c07c73
ABL
3 years ago
0e34dc49ca
Add BoltDB stats
3 years ago
d6be09ba65
Update Snapshot comment
3 years ago
5d48a18950
Check if slice is empty in addition to being nil
3 years ago
bb002bb81b
Block query transactions during serialize
3 years ago
a6da8d474f
Ensure (any) directory for on-disk-path exists
3 years ago
1f33dbb2cb
Fix up Store-open logging
3 years ago
105a894fe5
Support explicit SQLite path at Store level
3 years ago
a6346ba9d4
Fix unit test -- start muxes
3 years ago
dd692ecf4f
Add gauge-like metric for Snapshot times
3 years ago
9d87a15194
Small simplication
3 years ago
43e60a0469
Perform leader check first for STRONG query
...
Don't wait until Raft layer returns it. There is a small chance of a
false leadership detection, but the cluster would have to be in flux
anyway for that to be the case. Execute does this already.
3 years ago
fd98114839
HTTP-level testing passes
3 years ago
3580e341a8
Store-level testing passes
3 years ago
deb4dabf2e
Thread through foreign key constraint controls
3 years ago
0536bc3c1c
ABL
3 years ago
74d0780a5f
Merge pull request #845 from rqlite/sqlite-options
...
Add SQLite compile-time options to status output
3 years ago
39d0f2ad4a
Merge branch 'master' of github.com:rqlite/rqlite into sqlite-options
3 years ago
fc76327b46
Use FSM index in tests
3 years ago
b8d72f3fc4
New DB and FSM indexes to track state
3 years ago
ea723ee830
Ping databases on open to ensure full availability
3 years ago
696bac655f
Merge branch 'master' of github.com:rqlite/rqlite
3 years ago
f03900b282
Log type of database on start-up
3 years ago
b83424484d
Enhance SQLite database status
3 years ago
6edff5ba24
Deal with SQLite connection "caching"?
...
https://github.com/mattn/go-sqlite3/issues/966
3 years ago
6dc9583dab
Implement concurrent reads and writes
...
Disk doesn't support it yet, as it has not moved to WAL mode. Perhaps
wait for next release to do that.
3 years ago
e0da96dd55
Remove support for specifying SQLite DSN
...
This is a breaking change. However this feature hasn't been tested and
allows end-users to break the system too easily. Low-level control over
the SQLite database is better done with PRAGMA commands where possible.
3 years ago
8693694d14
Add current SQLite journal mode to status output
3 years ago
503825dbaf
Remove Abort functionality
...
No longer relevant since each Exec and Query gets its own connection.
3 years ago
561bb47a14
Port DB layer to sql/database
...
Empty statements now cause panic.
3 years ago
5f5f7dbbea
Sync use of DB connection
...
Possible, albeit simple, fix for https://github.com/rqlite/rqlite/issues/830
3 years ago
67b0368288
It's create in memory, not open
3 years ago
0a55f0bed1
It's create, not open
3 years ago
01e2403015
Count various Join scenarios
3 years ago
4591e2df35
Don't ignore -join even if previous state exists
3 years ago
f0417f2241
Correct typo in comment
3 years ago
58b567c27a
Only check non-voting nodes by default
3 years ago
4fd5555c41
ABL
3 years ago
f939d33fe5
Remove Metadata concept from system
...
System testing passes.
3 years ago
b2bcdb584d
Tweak Store API
3 years ago
b579e5c59f
Shut down Raft first
4 years ago
1dc156b965
Add total Raft directory size to Store stats
4 years ago
fd4b76a419
Convert Raft stats to numbers where possible
4 years ago
125074ab6e
Only shutdown BoltDB when Raft is done with it
4 years ago
3ff13cda74
More system testing
4 years ago
51c745ea23
More in-mem and on-disk system test
4 years ago
Philip O'Toole
chermehdi