Philip O'Toole
e3fa8584c0
Store can handle snapshotting empty WAL files
9 months ago
Philip O'Toole
36ad021fdd
Don't ignore errors
10 months ago
Philip O'Toole
e0c6861ca8
Improve test
10 months ago
Philip O'Toole
e6ddf27072
Unit test sync between chunked loading and Snapshotting
10 months ago
Philip O'Toole
59e928d058
Remove superfluous blank line
10 months ago
Philip O'Toole
df76560168
Sync and report number of loads in progress
10 months ago
Philip O'Toole
b0cd5c5a0c
Add more Load stats
10 months ago
Philip O'Toole
e0f361509a
Handle aborted chunks at lowest level
10 months ago
Philip O'Toole
dd6f386450
Refactor chunk loading
10 months ago
Philip O'Toole
94898ce43e
Track loads in progress
10 months ago
Philip O'Toole
44fa23b714
Use TrailingLogs = 0 to truncate log
10 months ago
Philip O'Toole
264e9822eb
Standardize on chunk size
10 months ago
Philip O'Toole
d726a4c780
Correct and simplify logic
10 months ago
Philip O'Toole
a96112b41d
Snapshot log after one-command load
10 months ago
Philip O'Toole
1f1ebf209f
Don't delete last log
10 months ago
Philip O'Toole
97e9b14bdf
Fix race during testing
10 months ago
Philip O'Toole
8adc11fb83
Snapshot in a goroutine
10 months ago
Philip O'Toole
4b272d04f1
Better use of counters in testing
10 months ago
Philip O'Toole
c39d3472bc
Check snapshot occurs at unit level
10 months ago
Philip O'Toole
c93447e29f
Delete chunked load entries in log after loading
10 months ago
Philip O'Toole
0dcc67f6f1
Perform full snapshot after chunked load
...
This must happen as chunked loading directly changes the SQLite file,
invalidating any existing Snapshots. Also, add missing unit tests for
chunked loading.
10 months ago
Philip O'Toole
31aef91d17
Add FullNeeded() to Snapshot store
...
It may be necessary to override the FullNeeded definition, so wrap it.
10 months ago
Philip O'Toole
8211af961d
Move some VACUUM checks to the Store
10 months ago
Philip O'Toole
5c7996fd98
Support optional VACUUM on backup
10 months ago
Philip O'Toole
a0bf10b021
Move VACUUM to DB layer
10 months ago
Philip O'Toole
f26f15476e
Actually add Provider type
10 months ago
Philip O'Toole
dfb8a60c17
Move Store Provide to a distinct type
10 months ago
Philip O'Toole
3ae4255e62
Backup-related cleanups
10 months ago
Philip O'Toole
4c31c480f5
Better name
10 months ago
Philip O'Toole
5a8a2744ff
Support skipping verifying frame data
...
This halves the Compacting Scan time, and doesn't seem necessary since
rqlite is the only system writing the WAL files.
10 months ago
Philip O'Toole
ee13427c67
Tigher logging
10 months ago
Philip O'Toole
bd9e96ea02
Integrate Compacting WAL writer
10 months ago
Philip O'Toole
d53a34f765
Don't log too much
10 months ago
Philip O'Toole
ba98e88a7b
Update store.go
10 months ago
Philip O'Toole
70d5e6de34
Check join and notify addresses are resolvable
10 months ago
Philip O'Toole
66c1002c32
Remove obsolete metrics
11 months ago
Philip O'Toole
b2663de630
Rename snapshot2 -> snapshot
11 months ago
Philip O'Toole
c11d059551
Better function name
11 months ago
Philip O'Toole
10e6f1b115
Clean up snapshot FDs
11 months ago
Philip O'Toole
93566415e3
Just run bad test
11 months ago
Philip O'Toole
47d877cc25
Don't close dst file until done
11 months ago
Philip O'Toole
09712788c9
Store now builds with snapshot2
11 months ago
Philip O'Toole
6a60108f0c
Start work on Restore and Recover
...
Builds, but still needs more work. It's referencing snapshot(1).
11 months ago
Philip O'Toole
d34c35df31
No need to Store-level WAL selection
11 months ago
Philip O'Toole
5e3d481a6e
Fill out methods on SnapshotStore v2
12 months ago
Philip O'Toole
7c420407c2
WIP snapshot2
...
Close() on the Sink is done (first pass). What's needed next are all the
methods on the Store (Open, List, FullNeeded). I also need to decide
what getSnapshots should really do -- return names or a slice of meta?
12 months ago
Philip O'Toole
974f939ffe
Add an interface between Store and Snapshot Store
...
This will make it easier to swap new implementations in.
1 year ago
Philip O'Toole
be17f2a3cc
Correct variable assignment order
1 year ago
Philip O'Toole
5b4ff9b8ce
Fix test code
1 year ago
Philip O'Toole
53b432a391
More DB and Snapshot stats
1 year ago
Philip O'Toole
59e6a23b02
Clean-up logging
1 year ago
Philip O'Toole
2fff800273
Substantially shrink Test_MultiNodeClusterLargeQueuedWrites
1 year ago
Philip O'Toole
378dd4b2a1
Simpler logging of first application
1 year ago
Philip O'Toole
5ab7069aa0
Update store.go
1 year ago
Philip O'Toole
a4f7431108
Log first time a log is applied
1 year ago
Philip O'Toole
ad86a2989b
Use non-racy function for Leader ID and address
1 year ago
Philip O'Toole
225b5346f5
Add basic stats for Snapshot store
1 year ago
Philip O'Toole
1ffb44cd38
ABL
1 year ago
Philip O'Toole
854fba0e66
Better way to detect previous close
1 year ago
Philip O'Toole
d78d2f4d9d
Short circuit EOF from decompressor
...
The gzip NewReader() seems to hang sometimes when reopened with the old
reader. Therefore just record that we have closed, and never reopen it.
1 year ago
Philip O'Toole
3eb4607a74
GoDoc
1 year ago
Philip O'Toole
a49385e1d3
Add end-to-end compress-decompress test
1 year ago
Philip O'Toole
52aa68497d
Better comment on disabling multistream
1 year ago
Philip O'Toole
06a94e50bf
Raft snapshotting runs to 100%
...
Still failing later.
1 year ago
Philip O'Toole
cb384a9df2
More test tweaks
1 year ago
Philip O'Toole
2395f30ddf
Be sure we don't write footer twice
1 year ago
Philip O'Toole
57eabce899
Protect against double-close of NodeTransport
1 year ago
Philip O'Toole
365abac57c
Simple unit test of NewNodeTransport
1 year ago
Philip O'Toole
fedbdb5478
Enable GZIP snapshot compression
...
Tests are failing.
1 year ago
Philip O'Toole
195981868f
Typo
1 year ago
Philip O'Toole
6efdf9f2b9
Add Decompressor
1 year ago
Philip O'Toole
1fe81713c9
More Compressor tests
1 year ago
Philip O'Toole
55b3aaa805
Add initial version of Gzipping reader
1 year ago
Philip O'Toole
d2c5acc447
Wrap transport as passthru
1 year ago
Philip O'Toole
47e9bbde49
Fix single-node upgrade test
1 year ago
Philip O'Toole
cb989d09a2
Some changes from PR v2
1 year ago
Philip O'Toole
1a24557c19
Merge remote-tracking branch 'origin' into wal-snapshot-store-v3
1 year ago
Philip O'Toole
208fc20659
Fix more uses of random
1 year ago
Philip O'Toole
54713b63f4
Merge remote-tracking branch 'origin' into wal-snapshot-store-v3
1 year ago
Philip O'Toole
c51b8364e8
Fix import path
1 year ago
Philip O'Toole
2288cfe2a9
random only needed by store
1 year ago
Philip O'Toole
c1a8b4a13a
Use global, sync'ed random source
...
Almost unbelievably it appears tests on AppVeyor get the same value when
seeding the random generator. See https://ci.appveyor.com/project/otoolep/rqlite/builds/47853399
1 year ago
Philip O'Toole
c6e214de1a
Remember store attributes
1 year ago
Philip O'Toole
dda86be0d3
Better Store open-and-close logging
1 year ago
Philip O'Toole
c44d020c11
Only attempt to checkpoint once
1 year ago
Philip O'Toole
d6b9c5b39e
Close streams
1 year ago
Philip O'Toole
ee8b23ba94
More logging
1 year ago
Philip O'Toole
e8e0381622
Log when Store closes
1 year ago
Philip O'Toole
a6134b02bc
Instrument the FSM snapshot process
1 year ago
Philip O'Toole
76867a682f
Time FSM snapshot creation
1 year ago
Philip O'Toole
ab6ca783e1
Checkpoint database during Recovery
1 year ago
Philip O'Toole
a9e895bee2
Some cleanups
1 year ago
Philip O'Toole
bd0f78391e
Remove unneeded FSM snapshot code
1 year ago
Philip O'Toole
47476d5b58
Integrate new snapshoting with Store
...
- Snapshot
- Restore
- RecoverNode
1 year ago
Philip O'Toole
5eb4e38cbc
Add comments and errors
1 year ago
Philip O'Toole
e4d5ffb8a9
Flesh our Store check()
1 year ago
Philip O'Toole
48ea4bbfd8
Use a single header
1 year ago
Philip O'Toole
ea8dc3ed18
More test fix-up post removal of in-memory
1 year ago
Philip O'Toole
6895525482
Rename some tests
1 year ago
Philip O'Toole
aa72408bc5
Remove racy use of rand
1 year ago