1
0
Fork 0

526 Commits (c9e97ac546b150a3d6f60b77461b8ae1f8e6553b)

Author SHA1 Message Date
Philip O'Toole 43e60a0469 Perform leader check first for STRONG query
Don't wait until Raft layer returns it. There is a small chance of a
false leadership detection, but the cluster would have to be in flux
anyway for that to be the case. Execute does this already.
3 years ago
Philip O'Toole 52ef9836ed Minor store-level test cleanup 3 years ago
Philip O'Toole fd98114839 HTTP-level testing passes 3 years ago
Philip O'Toole 3580e341a8 Store-level testing passes 3 years ago
Philip O'Toole 916342395e Add Store-level FK constraint test 3 years ago
Philip O'Toole a8d02bc807 Better JSON rendering of DB conf object 3 years ago
Philip O'Toole deb4dabf2e Thread through foreign key constraint controls 3 years ago
Philip O'Toole 9ed32972b1 Start adding FK option support 3 years ago
Philip O'Toole 0536bc3c1c ABL 3 years ago
Philip O'Toole 74d0780a5f
Merge pull request #845 from rqlite/sqlite-options
Add SQLite compile-time options to status output
3 years ago
Philip O'Toole 39d0f2ad4a Merge branch 'master' of github.com:rqlite/rqlite into sqlite-options 3 years ago
Philip O'Toole fc76327b46 Use FSM index in tests 3 years ago
Philip O'Toole b8d72f3fc4 New DB and FSM indexes to track state 3 years ago
Philip O'Toole ea723ee830 Ping databases on open to ensure full availability 3 years ago
Philip O'Toole 696bac655f Merge branch 'master' of github.com:rqlite/rqlite 3 years ago
Philip O'Toole f03900b282 Log type of database on start-up 3 years ago
Philip O'Toole b83424484d Enhance SQLite database status 3 years ago
Philip O'Toole 6edff5ba24 Deal with SQLite connection "caching"?
https://github.com/mattn/go-sqlite3/issues/966
3 years ago
Philip O'Toole 6dc9583dab Implement concurrent reads and writes
Disk doesn't support it yet, as it has not moved to WAL mode. Perhaps
wait for next release to do that.
3 years ago
Philip O'Toole e0da96dd55 Remove support for specifying SQLite DSN
This is a breaking change. However this feature hasn't been tested and
allows end-users to break the system too easily. Low-level control over
the SQLite database is better done with PRAGMA commands where possible.
3 years ago
Philip O'Toole 8693694d14 Add current SQLite journal mode to status output 3 years ago
Philip O'Toole 503825dbaf Remove Abort functionality
No longer relevant since each Exec and Query gets its own connection.
3 years ago
Philip O'Toole 561bb47a14 Port DB layer to sql/database
Empty statements now cause panic.
3 years ago
Philip O'Toole 5f5f7dbbea Sync use of DB connection
Possible, albeit simple, fix for https://github.com/rqlite/rqlite/issues/830
3 years ago
Philip O'Toole 67b0368288 It's create in memory, not open 3 years ago
Philip O'Toole 0a55f0bed1 It's create, not open 3 years ago
Philip O'Toole 01e2403015 Count various Join scenarios 3 years ago
Philip O'Toole 4591e2df35 Don't ignore -join even if previous state exists 3 years ago
Philip O'Toole f0417f2241 Correct typo in comment 3 years ago
Philip O'Toole 58b567c27a Only check non-voting nodes by default 3 years ago
Philip O'Toole 4fd5555c41 ABL 3 years ago
Philip O'Toole f939d33fe5 Remove Metadata concept from system
System testing passes.
3 years ago
Philip O'Toole b2bcdb584d Tweak Store API 3 years ago
Philip O'Toole b579e5c59f Shut down Raft first 4 years ago
Philip O'Toole 1dc156b965 Add total Raft directory size to Store stats 4 years ago
Philip O'Toole fd4b76a419 Convert Raft stats to numbers where possible 4 years ago
Philip O'Toole 125074ab6e Only shutdown BoltDB when Raft is done with it 4 years ago
Philip O'Toole 3ff13cda74 More system testing 4 years ago
Philip O'Toole 51c745ea23 More in-mem and on-disk system test 4 years ago
Philip O'Toole ae16a65d8d Handle snapshots with database data
This shouldn't happen normally, but technically could.
4 years ago
Philip O'Toole f8e4795d44 Merge branch 'master' into test-snap-restore-more 4 years ago
Philip O'Toole c2f40500a2 Close Bolt on Store close 4 years ago
Philip O'Toole e4c60bed7a Merge branch 'master' into test-snap-restore-more 4 years ago
Philip O'Toole ca1f8e1752 Whitebox testing of Noops on Store 4 years ago
Philip O'Toole aa2fec4a1f More useful store test helper functions 4 years ago
Philip O'Toole 0b84667b45 Fix on-disk create in Apply 4 years ago
Philip O'Toole a8559b49e8
Merge pull request #754 from rqlite/noop-store-test
Support Noop commands in Raft Log
4 years ago
Philip O'Toole 9b8016e079 Simple Noop unit test 4 years ago
Philip O'Toole 824dab5536 Doesn't work because Store can't be re-opened
Due to network issues.
4 years ago
Philip O'Toole d31105e0f0
Add missing return 4 years ago
Philip O'Toole ed080fe9d0 Close database before Restore 4 years ago
Philip O'Toole 9c53b0f8b7 Consolidate database creation functions 4 years ago
Philip O'Toole c782b43f2d
Build on-disk databases in-memory first
With this change, rqlite nodes running in "on-disk" mode build the database in memory first, and move it to the disk just before the Raft system starts. This means that on-disk nodes now initialize almost as quickly as in-memory nodes.
4 years ago
Philip O'Toole 4551bd30da Add LastCommandIndex 4 years ago
Philip O'Toole 33aa4522cf Better timing from first to last log 4 years ago
Philip O'Toole a4185172da Time snapshot, restore, and startup times 4 years ago
Philip O'Toole 80e0db9f52 Make logic easier to read 4 years ago
Philip O'Toole 9ea54b608f Use DeserializeInMemoryWithDSN for in-mem restore 4 years ago
Philip O'Toole fa43fd4dc0 Enhance in-mem snapshot unit test 4 years ago
Philip O'Toole ef1f0f1d57 Sync queries and restores 4 years ago
Philip O'Toole 0f1352fb01 Remove duplicate pre-restoration Close() 4 years ago
Philip O'Toole ce7a3a6912 Merge branch 'master' into no-disk-during-restore 4 years ago
Philip O'Toole 0b3a41bd3f
Clearer comment 4 years ago
Philip O'Toole 535d690d7c Tigthen up Store-level locking
Previous locking didn't look quite correct, and with the advent of the
Serialize functionality, it needs to be more precise.
4 years ago
Philip O'Toole 0d0bb3f44a Close database before restoration
This change also adds better logging, so the restoration process can be
better observed.
4 years ago
Philip O'Toole eb01e8dc9d Skip temp file when restoring in-memory database
This saves a read and write to disk, the size of the SQLite database.
4 years ago
Philip O'Toole db64f4c923 Don't use temp file during database snapshot
Snapshotting the database does not now need to write to disk, and copy
the SQLite database straight to RAM, for handoff to the Raft subsystem.
4 years ago
Philip O'Toole 93a78f09a0 Add call to Serialize function 4 years ago
Philip O'Toole d560356d8d Fix spelling mistakes 4 years ago
Philip O'Toole 8576ed3cfd Remove unused byte slice address 4 years ago
Philip O'Toole bb31aef20a ABL 4 years ago
Philip O'Toole 5295a0e6b1 Trivial cleanups 4 years ago
Philip O'Toole 9b0852e711 Implement policy over trailing logs
Don't allow control over trailing logs directly, just implement a policy
that scales the number by 1.25 relative to Snapshot threshold. Without
doing this it may be that setting the snapshot threshold lower doesn't
actually affect SQLite start up times, due to log application. Perhaps
this really needs exposure at the command line, but the command line is
starting to get complicated.
4 years ago
Philip O'Toole 2e6a93faa3 Convenience function to get both indexes at once 4 years ago
Philip O'Toole 8e11534139 Support getting log first and last index 4 years ago
Philip O'Toole 8cb88d7744 Count number of legacy commands unmarshaled 4 years ago
Philip O'Toole 4f0c3092b3 Better command decoding panic message 4 years ago
Philip O'Toole 7780a0987b Simplify variable declaration 4 years ago
Philip O'Toole e52b4c58c3 Factor out binary value functions 4 years ago
Philip O'Toole 011bc00784
Add SQLite database page-centric size to status output 4 years ago
Philip O'Toole 6d77d904e7
Compress SQLite database in Raft snapshot
Before Raft truncates the log, it needs to retrieve a snapshot of the SQLite database to represent the log entries that are removed during that truncation. Raft then writes that snapshot to disk. To reduce both disk IO and disk space this change compresses that SQLite snapshot before it is written to disk, and decompresses it during a restore operation.

This change also deals with older SQLite snapshots, which were not compressed.
4 years ago
Philip O'Toole d5322a1909
Ignore join request if node already part of cluster (#711)
rqlite used to work like this, but suffered a regression due to a change in how Hashicorp Raft worked. The manner it changed in was not public, so relying on it was always fragile.
4 years ago
Philip O'Toole 8c2298941a ABL 4 years ago
Philip O'Toole 59360b4792 ABL 4 years ago
Philip O'Toole 6575de779d
Use Protobuf for encoding Raft Log commands
This PR changes Raft Log Entry encoding from JSON to Protobuf. Furthermore, larger Raft commands (which can result from batching SQL statements, or individually long SQL statements) are compressed before encoding.

This primary reason for this change is to reduce IO load since that is one of the largest performance bottlenecks. It will also reduce internode traffic.

Legacy JSON-encoded commands are still handled by this code, so this change is backwards-compatible with previous releases in the v5 series.
4 years ago
Philip O'Toole b4867713e4 Better variable name, Query -> SQL 4 years ago
Philip O'Toole b6ebdf3ab1 Support setting Leader Lease timeout 4 years ago
Philip O'Toole c4b4cf2a51 Perform more testing of queries 4 years ago
Philip O'Toole 8a4a5c4901
Improve comments re opening of Store 4 years ago
Philip O'Toole f961bb4e44 Fix typos and comments 4 years ago
Philip O'Toole 1480d8898f Handle Raft commands missing Parameterized values
This allows this code to run with Raft logs from previous versions.
4 years ago
Philip O'Toole ac862e4091 Add system test for parameterized statements
Fix some bugs along the way too.
4 years ago
Philip O'Toole 212d4e5c86 Store and Raft layer support parameterized queires 4 years ago
Philip O'Toole d840adc2a9 Convert Store to use new DB Statements 4 years ago
Philip O'Toole f81ef9024e Add Raft log size on disk to status 4 years ago
Philip O'Toole e6e8ef36d2 Fix Snapshot test
If the snapshot count didn't change, the test didn't actually fail.
4 years ago
Philip O'Toole c4a89ede8d Add helper function for Raft log size 5 years ago
Philip O'Toole 487c0e3f8c Set Snapshot Interval to non-zero value
Snapshotting was *never* functional due to the zero interval value.
5 years ago
Philip O'Toole 9c4758ade7 Add comment 5 years ago
Philip O'Toole a1d0607fcd Sleep 1s so freshness interval passes 5 years ago
wangfenjin 3de2e675ba add strong level test
Change-Id: Ib91ffb22807993ffb7743b7fe7d0b41a62d0e696
5 years ago
wangfenjin 9c4151c37b move comment down
Change-Id: I2f8c435adab5e1c46da2e6b91205ad4df850decc
5 years ago
wangfenjin 9c295ca704 check level before freshness
Change-Id: I6d736ad09eac9f003f087adaa9c1f51691fd5c69
5 years ago
Philip O'Toole 536cac4fdf Allow log level control for Raft module
Default log level is INFO.
5 years ago
Philip O'Toole abb7772105 Support leader freshness with None consistency
This change allows the read request to specify the maximum time the node
receiving the request may have last heard from the cluster leader. It
only applies to a read consistency selection of None.
5 years ago
Philip O'Toole 81f5f6faf1 Add support for non-voting nodes
A non-voting node doesn't participate in Raft consensus, but does
subscribe to the committed log entries originating with the leader.
This means a non-voting node keeps up-to-date with the state machine,
without impacting write-latency. These non-voting nodes can provide
read scalability for the cluster.
5 years ago
Philip O'Toole 9d595e2ead ABL 5 years ago
Philip O'Toole e0cf01cefc Protect against possible nil pointer 5 years ago
Philip O'Toole a23af88235 Write backup and dump files to a writer
This avoids making an extra copy in RAM. Port of original PR at
https://github.com/rqlite/rqlite/pull/461/.
5 years ago
Philip O'Toole f57ace7da2 Broadcast Store meta via standard consensus
With this change the cluster metadata (arbitrary key-value data associated with each node) is now broadcast across the cluster using the standard consensus mechanism. Specifically the use case for this metadata is to allow all nodes know the HTTP API address of all other nodes, for the purpose of redirecting requests to the leader.

This change removed the need for multiplexing two logical connections
over the single Raft TCP connection, which greatly simplifies the
networking code generally.

Original PR https://github.com/rqlite/rqlite/pull/434
5 years ago
Philip O'Toole e376812907 Add method to Store that returns leader ID
With the advent of Raft IDs, this distinction matters.
5 years ago
Philip O'Toole 8a0e1ce8f0 Move to Hashicorp Raft v1 5 years ago
Philip O'Toole 7352c376c7
Don't ignore Join error, actually return it
Clearly a typo.
5 years ago
Philip O'Toole 085d2d23d8 ABL 5 years ago
Philip O'Toole 0b8cd8536f Remove apparently duplicated code 5 years ago
Philip O'Toole 0e69be3949 Store layer supports generating SQL format backups
Port PR453.
5 years ago
Philip O'Toole 0471877f26 Abort transaction if load operation fails 5 years ago
Philip O'Toole ccca1ecfa9 Fix DSN key in Store status output 5 years ago
Philip O'Toole d47808a26b Add expvar stats to Store 5 years ago
Philip O'Toole 2e91858e1e Allow Raft election timeout to be set 6 years ago
Philip O'Toole 6937a377ae Allow control of shutdown-on-remove
Defaults to leaving Raft run if the node is removed.
6 years ago
Philip O'Toole efec4d4e42 Return redirect if remove on follower
Fixes issue #391.
7 years ago
Philip O'Toole b5b2cb21db Log Store directory path on startup 7 years ago
Philip O Toole 6bbdc88c1d ABL
Always be linting.
7 years ago
Philip O'Toole 2a1b6c65c3 New QueryRequest and ExecuteRequest types
The Query() and Execute() functions on the Store now take a complex type
that encapsulates the statements, and associated parameters. This will
make it easier to add more control parameters when making requests.
7 years ago
Philip O'Toole 303be73dcb Add missing Store params to diagnostics 7 years ago
Philip O Toole a59e23cae0 Comment some unit tests 8 years ago
Philip O'Toole 8f2d8e8aa0 Copy JSON peers decode functionality from Raft
The main rqlite code needs this logic to check if the Discovery Service
should even be accessed. This code makes assumptions about how the Raft
module works, which is subject to change. The new unit tests should
capture that.
8 years ago
Philip O'Toole 2617962f5f Typo 8 years ago
WanliTian 4cb5035172 remove store.readPeersJSON 8 years ago
Philip O Toole aa5c946a40 Linting 8 years ago
Philip O'Toole 851f486ff8 Make waiting for logs optional 8 years ago
Philip O'Toole 059ab0757b Allow Store to wait for initial logs to be applied
Better fix for issue #260.
8 years ago
Philip O Toole 4b6c936d53 Fix comment typos 8 years ago
Philip O Toole ae0d119d83 Allow Raft Apply timeout to be configurable 8 years ago
Philip O Toole 5ed6ef0ae8 Covert dumps to queries in HTTP service
There is no need for special logic at the Store level.
8 years ago
Philip O Toole e6586f41f2 Use single load for dump processing
Writing a sufficiently sophisicated parser is too much work, and unlikely to be successful soon. Instead this change simply loads the entire dump as one command, and allows the underlying SQLite support to parse it correctly. This will definitely work, but since the load goes over Raft, it may hit limits with regards to network transfer sizes. Right now any limitations in that area are unknown.

Therefore this functionality remains somewhat experimental.
8 years ago
Philip O Toole 098578bda8 Clarify guarantees of some functions 8 years ago
Philip O Toole c2ab24378c Lint fixes 8 years ago
Philip O Toole 2fb99e5b20 Use Store logger everywhere in Store 8 years ago
Tycho Andersen dc2128d02a add a raw snapshotting API
This allows us to grab a raw snapshot of the databse.

v2: s/RawDBSnapshot/Database, remove reference to raft from the comment.
v3: add a leader arg to Database to do a leader check
v4: fix doc string.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8 years ago
Tycho Andersen 326ca757d7 add an API for using custom raft.PeerStore/log.Logger
This is useful in case the server needs to store other metadata (e.g. auth
data) along side the peer list.

The logging bit is handy in case something has its own logging framework
that it wants to use.

Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8 years ago
Philip O'Toole 7ac3450ab2 Suport observer deregistration 8 years ago
Tycho Andersen aa47089a32 expose raft's observer API via store
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8 years ago
Tycho Andersen 4cf094c6a4 store: use ErrNotLeader everywhere
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
8 years ago
Philip O'Toole 4242a641b1 Fix typo in comment 8 years ago
Philip O'Toole da21606644 Move chinook package to idiomatic testdata dir
https://github.com/golang/go/issues/14715
8 years ago
Philip O'Toole 634f55211c Unit test with Chinook database
https://chinookdatabase.codeplex.com/
8 years ago
Philip O'Toole eda120ffaa Correct unit tests 8 years ago
Philip O'Toole c00f1b460b Accept BEGIN and COMMIT on load() 8 years ago
Philip O'Toole 39ebf50cb7 Remove explicit control of FK constraints
It's not clear whether it's the right thing to do, and FK constraints
can still be controlled via PRAGMA commands.
8 years ago
Philip O'Toole 9ffa309f69 Stop database-level control of FK during load 8 years ago
Philip O'Toole ed304b7840 Remove useless comment 8 years ago
Philip O'Toole f87ee412e2 Correct unit tests
'PRAGMA foreign_keys=OFF' is no longer filtered.
8 years ago
Philip O'Toole 36368d6786 Start removing batch load 8 years ago
Philip O'Toole 68793cfcbb Use a transaction for loaded batch
Bump the batch size to 1000 too.
8 years ago
Philip O Toole b95af88367 Test SQL loading with blank lines 8 years ago
Philip O Toole f44c08a530 Skip loading blank lines 8 years ago
Philip O Toole 9c180d8f43 SQL parser now strips newlines 8 years ago
Philip O Toole 1f5cfc65d8 Trim leading and trailing newlines 8 years ago
Philip O Toole 84f3be9c10 Add missing package 8 years ago
Philip O Toole b87de293ce Move \n ; trimming to SQL line parser 8 years ago
Philip O Toole 6ee9cdbbdf Start integration of SQL line parser 8 years ago
Philip O Toole 3726d1bb20 Add DB conf to diagnostics 8 years ago
Philip O Toole cb7686fa46 Diagnostics show actual FK constraint state 8 years ago
Philip O Toole 2daac8704b Support batching during load 8 years ago
Philip O Toole 38f9b6ff20 Test loading no commands 8 years ago
Philip O Toole ab743b1ead Test loading an empty file 8 years ago
Philip O Toole ccd48161f3 Unit test Store Load() 8 years ago
Philip O Toole 31560160a1 Check FK state before load 8 years ago
Philip O Toole 4c6ee1ad39 Return error code during load if necessary 8 years ago
Philip O Toole 6007790ae4 Break loading when done 8 years ago
Philip O Toole 3286bf064d Start adding load-from-dump 8 years ago
Philip O'Toole 729221bf57 join endpoint returns HTTP 301 if necessary 8 years ago
Philip O Toole 95cb4baf2a More cyclo changes 8 years ago
Philip O Toole 7bdedf8cd9 Reduce cyclo complexity of opening store 8 years ago
Philip O'Toole c40fff4d46 Support controlling FK constraints at startup
This was possible previously, but would need to be set everytime on
startup via the API. This change allows it to set at startup AND enables
foreign constraint checking by default.
8 years ago
Philip O'Toole 7dd164915b Check database errors at store level 8 years ago
Philip O'Toole 0b13a1321d Simplify error handling for Update Peers 8 years ago
Philip O Toole aa467cffcd Fix misspelling in comment 8 years ago
Philip O'Toole 2982058a81 Support setting Raft heartbeat timeout 8 years ago
Philip O'Toole 8a05236768 Support configurable snapshot thresholds 8 years ago
Benjamen Keroack b67e56196e add cluster state and leader functions to store 8 years ago
Philip O'Toole 83026e4612 Add Peers to status output 8 years ago
Philip O'Toole 47539cf8e1 Add RemoveNode() to Store 8 years ago
Philip O'Toole ef18cd5b17 Expose Raft Peers() on Store 8 years ago
Philip O Toole bf2047b45b 'go lint' fixes 8 years ago
Philip O'Toole 8c0fee4b06 Include Raft dir in status 9 years ago
Philip O'Toole 5380c4d7e4 Map Raft "not leader" error to Store error (#112) 9 years ago
Philip O'Toole c04ff92d7a Fill our API peers lookups (#110) 9 years ago
Philip O'Toole fac48629bc Start initial integration of mux (#109)
Integrate TCP mux with cluster and store

This change allows any node, including followers, to use the Raft log to make changes to a cluster-wide state.
9 years ago
Philip O Toole 5aefda9983 Move to new rqlite org 9 years ago
Philip O'Toole e8c92e088c Don't join unless actually necessary 9 years ago
Philip O'Toole a02f44f089 Remove cluster-like code from Store 9 years ago
Philip O'Toole 9164550aab Remove debug print 9 years ago
Philip O'Toole bdeda47774 Initial implementation of Meta server
This should really be a distinct cluster package.
9 years ago
Philip O'Toole 0a3454a675 Methods on store to set and get API peers 9 years ago
Philip O'Toole 0bffa38319 Start implementing cluster meta comms 9 years ago
Philip O'Toole 16e3125149 Add support for cluster meta
This change required that the cluster meta become part of the Raft
snapshot.
9 years ago
Philip O'Toole 99ebaf9767 Add support for peersSub 9 years ago
Philip O'Toole ed56f77714 Add hierarchy to Raft commands 9 years ago
Philip O'Toole 099c7c18ac Enable TCP muxing layer for Raft commns 9 years ago
Philip O'Toole ef554e33a0 Update sqlite3 status output 9 years ago
Philip O'Toole a11b5fe2fe Unit test restoring in-memory databases 9 years ago
Philip O'Toole 91a552b7ec Further store-layer updates 9 years ago
Philip O'Toole 31cda32a38 'go fmt' fixes 9 years ago
Philip O'Toole 1ea4edd515 Store layer adapted to new DB layer 9 years ago
Philip O Toole c07c11f134 Fix trivial typo 9 years ago
Philip O Toole cafbc0722b 'go fmt' fixes 9 years ago
Philip O Toole 940c872dfd Explicitly test file and in-mem 9 years ago
Philip O'Toole b26bc3444c Unit test snapshot and restore 9 years ago
Philip O'Toole 7fd7a9a385 Finish multi-node cluster testing 9 years ago
Philip O'Toole 5e688bab93 Clarify it's localhost 9 years ago
Philip O'Toole cc3c75d9bf Re-enable multi-node testing on CirceCI 9 years ago
Philip O'Toole 5bd3ab574d Idiomatic network layer function names 9 years ago
Philip O'Toole 9693d7dc95 Obey 'timings' flag at database level 9 years ago
Philip O Toole 5f5a666a06 Move to "weak" and "strong" for consistency levels 9 years ago
Philip O Toole 1a16588450 Fix 'timings' JSON tag 9 years ago
Philip O Toole ac92530a6d Add sqlite file size to status 9 years ago
Philip O Toole db1fb33fd3 Allow timing control
Fixes issue #75.
9 years ago
Philip O Toole 8f674219eb Re-enable timings 9 years ago
Philip O Toole 1649b83d14 Fix unexpected command 'typ' 9 years ago
Philip O Toole 11c18cd769 Service and store support consistency levels
Fix issue #5.
9 years ago
Philip O Toole 20a05fd0a1 Refactor Raft command to support queries 9 years ago
Philip O'Toole 7b628686bf Add SQLite version to diagnostics 9 years ago
Philip O'Toole 62c17291b4 More diagnostic information 9 years ago
Philip O Toole e7e212411a Ensure Raft dir exists 9 years ago
Philip O'Toole 4524d014eb Leader redirection almost complete 9 years ago
Philip O'Toole 9d614b7c8d Add explicit error for "no leader" 9 years ago
Philip O'Toole 3f1a3b3777 Fix Raft snapshots of in-memory databases
The snapshot code assumed all databases were disk-based.
9 years ago
Philip O'Toole f28c0b650e Precise checking of CircleCI
Issue #64.
9 years ago
Philip O'Toole 534a1bd59b Unit test transactions at the store level 9 years ago
Philip O'Toole 8cb852d1cf Prettier store address in status 9 years ago
Philip O'Toole 59052ec0b4 More status output 9 years ago
Philip O'Toole e88f97ed06 Support cluster-level leader checks for queries 9 years ago
Philip O Toole b282aede1b 'go fmt' fixes 9 years ago
Philip O Toole a653a5feb5 Skip multi-node test on CircleCI
Seems like there might be something special about their networking.
9 years ago
Philip O Toole ce0a2ce269 Usage and comment fixes 9 years ago
Philip O Toole f1963221ca Small fixes and formatting 9 years ago
Philip O Toole cc73f0abbc Default to leader check before backup 9 years ago
Philip O Toole 2e7ae59245 Complete multi-node replication test 9 years ago
Philip O Toole b058458a0f Start adding multi-node tests 9 years ago
Philip O Toole 347109e4bb More unit tests of store 9 years ago
Philip O Toole d0f689c07c Start adding store-level unit tests 9 years ago
Philip O Toole 81a3af3b7e Allow leader check for queries 9 years ago
Philip O Toole 3ffed779f8 Move to POST for bulk queries
Backup endpoint also added, but not fully-functional.
9 years ago
Philip O Toole 170a916dd5 fmt and lint fixes 9 years ago
Philip O Toole 9c02e1edb6 Make it easier to understand DSN 9 years ago
Philip O Toole 4ff67c2c90 Support in-memory databases 9 years ago
Philip O Toole 2ba973d7b7 Allow control of time display from db layer
This is not a great solution, and somewhat of a hack to make
unit-testing easier. However, it will allow control over "time"
in the response in the future.
9 years ago
Philip O'Toole a8a03ddb07 Fix comment typo 9 years ago
Philip O Toole 7c75bd9494 Finalize name for sqlite file 9 years ago
Philip O Toole 17aea81d75 Add Raft statistics 9 years ago
Philip O Toole ee4ffe1d15 Comment cleanups 9 years ago
Philip O Toole b638fa7b87 Finish snapshot restore
Compiles, but not tested.
9 years ago
Philip O Toole a47da4065f Correctly handle database removal 9 years ago
Philip O Toole 35803cdf75 Shutdown Store on exit 9 years ago
Philip O Toole a6cb950ff4 Allow DSN config at command line
It might still need to be richer, so end-users could specify an
in-memory SQLite database. Specifying a DSN would require the user
to supply the full path to the SQLite database. This is OK. However,
the code then needs to be able to parse out the path to the database
so it can remove it before start up.
9 years ago
Philip O Toole 179ad650f3 Apply() and Snapshot NOT called concurrently
No reason to worry about this.
9 years ago
Philip O Toole 2ebec9b973 Start adding support for snapshots
This shows that passing the database into the Raft module is probably
not going to work, since the database could be opened in two ways -
 directly at startup, and be completely restored from the log, or with
a combination of restoring from a snapshot, followed by the remaining
log entries. In both cases the database must be opened using the
requested DSN settings.

A detailed config object for controlling SQLite behaviour, is probably
best, and it should be passed to the Raft store on start up.

Finally, file-level copying of the SQLite file can only take place if
no transaction is in effect. This might be handled by the use of a
RWLock. The write-lock is taken during Execute() and Snapshot, but
the Read lock is taken during Query(). Unfortunately this may reduce
the concurrency of inserts and updates. Perhaps the Backup call on
the Go SQLite library might be better, but it might be slow.
9 years ago
Philip O Toole d30d2fffa4 Query path now works 9 years ago
Philip O Toole ab71c3bdd2 Execute working now 9 years ago
Philip O Toole 639264db4f Hook up handleExecute 9 years ago
Philip O Toole 28f4a73773 Simplify use of ApplyFuture 9 years ago
Philip O Toole 3ab8e01d91 Start threading ApplyFuture for Execute response 9 years ago
Philip O Toole d70a9cb0ed Push database into Raft store
Allows the DSN to be fully set, if requested.
9 years ago
Philip O'Toole 99f5e63f49 Build fixes 9 years ago
Philip O'Toole 73d880c93c Start support for SQLite DSN
This isn't complete, since it prepends the Raft directory to the
filename, which would break a DSN.
9 years ago
Philip O'Toole 77b9077d9d Pass Execute results up
Doesn't compile.
9 years ago
Philip O'Toole 63c74cabea Fully build with new db package 9 years ago
Philip O'Toole f6984f4134 Small wrapper around sqlite3 9 years ago
Philip O'Toole 4f110e26ea Update README with desired behavior 9 years ago
Philip O'Toole b95c84d16e Add database interface to Store
CREATE TABLE now works.
9 years ago
Philip O'Toole e43b0bc421 Initialize SQLite store 9 years ago
Philip O'Toole 12aef34625 Start integrating actual SQLite support
Builds without error.
9 years ago
Philip O'Toole d7d46f79c0 Start refactor for Hashicorp
Heavily influenced by hraftd. rqlited builds without error.
9 years ago