This is a breaking change. However this feature hasn't been tested and
allows end-users to break the system too easily. Low-level control over
the SQLite database is better done with PRAGMA commands where possible.
This PR changes Raft Log Entry encoding from JSON to Protobuf. Furthermore, larger Raft commands (which can result from batching SQL statements, or individually long SQL statements) are compressed before encoding.
This primary reason for this change is to reduce IO load since that is one of the largest performance bottlenecks. It will also reduce internode traffic.
Legacy JSON-encoded commands are still handled by this code, so this change is backwards-compatible with previous releases in the v5 series.
This was possible previously, but would need to be set everytime on
startup via the API. This change allows it to set at startup AND enables
foreign constraint checking by default.
It's necessary to bypass sql.DB in order to gain access to SQLiteRows
instances, which is the first step toward inclusion of result column
type declarations in query results for issue #52.
This is not a great solution, and somewhat of a hack to make
unit-testing easier. However, it will allow control over "time"
in the response in the future.
It might still need to be richer, so end-users could specify an
in-memory SQLite database. Specifying a DSN would require the user
to supply the full path to the SQLite database. This is OK. However,
the code then needs to be able to parse out the path to the database
so it can remove it before start up.
This shows that passing the database into the Raft module is probably
not going to work, since the database could be opened in two ways -
directly at startup, and be completely restored from the log, or with
a combination of restoring from a snapshot, followed by the remaining
log entries. In both cases the database must be opened using the
requested DSN settings.
A detailed config object for controlling SQLite behaviour, is probably
best, and it should be passed to the Raft store on start up.
Finally, file-level copying of the SQLite file can only take place if
no transaction is in effect. This might be handled by the use of a
RWLock. The write-lock is taken during Execute() and Snapshot, but
the Read lock is taken during Query(). Unfortunately this may reduce
the concurrency of inserts and updates. Perhaps the Backup call on
the Go SQLite library might be better, but it might be slow.
This reverts commit f2de6cc26e, reversing
changes made to cdc6021ae7.
The change causes a panic when the server is started like so:
$GOPATH/bin/rqlite ~/node.1
db/db.go:14:1⚠️ dbName is unused
db/db.go:30:11⚠️ error return value not checked (os.Remove(dbPath))
db/db.go:63:18⚠️ error return value not checked (defer rows.Close())
db/db.go:15:⚠️ unused global variable dbName
Philip O'Toole
Benjamen Keroack
Zac Medico
Fernando Álvarez