9924fe75f5
Some small improvements
1 year ago
3058d3e9fc
Fix transaction handling in db.Request()
1 year ago
59786e818b
Initial unit tests for db.Requests()
...
Seems to work well.
1 year ago
4cbb541e9b
DB Request code-complete except for transactions
1 year ago
fe176a9752
Merge branch 'query-exec-stmt-with-conn' into unified-endpoint
1 year ago
497c6b3340
Elevate queryer type
1 year ago
3d652d929e
More factoring of DB query layer
1 year ago
de70664bf1
More code in db.Request()
1 year ago
6fcf85ef40
Flag major issue
1 year ago
1987f9b07b
Flesh out db.Request
...
Needs full unit testing.
1 year ago
b2b0b188e9
Factor out functions
1 year ago
014559bb06
Another DB read-only test
1 year ago
f84b258f99
Better variable name
1 year ago
c3e7044945
Final read-only test
1 year ago
2af5fc26aa
Move comment
1 year ago
8f45c4c8c9
Factor out read-only check
1 year ago
bb868595c1
Start prepping DB layer
1 year ago
92a604c4ec
Move SQLite validity checks to db module
1 year ago
fb1e5e3903
Use correct errors (thanks GPT!)
1 year ago
95a689b3cf
Some improvements thanks to ChatGPT 4
1 year ago
c1ba9157c2
More unit test of nil parameters
2 years ago
41bf265b02
Fix more handling of nil named values
2 years ago
053bde0ed6
Upgrade to latest SQLite driver
2 years ago
3d555a73cb
Add unit test showing malformed database issue
...
https://github.com/rqlite/rqlite/issues/1103
2 years ago
0702e39ace
Add ResetStats
2 years ago
e3cc4eb5d0
go mod updates ( #1120 )
...
* go mod updates
* CHANGELOG
2 years ago
e74bfc182a
Use new Encoder everywhere
2 years ago
0071867e34
Don't call t.Fatal() from goroutine
...
It wouldn't abort the test anyway, so just log.
2 years ago
5141eff5fc
test: use `T.TempDir` to create temporary test directory
...
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2 years ago
54aec1ca33
Tighten up backup code
2 years ago
221907f8a9
Support nil as paramterized values
2 years ago
c8b9fd72da
Sanity check supplied SQLite data
2 years ago
c91ca45ba4
Expose more DB attributes
2 years ago
a6a3567a4d
DB layer returns in-memory status
2 years ago
d84fb85f01
Don't HTML escape JSON output
3 years ago
869f3547b7
Update to SQLite 3.38
3 years ago
ebb2ff32c2
Send rand() once in DB layer
3 years ago
24a5dc8c4b
Much better random file names
3 years ago
3bfcbffc37
Fix typos and Grammar issues
3 years ago
7038b86dcb
One final named params unit test
3 years ago
01849e77d5
Add db-level support for named paremeters
3 years ago
239cdc04b5
Don't run heavy memory tests in parallel
...
Let's see if this helps these tests run faster, especially with -race,
and prevent CircleCI timeouts.
3 years ago
eab9183acd
Better comment in DB code
3 years ago
51bb54488b
Explicitly handle column type nil (null)
3 years ago
c639f21cde
Return time types in RFC3339 format
3 years ago
28063bfa0d
Support timestamp field
3 years ago
53e4b9ae3d
ABL
3 years ago
437d76bc1c
Update db.go
3 years ago
bed9dc2a49
Update db_test.go
3 years ago
393a1c67a0
Fix unit tests
3 years ago
3ed0909746
Read-only statements in query endpoint
3 years ago
519a91e635
Actual recovery node test passes on Linux
3 years ago
2bf7638a8d
Fix debug code in test
3 years ago
01895e501f
Serialize on-disk databases by simply reading file
...
This is safe according to https://sqlite.org/howtocorrupt.html and
avoids any sqlite3_serialize() limitations.
3 years ago
c0b22b0d03
Update CHANGELOG
3 years ago
08eb74493d
Add unit test for >1GiB in-memory databases
...
>2GiB in-memory are still not supported.
3 years ago
44e47686ee
Tests that demonstrate current isolation behavior
3 years ago
a4c4374daf
Update CHANGELOG
3 years ago
3907132761
Add detailed SQLite memstats to status/
3 years ago
aa8df67592
DB-level unit tests pass
3 years ago
161abb094d
Add db-level FK constraints unit test
3 years ago
deb4dabf2e
Thread through foreign key constraint controls
3 years ago
2cad491f40
Better creation of SQLite DSNs
3 years ago
f7b390ab28
Adding JSON1 unit test
3 years ago
96f9e85ee3
More concurrent query testing
3 years ago
3cee11de6f
Count number of query errors
3 years ago
6c69e8a9b5
Upgrade to new compile options
3 years ago
ea723ee830
Ping databases on open to ensure full availability
3 years ago
953e82f7c4
Add SQLite compile-time options to status output
3 years ago
11da97f455
Fix typo
3 years ago
497d3dd873
Remove obsolete foreign key constraint control
3 years ago
c086eba427
Remove debug statement
3 years ago
b83424484d
Enhance SQLite database status
3 years ago
cc8b924e64
Fix WaitGroup usage in test
3 years ago
4f2818d037
Add concurrent query test
3 years ago
eec7494d62
DB size requires a read-write connection
3 years ago
6edff5ba24
Deal with SQLite connection "caching"?
...
https://github.com/mattn/go-sqlite3/issues/966
3 years ago
bf51947ca9
Better error messages
3 years ago
d101e10469
Test writing via the query connection
...
It should fail.
3 years ago
854cf362fd
Don't specify mode=rw on file open
...
Not sure why.
3 years ago
6dc9583dab
Implement concurrent reads and writes
...
Disk doesn't support it yet, as it has not moved to WAL mode. Perhaps
wait for next release to do that.
3 years ago
db1768c63f
Initial implementation of two database connections
3 years ago
90a0d397c6
Switch from mode=memory to vfs=memdb for in-mem DB
3 years ago
791a1b0b72
Remove obsolete references to DSN
3 years ago
e0da96dd55
Remove support for specifying SQLite DSN
...
This is a breaking change. However this feature hasn't been tested and
allows end-users to break the system too easily. Low-level control over
the SQLite database is better done with PRAGMA commands where possible.
3 years ago
2924ed407d
Upgrade to rqlite/go-sqlite3 v1.20.4
3 years ago
028419af0f
Fix race in in-memory database testing
3 years ago
1ee7a9eb17
Limit in-memory connection pool to 1 connection
...
https://github.com/mattn/go-sqlite3/issues/959#issuecomment-890283264
3 years ago
911df4c03b
Don't leak a database if deserialization fails
...
Fixes https://github.com/rqlite/rqlite/issues/837 .
3 years ago
8693694d14
Add current SQLite journal mode to status output
3 years ago
503825dbaf
Remove Abort functionality
...
No longer relevant since each Exec and Query gets its own connection.
3 years ago
561bb47a14
Port DB layer to sql/database
...
Empty statements now cause panic.
3 years ago
eb8dcf74ff
Start moving to SQL layer
...
Builds.
3 years ago
5f5f7dbbea
Sync use of DB connection
...
Possible, albeit simple, fix for https://github.com/rqlite/rqlite/issues/830
3 years ago
c5e67a548c
Even more records during db test
4 years ago
665d9913bb
Add top-level Deserialize() to db layer
4 years ago
b83b2b241d
Merge branch 'master' into no-disk-during-restore
4 years ago
18a4f38701
Add Copy() method to db layer
4 years ago
ce7a3a6912
Merge branch 'master' into no-disk-during-restore
4 years ago
535d690d7c
Tigthen up Store-level locking
...
Previous locking didn't look quite correct, and with the advent of the
Serialize functionality, it needs to be more precise.
4 years ago
7900adbf53
Add Deserialize to DB layer
4 years ago
9210297413
Unit test DB serialize
4 years ago
93a78f09a0
Add call to Serialize function
4 years ago
488a9554ea
Switch to rqlite fork of go-sqlite3
4 years ago
011bc00784
Add SQLite database page-centric size to status output
4 years ago
6575de779d
Use Protobuf for encoding Raft Log commands
...
This PR changes Raft Log Entry encoding from JSON to Protobuf. Furthermore, larger Raft commands (which can result from batching SQL statements, or individually long SQL statements) are compressed before encoding.
This primary reason for this change is to reduce IO load since that is one of the largest performance bottlenecks. It will also reduce internode traffic.
Legacy JSON-encoded commands are still handled by this code, so this change is backwards-compatible with previous releases in the v5 series.
4 years ago
b4867713e4
Better variable name, Query -> SQL
4 years ago
ee9f929735
ABL
4 years ago
f961bb4e44
Fix typos and comments
4 years ago
69ab554b58
Unit test parameterized statements
4 years ago
1075315a11
DB Query now accepts complex Statements
4 years ago
db992d1dce
DB Execute now accepts complex Statements
4 years ago
3cfb0c3910
JSON types are also text
5 years ago
b283ecc86d
Ensure backup is correctly closed
...
Also catch any error that occurs during close.
5 years ago
0e69be3949
Store layer supports generating SQL format backups
...
Port PR453.
5 years ago
43ea24fb82
DB layer supports generating SQL format backups
...
Port of PR451.
5 years ago
0471877f26
Abort transaction if load operation fails
5 years ago
6d5d4b3d81
Fix unit tests due to SQLite master table changes
7 years ago
6bbdc88c1d
ABL
...
Always be linting.
7 years ago
bca0e43218
Unit test JOIN at the database level
7 years ago
20f0eb505c
Add test showing OK PRAGMA table_info
8 years ago
86a59adc57
Unit test query of SQLite master table
8 years ago
ffffd99f33
Test Common Table Expressions at database layer
8 years ago
b0e7432dc3
Check for nil row from Execer
8 years ago
9a7ff4c979
Handle "" types
...
Add unit test for concatenated queries. This deals with issue #240 but
this may be a hack. I need to dig into this issue more.
8 years ago
75417f6d1e
Start handling the various SQLite TEXT types
...
This is the start of a code change to handle this comprehensively.
8 years ago
d140b1e8c7
More database-level stats
8 years ago
48f7bb2c50
Add low-level DB expvar stats
8 years ago
efc49511a5
Test multiline statements at DB level
8 years ago
6815f09873
Database can return FK constraint state
8 years ago
1de7848362
Clean up and simplify DB tests
8 years ago
6b3704a9fa
Add db test helper functions
8 years ago
60443bee0b
Reduce test cyclo complexity
8 years ago
cb12dc17bf
Reduce cyclo complexity of query code
8 years ago
9fce85b059
Add UNIQUE constraint test
8 years ago
acb592aa0b
Tighten logic for FK constraint control
8 years ago
c40fff4d46
Support controlling FK constraints at startup
...
This was possible previously, but would need to be set everytime on
startup via the API. This change allows it to set at startup AND enables
foreign constraint checking by default.
8 years ago
56778f1764
'go lint' fixes
8 years ago
b248b0f4e1
Trivial formatting changes
8 years ago
3f064afc31
db: only convert row value to string if the column type is text
8 years ago
bf2047b45b
'go lint' fixes
8 years ago
6fde29ff62
Fix typos
9 years ago
6405dc01a8
Add function for loading in-memory db from file
9 years ago
31cda32a38
'go fmt' fixes
9 years ago
f4c9ff0c3b
Start with new DB layer
9 years ago
721445acd0
Update db.go
9 years ago
9b4f0d409e
More package doc for db
9 years ago
2b08c194ea
Update db.go
9 years ago
7b628686bf
Add SQLite version to diagnostics
9 years ago
1554934e88
'go fmt' fixes
9 years ago
285c8d4be6
Use sqlite3.SQLiteDriver directly, fix issue #73
9 years ago
300d58cfd3
'go fmt' fixes
9 years ago
9f348a6171
Use SQLiteRows.DeclTypes() to fix issue #52
9 years ago
f62787624c
Bypass sql.DB for queries (for direct SQLiteRows access)
...
It's necessary to bypass sql.DB in order to gain access to SQLiteRows
instances, which is the first step toward inclusion of result column
type declarations in query results for issue #52 .
9 years ago
3f1a3b3777
Fix Raft snapshots of in-memory databases
...
The snapshot code assumed all databases were disk-based.
9 years ago
f1963221ca
Small fixes and formatting
9 years ago
3ffed779f8
Move to POST for bulk queries
...
Backup endpoint also added, but not fully-functional.
9 years ago
524e6170bf
Final tests of backup
9 years ago
8b3c4202cc
Work for database backup
...
Tests are failing.
9 years ago
fac68c3cdd
Return time in seconds
9 years ago
170a916dd5
fmt and lint fixes
9 years ago
4ff67c2c90
Support in-memory databases
9 years ago
2ba973d7b7
Allow control of time display from db layer
...
This is not a great solution, and somewhat of a hack to make
unit-testing easier. However, it will allow control over "time"
in the response in the future.
9 years ago
f33ffc2936
Measure time per statement
9 years ago
1b41f789b8
Remove debug panic()
9 years ago
297479bc4e
Make multi-SELECT work
9 years ago
ee4ffe1d15
Comment cleanups
9 years ago
a6cb950ff4
Allow DSN config at command line
...
It might still need to be richer, so end-users could specify an
in-memory SQLite database. Specifying a DSN would require the user
to supply the full path to the SQLite database. This is OK. However,
the code then needs to be able to parse out the path to the database
so it can remove it before start up.
9 years ago
2ebec9b973
Start adding support for snapshots
...
This shows that passing the database into the Raft module is probably
not going to work, since the database could be opened in two ways -
directly at startup, and be completely restored from the log, or with
a combination of restoring from a snapshot, followed by the remaining
log entries. In both cases the database must be opened using the
requested DSN settings.
A detailed config object for controlling SQLite behaviour, is probably
best, and it should be passed to the Raft store on start up.
Finally, file-level copying of the SQLite file can only take place if
no transaction is in effect. This might be handled by the use of a
RWLock. The write-lock is taken during Execute() and Snapshot, but
the Read lock is taken during Query(). Unfortunately this may reduce
the concurrency of inserts and updates. Perhaps the Backup call on
the Go SQLite library might be better, but it might be slow.
9 years ago
1bbff9d53a
Lowest-layer database tests completed
9 years ago
e047d9e9d9
Test_SimpleFailingStatements ported
9 years ago
3c88305a77
Unit test failing statement
9 years ago
57b4dddd26
Add unit test of db package
9 years ago
1855f2d541
Fix up use of interface{} and *interface{}
9 years ago
e8490b842e
Query in db package almost there
...
It's the handling of the types, and ensuring they can be cast correctly,
which is what is next.
9 years ago
63c74cabea
Fully build with new db package
9 years ago
f6984f4134
Small wrapper around sqlite3
9 years ago
08c310400a
No need for db package
9 years ago
b95c84d16e
Add database interface to Store
...
CREATE TABLE now works.
9 years ago
e17cceed91
Better name for test helper
9 years ago
40b156fd47
Test_TransactionsHardFail now passes
9 years ago
226938c0d1
Test_TransactionsConstraintViolation now passing
9 years ago
801158fa83
Test_SimpleTransactions now passing
9 years ago
62137fc798
Test_FailingSimpleStatements now passing
9 years ago
ac3b5f2050
Unit-test simple database insertion cases
9 years ago
67e7fa3c8e
Continue moving db tests to standard framework
9 years ago
05fca403ab
Start removal of nonstandard test framework
9 years ago
e43b0bc421
Initialize SQLite store
9 years ago
12aef34625
Start integrating actual SQLite support
...
Builds without error.
9 years ago
022b536ade
Revert "Revert "Merge pull request #25 from oscillatingworks/deprecate-log4go""
...
This reverts commit fe415e6452
10 years ago
fe415e6452
Revert "Merge pull request #25 from oscillatingworks/deprecate-log4go"
...
This reverts commit f2de6cc26ecdc6021ae7
10 years ago
48f9b8bfd4
gometalinter: rqlite/db
...
db/db.go:14:1⚠️ dbName is unused
db/db.go:30:11⚠️ error return value not checked (os.Remove(dbPath))
db/db.go:63:18⚠️ error return value not checked (defer rows.Close())
db/db.go:15:⚠️ unused global variable dbName
10 years ago
eba40187a9
Deprecate code.google.com/p/log4go
...
- Add new internal `rqlite/log` package
- Replace `log4go` with `rqlite/log`
- Adapt log formatting in existing logs
Fixes #21
10 years ago
b91f275d1b
Enhance logging
10 years ago
4758106e53
Log a warning if a query might change the database
10 years ago
1629c63987
Construct some log strings more idiomatically
10 years ago
e69218ef4a
Layout source in a more conventional style
10 years ago
Philip O'Toole
Eng Zer Jun
chermehdi
Benjamen Keroack
Zac Medico
Fernando Álvarez