1276 Commits (next)

Author SHA1 Message Date
Sayan Nandan 8d2097e526
Add txn impls and tests for model DDL 1 year ago
Sayan Nandan 29d4137a2c
Add space DDL txns and full chain tests
Also ensure that the `env` key is not when merged
1 year ago
Sayan Nandan 39edfc64c9
Make sure props are stored inside space.meta.env
Also fixed a bug in the create space txn payload
1 year ago
Sayan Nandan ea072f281c
Add create space txn 1 year ago
Sayan Nandan 97a9471529
Utilize VFS for all testing 1 year ago
Sayan Nandan 9133949beb
Add IO tests for gns tx payloads 1 year ago
Sayan Nandan ea20611ebe
Cleanup gns txn impls 1 year ago
Sayan Nandan 13a39f1e1f
Simplify enc/dec misc procedures 1 year ago
Sayan Nandan 891252a89d
Add remaining DDL txns 1 year ago
Sayan Nandan 138753c4ad
Add `create model` txn impl 1 year ago
Sayan Nandan b78824d6d0
Simplify gns event impls 1 year ago
Sayan Nandan 7eff973b9e
Add `create space` txn impl 1 year ago
Sayan Nandan 79b450160e
Remove metadata spec and simplify obj enc/dec 1 year ago
Sayan Nandan d09df81757
Unlink MD traits from map enc/dec impls 1 year ago
Sayan Nandan df61b627b4
Add preliminary event impls 1 year ago
Sayan Nandan d8cabb9761
Add enc/dec for space 1 year ago
Sayan Nandan 70552a5df4
Impl enc/dec for model 1 year ago
Sayan Nandan 6e3f26ddbd
Make enc/dec for dicts more generic 1 year ago
Sayan Nandan 550a39ad99
Fix space DDL tests 1 year ago
Sayan Nandan 369abe9a22
Allow persist objects to have custom metadata 1 year ago
Sayan Nandan 2937cc7fbe
Simplify enc/dec spec traits 1 year ago
Sayan Nandan a619ea635c
Always assign UUIDs 1 year ago
Sayan Nandan 29457c6b89
Impl enc/dec routines for internal structures 1 year ago
Sayan Nandan 026b5ef0d2
Implement dict dec and resolve null disambiguation
The previous logic of using a different null was extremely confusing,
and we essentially ended up creating "two different null types."

That has been removed and we have implemented the dict dec methods and
rectified the enc methods.
1 year ago
Sayan Nandan bd3225693f
Upgrade deps 1 year ago
Sayan Nandan b6d5ad1c75
Add basic impl for `DictGeneric` 1 year ago
Sayan Nandan 7346035a27
Impl basic hl inf for persist 1 year ago
Sayan Nandan 3d2e4f2014
Simplify metadict semantics 1 year ago
Sayan Nandan 263e70905d
Rewrite journal: recovery impl and driver init
On initialization the driver will no longer attempt to truncate the
close event but instead use a truly AO style and utilize a new
directive.

We also now have a basic externally triggered on write and automatically
trigger on read recovery system in place. However, there are several
scopes of improvement to the implementation, such as tolerating even
more inconsistencies in the data.
1 year ago
Sayan Nandan e2b81ac77f
Upgrade deps 1 year ago
Sayan Nandan c1f8e2d1bd
Remove the delete entry once validated 1 year ago
Sayan Nandan dc4afdc257
Add mock txn impl and fix txn impls 1 year ago
Sayan Nandan 710fd79e64
Add methods to verify header data 1 year ago
Sayan Nandan 24c3b0f8a7
Fix test suite file writers 1 year ago
Sayan Nandan b0ef30853c
Upgrade deps 1 year ago
Sayan Nandan 843ff05d85
Impl txn log writer 1 year ago
Sayan Nandan 55f53456f8
Implement transaction reader 1 year ago
Sayan Nandan 9091db6bd3
Impl SDSS Writer 1 year ago
Sayan Nandan 17b07897e5
Implement `StartStop` 1 year ago
Sayan Nandan 2756682729
Implement auto generation for header objects 1 year ago
Sayan Nandan 45c93eac3c
Redefine SDSS Header records to have fixed layout 1 year ago
Sayan Nandan e126c826f9
Add new dynamic record definition and remove rw 1 year ago
Sayan Nandan 901150f98f
Impl basic header enc/dec methods 1 year ago
Sayan Nandan beb4d16ab4
Simplify header decode impl and fix MDR endian bug 1 year ago
Sayan Nandan 0e77946714
Add decode impls for headers 1 year ago
Sayan Nandan 96acecb2c3
Impl basic SDSSWriter (raw) 1 year ago
Sayan Nandan c8d3e6b739
Fix uptime impl 1 year ago
Sayan Nandan f29807446f
Add generation for full SDSS header 1 year ago
Sayan Nandan b0651c9492
Implement SDSS Header generators 1 year ago
Sayan Nandan dbc7128c41
Add more tests for update 1 year ago
Sayan Nandan 4580b76d55
Implement v1 executor for update 1 year ago
Sayan Nandan f2a0fda29d
Remove the shared impl of seq index
I actually realized this is a terrible abstraction the day I implemented
it and have since been planning to remove it.

The issue is with aliasing and potential corruption in a multi-threaded
environment. Even though we have never used it in such a context (aka
triggering UB) but we can potentially make an *accidental* use of it;
don't take it lightly: it's like modifying data while someone else is
reading it. That's not bad, that's a nightmare.
1 year ago
Sayan Nandan dae2052773
Implement and select and fix security bug in index
Discovered another security bug which could potentially be dangerous:
the low level impl of the index used a comparator where eager evaluation
of an expression led to an erroneous return.
1 year ago
Sayan Nandan abf1e29344
Add tests for insert and delete, and fix security bug
This commit makes an important security fix that caused non-matching
of field data which could have had terrible impacts.

Second, it adds tests for insert and delete.
1 year ago
Sayan Nandan b714647905
Add tests for insert 1 year ago
Sayan Nandan 2488053318
Implement delete and add impls required for other dml queries 1 year ago
Sayan Nandan 361acccc09
Use delta for schema changes
Also fixed an issue where an `alter model` would fail to keep added
fields in order.
1 year ago
Sayan Nandan 5838941ce8
Impl basic executor for ins 1 year ago
Sayan Nandan 60157e110b
Simplify cht impls 1 year ago
Sayan Nandan 05d93d2102
Add basic row definition 1 year ago
Sayan Nandan ac2ec6f71a
Move dc into data 1 year ago
Sayan Nandan 9297095f45
Add spl case tests 1 year ago
Sayan Nandan e2112c8bc1
Fix error causing dangling block 1 year ago
Sayan Nandan 4eb2851cf9
Improve word impls 1 year ago
Sayan Nandan 329ef1a27e
Add tests for large integers in data items 1 year ago
Sayan Nandan b25899e04b
Fix native qw storage for `Datacell` 1 year ago
Sayan Nandan eca185d560
Make rc modular 1 year ago
Sayan Nandan 554a478917
Add index key 1 year ago
Sayan Nandan 87adc1046d
Simplify entity handling 1 year ago
Sayan Nandan f98c5d3aa4
Reduce memory usage per GNS index item 1 year ago
Sayan Nandan 1586b05bbd
Add `drop model` exec without advanced params 1 year ago
Sayan Nandan fcc187901d
Require full entity in `create model` 1 year ago
Sayan Nandan 45ef72e400
Require full entity path in `alter model` 1 year ago
Sayan Nandan 30d1be4862
Add misc fixes 1 year ago
Sayan Nandan d626f9a302
Fix layer validation call and entity parse 1 year ago
Sayan Nandan ccfb7b2e12
Unsafe code review (partial)
Unsafe code review for March, 2023.

NEEDCHECK: Still need to verify unsafe code in index implementations
1 year ago
Sayan Nandan f0f67a98fc
Fix unsoundness in `alter space` and remove redundant macro usage 1 year ago
Sayan Nandan f351be2819
Remove partial entity syntax
This was a bad mistake that we made with the actions API, and we won't
make the same mistake with BlueQL again. PES has been a terrible idea
all along, and it was a leaky abstraction.

Instead, we'll be using a more robust framework for addressing entities.
2 years ago
Sayan Nandan 0e67872e69
Add tests for simple alter 2 years ago
Sayan Nandan c5d7f5f6f6
Test illegal type casts 2 years ago
Sayan Nandan 8f77c1d73a
Add basic alter exec impl 2 years ago
Sayan Nandan 9bc3d85b1c
Revise lit definitions and layout 2 years ago
Sayan Nandan 73df6f9af4
Define intents for sync 2 years ago
Sayan Nandan 1c95c2e70f
Implement alter model plan 2 years ago
Sayan Nandan 7069ab7bfb
Add exec for create model 2 years ago
Sayan Nandan 64c8b55b93
Add exec for drop space 2 years ago
Sayan Nandan feac086edc
Implement exec for create model 2 years ago
Sayan Nandan a88319b8b9
Enable fpath on nullable data 2 years ago
Sayan Nandan 188f9ecbb4
Support null in cell 2 years ago
Sayan Nandan b9453041e4
Trace fpath in layers 2 years ago
Sayan Nandan 1c9dfbaebe
Implement layered data validation
Also fixed stacked borrows
2 years ago
Sayan Nandan b8dd8135f1
Implement layer validation 2 years ago
Sayan Nandan 8d789bd166
Migrate to the newer datacell impl 2 years ago
Sayan Nandan 5d9851a427
Add model cell definition and more word impls 2 years ago
Sayan Nandan 3b8a251a01
Define larger words 2 years ago
Sayan Nandan 9d9ffeed1a
Skip expensive ql tests on miri [skip ci] 2 years ago
Sayan Nandan f2dbec0229
Use custom tags 2 years ago
Sayan Nandan ca04ea2cd1
Use custom lit impl 2 years ago
Sayan Nandan 66dab00eb3
Add lit impls
Also fixed 32-bit word store
2 years ago
Sayan Nandan 0410a019cf
Define explicit literals 2 years ago
Sayan Nandan 28af6f20ad
Add word definitions to `mem` 2 years ago
Sayan Nandan 5e8501320d
Implement vanilla patches and get rid of `Clone` bounds 2 years ago
Sayan Nandan 36b1e4fec2
Get rid of borrow in MT idx specs and related objects 2 years ago
Sayan Nandan 70810445cb
Make cmp more obvious 2 years ago
Sayan Nandan 1f608c1f6c
Support custom patch based mem tree delta 2 years ago
Sayan Nandan 34990c3186
Fix metadata merge in alter space exec for `null` prop cases 2 years ago
Sayan Nandan 3a992253c1
Fix alter space exec and add tests 2 years ago
Sayan Nandan bc7c7bf595
Implement basic exec for `alter space` 2 years ago
Sayan Nandan 1b2c6923bc
Refactor engine modules 2 years ago
Sayan Nandan f669cc7b00
Add patch based metadata merge algorithm 2 years ago
Sayan Nandan 997205a327
Exec queries to test 2 years ago
Sayan Nandan a33721e0a4
Implement basic executor for `create space`
Also:

- Added tests for `ItemID`
- Added tests for `create space` executor
2 years ago
Sayan Nandan 260dff954b
Use specific errors for AST nodes 2 years ago
Sayan Nandan 6899a35870
Use specific error kind for lex 2 years ago
Sayan Nandan 2f128c2cf2
Use `HSData` for dict entries for simplicity 2 years ago
Sayan Nandan fb2a218ead
Add `UArray` based str 2 years ago
Sayan Nandan 79f8630424
Use `Ident` to avoid major future refactors 2 years ago
Sayan Nandan 080563cc24
Re-organize imports to match import style [no ci] 2 years ago
Sayan Nandan 25ef03221b
Make `ASTNode` sensitive to COW style parsing 2 years ago
Sayan Nandan 35143b7960
Ensure we don't crash the CI runner
Too many input keys might consume far too much memory crashing the runner
2 years ago
Sayan Nandan 798871bdd6
Simplify wrapper based access 2 years ago
Sayan Nandan f98478b274
Fix state machines, refactor schema and add `ASTNode` def and impls
This is one huge commit that fixes a bunch of issues we've had with
the schema and surrounding impls. Here are some of them
- The type metadata syn parse was incredibly broken. More than
one trailing comma or other punctuation following a trailing symbol
(as in a comma) is illegal
- The `*_full` set of methods were repetitive and annoying. That has
been fixed (not the most elegant solution, but it works and fits
well)
- As a result of the first issue, fuzz targets kept blowing up,
rightfully. This has been fixed too. The fuzz needs to be more
deterministic covering all permutations, and this is something I'll
fix up ahead.

Finally, I can now happily say that the QL impl is rock solid. There is
another possible bug that I'm tracking down, but that will be fixed.
2 years ago
Sayan Nandan 64d87fe089
Refactor ddl module 2 years ago
Sayan Nandan e404a273ef
Simplify kw and sym lookup 2 years ago
Sayan Nandan 263a287edd
Fix double-free, use-after-free and other bugs in mtchm
The bug was actually because of my oversight in the VInline clone
impl for non-Copy Ts. Also added tests for worst case O(n)
performance.
2 years ago
Sayan Nandan 932bb4f58b
Fix remove len impl, use relaxed and add tests 2 years ago
Sayan Nandan 1920664273
Fix use-after-free in mtchm impl, add tests and add trait impls 2 years ago
Sayan Nandan 51622ba4d6
Add some smart pointers 2 years ago
Sayan Nandan 427ff72e42
Move iter into module for st seq idx 2 years ago
Sayan Nandan b21162c4dc
Make st idx generic on alloc strategy 2 years ago
Sayan Nandan 3eb00f892f
Add insert test 2 years ago
Sayan Nandan bec5e10f7b
Add mt idx empty tests 2 years ago
Sayan Nandan c9ea051bd3
Put state into config 2 years ago
Sayan Nandan 900e10efd7
Add mt idx impl (copy) 2 years ago
Sayan Nandan 57c7a6447b
Add mt idx impls 2 years ago
Sayan Nandan 48ada558ad
Clarify idx trait definitions
MT idx impls would have a fairly different interface when compared to ST
equivalents.
2 years ago
Sayan 29baeb4996
Implement mtchm 2 years ago
Sayan Nandan a59823fc48
Add misc lib changes 2 years ago
Sayan Nandan f26e71512e
Fix segfaults in UArray impl 2 years ago
Sayan Nandan 6fcf87c040
Add misc bounds and impls for `TMCell` 2 years ago
Sayan Nandan 548ed53e16
Do not enforce a `Clone` bound in idx impls 2 years ago
Sayan Nandan d831eb3a7c
Enable bi-directional iter on `VInline` 2 years ago
Sayan Nandan 89888483e2
Enable bi-directional iter on `UArray` 2 years ago
Sayan Nandan 86585ff864
Impl into_iter for `VInline` 2 years ago
Sayan Nandan 28b6ce0d96
Add misc trait impls for seq mem structures 2 years ago
Sayan Nandan f63ec2d37d
Add `IntoIter` for `UArray` 2 years ago
Sayan Nandan 14c1d94027
Refactor `mem` modules 2 years ago
Sayan Nandan 49dfac11b5
Add remove and capacity optimization ops to VInline
Also updated tests to make debugging easier (the previous test simply
placed all operations in one test making it harder to debug any bugs)
2 years ago
Sayan Nandan 10a36fa77d
Add `UArray` definition 2 years ago
Sayan Nandan ab30a74173
Re-export at mod level 2 years ago
Sayan Nandan 9d3115a7cf
Enable idx init with cap 2 years ago
Sayan Nandan a9900e2bd6
Revise definitions for ST idx 2 years ago
Sayan Nandan 7a2e8e4b58
Impl idx traits for `StdMap` 2 years ago
Sayan Nandan fa17f72519
Add `VInline` impl 2 years ago
Sayan Nandan 2dfe7227aa
Add `TMCell` 2 years ago
Sayan Nandan 44aa57a25a
Add sync module 2 years ago
Sayan Nandan 27e462fb4f
Refactor modules 2 years ago
Sayan Nandan 9d3cef9685
Add tests for ord index 2 years ago
Sayan Nandan 01f2bca81f
Add basic tests for ord index 2 years ago
Sayan Nandan ae516b7168
Add Index trait impls 2 years ago
Sayan Nandan 29a4cceea1
Add ordered iter impls for ord idx 2 years ago
Sayan Nandan b5ec9da926
Add ord index impls and fix UB
Added all basic impls for the ord index, also fixed dealloc on a nullptr
in drop impl
2 years ago
Sayan Nandan 530e76eca4
Add primitive methods to ord idx impl 2 years ago
Sayan Nandan 603ea85e8c
Add index definitions 2 years ago
Sayan Nandan ee32df0916
Add iterator definitions to ME trait 2 years ago
Sayan Nandan a116e49953
Add initial definition of `MemoryEngine` 2 years ago
Sayan Nandan fd22b51183
Use `State` for all other DDL statements 2 years ago
Sayan Nandan 763007a98c
Use `State` for `CREATE` and `ALTER` statements 2 years ago
Sayan Nandan 2756b1d070
Simplify lit to owned data clone 2 years ago
Sayan Nandan 4d140c74fd
Support built-in fn substitution 2 years ago
Sayan Nandan 0f064e1087
Use `State` for more reliable QP 2 years ago
Sayan Nandan 1df6f33032
Remove `RawSlice` usage
This hasn't been an easy task, but I'm finally very happy that we've
gotten rid of all usage instances of `RawSlice` within `engine::ql`.
This is a great feat in terms of codebase cleanup, removing unnecessary
usage of `unsafe` code.

However, this had made lifetimes slightly more complex; for example, we
can no longer directly return from methods that use `&[Token]` since all
returned data is now bound to the same lifetime.

Either way, with some care such issues can be avoided.
2 years ago
Sayan Nandan 255d058793
Add tests for secure list parsing 2 years ago
Sayan Nandan 16fb7a7d1b
Support parameters in DDL queries 2 years ago
Sayan Nandan ce8bba18cd
Add support for parameterization 2 years ago
Sayan Nandan 3257798ee0
Use `LitIR` throughout `dml` 2 years ago
Sayan Nandan 40b4c5042a
Define `QueryInterface` 2 years ago
Sayan Nandan 82f8cea5d1
Fix tests in `tests::dml_tests` 2 years ago
Sayan Nandan 4c5e919954
Store rel expr operands as `LitIR` reprs 2 years ago
Sayan Nandan b918af6d96
Use flags for DT 2 years ago
Sayan Nandan 5d527408cd
Fix number decode algorithm for signed numbers
Also added preliminary tests for the same
2 years ago
Sayan Nandan dae3faf87d
Implement insecure lexer 2 years ago
Sayan Nandan 5a7145fa40
Implement parameterization and remove unsafe literals
Also refactored tests for clarity
2 years ago
Sayan Nandan 79050e5fff
Rename `UnsafeLit` to `SafeLit`
This is more logical due to the inherent meaning of `unsafe`
2 years ago
Sayan Nandan bd0c06652c
Use `where` clauses by default for DML queries
Along with this, here's a summary of other changes:
- `RawSlice` now uses `NonNull` to help the compiler with size opts
- The lexer now allows case insensitive keyword usage (I still DO NOT
particularly approve of it, because enforcing is important)
2 years ago
Sayan Nandan 2dec28d989
Add `where` clause parsing 2 years ago
Sayan Nandan cac7bd4860
Add new keywords 2 years ago
Sayan Nandan 7e5e2838cc
Add support for relational expressions 2 years ago
Sayan Nandan ddba886a0f
Add operating modes 2 years ago
Sayan Nandan d1cba5d8b4
Support signed integers 2 years ago
Sayan Nandan 9d51dc70fe
Make sure we accept all literal types in queries 2 years ago
Sayan Nandan 986cb26ebd
Support `null` in dict and tymeta 2 years ago
Sayan Nandan 173832dd13
Add benches for `drop` 2 years ago
Sayan Nandan c36693afd5
Add benches for `use` and `inspect` 2 years ago
Sayan Nandan 5711fd1089
Improve DDL query parsing
Summary of changes:
- Remaining query parse methods that directly used the `Compiler` were
modified to use token streams instead
- `Entity::parse_from_tokens` to fix assignment to counter instead of
increments
- Allow entity in `drop model`
- `alter model` and `alter space` directly handle space/model names
instead of depending on passing value via args
- Tests added for `drop` and `inspect`
2 years ago
Sayan Nandan 89c6035367
Fix some DDL query implementations
Summary of changes:
- The `drop` queries now use the `DropItem` for drop definitions
- `create model` is now entirely handled by the function in `schema`
- `create space` is now entirely handled by the function in `schema`
- Tests were added for drop (they were never present before)
2 years ago
Sayan Nandan 05b8fe81c0
Add basic benches for `ql`
The benches for the protocol were fixed as well.
2 years ago
Sayan Nandan db35f8a31b
Fix update to allow single entities to be passed 2 years ago
Sayan Nandan 943125116b
Add parsing for delete queries 2 years ago
Sayan Nandan a1c85fb305
Add parsing for update queries 2 years ago