{ "cells": [ { "cell_type": "markdown", "id": "6e383e96-6c6b-46e3-a061-7b75a96ec756", "metadata": {}, "source": [ "# Storage and transactions" ] }, { "cell_type": "code", "execution_count": 1, "id": "1f17d433-96d1-47c1-8e3d-af5d996542e2", "metadata": {}, "outputs": [], "source": [ "%reload_ext pycozo.ipyext_direct" ] }, { "cell_type": "code", "execution_count": 2, "id": "5b4a724b-f8d9-44cd-8821-77edeadf772d", "metadata": {}, "outputs": [], "source": [ "%cozo_auth *******" ] }, { "cell_type": "markdown", "id": "c364f130-2d84-4563-9f35-18048025f9f9", "metadata": {}, "source": [ "## Stored relations" ] }, { "cell_type": "markdown", "id": "7b09219b-9a7c-4fa2-85ab-52b95ad69469", "metadata": {}, "source": [ "An obvious shortcoming of our previous acrobatics is that we have to carry around our love triangles network and enter it anew for every query, which leads to rapid deterioration of the `CTRL`, `C` and `V` keys. So let's fix that:" ] }, { "cell_type": "code", "execution_count": 3, "id": "5f1bc86f-eaa6-4793-8a56-78daa1288540", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] <- [['alice', 'eve'],\n", " ['bob', 'alice'],\n", " ['eve', 'alice'],\n", " ['eve', 'bob'],\n", " ['eve', 'charlie'],\n", " ['charlie', 'eve'],\n", " ['david', 'george'],\n", " ['george', 'george']]\n", " \n", ":create triangles" ] }, { "cell_type": "markdown", "id": "584b5cbb-7859-4808-9ec8-58fe7821c5e5", "metadata": {}, "source": [ "We have the _query directive_ `:relation create` together with a normal query. The results will then be stored on your disk with the name `triangles` instead of returned to you." ] }, { "cell_type": "markdown", "id": "ef5a9f83-30a5-4ed9-ae17-2d0ef6f0ef9a", "metadata": {}, "source": [ "You will receive an error if you try to run this script twice. In which case don't worry and continue." ] }, { "cell_type": "markdown", "id": "f5e58170-8f9e-4216-b6e7-22c30ea60a6d", "metadata": {}, "source": [ "Stored relations are safe from restarts and power failures. Let's query against it:" ] }, { "cell_type": "code", "execution_count": 4, "id": "2d76fb9c-ee23-4d95-b60f-b06eb5faf5b3", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 ab
0aliceeve
1bobalice
2charlieeve
3davidgeorge
4evealice
5evebob
6evecharlie
7georgegeorge
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :triangles[a, b]" ] }, { "cell_type": "markdown", "id": "4ab14596-93e4-4205-af8d-4b9da828cc1a", "metadata": {}, "source": [ "The colon `:` in front of the name tells the database that we want a _stored_ relation instead of a relation defined within the query itself." ] }, { "cell_type": "markdown", "id": "11c4a219-1e07-4f61-b40f-a602b414ddab", "metadata": {}, "source": [ "Now, Fred finally comes to the party and Fred loves Alice and Eve. We add these facts in the following way:" ] }, { "cell_type": "code", "execution_count": 5, "id": "099f6ec2-a55c-4116-ac8b-efe564657f99", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] <- [['fred', 'alice'],\n", " ['fred', 'eve']]\n", "\n", ":put triangles" ] }, { "cell_type": "code", "execution_count": 6, "id": "bcb24ccf-9956-49e8-b286-67d5464bb21e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 ab
0aliceeve
1bobalice
2charlieeve
3davidgeorge
4evealice
5evebob
6evecharlie
7fredalice
8fredeve
9georgegeorge
\n" ], "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :triangles[a, b]" ] }, { "cell_type": "markdown", "id": "016cbbd1-b65a-4b74-b7ce-f5b302b835d8", "metadata": {}, "source": [ "Notice that we used `:relation put` instead of `:relation create`. In fact, you can use `:relation put` before any call to `:relation create`. The `create` op just ensures that the insertion is into a new stored relation." ] }, { "cell_type": "markdown", "id": "649c8836-bdfa-4d18-ac6b-277993c76a3c", "metadata": {}, "source": [ "Now Eve no longer loves Alice and Charlie! Let's reflect this fact by using `retract`" ] }, { "cell_type": "code", "execution_count": 7, "id": "f6e400d6-6571-42e7-8044-5807bb789f8f", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] <- [['eve', 'charlie'],\n", " ['eve', 'alice']]\n", "\n", ":rm triangles" ] }, { "cell_type": "code", "execution_count": 8, "id": "c1ffa2b1-2ce4-4edb-a1db-ed03f2e4ddfd", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 ab
0aliceeve
1bobalice
2charlieeve
3davidgeorge
4evebob
5fredalice
6fredeve
7georgegeorge
\n" ], "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :triangles[a, b]" ] }, { "cell_type": "markdown", "id": "2c58a423-9c60-4230-ab4c-7b0dd18ab1d3", "metadata": {}, "source": [ "It is OK to retract non-existent facts, in which case the operation does nothing." ] }, { "cell_type": "markdown", "id": "ddccd02a-0184-4e3a-9cfd-6344252fe3d6", "metadata": {}, "source": [ "You can also reset the whole relation with `rederive`:" ] }, { "cell_type": "code", "execution_count": 9, "id": "c72cc224-2b50-441b-803f-f622968a6b77", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] <- [['eve', 'charlie'],\n", " ['eve', 'alice']]\n", "\n", ":replace triangles" ] }, { "cell_type": "code", "execution_count": 10, "id": "1e6b41eb-baf9-4933-8c82-5dfbc8c9d34e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 ab
0evealice
1evecharlie
\n" ], "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :triangles[a, b]" ] }, { "cell_type": "markdown", "id": "894e4980-6c3d-4f60-b77e-bd2659fda4a1", "metadata": {}, "source": [ "Only the `rederive`ed tuples remain." ] }, { "cell_type": "markdown", "id": "6cb40ca9-a82c-4712-9f15-4ecae101dcd4", "metadata": {}, "source": [ "You can see what stored relations you currently have in your database by running the following _system directive_:" ] }, { "cell_type": "code", "execution_count": 11, "id": "f051c4c6-55db-4099-b2da-1e55cea5deb3", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 namearityn_keysn_non_keysn_put_triggersn_rm_triggersn_replace_triggers
0airport11110000
1triangles220000
\n" ], "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "::relations" ] }, { "cell_type": "markdown", "id": "55b44e88-8024-4e65-92ec-f4ccdc24b1dd", "metadata": {}, "source": [ "Relations can be renamed:" ] }, { "cell_type": "code", "execution_count": 12, "id": "01baf1c4-40de-4edc-b9c8-efba9c3c2cb0", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "::relation rename triangles -> love_triangles" ] }, { "cell_type": "code", "execution_count": 13, "id": "8932c693-590a-43b3-83ee-8a99ee92278a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 6ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 namearityn_keysn_non_keysn_put_triggersn_rm_triggersn_replace_triggers
0airport11110000
1love_triangles220000
\n" ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "::relations" ] }, { "cell_type": "code", "execution_count": 14, "id": "ede55ffc-ca41-4e08-9938-e88c139c0937", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 ab
0evealice
1evecharlie
\n" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :love_triangles[a, b]" ] }, { "cell_type": "markdown", "id": "83409897-6d5d-4e4b-ac7b-54e128abc5e6", "metadata": {}, "source": [ "Now this triangles business is becoming tiring. Let's get rid of it:" ] }, { "cell_type": "code", "execution_count": 15, "id": "bc525dc4-257b-48d9-b2ee-3238ed961fad", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Completed in 0ms

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 status
0OK
\n" ], "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "::relation remove love_triangles" ] }, { "cell_type": "markdown", "id": "e744e86c-ccc5-4069-b071-bc3facea9eb3", "metadata": {}, "source": [ "Since we do not have any queries to run when nuking relations, we use a system directive instead of a query directive. Now you can no longer query the triangles:" ] }, { "cell_type": "code", "execution_count": 16, "id": "f39a7c6c-3d78-4ca4-b2d6-cc5855343200", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\u001b[31mquery::relation_not_found\u001b[0m\n", "\n", " \u001b[31m×\u001b[0m Cannot find requested stored relation 'love_triangles'\n" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "?[a, b] := :love_triangles[a, b]" ] }, { "cell_type": "markdown", "id": "c813ea00-4f66-492b-bd31-0f2ce534dccd", "metadata": {}, "source": [ "This completes all the operations on stored relations: `create`, `put`, `retract`, `rederive`. The syntax for `remove` is different from the rest for technical reasons.\n", "\n", "All these operations are _atomic_, meaning that for all the tuples they affect, either all are affected at the same time, or the operation completely fails. There is no in-between, corrupted state.\n", "\n", "Stored relations are simple, fast, and very raw. They can be used in exactly the same way as rules defined inline with the query. The way you use them is also not very different than in a traditional SQL database.\n", "\n", "Stored relations are suitable for data that has a well-defined structure at the onset, and which is loaded and updated in bulk. For example, you may have obtained from domain experts an [ontology](https://www.wikiwand.com/en/Ontology_\\(information_science\\)) in the form of a network of metadata. The ontology comes in nice tables with clear, detailed documentation. You store this ontology as a group of stored relations, and use them to extract insights from your business data. The ontology is updated periodically, and when an update comes you just use the `rederive` operation to replace the old version. Very simple and efficient. But if you need more guarantees for your data, or your data shapes change rapidly, use the triple store instead: see a later tutorial for how to use it." ] }, { "cell_type": "code", "execution_count": null, "id": "82735cec-aa4e-4a0e-8959-f522ea05c990", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" } }, "nbformat": 4, "nbformat_minor": 5 }