fluidB
/
fluexec
1
0
Fork 0
Code Pull Requests Activity

Updated documents

Browse Source 
Signed-off-by: Dvir Volk <dvirsky@gmail.com>
master
Dvir Volk 8 years ago
parent 0cfd1f5db3
commit e7495e0234
2 changed files with 1302 additions and 1608 deletions
Show Stats Download Patch File Download Diff File

523
API.md
Unescape Escape View File

@ -1,52 +1,62 @@
Redis Modules API reference manual Redis Modules: an introduction to the API
=== ===
Redis modules make possible the extension of Redis functionality using The modules documentation is composed of the following files:
external modules, by creating new Redis commands with performance and
features similar to what can be done inside the core itself.
Redis modules are dynamic libraries loaded into Redis at startup or * `INTRO.md` (this file). An overview about Redis Modules system and API. It's a good idea to start your reading here.
using the `MODULE LOAD` command. Redis exports a C API, in the * `API.md` is generated from module.c top comments of RedisMoule functions. It is a good reference in order to understand how each function works.
* `TYPES.md` covers the implementation of native data types into modules.
* `BLOCK.md` shows how to write blocking commands that will not reply immediately, but will block the client, without blocking the Redis server, and will provide a reply whenever will be possible.
Redis modules make possible to extend Redis functionality using external
modules, implementing new Redis commands at a speed and with features
similar to what can be done inside the core itself.
Redis modules are dynamic libraries, that can be loaded into Redis at
startup or using the `MODULE LOAD` command. Redis exports a C API, in the
form of a single C header file called `redismodule.h`. Modules are meant form of a single C header file called `redismodule.h`. Modules are meant
to be written in C, or any language with C binding functionality to be written in C, however it will be possible to use C++ or other languages
like C++. that have C binding functionalities.
Modules are Redis-version agnostic: a given module does not need to be Modules are designed in order to be loaded into different versions of Redis,
designed, or recompiled, in order to run with a specific version of so a given module does not need to be designed, or recompiled, in order to
Redis. In addition, they are registered using a specific Redis modules run with a specific version of Redis. For this reason, the module will
API version. The current API version is "1". register to the Redis core using a specific API version. The current API
version is "1".
This document describes the alpha version of Redis modules. API, This document is about an alpha version of Redis modules. API, functionalities
functionality and other details may change in the future. and other details may change in the future.
# Loading modules # Loading modules
In order to test a new Redis module, use the following `redis.conf` In order to test the module you are developing, you can load the module
configuration directive: using the following `redis.conf` configuration directive:
loadmodule /path/to/mymodule.so loadmodule /path/to/mymodule.so
Load a module at runtime with the following command: It is also possible to load a module at runtime using the following command:
MODULE LOAD /path/to/mymodule.so MODULE LOAD /path/to/mymodule.so
To list all loaded modules, use: In order to list all loaded modules, use:
MODULE LIST MODULE LIST
Finally, unload (or reload) a module using the following command: Finally, you can unload (and later reload if you wish) a module using the
following command:
MODULE UNLOAD mymodule MODULE UNLOAD mymodule
Note that `mymodule` is the name the module used to register itself Note that `mymodule` above is not the filename without the `.so` suffix, but
with the Redis core, and **is not** the filename without the instead, the name the module used to register itself into the Redis core.
`.so` suffix. The name can be obtained using `MODULE LIST`. It is The name can be obtained using `MODULE LIST`. However it is good practice
recommended to use the same filename for the dynamic library and module. that the filename of the dynamic library is the same as the name the module
uses to register itself into the Redis core.
# A Hello World module # The simplest module you can write
In order illustrate the basic components of a module, the following In order to show the different parts of a module, here we'll show a very
implements a command that outputs a random number. simple module that implements a command that outputs a random number.
#include "redismodule.h" #include "redismodule.h"
#include <stdlib.h> #include <stdlib.h>
@ -56,7 +66,7 @@ implements a command that outputs a random number.
return REDISMODULE_OK; return REDISMODULE_OK;
} }
int RedisModule_OnLoad(RedisModuleCtx *ctx) { int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx,"helloworld",1,REDISMODULE_APIVER_1) if (RedisModule_Init(ctx,"helloworld",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR; == REDISMODULE_ERR) return REDISMODULE_ERR;
@ -68,17 +78,21 @@ implements a command that outputs a random number.
} }
The example module has two functions. One implements a command called The example module has two functions. One implements a command called
`HELLOWORLD.RAND`. This function is specific to that module. In HELLOWORLD.RAND. This function is specific of that module. However the
addition, `RedisModule_OnLoad()` must be present in each Redis module. other function called `RedisModule_OnLoad()` must be present in each
It is the entry point for module initialization, and registers its Redis module. It is the entry point for the module to be initialized,
commands and other private data structures. register its commands, and potentially other private data structures
it uses.
In order to avoid namespace collisions, module commands should use the
dot-notation, for example, `HELLOWORLD.RAND`. Note that it is a good idea for modules to call commands with the
name of the module followed by a dot, and finally the command name,
Namespace collisions will cause `RedisModule_CreateCommand` to fail in like in the case of `HELLOWORLD.RAND`. This way it is less likely to
one or more modules. Loading will abort and an error condition have collisions.
returned.
Note that if different modules have colliding commands, they'll not be
able to work in Redis at the same time, since the function
`RedisModule_CreateCommand` will fail in one of the modules, so the module
loading will abort returning an error condition.
# Module initialization # Module initialization
@ -89,9 +103,9 @@ The following is the function prototype:
int RedisModule_Init(RedisModuleCtx *ctx, const char *modulename, int RedisModule_Init(RedisModuleCtx *ctx, const char *modulename,
int module_version, int api_version); int module_version, int api_version);
The `Init` function announces to the Redis core that the module has a The `Init` function announces the Redis core that the module has a given
given name, its version (reported by `MODULE LIST`), and that it uses name, its version (that is reported by `MODULE LIST`), and that is willing
a specific version of the API. to use a specific version of the API.
If the API version is wrong, the name is already taken, or there are other If the API version is wrong, the name is already taken, or there are other
similar errors, the function will return `REDISMODULE_ERR`, and the module similar errors, the function will return `REDISMODULE_ERR`, and the module
@ -100,15 +114,15 @@ similar errors, the function will return `REDISMODULE_ERR`, and the module
Before the `Init` function is called, no other API function can be called, Before the `Init` function is called, no other API function can be called,
otherwise the module will segfault and the Redis instance will crash. otherwise the module will segfault and the Redis instance will crash.
The second function called, `RedisModule_CreateCommand`, registers The second function called, `RedisModule_CreateCommand`, is used in order
commands with the Redis core. The following is the prototype: to register commands into the Redis core. The following is the prototype:
int RedisModule_CreateCommand(RedisModuleCtx *ctx, const char *cmdname, int RedisModule_CreateCommand(RedisModuleCtx *ctx, const char *cmdname,
RedisModuleCmdFunc cmdfunc); RedisModuleCmdFunc cmdfunc);
Most Redis modules API calls have the `context` of the module As you can see, most Redis modules API calls all take as first argument
as first argument, in order to reference the calling module's context the `context` of the module, so that they have a reference to the module
and the client executing a given command. calling it, to the command and client executing a given command, and so forth.
To create a new command, the above function needs the context, the command To create a new command, the above function needs the context, the command
name, and the function pointer of the function implementing the command, name, and the function pointer of the function implementing the command,
@ -121,9 +135,9 @@ The command function arguments are just the context, that will be passed
to all the other API calls, the command argument vector, and total number to all the other API calls, the command argument vector, and total number
of arguments, as passed by the user. of arguments, as passed by the user.
The arguments are provided as pointers to a specific data type, the As you can see, the arguments are provided as pointers to a specific data
`RedisModuleString`. This is an opaque data type with API functions type, the `RedisModuleString`. This is an opaque data type you have API
enabling access and use. Direct access to its fields is never needed. functions to access and use, direct access to its fields is never needed.
Zooming into the example command implementation, we can find another call: Zooming into the example command implementation, we can find another call:
@ -134,29 +148,52 @@ exactly like other Redis commands do, like for example `INCR` or `SCARD`.
# Setup and dependencies of a Redis module # Setup and dependencies of a Redis module
Redis modules do not depend on Redis or 3rd party libraries, nor do they Redis modules don't depend on Redis or some other library, nor they
need to be compiled with a specific `redismodule.h` file. In order need to be compiled with a specific `redismodule.h` file. In order
to create a new module, just copy a recent version of `redismodule.h` to create a new module, just copy a recent version of `redismodule.h`
in your source tree, link all the libraries you want, and create in your source tree, link all the libraries you want, and create
a dynamic library and export the `RedisModule_OnLoad()` function symbol. a dynamic library having the `RedisModule_OnLoad()` function symbol
exported.
The module will be able to load into different versions of Redis. The module will be able to load into different versions of Redis.
# Passing configuration parameters to Redis modules
When the module is loaded with the `MODULE LOAD` command, or using the
`loadmodule` directive in the `redis.conf` file, the user is able to pass
configuration parameters to the module by adding arguments after the module
file name:
loadmodule mymodule.so foo bar 1234
In the above example the strings `foo`, `bar` and `123` will be passed
to the module `OnLoad()` function in the `argv` argument as an array
of RedisModuleString pointers. The number of arguments passed is into `argc`.
The way you can access those strings will be explained in the rest of this
document. Normally the module will store the module configuration parameters
in some `static` global variable that can be accessed module wide, so that
the configuration can change the behavior of different commands.
# Working with RedisModuleString objects # Working with RedisModuleString objects
The command argument vector `argv` passed to module commands, and the The command argument vector `argv` passed to module commands, and the
return value of other module APIs functions, are of type `RedisModuleString`. return value of other module APIs functions, are of type `RedisModuleString`.
Most often Redis module strings are passed directly to other API calls. Usually you directly pass module strings to other API calls, however sometimes
However, a number of API functions enable direct access to the string you may need to directly access the string object.
object. For example,
There are a few functions in order to work with string objects:
const char *RedisModule_StringPtrLen(RedisModuleString *string, size_t *len); const char *RedisModule_StringPtrLen(RedisModuleString *string, size_t *len);
returns a pointer to `string` and sets its length in `len`. The `const` The above function accesses a string by returning its pointer and setting its
modifier prevents direct modification. length in `len`.
You should never write to a string object pointer, as you can see from the
`const` pointer qualifier.
New string objects are created using the following API: However, if you want, you can create new string objects using the following
API:
RedisModuleString *RedisModule_CreateString(RedisModuleCtx *ctx, const char *ptr, size_t len); RedisModuleString *RedisModule_CreateString(RedisModuleCtx *ctx, const char *ptr, size_t len);
@ -165,12 +202,14 @@ call to `RedisModule_FreeString()`:
void RedisModule_FreeString(RedisModuleString *str); void RedisModule_FreeString(RedisModuleString *str);
Alternatively, the automatic memory management API, covered later in However if you want to avoid having to free strings, the automatic memory
this document, can be employed to automatically free string handles. management, covered later in this document, can be a good alternative, by
doing it for you.
Note that the strings provided via the argument vector `argv` never need Note that the strings provided via the argument vector `argv` never need
to be freed. Only strings created within a module need to be freed, or to be freed. You only need to free new strings you create, or new strings
new strings returned by other APIs where specified. returned by other APIs, where it is specified that the returned string must
be freed.
## Creating strings from numbers or parsing strings as numbers ## Creating strings from numbers or parsing strings as numbers
@ -189,44 +228,46 @@ Similarly in order to parse a string as a number:
## Accessing Redis keys from modules ## Accessing Redis keys from modules
Most Redis modules, in order to be useful, have to interact with the Redis Most Redis modules, in order to be useful, have to interact with the Redis
data space. An exception would be an ID generator that never accesses data space (this is not always true, for example an ID generator may
Redis keys. Redis modules have two different APIs in order to never touch Redis keys). Redis modules have two different APIs in order to
access the Redis data space. A **low level API** providing very access the Redis data space, one is a low level API that provides very
fast access and a set of functions to manipulate Redis data structures. fast access and a set of functions to manipulate Redis data structures.
A **high level API** is provided to allow calling Redis commands and The other API is more high level, and allows to call Redis commands and
retrieving results, similar to how Lua scripts access Redis. fetch the result, similarly to how Lua scripts access Redis.
The high level API is also useful in order to access Redis functionalities The high level API is also useful in order to access Redis functionalities
that are not available as APIs. that are not available as APIs.
In general, module developers should prefer the low level API, because commands In general modules developers should prefer the low level API, because commands
implemented using the low level API run at a speed comparable to the speed implemented using the low level API run at a speed comparable to the speed
of native Redis commands. However there are definitely use cases for the of native Redis commands. However there are definitely use cases for the
higher level API. For example often the bottleneck could be processing the higher level API. For example often the bottleneck could be processing the
data and not accessing it. data and not accessing it.
Also note that in some cases the low level API is is as simple as the Also note that sometimes using the low level API is not harder compared to
high level API. the higher level one.
# Calling Redis commands # Calling Redis commands
The high level API to access Redis combines the `RedisModule_Call()` The high level API to access Redis is the sum of the `RedisModule_Call()`
function with the functions needed to access the reply object returned function, together with the functions needed in order to access the
by `Call()`. reply object returned by `Call()`.
`RedisModule_Call` uses a special calling convention, with a format `RedisModule_Call` uses a special calling convention, with a format specifier
specifier used to define the types of objects passed as arguments. that is used to specify what kind of objects you are passing as arguments
to the function.
Redis commands are invoked using a command name and a list of arguments. Redis commands are invoked just using a command name and a list of arguments.
However, when calling commands, the arguments may originate from different However when calling commands, the arguments may originate from different
kind of strings: null-terminated C strings, RedisModuleString objects as kind of strings: null-terminated C strings, RedisModuleString objects as
received from the `argv` parameter in the command implementation, binary received from the `argv` parameter in the command implementation, binary
safe C buffers with a pointer and a length, and so forth. safe C buffers with a pointer and a length, and so forth.
To call `INCRBY` using as first argument (the key) a string received in For example if I want to call `INCRBY` using a first argument (the key)
the argument vector `argv`, which is an array of RedisModuleString a string received in the argument vector `argv`, which is an array
object pointers, and a C string representing the number "10" as a second of RedisModuleString object pointers, and a C string representing the
argument (the increment), use the following function call: number "10" as second argument (the increment), I'll use the following
function call:
RedisModuleCallReply *reply; RedisModuleCallReply *reply;
reply = RedisModule_Call(ctx,"INCR","sc",argv[1],"10"); reply = RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
@ -248,15 +289,13 @@ This is the full list of format specifiers:
* **v** -- Array of RedisModuleString objects. * **v** -- Array of RedisModuleString objects.
* **!** -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing. * **!** -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
The function returns either a `RedisModuleCallReply` object on success, The function returns a `RedisModuleCallReply` object on success, on
or NULL on error. error NULL is returned.
NULL is returned when the command name is invalid, the format specifier NULL is returned when the command name is invalid, the format specifier uses
uses characters that are not recognized, or when the command is called characters that are not recognized, or when the command is called with the
with the wrong number of arguments. In the above cases the `errno` var wrong number of arguments. In the above cases the `errno` var is set to `EINVAL`. NULL is also returned when, in an instance with Cluster enabled, the target
is set to `EINVAL`. NULL is also returned when, in an instance with keys are about non local hash slots. In this case `errno` is set to `EPERM`.
Cluster enabled, the target keys are non local hash slots. In this
case `errno` is set to `EPERM`.
## Working with RedisModuleCallReply objects. ## Working with RedisModuleCallReply objects.
@ -288,8 +327,7 @@ is used:
size_t reply_len = RedisModule_CallReplyLength(reply); size_t reply_len = RedisModule_CallReplyLength(reply);
In order to obtain the value of an integer reply, the following function In order to obtain the value of an integer reply, the following function is used, as already shown in the example above:
is used, as already shown in the example above:
long long reply_integer_val = RedisModule_CallReplyInteger(reply); long long reply_integer_val = RedisModule_CallReplyInteger(reply);
@ -304,51 +342,53 @@ Sub elements of array replies are accessed this way:
The above function returns NULL if you try to access out of range elements. The above function returns NULL if you try to access out of range elements.
Strings and errors (which are like strings but with a different type) can Strings and errors (which are like strings but with a different type) can
be accessed in the following way, making sure to never write to be accessed using in the following way, making sure to never write to
the resulting pointer (that is returned as as const pointer so that the resulting pointer (that is returned as as `const` pointer so that
misusing must be pretty explicit):: misusing must be pretty explicit):
size_t len; size_t len;
const char *ptr = RedisModule_CallReplyStringPtr(reply,&len); char *ptr = RedisModule_CallReplyStringPtr(reply,&len);
If the reply type is not a string or an error, NULL is returned. If the reply type is not a string or an error, NULL is returned.
RedisCallReply objects are not the same as module string objects RedisCallReply objects are not the same as module string objects
(RedisModuleString types). When an API function expects a module string, (RedisModuleString types). However sometimes you may need to pass replies
the following function can be employed to create a new of type string or integer, to API functions expecting a module string.
`RedisModuleString` object from a call reply of type string, error or
integer:
RedisModuleString *mystr = RedisModule_CreateStringFromCallReply(myreply); When this is the case, you may want to evaluate if using the low level
API could be a simpler way to implement your command, or you can use
the following function in order to create a new string object from a
call reply of type string, error or integer:
Alternatively, one could evaluate whether using the low level API is as RedisModuleString *mystr = RedisModule_CreateStringFromCallReply(myreply);
simple (and potentially faster).
If the reply is not of the right type, NULL is returned. The returned If the reply is not of the right type, NULL is returned.
string object should be released with `RedisModule_FreeString()`, or by The returned string object should be released with `RedisModule_FreeString()`
enabling automatic memory management (see section below). as usually, or by enabling automatic memory management (see corresponding
section).
# Releasing call reply objects # Releasing call reply objects
Reply objects must be freed using `RedisModule_FreeCallRelpy`. For arrays, Reply objects must be freed using `RedisModule_FreeCallReply`. For arrays,
only the top level reply needs to be freed, but not the nested replies. you need to free only the top level reply, not the nested replies.
Currently, the module implementation provides protection in order to avoid Currently the module implementation provides a protection in order to avoid
crashing if a nested reply object is freed on error--however, *this crashing if you free a nested reply object for error, however this feature
protective feature may not be available in future versions, and should is not guaranteed to be here forever, so should not be considered part
not be considered part of the API*. of the API.
Automatic memory management can take care of freeing replies (see If you use automatic memory management (explained later in this document)
section below). Alternatively, memory can be released ASAP. you don't need to free replies (but you still could if you wish to release
memory ASAP).
## Returning values from Redis commands ## Returning values from Redis commands
Like normal Redis commands, new commands implemented via modules must be Like normal Redis commands, new commands implemented via modules must be
able to return values to the caller. Towards this end, The API exports a able to return values to the caller. The API exports a set of functions for
set of functions in order to return the usual Redis protocol types, and this goal, in order to return the usual types of the Redis protocol, and
arrays of such types (as elemented). Also, errors can be returned with any arrays of such types as elemented. Also errors can be returned with any
error code and string, where the error code is the initial uppercase error string and code (the error code is the initial uppercase letters in
letters in the error message--for example, the "BUSY" string in the the error message, like the "BUSY" string in the "BUSY the sever is busy" error
"BUSY the sever is busy" error message. message).
All the functions to send a reply to the client are called All the functions to send a reply to the client are called
`RedisModule_ReplyWith<something>`. `RedisModule_ReplyWith<something>`.
@ -369,8 +409,8 @@ We already saw how to reply with a long long in the examples above:
RedisModule_ReplyWithLongLong(ctx,12345); RedisModule_ReplyWithLongLong(ctx,12345);
To reply with a simple string like "OK", that can't contain binary To reply with a simple string, that can't contain binary values or newlines,
values or newlines, use: (so it's suitable to send small words, like "OK") we use:
RedisModule_ReplyWithSimpleString(ctx,"OK"); RedisModule_ReplyWithSimpleString(ctx,"OK");
@ -381,18 +421,18 @@ two different functions:
int RedisModule_ReplyWithString(RedisModuleCtx *ctx, RedisModuleString *str); int RedisModule_ReplyWithString(RedisModuleCtx *ctx, RedisModuleString *str);
The first function gets a C pointer and length. The second a RedisModuleString The first function gets a C pointer and length. The second a RedisMoudleString
object. Use one or the other depending on the source type you have at hand. object. Use one or the other depending on the source type you have at hand.
In order to reply with an array, use a function to emit the array In order to reply with an array, you just need to use a function to emit the
length, followed by as many calls to the above functions as there are array length, followed by as many calls to the above functions as the number
elements in the array: of elements of the array are:
RedisModule_ReplyWithArray(ctx,2); RedisModule_ReplyWithArray(ctx,2);
RedisModule_ReplyWithStringBuffer(ctx,"age",3); RedisModule_ReplyWithStringBuffer(ctx,"age",3);
RedisModule_ReplyWithLongLong(ctx,22); RedisModule_ReplyWithLongLong(ctx,22);
Returning nested arrays is easy--the nested array element just uses another To return nested arrays is easy, your nested array element just uses another
call to `RedisModule_ReplyWithArray()` followed by the calls to emit the call to `RedisModule_ReplyWithArray()` followed by the calls to emit the
sub array elements. sub array elements.
@ -401,16 +441,16 @@ sub array elements.
Sometimes it is not possible to know beforehand the number of items of Sometimes it is not possible to know beforehand the number of items of
an array. As an example, think of a Redis module implementing a FACTOR an array. As an example, think of a Redis module implementing a FACTOR
command that given a number outputs the prime factors. Instead of command that given a number outputs the prime factors. Instead of
factorializing the number, storing the prime factors in an array, and factorializing the number, storing the prime factors into an array, and
then producing the command reply, a better solution is to start an array later produce the command reply, a better solution is to start an array
reply where the length is not known, and set it later. This is accomplished reply where the length is not known, and set it later. This is accomplished
with a special argument to `RedisModule_ReplyWithArray()`: with a special argument to `RedisModule_ReplyWithArray()`:
RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_ARRAY_LEN); RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_ARRAY_LEN);
The above call starts an array reply, and more `ReplyWith` calls can The above call starts an array reply so we can use other `ReplyWith` calls
then be used to produce the array items. Finally in order to set the in order to produce the array items. Finally in order to set the length
length use the following call: se use the following call:
RedisModule_ReplySetArrayLength(ctx, number_of_items); RedisModule_ReplySetArrayLength(ctx, number_of_items);
@ -428,7 +468,7 @@ to this:
Another common use case for this feature is iterating over the arrays of Another common use case for this feature is iterating over the arrays of
some collection and only returning the ones passing some kind of filtering. some collection and only returning the ones passing some kind of filtering.
It is possible to have multiple nested arrays with a postponed reply. It is possible to have multiple nested arrays with postponed reply.
Each call to `SetArray()` will set the length of the latest corresponding Each call to `SetArray()` will set the length of the latest corresponding
call to `ReplyWithArray()`: call to `ReplyWithArray()`:
@ -444,7 +484,8 @@ This creates a 100 items array having as last element a 10 items array.
# Arity and type checks # Arity and type checks
Often commands need to check that the number of arguments and type of the key Often commands need to check that the number of arguments and type of the key
is correct. To report a wrong arity, use `RedisModule_WrongArity()`: is correct. In order to report a wrong arity, there is a specific function
called `RedisModule_WrongArity()`. The usage is trivial:
if (argc != 2) return RedisModule_WrongArity(ctx); if (argc != 2) return RedisModule_WrongArity(ctx);
@ -466,8 +507,8 @@ is of the expected type, or if it's empty.
## Low level access to keys ## Low level access to keys
Low level access to keys enable operations on value objects associated Low level access to keys allow to perform operations on value objects associated
with keys directly, with a speed similar to what Redis uses internally to to keys directly, with a speed similar to what Redis uses internally to
implement the built-in commands. implement the built-in commands.
Once a key is opened, a key pointer is returned that will be used with all the Once a key is opened, a key pointer is returned that will be used with all the
@ -477,44 +518,39 @@ associated value.
Because the API is meant to be very fast, it cannot do too many run-time Because the API is meant to be very fast, it cannot do too many run-time
checks, so the user must be aware of certain rules to follow: checks, so the user must be aware of certain rules to follow:
* Opening the same key multiple times where at least one instance is * Opening the same key multiple times where at least one instance is opened for writing, is undefined and may lead to crashes.
opened for writing, is undefined and may lead to crashes. * While a key is open, it should only be accessed via the low level key API. For example opening a key, then calling DEL on the same key using the `RedisModule_Call()` API will result into a crash. However it is safe to open a key, perform some operation with the low level API, closing it, then using other APIs to manage the same key, and later opening it again to do some more work.
* While a key is open, it should only be accessed via the low level key
API. For example opening a key, then calling DEL on the same key using In order to open a key the `RedisModule_OpenKey` function is used. It returns
the `RedisModule_Call()` API will result in a crash. However it is safe a key pointer, that we'll use with all the next calls to access and modify
to open a key, perform some operation with the low level API, close it,
then use other APIs to manage the same key, and later opening it again
to do some more work.
In order to open a key the `RedisModule_OpenKey` function is used. It
returns a key pointer, used in subsequent calls to access and modify
the value: the value:
RedisModuleKey *key; RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ); key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ);
The second argument is the key name, and must be a `RedisModuleString` The second argument is the key name, that must be a `RedisModuleString` object.
object. The third argument is the mode: `REDISMODULE_READ` or The third argument is the mode: `REDISMODULE_READ` or `REDISMODULE_WRITE`.
`REDISMODULE_WRITE`. It is possible to use `|` to bitwise OR the two It is possible to use `|` to bitwise OR the two modes to open the key in
modes to open the key in both modes. Currently, a key opened for writing both modes. Currently a key opened for writing can also be accessed for reading
can also be accessed for reading but this is to be considered an but this is to be considered an implementation detail. The right mode should
implementation detail. The right mode should be used in sane modules. be used in sane modules.
You can open non-existent keys for writing, and the keys will be created You can open non exisitng keys for writing, since the keys will be created
when an attempt to write to the key is performed. However when opening when an attempt to write to the key is performed. However when opening keys
keys just for reading, `RedisModule_OpenKey` will return NULL if the key just for reading, `RedisModule_OpenKey` will return NULL if the key does not
does not exist. exist.
A key is closed by calling: Once you are done using a key, you can close it with:
RedisModule_CloseKey(key); RedisModule_CloseKey(key);
With automatic memory management enabled, Redis will close all open keys Note that if automatic memory management is enabled, you are not forced to
when the module function returns close keys. When the module function returns, Redis will take care to close
all the keys which are still open.
## Getting the key type ## Getting the key type
In order to obtain the value of a key, use `RedisModule_KeyType()`: In order to obtain the value of a key, use the `RedisModule_KeyType()` function:
int keytype = RedisModule_KeyType(key); int keytype = RedisModule_KeyType(key);
@ -527,9 +563,9 @@ It returns one of the following values:
REDISMODULE_KEYTYPE_SET REDISMODULE_KEYTYPE_SET
REDISMODULE_KEYTYPE_ZSET REDISMODULE_KEYTYPE_ZSET
The above are standard Redis key types, with the addition of an empty The above are just the usual Redis key types, with the addition of an empty
type signalling the key pointer is associated with an empty key that type, that signals the key pointer is associated with an empty key that
does not exist yet. does not yet exists.
## Creating new keys ## Creating new keys
@ -550,9 +586,9 @@ Just use:
The function returns `REDISMODULE_ERR` if the key is not open for writing. The function returns `REDISMODULE_ERR` if the key is not open for writing.
Note that after a key gets deleted, it is setup in order to be targeted Note that after a key gets deleted, it is setup in order to be targeted
by new key commands. For example `RedisModule_KeyType()` will return it by new key commands. For example `RedisModule_KeyType()` will return it is
as an empty key, and writing to it will create a new key, possibly of an empty key, and writing to it will create a new key, possibly of another
another type (depending on the API used). type (depending on the API used).
## Managing key expires (TTLs) ## Managing key expires (TTLs)
@ -589,7 +625,7 @@ performed.
## Obtaining the length of values ## Obtaining the length of values
There is a single function in order to retrieve the length of the value There is a single function in order to retrieve the length of the value
associated with an open key. The returned length is value-specific, and is associated to an open key. The returned length is value-specific, and is
the string length for strings, and the number of elements for the aggregated the string length for strings, and the number of elements for the aggregated
data types (how many elements there is in a list, set, sorted set, hash). data types (how many elements there is in a list, set, sorted set, hash).
@ -608,11 +644,11 @@ The function works exactly like the Redis `SET` command itself, that is, if
there is a prior value (of any type) it will be deleted. there is a prior value (of any type) it will be deleted.
Accessing existing string values is performed using DMA (direct memory Accessing existing string values is performed using DMA (direct memory
access) for speed. The API will return a pointer and a length, so that access) for speed. The API will return a pointer and a length, so that's
it's possible to access and, if needed, modify the string directly. possible to access and, if needed, modify the string directly.
size_t len, j; size_t len, j;
char *myptr = RedisModule_StringDMA(key,REDISMODULE_WRITE,&len); char *myptr = RedisModule_StringDMA(key,&len,REDISMODULE_WRITE);
for (j = 0; j < len; j++) myptr[j] = 'A'; for (j = 0; j < len; j++) myptr[j] = 'A';
In the above example we write directly on the string. Note that if you want In the above example we write directly on the string. Note that if you want
@ -642,31 +678,53 @@ It's possible to push and pop values from list values:
int RedisModule_ListPush(RedisModuleKey *key, int where, RedisModuleString *ele); int RedisModule_ListPush(RedisModuleKey *key, int where, RedisModuleString *ele);
RedisModuleString *RedisModule_ListPop(RedisModuleKey *key, int where); RedisModuleString *RedisModule_ListPop(RedisModuleKey *key, int where);
The `where` argument specifies whether to push or pop from the tail In both the APIs the `where` argument specifies if to push or pop from tail
or head, using the following macros: or head, using the following macros:
REDISMODULE_LIST_HEAD REDISMODULE_LIST_HEAD
REDISMODULE_LIST_TAIL REDISMODULE_LIST_TAIL
Elements returned by `RedisModule_ListPop()` are like strings created with Elements returned by `RedisModule_ListPop()` are like strings craeted with
`RedisModule_CreateString()`, they must be released with `RedisModule_CreateString()`, they must be released with
`RedisModule_FreeString()` or by enabling automatic memory management. `RedisModule_FreeString()` or by enabling automatic memory management.
## Set type API ## Set type API
See [FUNCTIONS.md](FUNCTIONS.md) Work in progress.
## Sorted set type API ## Sorted set type API
See [FUNCTIONS.md](FUNCTIONS.md) Documentation missing, please refer to the top comments inside `module.c`
for the following functions:
* `RedisModule_ZsetAdd`
* `RedisModule_ZsetIncrby`
* `RedisModule_ZsetScore`
* `RedisModule_ZsetRem`
And for the sorted set iterator:
* `RedisModule_ZsetRangeStop`
* `RedisModule_ZsetFirstInScoreRange`
* `RedisModule_ZsetLastInScoreRange`
* `RedisModule_ZsetFirstInLexRange`
* `RedisModule_ZsetLastInLexRange`
* `RedisModule_ZsetRangeCurrentElement`
* `RedisModule_ZsetRangeNext`
* `RedisModule_ZsetRangePrev`
* `RedisModule_ZsetRangeEndReached`
## Hash type API ## Hash type API
See [FUNCTIONS.md](FUNCTIONS.md) Documentation missing, please refer to the top comments inside `module.c`
for the following functions:
* `RedisModule_HashSet`
* `RedisModule_HashGet`
## Iterating aggregated values ## Iterating aggregated values
See [FUNCTIONS.md](FUNCTIONS.md) Work in progress.
# Replicating commands # Replicating commands
@ -676,41 +734,43 @@ it is important for module commands to handle their replication in a consistent
way. way.
When using the higher level APIs to invoke commands, replication happens When using the higher level APIs to invoke commands, replication happens
automatically when using the "!" modifier in the format string of automatically if you use the "!" modifier in the format string of
`RedisModule_Call()` as in the following example: `RedisModule_Call()` as in the following example:
reply = RedisModule_Call(ctx,"INCR","!sc",argv[1],"10"); reply = RedisModule_Call(ctx,"INCR","!sc",argv[1],"10");
The bang is not parsed as a format specifier, but it internally flags As you can see the format specifier is `"!sc"`. The bang is not parsed as a
the command as "must replicate". format specifier, but it internally flags the command as "must replicate".
For more complex scenarios than that, use the low level API. In this If you use the above programming style, there are no problems.
case, if there are no side effects in the command execution, and However sometimes things are more complex than that, and you use the low level
it always consistently performs the same work, it is possible API. In this case, if there are no side effects in the command execution, and
to replicate the command verbatim as the user executed it. To do that, it consistently always performs the same work, what is possible to do is to
call the following function: replicate the command verbatim as the user executed it. To do that, you just
need to call the following function:
RedisModule_ReplicateVerbatim(ctx); RedisModule_ReplicateVerbatim(ctx);
When you using the above API, do not use any other replication function When you use the above API, you should not use any other replication function
since they are not guaranteed to mix well. since they are not guaranteed to mix well.
An alternative is to tell Redis exactly which commands to replicate as However this is not the only option. It's also possible to exactly tell
the effect of the command execution, using an API similar to Redis what commands to replicate as the effect of the command execution, using
`RedisModule_Call()`. Instead of calling the command they are sent to an API similar to `RedisModule_Call()` but that instead of calling the command
the AOF / slaves stream. For example: sends it to the AOF / slaves stream. Example:
RedisModule_Replicate(ctx,"INCRBY","cl","foo",my_increment); RedisModule_Replicate(ctx,"INCRBY","cl","foo",my_increment);
It's possible to call `RedisModule_Replicate` multiple times, and each It's possible to call `RedisModule_Replicate` multiple times, and each
will emit a command. The entire sequence emitted is wrapped in a will emit a command. All the sequence emitted is wrapped between a
`MULTI/EXEC` transaction, so that the AOF and replication effects are the `MULTI/EXEC` transaction, so that the AOF and replication effects are the
same as executing a single command. same as executing a single command.
It is not a good idea to mix both forms of replication if there are Note that `Call()` replication and `Replicate()` replication have a rule,
simpler alternatives. However, when mixing note that commands replicated in case you want to mix both forms of replication (not necessarily a good
with `Call()` are always the first emitted in the final `MULTI/EXEC` idea if there are simpler approaches). Commands replicated with `Call()`
block, while all the commands emitted with `Replicate()` will follow. are always the first emitted in the final `MULTI/EXEC` block, while all
the commands emitted with `Replicate()` will follow.
# Automatic memory management # Automatic memory management
@ -718,20 +778,20 @@ Normally when writing programs in the C language, programmers need to manage
memory manually. This is why the Redis modules API has functions to release memory manually. This is why the Redis modules API has functions to release
strings, close open keys, free replies, and so forth. strings, close open keys, free replies, and so forth.
However since commands are executed in a contained environment and However given that commands are executed in a contained environment and
with a set of strict APIs, Redis is able to provide automatic memory with a set of strict APIs, Redis is able to provide automatic memory management
management to modules, at the cost of some performance (most of the to modules, at the cost of some performance (most of the time, a very low
time, a very low cost). cost).
When automatic memory management is enabled, there is **no need to**: When automatic memory management is enabled:
1. Close open keys. 1. You don't need to close open keys.
2. Free replies. 2. You don't need to free replies.
3. Free RedisModuleString objects. 3. You don't need to free RedisModuleString objects.
Automatic and manual memory management can be combined. For example, However you can still do it, if you want. For example, automatic memory
automatic memory management may be active, but inside a loop allocating management may be active, but inside a loop allocating a lot of strings,
a lot of strings, you may still want to free strings no longer used. you may still want to free strings no longer used.
In order to enable automatic memory management, just call the following In order to enable automatic memory management, just call the following
function at the start of the command implementation: function at the start of the command implementation:
@ -742,15 +802,56 @@ Automatic memory management is usually the way to go, however experienced
C programmers may not use it in order to gain some speed and memory usage C programmers may not use it in order to gain some speed and memory usage
benefit. benefit.
# Allocating memory into modules
Normal C programs use `malloc()` and `free()` in order to allocate and
release memory dynamically. While in Redis modules the use of malloc is
not technically forbidden, it is a lot better to use the Redis Modules
specific functions, that are exact replacements for `malloc`, `free`,
`realloc` and `strdup`. These functions are:
void *RedisModule_Alloc(size_t bytes);
void* RedisModule_Realloc(void *ptr, size_t bytes);
void RedisModule_Free(void *ptr);
void RedisModule_Calloc(size_t nmemb, size_t size);
char *RedisModule_Strdup(const char *str);
They work exactly like their `libc` equivalent calls, however they use
the same allocator Redis uses, and the memory allocated using these
functions is reported by the `INFO` command in the memory section, is
accounted when enforcing the `maxmemory` policy, and in general is
a first citizen of the Redis executable. On the contrar, the method
allocated inside modules with libc `malloc()` is transparent to Redis.
Another reason to use the modules functions in order to allocate memory
is that, when creating native data types inside modules, the RDB loading
functions can return deserialized strings (from the RDB file) directly
as `RedisModule_Alloc()` allocations, so they can be used directly to
populate data structures after loading, instead of having to copy them
to the data structure.
## Pool allocator
Sometimes in commands implementations, it is required to perform many
small allocations that will be not retained at the end of the command
execution, but are just functional to execute the command itself.
This work can be more easily accomplished using the Redis pool allocator:
void *RedisModule_PoolAlloc(RedisModuleCtx *ctx, size_t bytes);
It works similarly to `malloc()`, and returns memory aligned to the
next power of two of greater or equal to `bytes` (for a maximum alignment
of 8 bytes). However it allocates memory in blocks, so it the overhead
of the allocations is small, and more important, the memory allocated
is automatically released when the command returns.
So in general short living allocations are a good candidates for the pool
allocator.
# Writing commands compatible with Redis Cluster # Writing commands compatible with Redis Cluster
Work in progress, see [FUNCTIONS.md](FUNCTIONS.md) for the following API: Documentation missing, please check the following functions inside `module.c`:
RedisModule_IsKeysPositionRequest(ctx); RedisModule_IsKeysPositionRequest(ctx);
RedisModule_KeyAtPos(ctx,pos); RedisModule_KeyAtPos(ctx,pos);
Command implementations, on keys position request, must reply with
`REDISMODULE_KEYPOS_OK` to signal the request was processed, otherwise
Cluster returns an error for those module commands that are not able to
describe the position of keys.

2363
FUNCTIONS.md
View File

File diff suppressed because it is too large Load Diff
Write Preview
Loading…
Cancel
Save