Redis key schema

Although the official Redis tutorial on data types1 recommends using a consistent schema when naming keys, Redis itself does not have any schema checking or validation functions although some basic validation can be done through the use of the EXISTS and TYPE Redis commands. If your application requires that a Redis key with a certain type exists in the instance, checking for the key's existence is easily accomplished with the EXISTS command followed by the subsequent TYPE command to confirm that the key is the expected Redis data structure stored in that key location. Beyond these two commands, validating the Redis key syntax and structure requires client-side code.

Adding this additional validation logic layer to your application may be useful if your Redis application is to be shared across different systems and organizations. An accurate and detailed Redis key schema can greatly assist you and the application developers and operators in troubleshooting or debugging problems. Another avenue to validate your Redis Key schema would be to include specific unit tests in your Redis application that test for boundary conditions, schema key syntax, and structure, along with the expected data structures for each validated key. The third option for validating your Redis key schema is to use a DTD or another XML-based validation of your key structure or to use a new schema validation technology such as JSON Schema available at http://json-schema.org/.

Options for validating Redis keys

A good key schema should also provide guidance for adding new Redis keys to an existing Redis-based application. There should not be any mysteries about what the name of a new Redis key should be if the schema is descriptive and consistent. Know and use both singular and plural forms of nouns to identify what and how many of an entity is being saved to Redis. For example, book:1 could be a Redis hash storing field related to a single book, while the Redis key books:sci-fiction could be a set of all books that are classified as part of the science fiction genre. A sorted set could be used for book sales ranking with the books:sales-rank key name with the number of books sold as the weight or the sorted set score and the book key as the value.

An example of a text-based Redis schema for a simple book application could look like the following:

Adding key schema documentation as part of your project's source code repository is a good practice to follow, even with simple, one-off Redis projects.

Key delimiters and naming conventions

You'll notice that in the last example that colon ':' is being used as a key delimiter. The colon is one of the suggested delimiters for compound Redis keys. This is just a convention, and you should feel free to use any other delimiter in your own application. For a web application, using a forward slash '/' may make better sense (although of course, you should never pass public URL requests directly to Redis without some preprocessing to sanitize the user's request).

Another delimiter for Redis keys is the period '.' making it easier to map to a common object-oriented syntax favored by many of the most popular programming languages such as C++, Object C, Python, Swift, and Ruby. There is nothing precluding or preventing you from mixing delimiters in your Redis key schema as long as the delimiter use is consistent and properly documented. The goal in choosing which delimiters to use should be intelligibility and consistency both to you and the eventual users of your application.

Where an effective Redis key schema excels is in establishing naming conventions relating keys together. These relationships loosely couple Redis keys together onto which application and business logic can be applied through client code. A Redis key schema weaves a narrative relating your data in a way that is intelligible to you and that meets your users' needs. Take, for example, the earlier book key schema and expanding the requirements to include other media types in your Redis databases, running a KEYS command from the Redis command-line tool, and doing some formatting. These display a pattern and implicit relationships through the Redis key schema:

all:sales-rank
global:book
book:1
book:2
book:3
books:genre:popular-fiction
books:genre:sci-fiction
books:format:ebook
books:format:paperback
books:sales-rank
global:film
film:1
film:2
films:genre:comedy
films:genre:drama
films:format:bluray
films:format:dvd
films:sales-rank

In this Redis application, both book and film provide the base prefix from which supporting data structures are associated with either a singular book or film key or a collection that contains additional entity hashes.

Each work is a hash with the Redis key being the prefix along with a global counter. Other supporting data structures, in this case, a book's or film's genre and format, are Redis sets that store all the book or film keys that are classified as belonging to a particular genre or format. For example, properties of Isaac Asimov's Foundation would be stored in the book:2 hash, and book:2 would also be a member of the books:genre:sci-fiction and books:format:paperback sets with entries in both the books:sales-rank and all:sales-rank sorted sets. Likewise, Orson Well's Citizen Kane, would be stored in the film:1 hash, with membership in the films:genre:drama and films:format:dvd sets and entries in the films:sales-rank and all:sales-rank sorted sets.

A common requirement in an application is the need to retrieve a collection of values on the basis of common characteristics. In the book example, we may be tempted to retrieve all the book genres with the KEYS command and a books:genre:* pattern. The use of the Redis KEYS command is highly discouraged for applications running in production as Redis needs to iterate through every single key in the datastore. With a consistent naming convention and appropriate data structures such as set, hash, or sorted sets, your application should not need to use the KEYS command for retrieving values. The SCAN command, with an option for retrieving values from Redis should not be thought of as a replacement for the KEYS command. The SCAN command extracts a random slice of key and then, applies any existing pattern provided with the MATCH option to the random slice. Going back to the previous example, the following use of SCAN in a running Redis-cli program only works if you have a small datastore:

127.0.0.1:6379> SCAN 0 MATCH books:genre*
1) "0"
2) 1) "books:genre:popular-fiction"
   2) "books:genre:mystery"
   3) "books:genre:teen"
   4) "books:genre:sci-fiction"
   5) "books:genre:fantasy"
   6) "books:genre:romance"

If your datastore is larger, using the same SCAN command may not return any matches or only a small subset of the total number of matches. Much better would be to store all the genre keys in a books:genres set that your application uses as an index to quickly retrieve all of the book genre keys with an SMEMBERS command:

127.0.0.1:6379> SMEMBERS books:genres
1) "books:format:ebook"
2) "books:genre:popular-fiction"
3) "books:format:paperback"
4) "books:genre:sci-fiction"
5) "books:sales-rank"

Testing your relationships and how they relate to each other through your Redis key naming conventions will depend on a number of factors, including if your application interacts directly with an active Redis instance. Adding unit tests that specifically check your key delimiters as well as the naming conventions in your Redis application ensures that the assumptions and requirements that your application depends upon are accurately represented in the data being stored in your Redis database. Here is an example of a unit test in Python that tests for a colon delimiter for the Redis key schema book example (the full example is available for download at http://mastering-redis.com).

def  test_delimiter(self):"""Method tests for a colon in  Redis keys in the datastore."""first_key = self.test_db.scan(0, "book*", 1)[1][0].decode()self.assertTrue(first_key.startswith("book:"))

The first line uses the Python Redis client to run a book:* pattern with the scan starting with an initial cursor of 0 and a count of one command returning the first instance and decoding the string to unicode while saving it to the first_key variable. The second line asserts that first_key starts with the prefix and delimiter that we expected in our Redis schema. We'll next take a different scenario and start by first outlining the major types of data that we want to capture in our Redis application and then, discussing how we can use our business requirements and nomenclature to create the corresponding Redis keys and data structures.