Deconstructing a Redis object mapper

Redis's rich ecosystem offers a number of object mappers for Redis that hide the key naming management from the designer and the user while offering, through client-side code, a functionality that may be present in other data storage technologies. Examining how a Redis object mapper implements this functionality with a particular pattern of keys and data structures can help you learn about existing patterns and allow you to extend and improve your own Redis-based applications. Using a Redis object mapper can also be helpful if you do not want to re-implement a functionality that may already exist and run it in production environments in your code base. A few of the more popular programming languages have these object mapper projects that all provide ways to persist object semantics and data in Redis while offering more object-oriented methodologies and techniques for the developer who may be more familiar with these techniques and ideas in their preferred programing language. These object mappers manipulate Redis keys and values by using the nomenclature and conventions of the programming language that the object mapper has developed while hopefully reducing the maintenance and the training overhead for Redis-based solutions in the organization.

For the node.js Redis object mapper called Nohm (available at https://github.com/maritz/nohm/), the Redis schema is created through a JavaScript object model. Returning to the previous paper-product web storefront example, modeling a stationery entity with Nohm first requires defining a stationary JavaScript model with color, height, and width properties by using the following code:

nohm.model('Stationary', {
 properties: {
   color: {
    type: 'string',
    unique: false,
    validations: [
    'notEmpty'
   ]
  },
  height: {
   type: 'string',
   unique: false
  },
  sheets: {
   type: 'integer',
   defaultValue: 20
  },
  width: {
   type: 'string',
   unique: false
  }
 }
});

Creating an equivalent stationary object in the stationery:1 hash from the previous section generates the following Redis commands and values on a running Redis database for a new stationary Javascript object with color, width, and height being set to the same values as in stationery:1. Running the MONITOR command from the redis-cli program provides the following output:

1431204654.386408 [0 127.0.0.1:61217] "info"1431204654.404005 [0 127.0.0.1:61217] "get" "paper:meta:version:Stationary"1431204654.405394 [0 127.0.0.1:61217] "sismember" "paper:idsets:Stationary" "-1431204204839"1431204654.406943 [0 127.0.0.1:61217] "set" "paper:meta:version:Stationary" "1bf8ca04e698cd589baa17c661498b1109f8d65c"1431204654.407516 [0 127.0.0.1:61217] "set" 
"paper:meta:idGenerator:Stationary" "default"1431204654.407547 [0 127.0.0.1:61217] "set" "paper:meta:properties:Stationary" "{\"color\":{\"type\":\"string\",\"unique\":false,\"validations\":[\"notEmpty\"]},\"height\":{\"type\":\"string\",\"unique\":false},\"sheets\":{\"type\":\"integer\",\"defaultValue\":20},\"width\":{\"type\":\"string\",\"unique\":false}}"1431204654.411575 [0 127.0.0.1:61217] "sadd" "paper:idsets:Stationary" "i9hiar0q75vit5d9rgc5"1431204654.418524 [0 127.0.0.1:61217] "MULTI"1431204654.419119 [0 127.0.0.1:61217] "hmset" "paper:hash:Stationary:i9hiar0q75vit5d9rgc5" "color" "blue" "height" "40 cm" "sheets" "20" "width" "30 cm" "__meta_version" "1bf8ca04e698cd589baa17c661498b1109f8d65c"

Unpacking this Redis database activity when a Nohm stationary object is saved to Redis, we will examine each Redis key from and what the object mapper is doing with the Redis key and the corresponding data structure in the Redis database. From this analysis, the Redis key schema being used by Nohm becomes more intelligible. We start building the Redis schema as a Nohm's Redis schema by following a very common Redis design pattern of using a paper namespace for all the object mapper's Redis keys and note that Nohm uses a colon as a key delimiter in its underlying schema.

  • paper:meta:version:Stationary: This Redis metadata key stores a string version used for stationary stores the version. A random metadata version string of 1bf8ca04e698cd589baa17c661498b1109f8d65c is then set as the current value of this key. Nohm tracks each change that we make to the stationary model and then stores version information of our model.
  • paper:idsets:Stationary: This Redis set stores all stationary IDs. This set is first checked with a negative UNIX timestamp, and then, an ID string of i9hiar0q75vit5d9rgc5 is generated and added to this set. This set is used to track stationary objects, and a random value should minimize problems related to duplicate keys.
  • paper:meta:idGenerator:Stationary: This Redis string is used by Nohm to determine the method for generating an ID. The default option generates a random string, while the increment option uses an integer counter.
  • paper:meta:properties:Stationary: A Redis string stores the serialized JSON metadata for the stationary object.
  • paper:hash:Stationary:i9hiar0q75vit5d9rgc5: Wrapped in a Redis transaction, the stationary Javascript object instance stores its property values in a Redis hash by using i9hiar0q75vit5d9rgc5 as the last part of its Redis key.

Next, add a second stationary package, say a red square, 45-cm high × 45-cm wide, with an initial sheet count of 15 results to the following Redis keys in our database:

paper:meta:properties:Stationary
paper:meta:idGenerator:Stationary
paper:idsets:Stationary
paper:hash:Stationary:i9hjsdjv4o9csf8eeonj
paper:meta:version:Stationary
paper:hash:Stationary:i9hiar0q75vit5d9rgc5

We'll see how a more complicated Redis key schema comes into play using Nohm to model the sales of a stationery item by using two supporting classes from the schema.org metadata vocabulary, namely an offer (http://schema.org/Offer) class and an order (http://schema.org/Order) class. The schema.org vocabulary is cosponsored by Google, Microsoft, Yahoo, and Yandax for representing structured data on the web. The Offer class contains the price and the available inventory along with a priceCurrency property to support offers in other currencies. For now, our default currency for priceCurrency will be the United States dollar. Our Order class contains the acceptedOffer and orderDate properties, with the acceptedOffer property linking to the specific order that we created for our stationery. So far, we have only replicated the initial storage of each stationery package with Nohm. Adding two new models to represent our sales, namely offer and order, we'll want to be able to use the relationship modeling available in Nohm to link the stationery objects. Unlike other object mappers for SQL-based databases that require the relationship to be predefined before use, Nohm allows any model to be associated with another model through a link method.

nohm.model('Offer', {
  properties: {
    inventoryLevel: {
      type: 'integer',
      unique: false
    },
    price: {
      type: 'float',
      unique: false
    },
    priceCurrency: {
      type: 'string',
      unique: false,
      defaultValue: 'USD'
   }
}
});

The order class contains two properties, namely orderDate and orderedItem, although the order class could be expanded to include other order properties from the schema.org vocabulary such as customer and discount as the requirements change for the paper stationery web storefront. You'll notice that we didn't add orderedItem as a formal property for the order class because we will be creating orderedItem through a Nohm link to the item stationary.

nohm.model('Order', {
  properties: {
    orderDate: {
      type: 'datetime'
    },
 }
});

When a sale occurs, the Nohm approach is to create a linkage between the offer, the order, and the stationery objects. After creating a new order instance with a timestamp of when a sales transaction occurred, Nohm uses a couple of supporting Redis sets to model the relationships between the three different classes. Nohm stores the relationship information in a few different sets as seen from these snippets from the Redis cli program:

First, paper:hashOffer: ia4ev8iu8cns7w6p968h is a hash key that sets its inventory level property as 50 and the price as 15 for the red stationery.

1432589868.318914 [0 10.0.2.2:55200] "hmset" "paper:hash:Offer:ia4ev8iu8cns7w6p9 68h" "inventoryLevel" "50" "price" "15" "priceCurrency" "USD" "__meta_version" " 229e1d3b89b02804b4bdad9909fa75aa442197d5"

Next, the paper:relationKeys:Offer:ia4ev8iu8c ns7w6p968h and paper:relations:Offer:itemOffered:S tationery:ia4ev8iu8cns7w6p968h sets are created; the first set stores all the keys to the sets that create the linkages between offer and stationery with the itemOffered property. The second set stores all the individual stationery IDs by creating a specific link between this specific offer and the stationery.

1432589868.323265 [0 10.0.2.2:55200] "sadd" "paper:relationKeys:Offer:ia4ev8iu8cns7w6p968h" "paper:relations:Offer:itemOffered:Stationery:ia4ev8iu8cns7w6p968h"
1432589868.323281 [0 10.0.2.2:55200] "sadd" "paper:relations:Offer:itemOffered:Stationary:ia4ev8iu8cns7w6p968h"  ia4ev8itec2wq9gc0qnt"

When an order is received and thereby a sale is recognized, first, a Redis paper:hash:Order:1 hash is created with the order date field, and with all Nohm, a metadata version id is also stored with the hash as a field.

1432589868.325604 [0 10.0.2.2:55200] "hmset" "paper:hash:Order:1" "orderDate" "Mon May 25 2015 15:33:33 GMT-0600 (Mountain Daylight Time)" "__meta_version" "a881a941cb6ff674a79c7f652f8d8153b7b47b"

Two additional sets, namely paper:relationKeys:Order:1 and paper:relations:Order:offer:Offer:1, create the linkage between our order and offer with the first set storing all the relationship links for order and the second set storing the specific Offer for the order that was added in the previous command:

1432589868.327808 [0 10.0.2.2:55200] "sadd" "paper:relationKeys:Order:1" "paper: relations:Order:offer:Offer:1"
1432589868.327829 [0 10.0.2.2:55200] "sadd" "paper:relations:Order:offer:Offer:1"  "ia4ev8iu8cns7w6p968h"

The following graphic illustrates the JavaScript code flow that creates this linkage between stationery, offer, and order for our online paper store.

Deconstructing a Redis object mapper

Key expiration

A significant feature of Redis is the ability to set the expiration time for a key. By being able to automatically delete expired keys, a Redis application can better manage both the size and the memory usage of the datastore as well as reduce the amount of client code for keeping track of every key in the datastore.

In the next chapter on optimizing and managing RAM for your Redis instance, the topic of key expiration will be examined in more detail. Key expiration is most often discussed in the context of keeping the memory usage of a Redis instance within acceptable performance limits by ejecting the expired keys from the database. Redis offers a number of different modes for setting the automatic ejection of expired keys depending on your application's needs and performance limits, which can be set either by setting an option in the your Redis configuration file or by run-time commands sent to your Redis database.

Key cautions

Over the years, certain best practices have emerged that are briefly articulated in Redis tutorial1. The practices revolve around the legibility and performance trade-offs in your running Redis database and supporting client code. The size of the Redis keys should be limited not only because of memory issues that may arise if the key size is greater than 1024 bytes long but also because larger-size keys can be confusing to the developer and the user of the Redis instance. Another problem of larger key names is that as the size of the Redis instance increases, each one of these larger key names begins to consume larger amounts of memory, thereby reducing the amount of available memory for the data.

Likewise, if the key name is too small, the extra memory saved may not be worth the problems that can occur later when trying to troubleshoot Redis or adding a functionality through new Redis keys. For example, a key name of u11:2 may be short but does not convey the meaning of what value is being managed while a key name of the same data, user:11:clicks, is a better descriptor for the value stored in this Redis key and the application context of this key. This can be a challenge for applications that develop and evolve over time but can be mitigated by adopting a consistent Redis key schema that allows room for further growth in the future. Even a small amount of time devoted to thinking of possible future uses when developing a Redis key schema can alleviate massive refactoring of the client code and data migration in Redis to handle emerging needs from the use of your application by individuals and other programs.

The Redis KEYS command should be used as a last resort as its use creates a long-running blocking call on the Redis instance and can even result in Redis running out of memory. SCAN provides an iterator over all of the keys in the Redis instance that can be incrementally called upon all of the keys. The Redis SCAN, and the equivalent HSCAN, SSCAN, and ZSCAN commands for hashes, sets, and sorted sets, respectively, are relatively newer commands that meet a real requirement for Redis applications. A note of caution when using SCAN and its related iterator commands is that SCAN cannot guarantee that an element will be returned if that element was not consistently present from the start to the end of the iteration.

上一章目录下一章