Partitioning is used to split your Redis data into multiple Redis instances so that every instance contain only a subset of your keys.
It is generally used for large databases.
Types of Partitioning
There are two types of partitioning in redis:
- Range Partitioning
- Hash Partitioning
Range partitioning is one of the simpliest way to perform partitioning. It is done by mapping ranges of objects into specific Redis instances.
Suppose that you have 3000 users. So you can say that users from ID 0 to ID 1000 will go into instance R0, while users form ID 1001 to ID 2000 will go into instance R1, users form ID 2001 to ID 3000 will go into instance R2 and so forth.
Hash partitioning is an alternative of Range partitioning. In Hash partitioning, a hash function is used to convert the key into a number and then the data is stored in different-different Redis instances.
Advantage of Redis Partitioning
- Partitioning facilitates you to use the collective memory of multiple computers. For example: In the case of larger databases you need a large amount of memory, so partitioning provides sum of memory from different computers. Without partitioning you can use only a limited amount of memory that a single computer can support.
- Partitioning is also used for scaling the computational power to multiple cores and multiple computers, and the network bandwidth to multiple computers and network adapters.
Disadvantage of Redis Partitioning
There are some disadvantages of partitioning because some features of Redis are obstructed with partitioning.
- Partitioning doesn't usually support the operations which have multiple keys. For example, you can't perform the intersection between two sets if they are stored in keys that are mapped to different Redis instances.
- Partitioning doesn't support the transactions which have multiple keys.
- The partitioning granularity is the key, so it is not possible to shard a dataset with a single huge key like a very big sorted set.
- When partitioning is used, data handling is more complex, for instance you have to handle multiple RDB / AOF files, and to make a backup of your data you need to aggregate the persistence files from multiple instances and hosts.
- Adding and removing capacity can be complex. For instance Redis Cluster supports mostly transparent rebalancing of data with the ability to add and remove nodes at runtime, but other systems like client side partitioning and proxies don't support this feature. However a technique called Pre-sharding helps in this regard.