I have been playing with AWS Elasticache Redis for some time now. Here are the 4 key points about AWS Elasticache Redis Scaling and Monitoring that helped me to understand the concepts of scaling and monitoring better.

1. Cluster, Shards and Replicas

AWS Elasticache Redis can be setup in stand-alone and cluster mode. In Cluster-Mode, there are two concepts to be aware of.

Shards: By default, in the Cluster-Mode there will be a single shard created. Shard hold the actual Redis key stores. As of this writing, a single cluster can contain up to 500 shards.

Replicas: Each shard will have its replica for redundancy and high availability. Need atleast a single replica per shard. Having more replicas can help to get distributed in multiple availability zones.

• If creation of cluster with 3 shards and 2 replicas, this means the creation of 3 primary nodes and 6 replicas. 9 nodes in total will be available.
• If the primary nodes fail, one of the replicas becomes the primary node.
• If the replicas also fail for the primary node, then backup restoration may be the only choice.
• For scaling, the standard approach is horizontal scaling . This involves adding more shards to the cluster.

2. HashSlots and KeySpaces

Redis uses the concept of HashSlots/ Keyspaces. These are the main concepts for storing and retrieving data in the shards. During creation of shards , Redis internally performs the allocation of the slots based on the total available shards .

During auto-scaling, the slots are re-calculated. Re-balancing of slots lead to movement of data across the shards. There are no down time during this process.

This article helps to get an in-depth understanding of the concepts.

3. Monitoring Metrics and Reserved Memory

It is important to understand the keys metrics and the monitoring is set accordingly. The concept of Reserved Memory has to be well understood. Redis takes around 25% for Reserved Memory for backup and replication. So the monitor levels should factor in this number.

For example, for cache.r5.large instance, the available memory is 13 GiB. 25% of this will around 3.25 GiB. This is for Reserved Memory and the remaining available memory is 9.75 GiB. Monitors for warn and alert levels must be set for this only.

If the intention is to set warn and alert levels at 75 % or 80%, then the monitor must be set at 7.3 GiB ( 9.75 * .75 ) and 7.8 ( 9.75 * .8) GiB respectively.

Ref (1) and (2)

4. Redis Key Eviction and Eviction Policies

Redis Key eviction will happen if the cluster runs out of memory . It is always better to maintain the cluster health and memory levels to avoid eviction. Once the eviction triggers , the cache will not be available for the evicted keys. This will create more pressure on the downstream systems.

There are a handful of eviction policies available to choose from based on the need. Careful selection of eviction policies will help to manage the performance of the cluster.

These are the 4 key points that helped me to understand scaling and monitoring in AWS Elasticache Redis.