Caching

In this post i want to discuss about how we used Caching effectively in one of our Financial Systems

What did we cache/ Why did we need cache

1. Authentication Tokens

In first version of one of our applications, we stored authentication tokens in database. This turned out to be very poor design choice . Two side affects of this

• For every access by every user — there would be 1 read for validation of token and 1 write for updation of same. Consequently as number of concurrent users increased in system, performance of system degraded exponentially
• Over period of time count of tokens increased and we had to flush/truncate table.

To solve this we employed InMemory caching and overnight performance of application sky-rocketed. No slowness of system. No stuck query. No restarts

Once user logged out then token was removed from cache.

2. Customer/Account Info

In our system we observed that during one transaction cycle, there would be about 10–15 calls/queries to db at different stages to retrieve Customer or related information. This presented a good scenario for caching

First access of day for specific customer would be obtained from database and then all subsequent access for that customer for that particular day would be obtained from cache

So during a transaction number of calls was reduced to 1 from 15 and further if that customer performs transaction again for that day, no db call would be needed

Any changes in customer details- cache would be cleared

Further at end of day, entire cache would be cleared and next day process would start fresh again.

Benefits

• Saved query on databases to factor of 10000 or more
• Reduced latency for transaction
• Increase TPS
• Increased number of concurrent users
• Reduced Load of database

3. Config Parameters

Again this is good source of caching. All config parameters which usually stays constant was cached.

Whenever there was a change in any of parameter, cache would be re-loaded

Further similar to customer info, this cache would be reloaded at start of day.

These were some of caching techniques which we used, which helped us increase performance of application

Examples of caching in real world

Below are some of examples where you run into caching on daily basis

• Processors/ Laptops/Smart Phones
• Operating systems
• Web browsers
• Delivery Networks

Commercial / Open source alternatives

Although in our applications , we employed InMemory caching (Concurrent HashMap) , for bigger scale there are open source alternatives which can be explored, concept remains same though.

• Redis
• Mem Cached

Further Light Reading on Topic

Book Algorithms to Live By , offers good light reading on topic. Some of key take aways for me from chapter on Caching

• One of fundamental problems of computer science — How to store, arrange and retrieve
• Different cache management algorithms — Random Eviction, FIFO and LRU. However LRU seems most effective in various scenarios
• LRU is simple and effective due to principle of Temporal Locality. If anything is used/accessed, there are chances it will be accessed/used again
• Limitation of having Infinite size cache
• Author offers various examples regards caching in variety of areas - Organising Books in Library, Inventory Stock organisation — Amazon , Organising files in Filing system — Yukoi Nohuchi, Limitations of Human memory, why we forget as we age