"A cache (pronounced /'kæ?/ kash) is a component that improves performance by transparently storing data such that future requests for that data can be served faster" Wikipedia
Caching is everywhere around us these days in all shapes and forms: query caching, data caching, page fragment caching, event caching, etc. Much like how in Forrest Gump you could do all kinds of shrimp dishes :)
In its most basic form, we use caching to accelerate and scale our processes. We might have a certain request that takes time to process due to let’s say in colloquial terms "a big ass query". We can leverage caching by slapping that query up and placing it in cache so we don't have to wait for it again. Our process now only at most take 1 big hit and then can be merrily on their way serving requests. Again, this is a very practical approach to caching. As you will see from this guide, caching can get very very complex and you need the right tools to be able to scale out, configure and tune your caching approaches.
When building enterprise class applications we are faced with many problems dealing with architecture, performance, scalability and so much more. One of these issues might be dealing with expensive queries, but there are also lots of other issues to consider like: expense of object creation/configuration, data retrieval, data/system availability, page output caching, etc. All of these issues can benefit from some kind of caching in order to optimize performance, availability and scalability.
Let's explore some benefits of caching:
Reduce the amounts of data transfers between processes, network or applications
Reduce processing time within a system or application
Reduce I/O access operations
Reduce database access operations
Reduce expensive data transformations and live by a 1 hit motto
When working with a cache engine you also have to consider how you will get data INTO the cache before you work with it. There are several approaches but I will talk about two methods of content loading:
Lazy Loading or Reactive Loading : This type of loading is the most typical way to load data into a cache and it happens once data is requested from an external process. Below is a typical example using CacheBox in a service layer call:
This is great for small content pieces and works well for applications. However, if you have large amounts of data that must be available at application startup or cache startup. Then we recommend looking at the next method.
Proactive Loading : This approach focuses itself on the loading of resources before the application starts up so content can be loaded. EHCache for example offers cache loaders that you can create that will populate the cache on initializations. CacheBox offers the same capabilities through its event model, so you can tap into the necessary events and then carry your population and loading procedures. For example, you might tap into the afterCacheRegistration event in CacheBox so you might listen to when a cache engine get's created and configured so you can start populating it right at application/cache startup. In this loader you will then have the opportunity to load your data either asynchronously or synchronously. If you will be loading large amounts of data or processor intensive data, we recommend you leverage cfthread and do the loading asynchronously within your event interceptor.
Cache Hit : An event that occurs when an element requested from cache is found in the cache.
Cache Miss : An event that occurs when an element requested form cache is NOT found in the cache.
Eviction : The act of removing an element from the cache due to certain criteria algorithm that directly is connected to the state of the element.
Eviction Policy : The algorithm that decides what element(s) will be evicted from a cache when full or a certain criteria has been met in the cache. (Cache Algorithms)
LRU : An eviction policy that discards the least recently used items first.
LFU : An eviction policy that counts how often an item has been accessed and it will discard the items that have been used the least.
FIFO : An eviction policy that works as a queue, first object in will be the first object out of the cache. So older staler objects are purged.
Cache Provider : A concrete implementation to a caching engine within CacheBox
Object Store : An object that stores cached elements and indexes them
Idle or Last Access Timeout : An expiration time an element receives if it is not accessed.
Reap : Usually an asynchronous event that cleans a cache from dead elements or expired elements
Distributed/Partitioned Cache : A form of caching that allows the cache to span multiple servers so that it can grow in size and in capacity. Each machine contains a unique partition of the dataset. Adding machines to the cluster will increase the capacity of the cache. Both read and write access involve network transfer and serialization/deserialization.
Near Cache : Each cache server contains a small local cache which is synchronized with a larger distributed/partitioned cache, optimizing read performance. There is some overhead involved with synchronizing the caches.
Replicated : Each cache server contains a full copy of the elements cached
In Process Cache : A cache that lives inside of the same heap as the application using it.
Out of Process Cache : A cache that lives as its own process and heap in the most likely the same server as the application using it.
Eternal Objects : Eternal objects are objects that will never be purged or evicted by the caching engine automatically. The only way to remove them is to recreate the cache or clear them manually.
Cache Topology : A concept that refers to where data physically resides and how it is accessed in a distributed environment
Whenever you are leveraging the power of caching, we recommend that you also take into consideration several key factors that need to be evaluated and put into practice in your applications. Below are some key factors that we recommend:
Thread Safety : Take into consideration that when you cache data/objects or whatever, multiple threads will be trying to access it in your application. Make sure that you have the appropriate locking and synchronization techniques so multiple threads don't interfere with one another. This might be from simple testing procedures to make sure an element exists, to a more strict approach where you create a cache decorator that does hard blocks on cache access via named locks. Don't ever assume that what you ask from the cache actually exists.
Serialization : If you will be caching to disk, database or using replication, distribution, most likely the caching engines will be serializing or converting your data and objects into byte or string format in order to persist them. Therefore, your objects must be able to support serialization and also be aware that if you are caching complex objects, those objects will be serialized alongside the target object. This is what's called an object graph. Where a serializer will go through the entire object graph (object/data relationships) of the target object and serialize everything. If you have circular references or references to tons of objects, serialization performance degrades and could potentially cause a heap overload and crash your application server as it will recurse forever. ColdFusion 9 introduced the component and property attribute called serializable, which is a boolean attribute you can add to the cfcomponent
and cfproperty
tags. We recommend using it and marking components and relationships that do NOT need serialization.
Caution The default value for the serializable attribute is true.
Data Format : Determine how you want to store data so it is easy to take that snapshot out of a cache and use. This can be from implementing object state/memento patterns, to leveraging serialization, to pre-defined data structures. Think ahead of HOW your data will be stored in the caches.
Cache Limits : There are definite benefits of demarcating limits on your cache. This involves setting up default limits for timeouts, idle timeouts and maximum number of objects within a cache. Studies have shown that leverage memory sensitive data constructs with fixed limit caching parameters can increase performance and overall JVM stability. Look at our cache resources for these studies as they are very interesting.
"In the past, developers didn't have much control over garbage collection or memory management. Java2 has changed that by introducing the java.lang.ref package. The abstract base class Reference is inherited by classes SoftReference, WeakReference and PhantomReference. SoftReferences are well suited for use in applications needing to perform caching."
Why is this? Well, a soft reference is nothing but a wrapper class that wraps an object in memory. Whenever the JVM requires memory and runs its reference rules, it can detect these soft references and decide to purge them if it meets the garbage collector rules. If it purges them, the wrapped object inside of the soft reference is marked for collection, but the java soft reference itself still exists. Therefore, the programmer can determine if the object inside the reference exists or not. If it does not exist, it means the object was garbage collected and I have no more object. I won't go into the implementation semantics of the cache, just theory. Please visit the references to understand more of how the ColdBox cache was built.
In summary, soft references are what determine if an object is still available or not. CacheBox cache can then run maintenance on itself and clean out all of its references for you if you are using the ConcurrentSoftReferenceStore
object store.
Cache Layers : In our experience, we have seen greater improvement in performance and scalability by planning caching layers. Caching layers is what are the different layers of caching your application will provide from data to objects to event and view fragments. We recommend you analyze your cache layers and plan how they will behave. A perfect live example of caching layers can be seen in our open source ColdFusion CMS, . ContentBox leverages query caching, XML/Feed caching, object caching of DAO's, services, some business objects, plugins and event handlers, to view fragment caching and overall event output caching. We use it all baby!