Friday, May 31, 2013

Custom framework for easy multi-threading

A big part of my job at Terracotta is to not only demonstrate the various features of Terracotta BigMemory (efficiently using all the RAM available on our server within a single java process, ease of use, high availability, consistency, multi-tenancy, etc...) but also the performance improvements (maximizing transaction/seconds or minimizing processing response times) you get by introducing BigMemory in your environment/application(s).
But to fully demonstrate how Terracotta BigMemory can provide and consistently guarantee Microsecond speed at TerraByte (TB) scale, I needed (still need) on a regular basis to be able to:
  1. Create an way to load Terracotta with massive amount of data, and
  2. Create a way to "flood" Terracotta BigMemory with massive amount of requests (gets, puts, searches)
  3. While making it easy to extend for any test use case, and 
  4. Allowing for the loading of all different "specialized" business objects into BigMemory (because customers don't want to necessarily test only with generic data structures or byte arrays)
What I started to do initially was to build for each new use case a different program using thread pool executor, queue, deques, etc...and doing a lot of copy pasting, and of course introducing some nice concurrency bugs at the same time :) !! (for example, I should have known the class "Random" is thread-safe...hence uses locks if accessed by multiple threads = not good for a concurrent framework)
So I decided to build a highly concurrent framework that I could rely on and could reuse over and over without having to rebuild the wheel each time, and especially without having to wonder: is the performance I'm measuring right now the actual performance of the system under test, or is it measuring - without my knowledge - the time taken by my threads to "lock" on top of each other!
Enter "JPerftester" (I wish I had found a more awesome name for it :) ) available on my github account at
I tried to architect it in a way that it's modular enough to use it for Terracotta-specific use cases, but possibly also for other use case as well (like for example using multi-threading to load billions of records into a DB, or do comparative load testing against other systems out there). Here is the structure:
  • Base -----> [maven base pom for global plugins/dependencies]
  • BaseEngine -----> [multi-threading base + generic framework objects and helpers]
  • TerracottaEhCacheTesterSuite -----> [Terracotta specific suite]
    • Base -----> [maven base pom for global Terracotta/Ehcache plugins/dependencies]
    • BaseEngine  -----> [specific framework objects for terracotta such as key-value pair operation executors etc...]
    • TesterClients  -----> [actual running examples]
      • CacheWriterTester  -----> [testing cache writing scenarii under load]
      • CustomCacheSearchTester  -----> [testing searches]
      • POJOCacheTester  -----> [testing get/puts with some actual POJOs]
      • ...there could be many more here
  • Other suite 1
  • Other suite 2
  • etc...
To build, simply go to JPerfTester root folder and run "mvn clean install"...that should be it to compile it all + package the sample TesterClients apps in tar.gz files ("dist" folders after build phase) that you can deploy anywhere you have JAVA installed (and Terracotta, since these sample app are terracotta testers)
In further posts, I'll go over in more detail how to run these test clients in your environment, and how to create a new "Tester Client" (terracotta-specific or not) for your needs.