Friday, February 26, 2010

How Generational Garbage Collector chokes on high request rate

Who doesn't want their server to be able to handle a large amount of requests gracefully? So you studied the programming languages shootout and picked one that performs the best—but not C/C++ because you don't want to bother with memory management—then wrote your server application, carefully multi-threaded your code, and tuned the code to your satisfaction. Only you find that the more concurrent requests you handle, the more time it takes to service each request. You profiled the running time and found that your server spends more time garbage collecting when the number of concurrent requests is higher.

Generational garbage collector assumes that most objects die young, so they would perform quick, minor collection on a nursery (also called "young" or "new object" heap). The assumption is that the nursery would contain few live objects, so the collection would be quick. The survivors are moved to the tenure generation (also called "old" heap), which is garbage collected less frequently. When the tenure heap runs out, the garbage collector engages a full collection. This heuristic addresses the problem that garbage collectors spend a lot of time traversing and moving reachable objects in order to get rid of the abandoned ones.

The lifetime of an object is measured by the number of bytes being allocated, not by the number of elapsed CPU cycles. A minor collection is triggered when the nursery is full. If the nursery is too small, the probability that a short-lived object gets promoted to tenure status increases. This increases the chance that we need a full collection later. We want the nursery to be large enough to give most short-lived objects a chance to die.

The "most objects die young" heuristic works well for single threaded programs. However, in a server application, when the server is handling multiple concurrent requests, the rate of allocation also increases proportionally—by virtue that multiple threads are allocating concurrently, making the object lifetime as seen by one thread of execution seem longer relative to the amount of objects the nursery could hold. The more concurrent requests your server handles, the more short-lived objects make it to tenure, so the tenure heap runs out more quickly. Then the garbage collector must spend more time doing a full collection.

One solution is to increase the nursery size. If a 4MB nursery is sufficient for one request, you should increase the nursery size to 400MB if you want to handle 100 concurrent requests. The tenure generation should not be increased—its size should be proportional to the amount of live objects you plan to keep in memory throughout the lifetime of the server, not to the number of concurrent requests. If your garbage collector gives you the option to specify the size of a per-thread nursery, that would be ideal.

Another solution is to limit the number of concurrent requests per server process. If you want better concurrency, you can always spawn new server processes. To one extreme, you can assign one worker process per request. The processes can all run on different processors, but each process is single threaded. When the request is finished, the process would make itself available for the next request (without quitting and restarting). A middle ground is to have several processors run a low number (< 20) of concurrent threads.

Further reading:

14 comments:

Thomas M. Baily said...


Hey I read your post its very very good assignment helper
best post keep up this work thanks.

rickypauls said...

Great write-up! All doubts regarding Assignment Help solved after reading the post. Sometimes, it becomes difficult to manage the time for writing homework due to some unavoidable reasons. I have recommendation for website greatassignmenthelp for ordering best writing services to complete your academic task.
Assignment Helper
Assignment Help Online
Online Assignment Help
Assignment Help Online Services
Assignment Assistance
Assignment Help Experts
Assignment Writing Help

Online Assignment Help said...

Universities all around the United States assign CPM homework to students, who lack the basic mathematical skills in elementary colleges. Get CPM homework help online from us, in case you find any problem while solving the mathematical problems.

Tillock Watson said...

'I'm highly impressed by the piece of thoughts you have shared on this portal. all the best
connect us on Assignment Help can shed your burden of assignments with a return of qualitative assignments.
Online Assignment Help

Max Willor said...

Very informative and well written post! Quite interesting and nice topic chosen for the post. My Assignment Help Au has been a pioneer within the Best Assignment Help on-line services and within the Australia and different countries ceaselessly for over Seven years. Make an order now for professional work, delivered on time and pure. You can email us at cs@Myassignmenthelpau.Com or Phone Number: +61-2-8005-8227.

poly said...

I want to thank you for this excellent read!! I certainly loved every little bit of it. I have got you book marked to check out new stuff you post. Our tested and proven writers spend many quality hours researching and writing our customers’ instructions. That’s how they create high-quality custom papers that help our customers better their writing skills. Even the strictest of teachers find it hard to deny those we help the “As” and “A-pluses” they deserve. Find more from Writing Assignment Help

Dan Smith said...

Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts. If you're looking for CDR Report Help. You can email us at cs@needcdrreport.com or Phone Number: +61-2-8005-8227

admin-writers said...

This post was awesome thank you for sharing. Nursing students are normally required to write several academic papers before they can graduate. Those students have great writing skills, access to material resources and sufficient time normally find writing such papers to be quite interesting. Learn more from Nursing Assignment Writers

assignmentauthors said...

This is because the finance relies on the data generated in these departments to file claims from the agencies for the patients that have been treated with insurance freelance writer for hire

mhesh said...

I don’t know where on the internet I would have gotten such a great content thumb up. Did you know? Water quality concerns are becoming more of a focus for the public. You may have heard about contaminants in the drinking water such as Arsenic, Chromium, Cryptosporidium or Giardia. Visit Premier Ro Pure Plus Manual for more.

My Assignment Help Online said...

My assignment help Australia is one-stop solution for students looking for quality assignment help for meeting academic writing commitments. Our subject experts help students to achieve the aim of winning accolades and success in career. Contact us to access the best team of writers, proofreaders, editors and project managers offering 100% authentic write-ups.

Jenifer Discota said...

Looking for cost accounting assignment help to score high grades? We have experts who can do it for you. Contact us and get your assignment done today.

mhesh said...

Thank you for bringing to a halt my long search topic. I really benefited from your content. If you are experiencing trouble striking a balance between your busy academic life and handling your online classes, you can get help from a professional who will take online classes on your behalf. It doesn’t matter the kind of help you need, whether it is passing your exam, finishing your assignments or even the entire classwork, you can get help here at Online Class Help .

Assignment Help Australia said...

Assignment Help services work like wonder when you don't have sufficient time to write your homework. Take the guidance of professional academic writers using online academic writing and finish your work as soon as possible even if you are in Australia.
My Assignment Help
Assignment Help Online
Online Assignment Help