Friday, February 26, 2010

How Generational Garbage Collector chokes on high request rate

Who doesn't want their server to be able to handle a large amount of requests gracefully? So you studied the programming languages shootout and picked one that performs the best—but not C/C++ because you don't want to bother with memory management—then wrote your server application, carefully multi-threaded your code, and tuned the code to your satisfaction. Only you find that the more concurrent requests you handle, the more time it takes to service each request. You profiled the running time and found that your server spends more time garbage collecting when the number of concurrent requests is higher.

Generational garbage collector assumes that most objects die young, so they would perform quick, minor collection on a nursery (also called "young" or "new object" heap). The assumption is that the nursery would contain few live objects, so the collection would be quick. The survivors are moved to the tenure generation (also called "old" heap), which is garbage collected less frequently. When the tenure heap runs out, the garbage collector engages a full collection. This heuristic addresses the problem that garbage collectors spend a lot of time traversing and moving reachable objects in order to get rid of the abandoned ones.

The lifetime of an object is measured by the number of bytes being allocated, not by the number of elapsed CPU cycles. A minor collection is triggered when the nursery is full. If the nursery is too small, the probability that a short-lived object gets promoted to tenure status increases. This increases the chance that we need a full collection later. We want the nursery to be large enough to give most short-lived objects a chance to die.

The "most objects die young" heuristic works well for single threaded programs. However, in a server application, when the server is handling multiple concurrent requests, the rate of allocation also increases proportionally—by virtue that multiple threads are allocating concurrently, making the object lifetime as seen by one thread of execution seem longer relative to the amount of objects the nursery could hold. The more concurrent requests your server handles, the more short-lived objects make it to tenure, so the tenure heap runs out more quickly. Then the garbage collector must spend more time doing a full collection.

One solution is to increase the nursery size. If a 4MB nursery is sufficient for one request, you should increase the nursery size to 400MB if you want to handle 100 concurrent requests. The tenure generation should not be increased—its size should be proportional to the amount of live objects you plan to keep in memory throughout the lifetime of the server, not to the number of concurrent requests. If your garbage collector gives you the option to specify the size of a per-thread nursery, that would be ideal.

Another solution is to limit the number of concurrent requests per server process. If you want better concurrency, you can always spawn new server processes. To one extreme, you can assign one worker process per request. The processes can all run on different processors, but each process is single threaded. When the request is finished, the process would make itself available for the next request (without quitting and restarting). A middle ground is to have several processors run a low number (< 20) of concurrent threads.

Further reading:

28 comments:

Thomas M. Baily said...


Hey I read your post its very very good assignment helper
best post keep up this work thanks.

rickypauls said...

Great write-up! All doubts regarding Assignment Help solved after reading the post. Sometimes, it becomes difficult to manage the time for writing homework due to some unavoidable reasons. I have recommendation for website greatassignmenthelp for ordering best writing services to complete your academic task.
Assignment Helper
Assignment Help Online
Online Assignment Help
Assignment Help Online Services
Assignment Assistance
Assignment Help Experts
Assignment Writing Help

Online Assignment Help said...

Universities all around the United States assign CPM homework to students, who lack the basic mathematical skills in elementary colleges. Get CPM homework help online from us, in case you find any problem while solving the mathematical problems.

Tillock Watson said...

'I'm highly impressed by the piece of thoughts you have shared on this portal. all the best
connect us on Assignment Help can shed your burden of assignments with a return of qualitative assignments.
Online Assignment Help

Max Willor said...

Very informative and well written post! Quite interesting and nice topic chosen for the post. My Assignment Help Au has been a pioneer within the Best Assignment Help on-line services and within the Australia and different countries ceaselessly for over Seven years. Make an order now for professional work, delivered on time and pure. You can email us at cs@Myassignmenthelpau.Com or Phone Number: +61-2-8005-8227.

poly said...

I want to thank you for this excellent read!! I certainly loved every little bit of it. I have got you book marked to check out new stuff you post. Our tested and proven writers spend many quality hours researching and writing our customers’ instructions. That’s how they create high-quality custom papers that help our customers better their writing skills. Even the strictest of teachers find it hard to deny those we help the “As” and “A-pluses” they deserve. Find more from Writing Assignment Help

Dan Smith said...

Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts. If you're looking for CDR Report Help. You can email us at cs@needcdrreport.com or Phone Number: +61-2-8005-8227

admin-writers said...

This post was awesome thank you for sharing. Nursing students are normally required to write several academic papers before they can graduate. Those students have great writing skills, access to material resources and sufficient time normally find writing such papers to be quite interesting. Learn more from Nursing Assignment Writers

assignmentauthors said...

This is because the finance relies on the data generated in these departments to file claims from the agencies for the patients that have been treated with insurance freelance writer for hire

Smith said...

I don’t know where on the internet I would have gotten such a great content thumb up. Did you know? Water quality concerns are becoming more of a focus for the public. You may have heard about contaminants in the drinking water such as Arsenic, Chromium, Cryptosporidium or Giardia. Visit Premier Ro Pure Plus Manual for more.

My Assignment Help Online said...

My assignment help Australia is one-stop solution for students looking for quality assignment help for meeting academic writing commitments. Our subject experts help students to achieve the aim of winning accolades and success in career. Contact us to access the best team of writers, proofreaders, editors and project managers offering 100% authentic write-ups.

Jenifer Discota said...

Looking for cost accounting assignment help to score high grades? We have experts who can do it for you. Contact us and get your assignment done today.

Smith said...

Thank you for bringing to a halt my long search topic. I really benefited from your content. If you are experiencing trouble striking a balance between your busy academic life and handling your online classes, you can get help from a professional who will take online classes on your behalf. It doesn’t matter the kind of help you need, whether it is passing your exam, finishing your assignments or even the entire classwork, you can get help here at Online Class Help .

Assignment Help Australia said...

Assignment Help services work like wonder when you don't have sufficient time to write your homework. Take the guidance of professional academic writers using online academic writing and finish your work as soon as possible even if you are in Australia.
My Assignment Help
Assignment Help Online
Online Assignment Help

amelie said...

Get assignment help in Australia with 20% off.our experts deliver you best Australian assignment help to students with covering all subjects.our team have experience in there field and provide top assignment at affordable prices.

Amelie said...

Deco-Cemento Ltd. produces a great range of polished plaster finishes and Microcement Coatings London hat add deepness and craftsmanship to commercial, residential and retail spaces. From a way of luxurious quality with metal finishes like Smooth and Pitted right the way through to custom surfaces that supply a way of occasion, there are finishes to reinforce the planning implication of each project.

Emma said...

Thanks for your Informative Blog. Meadows Office Suites provide short and long term rentals of furnished executive offices, flexible coworking office spaces, coworking space in Alpharetta, office space in Atlanta, Office Space near to metro atlanta as well as virtual office near to metro atlanta . We offer convenience and attractive place and environment at best price to stimulate the creativity and productivity of your businesse. Utilities and super fast internet are all included. Our conference room and training rooms are available to reinvigorate your workforce.

Jessica said...

The Acube Infotech offers the Expert RFID Solution. We provide the professional and the best solution including RFID Tags, RFID Handheld Reader, RFID Reader, Rfid jewellery solution, Rfid Asset tracking, Rfid jewellery tracking, jewellery tracking system, Jewellery Management System, ACUBE JIMS, RFID Inventory Management, RFID Library Management, RFID Metal Tags, Zebra RFID, RFID Warehouse Management, RFID Laundry Management and more to our reliable customer across the globe. The team that is involved in this procedure are well-grounded and are experienced in performing and execution. The ESMA has the sanction to be the franchise to supply the RFID Tags services in the United Arab Emirates. There are many companies who are searching for the RFID Solutions Provider but now you do not have to worry much because Acube Infotech is one of the leading RFID Solution providers in the Middle East is here to help you out.

Smith said...

Thank you for predicting our needs and writing great and informative article. Water is vital when it comes to plants’ growth. However, it is hard to get constant water supply during such times when the weather patterns are so unpredictable in Kenya. The rains no longer come regularly like they used to, and drought has struck several regions in the country. Visit Irrigation Equipment Suppliers in Kenya

online assignment writing help said...

This comment is about the assignment writing services online top assignment writing website in Australia. Assignments are an essential part of the academic curriculum and they also contribute a whole lot to the final grades. This is what makes them so important and this is why we must ensure that we get good grades in the assignments.
top assignment help australia
assignment service provider
best assignment writing help australia
best online assignment writing help
best assignment writing help

Silvia said...

Hello there, great stuff you’ve shared here I will be checking regularly for more articles. But wait! When it becomes hard to manage your online assignment, availing Online Assignment Help is the best choice. Many students often fail to submit their assignment before deadlines and end up scoring low grades because they didn’t have enough time to write the assignment given on time and satisfactorily. Visit Assignment Help . for assistance.

Smith said...

What a wonderful piece of article I have bookmarked for future checkup. Capstones are important papers that are written before one completes his/her course. These papers need the writers to have substantial background knowledge about the course and that is why our helpers for nursing capstone projects are all qualified professionals in nursing. Besides, our writers and editors are highly experienced in writing capstone projects after successfully completing hundreds of such papers. Learn more on Nursing Capstone Writing Help .

Smith said...

This is a very nice blog you have shared keep up good work. Is your dissertation troubling you? Do not worry anymore; our site has served many happy scholars with premium services in dissertation writing. You are therefore welcome to try us anytime you feel “I need a credible writer for my prospectus”. Well, you can Hire Phd Prospectus Writers Online

deepak said...

Online assingment

James Martin said...

Overcome the issues of missing due dates using Assignment Help from Qatar service provider. Pick the best writers even in Qatar to complete your papers on time via Online Assignment Help.Visit us : Assignment Help Online

Anish Pathak said...

If your are looking for assignment writing services in Australia then visit yesswork.

Mike Rogers said...

I work in Guruji Educational is a Brand deal in online Education through different apps, prepared for the purpose of documenting professional development in the Education field such as
Logical Reasoning
English Grammar
Hindi Grammar
English Stories
English Stories kids
SSC CGL
SSC CGL 2020
English
1000 English stories
PreSchool Learning
Kids Game
Moral English Stories
Moral Stories

assignmentworkhelp.com said...

We provide affordable assignment writing services of your operation management homework help at competitive prices so that you do not have to compromise on the quality of your assignment and get the required help for operations management assignment writing help from our team of specialist writers .
operation management assignment help
operation management homework help USA
operation management assignment help USA