Friday, February 26, 2010

How Generational Garbage Collector chokes on high request rate

Who doesn't want their server to be able to handle a large amount of requests gracefully? So you studied the programming languages shootout and picked one that performs the best—but not C/C++ because you don't want to bother with memory management—then wrote your server application, carefully multi-threaded your code, and tuned the code to your satisfaction. Only you find that the more concurrent requests you handle, the more time it takes to service each request. You profiled the running time and found that your server spends more time garbage collecting when the number of concurrent requests is higher.

Generational garbage collector assumes that most objects die young, so they would perform quick, minor collection on a nursery (also called "young" or "new object" heap). The assumption is that the nursery would contain few live objects, so the collection would be quick. The survivors are moved to the tenure generation (also called "old" heap), which is garbage collected less frequently. When the tenure heap runs out, the garbage collector engages a full collection. This heuristic addresses the problem that garbage collectors spend a lot of time traversing and moving reachable objects in order to get rid of the abandoned ones.

The lifetime of an object is measured by the number of bytes being allocated, not by the number of elapsed CPU cycles. A minor collection is triggered when the nursery is full. If the nursery is too small, the probability that a short-lived object gets promoted to tenure status increases. This increases the chance that we need a full collection later. We want the nursery to be large enough to give most short-lived objects a chance to die.

The "most objects die young" heuristic works well for single threaded programs. However, in a server application, when the server is handling multiple concurrent requests, the rate of allocation also increases proportionally—by virtue that multiple threads are allocating concurrently, making the object lifetime as seen by one thread of execution seem longer relative to the amount of objects the nursery could hold. The more concurrent requests your server handles, the more short-lived objects make it to tenure, so the tenure heap runs out more quickly. Then the garbage collector must spend more time doing a full collection.

One solution is to increase the nursery size. If a 4MB nursery is sufficient for one request, you should increase the nursery size to 400MB if you want to handle 100 concurrent requests. The tenure generation should not be increased—its size should be proportional to the amount of live objects you plan to keep in memory throughout the lifetime of the server, not to the number of concurrent requests. If your garbage collector gives you the option to specify the size of a per-thread nursery, that would be ideal.

Another solution is to limit the number of concurrent requests per server process. If you want better concurrency, you can always spawn new server processes. To one extreme, you can assign one worker process per request. The processes can all run on different processors, but each process is single threaded. When the request is finished, the process would make itself available for the next request (without quitting and restarting). A middle ground is to have several processors run a low number (< 20) of concurrent threads.

Further reading:

24 comments:

Indian Assignment Help Experts said...

Medical science is a multidisciplinary subject that covers the understanding of the human body and its elements. The different phenomena of the human body and how it acts and reacts to the associated habit and situation that it is exposed to are studied in health science. Writing the assignments on health and medical science is an extremely difficult task as there are a lot of topics in these subjects that are not very easy to grasp. Therefore, it is fairly common for the students of Medical Science to take online academic assignment writing help to get good grades and accomplish the task of assignment writing efficiently.
Medical Assignment
Medical Assignment Help
Medical Science Assignment

newassignmenthelpaustralia said...

Get the best statistics assignment help from Australia by experts at affordable prices. Get the best statistics assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

Assignments Help In UK said...

Now you can join the millions students who have trusted Assignmentshelp.uk, and now have upraised their grades.

Assignment writing in United Kingdom

newassignmenthelpaustralia said...

Get the best risk management assignment help from Australia by experts at affordable prices. Get the best risk management assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

Veronica Kate said...

The blog written is extremely impressive, with a great topic. However, a bit more research could have strengthened qit even further. You can explore the services as offered by livewebtutors.com, a premium academic writing services platform offering the best of Thesis editing service teamed with knowledge and experience.

assignment help said...

Get the best corporate finance assignment help from Australia by experts at affordable prices. Get the best corporate finance assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

newassignmenthelpaustralia said...

Get the best financial management assignment help from Australia by experts at affordable prices. Get the best financial management assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

assignment help said...

Get the best cookery assignment help from Australia by experts at affordable prices. Get the best cookery assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

Arif said...

Thank you for sharing such a great article. Just to point out that we specialises in custom academic papers for all levels of students. From students who are in high school, through to college graduates and undergraduates, all the way up to university students at master’s degree and even doctoral levels. Essay Writing Help offer an academic writing service across all subjects and their sub-topics and we offer every essay type from a thesis based argumentative essay to a literary review.

Roxane Fabin said...

I didn’t know that you guys were so good at providing programming assignment help. The Java assignment I sent is back and it’s simply excellent. However, the Java homework help tutor I was assigned, has saddened me by telling me that she is leaving your team. All in all, I am happy for completing my work on time and I will continue engaging you more in the future. If you guys handled other assignments other than programming I would hire you as well. However, if I get another programming one it will surely come your way.

Roxane Fabin said...

As a statistics homework help expert can you do my statistics assignment within a day? The paper is not very hard but I need a very experienced Econometrics assignment help expert to handle it. If you don't have anyone who can guarantee an A then please don't pick the job. I strictly want to get everything in the paper. I have sent it to you and therefore go through it and let me know whether you can handle it well.

Roxane Fabin said...

I am a Matlab homework help expert and would like to know, how is sampling done when a sensing array is used for image acquisition? I have looked for answers to this question on the web but I can’t seem to find anything satisfactory? Any matlab assignment help suggestions on where I can get more information on this topic and image processing as a whole? It would really help improve my skills as a Matlab homework help tutor.

Roxane Fabin said...

Which country do you operate in? I am looking for an economics assignment help expert from Australia. I want someone who can help me while relating to economic issues in our country. I am almost giving up on finding one from Australia. Therefore if you are from Australia or can get an economics homework help expert from Australia then you would have helped me in a great way.

Sadiya said...

On your post, you claim to provide first-class java homework help. If the assignment you send to me is not as satisfactory as you claim am I allowed to claim my money back? I have had some very horrible experiences with online programming assignment help companies. I just want assurance that whatever you claim to offer is what you offer. With that, I would give you all my assignments for the next two years.

Sadiya said...

I tend to come across two major equations while studying probability theory. They are Chebyshev's inequality and the Markov inequality. From experience, they are not that easy to understand. Apparently, the two are related. I need someone experienced, i.e., a statistics homework help or a professor, to show me if the two are related. In addition I also need a SAS assignment help expert to work on my paper.

Sadiya said...

I hate doing control systems assignments because they take too much of my time. I always come to this site when I need Matlab assignment help and never had problems but I didn’t like my last experience at all probably because the person who handled my task didn’t seem to have enough experience with the topic.I had to hire a different expert for the job and this almost made me miss my assignment submission deadline. I am not saying that this is not a good site to hire a Matlab homework help expert. All I am saying is that the company should take another look at their data mining experts to avoid future inconveniences.

Sadiya said...

Hello, I need an economics assignment help with regression and unemployment. I am not able to describe everything here but I have submitted the entire task on your provided website for review. I just hope that I will get the right economics homework help from you. Should you have any questions on the work I have provided please feel free to contact me. In addition, let me know how much I need to pay for the work to be completed.

newassignmenthelpaustralia said...

Get the best cipd assignment help from Australia by experts at affordable prices. Get the best cipd assignment writing services by professional assignment writers of Australia. On-Time Delivery. A+ Quality.

James Ian said...

This blog has got all what i was looking for. Thank you for sharing. Can i ask if you find it all too perplexing to explore the nitty-gritty of the assignment you’ve been given recently? Does the topic of your term paper trip you up? Save yourself from slaving over it by letting HND Assignment Help get this ordeal over with. We specialize in expert assignment writing services for a variety of fields, taking the onerous burden off your shoulders. Whether you’re a student who can’t make heads or tails of that essay title or an academician looking for advanced training assistance, we’ve got your back.

James Ian said...

How wonderful to wake on this perfect piece of article. Keep it up. Can i point out that our nursing essay writing service is designed to help nurses and those studying nursing, and will provide the extra support they need in completing their next nursing or healthcare essay. We match the best academic writers, many of whom work as nurses or healthcare practitioners within the NHS, to requests for help from students just like you. Writing in perfect English, our writers will create a custom piece of work designed just for you and to help you reach the grade you require. Get Nursing Essay Writers

Caleb said...

I’m amazed by your art of writing . You’re a fresh DNP candidate. At this time, it’s too early to start worrying about your DNP capstone project. Possibly, the idea of working with a quality DNP capstone project writing service hasn’t crossed your mind. You still have time, don’t you? But time keeps playing tricks on people. At the start, it may feel like you’ve got more than enough time. Time flies, though. Before you know it, you’ve run out of all of the time. Read more about DNP Capstone Project Writers

Arif said...

This must be a popular blog. The content and knowledge shared here is of high quality. Gas and electric tankless water heaters run more efficiently than the conventional water heaters of the same fuel type. We rated the annual energy consumption cost Excellent for a gas model but only Fair for an electric, but both rate Very Good for energy efficiency. Read more on Tankless Water Heater Reviews

Best Assignment Experts said...

C programming can directly assist you to speak with the hardware devices. It became the favourite language of the programmer because it executes the program quickly as compared to other languages. Best Assignment Experts provide C Programming assignment help

Technology Data said...

I have been looking for this information for quite some times. Will look around your website.
1milliionhacktreatindiablogspotcom
1milliionhacktreatindiablogspotcom
1milliionhacktreatindiablogspotcom
1milliionhacktreatindiablogspotcom
1milliionhacktreatindiablogspotcom