ParallelGC and ParallelGCThreads
When it comes to tuning garbage collection in Java, there are too many options available. There is no single default configuration that works best for your application, so you have to try different options.
When selecting ParallelGC for minor collection, it is important to set right number of garbage collector threads for ParallelGC. If you turn on ParallelGC for your application and do not set –XX:ParallelGCThreads, then the JVM will set number of parallel GC threads based on number of processors (cores) available on a machine where your application is running. The JVM will create one parallel GC thread per processor (core) for young generation.
An example helps. Imagine you have a machine with 16 processors (32 cores). For simplicity sake, you have an application running in 8 clones (JVMs) for load balancing reason. You have decided to turn on ParallelGC for all 8 clones (JVMs) for higher throughput. It means by default each JVM will create 32 garbage collector threads (one thread per core). You have 8 clones (JVMs), so you will end up with 32 x 8 = 256 garbage collector threads in your system. This is not good. You obviously do not need 32 garbage collector threads per JVM – it is overdone and a waste of system resources. In this case, you must set –XX:ParallelGCThreads property and set low number of threads (e.g. 4 or 6 threads per JVM rather than keeping it default).
Remember, more garbage collector threads can take system resources and more importantly it can increase fragmentation in old generation. There is a space reserved in old generation for each young generation's garbage collector thread for promotions. The division in space can cause fragmentation and young generation garbage collector threads may not be able to work efficiently when copying large amount of survived data from young generation to old generation. In order to fix fragmentation, the old generation will have to go through compaction process, which can increase major collection time.
If you want to make sure number of parallel GC threads is running as expected, then take a thread dump of your application and use an analysis tool to check whether you have correct number of parallel GC threads running.
So remember to set –XX:ParallelGCThreads=N whenever you use -XX:+UseParallelGC in a large multi-processor system.