Troubleshooting WebLogic Server 12c - Oracle WebLogic Server 12c Advanced Administration - page 192

7184774KB->7152045KB (8388608KB), 5.918 s, sum of pauses 5759.204 ms,
longest pause 5759.204 ms.
[memory ][Wed Nov 14 09:46:09 2012][OC#426] 20833.656-20839.466: OC
7192291KB->7155809KB (8388608KB), 5.809 s, sum of pauses 5648.369 ms,
longest pause 5648.369 ms.
The heap free is at 15 percent after the first full GC at 09:45:46 AM (OC #422). The time
spent is about 5.5 seconds. Just after it, another full GC is triggered at 09:45:51 AM (OC
#423). Again, the heap free is about 15 percent and the time spent is 5.7 seconds, and
another full GC is triggered. This pattern continues, indicating that the JVM is spending almost
all the CPU time on garbage collections; so, the machine high load is probably a consequence
of the GC threads working in a non-stop fashion, trying to free the heap memory.
Excessive utilization of the heap can be the consequence of a peak load in the WebLogic
Managed Server. Too many threads are executing at the same time, so too many objects are
live in the heap. The problem may be the consequence of a heap size being too small or the
concurrency being too high. Raise the heap size according to the footprint needed or limit the
threads' concurrency. Adding more Managed Servers to the cluster is also an option.
Try to find a heap size that has at least 30-40 percent of the heap free
after a full GC during a peak load test.
The high heap usage can also be the result of a memory leak. Memory leaks are usually
caused by application bugs that can lead to a continuous rise of the heap utilization until
an Out of Memory (OOM) occurs.
The verbose GC has a little-to-none overhead and can be
used in production environments.
The GC analysis can also be made using a graphical tool, such as the Monitoring Dashboard
or the JRockit Mission Control. For troubleshooting purposes, the verbose GC is very precise
and does not need the JVM to be up and running since the information is already written in
the logfile.
There's more...
The verbose GC can be enabled at runtime with the jrcmd command.