Tuning the Java Virtual Machine (JBoss AS 5) Part 1

In this topic, we will begin our performance tour with the Java Virtual Machine (JVM) tuning. As you know, every Java application requires a JVM environment to be executed and the JBoss application server is no exception.

With every new release of Java, many improvements have been made at all levels of the runtime system including in the garbage collector, in the code, in the VM handling of objects and threads, and in the compiler optimizations.

We have now got a much faster Virtual Machine but at the price of enhanced complexity due to the increase of the available algorithms and options. Therefore, our primary goal will be to simplify the management of the Virtual Machine and concentrate our efforts on the fixes that will surely pay good dividends such as Java heap and garbage collector settings.

Summing up, this topic is organized into the following sections:

• The first section introduces the basics of Java Virtual Machine

• The next section discusses optimal JVM settings, focusing on the most appropriate memory and garbage collection settings

• The last section shows how to put in practice the theoretical concepts, just learnt, with a concrete use case

The starting point

So every Java program runs in a Virtual Machine environment, but how is a Virtual Machine made up? Essentially, there are two areas of memory which are used by Java programs. These are commonly known as the stack and the heap.


The stack is the memory set aside as scratch space for a thread of execution. When a method is called, a block is reserved on the top of the stack for local variables and some bookkeeping data. When the method returns, the block becomes unused and can be recycled the next time a function is called. The stack is always reserved in a LIFO (Last In First Out) order, which means that the most recently reserved block is always the next block to be freed. This makes it really simple to keep track of the stack meaning that freeing a block from the stack is nothing more than adjusting one pointer.

The heap is the area of memory set aside for dynamic allocation. Unlike the stack, you can allocate a block at any time in your code and free it as well at any time. This makes it much more complex to handle, as variables on the heap must be destroyed manually and never fall out of scope. If you don’t keep track of your objects allocated on the heap, there can be memory leaks in your application.

One of the strengths of the J2SE platform is that it shields the developer from the complexity of memory allocation and garbage collection. As a matter of fact, the JVM handles these complex issues. However when memory management and garbage collection becomes a bottleneck in your applications, it is worth learning more about the internals of the JVM.

As you can see from the next picture, the Java heap is basically divided into two main parts. These are known as the old generation which is sometimes called the tenured space, and the young generation which is known as the new space. There is also a third generation which appears in the picture named as the permanent space. The permanent space is the area of the JVM that is used to store data structures and class information. We will deal with this area later on in this topic, in the Making good use of memory section.

tmp39-56_thumb

The young generation is further subdivided into three partitions. These are Eden and two Survivor spaces, called the From space (S1) and the To space (S2).

Objects are initially allocated in Eden. One Survivor space is empty at any time, and serves as a destination of the next, copying the collection of any live objects in Eden and the other Survivor space. Objects are copied between Survivor spaces in this way until they are old enough to be tenured, or copied to the Tenured generation.

In short, as depicted in the next picture, all objects that survived multiple garbage collections are finally moved from the young generation to the old generation.

tmp39-57_thumb

When is an Object eligible for garbage collection?

An object is considered garbage when it can no longer be reached from any pointer in the running program. The most straightforward garbage collection algorithms simply iterate over every reachable object. Any objects left over are then considered garbage.

You might wonder why the structure of Java heaps matter to the user. The answer is that these two partitions named young and tenured, are collected using two different algorithms. When the young generation is full, a minor collection is triggered, while the tenured generation can only be freed with a major collection.

As the name implies, a minor collection has a low impact on performance since it’s targeted on a smaller area of memory where most of the objects are quickly reclaimed (that is have died). A major collection, on the other hand, involves the entire Java heap where lots of live objects are piled. This means that it is a lot more expensive.

Consequently, by setting the appropriate size for the young and old generation, you can reduce the time spent in garbage collections. In particular, this can decrease the need for costly major collections.

Choosing the JVM settings

You can control the JVM heap settings using a set of parameters known as non-standard options, which are prefixed by the -X flag. Our analysis will be split into three main areas:

1. At first, we will review the parameters that define the minimum and maximum heap sizes.

2. After that, we will inspect the flags which can be used to fine-tune the Young / tenured generation ratio.

3. Finally, we will examine the most complex part, which concerns the garbage collector behavior and that can be further optimized by choosing the optimal garbage collector algorithm.

Setting the correct heap size

You can configure the desired heap size using two switches: the -Xms option lets you decide the initial heap size. On the other hand, the -Xmx option is where you can set the maximum memory granted to the heap. If JVM tries to reclaim more memory when the maximum heap size is reached, a message saying java.lang.

OutOfMemoryError: Java heap space is issued.

For example, the following command launches the MyApplication class using 128 MB as starting heap memory. This allows it to grow up to 512 MB.

tmp39-58_thumb

If you don’t specify a value for the initial and maximum heap size, JVM will choose some defaults. Prior to J2SE 5.0, the default values were 4 MB for the initial heap size, and 64 MB for the maximum allowed heap size.

Prior to J2SE 5.0 Settings

Heap setting

Default value

-Xms

4 MB

-Xmx

64 MB

You might imagine that these values are grossly inadequate for most server applications. Moreover, since the 1.5 release of JVM, the default selection values have changed due to the introduction of Ergonomics self-tuning strategies.

In practice, the default values for the Java heap and garbage collector algorithms are calculated based on the hardware capabilities of your machine. If your machine has got two or more processors, or at least 2 GB of memory available, it’s considered as a server class machine. Therefore it has higher default settings.

For server-class machines, the default value for the initial heap size is 1/64 of physical memory up to 1 GB. The maximum heap size is one-fourth of physical memory up to 1 GB. The most notable exception to this rule is JVM for Windows (32 bit), which always uses the older default values.

The following table summarizes the default values for the Java heap in both scenarios:

J2SE 5.0 Settings

Heap setting

Not Server machine

Server machine

(or Windows 32 bit)

-Xms

4 MB

1/64 of physical memory (up to 1 GB)

-Xmx

64 MB

V4 of physical memory (up to 1 GB)

So drawing some sums, if your machine is qualified as a server class machine and has got 2 GB available, it will use 32 MB as the initial heap size and 512 MB as the maximum size. That’s the fairest configuration default. However for an application server, it can be still considered insufficient.

The correct amount of memory to grant your application

So you know, because of the nature of the beast of the application server, you cannot rely on the default values for the heap size; furthermore if you make a poor choice, the JVM can’t compensate for it without a restart.

Luckily, setting the correct heap values is quite simple and requires just some trials.

Step # 1: Finding the maximum heap (-Xmx)

Finding the maximum size allowed for the Java heap is just a matter of observing the application under a consistent load. Suppose that your application has reached a peak load of 768 MB as depicted by the following screenshot:

tmp39-59

Then, you can add a bonus of 25-30% so that the JVM can handle an additional burst of request. This would turn out to a suggested maximum heap size of around 1024 MB

tmp39-60_thumb

For the best performance, it’s important to make sure that no more than 70-80% of the heap is occupied, otherwise the frequency of the garbage collector would be too great and your application would freeze until the amount of memory drops down consistently.

Also consider that the maximum amount of memory is limited by the operating system limits (see next tip I cannot allocate enough memory for JVM) and applications running on the machine. As a general rule, it is wise to consider as an upper limit for JVM, the amount of physical memory available on the machine, minus about 1 GB which should be left to the operating system.

Translating this to numbers, if you have got 3 GB of RAM, this is the maximum amount of memory you should allocate.

tmp39-61_thumb

Assigning over 2 GB of RAM to your application server is, anyway, considered a bad practice because with extra large heaps the time spent in major garbage collection grows up accordingly. So, if you want fast and robust applications consider 2 GB as the maximum amount of memory for the Java heap.

Step # 2: Finding the initial heap size (-Xms)

Setting the amount of initial memory for your heap is generally a bit more controversial. We will soon see why. As for the maximum heap size, the strategy stays the same; that is you should observe your application when it’s consistently loaded.

In many cases, you should just aim at a different target. Consider the following screenshot:

tmp39-62

Almost all applications, which are developed without memory leaks, exhibit a heap graph bouncing around a constant amount of memory. In the preceding screenshot, the application bounces around a minimal value of 320 MB which should be apparently considered the optimal initial size.

tmp39-63_thumb

We said apparently because in real world Enterprise applications, you will hardly find a minimal or maximum size with such a difference. This is because most tuning experts agree that it is worthy to set the initial size of the JVM at the same level as the maximum heap size; Furthermore, by using a fixed size for your heap, it reduces the overhead of computing a new heap size as new memory is reclaimed by the JVM.

The critics say that this would lead to a potential waste of memory, especially in the initial stages of the application execution. In addition, this introduces delays in the start of garbage collection, which will be a very expensive operation the first times it is triggered.

Even if there’s some truth in this criticism, in a long-term perspective the application will gradually reach 70-80% of the maximum memory (in our example 768 MB) and the garbage collector will stabilize as well. So, if you figure that you are going to get to the maximum heap anyway, then growing the JVM memory can be considered as pure overhead, which we should avoid.

The hardware resources available on the machine can have the last word in this contest. If your JBoss AS is running on a dedicated machine, then you should wisely set the initial size equal to the maximum heap size.

On the other hand, if your JBoss AS is competing with other applications for the same hardware (this typically happens on development machines) then it makes sense to set the initial size to the JVM minimal size, as pointed out by the preceding screenshot.

Next post:

Previous post: