Tuning the JBoss AS Part 2

JBoss AS pools

Most services provided by the application server are granted by means of the pooling of resources. Keeping these resources in a pool avoids the burden of re-creating costly resources like database connections and can be used as well to limit the amount of requests to be processed by the application server. As a matter of fact, allowing an indiscriminate number of requests to access your resources can be quite dangerous and can expose your system to potential threats.

JBoss AS defines a large list of pools, which, depending on the nature of your application, can become a potential bottleneck if you don’t configure them appropriately.

The following table introduces the list of pools, adding the relevant configuration information for it. You need to replace "<server>" with your server configuration, for example JBOSS_HOME/server/default is the path of the default configuration.

Pool	Used for	Configuration
System thread pool	For JNDI naming
JDBC connection pool	When making JDBC connections
HTTP thread pool	When handling HTTP requests
AJP thread pool	When handling HTTP requests through mod jk
WorkManager thread pool	In conjunction with JMS, as JBoss Messaging uses JCA inflow as the integration into EAP
JBoss Messaging thread pool (for remote clients)	Pools the TCP sockets
EJB 3 (same JVM)	Will use the same thread pool that invoked the EJB. For example, local HTTP clients will use the HTTP thread pool to invoke the EJB
EJB (remote clients)	When making remote EJB calls

In this topic, we will cover the System thread pool, which is used for accessing resources through Java Naming and Directory Interface (JNDI), and the Java Database Connectivity (JDBC) connection pool which handles database connections.

The JBoss Messaging pools and EJB pools are described in next topic, Tuning the Middleware Services, that is devoted to middleware services tuning.

The HTTP pool and AJP pool on the other hand, are covered in next topic, Tomcat Web Server Tuning, along with the web container-tuning configuration.

The System thread pool

The System thread pool is used by JBoss AS to control the requests which are accessing the application server resources through the JNDI tree. The Java Naming and Directory Interface is a Java API that allows Java software clients to discover and look up data and objects using a name.

This pool is defined in the <server>/conf/jboss-service.xml file. Here’s the core section of it:

And this is the meaning of these parameters:

• MinimumPoolsize: The minimum number of threads to keep active. By default 0.

• MaximumPoolsize: The maximum number of threads which can be active (default 100).

• KeepAliveTime: How long to keep threads alive when there is nothing to do. Time is expressed in milli-seconds (default 60000 = 1 minute).

• MaximumQueueSize: The maximum number of requests that are waiting to be executed (default 1024).

• When all your threads are busy and the waiting Queue has also reached the MaximumQueueSize, then the last parameter (BlockingMode) comes into play.

Setting BlockingMode to abort will determine a RuntimeException when a new request attempts to enter the busy thread queue. On the contrary, the default run will give priority to the calling thread, which will be able to execute the task. Setting the parameter to wait will force the calling thread to wait until the thread queue has room, while the option discard will simply discard the calling thread:

Actually, there is a last option "discardOldest" which works exactly as "discard", except that it ultimately performs a scan of the thread pool to see if a thread is about to complete before discarding the request.

JNDI lookups are relatively expensive, so caching an object that requires a lookup in client code or application code avoids incurring this performance hit more than once.

From Java EE 1.5 onwards, you can also use dependency injection to perform a single lookup to add the resources to your classes. For example, the following annotation:

is translated by the application server into:

Notice that when using EJB 3.0′s dependency injection there’s a slight overhead because, behind the scenes, the application server uses reflection to set the EJB variable field in your class.

Finding out if the System thread pool is a bottleneck

The default settings for the System thread pool are generally good for most applications so we will not take long with this topic.

However, in the unfortunate event that you have to optimize an application which does not cache JNDI entries, the System thread pool can become an additional bottleneck for your application.

Discovering this threat is not complicated and can be done with a simple configuration hack. In practice, you should set the MaximumQueueSize to 0, which means that the application server will not accept any new task when the System thread pool is full:

Now load test your application and check for any CommunicationException in your client console. You may find the following error trace thrown by the org.jnp. interfaces.NamingContext class:

 

 

 

Then it’s time to upgrade your System thread pool configuration, increasing the MaximumPoolsize attribute.

JDBC connection pool

A connection pool is a cache of database connections maintained by the application server so that they can be reused when the database receives future requests for data.

Connection pools can dramatically enhance the performance of executing commands against a database as opening and maintaining a database connection for each user is a quite costly operation.

Configuring a connection pool with JBoss AS can be done by means of a simple -ds.xml file which needs to be dropped in the deploy folder of the application server.

The basic pool parameters are the min-pool-size attribute, which is used to determine the minimum connections to store in a pool.

The max-pool-size element corresponds to the maximum limit of connections allowed in a pool.

The actual amount of connections stored in a connection pool is a variable attribute, in our example, connections that are idle for over 10 minutes will be automatically destroyed, and the poll will be resized up to the min-pool-size attribute.

Since JBoss AS 5, there’s an additional attribute, which can be used to bypass the min-pool-size limit, where you have idle connections. The attribute is use-strict-min—which therefore allows connections below the min-pool-size to be closed.

If you attempt to acquire a new connection from the pool when all resources are checked out, the application server will wait for a maximum time (in milliseconds) specified by the blocking-timeout-millis. If no resources can be acquired by that time, the following exception will be raised:

Calculating the optimal min-pool-size and max-pool-size

In order to calculate the optimal pool size it’s important to know how many connections your application requires. The pool attributes can be monitored using the JMX-console, which contains a reference to the connection pool in the jboss.jca domain and an MBean definition in name=<DataSourceName>,service=ManagedCon nectionPool.

For example, the DefaultDS can be configured by means of the MBean name=Defaul tDS,service=ManagedConnectionPool.

ConnectionCount is the number of connections established with the database. These connections are not necessarily in use and after the idle timeout minute, they are candidates to be destroyed (provided that the application has not reached a min-pool-size).

If you want to know the actual amount of connections in use, you have to check the InUseConnectionCount attribute.

The most important attribute for calculating the optimal pool size is the Max Connections In Use Count, which can be obtained by invoking the liststatistics method of the managed pool.

The following is a sample output from the action:

If your Max Connections In Use Count equals to the max-pool-size, then your application is likely to have a bottleneck in your datasource configuration. The Block Time statistics should indicate exactly how much time has been spent waiting for a new connection.

To correctly size a connection pool for your application, you should create load test scripts and set the max-pool-size to a consistent value (let’s say you expect a load of 100 concurrent users, then you could set this value to about 150).

Then, when the test is completed, check for the Max Connection In Use Count and set the max-pool-size to a relatively larger attribute (let’s say 20% more).

If you want to avoid the overhead of pool resizing, you can set the min-pool-size to the same value as max-pool-size. You should as well add the following element to your DataSoruce configuration so that JBoss AS attempts to prefill the pool size at deployment time; otherwise the first user accessing the pool will pay the pool build-up cost:

The major limitation to this sort of eager initialization is that database connections are a shared resource, which should not be monopolized by a single application that never sets them free back to the database.

Also remember that setting the pool size attributes doesn’t mean that you will actually be able to use that number of connections. Every RDMS has got a limited number of cursors, which are available for executing SQL statements. For example, Oracle DB has the OPEN_CURSOR parameter, which triggers the message "ORA-01000: maximum open cursors" exceeded if you try to reclaim new connections beyond this limit.