Database Reference
In-Depth Information
So we take a look at the
$MW_HOME/user_projects/domains/
soa_domain/servers/AdminServer/logs/AdminServer.log
log file.
Here, we see the stuck thread, which apparently had kept on working for over
10 minutes:
####<Dec 23, 2011 6:03:49 PM EST> <Error> <WebLogicServer> <soahost1> <AdminServer> <[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <cb680017c6a0acfe:-606797c4:134357968da:-8000-0000000000001062> <1324681429443> <BEA-000337> <
[STUCK] ExecuteThread: '0'
for queue: 'weblogic.kernel.Default (self-tuning)'
has been busy for "658" seconds
working on the request "weblogic.kernel.WorkManagerWrapper$1@2a8605ce", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.
It turns out that we had just tried to unsuccessfully start the BAM managed serv-
er 10 minutes earlier. By observing
$MW_HOME/user_projects/domains/
soa_domain/servers/bam_server1/logs/bam_server1.log
, we find
the same
ExecuteThread '0'
unable to register an MBean due to a
java.lang.OutOfMemoryError
exception:
####<Dec 23, 2011 5:53:36 PM EST> <Error> <JMX> <soahost1> <bam_server1> <[ACTIVE]
ExecuteThread: '0'
for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1324680816405> <BEA-149500> <
An exception occurred while registering the MBean
com.bea:Name=AdminServer,Type=WebServiceRequestBufferingQueue, WebServiceBuffering=AdminServer,Server=AdminServer, WebService=AdminServer.
java.lang.OutOfMemoryError: PermGen space
Fortunately, looking back at
AdminServer.log
a few hours later, we see that
ExecuteThread '0'
has become
"unstuck":
####<Dec 23, 2011 10:42:42 PM EST> <Info> <WebLogicServer> <soahost1> <AdminServer> <[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <cb680017c6a0acfe:-606797c4:134357968da:-8000-0000000000001070> <1324698162702> <BEA-000339> <[STUCK]
ExecuteThread: '0' for queue:
'weblogic.kernel.Default (self-tuning)'
has become "unstuck"
.>
In this example, we navigated multiple areas starting from the stuck thread to
get to the root cause of the warning on the health state of
AdminServer
. To
recap:
•
We found
AdminServer
to be in the
Warning
state, which is usually due to
a stuck thread.
•
We confirmed that there was indeed a stuck
ExecuteThread
as shown
on both the Oracle WebLogic Administration Console and the
AdminServ-
er.log
file.
•
By reviewing the
soa_server.log
and
bam_server1.log
files, we
found startup errors in the BAM server log.
