IBM WebSphere Application Server Performance Cookbook

Troubleshooting WXS

Hung Thread Detection

WXS has hung thread detection similar to that available in WAS. For example:

[3/13/15 7:33:09:631 PDT] 00000032 XSThreadPool  W   CWOBJ7853W: Detected a hung thread named "XIOPrimaryPool : 202" TID:3c14 BLOCKED.
Executing since 3/13/2015 07:32:40:520 -0700.
Stack Trace:
    com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java:549)
    java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    com.ibm.ws.xs.util.XSUtilities.loadClass(XSUtilities.java:77)
    com.ibm.ws.xs.io.ObjectStreamPool$ClassForNamePrivilegedAction.run(ObjectStreamPool.java:467)
    com.ibm.ws.xs.io.ObjectStreamPool$ReusableInputStream.resolveClass(ObjectStreamPool.java:388)
    java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1610)
    java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515)
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769)
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    java.util.HashMap.readObject(HashMap.java:1155)
    sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.lang.reflect.Method.invoke(Method.java:606)
    java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
    java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    com.ibm.ws.objectgrid.datagrid.BaseAgentCommand.inflateAgent(BaseAgentCommand.java:323)
    com.ibm.ws.objectgrid.datagrid.BaseAgentCommand.setBaseMap(BaseAgentCommand.java:173)
    com.ibm.ws.objectgrid.server.impl.ServerCoreEventProcessor.processCommand(ServerCoreEventProcessor.java:1454)
    com.ibm.ws.objectgrid.server.impl.ServerCoreEventProcessor.processClientServerRequest(ServerCoreEventProcessor.java:2596)
    com.ibm.ws.objectgrid.server.impl.ShardImpl.processMessage(ShardImpl.java:1469)
    com.ibm.ws.objectgrid.server.impl.ShardActor.handleContainerMessage(ShardActor.java:503)
    com.ibm.ws.objectgrid.server.impl.ShardActor.receive(ShardActor.java:333)
    com.ibm.ws.xs.xio.actor.impl.XIOReferableImpl.dispatch(XIOReferableImpl.java:110)
    com.ibm.ws.xsspi.xio.actor.XIORegistry.sendToTarget(XIORegistry.java:977)
    com.ibm.ws.xs.xio.transport.channel.XIORegistryRunnable.run(XIORegistryRunnable.java:88)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    com.ibm.ws.objectgrid.thread.XSThreadPool$Worker.run(XSThreadPool.java:309)

[3/13/15 7:34:20:345 PDT] 000035ab XSThreadPool  W   CWOBJ7854W: Thread named "WXS : 89" TID:35ab RUNNABLE is no longer hung.
Runnable: com.ibm.ws.objectgrid.util.security.SecurityContextRunnable@5fa09130.

Unable to Load Data into the Grid

Strategy 1: Object grid failed to start

Execute the command $WXS_HOME/bin/xsadmin.sh -primaries

Two common culprits are:

  1. Firewall rules blocking something that shouldn't be
  2. The numInitialContainers value is set higher than the total number of containers being started (in deploy.xml).