Startup
Common Tuning
Context and Dependency Injection (CDI)
- With tWAS >= 9, CDI is enabled by default. With tWAS < 9, CDI
is only enabled if an application has a
beans.xmlfile - If all applications do not use CDI
beans.xml, then set-Dcom.ibm.ws.cdi.enableImplicitBeanArchives=falseor to disable CDI completely,-Dcom.ibm.ws.cdi.enableCDI=false - To disable CDI on a per-application basis, use the amm.filter.properties file.
Application Startup
There is a "server.startup" thread pool (default maximum size of 3) in which applications start. If this thread pool (actually, its queue of work) is full, the following message will be printed in SystemOut.log:
[10/9/18 19:07:24:002 CEST] 00000001 ThreadPool I WSVR0629I: The request buffer for thread pool "server.startup" has reached its capacity.However, simply increasing the server.startup thread pool may not help because large parts of application startup within the WebContainer are single threaded and you may see contention in com.ibm.ws.webcontainer.component.WebContainerImpl. A feature request was opened but deemed too risky and complex: https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=21322
As per Amdahl's Law, the best thing to do is to profile and optimize the startup time of the application(s) taking the longest time to start. For J9-based JVMs, use IBM Java Health Center and for HotSpot-based JVMs, use OpenJDK Mission Control. More simply, take a bunch of thread dumps during startup to see where most of the time is spent. If there aren't many opportunities to optimize startup, then consider splitting applications into separate clusters.
Alternatively, there is a feature called "Launch application before server completes startup" which may be disabled, although note: "A setting of true informs the product that the application might start on a background thread and thus server startup might continue without waiting for the application to start. Thus, the application might not be ready for use when the application server starts."
You may see a stack with a synchronization block such as:
3XMTHREADINFO "server.startup : 2" J9VMThread:0xDE4DAD00, j9thread_t:0xDE3E61D4, java/lang/Thread:0xF6424EB8, state:CW, prio=5
3XMTHREADINFO1 (native thread ID:0x12AF, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO2 (native stack address range from:0x01981000, to:0x019C2000, size:0x41000)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/Thread.sleep(Native Method)
4XESTACKTRACE at java/lang/Thread.sleep(Thread.java:893)
4XESTACKTRACE at com/ibm/test/ApplicationStartup.contextInitialized(ApplicationStartup.java:12)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebApp.notifyServletContextCreated(WebApp.java:1651)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebAppImpl.initialize(WebAppImpl.java:410)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
4XESTACKTRACE at com/ibm/ws/webcontainer/VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
4XESTACKTRACE at com/ibm/ws/webcontainer/WSWebContainer.addWebApp(WSWebContainer.java:749)
4XESTACKTRACE at com/ibm/ws/webcontainer/WSWebContainer.addWebApplication(WSWebContainer.java:634)
4XESTACKTRACE at com/ibm/ws/webcontainer/component/WebContainerImpl.install(WebContainerImpl.java:422)
4XESTACKTRACE at com/ibm/ws/webcontainer/component/WebContainerImpl.start(WebContainerImpl.java:714)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.start(ApplicationMgrImpl.java:1154)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1369)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedModuleImpl.start(DeployedModuleImpl.java:638)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedApplicationImpl.start(DeployedApplicationImpl.java:967)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:760)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.start(ApplicationMgrImpl.java:2147)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:445)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitImpl.start(CompositionUnitImpl.java:123)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:388)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:116)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:994)
4XESTACKTRACE at com/ibm/wsspi/runtime/component/WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:349)
4XESTACKTRACE at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1659)
3XMTHREADINFO "server.startup : 1" J9VMThread:0xDE4D7100, j9thread_t:0xDE3E5EB8, java/lang/Thread:0xF6424F40, state:B, prio=5
3XMTHREADINFO1 (native thread ID:0x12AE, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO2 (native stack address range from:0x01940000, to:0x01981000, size:0x41000)
3XMTHREADBLOCK Blocked on: com/ibm/ws/webcontainer/component/WebContainerImpl@0xE72A44F8 Owned by: "server.startup : 2" (J9VMThread:0xDE4DAD00, java/lang/Thread:0xF6424EB8)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at com/ibm/ws/webcontainer/component/WebContainerImpl.start(WebContainerImpl.java:714)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.start(ApplicationMgrImpl.java:1154)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1369)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedModuleImpl.start(DeployedModuleImpl.java:638)
4XESTACKTRACE at com/ibm/ws/runtime/component/DeployedApplicationImpl.start(DeployedApplicationImpl.java:967)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:760)
4XESTACKTRACE at com/ibm/ws/runtime/component/ApplicationMgrImpl.start(ApplicationMgrImpl.java:2147)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:445)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitImpl.start(CompositionUnitImpl.java:123)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:388)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:116)
4XESTACKTRACE at com/ibm/ws/runtime/component/CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:994)
4XESTACKTRACE at com/ibm/wsspi/runtime/component/WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:349)
4XESTACKTRACE at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1659)Enabling Diagnostic Trace during Startup
The normal mechanism to enable diagnostic trace at runtime is through the Runtime tab or an MBean call; however, both are unavailable during startup. Instead, Java Surgery (https://www.ibm.com/support/pages/ibm-runtime-diagnostic-code-injection-java-platform-java-surgery) may be used to call the static method to set trace dynamically and this works even during startup. Example:
java -jar surgery.jar -pid ${PID} -command ExecuteMethod -class com.ibm.ejs.ras.ManagerAdmin -method setTraceState -arg "*=info:WAS.j2c=all:RRA=all:Transaction=all"Startup Order
The startup order of applications may be controlled with the Startup order value.