IBM WebSphere Application Server Performance Cookbook

J9 Health Center Enable at Runtime of Limited Duration

The Health Center agent is shipped with IBM Java and IBM Semeru Runtimes on z/OS (though not z/Linux), but it is not currently shipped with IBM Semeru Runtimes on other platforms, though there are some unsupported workarounds. The Health Center agent neither ships with nor works with HotSpot JVMs.

The following instructions are for non-z/OS platforms (in this context, z/Linux is considered non-z/OS). For z/OS, see alternate instructions.

  1. By default, late attach is enabled but double check that the following option has not been set to disable it: -Dcom.ibm.tools.attach.enable=no
  2. Log on as the same user that the JVM is running under.
  3. Execute the following, replacing $JAVA_HOME twice, $PID with the process ID, and change 30 to the number of minutes to run (see notes below on duration considerations): 
    $JAVA_HOME/bin/java -jar $JAVA_HOME/jre/lib/ext/healthcenter.jar ID=$PID level=headless -Dcom.ibm.java.diagnostics.healthcenter.headless.run.number.of.runs=1 -Dcom.ibm.java.diagnostics.healthcenter.headless.run.duration=30
  4. After the number of minutes elapses, gather the *.hcd file from the current working directory of the process. By default:
    • WebSphere Liberty: $LIBERTY/usr/servers/$SERVER/
    • WAS traditional: $TWAS/profiles/$PROFILE/

Warnings and notes:

  1. Every time a new HCD collection is started, the agent starts to look up method address to method name mappings for all loaded methods at the start of the HCD. By default, the agent queries up to 3,000 unresolved method name mappings every 5 seconds. Therefore, for proper profiled method name evaluation, the minimum duration per-HCD should be specified based on the number of loaded methods. It is common for an enterprise application to load hundreds of thousands of methods, so a minimum of 5-10 minutes is a good start. Tracing to show this behavior and the number of methods is -Dcom.ibm.diagnostics.healthcenter.logging.methodlookup=debug -Dcom.ibm.diagnostics.healthcenter.logging.MethodLookupProvider=debug and search for com.ibm.diagnostics.healthcenter.methodlookup.debug DEBUG: N methods to lookup. Methods created during the HCD interval are captured seperately.
  2. If the JVM could not be stopped gracefully, gather the temporary files from a subdirectory of the output called tmp_${STARTDAY}${STARTMONTH}${STARTYEAR}_${STARTHOUR}${STARTMINUTES}${STARTSECONDS}_
  3. Use the additional JVM option -Dcom.ibm.java.diagnostics.healthcenter.headless.output.directory=$DIR to redirect Health Center files to a different directory instead of the working directory.
  4. Note that this does not work with Liberty if some jndi-1.0-related features are loaded and there is a request for enhancement.
  5. If using Liberty and you specify -Xtrace:buffers={2m,dynamic} to minimize Health Center method metadata loss, since Liberty defaults to an unlimited maximum thread pool designed to maximize throughput, consider capping this with <executor maxThreads="N" /> based on available native memory to avoid native memory exhaustion, or use a smaller -Xtrace buffer size such as -Xtrace:buffers={128k,dynamic} (or lower).
  6. We have observed that some monitoring agents cause problems with Health Center. Consider removing other monitoring agents that use -agentpath while using HealthCenter, engage IBM and the agent company support teams to investigate, or use -Xbootclasspath/p to healthcenter.jar and -agentpath to libhealthcenter.so.

For details, see the Health Center chapter.