IBM WebSphere Application Server Performance Cookbook

HotSpot Native Memory Usage Recipe

HotSpot Native Memory Tracking has an overhead of ~5-10%.

  1. Ideally, restart with verbosegc (e.g. WebSphere Liberty)
  2. Restart with -XX:NativeMemoryTracking=detail
  3. Let the JVM warm-up for some time (ideally, at least some number of hours and taking significant workload). This is because it's normal for significant memory usage increase as the JIT code cache warms up.
  4. Register a native memory baseline (replace $PID with the target process ID): 
    jcmd $PID VM.native_memory baseline
  5. If the overhead is acceptable, gather an HPROF heapdump (replace $PID with the target process ID): 
    jcmd $PID GC.heap_dump dump1_$(hostname)_$(date +%Y%m%d_%H%M%S).hprof
  6. Let the JVM increase native memory significantly (at least a few hundred MB and ideally 1GB or more).
  7. Register a native memory baseline (replace $PID with the target process ID): 
    jcmd $PID VM.native_memory detail.diff > diag_nmt2_$(hostname)_$(date +%Y%m%d_%H%M%S).txt
  8. If the overhead is acceptable, gather an HPROF heapdump (replace $PID with the target process ID): 
    jcmd $PID GC.heap_dump dump2_$(hostname)_$(date +%Y%m%d_%H%M%S).hprof
  9. Restart the JVM and remove -XX:NativeMemoryTracking or stop the native tracker at runtime with (replace $PID with the target process ID): 
    jcmd $PID VM.native_memory shutdown
  10. Upload diag_nmt*.txt, both *.hprof files if taken, and any other JVM logs (e.g. stdout/stderr logs, verbosegc if captured, application logs, etc.).

Notes:

  1. Overhead may be reduced (with less effective diagnostics) by switching detail to summary in steps 1 and 7.
  2. There are orthogonal, OS-specific tools that can augment or replace the above steps (e.g. eBPF on Linux, etc.).