Java Diagnostics Guide 5.0
About the Diagnostics Guide
What does the "Java Virtual Machine (JVM)" mean?
Other sources of information
Reporting problems
Conventions and terminology
How to send your comments
Contributors
Summary of changes
Understanding the IBM Software Developers Kit (SDK) for Java
The building blocks of the IBM Virtual Machine for Java
Java application stack
Components of the IBM Virtual Machine for Java
JVM Application Programming Interface (API)
Diagnostics component
Memory management
Class loader
Interpreter
Platform port layer
Memory management
Overview of memory management
Object allocation
Reachable objects
Garbage collection
Heap sizing problems
Allocation
Heap lock allocation
Cache allocation
Large Object Area
Initialization and the LOA
Expansion and shrinkage of the LOA
Allocation in the LOA
Detailed description of garbage collection
Mark phase
Mark stack overflow
Parallel mark
Concurrent mark
Sweep phase
Parallel bitwise sweep
Concurrent sweep
Compaction phase
Subpool (AIX, Linux PPC and zSeries, z/OS and i5/OS only)
Reference objects
Soft, weak, and phantom reference processing
Final reference processing
JNI weak reference
Heap expansion
Heap shrinkage
Generational Concurrent Garbage Collector
Tenure age
Tilt ratio
How to do heap sizing
Initial and maximum heap sizes
Using verbose:gc
Using fine tuning options
Interaction of the Garbage Collector with applications
How to coexist with the Garbage Collector
Root set
Thread local heap
Bug reports
Finalizers
Nature of finalizers
Finalizers and garbage collection
How finalizers are run
Summary and alternative approach
Manually starting the Garbage Collector
Frequently asked questions about the Garbage Collector
Class loading
The parent-delegation model
Namespaces and the runtime package
Custom class loaders
Class data sharing
The JIT compiler
JIT compiler overview
How the JIT compiler optimizes code
Phase 1 - inlining
Phase 2 - local optimizations
Phase 3 - control flow optimizations
Phase 4 - global optimizations
Phase 5 - native code generation
Frequently asked questions about the JIT compliler
Java Remote Method Invocation
The RMI implementation
Thread pooling for RMI connection handlers
Understanding distributed garbage collection
Debugging applications involving RMI
The ORB
CORBA
RMI and RMI-IIOP
Java IDL or RMI-IIOP?
RMI-IIOP limitations
Further reading
Examples of client–server applications
Interfaces
Remote object implementation (or servant)
Stubs and ties generation
Server code
Summary of major differences between RMI (JRMP) and RMI-IIOP
Summary of differences in server development
Summary of differences in client development
Using the ORB
How the ORB works
The client side
Stub creation
ORB initialization
Obtaining the remote object
Remote method invocation
The server side
Servant implementation
Tie generation
Servant binding
Processing a request
Additional features of the ORB
Portable object adapter
Fragmentation
Portable interceptors
Interoperable Naming Service (INS)
The Java Native Interface (JNI)
Overview of JNI
The JNI and the Garbage Collector
Overview of JNI object references
JNI transitions
J2J and N2N transitions
N2J transitions
J2N calls
J2N returns
Copying and pinning
Using the isCopy flag
Using the mode flag
A generic way to use the isCopy and mode flags
Handling exceptions
Synchronization
Debugging the JNI
JNI checklist
Submitting problem reports
Problem determination
First steps in problem determination
AIX problem determination
Setting up and checking your AIX environment
Enabling full AIX core files
General debugging techniques
AIX debugging commands
lsconf
netpmon
netstat
ps
sar
svmon
topas
trace
vmstat
DBX Plug-in
Diagnosing crashes
Documents to gather
Locating the point of failure
Debugging hangs
AIX deadlocks
AIX busy hangs
Poor performance on AIX
Understanding memory usage
32- and 64-bit JVMs
The 32-bit AIX Virtual Memory Model
The 64-bit AIX Virtual Memory Model
Changing the Memory Model (32-bit JVM)
The native and Java heaps
The AIX 32-bit JVM default memory models
Monitoring the native heap
Native heap usage
Specifying MALLOCTYPE
Monitoring the Java heap
Receiving OutOfMemoryError exceptions
Is the Java or native heap exhausted?
Java heap exhaustion
Native heap exhaustion
AIX fragmentation problems
Submitting a bug report
Debugging performance problems
Finding the bottleneck
CPU bottlenecks
Memory bottlenecks
I/O bottlenecks
JVM heap sizing
JIT compilation and performance
Application profiling
MustGather information for AIX
Linux problem determination
Setting up and checking your Linux environment
General debugging techniques
Using system dump tools
Examining process information
ldd
Tracing tools
Debugging with gdb
Diagnosing crashes
Debugging hangs
Debugging memory leaks
Debugging performance problems
Finding the bottleneck
CPU usage
Memory usage
Network problems
JVM heap sizing
JIT compilation and performance
Application profiling
MustGather information for Linux
Known limitations on Linux
Windows problem determination
Setting up and checking your Windows environment
Windows 32-bit large address aware support
General debugging techniques
System dump
Diagnosing crashes in Windows
Data to send to IBM
Debugging hangs
Getting a dump from a hung JVM
Analyzing deadlocks
Debugging memory leaks
The Windows memory model
Classifying leaks
Tracing leaks
–Xrunjnichk option
–memorycheck option
Using Heapdump to debug memory leaks
OutOfMemoryError creating a thread
Debugging performance problems
Finding the bottleneck
Windows systems resource usage
JVM heap sizing
JIT compilation and performance
Application profiling
MustGather information for Windows
z/OS problem determination
Setting up and checking your z/OS environment
Maintenance
LE settings
Environment variables
Private storage usage
Setting up dumps on z/OS
General debugging techniques
Using IPCS commands
Using dbx
Interpreting error message IDs
Diagnosing crashes
Documents to gather
Determining the failing function
Working with TDUMPs using IPCS
Useful IPCS commands and some sample output
Debugging hangs
The process is deadlocked
The process is looping
The process is performing badly
Understanding Memory Usage
Allocations to LE HEAP
z/OS virtual storage
OutOfMemoryError exceptions
Debugging performance problems
Finding the bottleneck
z/OS systems resource usage
JVM heap sizing
JIT compilation and performance
Application profiling
MustGather information for z/OS
IBM i problem determination
Determining which VM is in use
Setting up your IBM Technology for Java Environment
Required Software and Licensing
Configuring JAVA_HOME
Enabling i5/OS PASE core files
Operating system settings
Java Virtual Machine settings
Available disk space
Setting environment variables for i5/OS PASE or QShell
Determining the home directory for a user
Setting default Java command-line options
General debugging techniques
Diagnosing problems at the command line
IBM i debugging commands
Work with Active Jobs (WRKACTJOB)
Work with Job (WRKJOB)
Work with System Status (WRKSYSSTS)
Work with Disk Status (WRKDSKSTS)
Process Status (ps)
Debugger (dbx)
Debugging performance problems
Analyzing CPU bottlenecks
Work with Active Job (WRKACTJOB)
Work with System Activity (WRKSYSACT)
Performance Explorer
Analyzing memory problems
Work with System Status (WRKSYSSTS)
Process status (ps)
Analyzing I/O problems
Work with System Status (WRKSYSSTS)
Work with Disk Status (WRKDSKSTS)
Work with TCP/IP Network Status (WRKTCPSTS or NETSTAT)
Communications trace
Print from raw data collected
Print from stream file
Diagnosing crashes
Checking the system environment
Finding out about the Java environment
Detailed crash diagnosis
Diagnosing hangs
i5/OS deadlocks
i5/OS busy hangs
Understanding memory usage
The 32-bit i5/OS PASE Virtual memory model
The process and garbage-collected heaps
Monitoring the garbage-collected heap
Process heap usage
OutOfMemoryError exceptions
Garbage-collected heap exhaustion
Submitting a bug report
Using dbx
Using the DBX Plug-in for Java
Example: testing the integrity of the heap image
Important dbx usage notes and warnings
Using dbx to investigate a Java system dump
Starting dbx on a system dump (core.{date}.{time}.{pid}.dmp)
Sun Solaris problem determination
Hewlett-Packard SDK problem determination
ORB problem determination
Identifying an ORB problem
Debug properties
ORB exceptions
Completion status and minor codes
Java security permissions for the ORB
Interpreting the stack trace
Description string
Interpreting ORB traces
Message trace
Comm traces
Client or server
Service contexts
Common problems
ORB application hangs
Running the client without the server running before the client is started
Client and server are running, but not naming service
Running the client with MACHINE2 (client) unplugged from the network
IBM ORB service: collecting data
Preliminary tests
NLS problem determination
Overview of fonts
Font utilities
Common NLS problem and possible causes
Attach API problem determination
Using diagnostic tools
Overview of the available diagnostics
Categorizing the problem
Summary of diagnostic information
Summary of cross-platform tooling
Heapdump analysis tooling
Cross-platform dump viewer
JVMTI tools
JVMPI tools
JPDA tools
DTFJ
Trace formatting
JVMRI
Using dump agents
Using the -Xdump option
Merging -Xdump agents
Dump agents
Console dumps
System dumps
Stack dumps
LE CEEDUMPs
Tool option
Javadumps
Heapdumps
Snap traces
Dump events
Advanced control of dump agents
exec option
file option
filter option
opts option
Priority option
range option
request option
defaults option
Dump agent tokens
Default dump agents
Removing dump agents
Dump agent environment variables
Signal mappings
Windows, Linux, AIX, and i5/OS specifics
Using dump agents on z/OS
Using Javadump
Enabling a Javadump
Triggering a Javadump
Interpreting a Javadump
Javadump tags
TITLE, GPINFO, and ENVINFO sections
Storage Management (MEMINFO)
Locks, monitors, and deadlocks (LOCKS)
Threads and stack trace (THREADS)
Understanding Java thread details
Classloaders and Classes (CLASSES)
Environment variables and Javadump
Using Heapdump
Information for users of previous releases of Heapdump
Getting Heapdumps
Enabling text formatted ("classic") Heapdumps
Available tools for processing Heapdumps
Using -Xverbose:gc to obtain heap information
Environment variables and Heapdump
Text (classic) Heapdump file format
Using system dumps and the dump viewer
Overview of system dumps
System dump defaults
Overview of the dump viewer
Using the dump extractor, jextract
Using the dump viewer, jdmpview
Dump viewer commands
General commands
Working with locks
Showing details of a dump
Analyzing the memory
Working with classes
Working with objects
Generating Heapdumps
Working with trace
Example session
Tracing Java applications and the JVM
What can be traced?
Types of tracepoint
Default tracing
Where does the data go?
Writing trace data to memory buffers
Writing trace data to a file
External tracing
Tracing to stderr
Trace combinations
Controlling the trace
Specifying trace options
Detailed descriptions of trace options
none
properties[=<filename>]
buffers=nnnk|nnnm[,dynamic|nodynamic]
Options that control tracepoint activation
Tracepoint specification
Trace levels
method=<method_specification>[,<method_specification>]
output=<filename>[,sizem[,<generations>]]
exception.output=<filename>[,nnnm]
resume
resumecount=<count>
stackdepth=<n>
suspend
suspendcount=<count>
trigger=<clause>[,<clause>][,<clause>]...
Using the Java API
Using the trace formatter
Determining the tracepoint ID of a tracepoint
Application trace
Implementing application trace
Registering for trace
Tracepoints
Printf specifiers
Example HelloWorld with application trace
Using application trace at run time
Using method trace
Running with method trace
Untraceable methods
Examples of use
Example of method trace output
JIT problem determination
Diagnosing a JIT problem
Disabling the JIT compiler
Selectively disabling the JIT compiler
Locating the failing method
Identifying JIT compilation failures
Performance of short-running applications
JVM behavior during idle periods
The Diagnostics Collector
Introduction to the Diagnostics Collector
Using the Diagnostics Collector
Collecting diagnostics from Java runtime problems
Verifying your Java diagnostics configuration
Configuring the Diagnostics Collector
Diagnostics Collector settings
Known limitations
Garbage Collector diagnostics
How do the garbage collectors work?
Common causes of perceived leaks
Listeners
Hash tables
Static class data
JNI references
Objects with finalizers
-verbose:gc logging
Global collections
Garbage collection triggered by System.gc()
Allocation failures
New area allocation failures
Tenured allocation failures
Scavenger collections
Concurrent garbage collection
Concurrent sweep completed
Concurrent kickoff
Allocation failures during concurrent mark
Concurrent aborted
Concurrent halted
Concurrent collection
System.gc() calls during concurrent mark
Timing problems during garbage collection
-Xtgc tracing
-Xtgc:backtrace
-Xtgc:compaction
-Xtgc:concurrent
-Xtgc:dump
-Xtgc:excessiveGC
-Xtgc:freelist
-Xtgc:parallel
-Xtgc:references
-Xtgc:scavenger
-Xtgc:terse
Finding which methods allocated large objects
Class-loader diagnostics
Class-loader command-line options
Class-loader runtime diagnostics
Loading from native code
Shared classes diagnostics
Deploying shared classes
Cache naming
Cache access
Cache housekeeping
Cache performance
Compatibility between service releases
Nonpersistent shared cache cleanup
Dealing with runtime bytecode modification
Potential problems with runtime bytecode modification
Modification contexts
SharedClassHelper partitions
Using the safemode option
Further considerations for runtime bytecode modification
Understanding dynamic updates
Using the Java Helper API
SharedClassHelper API
Understanding shared classes diagnostics output
Verbose output
VerboseIO output
VerboseHelper output
printStats utility
printAllStats utility
Debugging problems with shared classes
Using shared classes trace
Why classes in the cache might not be found or stored
Dealing with initialization problems
Dealing with verification problems
Dealing with cache problems
Class sharing with OSGi ClassLoading framework
Using the Reliability, Availability, and Serviceability Interface
Preparing to use JVMRI
Writing an agent
Registering a trace listener
Changing trace options
Starting the agent
Building the agent
Agent design
JVMRI functions
API calls provided by JVMRI
CreateThread
DumpDeregister
DumpRegister
DynamicVerbosegc
GenerateHeapdump
GenerateJavacore
GetComponentDataArea
GetRasInfo
InitiateSystemDump
InjectOutOfMemory
InjectSigSegv
NotifySignal
ReleaseRasInfo
RunDumpRoutine
SetOutOfMemoryHook
TraceDeregister
TraceDeregister50
TraceRegister
TraceRegister50
TraceResume
TraceResumeThis
TraceSet
TraceSnap
TraceSuspend
TraceSuspendThis
RasInfo structure
RasInfo request types
Intercepting trace data
The -Xtrace:external=<option>
Calling external trace
Formatting
Using the HPROF Profiler
Explanation of the HPROF output file
Using the JVMTI
Using the Diagnostic Tool Framework for Java
Using the DTFJ interface
DTFJ example application
Using JConsole
Using the IBM Monitoring and Diagnostic Tools for Java - Health Center
Introduction
Platform requirements
Monitoring a running Java application
Installing the Health Center agent
Starting a Java application with the Health Center agent enabled
Connecting to a Java application using the Health Center client
Connecting to a Java application using authentication
Configuring WebSphere or Rational product environments
Configuring WebSphere Application Server environments
Configuring WebSphere Integration Developer environments
Configuring Rational Application Developer environments
Data available on connection to a running Java application
Controlling the amount of data generated
Saving data
Opening files from disk
Classes perspective
Using the classes perspective
Class references
Environment perspective
Environment references
Garbage collection perspective
Using the garbage collection perspective
Garbage collection references
I/O perspective
Locking perspective
Using the Locking perspective
Resolving lock contention
Locking references
Native memory perspective
Profiling perspective
Method profiling
Method Profile view
Invocation Paths
Called methods
Timeline
Method profiling references
WebSphere Real Time perspective
Introduction to the WebSphere Real Time perspective
Setting preferences for the WebSphere Real Time perspective
Views within the WebSphere Real Time perspective
Customizing the WebSphere Real Time perspective
Troubleshooting
Cannot connect to an application
Cleaning up temporary files
Data disappears
GUI unresponsive
Hangs
Monitored application runs out of native memory or crashes
No data present
No I/O information present
No method names showing
Only the first character of file names showing
Out of memory errors and ISA 4.1
Printing
Showing the Status perspective
Problems when using WebSphere Application Server - Community Edition
Resetting displayed data
Cropping data
Controlling the units
Filtering
Performance hints
Reference
Command-line options
Specifying command-line options
General command-line options
System property command-line options
JVM command-line options
-XX command-line options
JIT command-line options
Garbage Collector command-line options
Messages
DUMP messages
JVMDUMP000E
JVMDUMP001E
JVMDUMP002W
JVMDUMP003E
JVMDUMP004E
JVMDUMP005E
JVMDUMP006I
JVMDUMP007I
JVMDUMP008I
JVMDUMP009E
JVMDUMP010I
JVMDUMP011I
JVMDUMP012E
JVMDUMP013I
JVMDUMP014E
JVMDUMP015I
JVMDUMP016I
JVMDUMP017I
JVMDUMP018W
JVMDUMP019I
JVMDUMP020I
JVMDUMP021W
JVMDUMP022W
JVMDUMP023W
JVMDUMP024W
JVMDMP025I
JVMDUMP026
JVMDUMP027W
JVMDUMP028W
JVMDUMP029W
JVMDUMP030W
JVMDUMP031W
JVMDUMP032I
JVMDUMP033I
JVMDUMP034I
JVMDUMP035I
JVMDUMP036I
JVMDUMP037E
JVMDUMP038
J9VM messages
JVMJ9VM000
JVMJ9VM001
JVMJ9VM002
JVMJ9VM003
JVMJ9VM004E
JVMJ9VM005E
JVMJ9VM006E
JVMJ9VM007E
JVMJ9VM008
JVMJ9VM009
JVMJ9VM010W
JVMJ9VM011W
JVMJ9VM012W
JVMJ9VM013W
JVMJ9VM014W
JVMJ9VM015W
JVMJ9VM016W
JVMJ9VM017
JVMJ9VM018
JVMJ9VM032
JVMJ9VM033
JVMJ9VM035
JVMJ9VM038E
JVMJ9VM039
SHRC messages
JVMSHRC004E
JVMSHRC005
JVMSHRC006I
JVMSHRC007I
JVMSHRC008I
JVMSHRC009I
JVMSHRC010I
JVMSHRC012I
JVMSHRC013E
JVMSHRC014E
JVMSHRC015
JVMSHRC017E
JVMSHRC018
JVMSHRC019
JVMSHRC020
JVMSHRC021
JVMSHRC022
JVMSHRC023E
JVMSHRC024
JVMSHRC025
JVMSHRC026E
JVMSHRC027E
JVMSHRC028E
JVMSHRC029E
JVMSHRC030E
JVMSHRC031E
JVMSHRC032E
JVMSHRC057
JVMSHRC058
JVMSHRC059
JVMSHRC060
JVMSHRC061
JVMSHRC062
JVMSHRC063
JVMSHRC064
JVMSHRC065
JVMSHRC066
JVMSHRC067
JVMSHRC068
JVMSHRC069
JVMSHRC070
JVMSHRC071
JVMSHRC072
JVMSHRC077
JVMSHRC078
JVMSHRC079
JVMSHRC080
JVMSHRC081
JVMSHRC082
JVMSHRC083
JVMSHRC084
JVMSHRC085
JVMSHRC086
JVMSHRC087
JVMSHRC088
JVMSHRC089
JVMSHRC090
JVMSHRC091
JVMSHRC092
JVMSHRC093
JVMSHRC094
JVMSHRC095
JVMSHRC096
JVMSHRC097
JVMSHRC125
JVMSHRC126
JVMSHRC127
JVMSHRC128
JVMSHRC129
JVMSHRC130
JVMSHRC131
JVMSHRC132
JVMSHRC133
JVMSHRC134
JVMSHRC135
JVMSHRC136
JVMSHRC137
JVMSHRC147
JVMSHRC154
JVMSHRC155
JVMSHRC156
JVMSHRC157
JVMSHRC158
JVMSHRC159
JVMSHRC160
JVMSHRC161
JVMSHRC162
JVMSHRC166
JVMSHRC168
JVMSHRC169
JVMSHRC171
CORBA minor codes
CONN_CLOSE_REBIND
CONN_PURGE_ABORT
CONNECT_FAILURE_1
CONNECT_FAILURE_5
CREATE_LISTENER_FAILED
LOCATE_UNKNOWN_OBJECT
NULL_PI_NAME
ORB_CONNECT_ERROR_6
POA_DISCARDING
RESPONSE_INTERRUPTED
TRANS_NC_LIST_GOT_EXC
UNEXPECTED_CHECKED_EXCEPTION
UNSPECIFIED_MARSHAL_25
Environment variables
Displaying the current environment
Setting an environment variable
Separating values in a list
JVM environment settings
z/OS environment variables
Default settings for the JVM
Notices
Trademarks
ibm.com: About IBM - Privacy - Contact
About IBM
Privacy
Contact