IBM WebSphere Application Server Performance Cookbook

WebSphere Liberty OpenTelemetry log Recipe

WebSphere Liberty OpenTelemetry may be used to periodically log performance statistics (e.g. thread pool utilization, JDBC connection pool utilication, etc.) to disk.

  1. The mpTelemetry feature must be installed. If needed, verify with productInfo; for example: 
    $ LIBERTY/bin/productInfo featureInfo | grep mpTelemetry
mpTelemetry-2.1
    • If it's not installed, you may install it with installUtility and then restart Liberty; for example: 
      LIBERTY/bin/installUtility install mpTelemetry-2.1
  2. Set the following environment variables to send the data to messages.log; for example, in server.env: 
    OTEL_METRIC_EXPORT_INTERVAL=60000
OTEL_SDK_DISABLED=false
OTEL_SERVICE_NAME=otellog
OTEL_LOGS_EXPORTER=none
OTEL_TRACES_EXPORTER=none
OTEL_METRICS_EXPORTER=console
OTEL_JAVA_DISABLED_RESOURCE_PROVIDERS=io.opentelemetry.instrumentation.resources.HostResourceProvider,io.opentelemetry.instrumentation.resources.ProcessResourceProvider,io.opentelemetry.instrumentation.resources.OsResourceProvider,io.opentelemetry.instrumentation.resources.ProcessRuntimeResourceProvider
  3. Add the mpTelemetry-2.1 feature to server.xml (or a configDropin override). For example: 
    <featureManager><feature>mpTelemetry-2.1</feature></featureManager>
    1. Note that Liberty defaults to a maximum of 20MB for messages.log with up to one historical file. If needed, you may increase this; for example, for a total of up to 1GB with 10 files up to 100MB each: 
      <logging maxFileSize="100" maxFiles="10"  />
  4. Restart Liberty to pick up the new envars.
  5. Every OTEL_METRIC_EXPORT_INTERVAL milliseconds, you should see OpenTelemetry statistics output in messages.log; for example, here's a statistic output for active threads (io.openliberty.threadpool.active_threads) for the Default Executor thread pool showing value=0 over 2 minutes: 
    [8/19/26, 10:39:21:572 CDT] 00000045 io.opentelemetry.exporter.logging.LoggingMetricExporter      I metric: ImmutableMetricData{resource=Resource{schemaUrl=null, attributes={service.instance.id="8b388da6-56ea-4920-8b1c-23076e381adf", service.name="otellog", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.48.0"}}, instrumentationScopeInfo=InstrumentationScopeInfo{name=io.openliberty.monitor.metrics, version=null, schemaUrl=null, attributes={}}, name=io.openliberty.threadpool.active_threads, description=The number of threads that are actively running tasks., unit={thread}, type=LONG_SUM, data=ImmutableSumData{points=[ImmutableLongPointData{startEpochNanos=1787153901540000000, epochNanos=1787153961544000000, attributes={io.openliberty.threadpool.name="Default Executor"}, value=0, exemplars=[]}], monotonic=false, aggregationTemporality=CUMULATIVE}}
[8/19/26, 10:40:21:559 CDT] 00000045 io.opentelemetry.exporter.logging.LoggingMetricExporter      I metric: ImmutableMetricData{resource=Resource{schemaUrl=null, attributes={service.instance.id="8b388da6-56ea-4920-8b1c-23076e381adf", service.name="otellog", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.48.0"}}, instrumentationScopeInfo=InstrumentationScopeInfo{name=io.openliberty.monitor.metrics, version=null, schemaUrl=null, attributes={}}, name=io.openliberty.threadpool.active_threads, description=The number of threads that are actively running tasks., unit={thread}, type=LONG_SUM, data=ImmutableSumData{points=[ImmutableLongPointData{startEpochNanos=1787153901540000000, epochNanos=1787154021544000000, attributes={io.openliberty.threadpool.name="Default Executor"}, value=0, exemplars=[]}], monotonic=false, aggregationTemporality=CUMULATIVE}}
    1. All statistics will be output except those excluded with OTEL_JAVA_DISABLED_RESOURCE_PROVIDERS. In the example above, a lot more statistics were printed each interval but were removed for brevity.
  6. Reproduce the problem.
  7. Upload the following files:
    1. logs/messages*log
    2. logs/console.log
    3. logs/ffdc/*

Notes:

  1. A simple server without an application generates about 20KB per iteration, including the Default Executor thread pool size and active count, Java heap usage, GC duration, number of CPUs, JVM CPU utilization, JVM thread count, class loading and unloading, etc., but this will increase as the number of resources increases (e.g. connection pools, etc.).