Linux Simple perf Recipe

Linux perf record gathers high frequency native stack samples of what's using the CPU. The overhead is generally in the range of a few % and is production ready although it's best to gauge the overhead in a test environment.

1. Install perf if it's not installed. To check if it's installed:

   perf version

2. As root, during the performance problem, run the following command. Change 60 at the end to the number of seconds you want to gather data for:

   date +'%Y-%m-%d %H:%M:%S.%N %Z' &>> diag_starttimes_$(hostname).log; cat /proc/uptime &>> diag_starttimes_$(hostname).log; perf record --call-graph dwarf,5528 -F 99 -a -g -T -- sleep 60

3. Wait for the above command to complete.
4. As root from the directory where perf record was run, execute the following commands:

   perf script --header -I -f -F comm,cpu,pid,tid,time,event,ip,sym,dso,symoff > diag_perfscript_$(hostname)_$(date +%Y%m%d_%H%M%S_%N).stdout.txt 2> diag_perfscript_$(hostname)_$(date +%Y%m%d_%H%M%S_%N).stderr.txt
   perf report -n --show-cpu-utilization -v --stdio > diag_perfreport_$(hostname)_$(date +%Y%m%d_%H%M%S_%N).stdout.txt 2> diag_perfreport_$(hostname)_$(date +%Y%m%d_%H%M%S_%N).stderr.txt
   perf archive --all >diag_archive.txt 2>&1 || perf archive >>diag_archive.txt 2>&1
   tar czhvf diag_perfall_$(hostname)_$(date +%Y%m%d_%H%M%S).tar.gz diag* perf*bz2

5. Upload diag_perfall_*.tar.gz

For background, see Linux perf.

Previous Section (Linux syscall times) | Next Section (Troubleshooting AIX Recipes) | Back to Table of Contents