Troubleshooting AIX
Java interaction
If Java uses shmget to allocate the Java heap, then it
will immediately mark the shared memory region for deletion, so that if
the JVM crashes, the memory will be released. Therefore, if you find
large shared memory regions marked for deletion
(ipcs -Smqsa -1 | egrep "^m" | egrep " D"), this is most
likely the reason and expected.
Request core dump (also known as a "system dump" for IBM Java)
Additional methods of requesting system dumps for IBM Java are documented in the Troubleshooting IBM Java and Troubleshooting WAS chapters.
The gencore command pauses the process while the core is generated and then the process should continue. Replace PIDinthefollowingexamplewiththeprocessID.Youmusthavepermissionstotheprocess(i.e.eitherrunastheowneroftheprocessorasroot)andthecoreulimitmustbeunlimited((ulimit -c unlimited) before starting the process). The size of the core file will be the size of the virtual size of the process (ps VSZ). If there is sufficient free space in physical RAM and the filecache, the core file will be written to RAM and then asynchronously written out to the filesystem which can dramatically improve the speed of generating a core and reduce the time the process is paused. In general, core dumps compress very well (often up to 75%) for transfer.
gencore PIDcore.(date +%Y%m%d.%H%M%S).dmp
Signals
Signal mappings and detailed error codes may be found in /usr/include/sys/signal.h.
A simpler listing may be performed with kill -l:
$ kill -l
1) HUP 14) ALRM 27) MSG 40) bad trap 53) bad trap
2) INT 15) TERM 28) WINCH 41) bad trap 54) bad trap
3) QUIT 16) URG 29) PWR 42) bad trap 55) bad trap
4) ILL 17) STOP 30) USR1 43) bad trap 56) bad trap
5) TRAP 18) TSTP 31) USR2 44) bad trap 57) bad trap
6) ABRT 19) CONT 32) PROF 45) bad trap 58) RECONFIG
7) EMT 20) CHLD 33) DANGER 46) bad trap 59) CPUFAIL
8) FPE 21) TTIN 34) VTALRM 47) bad trap 60) GRANT
9) KILL 22) TTOU 35) MIGRATE 48) bad trap 61) RETRACT
10) BUS 23) IO 36) PRE 49) bad trap 62) SOUND
11) SEGV 24) XCPU 37) VIRT 50) bad trap 63) SAK
12) SYS 25) XFSZ 38) ALRM1 51) bad trap
13) PIPE 26) bad trap 39) WAITING 52) bad trapFind PID that owns a socket
Method 1
Install the optional lsof tool.
Method 2
$ netstat -Aan | grep "*.2[2,5].*LISTEN"
f1000e000531a3b8 tcp4 0 0 *.22 *.* LISTEN
f1000e00040babb8 tcp4 0 0 *.25 *.* LISTEN$ rmsock f1000e000531a3b8 tcpcb
The socket 0xf1000e000531a008 is being held by proccess 9830644
(sshd).
$ rmsock f1000e00040babb8 tcpcb
The socket 0xf1000e00040ba808 is being held by proccess 6684754
(sendmail).
Method 3
Find the socket of interest with netstat -A and copy the first hexadecimal address which is the socket ID:
# netstat -Aan | grep 32793 **f1000e00032203b8** tcp4 0 0 127.0.0.1.32793 *.* LISTENSend this socket ID into kdb:
# echo "sockinfo f1000e00032203b8 tcpcb" | kdb | grep proc F1000F0A00000000 F1000F0A10000000 pvproc+000000 proc/fd: 65/8 proc/fd: fd: 8 pvproc+010400 65*kuxagent ACTIVE **0410058** 0000001 00000008A84D5590 0 0030Take the first hexadecimal address after ACTIVE and convert it to decimal:
# echo "hcal 0410058" | kdb | grep Value Value hexa: 00410058 Value decimal: 4259928Search for this PID in
ps:# ps -elf | grep 4259928
Kernel Trace
Trace source of kill signal
It may be useful to understand what PID is sending a kill signal to a process on AIX. You can use this kernel trace:
Login as root
# rm -rf /tmp/aixtrace; mkdir /tmp/aixtrace/; cd /tmp/aixtrace/
# trace -C all -a -T 10M -L 20M -n -j 134,139,465,14e,46c -o ./trc
... Reproduce the problem ... e.g. kill -3 7667754
# trcstop
# cp /etc/trcfmt .
# trcnm -a > trace.nm
# LDR_CNTRL=MAXDATA=0x80000000 gensyms > trace.syms
# LDR_CNTRL=MAXDATA=0x80000000 gennames -f > gennames.out
# pstat -i > trace.inode
# ls -al /dev > trace.maj_min2lvEither zip and send these files to a PMR or analysis machine, or run these commands directly to process the trace:
# trcrpt -C all -r -o trc.tr trc
# trcrpt -C all -t trcfmt -n trace.nm -x -O pid=on,tid=on,svc=on,exec=on,cpuid=on,PURR=on -o trc.txt trc.trMake sure the trace buffers did not wrap:
# grep WRAP trc.txtIf there are no results, then you're good; otherwise, if you see lines such as:
006 --1- -1 -1 -1 963.205627656 0.002912 963.205627656 TRACEBUFFER WRAPAROUND 0003
005 -4916246- -1 4916246 113967573 963.205627656* 963.205627656 LOGFILE WRAPAROUND 0002Then, either try increasing buffer sizes or reducing your test case or system load (or the tracepoints in -j).
Finally, search for the signal:
# grep -Ei "^14e|46c" trc.txt | grep -E "signal 3|SIGQUIT"
14E ksh 0 10879036 62128373 28.157542500 0.128249 28.157542500 kill: signal SIGQUIT to process ? javaThe time of the signal is the ELAPSED_SEC column added to the date at the top of trc.txt:
# head -2 trc.txtWed Aug 21 05:10:28 2013
Thus the kill was sent at 05:10:56 by PID 10879036 (ksh). If this is a long running process, then you can reference ps.out for more details. The entry may not print the PID the signal was sent to (notice the question mark), but you should be able to figure that out based on other artifacts produced at that time such as javacores.
Useful Commands
genkld: List all shared objects loaded on the system.genld: List all shared objects loaded by a process.slibclean: Remove shared libraries from memory that have 0 use and load counts.
Querying Queue Depth with Netstat
Example AIX netstat output:
$ netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 10.30.30.36.80 10.30.30.113.38378 ESTABLISHED [...]AIX has a somewhat strange format with the IP address followed by a period (instead of a colon) and the port number.
To find the queue depth of a server, first search for the local IP followed by the listening port of the server (it doesn't matter which column it's in because it's impossible for a server to both listen on a port and use that same port as a client port for some other socket), and filter the output for sockets in ESTABLISHED or SYN_RECEIVED states. Sockets may also be in various closing states such as CLOSE_WAIT, TIME_WAIT, FIN_WAIT_1, and FIN_WAIT_2 (depending on which side closed its half of the socket first): these states can cause other types of bottlenecks but they are not related to queue depth.
For example, let's say the web server above is listening on port 80:
$ netstat -an | grep -F 10.30.30.36.80 | grep -e ESTABLISHED -e SYN_RECEIVED | wc -l
14Then, to get the queue depth, subtract this number from the maximum number of servers (e.g. MaxClients, WebContainer maximum size, etc.).
Debug Symbols
AIX: "Specifying -g will turn off all inlining unless you explicitly request it with an optimization option." (http://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.ibm.xlc1313.aix.doc/compiler_ref/opt_g_lower.html)
Use stabsplit to create separate symbol files: http://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/com.ibm.aix.cmds3/ld.htm
Analyzing Native Memory with svmon
AIX, like all other modern operating systems, aggressively uses RAM as a file cache; therefore, it's very common for the "free" RAM in AIX to show as very low. However, in general, this file cache can be pushed out of RAM to make space for program demands. Therefore, to understand "effective RAM usage", you must subtract non-pinned filecache from used RAM.
aixmem.sh:
#!/bin/sh
PID=$1
INTERVAL=3600
echo "PID=$PID, INTERVAL=$INTERVAL"
while ([ -d /proc/$PID ]); do
date
svmon -G
svmon -r -m -P $PID
kill -3 $PID
sleep $INTERVAL
doneRun the following to make the script executable:
chmod a+x aixmem.shStart the script (and replace $PID with the target Java process ID):
nohup ./aixmem.sh $PID > nativemem.txt 2>&1 &This runs at an interval of 30 minutes and has a very low overhead
(assuming you have not changed the default behavior of
kill -3 to only produce a javacore).
After enough data has been captured, kill the script:
kill $(ps -ef | grep nativemem.sh | grep -v grep | awk '{print $2}')The following command line snippet on Linux (should be easy to convert to ksh) may be used to analyze the output of the AIX memory script above and display the effectively used RAM (i.e. RAM usage - file cache):
grep -e TZ -e "^memory" -e "^in use" nativemem.txt | \
while read line; do \
read line2;
read line3;
echo $line;
rambytesused="$(echo "${line2}" | awk '{print $3}' | echo "$(cat -)*4096" | bc)";
rambytesfilecache="$(echo "${line3}" | awk '{print $5}' | echo "$(cat -)*4096" | bc)";
printf "Total RAM usage (bytes): %s\n" "${rambytesused}";
printf "File cache RAM usage (bytes): %s\n" "${rambytesfilecache}";
printf "Effectively used RAM (bytes): %s\n\n" "$(echo "${rambytesused}-${rambytesfilecache}" | bc)";
doneNative Memory Leaks
Restart the process with the MALLOCDEBUG envar to track un-freed mallocs. This may have a significant performance overhead:
export MALLOCDEBUG=report_allocations,stack_depth:3Reproduce the problem and stop the process gracefully. A report is produced in stderr with each un-freed allocation:
Allocation #0: 0x3002FD60
Allocation size: 0x2A0
Allocated from heap: 0
Allocation traceback:
0xD01DC934 malloc
0xD01288AC init_malloc
0xD012A1F4 malloc
0xD04DFF34 __pth_initSnapshots of this data may also be captured with a core dump (see above) and the dbx $(malloc) command (see below).
With Java, careful of using stack depths greater than 3:
"The stack depth of 3 provides only a limited stack trace. However, the use of larger stack depths with a Java application can cause crashes because the debug malloc facility does not understand the stack frames used for JIT compiled code."
Run format_mallocdebug_op.sh to aggregate and summarize the stacks: https://www.ibm.com/developerworks/aix/library/au-mallocdebug.html
Example output:
ZIP_Put_In_Cache
readCEN
calloc_common
malloc
################################
533676 bytes leaked in 127 Blocks
################################dbx
Analyze a core file:
$ dbx ${PATH_TO_EXECUTABLE} ${PATH_TO_CORE}To load shared libraries from a particular folder, user -p:
$ dbx -p /=./ ${PATH_TO_EXECUTABLE} ${PATH_TO_CORE}If you see the following warning:
warning: The core file is not a fullcore. Some info may not be available.Then the core is probably truncated. As recommended in the Java documentation, enable fullcore and reproduce the issue (http://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.aix.80.doc/diag/problem_determination/aix_setup_full_core.html):
# chdev -l sys0 -a fullcore='true' -a pre430core='false'Tips
Type help $COMMAND to print the summary and options of a
command. For example:
(dbx) help proc
proc [raw] [cred | cru | rlimit | ru | sigflags | signal]
Display process information. "raw" displays in raw hex format.
"cred" shows credentials, "cru", "ru" and "rlimit" resource info,
"sigflags" and "signal" information about signals and handlers.Command output may be redirected to files in the same directory. For example:
(dbx) coremap > coremap.txtproc
proc prints general process information. For
example:
(dbx) proc
pi_pid: 9306144 pi_sid: 10354784
pi_ppid: 9961578 pi_pgrp: 204
pi_uid: 204 pi_suid: 204
pi_thcount: 342 pi_cpu: 0
pi_start: Tue Dec 9 06:09:20 2014
pi_tsize: 0x0000000000013eeb pi_dsize: 0x000000003ba99c00...thread
thread prints a list of all native threads. The thread
preceded with ">" is the current thread:
(dbx) thread
thread state-k wchan state-u k-tid mode held scope function
$t1 run 0xf1000f0a1015c140 blocked 96535077 k no sys _event_sleep
$t190 run running 123339089 k no sys pollset_poll
>$t286 run running 96272413 k no sys genSystemCoreUsingGencore...Native Stacks
On AIX, to calculate the total native memory used by native stacks:
(dbx) thread infoFor example, under "stack storage," the native stack size is next to size=:
thread state-k wchan state-u k-tid mode held scope function
$t1 run blocked 28999789 u no sys _event_sleep...
stack storage:
base = 0x2df23000 size = 0x1ffc830where
where prints a native stack of the current thread. For
example:
(dbx) where
genSystemCoreUsingGencore() at 0x9000000177a6ef4
j9dump_create() at 0x9000000177a6370
doSystemDump() at 0x9000000177ed2cc
protectedDumpFunction() at 0x9000000177f2b54
j9sig_protect() at 0x90000001777dc9c
runDumpFunction() at 0x9000000177f2aa4
runDumpAgent() at 0x9000000177f26cc
createAndRunOneOffDumpAgent() at 0x9000000177f22a0
triggerOneOffDump() at 0x900000017814598
Java_com_ibm_jvm_Dump_SystemDumpImpl() at 0x9000000186cb198To print the stack of a particular thread, reference the number with $t#:
(dbx) where $t190
Thread $t190
warning: Thread is in kernel mode, not all registers can be accessed.
pollset_poll(??, ??, ??, ??) at 0x90000000014e56c
pollStatus() at 0x900000005e8ca30map
map prints a list of all loaded modules. For
example:
(dbx) map
Entry 1:
Object name: ./opt/BPM/8.5/WebSphere/AppServer/java/jre/bin/java
Text origin: 0x10000000000
Text length: 0x17236
Data origin: 0x1001000012b
Data length: 0x194d
File descriptor: 0x5
...
Entry 64:
Object name: ./usr/lib/libc.a
Member name: shr_64.o
Text origin: 0x900000000000c80
Text length: 0x43b3bf
Data origin: 0x9001000a00007e0
Data length: 0x11d8a0
File descriptor: 0x83coremap
coremap prints a list of all memory mappings. For
example:
Mapping: Shared Memory (size=0x280000000)
from (address): 0xa00000000000000 - 0xa00000280000000
to (offset) : 0x421680be - 0x2c21680be
in file : core.20141209.175356.9306144.0002.dmpIn the above example, the virtual address range is from 0xa00000000000000 to 0xa00000280000000 (which is of length 0x280000000 reported in the first line), and the raw data may be found in the file core.20141209.175356.9306144.0002.dmp in the range 0x421680be to 0x2c21680be. We can verify this by dumping the first 2 words of the virtual address:
(dbx) 0xa00000000000000/2X
0x0a00000000000000: 00000100 13737f3cThis matches dumping the same bytes from the core file at the offset:
$ od -N 8 -x core.20141209.175356.9306144.0002.dmp +0x421680be
421680be 0000 0100 1373 7f3cPrint memory
An address followed by a slash, a number, and a format character may be used to print raw memory. For example:
(dbx) 0x10000000000/8X
0x0000010000000000: 01f70005 51c013e3 00000000 0003a516
0x0000010000000010: 00781182 000015af 010b0001 00000000The help command for this is help display.
For null-terminated strings, simply type the address followed by /s.
For character bytes, use /c. For example:
(dbx) 0x20000000000/8c
0x0000020000000000: 'A' 'B' 'C' 'D' 'E' 'F' '1' '2'malloc
malloc prints a summary of the malloc subsystem. For
example:
(dbx) malloc
The following options are enabled:
Implementation Algorithm........ Default Allocator (Yorktown)
Statistical Report on the Malloc Subsystem:
Heap 0
heap lock held by................ pthread ID 0x1001000ee90
bytes acquired from sbrk()....... 1000964480
bytes in the freespace tree...... 125697184
bytes held by the user........... 875267296
allocations currently active..... 56012
allocations since process start.. 65224401
The Process Heap
Initial process brk value........ 0x0000010010001a80
current process brk value........ 0x000001004ba99c00
sbrk()s called by malloc......... 7180corefile
corefile prints information about a loaded core file.
For example:
(dbx) corefile
Process Name: /opt/IBM/WebSphere/AppServer/java/jre/bin/java
Version: 500
Flags: FULL_CORE | CORE_VERSION_1 | MSTS_VALID | UBLOCK_VALID | USTACK_VALID | LE_VALID
Signal:
Process Mode: 64 bitfd
fd prints information about open file handles.
Related Commands
- Print all printable strings and their hex offsets (at least N
printable characters followed by a null, default N is 4):
$ strings -t x $CORE_FILE - Print all bytes in both hexadecimal and character from the core file
starting at offset 0x988 and only show 100 bytes:
$ od -N 100 -xc $CORE_FILE +0x988 - Alternatively:
$ od -v -A x -N 100 -j 0x2B521000 -t xc $CORE_FILE
Compressing Files
compress
The compress command is a built-in command to compress files. If you're okay with replacing the file, then you can run it in place like this:
compress -f core.dmpThis will replace the file with a core.dmp.Z file which is compressed.
If you want to create a separately compressed file instead of replacing in-place, then you just pipe the file into compress and write to the desired file:
cat core.dmp | compress -f > core.dmp.ZTo package multiple files, se tar
to package multiple files, pipe that to stdout, then use compress to
take that tar data from stdin and then write that to a .Z
file:
tar -cvf - $FILES_OR_DIRS | compress -f > $NAME.ZThe reason for always using the -f flag is both to
overwrite existing files and also to avoid the issue of "This file is
not changed; compression does not save space." not writing the file.
tar.gz
If gzip
is available, use tar
to package multiple files, pipe that to stdout, then use gzip to take
that tar data from stdin and then write that to a .tar.gz
file:
tar -cvf - $FILES_OR_DIRS | gzip > $NAME.tar.gzSplitting Files
The split command splits a file into smaller files.
For example, let's say you have a core.dmp file. You can split it into 100MB files like this:
split -b 100m core.dmp core.dmp.split.This will produce files like this, each of which is no larger than 100MB:
core.dmp.split.aa
core.dmp.split.ab
core.dmp.split.acNetwork
DNS Cache
DNS caching is enabled with netcd. See if
netcd is enabled by running
lssrc -s netcd.
The netcdctrl -t all -a outputfile command dumps out the
DNS cache.
The netcdctrl -f -t $TYPE command flushes the cache.
Specify $TYPE as local, dns, nis, yp, ulm, or all.
ARP
- List ARP entries:
arp -a - There isn't an option to clear the entire ARGP cache but you may
delete an individual entry with
arp -d $HOST_OR_IP. This could be automated witharp -a | grep "stored"and then a script to delete
Example Script to Deny/Allow Packets on a port
#!/bin/ksh
WAS_PORT=9443
sample_intvl=20
curl_id=2172797
curl_msg="200 ok"
do_v6=0 # set to 1 if enabling firewall for IP V6 also
#------------------
# FUNCTIONS
show_usage()
{
echo "Usage: $0 [-p port] [-s sample_intvl][-F on|off]
echo "-p port\tspecifies the WAS port; default is 9443"
echo "-s sec\tspecifies how often in seconds to sample the port state"
echo "-F on/off\tManually turn on the firewall or turn off the firewall"
exit
}
create_filter()
{
genfilt -v4 -P $WAS_PORT -O eq -w I -a D -s 0 -m 0 -M 0 -D "Deny packets to port $WAS_PORT from all but loopback"
if [ "$do_v6" = 1 ]; then
genfilt -v6 -P $WAS_PORT -O eq -w I -a D -s 0 -m 0 -M 0 -D "Deny packets to port $WAS_PORT from all but loopback"
fi
}
chk_MFP_up()
{
while :; do
/usr/bin/netstat -an | /usr/bin/grep "\.$WAS_PORT .*LISTEN" >/dev/null
if [ $? != 0 ]; then
return
fi
sleep $sample_intvl
done
}
chk_url_ok()
{
while :; do
curl https://localhost:$WAS_PORT/api/adapter/is_okay?id=$curl_id | grep -i "$curl_msg" >/dev/null
if [ $? = 0 ]; then # 200_ok was returned
return
fi
sleep $sample_intvl
done
}
enable_firewall()
{
mkdev -l ipsec_v4 # enable IPsec kernel extension for IP V4
mkfilt -v4 -u # activate the filters
if [ "$do_v6" = 1 ]; then
mkdev -l ipsec_v6 # enable IPsec kernel extension for IP V4
mkfilt -v6 -u # activate the filters for IP V4
fi
}
disable_firewall()
{
rmfilt -v4 -d # disable the filter
rmdev -l ipsec_v4 # disable IPsec kernel extension
if [ "$do_v6" = 1 ]; then
rmfilt -v6 -d # disable the filter for IP V6
rmdev -l ipsec_v6 # disabl3 IPsec kernel extension for IP V6
fi
}
#--------------------------------------------------------
# MAIN PROGRAM
#
fw_state=""
while getopts p:s:F: flag; do
case $flag in
p) WAS_PORT=$OPTARG;;
s) sample_intvl=$OPTARG;;
F) fw_state=$OPTARG;;
\?) show_usage;;
esac
done
create_filter # create the filter at the beginning; okay if duplicate filter errmsg is returned
#
if [ "$fw_state" = "on" ]; then
enable_firewall
exit
elif [ "$fw_state" = "off" ]; then
disable_firewall
exit
fi
while :; do
chk_MFP_up
enable_firewall
chk_url_ok
disable_firewall
doneerror.h and errno.h
#define EPERM 1 /* Operation not permitted */
#define ENOENT 2 /* No such file or directory */
#define ESRCH 3 /* No such process */
#define EINTR 4 /* interrupted system call */
#define EIO 5 /* I/O error */
#define ENXIO 6 /* No such device or address */
#define E2BIG 7 /* Arg list too long */
#define ENOEXEC 8 /* Exec format error */
#define EBADF 9 /* Bad file descriptor */
#define ECHILD 10 /* No child processes */
#define EAGAIN 11 /* Resource temporarily unavailable */
#define ENOMEM 12 /* Not enough space */
#define EACCES 13 /* Permission denied */
#define EFAULT 14 /* Bad address */
#define ENOTBLK 15 /* Block device required */
#define EBUSY 16 /* Resource busy */
#define EEXIST 17 /* File exists */
#define EXDEV 18 /* Improper link */
#define ENODEV 19 /* No such device */
#define ENOTDIR 20 /* Not a directory */
#define EISDIR 21 /* Is a directory */
#define EINVAL 22 /* Invalid argument */
#define ENFILE 23 /* Too many open files in system */
#define EMFILE 24 /* Too many open files */
#define ENOTTY 25 /* Inappropriate I/O control operation */
#define ETXTBSY 26 /* Text file busy */
#define EFBIG 27 /* File too large */
#define ENOSPC 28 /* No space left on device */
#define ESPIPE 29 /* Invalid seek */
#define EROFS 30 /* Read only file system */
#define EMLINK 31 /* Too many links */
#define EPIPE 32 /* Broken pipe */
#define EDOM 33 /* Domain error within math function */
#define ERANGE 34 /* Result too large */
#define ENOMSG 35 /* No message of desired type */
#define EIDRM 36 /* Identifier removed */
#define ECHRNG 37 /* Channel number out of range */
#define EL2NSYNC 38 /* Level 2 not synchronized */
#define EL3HLT 39 /* Level 3 halted */
#define EL3RST 40 /* Level 3 reset */
#define ELNRNG 41 /* Link number out of range */
#define EUNATCH 42 /* Protocol driver not attached */
#define ENOCSI 43 /* No CSI structure available */
#define EL2HLT 44 /* Level 2 halted */
#define EDEADLK 45 /* Resource deadlock avoided */
#define ENOTREADY 46 /* Device not ready */
#define EWRPROTECT 47 /* Write-protected media */
#define EFORMAT 48 /* Unformatted media */
#define ENOLCK 49 /* No locks available */
#define ENOCONNECT 50 /* no connection */
#define ESTALE 52 /* no filesystem */
#define EDIST 53 /* old, currently unused AIX errno*/
/* non-blocking and interrupt i/o */
/*
* AIX returns EAGAIN where 4.3BSD used EWOULDBLOCK;
* but, the standards insist on unique errno values for each errno.
* A unique value is reserved for users that want to code case
* statements for systems that return either EAGAIN or EWOULDBLOCK.
*/
#if _XOPEN_SOURCE_EXTENDED==1
#define EWOULDBLOCK EAGAIN /* Operation would block */
#else /* _XOPEN_SOURCE_EXTENDED */
#define EWOULDBLOCK 54
#endif /* _XOPEN_SOURCE_EXTENDED */
#define EINPROGRESS 55 /* Operation now in progress */
#define EALREADY 56 /* Operation already in progress */
/* ipc/network software */
/* argument errors */
#define ENOTSOCK 57 /* Socket operation on non-socket */
#define EDESTADDRREQ 58 /* Destination address required */
#define EDESTADDREQ EDESTADDRREQ /* Destination address required */
#define EMSGSIZE 59 /* Message too long */
#define EPROTOTYPE 60 /* Protocol wrong type for socket */
#define ENOPROTOOPT 61 /* Protocol not available */
#define EPROTONOSUPPORT 62 /* Protocol not supported */
#define ESOCKTNOSUPPORT 63 /* Socket type not supported */
#define EOPNOTSUPP 64 /* Operation not supported on socket */
#define EPFNOSUPPORT 65 /* Protocol family not supported */
#define EAFNOSUPPORT 66 /* Address family not supported by protocol family */
#define EADDRINUSE 67 /* Address already in use */
#define EADDRNOTAVAIL 68 /* Can't assign requested address */
/* operational errors */
#define ENETDOWN 69 /* Network is down */
#define ENETUNREACH 70 /* Network is unreachable */
#define ENETRESET 71 /* Network dropped connection on reset */
#define ECONNABORTED 72 /* Software caused connection abort */
#define ECONNRESET 73 /* Connection reset by peer */
#define ENOBUFS 74 /* No buffer space available */
#define EISCONN 75 /* Socket is already connected */
#define ENOTCONN 76 /* Socket is not connected */
#define ESHUTDOWN 77 /* Can't send after socket shutdown */
#define ETIMEDOUT 78 /* Connection timed out */
#define ECONNREFUSED 79 /* Connection refused */
#define EHOSTDOWN 80 /* Host is down */
#define EHOSTUNREACH 81 /* No route to host */
/* ERESTART is used to determine if the system call is restartable */
#define ERESTART 82 /* restart the system call */
/* quotas and limits */
#define EPROCLIM 83 /* Too many processes */
#define EUSERS 84 /* Too many users */
#define ELOOP 85 /* Too many levels of symbolic links */
#define ENAMETOOLONG 86 /* File name too long */
/*
* AIX returns EEXIST where 4.3BSD used ENOTEMPTY;
* but, the standards insist on unique errno values for each errno.
* A unique value is reserved for users that want to code case
* statements for systems that return either EEXIST or ENOTEMPTY.
*/
#if defined(_ALL_SOURCE) && !defined(_LINUX_SOURCE_COMPAT)
#define ENOTEMPTY EEXIST /* Directory not empty */
#else /* not _ALL_SOURCE */
#define ENOTEMPTY 87
#endif /* _ALL_SOURCE */
/* disk quotas */
#define EDQUOT 88 /* Disc quota exceeded */
#define ECORRUPT 89 /* Invalid file system control data */
#define ECORRUPT 89 /* Invalid file system control data */
/* errnos 90-92 reserved for future use compatible with AIX PS/2 */
/* network file system */
#define EREMOTE 93 /* Item is not local to host */
/* errnos 94-108 reserved for future use compatible with AIX PS/2 */
#define ENOSYS 109 /* Function not implemented POSIX */
/* disk device driver */
#define EMEDIA 110 /* media surface error */
#define ESOFT 111 /* I/O completed, but needs relocation */
/* security */
#define ENOATTR 112 /* no attribute found */
#define ESAD 113 /* security authentication denied */
#define ENOTRUST 114 /* not a trusted program */
/* BSD 4.3 RENO */
#define ETOOMANYREFS 115 /* Too many references: can't splice */
#define EILSEQ 116 /* Invalid wide character */
#define ECANCELED 117 /* asynchronous i/o cancelled */
/* SVR4 STREAMS */
#define ENOSR 118 /* temp out of streams resources */
#define ETIME 119 /* I_STR ioctl timed out */
#define EBADMSG 120 /* wrong message type at stream head */
#define EPROTO 121 /* STREAMS protocol error */
#define ENODATA 122 /* no message ready at stream head */
#define ENOSTR 123 /* fd is not a stream */
#define ECLONEME ERESTART /* this is the way we clone a stream ... */
#define ENOTSUP 124 /* POSIX threads unsupported value */
#define EMULTIHOP 125 /* multihop is not allowed */
#define ENOLINK 126 /* the link has been severed */
#define EOVERFLOW 127 /* value too large to be stored in data type */