z/OS DGW/Apache conversion¶
This document will use shorthand names for the HTTP Servers for z/OS. These are NOT official product names.
"DGW" will refer to the old HTTP Server (V5R3), distributed with z/OS.
"Apache" or "IBM HTTP Server" will refer to the new IBM HTTP Server provided with WebSphere Application Server and z/OS Ported Tools.
This description covers only a few highlights of the differences you might encounter when you try to convert httpd.conf for DGW to Apache. It is not a complete description. When you install Apache, it will generate a sample httpd.conf for you. You should start with that file and tailor the directives as needed.
General facilities and directives¶
The DGW main program is a member of a PDS. It runs as a single address
space (process) with a fixed number of threads within it. It uses
multiple processes only if it is in scalable mode. DGW attempts to keep
the server running even after a program check.
The Apache main program is an hfs file, and it uses a different
strategy. Apache runs multiple processes, each with multiple threads.
Apache increases and decreases the number of processes and threads
dynamically based on demand. If a process is not busy enough or if it
takes a program check, Apache may cause an orderly termination of the
process, and other processes will handle any new requests.
Apache supports SSL, but the configuration is slightly different from other Apache-based servers -- See item 2 below. Apache supports SSL proxy connections to content servers, while DGW does not.
A CGI program that runs under DGW might run unchanged under Apache, except for the EBCDIC issues noted below.
Both servers support SAF authentication/authorization and RunAs (%%CLIENT%% and %%CERTIF%%). Both support LDAP authentication and authorization. Apache supports the use of a local password file and group files, but the password file is not compatible with the ones created using DGW's htadm program, and the group file format is more restrictive. See item 7 below.
Apache supports IPv6, while DGW does not.
Apache does not support FRCA or scalable mode (WLM). Apache writes relatively basic SMF records compared to DGW, starting in IBM HTTP Server 8.5.5.0.
Apache comment lines in httpd.conf must have a '#' in the first position on each line. Apache has no provision for comments on the same line as a directive. In general, blanks at the beginning of lines are ignored.
You can run DGW and Apache on the same machine at the same time, as long as they use different ports, log files, and pid files.
Non-SSL and SSL listening ports¶
DGW supports the use of only two ports, one non-SSL and one SSL. DGW's port, sslport, sslmode, and normalmode are replaced by listen and SSLEnable, as shown below. Apache supports the use of multiple ports of each flavor, and you can even have SSL ports using SAF keyrings and SSL ports using kdb files in the same server. You define the SSL options in a virtual host container. For example, here is part of an httpd.conf for a server listening on non-SSL port 80 and two SSL ports:
Listen 80 Listen 443 Listen 8443 <VirtualHost *:443> ServerName www.company1.com:443 SSLEnable Keyfile /saf company1_SAF_ring SSLProtocolDisable SSLv2 </VirtualHost> <VirtualHost *:8443> ServerName www.company2.com:8443 SSLEnable SSLProtocolDisable SSLv2 TLSv1 SSLCipherSpec 3A SSLCipherSpec 35 SSLCipherSpec 34 Keyfile /usr/lpp/internet/company2.kdb SSLServerCert KDBLabel </VirtualHost>
SSL Details¶
If you want an SSL-only server, make sure each "Listen" directive is for a virtual host that has "SSLEnable".
DGW's SSLClientAuth local becomes Apache's "SSLClientAuth required". To turn on tracing to debug SSL problems, see the
LogLevel
andSSLTrace
directives.DGW required you to code every SSLCipherSpec you wanted to enable, but modern releases of IBM HTTP Server (8.0 and later) enable a reasonable default set of ciphers.
Keyfile: DGW's “Keyfile company1_SAF_ring SAF” becomes “Keyfile /saf company1_SAF_ring”.
DGW's “Keyfile /usr/lpp/internet/company2.kdb” becomes “Keyfile /usr/lpp/internet/company2.kdb”.DGW's SSLCipherSpec maps closely to Apache, but if you do not specify any specs, DGW defaults to none, while Apache defaults to all specs, except for a few weak ones. Note the Apache SSLProtocolDisable directive allows disabling some protocols. For example, you can disable SSLv3 while allowing TLSv1.
DGW's SSLServerCert applied to an IP address only, not including the port. Apache's SSLServerCert directive specifies only a label without an IP or port. Since this Apache directive is inside a virtual host container with a port specified, it applies to a port within an IP address.
If you use the GSKSRVR started task for sysplex-wide session ID caching, code this directive to disable Apache's own session ID caching:
SSLCacheDisable
Pass and other directives defining which files are used to respond to a request¶
The simplest and most common way of routing a request's URL to an HTML file in DGW is the Pass directive and its cousins Exec, Map, Fail, Redirect, and Proxy. For example:
Pass /abc/* /usr/lpp/internet/docs/*
In this example, a URL requesting /abc/xyz.html would cause the server to return the file /usr/lpp/internet/docs/xyz.html.
The approximate equivalent to DGW's Pass directive in Apache is the Alias directive:
Alias /abc/ "/usr/lpp/internet/docs/" Note that Apache does not use the trailing asterisks. If your DGW URL template has an asterisk or other wildcard characters in any position except the last (on the end of the URL), you will have to use the related directive AliasMatch instead of Alias. The DGW catchall directive ("Pass /*") is covered under item 10 below.Item 6 below addresses the use of the Pass directive's optional third argument, the virtual host, in DGW.
The Exec directive in DGW corresponds to the
ScriptAlias
directive in Apache. Any wildcards except an asterisk on the end will require you to use ScriptAliasMatch. See the CGI section below.You might convert a Map directive to one of the many Rewrite directives, but this is too extensive to document here. Please see the Apache documentation.
DGW's Redirect directive is still called Redirect in Apache, and Apache offers some other flavors, such as RedirectMatch, RedirectPermanent, and RedirectTemp.
The Proxy directive's closest equivalent for performing reverse proxy is the "ProxyPass" directive. In addition, Apache provides a family of related proxy directives for finer control. DGW does not support SSL proxy connections to content servers, but Apache does support this capability.
You can convert DGW's Fail directive to an Apache container with a "deny" subdirective. For the Protection and Protect directives and their subdirectives, see item 7 below on resource protection.
For Service and other GWAPI directives, see item 5 below.
WebSphere plug-in¶
The WebSphere plug-in directives look like this in Apache:
LoadModule was_ap22_module modules/mod_was_ap22_http.so
WebSpherePluginConfig /usr/lpp/internet/config/plugin-cfg.xml
There are no Service, ServerInit, or ServerTerm directives in Apache. The plug-in for Apache uses the values in plugin-cfg.xml to determine if it should service a request. DGW, on the other hand, uses a combination of the values in plugin-cfg.xml and in the Service directives, and these two sources of information have to be synchronized, essentially duplicates of each other. Some DGW customers use a line in plugin-cfg.xml similar to this:
<Uri Name="/*" />
This causes the plug-in to try to service every request that DGW routes
to it. Since DGW routes only selected requests to the plug-in, as
determined by the Service directives, it would still be possible for DGW
to service some requests, such as those for static files, without
invoking the plug-in at all. With Apache, on the other hand, the plug-in
is invoked on every request, and the plug-in decides whether or not to
service it. If the plug-in is configured to service every request, then
the web server will not have the opportunity to serve any requests
locally.
It is important to note that DGW ignores the "keyring" and "stashfile"
values in plugin-cfg.xml, but Apache respects them:
<Transport Hostname="www.whatever.com" Port="443" Protocol="https">
<Property name="keyring" value="/usr/lpp/internet/server1.kdb" />
<Property Name="stashfile" Value="/usr/lpp/internet/server1.sth" />
</Transport>
<Transport Hostname="www.another.com" Port="9443" Protocol="https">
<Property name="keyring" value="RACFKeyRing" />
</Transport>
Service, ServerInit, and other GWAPI directives¶
The way that Apache uses modules is very different from the way DGW defined and invoked them, and the API is entirely different. DGW referred to these dynamically loaded modules or DLLs by the term "exits" or "plug-ins" or "GWAPIs". Apache generally uses the term "modules", but Apache also has other special terms, like "hook" and "handler".
The GWAPI directives -- such as ServerInit, Service, ServerTerm, WLMCLassify, PreExit, Authentication, Authorization, NameTrans, ObjectType, DataFilter, Log, Error, PostExit, and PICSDBLookup -- refer to programs written for DGW's API. Apache can use neither the directives nor the programs. In general, Apache offers a richer set of possibilities for modules. In some cases, such as the WebSphere plug-in or the FCGI module, Apache has readymade alternatives. In some other cases, such as HTCounter and MVSDS, you will have to pursue other solutions.
An automated conversion is generally impossible, though one might be able to approach it in a few special cases, like that of the WebSphere plug-in. DGW's use of exit programs is generally oriented around the URL of the request. The directives that cause DGW to call an exit program closely resemble the Pass directive. In Apache, a module's code usually determines its own applicability to a request.
Because Apache is open source, all of the standard modules are available with full source from http://httpd.apache.org.
Virtual hosts¶
The method of handling many configuration options is much moe robust in Apache. Most protection setups and information about how to handle a request are organized around containers, such as VirtualHost, Directory, and Location. For example:
<VirtualHost 123.45.67.89:80/ >
... various options for this virtual host...
</VirtualHost>
<Directory "/usr/lpp/internet/htdocs"/ >
... various options for this hfs directory...
</Directory>
<Location /abc/ >
... various options for this URL...
</Location>
Regarding the use of the Pass directive's optional third argument,
virtual host, in DGW:
In DGW, if you wanted to respond to request "/abc/..."
... - with a file from directory /usr/include/net if the request is from
IP address 9.8.7.6;
- with a file from directory /usr/include/sys if the request has a Host
request header "www.myserver.org"; and
- with a file from directory /usr/include otherwise;
then you might code these directives:
Pass /abc/* /usr/include/net/* 9.8.7.6
Pass /abc/* /usr/include/sys/* www.myserver.org
Pass /abc/* /usr/include/*
Apache approaches the virtual host from an entirely different direction. It might use something like this to accomplish the same file selection described above:
Alias /abc/ "/usr/include/"
<VirtualHost 9.8.7.6:80>
Alias /abc/ "/usr/include/net/"
</VirtualHost>
<VirtualHost www.myserver.org>
Alias /abc/ "/usr/include/sys/"
</VirtualHost>
The first Alias directive is not in a virtual host container, but works at the server level for all requests that are not controled by some kind of container. The other two Alias directives are applied only for requests applicable to their respective virtual hosts.
ASCII/EBCDIC considerations¶
The HTTP protocol stipulates that all request headers and response headers should be transmitted in 8-bit ASCII, and POST content and response content is generally in ASCII, if it is text. Most programs running in DGW and in Apache will examine and create the headers in EBCDIC, and the server will translate them. The programs will usually generate text content in EBCDIC. Apache will translate Request POST content to EBCDIC if it is text, that is, if its Content-type starts with "text/". To make this work in Apache, use directives similar to this:
LoadModule charset_lite_module modules/mod_charset_lite.so
<IfModule mod_charset_lite.c>
<Location / >
CharsetSourceEnc IBM-1047
CharsetDefault ISO8859-1
</Location>
</IfModule>
IBM-1047 is an EBCDIC character set, and ISO8859-1 is an ASCII set. If you have documents stored in ASCII in the z/OS hfs, tell Apache by adding directives similar to these just before the closing "</IfModule>" above:
<Location /ascii_text/ >
CharsetSourceEnc ISO8859-1
CharsetDefault ISO8859-1
</Location>
Alias /ascii_text/ "/usr/lpp/internet/ascii/"
The AddType directive in Apache is similar to that in DGW, but its
options are in a different order, and Apache has no options for encoding
(that is, ASCII/EBCDIC/binary) or for quality ratings.
In addition, DGW allowed a CGI to write a header "Content-Encoding:
ascii" or "Content-Encoding: binary" to control translation of response
content from EBCDIC to ASCII. Apache ignores this response header.
Therefore, if it is impractical to separate the ASCII files from the
EBCDIC files in the hfs, and they are identifiable by the filename
extension, you might consider using a LocationMatch directive. For
example, if your files that have the extension ".ascii" are HTML files
in ASCII, and your files that have the extension ".asctext" are plain
text files in ASCII, you might consider directives like this:
AddType text/html .ascii
AddType text/plain .asctext
<LocationMatch "\.(ascii|asctext)$" >
CharsetSourceEnc ISO8859-1
CharsetDefault ISO8859-1
</LocationMatch>
Regarding SSI files (server-side includes, usually named with the suffix ".shtml"): DGW handles these even if they are stored on disk in ASCII. Apache requires that they be stored in EBCDIC.
Before IBM HTTP Server 8.5.5, Javascript files (.js) don't get translated by default because their default content-type is "application/x-javascript" which isn't a text type. Two ways to solve this are:
Add
AddType text/javascript .js
if serving these files with content-type "text/javascript" is acceptable.
Add
<FilesMatch "\.js$"> CharsetOptions TranslateAllMimeTypes </FilesMatch>
to tell mod_charset_lite to translate all .js files regardless of content-type.
For a CGI that emits its response content in ASCII, use directives similar to these:
<Location /ascii_exec/ >
CharsetSourceEnc ISO8859-1
CharsetDefault ISO8859-1
</Location>
ScriptAlias /ascii_exec/ "/usr/lpp/internet/ascii_e/"
Note that if this CGI is a z/OS shell script that emits ASCII content, it should be stored in EBCDIC for the command interpreter, and it should still write its headers in EBCDIC, then write its response content in ASCII. Note that the input content side of the request uses the same rules, that is, if the CGI is governed by this ASCII Location container, Apache will NOT translate Request POST content to EBCDIC.
DGW has a number of special options for GWAPI programs that allow the program to control these ASCII/EBCDIC options. Apache cannot use these programs or these options.
These directives in DGW have NO counterpart in Apache:
PostDataConv
DetectUTF8 ON
ENUExecs
AddEncoding
These directives in DGW have these approximate counterparts in Apache:
DefaultFsCp - CharsetSourceEnc
DefaultNetCp - CharsetDefault
AddLanguage - AddLanguage
AddCharSet - AddCharSet
AddType - AddType - The information on LocationMatch above describes important differences.
The special case for "nph-" output is the same in DGW and Apache.
IBM HTTP Server 8.5.5.0 adds a special "DGWCompat" option to mod_charset_lite which allows the special values of the Content-Encoding header used by DGW to determine if a CGI has emitted data requiring conversion.
Logging and reporting¶
The DGW log directives -- AccessLog, AgentLog, RefererLog, ErrorLog, CgiErrorLog, ProxyAccessLog, and CacheAccessLog -- are replaced in Apache with a more powerful and flexible set of log directives. In general, Apache defines just the accesslog and the errorlog. The simplest Apache logging directives are similar to these:
ErrorLog logs/error.log LogLevel [ warn | notice | info | debug ] LogFormat "%h %l %u %t \"%r\" %/>s %b" common CustomLog logs/access.log common
You can code the Apache log directives for a virtual host container or the whole server.
Apache generally writes error messages and traces to the errorlog whenever you enable tracing with the LogLevel directive.
DGW's CgiErrorLog directive is somewhat similar to Apache's ScriptLog, but ScriptLog is not efficient, and is intended only for debugging.
Note that Apache does not append the date to the log file names, and it does not automatically start a new log at mignight. To start a new log every 24 hours and append a date, you can use the rotatelogs program. For details on how to use this, see http://httpd.apache.org/docs/2.2/programs/rotatelogs.html or http://httpd.apache.org/docs/2.2/logs.html#piped.
A better solution might be the one documented in http://httpd.apache.org/docs/2.2/logs.html#rotation:
- Rename the log files at midnight (Yes, Unix and USS will let you do this!)
- Perform a graceful restart of Apache using the command "apachectl -k graceful" or equivalent JCL.Apache does not provide its own log analysis programs, but there are several log analysis programs available from other sources for free download and for purchase.
Apache never deletes its own log files.
Welcome, Servertoken, and miscellaneous directives¶
DGW's "Welcome index.html" becomes Apache's "DirectoryIndex index.html".
DGW's various "Dir..." directives, such as DirShowDate, are covered by Apache's "IndexOptions".
DGW's DirAccess off is Apache's "Options -Indexes".
DGW's "Imbeds" is Apache's "Options +Includes".
DGW's "Imbeds on noexec" is Apache's "Options IncludesNOEXEC".
For example, to prevent directory listings and to exclude SSI #exec CGI:<Directory "/"> Options -Indexes -Includes +IncludesNOEXEC </Directory>
When using server-side includes (SSI, usually embedded in .shtml files), note that the Requires for the initial .shtml file should provide sufficient access for permissions and Requires of the "#include" and "#exec cgi" and "#exec cmd" therein, or they will fail. The server will not prompt for a username and password for these files.
DGW ServerRoot is still "ServerRoot" in Apache.
DGW's catchall Pass directive...
Pass /* /usr/wherever/*
becomes Apache's
DocumentRoot "/usr/wherever"
DGW uses the ServerToken directive to suppress text which would identify your server's type...
- in the Server response header,
- in the Via header (when you are using your server as a proxy), and
- in the errorpages (when you do not use your own errorpages).
Apache has a "Header" directive with set and unset options that sound as if they should take care of the Server header, but it does not work as you might expect.
Apache has a ProxyVia directive that addresses the Via header. As long as you let it default to "off", the server will not generate a Via header.
DGW had an ErrorPage directive. Apache has a similar directive, ErrorDocument, that addresses the problem of the text sent when there is an error, but you will have to code an ErrorDocument directive for each numeric HTTP response code. The format is "ErrorDocument <error-code> <document>". For example:ErrorDocument 401 /subscription_info.html
PidFile is still "PidFile".
Timeouts¶
PersistTimeout becomes "KeepAliveTimeout", and the time must be expressed in seconds. Apache also uses the "KeepAlive on" and the "KeepAliveTimeout" and the "MaxKeepAliveRequests" directives. InputTimeout and OutputTimeout and ScriptTimeout become "Timeout", and the time must be expressed in seconds. Apache uses the OutputTimeout effect to govern the time between response packets, not the total request response time.
If you include these directives in the Apache httpd.conf, you can use the "/status" and "/info" requests to show information about your server:
<Location /status/ >
order deny,allow
allow from all
SetHandler server-status
</Location>
<Location /info/ >
order deny,allow
allow from all
SetHandler server-status
SetHandler server-info
</Location>
CGI, FCGI, Rexx programs, MVSDS, HTCounter¶
A CGI program that runs under DGW might run unchanged under Apache,
except for the EBCDIC issues noted elsewhere. To repeat, the special
case for "nph-" output is the same in DGW and Apache.
You will need this directive to use CGI:
LoadModule cgi_module modules/mod_cgi.so
You may code a ScriptAlias directive for each URL requesting a CGI. For example,
ScriptAlias /cgi-bin/ "/usr/lpp/internet/cgi-bin/"
CGI “shebang” line¶
The CGI can be any executable, such as a compiled module, a shell script, a Rexx script, or a perl script. CGIs which require an interpreter must have a shebang line on the first line of the script to work with Apache, or the script will fail to run and a 500 error will be returned to the client. The shebang line specifies the name of the program which is used to interpret the script, as with the first line of the following example:
#!/bin/sh
echo "Content-Type: text/plain"
echo ""
echo "Output from CGI"
DGW is more tolerant.
CGI Content-Type header¶
When the CGI does not output a Content-Type header to describe the
response, Apache sends the content type of the script itself. For
example, if the script name is genrsp.sh
and it does not output a
Content-Type header, the content-type of the response will be
application/x-sh
since that is the content-type of the CGI script
itself. DGW is more tolerant of a CGI which does not write a
Content-Type header.
FastCGI¶
Apache does not use the ServerInit, Service, and ServerTerm directives for FastCGI, and there is no FastCGI configuration file, such as was used on DGW's ServerInit directive. FastCGI programs compiled for DGW should work with Apache without re-compiling. Consider this DGW Service directive for FastCGI programs:
Service /fcgi-bin/* /usr/lpp/internet/bin/libfcgi.so:FCGIDispatcher*
The Apache equivalent might be like this, with a FastCGIServer directive for each FastCGI program in directory /usr/fcgi/:
LoadModule fastcgi_module modules/mod_fastcgi.so
ScriptAlias /fcgi-bin/ "/usr/fcgi/"
<Directory "/usr/lpp/internet/bin/fcgi/">
AllowOverride None
Options +ExecCGI
SetHandler fastcgi-script
</Directory>
FastCGIServer "/usr/fcgi/size" -processes 1
FastCGIServer "/usr/fcgi/tinyfcgi" -processes 1 -port 8765
Apache's support for RunAs (%%CLIENT%% or similar) for FastCGI requests is very different from DGW's support. The FastCGI program will not run under the UID set by SAFRunAs, but under the UID used to start the FastCGI program, which is generally the server's UID. There is some processing of the request that takes place inside the HTTP server before it is sent to the FastCGI program. If this processing is in a container that uses SAFRunAs, then you must add the "-port" option to the FastCGIServer directive.
Rexx programs¶
As noted above, Apache supports the use of Rexx programs as CGIs, exactly like any other script. It does not support Rexx scripts as modules (GWAPIs).
MVSDS¶
DGW offers a GWAPI program named mvsds.so, which allows customers to access MVS files (datasets). IBM HTTP Server after 8.5.5 includes a similar module, mod_mvsds.so. Prior to 8.5.5, shell script could be used instead for rudimentary dataset access.
HTCounter¶
DGW offers a GWAPI program named HTCounter, which has no counterpart in Apache. There are some programs and scripts publicly available on the internet which provide a similar capability, so you can minimize HTML changes in documents that use the old HTCounter. It is possible to run the DGW HTCounter program as a CGI in an SSI file, as documented in the DGW manual, using HTML similar to this:
<!--#exec cgi="/cgi-bin/apicounter/sample1.ctr"-->
You can also use DGW's HTCounter program in a similar way in an SSI file in Apache, but there are some limitations which may make it impractical. For example, HTCounter as a CGI can not use the Format options; in addition, it uses some environment variables which must be set as HTCounter expects, among them PATH_INFO and URL; and the CGI version is unable to use the INIT_STRING of the ServerInit directive.
Javelin¶
DGW offered a program known as Javelin, which provided some proxy services. There is no exactly equivalent offering in Apache.
Changelog¶
20141111: Initial conversion from internal z/conversion.html