With the installation, update and detailed configuration of the WASD Web Services package provided in ([-.CONFIG]DOC_config.sdml) why have an introduction in this subsequent document? After getting the basics up and running (often the first thing we want to do) it's time to stop and consider the tool and what we're trying to accomplish with it. So this section provides an overview of the package's design philosophy, history and significant features and capabilities by topic.

The document (assumes\BOLD) a basic understanding of Web technologies and uses terms without explaining them (e.g. HTTP, HTML, URL, CGI, SSI, etc.) The reader is refered to documents specifically on these topics. (HTML/OFF)

It is strongly suggested those using printed versions of this document also access the Web version. It provides online demonstrations of some concepts. (HTML/ON) (Objectives\hd_objective)

WASD Web Services originated from a 1993 decision by Wide Area Surveillance Division (WASD) management (then High Frequency Radar Division, HFRD) to make as much information as possible, both administrative and research, available online to a burgeoning personal desktop workstation and PC environment (to use the current term an (intranet)) using the then emerging Web technologies.

It then became the objective of this author to make (all) of our systems' VMS-related resources available via HTTP and HTML, regardless of the underlying data or storage format. An examination of the WASD package will show that this objective is substantially achieved. (Reasons For Yet Another Web Package\hd_local_httpd)

Reasons for developing (remember; back in 1994!) a local HTTP server were few but compelling: (UNNUMBERED) The WASD Web implementation began mid-1994. It was prefered to support this environment on a VMS platform; at the time the most widely used and accessible environment within WASD. At that time servers (and even then there were quite a few variations) were largely Unix based, although it was being supported (to a greater or lesses extent) across a wide range of platforms. Ports to VMS, if they existed, were often in-progress or half-baked, employing (Unix)isms that don't translate elegantly to the VMS environment. The VMS version of the CERN server (3.0-6) was evaluated during mid-1994: (UNNUMBERED\-) It was (still is) not multi-threaded under VMS (i.e. cannot support concurrent clients). For example, a lengthy search may delay other clients for unacceptable periods. The performance was good with document transfers, but became poor when running a (script). It is acknowleged in the release notes that it cannot handle a client cancelling a data transfer (a not-uncommon action). This was confirmed experimentally. An early version of the OSU server was evaluated via documentation mid-1994. The author considered that the DECthreads of the time to have limitations (including frequent, show-stopping bugs) and OSU had a number of implementation idiosyncracies (e.g. DECnet based scripting). HTTP, in the then standard implementation (HTTP/1.0, RFC1945), was relatively simple to implement to the level required to support intra-Divisional requirements. Since that time (UNNUMBERED\-) (As of December 1995\BOLD) the server has worked extremely well and has a number of facilities tailored for the VMS environment. It can continue to be utilized until there are overwhelming reasons for implementing something else. (June 1997\BOLD) the server and associated software continues to evolve and provide a stable and effective VMS Web environment, even with the advent of a small number of commercial VMS Web products. (October 1999\BOLD) the package is beginning to mature as an HTTP/1.0 solution, providing not only a fast and stable server but an increasingly extensive collection of applications and tools. (July 2002\BOLD) it continues to be refined and extended. A greater emphasis on (commercial) functionality has occured over the past couple of years. (December 2004\BOLD) it now complies with the HTTP/1.1 specification (RFC2616) and provides a very respectable range of functionality and the fastest and most efficient serving environment for VMS. (A decade on (2014)\BOLD) it continues to be adopted by sites wanting fast, efficient, capable and often philosophically VMS infrastructure. WASD continues to be enhanced and bug-fixed (two decades) after its initial, tentative steps into the World-Wide information Web. (May 2016\BOLD) brings HTTP/2 (RFC 7540, RFC 7541) to WASD. A replacement for how HTTP is expressed (on the wire), it is not a ground-up rewrite of the protocol; HTTP methods, status codes and semantics are the same. The focus of the protocol is on performance; specifically, end-user perceived latency, network and server resource usage. (--------------------------------------------------------------------) (Package Overview\cr_httpd)

The most fundamental component of the WASD VMS Web Services environment is the HTTP server (HyperText Transport Protocol Daemon, or HTTPd). WASD has a single-process, multi-threaded, asynchronous I/O design.

The following bullet-points summarise the features and facilities, many of which are described in significant detail in following chapters. (General\hd_features_general) (UNNUMBERED) concurrent, multi-threaded client support HTTP/2 compliant (RFC 7540, RFC 7541) HTTP/1.1 compliant (RFC 2616, RFC 7230 and family) HTTP/1.0 compliant (RFC 1954) WebDAV 1,2 support (RFC 4918) Cross-Origin Resource Sharing (CORS) virtual services (servers) IPv4 and IPv6 support (requires underlying TCP/IP support) requests above a configurable limit can be queued ((throttling)) enhanced privacy using Secure Sockets Layer (SSL) technology, including OpenSSL Toolkit, WASD OpenSSL, and HP SSL (Secure Sockets Layer) for OpenVMS Alpha, Itanium and (from late 2003) VAX product serves ODS-2 and ODS-5 (EFS) volumes, as well as file names encoded using the PATHWORKS 4/5, Advanced Server (PATHWORKS 6) and SRI (MultiNet NFS, etc.) schemas versatile directory listing (generic and VMS-style) Server-Side Includes (SSI HTML pre-processing) configurable cache, with time-based and forced revalidation (reload) byte-range support with 206 partial responses (useful for PDF and restarting file download by modern browsers) proxy serving, with local file-system caching, plus the CONNECT method (also allowing a number of esoteric SSL tunnelling configurations), along with FTP proxy gatewaying between Web protocols (HTTP-to-SSL, SSL-to-HTTP, HTTP-to-FTP) gatewaying between IP protocols (IPv4-to-IPv6, IPv6-to-IPv4) (Scripting\hd_features_scripting) (UNNUMBERED) CGI 1.1 compliant scripting (RFC 3875) non-server and user account scripting (CGIplus) scripting (offering reduced latency, increased throughput and reduced system impact) (Persistent) scripting, Run-Time Environments (RTEs) that provide for simple persistent scripting WebSocket scripting environment; a capability introduced with HTML5, providing an asynchronous, bidirectional, full-duplex connection. (RawSocket) scripting environment; providing an protocol-agnostic asynchronous, bidirectional, full-duplex connection. (ISAPI) extensions/scripting (also offering reduced latency, increased throughput and reduced system impact) DECnet-based CGI scripting (with connection reuse) OSU (DECthreads server) scripting emulation, with connection reuse (as per OSU 3.3a), allowing many OSU scripts to be employed unmodified script processor (e.g. PERL, PHP, Python) configurable on file type (suffix) configurable, automatic, MIME content-type initiated scripting ((presentation) scripting) (Access Control\hd_features_access) (UNNUMBERED) host-level, on per-host or per-domain (Basic) and (Digest) user authentication and path/group-based authorization WASD-specific user databases SYSUAF-authentication and VMS user security profile based file access control ACME service authentication (on applicable platforms) X.509 client certificate authentication (for SSL transactions) RFC 1413 ((ident) daemon) (authentication) Example LDAP authenticators (Administration\hd_features_admin) (UNNUMBERED) multiple (instances) (server processes) executing on the one system allow continuous availability via rolling restarts and (fail-through) processing (one-button) control of multiple (instances) on both single systems and across clusters online server configuration, including reports on requests, loaded configuration, mapping rules, authorization information and graphical activity displays online, live server processing event report (WATCH) Web-standard, (common) and (combined) access log formats (allowing processing by most log-analysis tools), along with a user-definition capability allowing custom log formats logging periods, where log files automatically change on a daily, weekly or monthly basis (keeps log files ordered and at a managable size) customizable message database (capable of supporting non-English and concurrent, multiple languages) (Server Behaviour\hd_httpd_behaviour)

The technical aspects of server design and behaviour are described in ([-.CONFIG]DOC_readmore.sdml) (VMS Versions\hd_vms_versions)

The WASD server is supported on any VMS version from V7.0 upwards, on Alpha, Itanium and VAX architectures. The current version (as of late 2014), V8.4 Alpha and Itanium, as is commonly the case on VMS platforms, required nothing more than relinking. Obviously no guarantees can be made for yet-to-be-released versions but at a worst-case these should only require the same.

Up until v10.1 WASD was supported on VMS V6.0 and later. Eventually it had to be dragged kicking and screaming into the mid-1990s!

The WASD distribution and package organisation fully supports mixed-architecture clusters (Alpha, Itanium and/or VAX in the one cluster) as one integrated installation. (TCP/IP Packages\hd_tcpip_packages)

The WASD server uses the Compaq TCP/IP Services (UCX) BG $QIO interface. The following packages support this interface and may be used. (UNNUMBERED) TCP/IP Services for OpenVMS (Hewlett Packard Corporation), any version Digital TCP/IP Services for OpenVMS (aka UCX), any version MultiNet for OpenVMS (Process Software Corporation), any version

To deploy IPv6 services this package must support IPv6. (International Features\hd_international)

WASD provides a number of features that assist in the support of non-English and multi-language sites. These (international) features only apply to the server, not necessarily to any scripts! (UNNUMBERED) (Language Variants\BOLD)

A directory may contain language-specific variants of a basic document. When requesting the basic document name these variants are automatically and transparently provided as the response if one matches preferences expresses in the request's (Accept-Language:) request header field. Both text and non-text documents (e.g. images) may be provided using this mechanism.

Configuration information is provided in (HREF=[-.config]config.sdml!../config/#hd_language_variants)([-.CONFIG]DOC_config.sdml). (Character Sets\BOLD)

Generally the default character set for documents on the Web is ISO-8859-1 (Latin-1). The server allows the specification of any character set as a default for text document responses (plain and HTML). In addition, text document file types may be modified or additional ones specified that have a different character set associated with that type. Furthermore, specific character sets may be associated with mapping paths. A site can therefore relatively easily support multiple character set document resources.

In addition the server may be configured to dynamically convert one character set to another during request processing. This is supported using the VMS standard NCS character set conversion library.

For further information see [CharsetDefault], [CharsetConvert] and [AddType] in (HREF=[-.config]config.sdml!../config/#hd_config_directives_alpha)([-.CONFIG]DOC_config.sdml). (Server Messages\BOLD)

The server uses an administrator-customizable database of messages that can contain multiple language instances of some or all messages, using the Latin-1 character set (ISO8859-1). Although the base English messages can be completely changed and/or translated to provide any message text required or desired, a more convenient approach is to supplement this base set with a language-specific one.

One language is designated the prefered language. This would most commonly be the language appropriate to the geographical location and/or clientele of the server. Another language is designated the base language. This must have a complete set of messages and is a fall-back for any messages not configured for the additional language. Of course this base language would most commonly be the original English version.

More than just two languages can be supported. If the browser has (prefered languages) set the server will attempt to match a message with a language in this preference list. If not, then the server-prefered and then the base language message would be issued, in that order. In this way it would be possible to simultaneously provide for English, French, German and Swedish audiences, just for example.

For message configuration information see (HREF=[-.config]config.sdml!../config/#cr_msg_config)([-.CONFIG]DOC_config.sdml). (Server Dates\BOLD)

Dates appearing in server-generated, non-administrative content (e.g. directory listings, not META-tags, which use Web-standard time formats) will use the natural language specified by any SYS$LANGUAGE environment in use on the system or specifically created for the server. (Virtual Services\BOLD)

Virtual-server-associated mapping, authorization and character-sets allow for easy multiple language and environment sites. Further per-request tailoring may be deployed using conditional rule mapping described below. Single server can support multi-homed (host name) and multiple port services.

For virtual services information see (HREF=[-.config]config.sdml!../config/#cr_config)([-.CONFIG]DOC_config.sdml). (Conditional Rule Mapping\BOLD)

Mapping rules map requested URL paths to physical or other paths (see (HREF=[-.config]config.sdml!../config/#cr_mapping_rule)([-.CONFIG]DOC_config.sdml)). Conditional rules are only applied if the request matches criteria such as prefered language, host address (hence geographical location to a certain extent), etc. This allows requests for generic documents (e.g. home pages) to be mapped to language versions appropriate to the above criteria.

For conditional mapping information see (HREF=[-.config]config.sdml!../config/#cr_conditional)([-.CONFIG]DOC_config.sdml). (--------------------------------------------------------------------)