WASD Web Environment

For version 12.0 release of WASD VMS Web Services.

Published November 2021

Document generated using wasDOC version 2.0.0

Abstract

This document is a guide to supporting Web documents within the WASD Web Services environment. It is not a tutorial on writing HTML documents.

For installation and update details see WASD Web Services - Installation

For detailed configuration information see WASD Web Services - Configuration

For the more significant features and facilities available see WASD Web Services - Features

For information on CGI, CGIplus, ISAPI, OSU, etc., scripting, see WASD Web Services - Scripting

Online Search

WASD VMS Web Services – Copyright © 1996-2021 Mark G. Daniel

Apache License, Version 2.0

License

Licensed under the Apache License, Version 2.0 (the "License");

Mark.Daniel@wasd.vsm.com.au
A pox on the houses of all spamers. Make that two poxes.

All copyright and trademarks within this document belong to their rightful owners. See 10. Attribution and Acknowledgement.

This is a static (file), single document.
Alternative multi-part static and dynamic documents.
Links followed by ⤤ open in a new page.

Table of Content

1.Introduction

This document provides an overview of non-administration aspects of the WASD package. Intended more for the end-user authoring documents or wishing to use some particular behaviour, most of the aspects descibed here can be used without recourse to the site administrator. Originally intended for an audience inside the organisation where WASD originated, at a time when the Web and web technologies were in their infancy (i.e. last century :-) the document is focused on WASD specifics and admittedly is a little dated in the some of the aspects covered but not in the detail. The descriptions in this document are kept current.

The document assumes a basic understanding of web technologies and uses some terms without explaining them (e.g. HTTP, HTML, URL, CGI, etc.) It is not a tutorial on writing HTML or any other documents.

2.Document Access and Specification

Arbitrary documents may not be accessed.

The server can only access files where the path is allowed according to a specified set of rules specified within the web environment.

Documents must be read-accessible.

The server can only access files that are world readable, or that have an ACL specifically controlling access for "HTTP$SERVER", the server account.

2.1Document Content Type

Document (file) retrieval is initiated by providing the server with the file specification as a URL path. Server configuration determines the format in which the file is returned to the client. It may contain text or images immediately diplayable by the browser, or by a viewer external to the browser may be spawned. The server may automatically activate a script to provide a gateway to non-native information (see description of [AddType] configuration directive in the Technical Overview). The file type (extension) determines the content type by which the server returns (and/or interprets) the file.

The following table lists some of the current file types (as examples) and their associated MIME-style content type. HTML documents are presented layed-up according to the full HTML-capabilities of the browser. Plain-text documents are presented in a fixed-font format. Other types require an external viewer to be activated. Here are a few examples.

If other file types are required to be defined contact the Web administrator.

2.2Explicitly Specifying Content-Type

When accessing files it is possible to explicitly specify the identifying content-type to be returned to the browser in the HTTP response header. Of course this does not change the actual content of the file, just the header content-type! This is primarily provided to allow access to plain-text documents that have obscure, non-"standard" or non-configured file extensions.

It could also be used for other purposes, "forcing" the browser to accept a particular file as a particular content-type. This can be useful if the extension is not configured (as mentioned above) or in the case where the file contains data of a known content-type but with an extension conflicting with an already configured extension specifying data of a different content-type.

Enter the file path into the browser's URL specification field ("Location:", "Address:"). Then, for plain-text, append the following query string:

For another content-type substitute it appropriately. For example, to retrieve a text file in binary (why I can't imagine :^) use

It is also posssible to "force" the content-type for all files in a particular directory. See 3.3.15 Specifying Content-Type.

Ignored Content-Type

Even then some browsers and/or some operating systems and/or some version combinations insist on ignoring the response header specified content-type and instead seem to second-guess (often incorrectly) based on the file name extension. A common example is the content of DCL procedures on Windows and up-until-fairly-recent versions of Internet Explorer.

Faux Extension

Notwithstanding, if a '$' and then a second extension is appended to the URI this is often sufficient to coerce the browser into displaying the content associated with the bogus extension. In the case of DCL procedure access on a Windows platform try using "$.txt", or for other purposes whatever extension fits the requirement, as in the following examples.

2.3Document Specification

For the "http:" protocol, file and directory locations are specified using URL path syntax where slash-separated ("/") elements delineate a hierarchy leading to a data item. Anyone familiar with the syntax of the Unix file system, or the MS-DOS file system (where back-slashes are hierarchy delimiters), will feel at home with URL syntax. Specifications under VMS are not case-sensitive.

A VMS directory specification

Master file directory [000000]

2.3.1Absolute File Path

A file may be specified using an absolute, or full path. This must specify the location of the file exactly. Absolute paths always begin with a forward-slash ("/"). For example:

2.3.2Partial (or Relative) File Path

(Strictly speaking, it is a function of the client to construct a full URL from such a relative URL before sending the request to the server.)

A file may be specified relative to its current location. That is, a current document (or menu) may specify another document file relative to itself. This may be at the current level, a subdirectory, or in another part of the directory tree related to the current. Relative paths never begin with forward-slash ("/").

For example, documents at the same level as the current may be specified without any hierachy being indicated:

Documents at an inferior point in the hierarchy may be specified as in the following example:

Documents in a related part of the hierarchy may be referenced using the "./" construct. As with MS-DOS and Unix this syntax indicates the immediately superior directory.

2.4Extended File Specifications (ODS-5)

OpenVMS Alpha V7.2 introduced a new on-disk file system structure, ODS-5. This brings to VMS in general, and WASD and other Web servers in particular, a number of issues regarding the handling of characters previously not encountered during (ODS-2) file system activities.

2.4.1Characters In Request Paths

There is a standard for characters used in HTTP requests paths and query strings (URLs). This includes conventions for the handling of reserved characters, for example "?", "+", "&", "=" that have specific meanings in a request, characters that are completely forbidden, for example white-space, control characters (0x00 to 0x1f), and others that have usages by convention, for example the "~", commonly used to indicate a username mapping. The request can otherwise contain these characters provided they are URL-encoded (i.e. a percentage symbol followed by two hexadecimal digits representing the hexadecimal-encoded character value).

There is also an RMS standard for handling characters in extended file specifications, some of which are forbidden in the ODS-2 file naming conventions, and others which have a reserved meaning to either the command-line interpreter (e.g. the space) or the file system structure (e.g. the ":", "[", "]" and "."). Generally the allowed but reserved characters can be used in ODS-5 file names if escaped using the "^" character. For example, the ODS-2 file name "THIS_AND_THAT.TXT" could be named "This^_^&^_That.txt" on an ODS-5 volume. More complex rules control the use of character combinations with significance to RMS, for instance multiple periods. The following file name is allowed on an ODS-5 volume, "A-GNU-zipped-TAR-archive^.tar.gz", where the non-significant period has been escaped making it acceptable to RMS.

The WASD server will accept request paths for file specifications in both formats, URL-encoded and RMS-escaped. Of course characters absolutely forbidden in request paths must still be URL-encoded, the most obvious example is the space. RMS will accept the file name "This^ and^ that.txt" (i.e. containing escaped spaces) but the request path would need to be specified as "This%20and%20that.txt", or possibly "This^%20and^%20that.txt" although the RMS escape character is basically redundant.

Unlike for ODS-2 volumes, ODS-5 volumes do not have "invalid" characters, so unlike with ODS-2 no processing is performed by the server to ensure RMS compliance.

2.4.2Characters In Server-Generated Paths

When the server generates a path to be returned to the browser, either in a viewable page such as a directory listing or error message, or as a part of the HTTP transaction such as a redirection, the path will contain the URL-encoded equivalent of the canonical form of an extended file specification escaped character. For example, the file name "This^_and^_that.txt" will be represented by "This%20and%20that.txt".

When presenting a file name in a viewable page the general rule is to also provide this URL-equivalent of the unescaped file name, with a small number of exceptions. The first is a directory listing where VMS format has been requested by including a version component in the request file specification. The second is in similar fashion, but with the tree facility, displaying a directory tree. The third is in the navigation page of the UPDate menu. In all of the instances the canonical form of the extended file specification is presented (although any actual reference to the file is URL-encoded as described above).

2.4.3Document Cache

The Web server is most commonly set up to cache static documents (files). A cache is higher speed storage, in-memory, in the server itself. Cached documents are checked periodically for changes when being requested. Changes to a file are determined by the comparing the modification date/time and file length. A common check period is one minute, though it can set longer or even disabled. If a document has changed the old one is discarded from cache (called invalidation) and the new one loaded into cache while being transfered to the client.

After making changes to a document it is possible the server will continue to serve the old one for a short period. This can be overridden by using the browser's Reload facility. This directs the server to go and check the on-disk file regardless, invalidating it if necessary.

3.Directory Listing

A directory listing is sometimes refered to as a document Index, and is generally titled "Index of …".

Unless disabled by the server's configuration, a directory listing is recognised by the server whenever a wildcard is present in a specification and there is no query string directing another activity (e.g. a document search). Compliant with other web implementations, a directory listing is also generated if a URL specifies a directory only and that directory contains no home page.

All specifications must be made using URL-style paths. See 2.3 Document Specification.

The directory listing is designed to look very much like the basic layout of other servers, except that all directories are grouped at the top. In the opinion of the author, this looks and functions better than when interspersed with the files, as is otherwise common. The default listing provides:

• iconic indication of the data type
• file name
• last revison date/time
• size
• description

The description can be either just that, a description of the role of that type of file under VMS, or if presented within quotes, an HTML document's own internal description taken from the "<title></title>" element.

Note that directory listings only processes the physical file system. This may or may not correspond to the web environment's virtual mappings.

The following link illustrates the directory listing format: /wasd_root/wasdoc/*.*

VMS-ish Format

The default listing has a generic WWW look about it, however it can be made to look a little more like the format of the VMS "DIRECTORY" command. In this mode the directories are presented as VMS subdirectories, the version number is shown, if a version wildcard was included in the specification then all matching versions are shown, the size is presented in used and allocated blocks, and automatic script activation is disabled. The VMS-style format is enabled by providing an explicit or wildcard version number with the specification, as in the following example: /wasd_root/wasdoc/*.*;

Listing Icons

By default (and generally) WASD installations are configured to return a binary file for unknown content-types (usually triggering a browser "save-as" dialog). For such files, and in fact all files in general, a directory listing icon is usually a link to a plain-text version of the file (regardless of the actual content). This becomes convenient way to access the content for files with "interesting" file name extensions and the actual markup of HTML source files, etc. Of course some files containing non-textual data will be variously displayed as gibberish depending on the browser.

3.1Controlling Access To A Directory

The following files (empty, or not), when within a specific directory regulate access to that directory, and the listing of any parent directory or subdirectories.

• .WWW_HIDDEN – Renders the directory completely invisible to the directory listing mechanism. Files within the directory may still be accessed if specified explicitly but the directory content itself cannot be listed by any means.
• .WWW_NOWILD – Renders the directory incapable of being listed using "*.*" characters at the end of the path, even if allowed by the server. This is a little different to .WWW_HIDDEN, which hides the directory completely. The no-wild still allows a directory without a home page to list as a directory, it does however prevent the forced listing using the "*.*" syntax.
• .WWW_NOP – Any parent directory is not listed.
• .WWW_NOS – Any subdirectories are not listed.
• .WWW_NOPS – Any parent directory or subdirectories are not listed.

3.2"Hidden" Files

Any file name beginning with a period is hidden from the directory listing mechanism (i.e. in VMS parlance it has only a type/suffix/extension). If specifically accessed they will be retrieved however. Hence the following files would not appear in a directory listing:

3.3Server Directives

The WASD server behaviour can be modified using server directives. For directory listings this involves the inclusion of a query string beginning with "?httpd=index". The server detects this URI query string and processes it internally, changing the default action of directory listings.

Multiple directives can be combined by concatenating them with intervening ampersands, as per normal URI syntax.

Server directives specified in the URI propagate when moving between directories unless the query string also contains

3.3.1.WWW_WASD

The control file .WWW_WASD can be used to contain server directives. Directives are generally included one per line with "#" prefixed comment lines allowed.

When a .WWW_WASD file is present in a directory it takes precedence over any SSI or URI (query string) directives also being applied, unless it includes an override=yes directive. When this is present any SSI and/or URI directive string is also applied to the request.

3.3.2Layout

Allows specification of the directory listing layout from the URL, overriding the server default. The layout directive is a short, case-insensitive string that specifies the included fields, relative placement and optionally the width of the fields in a directory listing. Each field is controlled by a single letter (one with colon-separated parameter) and optional leading decimal number specifying the width. When a width is not specified an appropriate default applies. An underscore is used to indicate a single space and is used to separate the fields (two consecutive works well).

• C – creation date
• D – description (often best specified last)
  • D:L – for files, make a link out of the description text
• I – icon (takes no field-width attribute)
  • L – link (highlighted anchor using the name of the file)
  • L:F – file-system name (for ODS-5 displays spaces, etc.)
  • L:N – name-only, do not display the extension
  • L:U – force name to upper-case
• N – name (no link, why bother? who knows!)
• O – owner (can be disabled)
• R – revision date
• S – size
  • S:B – in bytes (comma-formatted)
  • S:D – decimal kilos (see below)
  • S:F – kilo and mega are displayed to one decimal place
  • S:K – in kilo-bytes (and fractions thereof)
  • S:M – in mega-bytes (and fractions thereof)
• U – upper-case file and directory names (must be the first character)

The default layout is:

The following provide other examples:

By default the size of files is calculated as 1024 byte kilos.

When using the "S:D", the size is displayed as per "F" with 1000 byte kilos. If it is prefered to have the default display in 1000 byte kilos then set the directory listing layout using:

If unsure of the kilo value being used check the "<meta>" information in the directory listing.

The following links illustrate this functionality by listing this document's directory in various formats.

3.3.3Directory Style

WASD has a default directory listing style where the page title is a series of links to the various directories in the path, then column labels are separated from directory links by a horizontal line, the after the listed directories a blank line, and the listed files, followed by another horizontal line, all in monospace font. There have been a small number of variations on this over the years. The current default (post-v10.4) style is implemented using an HTML table and differs from the historical preformatted text approach.

For each style there is a second variant where the horizontal lines partitioning the listing are not present.

The various directory listing styles are illustrated below.

Sortable Listing

There is also a useful alternate style that allows manual, automatic and programmatic sorting of the listing content. The sort is performed by JavaScript and does not require a re-request of the server. A sortable listing is indicated by ▼▲  in the header icon column (if present). When a column has been sorted ascending/descending symbols are present adjacent to the column label. The taper of the symbol shape ▼  indicating larger/later to smaller/earlier top to bottom (descending), and ▲  smaller/earlier to larger/later top to bottom (ascending). Manual sorting is accomplished by clicking on the column label. This reverses the current sort order.

The examples above include the basic sortable listing.

The sortable listing by default generates the standard WASD ascending by name listing. It is also possible to automatically sort a listing on a particular column when it is generated. This is done with an additional parameter specifying the layout column character as used in . The default is to list top-to-bottom ascending order. An optional "-" lists top-to-bottom in descending order (the redundant "+") is also accepted.

Examples of automatic sort on size, descending size, revision data and descending revision date are below.

Programmatic listing from JavaScript embedded in a README.HTML or enclosing SSI document can be performed by calling the wasdDirSort() function with a string parameter containing the column character and optional order character. The following example sorts descending by size.

To make all directory listings for a server or site sortable, place the following mapping rule appropriately close to the top of relevant mapping rules.

3.3.4Selective Listing

The WASD directory listing permits the use of VMS wildcards in the file specification. This directly allows selected listings of files. For example, all files, all the SDML files in this document's directory, all the HTML files, and finally all the files containing 01 in the name.

In addition, it is possible to selectively include and/or exclude selected files using the server directive httpd=index&these=. The parameter is one or more, comma-separated file name specifications which may contain wildcards. The directory listing usually would have file name and type wildcard (though can be a selector in itself) and then files are selected using using the above directive. If a selector is preceded by an exclamation point ("!") this excludes matching files. Matching is first to last and the first match hit is applied.

The following examples provide the equivalent listings to the examples above.

This example lists all of the files above in the one listing.

This example lists all files except those with an .HTML type.

3.3.5Listing Title

The listing (page) title may be explicitly supplied using the httpd=index&title=.. directive.

The keywords parallel those provided for path mapping SET dir=title=...

• "0" (digit zero) suppress any title
• "1..99" where 1 is the top-level directory (device), 2 is the second-level directory, 3 … 99 the current directory
• "DEFAULT" the default for the directory style
• "OWNER" the VMS account owning the directory
• "REMOTE" the remote user name (for X509 authentication the certificate common-name)
• "THIS=<string>" a literal string

3.3.6Listing Font

Historically and by default, directory listings are provided in a monospace font. The parent document or browser (proprtional) font may be used instead with the httpd=index&font=inherit directive. This can only be used with the sortable and tabular (post-v10.4) listings. Historical listing styles are only suitable for monospace fonts.

There is the per-path SET dir=font=inherit mapping rule equivalent.

3.3.7File Versions

As described above, a directory listing can be requested in a VMSish style by appending a version delimter to the URL file specification, with multiple versions of the file, where they exist, displayed with version number.

Multiple versions may also be selectively listed using the httpd=index&versions= directive. The parameter can be an integer representing the maximum number of versions to be listed, or an asterisk wildcard indicating all versions should be listed.

There is the per-path SET dir=versions= mapping rule equivalent.

3.3.8Readme Files

When a directory listing is generated any "README.", "README.TXT" or "README.HTML" file (or others as configured for the particular server) in the directory will have the contents displayed immediately below the title of the page. This allows additional information on the directory's contents, function, etc., to be presented. This can be suppressed by appending the following query-string to the directory specification, as in the accompanying example:

Read-me files can be SSI documents if configured by the server administrator. General SSI guidelines apply to these, see 4. Server Side Includes (SSI)

3.3.9Listing Delimiters

A directory listing is normally delimited by a header, comprising an "Index of", column headings and horizontal line, and a footer, comprising a horizintal line. This default behaviour may be modified using the "delimit=" directive.

• header – a header comprising a horizontal rule and column heading is generated
• footer – a footer comprising a horizontal rule is generated
• none – no header or footer is generated
• both – both header and footer is generated (default)

3.3.10Suppressing Directories

Parent and subdirectories may be suppressed in a listing using the "nop", "nops" and "nos" directives. These parallel the purpose of the directory listing control files described in 3.1 Controlling Access To A Directory, and if set to true suppress the listing of the corresponding directories.

• nop – any parent directory is not listed
• nops – any subdirectories are not listed
• nos – any parent directory or subdirectories are not listed

3.3.11Listing Refresh and Expiry

Directory listings and trees may be pre-expired. That is, the listing is reloaded each time the page is referenced. This is convenient in some environments where directory contents change frequently, but adds considerable over-head and so is often disabled by default. Individual directory listings may have either default behaviour over-ridden using syntax similar to the following examples:

3.3.12Scripting From Directory Listings

When a directory listing is requested a script name can be specified to be used as a prefix to all of the file links in the listing. When the client selects a file link the script specified is implicitly activated.

The following link illustrates this facility by specifying the liner script. When a link is selected the liner script presents the file with preprended line numbers:

3.3.13Auto-Scripting

The server's auto-scripting facility (see description of [AddType] configuration directive in the Technical Overview) can be suppressed by appending the following query-string to the directory specification, as in the accompanying example:

This implies that any file accessed from the listing will be transfered without any data conversion possible due to script activation. The browser must then process the document in some fashion (often by activating a save as dialog).

3.3.14Target Window

Files generally open in the current browser window. It is possible to open a file in another window using the target= directive.

Browser (HTML) Supported Targets

3.3.15Specifying Content-Type

When accessing files it is possible to explicitly specify the identifying content-type to be returned to the browser in the HTTP response header. Of course this does not change the actual content of the file, just the header content-type! This is primarily provided to allow access to plain-text documents that have obscure, non"-standard" or non-configured file extensions. See 2.2 Explicitly Specifying Content-Type.

It could also be used for other purposes, "forcing" the browser to accept a particular file as a particular content-type. This can be useful if the extension is not configured (as mentioned above) or in the case where the file contains data of a known content-type but with an extension conflicting with an already configured extension specifying data of a different content-type.

It is posssible to "force" the content-type for all files in a particular directory. Enter the path to the directory and then add

(or what-ever type is desired). Links to files in the listing will contain the appropriate "?httpd=content&type=..." appended as a query string.

This is an example:

3.3.16Icon Plain Text Link

Directory listing icons are generally links to the content of the file displayed as plain-text (‘Listing Icons’ in 3. Directory Listing). This can be disabled using the ilink directive.

3.3.17Query String

The query=<string> directive can be used to propagate an explicit query string to subsequent directory requests. This is intended to be applied from a .WWW_WASD control file because actual URI query strings are propagated by default.

3.3.18Allowing Override

When a .WWW_WASD file is present in a directory it takes precedence over any SSI or URI (query string) directives also available, unless it includes an override=yes directive. When this is present any SSI and/or URI directive string is also applied to the request.

3.4Directory Tree

The "Tree" internal script allows a directory tree to be generated. This script is supplied with a directory name from which it displays all subdirectories in a hierarchical layout, showing subordinancies. Selecting any one of the subdirectories displayed generates a directory listing (see 3. Directory Listing).

Appending a file specification (with or without wildcards) to the directory name results in the any directory listing displaying only files matching the specification. To display all files a "*.*" should always be appended.

Note that this script only processes the physical file system. This may or may not correspond to the web environment's virtual mappings.

To enable the VMS-style directory listing format, or to use any of the directory server directives, append one, or a combination of, the following query strings to the directory specification:

The following links provide online demonstrations:

Note that this activity is I/O intensive, and can take a considerable period if the tree is extensive.

4.Server Side Includes (SSI)

The HTML pre-processor is used to provide dynamic information inside of an otherwise static, HTML (HyperText Markup Language) document. The HTTPd server provides this as internal functionality, scanning the input document for special pre-processor directives, which are replaced by dynamic information based upon the particular directive.

As of version 5.1 WASD SSI has been enhanced to provide flow-control statements, allowing blocks of the document to be conditionally processed, see 4.7 Flow Control. These extensions allow quite versatile documents to be created without resorting to script processing.

Two documents are provided as examples of SSI processing.

• A simple SSI document.

  WASD_ROOT:[WASDWASDOC.ENV]SSI.SHTML

  (access as SSI)

• An SSI document using variable assignment and flow-control.

  WASD_ROOT:[WASDWASDOC.ENV]XSSI.SHTML

  (access as SSI)

By default the HTML pre-processor is invoked when the document file's extension is ".SHTML". As there is a significant overhead with pre-processed HTML compared to normal HTML, it should only be used when it serves a useful documentary purpose, and not just for the novelty.

Essential compatibility with OSU Server Side Includes is provided. This may ease any transition between the two. See 4.11 OSU Compatibility for further information.

4.1Virtual Documents

One effective use for pre-processed HTML is the creation of single virtual documents from two or more physical documents. That is, the pre-processed document is used to include multiple physical documents, that may even be independently administered, to return a composite document to the client. This is a relatively low-overhead activity as SSI goes, but because it is a dynamic document, without some extra considerations (see 4.2 Last-Modified Information).

Example 1

This provides an example of the efficient use of SSI processing to create virtual documents. Each page will comprise a header (containing the body tag and page header, etc), the document proper and a footer (containg the end-of-page information, modification date, and end-body tag, etc).

A more efficient variant places the document proper in its own, plain HTML file which is then #included (it is much, much, much more efficient for the server to throw a file at the network, than parse every character in one ;^)

Example 2

This example provides a seemingly more convoluted, but very much more powerful configuration, that uses recursion to greatly simplify maintenance of common-layout documents for the end-user.

File 1; the document accessed via the browser URL, doesn't matter what its name is, this configuration is completely naming independent.

File 2; the TEMPLATE.SHTML refered to by the first include above.

File 3; the DOCUMENT.HTML refered to by the second include in file 1.

This is just a bunch of HTML!

This is an explanation of how it works …

1. the browser accesses file 1 via a URL
2. processing begins with file 1
3. file 1 checks if it has a parent (is the first file processed),
   it doesn't and so …
4. file 1 set a variable named TITLE
5. file 1 #includes file 2, a site-common template
6. file 2 substitutes the TITLE variable contents as the document title
7. file 2 #includes a site-common header
8. file 2 #includes an optional document-local header
9. here's the interesting bit …
   file 2 now re-#includes the original document, file 1 (!!)
10. file 1 checks if it has a parent (is the first file processed),
    it does and so …
11. file 1 #includes file 3 (the actual contents of the document)
12. file 3 is a plain HTML document, just added to the output
13. file 1 is now exhausted and processing returns to file 2
14. file 2 #includes an optional document-local footer
15. file 2 #includes a site-common footer
16. file 2 is exhausted and processing returns to file 1
17. file 1 is exhausted and processing stops

The following link provides an example of a such a virtual document.

4.2Last-Modified Information

SSI documents generally contain dynamic elements, that is those that may change with each access to the document (e.g. current date/time). This makes evaluation of any document modification date difficult and so by default no "Last-Modified: timestamp" information is supplied against an SSI document. The potential efficiencies of having document timestamps, so that requests can be made for a document to be returned only if modified after a certain date/time ("If-Modified-Since: timestamp"), are significant against the CPU overheads of processing SSI documents.

WASD allows the document author to determine whether or not a last-modified header field should be generated for a particular document and which contributing file(s) should be used to determine it. This is done using the #modified directive. If a virtual document is made up of multiple source documents (files) each can be assessed using multiple virtual= or file= tags, the most recently modified will be used to determine if the virtual document has been modified, and also to generate the last-modified timestamp.

The if-modified-since tag compares the determined revision date/time of the document file(s) with any "If-Modified-Since:" timestamp supplied with the request. If the virtual document's revision date/time is the same or older than the request's then a not-modified (304 status) header is generated and sent to the client and document processing ceases. If more recent an appropriate "Last-Modified:" header field is added to the document and it continues to be processed.

If a request has a "Pragma: no-cache" field (as with Navigator's reload function) the document is always generated (this is consistent with general WASD behaviour). The following example illustrates the essential features.

This construct should be placed at the very beginning of the SSI document, and certainly before there is any chance of output being sent to the browser. Once output to the client has occured there can be no change to the response header information (not unreasonably).

4.3Pre-Expiring Documents

SSI preprocessed documents are dynamic in the sense that the information presented can be different every time the document is generated (e.g. if time directives are included). If it is important that each time the document is accessed it is regenerated then an HTML META tag can be included in the HTML header to cause the document to expire. This will result in the document being reloaded with each access. This can be accomplished two ways.

• Use the #modified directive to include an "Expires: timestamp" response header field. Place the following construct at the beginning of the SSI document.
  <!--#modified expires="Fri, 13 Jan 1978 14:00:00 GMT" -->
  An alternative, if the objective to to pre-expire the document, is to specify an expiry of zero. The is specially handled by the SSI engine. It adds an expiry response header field, plus cache-control header fields to suppress document caching (on compliant browsers).
  <!--#modified expires="0" -->

4.4Directive Syntax

The syntax follows closely that used by the other implementations, but some directives are tailored to the WASD and VMS environment. The directive is enclosed within an HTML comment and takes the form:

A tag provides parameter information to the directive. A directive may have zero, one or more parameters. Values supplied with any tag may be literal or via variable substitution (see 4.6 Variables). A value must be encolosed by quotation marks if it contains white-space.

A directive can be split over multiple lines provided the new line begins naturally on white-space within the directive. For example, this is correctly split

Directive and tag keywords are case insensitive. The tag value may or may not be case sensitive, depending upon the command/tag. Generally the effect of a command is to produce additional text to be inserted in the document, although it is possible to control the flow of processing in a document with decision structures.

4.5Directives

SSI Directives

4.5.1#ACCESSES

The #accesses directive allows the number of times the document has been accessed to be included. It does this by creating a counter file in the same location and using the same name with a dollar symbol appended to the type (extension). The count may be reset by deleting the file. This is an expensive function (in terms of file system activity) and so should be used appropriately. It can be disabled by server configuration. Three tags provide additional functionality:

• ORDINAL
  <!--#accesses ordinal -->

  Provides the count as 1st, 2nd, 3rd, 4th, 5th … 10th, 11th, 12th … 120th, 121st, 122nd, etc.

• SINCE
  <!--#accesses since="text" -->

  This tag includes the specified text immediately after the access count is displayed, then adds the creation date of the counter file.

• TIMEFMT
  <!--#accesses since="text" timefmt="[time-format]" -->
  Allows the time format of the since tag to be supplied, where time-format is specified according to 4.10 Time Format.

4.5.2#CONFIG

The #config directive allows time and file size formats to be specified for all subsequent directives providing these values. Optional specifications for individual directives may still be made, and override, do not supercede, any specification made using a config directive. A config directive may be made once, or any number of times in a document, and applies until another is made, or until the end of the document.

• ERRMSG
  <!--#config errmsg="string" -->

  This directive allows the error message generated if a problem problem processing the SSI document occurs (e.g. miss-spelled directive) to be specified in the document.

• TIMEFMT
  <!--#config timefmt="time-format" -->
  Where time-format is specified according to4.10 Time Format.
• SIZEFMT
  <!--#config sizefmt="size-format" -->
  Where size-format is specified using the following keywords:
  • "abbrev" (as bytes, kbytes, Mbytes)
  • "blocks" (VMS blocks, used)
  • "bytes" (e.g. "1,256,731 bytes")
• TRACE
  <!--#config trace="1|0" -->

  Switches document processing trace on or off, intended for use when debugging more complex or flow-controlled SSI documents.

  Output from a trace is colour-coded.

  • Blue – As a line is read from the document is is displayed in blue. The text is preceded by a square-bracketed source file line number and flow-control level.
  • Red – As an SSI statement is actually processed it is displayed in red. Due to document parsing this may occur at some point after the line is read from file.
  • Magenta – As variables are set or read the variable name and value is displayed. A variable set has the name separated from the value by an equate symbol ("="), when being read the character is a full-colon (":").
  • Black – Document (HTML and text) output is displayed as black plain text.

  The following link provides an example of a document trace.

4.5.3#DIR

The #dir directive generates an Index of … directory listing inside an HTML document. Apart from not generating a title (it is up to the pre-processed document to title, or otherwise caption, the listing) it provides all the functionality of the WASD HTTPd directory listing (see 3. Directory Listing), including query string format control via the "par=" parameter (note that from the "?httpd=index" introducer used with directory listings is not necessary from SSI). It is an WASD HTTPd extension to pre-processed HTML.

• FILE

  Listing specified using a VMS file path.

  <!--#dir file="file-name" [par="server-directive(s)"] -->
• VIRTUAL

  Listing specified using URL-style syntax.

  <!--#dir virtual="path" [par="server-directive(s)"] -->

For example:

4.5.4#DCL

The #dcl directive executes a DCL command and incorporates the output into the processed document. It is an WASD HTTPd extension to the more common exec directive, which is also included.

By default, output from the DCL command has all HTML-forbidden characters (e.g. "<", "&") escaped before inclusion in the processed document. Thus command output cannot interfere with document markup, but nor can the DCL command provide HTML markup. This behaviour may be changed by appending the following tag to the directive:

Some #dcl directives are for privileged documents only, documents defined as those being owned by the SYSTEM account, and not being world-writeable. The reason for this should be obvious. There are implicit security concerns about any document being able to execute any DCL command(s), even if it is being executed in a completely unprivileged process. Hence only innocuous commands are allowed in standard documents.

• SAY

  Execute the DCL "WRITE SYS$OUTPUT" command, using the specified parameter.

  <!--#dcl say="hello." -->
• SHOW

  Execute the DCL "SHOW" command, using the specified parameter.

  <!--#dcl show="device/full tape1:" -->
• DIR

  Execute the DCL "DIRECTORY" command, using the supplied file specification. Qualifiers may be included in the optional "par" tag to control the format of the listing.

  <!--#dcl dir="web:[000000]" --> <!--#dcl dir="web:[000000]" par="/nohead/notrail" --> <!--#dcl dir="web:[000000]" par="/size/date" -->
• EXEC (privileged)

  Execute the specified DCL command.

  <!--#dcl exec="show device/full tape1:" -->
• FILE (privileged)

  Execute the DCL command procedure specified as a VMS file path, with any specified parameters applied to the procedure.

  <!--#dcl file="WASD_ROOT:[SHTML]TEST.COM" par="PARAM1 PARAM2" -->
• VIRTUAL (privileged)

  Execute the DCL command procedure specified in URL-style syntax, with any specified parameters applied to the procedure.

  <!--#dcl virtual="../shtml/test.com" par="PARAM1 PARAM2" -->
• CGI

  Execute the specified CGI script. The CGI response header is suppressed and only the response body is included in the document.

  <!--#dcl cgi="/cgi-bin/calendar?2004" -->

4.5.5#ECHO

The #echo directive incorporates the specified information into the processed document. Multiple tags may be used within the one directive.

• VALUE=
  VAR=

  Any SSI variable (e.g. CREATED), CGI variable (e.g. HTTP_USER_AGENT), or document assigned variable (e.g. EXAMPLE1), see 4.6 Variables.

  <!--#echo value={created} var={example1} -->
• CREATED

  The date/time of the current document's creation.

  <!--#echo created[="time-format"] -->
• DATE_LOCAL

  Include the current date/time.

  <!--#echo date_local[="time-format"] -->
• DATE_GMT

  Include the current Greenwich Mean Time (UTC) date/time.

  <!--#echo date_gmt[="time-format"] -->
• DOCUMENT_NAME

  The current document's URL-style path.

  <!--#echo document_name -->
• FILE_NAME

  The current document's VMS file path.

  <!--#echo file_name -->
• HEADER

  Append the specified string to the response header (with correct carriage control). Should be used as early as possible in the SSI document.

  <!--#echo header="Pragma: no-cache" --> <!--#echo header="X-Extension-Header: just an example!" -->
• LAST_MODIFIED

  The date/time of the current document's last modification.

  <!--#echo last_modified[="time-format"] -->

4.5.6#ELIF

The #elif directive (else-if) allows blocks of HTML markup and SSI directives to be conditionally processed, see 4.7 Flow Control and 4.5.14 #IF. This directive effectively allows a case statement to be constructed.

4.5.7#ELSE

The #else directive allows blocks of HTML markup and SSI directives to be conditionally processed, see 4.7 Flow Control. It is the default block after an "#if", "#orif" or "#elif".

4.5.8#ENDIF

The #endif directive marks the end of a block of document text being conditionally processed, see 4.7 Flow Control.

4.5.9#EXEC

The #exec directive executes a DCL command and incorporates the output into the processed document. It is the VMS equivalent of the exec shell directive of some Unix implementations. It is implemented in the same way as the #DCL directive, and so the general detail of that directive applies. It supports both the cmd tag and the cgi tag, allowing execution of CGI scripts (the response header is absorbed).

The #exec directive is for privileged documents only, documents defined as those being owned by the SYSTEM account, and not being world-writeable. The reason for this should be obvious. There are implicit security concerns about any document being able to execute any DCL command(s), even if it is being executed in a completely unprivileged process.

4.5.10#EXIT

The #exit directive causes the server to stop processing the current SSI file. If the current file was an #included SSI file, processing continues back with the parent file. Note that the #stop directive also is available, it stops processing of the entire virtual document.

4.5.11#FCREATED

The #fcreated directive incorporates the creation date/time of a specified file/document into the processed document.

• FILE

  Document specified using a VMS file path.

  <!--#fcreated file="file-name" [fmt="time-format"] -->
• VIRTUAL

  Document specified using URL-style syntax.

  <!--#fcreated virtual="path" [fmt="time-format"] -->

4.5.12#FLASTMOD

The #flastmod directive incorporates the last modification date/time of a specified file/document into the processed document.

• FILE

  Document specified using a VMS file path.

  <!--#flastmod file="file-name" [fmt="time-format"] -->
• VIRTUAL

  Document specified using URL-style syntax.

  <!--#flastmod virtual="path" [fmt="time-format"] -->

4.5.13#FSIZE

The #fsize directive incorporates the size, in bytes, kbytes or Mbytes, of a specified file/document into the processed document.

• FILE

  Document specified using a VMS file path.

  <!--#fsize file="file-name" [fmt="size-format"] -->
• VIRTUAL

  Document specified using URL-style syntax.

  <!--#fsize virtual="path" [fmt="size-format"] -->

4.5.14#IF

The #if directive allows blocks of HTML markup and SSI directives to be conditionally processed, see 4.7 Flow Control.

• VAR=

  Variable the decision will be based upon.

  <!--#if var="[variable}|literal]" -->
• EQS=

  Is the string the same as in the variable?

• EQ=

  If the variable is a number is it the same as this?

• GT=

  If the variable is a number is it greater than this?

• LT=

  If the variable is a number is it less than this?

• SRCH=

  Search the variable for this string. May contain the * (asterisk) wildcard, matching one or more characters, and the % (percentage), matching any single character.

As in the following examples:

4.5.15#INCLUDE

The #include directive incorporates the contents of a specified file/document into the processed document.

• FILE

  Include the contents of the document specified using a VMS file specification.

  <!--#include file="file-name" -->
• VIRTUAL

  Include the contents of the document specified using URL-style syntax.

  <!--#include virtual="path" -->

The contents of the specified file are included differently depending on the MIME content-type of the file. Files of text/html content-type (HTML documents) are included directly, and any HTML tags within them contribute to the markup of the document. Files of text/plain content-type (plain-text documents) are encapsulated in "<pre></pre>" tags and have all HTML-forbidden characters (e.g. "<", "&") escaped before inclusion in the processed document. An HTML file can be forced to be included as plain-text by using the following syntax:

To "force" a file to be considered as text regardless of the actual content (as determined by the server from the file type), use on of the following depending on whether it should be rendered as plain or HTML text.

Other SSI files may be included and their content dynamically included in the resulting document. To prevent a recursive inclusion of documents the nesting level of SSI documents is limited to five.

4.5.16#MODIFIED

The #modified directive allows a document author to control the "Last-Modified:"/"If-Modified-Since:"/"304 Not modified" behaviour of an SSI document. See 4.1 Virtual Documents.

• no tag

  Get the last-modified date/time of the current document.

  <!--#modified -->
• FILE

  Get the last-modified date/time of the document specified using VMS file specification.

  <!--#modified file="file-name" -->
• VIRTUAL

  Get the last-modified date/time of the document specified using URL-style syntax.

  <!--#modified virtual="path" -->
• IF-MODIFIED-SINCE

  Compares any "If-Modified-Since:" request header timestamp to the revision date time obtained using file or virtual (most recent if multiple). If the document timestamp is more recent (has been modified) an appropriate "Last-Modified" response header field is generated and added to the response, and document processing continues. If it has not been modified a "304" response header is return (document not modified) and document processing stops.

  <!--#modified if-modified-since -->
• LAST-MODIFIED

  Adds a "Last-Modified:" response header field using a timestamp retrieved using file or virtual (note: unnecessary if the if-modified-since tag is used).

  <!--#modified last-modified -->
• EXPIRES

  Adds a "Expires:" response header field. The string literal should be a legitimate RFC-1123 date string. This can be used for pre-expiring documents (so they are always reloaded), set it to a date in the not-too-distant past (as in the example below). Of course it could also be used for setting the legitimate future expiry of documents.

  <!--#modified expires="Fri, 13 Jan 1978 14:00:00 GMT" -->

4.5.17#ORIF

The #orif directive (or-if) allows blocks of HTML markup and SSI directives to be conditionally processed, see 4.7 Flow Control and 4.5.14 #IF. In the absence of any real expression parser this directive allows a block to be processed if one of multiple conditions are met.

4.5.18#PRINTENV

The #printenv directive prints a plain-text list of all SSI-specific, then CGI, then document-assigned variables (see 4.6 Variables). This directive is intended for use when debugging flow-controlled SSI documents.

The following link uses the example SSI document WASD_ROOT:[WASDOC.ENV]XSSI.SHTML to demonstrate this.

4.5.19#SET

The #set directive allows a user variable to be assigned or modified, see 4.6 Variables.

Variables are always stored as strings and have a finite but generally usable length. Some comparison tags provided in the flow-control directives treat the contents of variables as numbers. A numeric conversion is done at evaluation time.

4.5.20#SSI

The #ssi directive allows multiple SSI directives to be used without the requirement to enclose them in the normal HTML comment tags (i.e. <!-- -->). This helps reduce the clutter in an SSI document that uses the extended capabilities of variable assignment and flow control. Document HTML cannot be included between the opening and closing comment elements of the "#ssi" tag, although of course document output can be generated using the "#echo" tag.

The example SSI document WASD_ROOT:[WASDOC.ENV]XSSI.SHTML will demonstrate this concept.

4.5.21#STOP

The #stop directive causes the server to stop processing the virtual document. It can be used with flow control structures to conditionally process only part of a virtual document. Note that the #exit directive also is available, it stops processing of the current file (for nested #includes, etc.).

4.6Variables

The SSI processor maintains information about the server, date and time, request path, request parameters, etc., accessible via variable name. Although these server variables cannot be modified by the document the processor also allows the author to create and assign new document variables by name. SSI variables have global scope, with a small number of exceptions listed below. That is, the same set of variables are shared with the parent document by any other SSI documents #included, and any included by those, etc.

Local variables:

• DOCUMENT_DEPTH – the current nesting level for #included SSI files
• PARENT_FILE_NAME – if an #included SSI file the name of the including file
• THIS_FILE_NAME – the name of the SSI file currently being processed

One other special-purpose variable, THE_FILE_NAME, see 4.9.1 THE_FILE_NAME.

Server assigned variables comprise some SSI-specific as well as the same CGI variables available to CGI scripts. These may be found listed in the CGI Variables in WASD Scripting document. <P> The following link provides a list of the SSI and CGI variables available to SSI documents.

Whenever a directive uses information from a tag (see 4.4 Directive Syntax) values from variables may be substituted as as a whole or partial value. This is done using curly braces to delimit the variable name. For example

Variables are considered numeric when they begin with a digit. Those beginning with an alphabetic are considered to have a numeric value of zero.

Variables are considered to be boolean false if empty and true when not empty.

Substrings

It is also possible to extract substrings from variables using the following syntax,

where the start-index begins with the zeroth character and numbers up to the last character in the string, and count may be zero or any positive number. If only one number is supplied it is regarded as a count and the string is extracted from the zeroth character.

To illustrate,

would output

Other "Functions"

• LENGTH - This "function" returns the length of the parameter string (or substring).
  {variable-name[,start-index],count]],length}

  For example

  <!--#set var=EXAMPLE value="This is an example!" --> <!--#echo "\"{EXAMPLE}\" is {EXAMPLE,length} characters long." --> <!--#echo "\"{EXAMPLE,5,2}\" is {EXAMPLE,5,2,length} characters long!" -->

  would output

  "This is an example!" is 19 characters long. "is" is 2 characters long!
• EXISTS – This "function" returns true if the variable exists and false if it does not. This is useful as accessing a non-existant variable will result in an SSI error message with document processing ceasing!
  var={variable-name,exists}

  For example

  <!--#set var=BOGUS_VARIABLE value="irrelevant" --> <!--#if var={BOGUS_VARIABLE,exists} --> &quot;BOGUS_VARIABLE&quot; exists! <!--#else --> &quot;BOGUS_VARIABLE&quot; does NOT exist! <!--#endif -->

The example SSI document WASD_ROOT:[WASDOC.ENV]XSSI.SHTML can demonstrate these concepts.

4.7Flow Control

WASD SSI allows blocks of document to be conditionally processed. This uses constructs in a similar way to any programming language. The emphasis has been on simplicity and speed of processing. No complex expression parser is provided. Despite this, complex document constructs can be implemented. Flow control structures may be nested up to eight levels.

• #if – Marks the start of a conditionally processed block. If evaluated true the block is processed.
• #orif – Allows a document block to be processed if one of multiple conditions are met. Must be used immediately following a "#if" or "elif". (This is the only really WASD-idiosyncratic element)
• #elif – Allows a series of conditionals to be tested each with its own document block available for processing. Allows a type of case statement to be constructed.
• #else – Provides a default document block following unsuccessful "#if", "orif" and "elif" testing and consequent non-processed blocks.
• #endif – Terminates a conditional block.

The "#if", "#orif" and "#elif" directives must provide an evaluation. This can be single variable, which if numeric and non-zero is considered true, if zero if false, or can be a string, which if empty is false, and if not empty is true. Tests can be made against the variable which when evaluated return a true or false. Multiple tests may be made against the one variable, or against more than one variable. Multiple tests act as a logical AND of the results and terminate when the first fails.

• eqs – If the supplied string is the same as the variable string.
• srch – If the variable string matches the supplied search string. The search string may contain the * (asterisk), matching any zero or more characters, and % (percentage), matching any one character.
• eq – If the numeric value of the variable is the same as that of the supplied number. For a numeric value test to be legitimate both values must begin with a digit. Those beginning with an alphabetic are considered zero.
• lt – If the numeric value of the variable is less than that of the supplied number.
• gt – If the numeric value of the variable is greater than that of the supplied number.

Any evaluation can have the result negated by prefixing it with an exclamation point. For instance, the first of these examples would produce a false result, the second true.

The following is a simple example illustration of variable setting, use of variable substrings, and conditional processing of document blocks.

The example SSI document WASD_ROOT:[WASDOC.ENV]XSSI.SHTML further illustrates these concepts.

4.8Query Strings

A query string may be passed to an SSI document in much the same way as to a CGI script. In this way the behaviour of the document can be varied in accordance to information explicitly passed to it when accessed. To prevent the server's default query engine being given the request precede any query string with "?httpd=ssi". The server detects this and passes the request instead to the SSI processor. Just append the desired query string components to this as if they were form elements. For example:

The following link uses the example SSI document WASD_ROOT:[WASDOC.ENV]XSSI.SHTML to demonstrate this. Look for the <QUOTE>(FORM_TEST1=one), etc.

4.9File and Virtual Specifications

Documents may be specified using either the "FILE" or "VIRTUAL" tags.

The "FILE" tag expects an absolute VMS file specification.

The "VIRTUAL" tag expects an URL-style path to a document. This can be an absolute or relative path. See 2.3 Document Specification for further details.

4.9.1THE_FILE_NAME

Generally, when an error are encountered document processing halts and and an error report is generated. For some common circumstances, in particular the existance or not of a particular file, may require an alternative action. For file activities (e.g. #include, #flastmod, #created, #fsize) the optional fmt="" tag provides some measure of control on error behaviour. If the format string begins with a "?" files not found are not reported as errors and processing continues. Other file systems errors, such as directory not found, syntax errors, etc., are always reported.

Every time a file is accessed (e.g. #include, #flastmod) the server variable THE_FILE_NAME gets set to that name if successful, or reset to empty if unsuccessful. This variable can be checked to determine success or otherwise.

• For #included files, the 'fmt="?"' just suppresses an error report, if the file exists then it is included.
• For #modified file specifications use 'fmt="?"' to suppress error reporting on evaluation of files that may exist but are not mandatory.
• For file statistic directives (e.g. #flastmod, #fcreated, #fsize) the 'fmt="?"' tag completely suppresses all output as well as error reporting. This can be used to check for the existance of a file. For example if the file TEST.TXT exists in the following example the variable THE_FILE_NAME would contain the full file name, if it does not exist it would be empty, and the code example would behave accordingly.
  <!--#fcreated virtual="TEST.TXT" fmt="?" --> <!--#if var={THE_FILE_NAME} eqs="" --> File does not exist! <!--#else --> File exists! <!--#endif -->

4.10Time Format

Whenever a time directive is used an optional tag can be included to specify the format of the output. The default looks a little VMS-ish. If a format specification is made it must confirm to the C programming language function strftime().

The format specifier follows a similar syntax to the C standard library printf() family of functions, where conversion specifiers are introduced by percentage symbols. Here are some example uses:

A problem with any supplied time formatting specification will be reported.

The following table provides the general conversion specifiers. For further information on the formatting process refer to a C programming library document on the strftime() function.

strftime() Format Directives

4.11OSU Compatibility

Essential compatibility with OSU Server Side Includes directives is provided. This is intended to ease any transition to WASD, as existing SSI documents will not need to be changed unless any of the WASD capabilities are required. To provide transparent processing of OSU .HTMLX files ensure the following WASD configuration is in place.

In HTTPD$CONFIG file:

Note that the content description must contain the string "OSU" to activate some compliancy behaviours.

In HTTPD$MAP file:

This provides a mechanism for the OSU part-document facility. (Yes, the "__part" has two leading underscores!)

OSU Directives

The following OSU directives are provided specifically for OSU compatibility, although there is no reason why most of these may not also be deployed in general WASD SSI documents if there is a requirement. Note that these are OSU-specifics, other OSU directives are provided by the standard WASD SSI engine.

OSU Compatible Directives

If WASD is configured for OSU SSI compatibility the following link provides an online demonstration as well as further explanation of the OSU SSI engine using an OSU preprocessor document from the distribution (included within copyright compliance).

/wasd_root/src/osu/preproc.htmlx

How do we know WASD is processing it? Look for the #echo var="GETENV=SYS$REM_ID" towards the end of the document. It should indicate "[VARIABLE_DOES_NOT_EXIST!]" because it's attempting to output a DECnet-related logical name!

OSU "Part"s

The OSU processor allows for delimited subsections of an #included document, or a URL referenced document for that matter, to be included in the output. This is supported, but only for compatibility. It is only enabled for ".HTMLX" documents and if otherwise used may interact unexpectedly with WASD SSI flow-control.

4.12Script-Generated SSI Documents

It is possible to have script output passed back through the SSI engine for markup. This approach might allow script output to automatically be wrapped in standard site headers and footers for example. Essentially the script must output an SSI-markup response body and include in the otherwise standard CGI response header a field containing "Script-Control: X-content-handler=SSI". The following example in DCL show the essential elements of such a script.

5.Document Searching

The query and extract scripts provide real-time searching of plain-text and HTML documents, and document retrieval. The search is a simple-string search, not a GREP-style search. It is designed to provide a useful mechanism for locating documents containing a keyword, not for document analysis. It has the useful feature for plain-text documents of allowing the selective extraction of only the portion near the hit.

Only files with a plain-text or HTML MIME data type (see 2. Document Access and Specification) will be searched. Others may be specified, or be selected from wildcard file specification, but they will not actually have their contents searched.

Directory specifications may include a wildcard elipsis (allowing a directory tree to be traversed) and/or file name wildcards. In other words, anything acceptable as VMS file system syntax (except in URL-format of course). See examples in 5.3.1 Standard Search Form.

5.1Plain-Text Search

A search of a plain-text file is straight-forward. Each line in the file is searched for the required string. The first time it is encountered is considered a hit. The line is not searched for any further occurances.

Searches of plain text files allow the subsequent selection of partial documents (i.e. the retrieval of only a number of lines around any actual hit). This allows the user to selectively extract a portion of a document, avoiding the need to explcitly scan through to the section of interest.

5.2HTML Search

A search of an HTML file is a little more complex. As might be expected, only text presented in the document text is searched, markup text is ignored. That is, all text not part of an HTML tag construct is extracted and searched. For example, out of the following HTML fragment

The mechanism for partial document retrieval available with plain-text files is not present with HTML documents. HTML files generally must be treated as a whole, with the formatting of current sections often very dependent on the formatting of previous sections. This makes extracting a subsection perilous without extensive syntactical analyis. On the positive side, HTML documents tend to be already divided into meaningful subdocuments (files), making retrieval of a hit naturally more-or-less within context.

Instead of partial document retrieval, the document is processed to place anchors for each hit, making it possible to jump directly to a particular section of interest. Generally this works well but may occasionally distort the presentation of a document.

5.3Search Syntax

A search may be initiated in basic three ways:

1. Appending a question-mark and search string to a file specification (the simple syntax of "ISINDEX"-style searching). This is standard HTTP, and of course must conform to HTTP syntax.
2. Providing the name of the query script followed by the directory path to be searched. The script then returns a standard search form.
3. Forms-based search, which allows the format and mechanism of the search to be controlled.

5.3.1Standard Search Form

Using the "QUERY" script name followed by a URL-format path specifying the directory to be searched returns a standard, script-generated search form.

The following link provides an online demonstration of the standard search form.

/cgi-bin/query/wasd_root/wasdoc/env/

As with all search specifications, the directory specification may include wildcard a elipsis (allowing a directory tree to be traversed) and/or file name wildcards. In other words, anything acceptable as VMS file system syntax (except in URL-format of course). See the following examples.

5.3.2Forms-Based Search

A "forms-based" search is initiated by the server receiving a file specification, which of course may contain wildcards, followed by a search parameter. This is a typical HTML forms format URL. For example:

The following link provides an online demonstration search using the form-based syntax.

/wasd_root/wasdoc/env/*.*?search=formatted

5.3.3Search Options

Additional URI components may be appended after the initial "search=" parameter. These are appended with intervening "&") characters.

• Case-Sensitivity – An optional URI component of "case=yes" or "case=no" makes the search case-sensistive or case-insensistive (the default). The following example illustrates the use of this syntax:

  /wasd_root/wasdoc/env/*.html?search=Protocol&case=yes	case-sensistive search for "Protocol"
  /wasd_root/wasdoc/env/*.html?search=PrOtOcOl&case=no	case-insensistive search for "PrOtOcOl"

• Hits – An optional URI component of "hits=document" or "hits=line" makes the search results be presented by-document (file) or by line-by-line (the default). The following example illustrates the use of this syntax: /web/html/.../*.html?search=protocol&hits=document /web/html/.../*.html?search=protocol&hits=line

  /wasd_root/wasdoc/env/*.html?search=protocol&hits=document	search result granularity by document
  /wasd_root/wasdoc/env/*.html?search=protocol&hits=line	search result granularity by line (the default)

5.3.4Example Search Form

To allow the client to enter a search string and submit a search to the server a HTML level 2 form construct can be used. Here is an example:

The following provides an online demonstration of the form used above:

Bells and Whistles

A form providing all the options refered to in 5.3.3 Search Options is shown below (some additional white-space introduced for clarity):

The following provides an online demonstration of the form used above:

6.VMS Help and Text Libraries

Affectionately known as Conan the Librarian – and with all due acknowlegement to Wierd Al.Yankovic   – this script makes VMS Help and Text libraries accessible in the web environment.

The librarian script will be automatically activated if the file specified has an extension of ".HLB" or ".TLB". Alternatively it may be explicitly activated by specifying as a prefix to the file specification (but the ability to provide a relative specification is lost). The following examples illustrate the syntax:

Other Librarian Functionality

To obtain an index of matching libraries explicitly activate the Conan script providing a wildcard file specification.

To obtain the library header information add a query string to the library file specification, as shown in the following example:

7.Bookreader Books and Libraries

While largely of historical interest these days, access to Bookreader format documentation (and any generated internally) is provided in the WASD web environment via two integrated scripts, HyperReader, which reads the books, and HyperShelf, which reads the Library and Shelf structures.

The HyperReader and HyperShelf scripts are automatically activated when the document file's extension is ".DECW$BOOK" and ".DECW$BOOKSHELF" respectively. Alternatively, the respective scripts may be explicitly specified (but the ability to provide a relative specification is lost).

If the server system supports Bookreader documentation collection one ot both the following links will provide an online demonstration:

8.Web Document Update

The Update facility allows Web documents and file environments to be administered from a standard browser. This facility is available to Web administrator and user alike. Availability and capability depends on the authorization environment within the server.

It should be stressed that this is not designed as a full web administration or authoring tool, and for document preparation relies on the editing capabilities of the "<textarea>" widget of the user's browser. It does however, allow ad-hoc changes to be made to documents fairly easily, as well as allowing documents to be deleted, and directories to be created and deleted.

Consult the current Update documentation for usage detail.

And depending on site configuration /upd/wasd_root/ an example of the interface.

Update Access Permission

Of course, the user must have write (POST/PUT) access to the document or area on the server (i.e. the path) and the server account have file system permission to write into the parent directory.

Contact the Web Administrator for further information on the availablility of authentication and authorization permissions to do online updates of Web paths.

The server will report "Insufficient privilege or object protection violation ... /path/" if it does not have file system permission to write into a directory.

Write access by the server into VMS directories (using the POST or PUT HTTP methods) is controlled using VMS ACLs. This is in addition to the path authorization of the server itself of course! The requirement to have an ACL on the directory prevents inadvertant mapping/authorization of a path resulting in the ability to write somewhere not intended.

Two different ACLs implement two grades of access.

1. If the ACL grants CONTROL access to the server account then only VMS-authenticated usernames with security profiles can potentially write to the directory. Only potentially, because a further check is made to assess whether that VMS account in particular has write access.

   This example shows a suitable ACL that applies only to the original directory:

   $ SET SECURITY directory.DIR - /ACL=(IDENT=HTTP$SERVER,ACCESS=READ+CONTROL)
   This example shows setting an ACL that will propagate to created files and importantly, subdirectories:
   $ SET SECURITY directory.DIR - /ACL=((IDENT=HTTP$SERVER,OPTIONS=DEFAULT,ACCESS=READ+WRITE+DELETE+CONTROL), - (IDENT=HTTP$SERVER,ACCESS=READ+WRITE+DELETE+CONTROL))
2. If the ACL grants WRITE access then the directory can be written into by any authenticated username for the authorized path.

   This example shows a suitable ACL that applies only to the original directory:

   $ SET SECURITY directory.DIR - /ACL=(IDENT=HTTP$SERVER,ACCESS=READ+WRITE)
   This example shows setting an ACL that will propagate to created files and importantly, subdirectories:
   $ SET SECURITY directory.DIR - /ACL=((IDENT=HTTP$SERVER,OPTIONS=DEFAULT,ACCESS=READ+WRITE+DELETE), - (IDENT=HTTP$SERVER,ACCESS=READ+WRITE+DELETE))

9.Index

10.Attribution and Acknowledgement

WASD VMS Web Services – Copyright © 1996-2021 Mark G. Daniel

Licensed under the Apache License, Version 2.0

None of the following licensing appears incompatible with the Apache License

Clark Cooper, et.al.

This package uses the Expat XML parsing toolkit.

Bjöern Höehrmann

This package uses essential algorithm and code from Flexible and Economical UTF-8 Decoder.

Free Software Foundation

This package contains software made available by the Free Software Foundation under the GNU General Public License.

Ohio State University

This package contains software provided with the OSU (DECthreads) HTTP server package, authored by David Jones:

OpenSSL Project

This product can include software developed by the OpenSSL Project for use in the OpenSSL Toolkit (https://www.openssl.org/).

Paul E. Jones

This package uses SHA-1 hash code.

RSA Data Security

This software contains code derived in part from RSA Data Security, Inc:

Stuart Langridge

SortTable version 2
Stuart Langridge, http://www.kryogenix.org/code/browser/sorttable/

Tatsuhiro Tsujikawa

nghttp2 - HTTP/2 C Library
Tatsuhiro Tsujikawa, https://github.com/tatsuhiro-t

VSI OpenVMS, VSI TCP/IP Services for OpenVMS, VSI C
are registered trademarks of VMS Software Inc.

OpenVMS, HP TCP/IP Services for OpenVMS, HP C, Alpha, Itanium and VAX
are registered trademarks of Hewlett Packard Enterprise

MultiNet and TCPware are registered trademarks of Process Software Corporation