| ↩︎ | ↖︎ | ↑︎ | ↘︎ | ↪︎ |
Web-based Distributed Authoring and (not) Versioning for the WASD package.
Effective WASD WebDAV file-space (without significant naming constraints) relies on being hosted on ODS-5 volumes. Behaviour hosting file-space on ODS-2 volumes is untested (though possible provided file naming is constrained to ODS-2 conventions).
WASD WebDAV methods and request headers, etc., are also propagated to the scripting environment and so functionality may be implemented using CGI, CGIplus or RTE based applications.
WASD proxy-serving supports WebDAV methods, header fields, etc.
Generally WebDAV clients are applications other than browsers and so response bodies with human-readable error explanations are unnecessary and consume bandwidth to no good purpose, and so not provided.
File-systems are notoriously latent components relative to the rest of the system (more so with VMS). Any operation to collections (directories) are not going to be atomic and for large collections requiring many sub-operations the potential for the process to be interrupted or otherwise disturbed are enormous. File-systems are not databases amenable to extensive ACID operations.
In addition each file under WebDAV management has the potential for an associated but independent metadata file. This of course means for every DAV-specific resource file activity there is at least a file-system action to check for a metadata file and for some actions such as COPY the potential for an associated but entirely independent file operation.
Of course WebDAV was not intended or designed as a general file-system protocol but one for distributed management of somewhat restricted collections of Web-related resources and so in context probably works well enough.
See sections below on file-system operation method restrictions.
during server WebDav file-system modifications is a recipe for inconsistency and/or corruption!
A list of WebDAV methods, what WASD does with them, and any limitations or restrictions. Some of these are familiar HTTP/1.n methods and other are RFC 4981 specific. Some of the HTTP/1.n methods are overloaded with additional or variant behaviours when used in a WebDAV context. Issues of atomicity with the manipulation of file-system trees containing numbers of individual files makes strict RFC 4918 compliance difficult. See "…Restrictions" below.
| Method | Description |
|---|---|
| COPY** | Reproduces both single resources (files) and collections (directory trees). Will overwrite files (if specified by the request) but will respond 209 (Conflict) if it would overwrite a tree. |
| DELETE** | deletes files and directory trees |
| GET | just the vanilla HTTP/1.1 behaviour |
| HEAD | ditto |
| LOCK** | see WEBDAV LOCKING below |
| MKCOL** | create a directory |
| MOVE** | Moves (rename or copy) a file or a directory tree. Will 'overwrite' files (if specified by the request) but will respond 209 (Conflict) if it would overwrite a tree. |
| OPTIONS | If WebDAV is enabled and available for the path this reports the WebDAV extension methods |
| PROPFIND** | Retrieves the requested file characteristics, DAV lock status and 'dead' properties for individual files, a directory and its child files, or a directory tree. |
| PROPPATCH** | set and remove 'dead' meta-data properties |
| PUT | Against a WebDAV resource behaves a little differently to historical WASD implementation of PUT. |
| UNLOCK** | see WebDAV locking below |
| **WebDAV RFC 4918 method |
WASD Statistics Reports gather WebDAV related data. Where a method can be used both for vanilla HTTP/1.n and WebDAV purposes it is counted in WebDAV statistics if the request header contains some other indication of a WebDAV activity.
Does not comply with the overwrite:T directive for collections (does so for files). Will not preemptively delete the existing tree. It returns a 209 (Conflict) response instead.
COPY does not maintain collection consistent URL namespace if a member resource cannot be moved as required by RFC 4918. It should maintain the source subtree completely uncopied. Instead it is best-effort and continues copying resources until exhausted. This is consistent with file-system behaviour. The RFC 4918 requirement, while not impossible, is fraught with issues inside a file-system.
Deletion of collections is particularly fraught with issues for a file-system. In userland it is almost impossible to predetermine if an individual file in a directory tree is going to resist deletion (due to locking, protections, etc) and in kernel land it's probably no easier. It leaves the undeleted tree hierachy (resource ancestors) intact. This is RFC 4918 compliant however!
So, in the case of WASD WebDAV it's just best-effort and if something down the tree won't disappear, it just reports the failure in the 207 response and carries merrily on through the tree regardless. This IS acceptable WebDAV server behaviour!
Does not comply with the overwrite:T directive for collections (does so for files). Will not currently pre-emptively delete the existing tree. It returns a 209 (Conflict) response instead.
MOVE first attempts to rename the file or directory. This is reasonably efficient, especially for directory trees but obviously only suitable for a target on the same disk volume. If a rename failure is due to a different device it falls back to using a COPY then DELETE in two separate phases. Needless-to-say this is hardly atomic and can lead to inconsistencies between source and target.
MOVE does not maintain collection consistent URL namespace if a member resource cannot be moved as required by RFC 4918. It should maintain the source subtree unmoved. Instead it is best-effort and continues moving resources until exhausted. This is consistent with file-system behaviour. The RFC 4918 requirement, while not impossible, is fraught with issues inside a file-system.
The conditional "If:" request header field does not have full RFC 4918 support. It implements lock token and etag token processing with parenthetical OR and NOT processing. For unsupported features WATCH reports that the header was not understood and always returns an abort status. WebDAV "If:" processing is an extrodinarily complex kludge for on-the-fly decision making by the server and much of what I have read indicates most clients only ever use extremely simple conditions anyway.
WebDAV and its features are globally enabled and configured using directives contained in the WASD_CONFIG_GLOBAL configuration file.
| Directive | Description |
|---|---|
| [PutMaxKBytes] | maximum size of a file (PUT and POST) |
| [WebDAV] | This directive enables and disables WebDAV. |
| [WebDAVlocking] | Enables and disables WebDAV locking. |
| [WebDAVlockTimeoutDefault] | see ‘Locking Timeout’ in 6.4 WebDAV Locking |
| [WebDAVlockTimeoutMax] | see ‘Locking Timeout’ in 6.4 WebDAV Locking |
| [WebDAVlockCollectionDepth] | See ‘Locking Depth’ in 6.4 WebDAV Locking |
| [WebDAVmetaDir] | see 6.3 WebDAV Metadata |
| [WebDAVquota] | Enables and disables RFC 4331 functionality (disk quota reporting). |
In addition these and other configurations are provided on a per-path basis using mapping rules.
WASD request processing rules (see Request Processing Configuration of WASD Configuration) may be used on a per-path basis to modify (some) global configuration settings and provide other WevDAV configuation.
| Rule | Description |
|---|---|
| ODS=NAME=8BIT|UTF8|DEFAULT | When a file is PUT using WebDAV (or upload), for non-7bit ASCII file names use native ODS-5 8bit syntax (default) or UTF-8 encoded character sequences (see 6.2.2 File Naming) |
| PUT=MAX=<integer> | * | Maximum number of kilobytes file size, if "*" then effectively unlimited (per-path equivalent of the global directive [PutMaxKBytes]). |
| WEBDAV=[NO]HIDDEN | list (default) or hide U*x hidden files (i.e. those with names beginning with period) |
| WEBDAV=[NO]LOCK | allow/apply WebDAV locking to this path |
| WEBDAV=[NO]PROFILE | WebDAV access according to SYSUAF profile |
| WEBDAV=[NO]PROP | allow/apply WebDAV 'dead' property(ies) to this path |
| WEBDAV=[NO]PUT=LOCK | a resource must be locked before a PUT is allowed |
| WEBDAV=[NO]READ | WebDAV methods allowed read this tree |
| WEBDAV=[NO]SERVER | WebDAV access as server account (best effort) |
| WEBDAV=[NO]WINPROP | when NOWINPROP windows properties are ignored and emulated |
| WEBDAV=[NO]WRITE | WebDAV methods allowed write to this path (implied read) |
| WEBDAV=LOCK=TIMEOUT=DEFAULT= | hh:mm:ss |
| WEBDAV=LOCK=TIMEOUT=MAX= | hh:mm:ss |
| WEBDAV=META=DIR= | per-path equivalent of global [WevbDAVmetaDir] (see 6.3 WebDAV Metadata) |
An essential function of the path setting rules is for specifying which paths in server Web-space are allowed to be accessed using the WebDAV protocol and what sort of access (read, write, etc.) that path is allowed.
By default files that are PUT via WebDAV (or upload) support the ISO Latin-1 character set. ASCII and non-7-bit file names use the native ODS-5 syntax. Where character sets other than ISO Latin-1, or where compatibility with other WebDAV implementations is desired (e.g. Apache), a path can be set to allow file names supplied using UTF-8 sequences.
For example, the English language word "naïve", having a diaeresis mark over the "i" character (indicating it is pronounced separately from the preceding vowel) is commonly respresented using the 8 bit character 0xEF, or as the two byte UTF-8 sequence 0xC3AF. This word if used as the file name with a type (extension) of ".TXT" by default would have the sequence of 8-bit characters
"Index of" (directory) listings will honour a path set ods=name=utf8 and make the listing character set UTF-8 resulting in a browser correctly rendering the name (WebDAV listings are by definition UTF-8).
While files and directories created via WebDAV will have a consistent naming schema applied those created by applications or manual operation on the VMS system can result in files that are not accessible with WebDAV.
For example the file name
In addition, the two files
To avoid this situation a potentially ambiguous file name containing an escaped period and no type (extension) is ignored by directory listings and WebDAV property lists. When an ambiguous file name is detected it is reported in WATCH reports.
Is controlled using the mapping rules:
| Rule | Description |
|---|---|
| WEBDAV=PROFILE | access using request SYSUAF-authenticated security profile |
| WEBDAV=WRITE | unconditional permission to read/write |
| WEBDAV=READ | unconditional permission to read |
| WEBDAV=SERVER | access using server account permissions |
All access by WebDAV operations must have at least one set against the path. If access is permitted by one of the above settings SYSPRV is enabled to allow that access using the server account. Therefore files and directories should have a SYSTEM:READ+WRITE+EXECUTE+DELETE protection or equivalent ACL permissions, or the access may fail totally or in some part of a supposedly atomic action.
These file-system access settings are applied in the order listed above. That is, if a path successively has one or more of the above settings applied during rule processing, when it comes to applying those access controls, SYSUAF profile is applied, then if no profile SETing access to read/write, then to read-only, then access via the server account.
In addition WebDAV access requires an authorisation rule against each path.
All access by WebDAV operations must have one set against the path.
All WebDAV access is a combination of WASD_CONFIG_MAP path setting and WASD_CONFIG_AUTH authorisation permissions. The least permissive of the two overrides the more. The combination of an authorisation rule and a path mapping rule mitigates the chance of opening unintended access into the file-system.
These is the test-bench environment used during development:
Note that WebDAV read/write access is a combination of the mapping and the authorisation rule (mapping WEBDAV=READ overrides authorisation read+write). Expect complications with Microsoft environments.
For test-benching you could avoid authorisation issues completely with:
A common requirement is to provide concurrent general access and authorised WebDAV acccess to the same Web-space. This is accomplished by using two paths mapped into the same file-system space, the general access (non-authorised) path, and a WebDAV (authorised) path. The WebDAV client uses the authorised path and can then apply WebDAV methods to maintain the resources.
The following configuration is taken from a site using WebDAV to allow users to manage their Web presence. The user mapping is a fairly standard configuration for VMS accounts (see Mapping User Directories (tilde character ("~")) of WASD Configuration). User Web areas are in the [.WWW] subdirectory of the account home area.
The four WebDAV access rules are located before the three general user access rules. The WebDAV rules are more specific. The first USER rule maps subdirectories - and the parent if a trailing slash is included. The second USER rule maps the parent directory for user agents that do not include trailing slash on their directory specifications (most it seems).
The second pair of rules reverse-maps the VMS file-system specifications represented by the result (right side) of the PASS rule into the path represented by the template (left side) of the PASS rule. Mapping from file-specifications to paths is necessary because of the way the PROPFIND method searches the file-system and then reports its results to the client as URLs.
The use of the notepad rule with a string of "webdav" (the actual string is not significant as long as it is unique within the rules) is used to conditionally process the reverse-mapping rules. They will be applied only to the requests originally mapped by the USER rules. The pass:-1 ensures the rules are only applied during reverse-mapping, not during request mapping.
The fifth rules maps general Web access to the user area. Remember, web access is to a user home subdirectory [.WWW].
The sixth and seventh rules reverse-map the VMS file-system specifications for the general USER rules for similar reasons to those described above. Why two? The user directories occur across two disk volumes and so each must be reverse-mapped.
As noted above, WASD WebDAV requires both mapping and authorization rules (even for "world" - or non-authenticated - access).
In this case authorisation is only required for WebDAV access. There are two rules. The first authorises subdirectories and parent directories for agents that supply a trailing slash. The second for agents that do not provide a trailing slash.
… two rules for each location? Why
For finer control. The first only matches requests with a path of "/~user/dav/subdir/" and "/~user/dav", whereas the latter matches "/~user/dav/subdir/" and "/~user/dav" and "/~user/david/" and "/~user/davros", etc.
Metadata is data (information) about data. WebDAV uses the concept of a resource property. There are "live" properties and "dead" properties. Essentially the live properties are the dynamic characteristics of a file-system object represented by creation and modification date-times, object size, etc. WebDAV dead properties are those supplied by WebDAV clients as XML entities and stored associated with the particular WebDAV object, in WASD's case the file-system object (file or directory). WASD also uses the file metadata to store resource lock data (see 6.4 WebDAV Locking).
WASD manages resource metadata using a separate file associated by name with the data file. This is done for reasons of programmatic simplicity and for the convenience of any command-line owner or sysadmin of the resources. No specialised tools are required. This metadata file can be stored in one of three locations.
For example, with the global directive
For example, with the global directive
The metadata file associated with a directory is stored in the same metadata location as files contained by that directory (not in the metadata location associated with the parent directory that contains the directory file). This metadata file is named ".DIR__wasdav" (i.e. no name, just an extension), with the following example illustrating how this would appear in each of the three metadata locations, for a subdirectory named "New Folder".
All metadata is stored using XML. Multiple XML data can be contained in a single metadata file. Each can be individually manipulated by a WebDAV client. The property elements are stored as-supplied by the client. It is presumed that their XML well-formedness is guaranteed by the original request XML parsing. Metadata files have content similar to the following:
This metadata example contains four properties; an exclusive write lock owned by "MGD" and three set by a client in three different (contrived) namespaces.
An example of such property meta-data generated by a Microsoft Windows (not Internet) Explorer client (example wrapped for presentation):
Every file written or modified by Windows Explorer generates this sort of metadata which is then stored in an associated metadata file and read each time the data file is accessed. Some might consider this unnecessary clutter in most circumstances (I do). WASD allows this metadata to be suppressed and equivalent data generated (fudged) from file live properties when accessed - often sufficient for purpose. To suppress the actual processing of Windows Explorer metadata set a path using the WEBDAV=NOWINPROP in WASD_CONFIG_MAP.
For efficiency and functionality considerations WebDAV locking may be enabled and disabled (default) as global functionality using the WASD_CONFIG_GLOBAL [WebDAVlocking] directive. Additionally the WEBVDAV=[NO]LOCKING path SETing can configure this on a per-path basis.
In common with RFC 4918 WASD WebDAV locking controls only write access. Both exclusive and shared locks are provided. Locking applies to the DELETE, LOCK, MKCOL, MOVE, PROPPATCH, PUT, and UNLOCK methods.
WASD WebDAV locking checks parent collections to a configurable depth. WASD_CONFIG_GLOBAL directive [WebDAVlockCollectionDepth] where the default (0 or 1) checks only WebDAV locking on files, 2 WebDAV locking on the parent directory, 3 on the grandparent, 4 the great-grandparent, etc. Of course each level can add significant latency (and expense) to some operations.
For more information on locking operation and implementation details see the DAVLOCK.C module and for meta-data in general the DAVMETA.C module.
When a client locks a resource it can specify the period for the lock. In the absence of such a specification WASD will apply the [WebDAVlockTimeoutDefault] value (by default 0-01:00:00 - one hour). WASD also applies the [WebDAVlockTimeoutMax] maximum lock period (by default 7-00:00:00 - one week). When the maximum period expires the lock is no longer valid.
WASD uses VMS locking to queue and arbitrate access to WebDAV resources and meta-files.
Two lock modes are employed; 'exclusive', when changes are to be made to the resource or its meta-data, and 'concurrent read', when resource and/or meta-data are only to be read. Concurrent read locks are compatible, but an exclusive queued against a resource currently being read waits, as does a read against a current exclusive.
WASD takes out its own VMS DLM locks on resources (files and directories) before beginning any WebDAV operation, and these prevent conflict with other WASD WebDAV operations on the same system or cluster, but RMS does not use these nor does WASD use RMS locks (except when actually acessing the file-system of course), and so there is potential for interactions between the two domains (in common with general file-system actvities). WASD WebDAV deliberately does not try to block file-system actions from other processing (except where RMS locks/blocks). Its own DLM locking is purely for internal purposes.
Some application/environment-specific considerations when using WASD WebDAV. Please report any you encounter for future inclusion in this section. Also see 6.6 Microsoft Miscellanea immediately below.
OS X Finder requires [WebDAVlocking] enabled for read/write access, otherwise access will be read-only.
Dreamwever 8 (at least, the only version I have access to) insists on using a URI with a trailing "/./" occasionally (I'm guessing to specify the "current" directory - cf. "/../", or "parent" syntax). Just absorb this internally using an appropriate mapping internal redirect.
A cornucopia of of minor and major considerations!
Microsoft approach WebDAV in their own inimitable fashion. Hence Microsoft agents, considering their ubiquity, including their mini-redirector are specifically looked for and functionality modified to accomodate them.
The following is a list topics/issues that were encountered/investigated during WASD WebDAV development. They may or may not be applicable to your site.
Some general references:
DOS/Windows command-line network configuration:
Microsoft agents (at least) seem to request the server OPTIONS of the server root regardless of any path provided with the NET USE or other network drive mapping employed. To selectively map such a request into a path that has WebDAV enabled on it (and will therefore respond with the DAV-related options) use a conditional redirect rule. For example
Subsequent rules will probably be required to map typeless directory requests to the actual directory required.
Requests containing paths /_vti_inf.html and /_vti_bin/* are related to FrontPage protocol discovery probing. They can be adequately handled using a mapping rule lsuch as the following:
See ‘Microsoft Metadata’ in 6.3 WebDAV Metadata.
If the server's response does not contain an MS-Author-Via header, the OLE DB Provider for Internet Publishing loads the WEC and WebDAV protocol drivers one at a time (WEC first, WebDAV second) and asks them, "Do you know how to handle this URL?", specifying the exact URL passed in by the client. The first protocol which responds "yes" is selected. If neither protocol driver responds "yes" then the method which triggered the automatic driver selection (usually IBindResource::Bind) fails with an OLE DB Provider for Internet Publishing specific error code IPP_E_SERVERTYPE_NOT_SUPPORTED.
Some Windows XP machines have a broken Web Folders installation. Microsoft includes a Web Folders repair utility built in to Windows to correct the problem. Use the following steps to fix the problem:
Attach the port-number (80 by default) to the http-address you enter into the field of the "My Network Places"-assistant. As you can see in the following image and the linked screenshot, this will force Windows XP to use the "Microsoft Data Access Internet Publishing Provider DAV 1.1" mechanism instead of "Microsoft-WebDAV-MiniRedir/5.1.2600".
It is also possible to add the number sign # to the http-address you enter into the field of the "My Network Places"-assistant. As you can see in the following image and the linked screenshot, this will also force Windows XP to use the "Microsoft Data Access Internet Publishing Provider DAV 1.1" mechanism instead of "Microsoft-WebDAV-MiniRedir/5.1.2600".
There is a third way to get this working from the client-site. As described in the Microsoft Knowledge Base, Article ID: 841215, Windows XP disables "Basic Auth" in his "Microsoft-WebDAV-MiniRedir/5.1.2600"-mechanism by default for security reasons. See description below.
You can enable BasicAuth by adding the following registry key and setting it to a non-zero value:
If you delete the registry key or set it to 0, the behavior reverts to the default, or disabling the use of BasicAuth.
Disabling Basic Authentication over a clear channel:
Because the DAVRdr is part of the remote file-system stack, a computer is open to attack whenever an attempt is made to remotely access files. Although the threat to other applications that use the Internet APIs is less severe than it is for the DAVRdr, a similar attack is possible whenever an application (or the user) attempts to access a URL. For this reason, WinInet is exposing the mechanism by which the DAVRdr disables BasicAuth to other users of the Internet APIs.
With Windows XP Service Pack 2, there are two ways to block the use of Basic Authentication over clear (or unencrypted) channels:
Create the following registry key and set it to a non-zero value.
This prevents WININET from attempting to use BasicAuth unless the channel is secured (HTTPS or SSL).
The application can disable the use of BasicAuth for its connections by setting the AUTH_FLAG_DISABLE_BASIC_CLEARCHANNEL flag (0x4) in the value supplied in the call to InternetSetOption using INTERNET_OPTION_AUTH_FLAGS.
You can enable BasicAuth by setting the following registry key to the value 3 and restarting the WebClient service:
"In my case I try to copy file over WEBDAV to WEB Client connection e.g. I have mapped drive to web site. file is about 70MB I can copy small files from the same WEBDav folder."
These are the resources used during WASD WebDAV development.
All these have been used during WASD WebDAV development.
| ↩︎ | ↖︎ | ↑︎ | ↘︎ | ↪︎ |