/*****************************************************************************/ #ifdef COMMENTS_WITH_COMMENTS /* qdLogStats.c Quick & Dirty LOG STATisticS. Utility to extract very elementary statistics from Web server common/combined format log files. A number of filters allow subsets of the log contents to be selected. If a filter pattern does not begin with the '^' character then it is considered to be a wildcard pattern. If the pattern begins with the '^' character then it is considered to be regular expression. A filter pattern result can be negated by the use of a leading '!' character. If the negation is used then the target string cannot be empty. If it's empty it always treated as no match (and filtered out). WILDCARD EXPRESSIONS -------------------- A wildcard expression uses the '*' to match any zero or more characters, the '%' to match any one character, the '?' to match any zero or one character. The '*', '%' and '?' are translated into an appropriate regular expression, with the start and end of the string anchored. This should act much the same as previous versions of QDLOGSTATS. If you want to do anything more complex than simple wildcard matching you will need to use a regular expression. The following expression would match all requests with a path that was rooted at "/wasd_root/". /wasd_root/* This expression matches all paths that accessed the Alpha object files in the WASD source (and build) tree. /wasd_root/*/obj_axp/* REGULAR EXPRESSIONS ------------------- Are provided using the GNU extended regular expression matching and search library, version 0.12 (from RX1.5 kit). Copyright (C) 1993 Free Software Foundation, Inc. Although a full explanation of regular expression matching is well beyond the scope of this source code description here's a quick mnemonic. \ Quote the next metacharacter ^ Match the beginning of the line . Match any character (except newline) $ Match the end of the line | Alternation (or) [abc] Match only a, b or c [^abc] Match anything except a, b and c [a-z0-9] Match any character in the range a to z or 0 to 9 * Match 0 or more times + Match 1 or more times ? Match 1 or 0 times {n} Match exactly n times {n,} Match at least n times {n,m} Match at least n but not more than m times Match-self Operator Ordinary characters. Match-any-character Operator . Concatenation Operator Juxtaposition. Repetition Operators * + ? {} Alternation Operator | List Operators [...] [^...] Grouping Operators (...) Back-reference Operator \digit Anchoring Operators ^ $ More detailed explanations abound. Keep in mind that this program uses the library in case-insensitive, extended Posix mode (basically EGREP look-alike). The following expression would match all requests with a path that was rooted at "/wasd_root/". ^^/wasd_root/.* This expression matches all paths that accessed the Alpha object files in the WASD source (and build) tree. ^^/wasd_root/.*/obj_axp/.* Note the leading '^' which directs the program that this is a regular expression. OTHER CONSIDERATIONS -------------------- Matching log file records can be viewed as processed using the /VIEW=[type] qualifier, where is ALL, MATCH (default) or NOMATCH. For fields that are commonly URL-encoded (or for any field that can usefully be URL-encoded to disguise the actual path) the /DECODE qualifier specifies that the field should be URL-decoded before filter matching. If the /DECODE qualifier is used without keywords all three of these fields are decoded, otherwise any combination of PATH, QUERY and REFERER may be used to selectively decode that field. For example /DECODE=(PATH,QUERY). Remember that decoding uses CPU cycles and adds processing time. A progress indicator can be displayed using the /PROGRESS qualifier. With this a "+" is output for each file processed and a "." for each 1000 records in any file. An optional integer can be provides as /PROGRESS=integer to change this record value. Note that all selections are based on string matching. Files and/or records cannot be selected on the basis of being before or after a particular date for instance. Also that records are accepted or rejected in the order in which the results are displayed, that is if a record is rejected against method it will never be assessed against user-agent ... UNLESS then /ALL qualifier is applied. When /ALL is used comparison continues against all filter strings even if one or more do not match. This is useful for displaying the totals that match each of multiple filter critera (not much use against a single one). If /LOOKUP[=] is enabled then literal client host addresses have their names resolved (if possible). This name is displayed when log line is output. When enabled the resolved name is also used when matching against client hosts. Although part of the WASD package there is no reason why this shouldn't be of use with other packages if desired. EXAMPLES -------- Requires a foreign verb or such (shorter version might save keystrokes :^) $ QDLOG == "$CGI-BIN:[000000]QDLOGSTATS" $ QDLOGSTATS == "$CGI-BIN:[000000]QDLOGSTATS" 1) Records from September 1999. $ QDLOGSTATS WASD_LOGS:*1999*.LOG /DATE="*/SEP/1999*" 2) Records where the browser was an X-based Netscape Navigator $ QDLOGSTATS WASD_LOGS:*.LOG /USERAGENT=*MOZILLA*X11* 3) Records of POST method requests $ QDLOGSTATS WASD_LOGS:*.LOG /METHOD=POST 4) Records requesting a particular path $ QDLOGSTATS WASD_LOGS:*.LOG /PATH="/cgi-bin/*" 5) Records with a query string $ QDLOGSTATS WASD_LOGS:*.LOG /QUERY="%*" $ QDLOGSTATS WASD_LOGS:*.LOG /QUERY="^^.+$" 6) Records without a query string (or without a specific query string) $ QDLOGSTATS WASD_LOGS:*.LOG /QUERY="^^$" $ QDLOGSTATS WASD_LOGS:*.LOG /QUERY="!*grotty*" $ QDLOGSTATS WASD_LOGS:*.LOG /QUERY="!^grotty" 7) Records requesting a particular path $ QDLOGSTATS WASD_LOGS:*.LOG /PATH="/cgi-bin/*" 8) Select proxy records requesting (a) particular site(s) $ QDLOGSTATS WASD_LOGS:*8080*.LOG /PATH="http://*.compaq.com*" $ QDLOGSTATS WASD_LOGS:*8080*.LOG /METHOD=POST /PATH="^http://.*sex.*/.*" 9) Records where the request was authenticated $ QDLOGSTATS WASD_LOGS:*.LOG /AUTHUSER="%*" $ QDLOGSTATS WASD_LOGS:*.LOG /AUTHUSER="^.+" $ QDLOGSTATS WASD_LOGS:*.LOG /AUTHUSER="DANIEL" 10) Records where the request was authenticated, excluding DANIEL $ QDLOGSTATS WASD_LOGS:*.LOG /AUTHUSER="!DANIEL" 11) Records with requests originating from (a) particular host(s) $ QDLOGSTATS WASD_LOGS:*.LOG /CLIENT=131.185.250.1 $ QDLOGSTATS WASD_LOGS:*.LOG /LOOKUP /CLIENT="*.vsm.com.au" 12) All requests between 26th September 2006 10:30 and midnight that same day $ QDLOGSTATS WASD_LOGS:*.LOG /DTSINCE=26-SEP-2006:10:30 - /DTBEFORE=26-SEP-2006:23:59 13) All requests during the previous day $ QDLOGSTATS WASD_LOGS:*.LOG /DTSINCE=YESTERDAY /DTBEFORE=TODAY CGI INTERFACE ------------- A CGI interface is available. It basically parallels the CLI behaviour described above. http://the.host.name/cgi-bin/qdlogstats So that it's not available for uncontrolled use the script *must* be subject to authorization (i.e. have a non-empty REMOTE_USER). For the WASD package this may be enabled using the following HTTPD$AUTH rule. [whatever-realm] /cgi-bin/qdlogstats r+w,~siteadmin For VMS Apache (CSWS) using SYSUAF authentication this might be configured using the following entries in [CONF]HTTPD.CONF LoadModule auth_openvms_module /apache$common/modules/mod_auth_openvms.exe_alpha AuthType Basic AuthName "OpenVMS authentication" AuthUserOpenVMS On require valid-user If there seems to be no relevant CGI variables the script displays a request form that allows the entering of strings against the various fields of the log file entries. This form uses the GET method (putting form elements into the query string) and so searches commonly or periodically made can be saved as bookmarks if desired. The account processing the script must have access to the log directory and files. This may be provided by the server account itself (i.e. the logs are normally accessable) or the scripting account itself has access. For instance, with WASD on VMS V6.2 or later this could be provided using the /PERSONA functionality and a HTTPD$MAP rule such as set /cgi-bin/qdlogstats script=as=SYSTEM An alternative is to install the script executable with sufficient privileges to access the files. Normally this would only be SYSPRV but as the script attempts to enable READALL as well so it *could* be installed with that if necessary. The following example DCL executed during server startup would ensure such access. $ INSTALL ADD /PRIVILEGE=READALL CGI-BIN:[000000]QDLOGSTATS.EXE For WASD 8.1 and later access control should not need to be changed. For non-WASD and WASD earlier than 8.1 the following should be applied. If this is provided it would also be prudent to control access to the executable from arbitrary accounts at the CLI, and restricting it to the scripting account using an ACL. For example $ SET SECURITY CGI-BIN:[000000]QDLOGSTATS.EXE - /ACL=((IDENT=HTTP$SERVER,ACCESS=R+E),(IDENT=*,ACCESS=NONE) The CGI version REQUIRES the following system-level logical name to locate the log files for it. If this name does not exist the CGI interface will not work. All log files must be available via this logical. The logical name is defined /SYSTEM to limit a possibly READALL privilege being used against arbitrary paths. $ DEFINE /SYSTEM QDLOGSTATS_LOGS "WASD_LOGS:" GEOLOCATION ----------- QdLogStats provides optional geolocation data, included between curly braces, following the client address/name. This is JavaScript enabled and performed by the client browser and so is only accessible with CGI usage. It also adds some latency to each lookup. The logical name QDLOGSTATS_GEOLOCATE provides a local URI or fully specified URL for a JavaScript resource (file) to perform the geolocation processing. This URI/URL is loaded into the browser when QdLogStats is activated. The file QDLOGSTATS_GEOLOCATION_GEOIPDB.JS provides a free service not requiring registration as well as a template for provding other services. Example basic geolocation is enabled using: $ DEFINE /SYSTEM QDLOGSTATS_GEOLOCATE - "/wasd_root/src/utils/qdlogstats_geolocate_geoipdb.js" QUALIFIERS ---------- /ALL compare all records to all filters /AUTHUSER= pattern (any authenticated username) /BEFORE= files (not records!) modified before VMS time /CLIENT= pattern (client host name or IP address) /DATETIME= pattern ("11/Jun/1999:14:08:49 +0930") /DBUG turns on all "if (Debug)" statements /DECODE[=keyword] URL-decode PATH, QUERY and/or REFERER before filtering /DTBEFORE= before date/time (VMS format) for filtering records /DTSINCE= since date/time (VMS format) for filtering records /IP= 4 or 6 to select either IPv4 or IPv6 records /LOG show the name of the log file the records come from /LOOKUP[=] lookup host name from literal address /METHOD= HTTP method ("GET", "POST", "HEAD", "(WebDAV)", etc.) /NOWASD remove likes of "POST /WASD::HTTPd:80-BEGIN-00000001" /OUTPUT= file specification /PATH= pattern (URL path component only) /PROGRESS[=integer] show progress during processing /PROTOCOL= HTTP protocol ("HTTP/2" or "HTTP/1.1", etc.) as a string /QUERY= pattern (URL query component only) /REFERER= pattern (HTTP "Referer:" field, COMBINED only) /REMOTEID= pattern (anything in the RFC819 remote ident field) /RESOLVE displays (name) if address and (address) if name /RESPONSE= HTTP response status code pattern /SINCE= files (not records!) modified since VMS time /SIZE=([MIN=|MAX=]) minimum and/or maximum response size /SOFTWAREID (and /VERSION) display QDLOGSTATS and CGILIB versions /USERAGENT= pattern (HTTP "User-Agent:", COMBINED only) /VIEW[=type] display matching records, where can be ALL, MATCH (default) or NOMATCH BUILD DETAILS ------------- See BUILD_QDLOGSTATS.COM procedure. COPYRIGHT --------- Copyright (C) 2000-2021 Mark G.Daniel Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Copyright (C) 1993 Free Software Foundation, Inc. Extended regular expression matching and search library, version 0.12. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. VERSION HISTORY (update SOFTWAREVN as well!) --------------- 09-MAY-2021 MGD v2.0.0, TcpIpLookup() refactor IP name/address resolution using getnameinfo() and getaddrinfo() FORM_RESOLVE and /RESOLVE displays (name) if address and (address) if name cosmetic refinements (perhaps just modifications) 18-MAY-2020 MGD v1.17.2, bugfix; protocol filter 18-JAN-2019 MGD v1.17.1, CgiForm() add HTTP status 418 to selector list 27-NOV-2018 MGD v1.17.0, refactor geolocation (see above for requirements) as well as general refactor and maintenance 09-NOV-2016 MGD v1.16.0, view (only) the first record for each client (allows close-enough assessment of unique access) JavaScript driven geolocation (with CGI interface) using the freegeoip.net or ip-api.com services HT_LOGS to WASD_LOGS (per WASD v10 and later :-) 06-SEP-2016 MGD v1.15.1, update to include HTTP/2 (of course this comes via the WASD 'RQ' logging directive, not standard) 04-JAN-2014 MGD v1.15.0, "interesting" response status codes option MATCHn() macros for improved efficiency text [pre-]wrap (more for printing) method and status percentages (of matched requests) bugfix; POST and PROP.. results transposition 10-AUG-2013 MGD v1.14.0, StatTimer() statistics 17-NOV-2011 MGD v1.13.3, 1nn response filter option 12-JUN-2010 MGD v1.13.2, bugfix; TcpIpLookupHostName() unresolved cache entry indicated by IP address with leading '?' 10-MAY-2010 MGD v1.13.1, NoWasdEntries test for "POST /*::WASD%:*-*-*" 01-NOV-2009 MGD v1.13.0, WebDAV methods view suspect records increase MAX_LINE_LENGTH to 16384 30-DEC-2008 MGD v1.12.0, bugfix; "error (no status)" response selector needs to be "0" not "000" (at least for WASD) 30-SEP-2006 MGD v1.11.0, access log date/time as since/before filter /DTBEFORE= for date/time filter /DTSINCE= for date/time filter selectors for log last modified and record date/time accumulate OPTIONS, TRACE and ? (unknown) methods 01-JUN-2005 MGD v1.10.3, update CgiForm() response statuses for HTTP/1.1, /NOWASD and equivalent checkbox, bugfix; date with leading space (ta Stuart Symonds) 10-MAY-2005 MGD v1.10.2, SWS 2.0 ignore query string components supplied as command-line parameters differently to CSWS 1.2/3 13-APR-2005 MGD v1.10.1, bugfix; missing assignments in GetParameters() 11-MAY-2004 MGD v1.10.0, IPv6 modifications /IP= to select either IPv4 or IPv6 records 23-DEC-2003 MGD v1.9.2, minor conditional mods to support IA64 02-DEC-2003 MGD v1.9.1, 206 partial content 12-AUG-2003 MGD v1.9.0, HTTP protocol filter 17-MAY-2003 MGD v1.8.0, use regular expression matching (GNU RX1.5 source) (this fundamentally changes some previous behaviours) allow for protocol in URL ("method path protocol") 02-JAN-2003 MGD v1.7.0, filtering on HTTP response status and response size 24-NOV-2002 MGD v1.6.0, URL-decoding path/query/referer before filtering, check for account SYSPRV before CLI activating 05-NOV-2002 MGD v1.5.1, extend CGI log search to check for QDLOGSTATS_LOGS then to fallback to HTTPD$LOG and HT_LOGS logicals 26-SEP-2002 MGD v1.5.0, make CGI log search dependent on the QDLOGSTATS_LOGS logical name being defined and log files within that 16-AUG-2002 MGD v1.4.0, add log file last modification before/since, some accomodations for CSWS 1.2, bugfix; silly boy - what about HEAD method! 30-MAR-2002 MGD v1.3.0, accumulate request methods 01-JUL-2001 MGD v1.2.2, add 'SkipParameters' for direct OSU support 30-JAN-2001 MGD v1.2.1, small improvement to lookup cache for noting addresses that never resolve (and do it slowly) 10-JAN-2001 MGD v1.2.0, lookup client host name 23-DEC-2000 MGD v1.1.0, CGI interface (still pretty q&d) 14-NOV-2000 MGD v1.0.0, initial development */ #endif /* COMMENTS_WITH_COMMENTS */ /*****************************************************************************/ #define COPYRIGHT_DATE "2000-2021" #define SOFTWAREVN "2.0.0" #define SOFTWARENM "QDLOGSTATS" #ifdef __ALPHA # define SOFTWAREID SOFTWARENM " AXP-" SOFTWAREVN #endif #ifdef __ia64 # define SOFTWAREID SOFTWARENM " IA64-" SOFTWAREVN #endif #ifdef __VAX # error VAX no longer implemented #endif #ifdef __x86_64 # define SOFTWAREID SOFTWARENM " X86-" SOFTWAREVN #endif /* standard C header files */ #include #include #include #include #include #include #include #include #include #include /* VMS-related header files */ #include #include #include #include #include #include #include #include #include #include #include #include /* require this for TCP/IP Services IPv6 */ #define _SOCKADDR_LEN /* BUT MultiNet BG driver does not support BSD 4.4 AF_INET addresses */ #define NO_SOCKADDR_LEN_4 /* TCP/IP-related header files */ #include #include #include /* application related header files */ #include #include /* regular expression processing */ #define REGEX_CHAR '^' /* compile flags, case-insensitive extended GREP */ #define REGEX_C_FLAGS (REG_EXTENDED | REG_ICASE) /* execution flags */ #define REGEX_E_FLAGS (0) /* one for each of the possible unique FilterThisOut()s */ #define REGEX_PATTERN_MAX 15 /* retry this many times (at a second interval) */ #define TCPIP_LOOKUP_HOST_NAME_RETRY 3 #define BOOL int #define true 1 #define false 0 typedef struct _IO_SB { unsigned short Status; unsigned short Count; unsigned long Unused; } IO_SB; #define VMSok(x) ((x) & STS$M_SUCCESS) #define VMSnok(x) (!((x) & STS$M_SUCCESS)) /* mainly to allow easy use of the __unaligned directive */ #define UINTPTR __unaligned unsigned int* #define ULONGPTR __unaligned unsigned long* #define USHORTPTR __unaligned unsigned short* #define INT64PTR __unaligned __int64* /* demonstrated to be an order of magnitude faster than memcmp() */ #define MATCH0(ptr1,ptr2,len) (memcmp(ptr1,ptr2,len) == 0) #define MATCH1(ptr1,ptr2) (*(char*)(ptr1) == *(char*)(ptr2)) #define MATCH2(ptr1,ptr2) (*(USHORTPTR)(ptr1) == *(USHORTPTR)(ptr2)) #define MATCH3(ptr1,ptr2) ((*(ULONGPTR)(ptr1) & 0x00ffffff) == \ (*(ULONGPTR)(ptr2) & 0x00ffffff)) #define MATCH4(ptr1,ptr2) (*(ULONGPTR)(ptr1) == *(ULONGPTR)(ptr2)) #define MATCH5(ptr1,ptr2) ((*(INT64PTR)(ptr1) & 0x000000ffffffffff) == \ (*(INT64PTR)(ptr2) & 0x000000ffffffffff)) #define MATCH6(ptr1,ptr2) ((*(INT64PTR)(ptr1) & 0x0000ffffffffffff) == \ (*(INT64PTR)(ptr2) & 0x0000ffffffffffff)) #define MATCH7(ptr1,ptr2) ((*(INT64PTR)(ptr1) & 0x00ffffffffffffff) == \ (*(INT64PTR)(ptr2) & 0x00ffffffffffffff)) #define MATCH8(ptr1,ptr2) (*(INT64PTR)(ptr1) == *(INT64PTR)(ptr2)) #define FI_LI "QDLOGSTATS",__LINE__ /* used in modulus for every so many records progress */ #define PROGRESS_RECORDS 1000 /* wrap every so many characters */ #define PROGRESS_WRAP_CLI 80 #define PROGRESS_WRAP_CGI 60 /* maximum log line/record length */ #define MAX_LINE_LENGTH 16384 #define DEFAULT_LOGS_LOGICAL "QDLOGSTATS_LOGS" #define DEFAULT_LOGS2_LOGICAL "HTTPD$LOG" #define DEFAULT_LOGS3_LOGICAL "WASD_LOGS" #define DEFAULT_LOGS_FILESPEC "*.LOG*;" #define DEFAULT_LOGS_DIR_APACHE "APACHE$SPECIFIC:[LOGS]" #define DEFAULT_LOGS_APACHE "*ACCESS_LOG*.;" #define DEFAULT_LOGS_DIR_OSU "WWW_ROOT:[SYSTEM]" #define DEFAULT_LOGS_OSU "ACCESS*.LOG;" #define DEFAULT_LOGS_DIR_WASD "WASD_LOGS:" #define DEFAULT_LOGS_WASD "*ACCESS*.LOG*;" #define VIEW_NONE 0 #define VIEW_LOG 1 #define VIEW_ALL 2 #define VIEW_MATCH 3 #define VIEW_NOMATCH 4 #define VIEW_SUSPECT 5 #define VIEW_UNIQUE 6 /* client */ char CopyrightInfo [] = "Copyright (C) " COPYRIGHT_DATE " Mark G.Daniel.\n\ This program, comes with ABSOLUTELY NO WARRANTY.\n\ This is free software, and you are welcome to redistribute it under the\n\ conditions of the GNU GENERAL PUBLIC LICENSE, version 3, or any later version.\n\ http://www.gnu.org/licenses/gpl.txt"; char CopyrightMeta [] = "Copyright (C) " COPYRIGHT_DATE " Mark G.Daniel - GPL licensed"; char Utility [] = "QDLOGSTATS"; BOOL Addr2Name, Debug, DoCgiStats, DoCliStats, FilterOnAll, GeoLocateForm, NoWasdEntries, ShowLogFile, UrlDecodePath, UrlDecodeQuery, UrlDecodeReferer; int AccessProblemCount, AuthUserFilterAcceptCount, AuthUserFilterRejectCount, ClientFilterAcceptCount, ClientFilterRejectCount, CombinedLogRecordCount, CommonLogRecordCount, DateTimeFilterRejectCount, DateTimeFilterAcceptCount, DateTimeProblemCount, DtBeforeFilterAcceptCount, DtBeforeFilterRejectCount, FileCountTotal, GeoLocateCodeLength, GeoLocateId, Ip4FilterAcceptCount, Ip4FilterRejectCount, Ip6FilterAcceptCount, Ip6FilterRejectCount, IpVersionFilter, LastModifiedIgnoredCount, LookupRetry, MethodFilterAcceptCount, MethodFilterRejectCount, MethodConnectCount, MethodConnectPercent, MethodCopyCount, MethodCopyPercent, MethodDeleteCount, MethodDeletePercent, MethodGetCount, MethodGetPercent, MethodHeadCount, MethodHeadPercent, MethodLockCount, MethodLockPercent, MethodMkColCount, MethodMkColPercent, MethodMoveCount, MethodMovePercent, MethodOptionsCount, MethodOptionsPercent, MethodPostCount, MethodPostPercent, MethodPropFindCount, MethodPropFindPercent, MethodPropPatchCount, MethodPropPatchPercent, MethodPutCount, MethodPutPercent, MethodTraceCount, MethodTracePercent, MethodUnknownCount, MethodUnknownPercent, MethodUnLockCount, MethodUnLockPercent, PathFilterAcceptCount, PathFilterRejectCount, ProgressCount, ProgressWrap, ProtocolFilterAcceptCount, ProtocolFilterRejectCount, QueryFilterAcceptCount, QueryFilterRejectCount, RecordCount, RecordCountTotal, SuspectRecordCountTotal, RequestCountTotal, RefererFilterAcceptCount, RefererFilterRejectCount, RemoteIdentFilterAcceptCount, RemoteIdentFilterRejectCount, ResponseMaxSize, ResponseMaxSizeAcceptCount, ResponseMaxSizeRejectCount, ResponseMinSize, ResponseMinSizeAcceptCount, ResponseMinSizeRejectCount, ShowProgress, DtSinceFilterAcceptCount, DtSinceFilterRejectCount, StatusFilterAcceptCount, StatusFilterRejectCount, UserAgentFilterAcceptCount, UserAgentFilterRejectCount, ViewRecords; int StatusCodeCount [6], StatusCodePercent [6]; unsigned long BinTime [2], DtBeforeFilterBinTime [2], DtSinceFilterBinTime [2], LmBeforeBinTime [2], LmSinceBinTime [2]; unsigned short NumTime [7]; float ByteCountTotal; char *AuthUserFilterPtr, *CgiFormLogFileSpecPtr, *CgiLibEnvironmentPtr, *CgiRequestUriPtr, *CgiScriptNamePtr, *CgiServerNamePtr, *ClientFilterPtr, *DateTimeFilterPtr, *GeoLocatePtr, *IpVersionFilterPtr, *MethodFilterPtr, *OutputPtr, *PathFilterPtr, *ProtocolFilterPtr, *QueryFilterPtr, *RegCompPatternPtr, *RefererFilterPtr, *RemoteIdentFilterPtr, *SearchDna, *ResponseFilterPtr, *UserAgentFilterPtr; char LogFileSpec [256], DtBeforeFilterString [32], DtSinceFilterString [32], LmBeforeString [32], LmSinceString [32], RegCompErrorString [64], SoftwareID [64], StartDateTime [32]; $DESCRIPTOR (DateTime20FaoDsc, "!20%D\0"); $DESCRIPTOR (DtBeforeFilterStringDsc, DtBeforeFilterString); $DESCRIPTOR (DtSinceFilterStringDsc, DtSinceFilterString); $DESCRIPTOR (FarEnoughBackDsc, "01-JAN-1970 00:00:00.00"); $DESCRIPTOR (FarEnoughForwardDsc, "31-DEC-2100 23:59:59.99"); $DESCRIPTOR (LmBeforeStringDsc, LmBeforeString); $DESCRIPTOR (LmSinceStringDsc, LmSinceString); $DESCRIPTOR (StartDateTimeFaoDsc, "!17%D\0"); $DESCRIPTOR (StartDateTimeDsc, StartDateTime); const uchar Ip4AddrZero [4], Ip6AddrZero [16]; #define INETACP$C_TRANS 2 #define INETACP_FUNC$C_GETHOSTBYNAME 1 #define INETACP_FUNC$C_GETHOSTBYADDR 2 $DESCRIPTOR (TcpIpDeviceDsc, "UCX$DEVICE"); /* required function prototypes */ void CgiForm (); void CgiStats (); void CliStats (); BOOL CompileFilters (); void DateTimeSelector (); void DateTimeCgi (); char* DateTimeFilter (char*); BOOL FilterBinDateTime (char*); BOOL FilterThisOut (char*, char*); int GetParameters (); void LogModifiedCgi (); void LogModifiedSelector (); void NeedsPrivilegedAccount (); void OutputLogRecord (char*); int ProcessLogFile (char*); int ProcessLogRecord (char*); int SearchFileSpec (char*); int ShowHelp (); int strsame (char*, char*, int); char* StatTimer (); char* SysGetMsg (int); char* SysTrnLnmSystem (char*); BOOL TcpIpIsAddress (char*); char* TcpIpLookup (char*, char*, uchar*, uchar*); BOOL UniqueClient (char*); /*****************************************************************************/ /* 'argc' and 'argv' are only required to support CgiLibEnvironmentInit(), in particular the OSU environment. */ main ( int argc, char *argv[] ) { /*********/ /* begin */ /*********/ sprintf (SoftwareID, "%s (%s)", SOFTWAREID, CGILIB_SOFTWAREID); sys$gettim (&BinTime); sys$numtim (&NumTime, &BinTime); /* always initialize the CGILIB so we can at least determine the mode */ if (getenv ("QDLOGSTATS$DBUG")) Debug = true; CgiLibEnvironmentSetDebug (Debug); if (Debug) fprintf (stdout, "Content-Type: text/plain\n\n"); CgiLibEnvironmentInit (argc, argv, false); GetParameters (); /* determine if we're at the command line or being used as a script */ if (!CgiLibVarNull ("WWW_GATEWAY_INTERFACE")) { DoCliStats = true; CliStats (); } else { DoCgiStats = true; CgiStats (); } exit (SS$_NORMAL); } /*****************************************************************************/ /* Generate command-line output containing the statistics. */ void CliStats () { int status; char *cptr; char AccessProblemString [64], AuthUserString [256], ByteString [256], ClientString [256], DateTimeString [256], DateTimeProblemString [64], IgnoredString [256], LastModifiedString [128], IpVersionString [128], MethodString [256], PathString [256], ProtocolString [256], QueryString [256], RefererString [256], RemoteIdentString [256], ResponseMaxSizeString [128], ResponseMinSizeString [128], ResponseString [128], UserAgentString [256]; unsigned long ResultBinTime [2]; /*********/ /* begin */ /*********/ NeedsPrivilegedAccount (); if (OutputPtr) if (!(stdout = freopen (OutputPtr, "w", stdout))) exit (vaxc$errno); if (!LogFileSpec[0]) { ShowHelp (); exit (SS$_NORMAL); } sys$fao (&StartDateTimeFaoDsc, NULL, &StartDateTimeDsc, 0); if (StartDateTime[0] == ' ') StartDateTime[0] = '0'; fprintf (stdout, "%s, %s\n", SoftwareID, StartDateTime); StatTimer (); if (LmBeforeBinTime[0]) { sys$fao (&DateTime20FaoDsc, NULL, &LmBeforeStringDsc, &LmBeforeBinTime); if (!LmSinceBinTime[0]) { /* create a time far enough in the past */ status = sys$bintim (&FarEnoughBackDsc, &LmSinceBinTime); if (VMSnok (status)) exit (status); } } if (LmSinceBinTime[0]) { sys$fao (&DateTime20FaoDsc, NULL, &LmSinceStringDsc, &LmSinceBinTime); if (!LmBeforeBinTime[0]) { /* create a time far enough in the future */ status = sys$bintim (&FarEnoughForwardDsc, &LmBeforeBinTime); if (VMSnok (status)) exit (status); } } if (LmSinceBinTime[0] || LmBeforeBinTime[0]) { /* sanity check on the filter window */ status = lib$sub_times (&LmBeforeBinTime, &LmSinceBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) exit (LIB$_NEGTIM); } if (LmSinceString[0] || LmBeforeString[0]) { LastModifiedString[0] = '\0'; if (LmSinceString[0]) sprintf (LastModifiedString, "since %s", LmSinceString); if (LmBeforeString[0]) { for (cptr = LastModifiedString; *cptr; cptr++); sprintf (cptr, "%sbefore %s", LastModifiedString[0] ? "\n" : "", LmBeforeString); } } else strcpy (LastModifiedString, ""); if (DtBeforeFilterBinTime[0]) { sys$fao (&DateTime20FaoDsc, NULL, &DtBeforeFilterStringDsc, &DtBeforeFilterBinTime); if (!DtSinceFilterBinTime[0]) { /* create a time far enough in the past */ status = sys$bintim (&FarEnoughBackDsc, &DtSinceFilterBinTime); if (VMSnok (status)) exit (status); } } if (DtSinceFilterBinTime[0]) { sys$fao (&DateTime20FaoDsc, NULL, &DtSinceFilterStringDsc, &DtSinceFilterBinTime); if (!DtBeforeFilterBinTime[0]) { /* create a time far enough in the future */ status = sys$bintim (&FarEnoughForwardDsc, &DtBeforeFilterBinTime); if (VMSnok (status)) exit (status); } } if (DtSinceFilterBinTime[0] || DtBeforeFilterBinTime[0]) { /* sanity check on the filter window */ status = lib$sub_times (&DtBeforeFilterBinTime, &DtSinceFilterBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) exit (LIB$_NEGTIM); } if (!CompileFilters ()) { fprintf (stdout, "%%%s-E-REGEX, expression \'%s\' has %s", Utility, RegCompPatternPtr, RegCompErrorString); exit (STS$K_ERROR | STS$M_INHIB_MSG); } ProgressWrap = PROGRESS_WRAP_CLI; /* make a guess at the log file defaults */ if (getenv("WASD_ROOT") || getenv("HT_ROOT")) { SearchDna = DEFAULT_LOGS_DIR_WASD; cptr = DEFAULT_LOGS_WASD; } else if (getenv("WWW_ROOT")) { SearchDna = DEFAULT_LOGS_DIR_OSU ; cptr = DEFAULT_LOGS_OSU; } else if (getenv("APACHE$SPECIFIC")) { SearchDna = DEFAULT_LOGS_DIR_APACHE; cptr = DEFAULT_LOGS_APACHE; } else { SearchDna = DEFAULT_LOGS_FILESPEC; cptr = DEFAULT_LOGS_FILESPEC; } if (LogFileSpec[0]) cptr = LogFileSpec; status = SearchFileSpec (cptr); if (ShowProgress && FileCountTotal) fputs ("\n", stdout); if (VMSnok (status) && status != RMS$_NMF) exit (status); if (AccessProblemCount) sprintf (AccessProblemString, ", %d access failed", AccessProblemCount); else AccessProblemString[0] = '\0'; if (LastModifiedIgnoredCount) sprintf (IgnoredString, ", %d ignored", LastModifiedIgnoredCount); else IgnoredString[0] = '\0'; if (ByteCountTotal > 1000000000.0) { ByteCountTotal /= 1000000000.0; sprintf (ByteString, "GBytes ....... %.3f", ByteCountTotal); } else if (ByteCountTotal > 1000000.0) { ByteCountTotal /= 1000000.0; sprintf (ByteString, "MBytes ........ %.3f", ByteCountTotal); } else if (ByteCountTotal > 1000.0) { ByteCountTotal /= 1000.0; sprintf (ByteString, "KBytes ........ %.3f", ByteCountTotal); } else sprintf (ByteString, "Bytes ......... %.0f", ByteCountTotal); if (!ResponseMinSize) strcpy (ResponseMinSizeString, ""); else if (ResponseMinSize < 1000) sprintf (ResponseMinSizeString, "%d", ResponseMinSize); else if (ResponseMinSize < 1000000) sprintf (ResponseMinSizeString, "%dk", ResponseMinSize / 1000); else sprintf (ResponseMinSizeString, "%dM", ResponseMinSize / 1000000); if (!ResponseMaxSize) strcpy (ResponseMaxSizeString, ""); else if (ResponseMaxSize < 1000) sprintf (ResponseMaxSizeString, "%d", ResponseMaxSize); else if (ResponseMaxSize < 1000000) sprintf (ResponseMaxSizeString, "%dk", ResponseMaxSize / 1000); else sprintf (ResponseMaxSizeString, "%dM", ResponseMaxSize / 1000000); if (FilterOnAll) { if (!ClientFilterPtr) strcpy (ClientString, ""); else sprintf (ClientString, "%s (%d nomatch, %d match)", ClientFilterPtr, ClientFilterRejectCount, ClientFilterAcceptCount); if (!IpVersionFilter) strcpy (IpVersionString, ""); else if (IpVersionFilter == 4) sprintf (IpVersionString, "4 (%d nomatch, %d match)", Ip4FilterRejectCount, Ip4FilterAcceptCount); else if (IpVersionFilter == 6) sprintf (IpVersionString, "6 (%d nomatch, %d match)", Ip6FilterRejectCount, Ip6FilterAcceptCount); if (!AuthUserFilterPtr) strcpy (AuthUserString, ""); else sprintf (AuthUserString, "%s (%d nomatch, %d match)", AuthUserFilterPtr, AuthUserFilterRejectCount, AuthUserFilterAcceptCount); if (DtSinceFilterString[0] || DtBeforeFilterString[0]) { DateTimeString[0] = '\0'; if (DtSinceFilterString[0]) sprintf (DateTimeString, "%ssince %s (%d nomatch, %d match)", DtBeforeFilterString[0] ? " " : "", DtSinceFilterString, DtSinceFilterRejectCount, DtSinceFilterAcceptCount); if (DtBeforeFilterString[0]) { if (DateTimeString[0]) strcat (DateTimeString, "\n "); for (cptr = DateTimeString; *cptr; cptr++); sprintf (cptr, "before %s (%d nomatch, %d match)", DtBeforeFilterString, DtBeforeFilterRejectCount, DtBeforeFilterAcceptCount); } } else if (!DateTimeFilterPtr) strcpy (DateTimeString, ""); else sprintf (DateTimeString, "%s (%d nomatch, %d match)", DateTimeFilterPtr, DateTimeFilterRejectCount, DateTimeFilterAcceptCount); if (!MethodFilterPtr) strcpy (MethodString, ""); else sprintf (MethodString, "%s (%d nomatch, %d match)", MethodFilterPtr, MethodFilterRejectCount, MethodFilterAcceptCount); if (!PathFilterPtr) strcpy (PathString, ""); else sprintf (PathString, "%s (%d nomatch, %d match)", PathFilterPtr, PathFilterRejectCount, PathFilterAcceptCount); if (!QueryFilterPtr) strcpy (QueryString, ""); else sprintf (QueryString, " (%d nomatch, %d match)", QueryFilterRejectCount, QueryFilterAcceptCount); if (!RefererFilterPtr) strcpy (RefererString, ""); else sprintf (RefererString, "%s (%d nomatch, %d match)", RefererFilterPtr, RefererFilterRejectCount, RefererFilterAcceptCount); if (!ProtocolFilterPtr) strcpy (ProtocolString, ""); else sprintf (ProtocolString, " (%d nomatch, %d match)", ProtocolFilterRejectCount, ProtocolFilterAcceptCount); if (!RefererFilterPtr) strcpy (RefererString, ""); else sprintf (RefererString, "%s (%d nomatch, %d match)", RefererFilterPtr, RefererFilterRejectCount, RefererFilterAcceptCount); if (!RemoteIdentFilterPtr) strcpy (RemoteIdentString, ""); else sprintf (RemoteIdentString, "%s (%d nomatch, %d match)", RemoteIdentFilterPtr, RemoteIdentFilterRejectCount, RemoteIdentFilterAcceptCount); if (!UserAgentFilterPtr) strcpy (UserAgentString, ""); else sprintf (UserAgentString, "%s (%d nomatch, %d match)", UserAgentFilterPtr, UserAgentFilterRejectCount, UserAgentFilterAcceptCount); if (!ResponseFilterPtr) strcpy (ResponseString, ""); else sprintf (ResponseString, "%s (%d nomatch, %d match)", ResponseFilterPtr, StatusFilterRejectCount, StatusFilterAcceptCount); if (ResponseMinSize) { for (cptr = ResponseMinSizeString; *cptr; cptr++); sprintf (cptr, " (%d nomatch, %d match)", ResponseMinSizeRejectCount, ResponseMinSizeAcceptCount); } if (ResponseMaxSize) { for (cptr = ResponseMaxSizeString; *cptr; cptr++); sprintf (cptr, " (%d nomatch, %d match)", ResponseMaxSizeRejectCount, ResponseMaxSizeAcceptCount); } } else { if (!ClientFilterPtr) strcpy (ClientString, ""); else sprintf (ClientString, "%s (%d nomatch)", ClientFilterPtr, ClientFilterRejectCount); if (!IpVersionFilter) strcpy (IpVersionString, ""); else if (IpVersionFilter == 4) sprintf (IpVersionString, "4 (%d nomatch)", Ip4FilterRejectCount); else if (IpVersionFilter == 6) sprintf (IpVersionString, "6 (%d nomatch)", Ip6FilterRejectCount); if (!AuthUserFilterPtr) strcpy (AuthUserString, ""); else sprintf (AuthUserString, "%s (%d nomatch)", AuthUserFilterPtr, AuthUserFilterRejectCount); if (DtSinceFilterString[0] || DtBeforeFilterString[0]) { DateTimeString[0] = '\0'; if (DtSinceFilterString[0]) sprintf (DateTimeString, "%ssince %s (%d nomatch)", DtBeforeFilterString[0] ? " " : "", DtSinceFilterString, DtSinceFilterRejectCount); if (DtBeforeFilterString[0]) { if (DateTimeString[0]) strcat (DateTimeString, "\n "); for (cptr = DateTimeString; *cptr; cptr++); sprintf (cptr, "before %s (%d nomatch)", DtBeforeFilterString, DtBeforeFilterRejectCount); } } else if (!DateTimeFilterPtr) strcpy (DateTimeString, ""); else sprintf (DateTimeString, "%s (%d nomatch)", DateTimeFilterPtr, DateTimeFilterRejectCount); if (!MethodFilterPtr) strcpy (MethodString, ""); else sprintf (MethodString, "%s (%d nomatch)", MethodFilterPtr, MethodFilterRejectCount); if (!PathFilterPtr) strcpy (PathString, ""); else sprintf (PathString, "%s (%d nomatch)", PathFilterPtr, PathFilterRejectCount); if (!QueryFilterPtr) strcpy (QueryString, ""); else sprintf (QueryString, "%s (%d nomatch)", QueryFilterPtr, QueryFilterRejectCount); if (!ProtocolFilterPtr) strcpy (ProtocolString, ""); else sprintf (ProtocolString, "%s (%d nomatch)", ProtocolFilterPtr, ProtocolFilterRejectCount); if (!RefererFilterPtr) strcpy (RefererString, ""); else sprintf (RefererString, "%s (%d nomatch)", RefererFilterPtr, RefererFilterRejectCount); if (!RemoteIdentFilterPtr) strcpy (RemoteIdentString, ""); else sprintf (RemoteIdentString, "%s (%d nomatch)", RemoteIdentFilterPtr, RemoteIdentFilterRejectCount); if (!UserAgentFilterPtr) strcpy (UserAgentString, ""); else sprintf (UserAgentString, "%s (%d nomatch)", UserAgentFilterPtr, UserAgentFilterRejectCount); if (!ResponseFilterPtr) strcpy (ResponseString, ""); else sprintf (ResponseString, "%s (%d nomatch)", ResponseFilterPtr, StatusFilterRejectCount); if (ResponseMinSize) { for (cptr = ResponseMinSizeString; *cptr; cptr++); sprintf (cptr, " (%d nomatch)", ResponseMinSizeRejectCount); } if (ResponseMaxSize) { for (cptr = ResponseMaxSizeString; *cptr; cptr++); sprintf (cptr, " (%d nomatch)", ResponseMaxSizeRejectCount); } } if (DateTimeProblemCount) sprintf (DateTimeProblemString, " (%d format problems)", DateTimeProblemCount); else DateTimeProblemString[0] = '\0'; fprintf (stdout, "\n\ Log ........... %s (%d matched%s%s)\n\ Modified ...... %s\n\ Client ........ %s\n\ IP Version .... %s\n\ Remote ID ..... %s\n\ Auth User ..... %s\n\ Date/Time ..... %s%s\n\ Method ........ %s\n\ Path .......... %s\n\ Query ......... %s\n\ Protocol ...... %s\n\ Referer ....... %s\n\ User Agent .... %s\n\ \n\ Response ...... %s\n\ Size Min ...... %s\n\ Size Max ...... %s\n\ \n\ Statistics .... %s\n\ Records ....... %d (%d suspect, %d common, %d combined)\n\ Requests ...... %d\n\ Methods ....... CONNECT:%d (%d%%) COPY:%d (%d%%) DELETE:%d (%d%%) \ GET:%d (%d%%) HEAD:%d (%d%%) LOCK:%d (%d%%) MKCOL:%d (%d%%) \ MOVE:%d (%d%%) OPTIONS:%d (%d%%) POST:%d (%d%%) PROPFIND:%d (%d%%) \ POPPATCH:%d (%d%%) PUT:%d (%d%%) TRACE:%d (%d%%) UNLOCK:%d (%d%%) \ ?:%d (%d%%) \n\ Responses ..... 1nn:%d (%d%%) 2nn:%d (%d%%) 3nn:%d (%d%%) 4nn:%d (%d%%) \ 5nn:%d (%d%%) ?:%d (%d%%) \n\ %s\n", LogFileSpec, FileCountTotal, IgnoredString, AccessProblemString, LastModifiedString, ClientString, IpVersionString, RemoteIdentString, AuthUserString, DateTimeString, DateTimeProblemString, MethodString, PathString, QueryString, ProtocolString, RefererString, UserAgentString, ResponseString, ResponseMinSizeString, ResponseMaxSizeString, StatTimer(), RecordCountTotal, SuspectRecordCountTotal, CommonLogRecordCount, CombinedLogRecordCount, RequestCountTotal, MethodConnectCount, MethodConnectPercent, MethodCopyCount, MethodCopyPercent, MethodDeleteCount, MethodDeletePercent, MethodGetCount, MethodGetPercent, MethodHeadCount, MethodHeadPercent, MethodLockCount, MethodLockPercent, MethodMkColCount, MethodMkColPercent, MethodMoveCount, MethodMovePercent, MethodOptionsCount, MethodOptionsPercent, MethodPostCount, MethodPostPercent, MethodPropFindCount, MethodPropFindPercent, MethodPropPatchCount, MethodPropPatchPercent, MethodPutCount, MethodPutPercent, MethodTraceCount, MethodTracePercent, MethodUnLockCount, MethodUnLockPercent, MethodUnknownCount, MethodUnknownPercent, StatusCodeCount[1], StatusCodePercent[1], StatusCodeCount[2], StatusCodePercent[2], StatusCodeCount[3], StatusCodePercent[3], StatusCodeCount[4], StatusCodePercent[4], StatusCodeCount[5], StatusCodePercent[5], StatusCodeCount[0], StatusCodePercent[0], ByteString); } /*****************************************************************************/ /* Generate a CGI response containing an HTML page of statistics. */ void CgiStats () { #ifndef PRV$M_READALL #define PRV$M_READALL 0x0008 #endif static unsigned long PrvMask [2] = { (unsigned long)PRV$M_SYSPRV | (unsigned long)PRV$M_READALL, 0 }; BOOL ShowProgressOnPage; int status; char *cptr, *ByteStringPtr; char AccessProblemString [64], AuthUserString [256], ByteString [16], ClientString [256], DateTimeString [256], DateTimeProblemString [64], IgnoredString [256], IpVersionString [128], LastModifiedString [128], MethodString [256], PathString [256], ProtocolString [256], QueryString [256], RefererString [256], RemoteIdentString [256], ResponseMaxSizeString [128], ResponseMinSizeString [128], ResponseString [128], UserAgentString [256], VmsMessage [512]; /*********/ /* begin */ /*********/ /* ensure one of the required log directory logicals is defined */ if (!(SearchDna = SysTrnLnmSystem(DEFAULT_LOGS_LOGICAL))) if (!(SearchDna = SysTrnLnmSystem(DEFAULT_LOGS2_LOGICAL))) if (!(SearchDna = SysTrnLnmSystem(DEFAULT_LOGS3_LOGICAL))) { CgiLibResponseError (FI_LI, 0, "Logical name " DEFAULT_LOGS_LOGICAL " is not defined."); return; } sys$fao (&StartDateTimeFaoDsc, NULL, &StartDateTimeDsc, 0); if (StartDateTime[0] == ' ') StartDateTime[0] = '0'; StatTimer(); cptr = CgiLibVar ("WWW_REMOTE_USER"); if (!cptr[0]) { CgiLibResponseError (FI_LI, 0, "Authorization mandatory."); return; } CgiLibEnvironmentRecordOut (); CgiLibEnvironmentPtr = CgiLibEnvironmentName(); CgiServerNamePtr = CgiLibVar ("WWW_SERVER_NAME"); CgiFormLogFileSpecPtr = CgiLibVarNull ("WWW_FORM_LOGFILESPEC"); if (!CgiFormLogFileSpecPtr) { CgiForm (); return; } LogModifiedCgi (); if (LmSinceString[0] || LmBeforeString[0]) { LastModifiedString[0] = '\0'; if (LmSinceString[0]) sprintf (LastModifiedString, "since %s", LmSinceString); if (LmBeforeString[0]) { for (cptr = LastModifiedString; *cptr; cptr++); sprintf (cptr, "%sbefore %s", LastModifiedString[0] ? "
" : "", LmBeforeString); } } else strcpy (LastModifiedString, "<none>"); /* get the filter components from the form */ ClientFilterPtr = CgiLibVar ("WWW_FORM_CLIENT"); if (!ClientFilterPtr[0]) ClientFilterPtr = NULL; IpVersionFilterPtr = CgiLibVar ("WWW_FORM_IPV"); if (*IpVersionFilterPtr == '4') IpVersionFilter = 4; else if (*IpVersionFilterPtr == '6') IpVersionFilter = 6; else IpVersionFilter = 0; RemoteIdentFilterPtr = CgiLibVar ("WWW_FORM_REMOTEID"); if (!RemoteIdentFilterPtr[0]) RemoteIdentFilterPtr = NULL; AuthUserFilterPtr = CgiLibVar ("WWW_FORM_AUTHUSER"); if (!AuthUserFilterPtr[0]) AuthUserFilterPtr = NULL; DateTimeCgi (); MethodFilterPtr = CgiLibVar ("WWW_FORM_METHOD"); if (!MethodFilterPtr[0]) MethodFilterPtr = CgiLibVar ("WWW_FORM_METHODLIST"); if (!MethodFilterPtr[0]) MethodFilterPtr = NULL; PathFilterPtr = CgiLibVar ("WWW_FORM_PATH"); if (!PathFilterPtr[0]) PathFilterPtr = NULL; cptr = CgiLibVar ("WWW_FORM_DECODEPATH"); if (cptr[0]) UrlDecodePath = true; else UrlDecodePath = false; ProtocolFilterPtr = CgiLibVar ("WWW_FORM_PROTOCOL"); if (!ProtocolFilterPtr[0]) ProtocolFilterPtr = CgiLibVar ("WWW_FORM_PROTOCOLLIST"); if (!ProtocolFilterPtr[0]) ProtocolFilterPtr = NULL; QueryFilterPtr = CgiLibVar ("WWW_FORM_QUERY"); if (!QueryFilterPtr[0]) QueryFilterPtr = NULL; cptr = CgiLibVar ("WWW_FORM_DECODEQUERY"); if (cptr[0]) UrlDecodeQuery = true; else UrlDecodeQuery = false; RefererFilterPtr = CgiLibVar ("WWW_FORM_REFERER"); if (!RefererFilterPtr[0]) RefererFilterPtr = NULL; cptr = CgiLibVar ("WWW_FORM_DECODEREFERER"); if (cptr[0]) UrlDecodeReferer = true; else UrlDecodeReferer = false; UserAgentFilterPtr = CgiLibVar ("WWW_FORM_USERAGENT"); if (!UserAgentFilterPtr[0]) UserAgentFilterPtr = NULL; ResponseFilterPtr = CgiLibVar ("WWW_FORM_RESPONSE"); if (!ResponseFilterPtr[0]) ResponseFilterPtr = CgiLibVar ("WWW_FORM_RESPONSELIST"); if (!ResponseFilterPtr[0]) ResponseFilterPtr = NULL; cptr = CgiLibVar ("WWW_FORM_VIEW"); ShowLogFile = ShowProgressOnPage = false; if (strsame (cptr, "ALL", -1)) ViewRecords = VIEW_ALL; else if (strsame (cptr, "ALL-LOG", -1)) { ViewRecords = VIEW_ALL; ShowLogFile = true; } else if (strsame (cptr, "LOG", -1)) { ViewRecords = VIEW_LOG; ShowLogFile = true; } else if (strsame (cptr, "MATCH", -1)) ViewRecords = VIEW_MATCH; else if (strsame (cptr, "MATCH-LOG", -1)) { ViewRecords = VIEW_MATCH; ShowLogFile = true; } else if (strsame (cptr, "NOMATCH", -1)) ViewRecords = VIEW_NOMATCH; else if (strsame (cptr, "NOMATCH-LOG", -1)) { ViewRecords = VIEW_NOMATCH; ShowLogFile = true; } else if (strsame (cptr, "PROGRESS", -1)) ShowProgressOnPage = true; else if (strsame (cptr, "SUSPECT", -1)) { ViewRecords = VIEW_SUSPECT; ShowLogFile = true; } else if (strsame (cptr, "UNIQUE", -1)) { ViewRecords = VIEW_UNIQUE; ShowLogFile = true; } cptr = CgiLibVar ("WWW_FORM_ALL"); if (cptr[0]) FilterOnAll = true; cptr = CgiLibVar ("WWW_FORM_REMWASD"); if (cptr[0]) NoWasdEntries = true; cptr = CgiLibVar ("WWW_FORM_RESOLVE"); if (cptr[0]) Addr2Name = true; cptr = CgiLibVar ("WWW_FORM_LOOKUP"); if (isdigit(cptr[0])) LookupRetry = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_MINSIZE"); if (!cptr[0]) cptr = CgiLibVar ("WWW_FORM_MINSIZELIST"); ResponseMinSize = atoi(cptr); if (isdigit(*cptr)) { while (isdigit(*cptr)) cptr++; if (tolower(*cptr) == 'k') ResponseMinSize *= 1000; if (toupper(*cptr) == 'M') ResponseMinSize *= 1000000 ; } if (!ResponseMinSize) strcpy (ResponseMinSizeString, "<none>"); else if (ResponseMinSize < 1000) sprintf (ResponseMinSizeString, "%d", ResponseMinSize); else if (ResponseMinSize < 1000000) sprintf (ResponseMinSizeString, "%dk", ResponseMinSize / 1000); else sprintf (ResponseMinSizeString, "%dM", ResponseMinSize / 1000000); cptr = CgiLibVar ("WWW_FORM_MAXSIZE"); if (!cptr[0]) cptr = CgiLibVar ("WWW_FORM_MAXSIZELIST"); ResponseMaxSize = atoi(cptr); if (isdigit(*cptr)) { while (isdigit(*cptr)) cptr++; if (tolower(*cptr) == 'k') ResponseMaxSize *= 1000; if (toupper(*cptr) == 'M') ResponseMaxSize *= 1000000; } if (!ResponseMaxSize) strcpy (ResponseMaxSizeString, "<none>"); else if (ResponseMaxSize < 1000) sprintf (ResponseMaxSizeString, "%d", ResponseMaxSize); else if (ResponseMaxSize < 1000000) sprintf (ResponseMaxSizeString, "%dk", ResponseMaxSize / 1000); else sprintf (ResponseMaxSizeString, "%dM", ResponseMaxSize / 1000000); if (!CompileFilters ()) { CgiLibResponseError (FI_LI, 0, "The regular expression  %s  has %s.", (char*)CgiLibHtmlEscape(RegCompPatternPtr, -1, NULL, 0), RegCompErrorString); return; } if (cptr = CgiLibVarNull ("WWW_FORM_GEOLOCATE")) { if (*cptr) { GeoLocateForm = true; if (GeoLocateForm) GeoLocatePtr = SysTrnLnmSystem("QDLOGSTATS_GEOLOCATE"); } } /* WASD stream mode will stop each fflush() having carriage control added */ CgiLibEnvironmentBinaryOut (); CgiLibResponseHeader (200, "text/html", "Script-Control: X-stream-mode\n"); fprintf (stdout, "\n\ \n\ \n\ \n\ \n\ \n\ \n\ qdLogStats@%s\n\ \n\ \n\ %s%s%s\ \n\ \n\ qdLogStats@%s
\n\    %s
\n\    %s\n", SoftwareID, CopyrightMeta, CgiLibEnvironmentPtr, SearchDna, CgiServerNamePtr, GeoLocatePtr ? "\n" : "" , CgiServerNamePtr, SOFTWAREID, StartDateTime); if (ViewRecords) fputs ("


\
", stdout);
   else
   {
      /* the CGI interface always provides hidden progress indication */
      ShowProgress = PROGRESS_RECORDS;
      if (ShowProgressOnPage)
         fputs ("
", stdout);
      else
         fputs ("\n", stdout);

   if (VMSnok (status) && status != RMS$_NMF && status != RMS$_FNF)
      sprintf (VmsMessage,
"error: \
%s\n\
\n",
               SysGetMsg(status), status);
   else
      VmsMessage[0] = '\0';

   if (AccessProblemCount)
      sprintf (AccessProblemString, ", %d access failed", AccessProblemCount);
   else
      AccessProblemString[0] = '\0';

   if (LastModifiedIgnoredCount)
      sprintf (IgnoredString, ", %d ignored", LastModifiedIgnoredCount);
   else
      IgnoredString[0] = '\0';

   if (ByteCountTotal > 1000000000.0)
   {
      ByteCountTotal /= 1000000000.0;
      ByteStringPtr = "GBytes";
      sprintf (ByteString, "%.3f", ByteCountTotal);
   }
   else
   if (ByteCountTotal > 1000000.0)
   {
      ByteCountTotal /= 1000000.0;
      ByteStringPtr = "MBytes";
      sprintf (ByteString, "%.3f", ByteCountTotal);
   }
   else
   if (ByteCountTotal > 1000.0)
   {
      ByteCountTotal /= 1000.0;
      ByteStringPtr = "KBytes";
      sprintf (ByteString, "%.3f", ByteCountTotal);
   }
   else
   {
      ByteStringPtr = "Bytes";
      sprintf (ByteString, "%.0f", ByteCountTotal);
   }

   if (FilterOnAll)
   {
      if (!ClientFilterPtr)
         strcpy (ClientString, "<none>");
      else
         sprintf (ClientString, ""%s"  (%d nomatch, %d match)",
                  (char*)CgiLibHtmlEscape (ClientFilterPtr, -1, NULL, 0),
                  ClientFilterRejectCount, ClientFilterAcceptCount);
      if (!IpVersionFilter)
         strcpy (IpVersionString, "<none>");
      else
      if (IpVersionFilter == 4)
         sprintf (IpVersionString, "4  (%d nomatch, %d match)",
                  Ip4FilterRejectCount, Ip4FilterAcceptCount);
      else
      if (IpVersionFilter == 6)
         sprintf (IpVersionString, "6  (%d nomatch, %d match)",
                  Ip6FilterRejectCount, Ip6FilterAcceptCount);
      if (!AuthUserFilterPtr)
         strcpy (AuthUserString, "<none>");
      else
         sprintf (AuthUserString, ""%s"  (%d nomatch, %d match)",
                  (char*)CgiLibHtmlEscape (AuthUserFilterPtr, -1, NULL, 0),
                  AuthUserFilterRejectCount, AuthUserFilterAcceptCount);

      if (DtSinceFilterString[0] || DtBeforeFilterString[0])
      {
         DateTimeString[0] = '\0';
         if (DtSinceFilterString[0])
            sprintf (DateTimeString, "%ssince %s  (%d nomatch, %d match)",
                     DtBeforeFilterString[0] ? "  " : "",
                     DtSinceFilterString, DtSinceFilterRejectCount,
                                          DtSinceFilterAcceptCount);
         if (DtBeforeFilterString[0])
         {
            if (DateTimeString[0]) strcat (DateTimeString, "
"); for (cptr = DateTimeString; *cptr; cptr++); sprintf (cptr, "before %s (%d nomatch, %d match)", DtBeforeFilterString, DtBeforeFilterRejectCount, DtBeforeFilterAcceptCount); } } else strcpy (DateTimeString, "<none>"); if (!MethodFilterPtr) strcpy (MethodString, "<none>"); else sprintf (MethodString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (MethodFilterPtr, -1, NULL, 0), MethodFilterRejectCount, MethodFilterAcceptCount); if (!PathFilterPtr) strcpy (PathString, "<none>"); else sprintf (PathString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (PathFilterPtr, -1, NULL, 0), PathFilterRejectCount, PathFilterAcceptCount); if (!QueryFilterPtr) strcpy (QueryString, "<none>"); else sprintf (QueryString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (QueryFilterPtr, -1, NULL, 0), QueryFilterRejectCount, QueryFilterAcceptCount); if (!ProtocolFilterPtr) strcpy (ProtocolString, "<none>"); else sprintf (ProtocolString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (ProtocolFilterPtr, -1, NULL, 0), ProtocolFilterRejectCount, ProtocolFilterAcceptCount); if (!RefererFilterPtr) strcpy (RefererString, "<none>"); else sprintf (RefererString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (RefererFilterPtr, -1, NULL, 0), RefererFilterRejectCount, RefererFilterAcceptCount); if (!RemoteIdentFilterPtr) strcpy (RemoteIdentString, "<none>"); else sprintf (RemoteIdentString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (RemoteIdentFilterPtr, -1, NULL, 0), RemoteIdentFilterRejectCount, RemoteIdentFilterAcceptCount); if (!UserAgentFilterPtr) strcpy (UserAgentString, "<none>"); else sprintf (UserAgentString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (UserAgentFilterPtr, -1, NULL, 0), UserAgentFilterRejectCount, UserAgentFilterAcceptCount); if (!ResponseFilterPtr) strcpy (ResponseString, "<all>"); else sprintf (ResponseString, ""%s"  (%d nomatch, %d match)", (char*)CgiLibHtmlEscape (ResponseFilterPtr, -1, NULL, 0), StatusFilterRejectCount, StatusFilterAcceptCount); if (ResponseMinSize) { for (cptr = ResponseMinSizeString; *cptr; cptr++); sprintf (cptr, "  (%d nomatch, %d match)", ResponseMinSizeRejectCount, ResponseMinSizeAcceptCount); } if (ResponseMaxSize) { for (cptr = ResponseMaxSizeString; *cptr; cptr++); sprintf (cptr, "  (%d nomatch, %d match)", ResponseMaxSizeRejectCount, ResponseMaxSizeAcceptCount); } } else { if (!ClientFilterPtr) strcpy (ClientString, "<none>"); else sprintf (ClientString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (ClientFilterPtr, -1, NULL, 0), ClientFilterRejectCount); if (!IpVersionFilter) strcpy (IpVersionString, "<none>"); else if (IpVersionFilter == 4) sprintf (IpVersionString, "4 (%d nomatch)", Ip4FilterRejectCount); else if (IpVersionFilter == 6) sprintf (IpVersionString, "6 (%d nomatch)", Ip6FilterRejectCount); if (!AuthUserFilterPtr) strcpy (AuthUserString, "<none>"); else sprintf (AuthUserString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (AuthUserFilterPtr, -1, NULL, 0), AuthUserFilterRejectCount); if (DtSinceFilterString[0] || DtBeforeFilterString[0]) { DateTimeString[0] = '\0'; if (DtSinceFilterString[0]) sprintf (DateTimeString, "%ssince %s (%d nomatch)", DtBeforeFilterString[0] ? "  " : "", DtSinceFilterString, DtSinceFilterRejectCount); if (DtBeforeFilterString[0]) { if (DateTimeString[0]) strcat (DateTimeString, "
"); for (cptr = DateTimeString; *cptr; cptr++); sprintf (cptr, "before %s (%d nomatch)", DtBeforeFilterString, DtBeforeFilterRejectCount); } } else strcpy (DateTimeString, "<none>"); if (!MethodFilterPtr) strcpy (MethodString, "<none>"); else sprintf (MethodString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (MethodFilterPtr, -1, NULL, 0), MethodFilterRejectCount); if (!PathFilterPtr) strcpy (PathString, "<none>"); else sprintf (PathString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (PathFilterPtr, -1, NULL, 0), PathFilterRejectCount); if (!QueryFilterPtr) strcpy (QueryString, "<none>"); else sprintf (QueryString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (QueryFilterPtr, -1, NULL, 0), QueryFilterRejectCount); if (!ProtocolFilterPtr) strcpy (ProtocolString, "<none>"); else sprintf (ProtocolString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (ProtocolFilterPtr, -1, NULL, 0), ProtocolFilterRejectCount); if (!RefererFilterPtr) strcpy (RefererString, "<none>"); else sprintf (RefererString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (RefererFilterPtr, -1, NULL, 0), RefererFilterRejectCount); if (!RemoteIdentFilterPtr) strcpy (RemoteIdentString, "<none>"); else sprintf (RemoteIdentString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (RemoteIdentFilterPtr, -1, NULL, 0), RemoteIdentFilterRejectCount); if (!UserAgentFilterPtr) strcpy (UserAgentString, "<none>"); else sprintf (UserAgentString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (UserAgentFilterPtr, -1, NULL, 0), UserAgentFilterRejectCount); if (!ResponseFilterPtr) strcpy (ResponseString, "<none>"); else sprintf (ResponseString, ""%s"  (%d nomatch)", (char*)CgiLibHtmlEscape (ResponseFilterPtr, -1, NULL, 0), StatusFilterRejectCount); if (ResponseMinSize) { for (cptr = ResponseMinSizeString; *cptr; cptr++); sprintf (cptr, "  (%d nomatch)", ResponseMinSizeRejectCount); } if (ResponseMaxSize) { for (cptr = ResponseMaxSizeString; *cptr; cptr++); sprintf (cptr, "  (%d nomatch)", ResponseMaxSizeRejectCount); } } if (DateTimeProblemCount) sprintf (DateTimeProblemString, " (%d format problems)", DateTimeProblemCount); else DateTimeProblemString[0] = '\0'; fprintf (stdout, "

\n\ %s\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \ \n\ \n\ \ \n\ \ \n\ \n\
Log: %s%s(%d matched%s%s)
Modified: %s
Client: %s
IP Version: %s
Remote ID: %s
Auth User: %s
Date/Time: %s%s
Method: %s
Path: %s
Query: %s
Protocol: %s
Referer: %s
User Agent: %s
Response: %s
Size Min: %s
Max: %s
Statistics: %s
Records: %d  (%d suspect, %d common, %d combined)
Requests: %d
Methods: CONNECT:%d (%d%%)  COPY:%d (%d%%)  \ DELETE:%d (%d%%)  GET:%d (%d%%)  \ HEAD:%d (%d%%)  LOCK:%d (%d%%)  \ MKCOL:%d (%d%%)  MOVE:%d (%d%%)  \ OPTIONS:%d (%d%%)  POST:%d (%d%%)  \ PROPFIND:%d (%d%%)  PROPPATCH:%d (%d%%)  \ PUT:%d (%d%%)  TRACE:%d (%d%%)  \ UNLOCK:%d (%d%%)  ?:%d (%d%%)
Responses: 1nn:%d (%d%%)  2nn:%d (%d%%)  \ 3nn:%d (%d%%)  4nn:%d (%d%%)  5nn:%d (%d%%)  \ ?:%d (%d%%) 
%s: %s
\n\ \n\ \n", VmsMessage, CgiFormLogFileSpecPtr, CgiFormLogFileSpecPtr[0] ? "  " : "", FileCountTotal, IgnoredString, AccessProblemString, LastModifiedString, ClientString, IpVersionString, RemoteIdentString, AuthUserString, DateTimeString, DateTimeProblemString, MethodString, PathString, QueryString, ProtocolString, RefererString, UserAgentString, ResponseString, ResponseMinSizeString, ResponseMaxSizeString, StatTimer(), RecordCountTotal, SuspectRecordCountTotal, CommonLogRecordCount, CombinedLogRecordCount, RequestCountTotal, MethodConnectCount, MethodConnectPercent, MethodCopyCount, MethodCopyPercent, MethodDeleteCount, MethodDeletePercent, MethodGetCount, MethodGetPercent, MethodHeadCount, MethodHeadPercent, MethodLockCount, MethodLockPercent, MethodMkColCount, MethodMkColPercent, MethodMoveCount, MethodMovePercent, MethodOptionsCount, MethodOptionsPercent, MethodPostCount, MethodPostPercent, MethodPropFindCount, MethodPropFindPercent, MethodPropPatchCount, MethodPropPatchPercent, MethodPutCount, MethodPutPercent, MethodTraceCount, MethodTracePercent, MethodUnLockCount, MethodUnLockPercent, MethodUnknownCount, MethodUnknownPercent, StatusCodeCount[1], StatusCodePercent[1], StatusCodeCount[2], StatusCodePercent[2], StatusCodeCount[3], StatusCodePercent[3], StatusCodeCount[4], StatusCodePercent[4], StatusCodeCount[5], StatusCodePercent[5], StatusCodeCount[0], StatusCodePercent[0], ByteStringPtr, ByteString); } /****************************************************************************/ /* */ void CgiForm () { char *cptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "CgiForm()\n"); GeoLocatePtr = SysTrnLnmSystem("QDLOGSTATS_GEOLOCATE"); /* make a guess at the log file defaults */ if (CgiLibEnvironmentIsWasd()) cptr = DEFAULT_LOGS_WASD; else if (CgiLibEnvironmentIsOsu()) cptr = DEFAULT_LOGS_OSU; else if (CgiLibEnvironmentIsApache()) cptr = DEFAULT_LOGS_APACHE; else cptr = DEFAULT_LOGS_FILESPEC; CgiLibEnvironmentPtr = CgiLibEnvironmentName(); CgiScriptNamePtr = CgiLibVar ("WWW_SCRIPT_NAME"); sys$fao (&StartDateTimeFaoDsc, NULL, &StartDateTimeDsc, 0); if (StartDateTime[0] == ' ') StartDateTime[0] = '0'; CgiLibResponseHeader (200, "text/html"); fprintf (stdout, "\n\ \n\ \n\ \n\ \n\ \n\ \n\ qdlogstats@%s\n\ \n\ \n\ \n\ \n\ qdLogStats@%s
\n\    %s
\n\    %s\n\

\n\

\n\ \n\ \n", SoftwareID, CopyrightMeta, CgiLibEnvironmentPtr, SearchDna, CgiServerNamePtr, CgiServerNamePtr, SOFTWAREID, StartDateTime, CgiScriptNamePtr, cptr); LogModifiedSelector (); fprintf (stdout, "\n\ \n\ \n\ \n\ \n\ \n\ \n\ \n", TCPIP_LOOKUP_HOST_NAME_RETRY); DateTimeSelector (); fprintf (stdout, "\n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \n"); fprintf (stdout, "\n\ \n\
Log Specification: \
Client: \n\  \  lookup name\n\
IP Version: \n\ \n\
Remote ID: 
Auth User: 
Method: \n\ \n\  or \ \n\
Path: \n\  \  decode first\n\
Query: \n\  \  decode first\n\
Protocol: \n\ \n\  or \ \n\
Referer: \n\  \  decode first\n\
User Agent: \n\
Response: \n\ \n\  or \ \n\
Size Min: \n\ \n\  or \ \n\
Max: \n\ \n\  or \ \n\
View: \n\ \n"); if (CgiLibEnvironmentIsWasd()) fprintf (stdout, " \  no WASD entries\n"); fprintf (stdout, " \  match all\n\  \  resolve\n"); if (GeoLocatePtr) fprintf (stdout, "\n\
\n\   \ \n\
\n\

\n\ \n\ \n"); } /****************************************************************************/ /* Generate HTML form selectors for the modified since and before log file filters. Default it to cover the last month (for want of something better). */ void LogModifiedSelector () { static char *MonthName [] = { "", "JAN", "FEB", "MAR", "APR", "MAY", "JUN", "JUL", "AUG", "SEP", "OCT", "NOV", "DEC" }; int cnt, lm; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "LogModifiedSelector()\n"); if ((lm = NumTime[1] - 1) < 1) lm = 12; fprintf (stdout, "Modified Since: \n\ \n\n\n\  \  apply since\n\ \n"); fprintf (stdout, "Before: \n\ \n\n\n\  \  apply before\n\ \n"); } /****************************************************************************/ /* Get the since and before filter form fields and convert these to binary date/times. If an error with the time report and exit script. */ void LogModifiedCgi () { static char *MonthName[] = { "***", "JAN", "FEB", "MAR", "APR", "MAY", "JUN", "JUL", "AUG", "SEP", "OCT", "NOV", "DEC" }; static $DESCRIPTOR (DateTimeFaoDsc, "!2ZL-!3AZ-!4ZL !2ZL:!2ZL:!2ZL.!2ZL"); int dd, mmm, yyyy, status; unsigned short slen; unsigned long ResultBinTime [2]; char *cptr; char DateTime [32]; $DESCRIPTOR (DateTimeDsc, DateTime); /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "LogModifiedCgi()\n"); if (CgiLibVarNull ("WWW_FORM_LMSFILTER")) { /* applying the since filter */ cptr = CgiLibVar ("WWW_FORM_LMSDD"); dd = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_LMSMMM"); mmm = atoi(cptr); if (mmm < 1 || mmm > 12) mmm = 0; cptr = CgiLibVar ("WWW_FORM_LMSYYYY"); yyyy = atoi(cptr); sys$fao (&DateTimeFaoDsc, &slen, &DateTimeDsc, dd, MonthName[mmm], yyyy, 0, 0, 0, 0); DateTimeDsc.dsc$w_length = slen; DateTime[slen] = '\0'; status = sys$bintim (&DateTimeDsc, &LmSinceBinTime); if (VMSnok (status)) { CgiLibResponseError (FI_LI, status, "log modified since filter ... %s", DateTime); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } if (CgiLibVarNull ("WWW_FORM_LMBFILTER")) { /* applying the before filter */ cptr = CgiLibVar ("WWW_FORM_LMBDD"); dd = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_LMBMMM"); mmm = atoi(cptr); if (mmm < 1 || mmm > 12) mmm = 0; cptr = CgiLibVar ("WWW_FORM_LMBYYYY"); yyyy = atoi(cptr); DateTimeDsc.dsc$w_length = sizeof(DateTime)-1; sys$fao (&DateTimeFaoDsc, &slen, &DateTimeDsc, dd, MonthName[mmm], yyyy, 23, 59, 59, 99); DateTimeDsc.dsc$w_length = slen; DateTime[slen] = '\0'; status = sys$bintim (&DateTimeDsc, &LmBeforeBinTime); if (VMSnok (status)) { CgiLibResponseError (FI_LI, status, "date/time before filter ... %s", DateTime); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } if (!LmSinceBinTime[0] && !LmBeforeBinTime[0]) return; if (LmSinceBinTime[0]) sys$fao (&DateTime20FaoDsc, NULL, &LmSinceStringDsc, &LmSinceBinTime); else { /* create a time far enough in the past */ status = sys$bintim (&FarEnoughBackDsc, &LmSinceBinTime); if (VMSnok (status)) exit (status); } if (LmBeforeBinTime[0]) sys$fao (&DateTime20FaoDsc, NULL, &LmBeforeStringDsc, &LmBeforeBinTime); else { /* create a time far enough in the future */ status = sys$bintim (&FarEnoughForwardDsc, &LmBeforeBinTime); if (VMSnok (status)) exit (status); } /* sanity check on the filter window */ status = lib$sub_times (&LmBeforeBinTime, &LmSinceBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) { CgiLibResponseError (FI_LI, 0, "Impossible to select since %s and before %s", LmSinceString, LmBeforeString); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } /****************************************************************************/ /* Generate HTML form selectors for the since and before date/time filters. Default it to cover the time period midnight to midnight of the current day. */ void DateTimeSelector () { static char *MonthName[] = { "", "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" }; int cnt; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "DateTimeSelector()\n"); fprintf (stdout, "Date/Time Since: \n\ \n\n\n\n\n\  \  apply since filter\n\ \n"); fprintf (stdout, "Before: \n\ \n\n\n\n\n\  \  apply before filter\n\ \n"); } /****************************************************************************/ /* Get the since and before filter form fields and convert these to binary date/times. If an error with the time report and exit script. */ void DateTimeCgi () { static char *MonthName[] = { "***", "JAN", "FEB", "MAR", "APR", "MAY", "JUN", "JUL", "AUG", "SEP", "OCT", "NOV", "DEC" }; static $DESCRIPTOR (DateTimeFaoDsc, "!2ZL-!3AZ-!4ZL !2ZL:!2ZL:!2ZL"); int dd, mmm, yyyy, hh, mm, status; unsigned short slen; unsigned long ResultBinTime [2]; char *cptr; char DateTime [32]; $DESCRIPTOR (DateTimeDsc, DateTime); /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "DateTimeCgi()\n"); if (CgiLibVarNull ("WWW_FORM_DTSFILTER")) { /* applying the since filter */ cptr = CgiLibVar ("WWW_FORM_DTSDD"); dd = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTSMMM"); mmm = atoi(cptr); if (mmm < 1 || mmm > 12) mmm = 0; cptr = CgiLibVar ("WWW_FORM_DTSYYYY"); yyyy = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTSHH"); hh = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTSMM"); mm = atoi(cptr); sys$fao (&DateTimeFaoDsc, &slen, &DateTimeDsc, dd, MonthName[mmm], yyyy, hh, mm, 0); DateTimeDsc.dsc$w_length = slen; DateTime[slen] = '\0'; status = sys$bintim (&DateTimeDsc, &DtSinceFilterBinTime); if (VMSnok (status)) { CgiLibResponseError (FI_LI, status, "date/time since filter ... %s", DateTime); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } if (CgiLibVarNull ("WWW_FORM_DTBFILTER")) { /* applying the before filter */ cptr = CgiLibVar ("WWW_FORM_DTBDD"); dd = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTBMMM"); mmm = atoi(cptr); if (mmm < 1 || mmm > 12) mmm = 0; cptr = CgiLibVar ("WWW_FORM_DTBYYYY"); yyyy = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTBHH"); hh = atoi(cptr); cptr = CgiLibVar ("WWW_FORM_DTBMM"); mm = atoi(cptr); DateTimeDsc.dsc$w_length = sizeof(DateTime)-1; sys$fao (&DateTimeFaoDsc, &slen, &DateTimeDsc, dd, MonthName[mmm], yyyy, hh, mm, 0); DateTimeDsc.dsc$w_length = slen; DateTime[slen] = '\0'; status = sys$bintim (&DateTimeDsc, &DtBeforeFilterBinTime); if (VMSnok (status)) { CgiLibResponseError (FI_LI, status, "date/time before filter ... %s", DateTime); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } if (!DtSinceFilterBinTime[0] && !DtBeforeFilterBinTime[0]) return; if (DtSinceFilterBinTime[0]) sys$fao (&DateTime20FaoDsc, NULL, &DtSinceFilterStringDsc, &DtSinceFilterBinTime); else { /* create a time far enough in the past */ status = sys$bintim (&FarEnoughBackDsc, &DtSinceFilterBinTime); if (VMSnok (status)) exit (status); } if (DtBeforeFilterBinTime[0]) sys$fao (&DateTime20FaoDsc, NULL, &DtBeforeFilterStringDsc, &DtBeforeFilterBinTime); else { /* create a time far enough in the future */ status = sys$bintim (&FarEnoughForwardDsc, &DtBeforeFilterBinTime); if (VMSnok (status)) exit (status); } /* sanity check on the filter window */ status = lib$sub_times (&DtBeforeFilterBinTime, &DtSinceFilterBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) { CgiLibResponseError (FI_LI, 0, "Impossible to filter since %s and before %s", DtSinceFilterString, DtBeforeFilterString); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } /****************************************************************************/ /* Passed a string containing the date/time field from the log record, convert it to a binary time and then compare to the since/before binary date/times, to filter on the date/time. Returns true if outside the time period specified. */ BOOL FilterBinDateTime (char *DateTimePtr) { static char DateTime [21]; static $DESCRIPTOR (DateTimeDsc, DateTime); int status; char *cptr, *sptr, *zptr; unsigned long DateTimeBinTime [2], ResultBinTime [2]; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "FilterBinDateTime()\n"); /* get the first 20 characters of the date/time field */ zptr = (sptr = DateTime) + sizeof(DateTime)-1; for (cptr = DateTimePtr; *cptr && sptr < zptr; *sptr++ = *cptr++); *sptr = '\0'; /* make it look VMS-ish */ if (DateTime[2] != '/') return (false); DateTime[2] = '-'; if (DateTime[6] != '/') return (false); DateTime[6] = '-'; if (DateTime[11] != ':') return (false); DateTime[11] = ' '; status = sys$bintim (&DateTimeDsc, &DateTimeBinTime); if (VMSnok (status)) { if (Debug) fprintf (stdout, "sys$bintim() %%X%08.08X |%s|\n", status, DateTime); DateTimeProblemCount++; return (true); } /* if the since is larger (later) than the date/time then return false */ status = lib$sub_times (&DateTimeBinTime, &DtSinceFilterBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) { DtSinceFilterRejectCount++; return (true); } /* if the date/time is larger (later) than the before then return false */ status = lib$sub_times (&DtBeforeFilterBinTime, &DateTimeBinTime, &ResultBinTime); if (Debug) fprintf (stdout, "lib$sub_times() %%X%08.08X\n", status); if (status == LIB$_NEGTIM) { DtBeforeFilterRejectCount++; return (true); } DtSinceFilterAcceptCount++; DtBeforeFilterAcceptCount++; return (false); } /****************************************************************************/ /* Turn the supplied string, for example "dd/aug/2002:dd:hh:mm +hhmm", into a wilcarded date/time filter, such as "@@/aug/2002" and return a pointer to that (sometimes modified) string. If the string has not been changed from the default value of "dd/mmm/yyyy:hh:mm:ss" then return NULL. */ char* DateTimeFilter (char *String) { char *cptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "DateTimeFilter() |%s|\n", String); if (!String) return (NULL); if (strsame (String, "dd/mmm/yyyy:hh:mm:ss", 20)) return (NULL); cptr = String; if (*cptr == 'd') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'd') *cptr = '*'; if (*cptr) cptr++; if (*cptr != '/') return (String); if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr != '/') return (String); if (*cptr) cptr++; if (*cptr == 'y') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'y') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'y') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'y') *cptr = '*'; if (*cptr) cptr++; if (*cptr != ':') return (String); if (*cptr) cptr++; if (*cptr == 'h') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'h') *cptr = '*'; if (*cptr) cptr++; if (*cptr != ':') return (String); if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr != ':') return (String); if (*cptr) cptr++; if (*cptr == 's') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 's') *cptr = '*'; if (*cptr) cptr++; if (!isspace(*cptr)) return (String); if (*cptr) cptr++; if (*cptr != '+' && *cptr != '-' && *(unsigned char*)cptr != 177) return (String); *cptr++ = '*'; if (*cptr == 'h') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'h') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr == 'm') *cptr = '*'; if (*cptr) cptr++; if (*cptr) return (String); if (cptr > String && !*cptr) cptr--; while (cptr > String && !isalnum(*cptr)) cptr--; while (*cptr && *cptr != '*') cptr++; if (*cptr == '*') cptr++; *cptr = '\0'; return (String); } /****************************************************************************/ /* Search against the command line file specification, passing each file found to be processed. */ int SearchFileSpec (char *FileSpec) { int idx, status, FileNameLength, Length; char FileName [256], ExpFileName [256]; struct FAB SearchFab; struct NAM SearchNam; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "SearchFileSpec() |%s|%s|\n", SearchDna, FileSpec); /* initialize the file access block */ SearchFab = cc$rms_fab; SearchFab.fab$l_dna = SearchDna; SearchFab.fab$b_dns = strlen(SearchDna); SearchFab.fab$l_fna = FileSpec; SearchFab.fab$b_fns = strlen(FileSpec); SearchFab.fab$l_nam = &SearchNam; SearchNam = cc$rms_nam; SearchNam.nam$l_esa = ExpFileName; SearchNam.nam$b_ess = sizeof(ExpFileName)-1; SearchNam.nam$l_rsa = FileName; SearchNam.nam$b_rss = sizeof(FileName)-1; if (VMSnok (status = sys$parse (&SearchFab, 0, 0))) return (status); while (VMSok (status = sys$search (&SearchFab, 0, 0))) { *SearchNam.nam$l_ver = '\0'; if (Debug) fprintf (stdout, "FileName |%s|\n", FileName); ProcessLogFile (FileName); FileCountTotal++; *SearchNam.nam$l_ver = ';'; } if (RequestCountTotal) { for (idx = 0; idx <= 5; idx++) StatusCodePercent[idx] = lround ((float)StatusCodeCount[idx] * 100.0 / (float)RequestCountTotal); MethodConnectPercent = lround ((float)MethodConnectCount * 100.0 / (float)RequestCountTotal); MethodCopyPercent = lround ((float)MethodCopyCount * 100.0 / (float)RequestCountTotal); MethodDeletePercent = lround ((float)MethodDeleteCount * 100.0 / (float)RequestCountTotal); MethodGetPercent = lround ((float)MethodGetCount * 100.0 / (float)RequestCountTotal); MethodHeadPercent = lround ((float)MethodHeadCount * 100.0 / (float)RequestCountTotal); MethodLockPercent = lround ((float)MethodLockCount * 100.0 / (float)RequestCountTotal); MethodMkColPercent = lround ((float)MethodMkColCount * 100.0 / (float)RequestCountTotal); MethodMovePercent = lround ((float)MethodMoveCount * 100.0 / (float)RequestCountTotal); MethodOptionsPercent = lround ((float)MethodOptionsCount * 100.0 / (float)RequestCountTotal); MethodPropFindPercent = lround ((float)MethodPropFindCount * 100.0 / (float)RequestCountTotal); MethodPropPatchPercent = lround ((float)MethodPropPatchCount * 100.0 / (float)RequestCountTotal); MethodPostPercent = lround ((float)MethodPostCount * 100.0 / (float)RequestCountTotal); MethodPutPercent = lround ((float)MethodPutCount * 100.0 / (float)RequestCountTotal); MethodTracePercent = lround ((float)MethodTraceCount * 100.0 / (float)RequestCountTotal); MethodUnLockPercent = lround ((float)MethodUnLockCount * 100.0 / (float)RequestCountTotal); MethodUnknownPercent = lround ((float)MethodUnknownCount * 100.0 / (float)RequestCountTotal); } return (status); } /****************************************************************************/ /* Open the log file, read and process each record from it, close the file! */ int ProcessLogFile (char *FileName) { BOOL accepted, ShowLog; int status, CtimeBefore, CtimeSince; char *cptr; char HtmlLine [MAX_LINE_LENGTH+(MAX_LINE_LENGTH/4)], Line [MAX_LINE_LENGTH], LineBuffer [MAX_LINE_LENGTH]; FILE *FilePtr; stat_t statBuffer; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "ProcessLogFile() |%s|\n", FileName); if (LmSinceBinTime[0] || LmSinceBinTime[1]) CtimeSince = decc$fix_time (&LmSinceBinTime); else CtimeSince = 0; if (LmBeforeBinTime[0] || LmBeforeBinTime[1]) CtimeBefore = decc$fix_time (&LmBeforeBinTime); else CtimeBefore = 0; if (status = stat (FileName, &statBuffer) < 0) { if (Debug) fprintf (stdout, "stat() %d %%X%08.08X\n", status, vaxc$errno); AccessProblemCount++; status = vaxc$errno; return (status); } if (CtimeBefore && statBuffer.st_mtime > CtimeBefore) { LastModifiedIgnoredCount++; return (SS$_NORMAL); } if (CtimeSince && statBuffer.st_mtime < CtimeSince) { LastModifiedIgnoredCount++; return (SS$_NORMAL); } if (ShowProgress && !ViewRecords) { /* this works for both CLI and CGI (inside ""!) */ if (ProgressCount == ProgressWrap) { ProgressCount = 0; fputc ('\n', stdout); } ProgressCount++; fputc ('+', stdout); fflush (stdout); } if (ShowLogFile) ShowLog = true; else ShowLog = false; RecordCount = 0; FilePtr = fopen (FileName, "r", "shr=get", "shr=put"); if (!FilePtr) { status = vaxc$errno; if (Debug) fprintf (stdout, "fopen() %%X%08.08X\n", status); AccessProblemCount++; return (status); } while (fgets (Line, sizeof(Line), FilePtr)) { RecordCount++; RecordCountTotal++; if (ViewRecords == VIEW_NONE) { /* again this works for both CLI and CGI (inside ""!) */ if (ShowProgress && !(RecordCount % ShowProgress)) { if (ProgressCount == ProgressWrap) { ProgressCount = 0; fputc ('\n', stdout); } ProgressCount++; fputc ('.', stdout); fflush (stdout); } } else if (ViewRecords == VIEW_ALL) { if (DoCliStats) OutputLogRecord (Line); else { CgiLibHtmlEscape (Line, -1, HtmlLine, sizeof(HtmlLine)); OutputLogRecord (HtmlLine); } } else strcpy (LineBuffer, Line); if (ViewRecords == VIEW_LOG) { if (ShowLog) { for (cptr = FileName; *cptr; cptr++); while (cptr > FileName && *cptr != ']' && *cptr != ':') cptr--; if (*cptr == ']' || *cptr == ':') cptr++; fprintf (stdout, "%s\n", cptr); ShowLog = false; } } if (Debug) fprintf (stdout, "|%s|\n", Line); accepted = ProcessLogRecord (Line); if ((ViewRecords == VIEW_MATCH && accepted) || (ViewRecords == VIEW_NOMATCH && !accepted) || (ViewRecords == VIEW_SUSPECT && accepted) || (ViewRecords == VIEW_UNIQUE && accepted)) { if (ShowLog) { for (cptr = FileName; *cptr; cptr++); while (cptr > FileName && *cptr != ']' && *cptr != ':') cptr--; if (*cptr == ']' || *cptr == ':') cptr++; fprintf (stdout, "%s\n", cptr); ShowLog = false; } if (DoCliStats) OutputLogRecord (LineBuffer); else { CgiLibHtmlEscape (LineBuffer, -1, HtmlLine, sizeof(HtmlLine)); OutputLogRecord (HtmlLine); fflush (stdout); } } } fclose (FilePtr); return (SS$_NORMAL); } /*****************************************************************************/ /* Output a single record from the log. If client host name lookup is enabled then check whether the client field is a dotted-decimal host address. If it is then lookup the host name and output that instead of the address. */ void OutputLogRecord (char *rptr) { static int LineCount; char *cptr, *sptr, *zptr; char client [256]; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "OutputLogRecord()\n"); if (!DoCliStats && (LineCount & 1)) fputs ("", stdout); zptr = (sptr = client) + sizeof(client)-1; while (*rptr && *rptr != ' ' && sptr < zptr) *sptr++ = *rptr++; *sptr = '\0'; while (*rptr && *rptr != ' ') rptr++; if (!DoCliStats && (LineCount & 1)) { for (sptr = rptr; *sptr && *sptr != '\n'; sptr++); *sptr = '\0'; } fputs (client, stdout); if (Addr2Name) { if (TcpIpIsAddress (client)) sptr = TcpIpLookup (NULL, client, NULL, NULL); else sptr = TcpIpLookup (client, NULL, NULL, NULL); if (DoCliStats) fprintf (stdout, " (%s)", sptr); else fprintf (stdout, " (%s)", sptr); } if (GeoLocatePtr) { if (!TcpIpIsAddress (sptr = client)) sptr = TcpIpLookup (client, NULL, NULL, NULL); if (TcpIpIsAddress (sptr)) { GeoLocateId++; fprintf (stdout, " \ ", GeoLocateId, GeoLocateId, sptr); } else fprintf (stdout, " (%s)", sptr); } /* output the rest of the log record */ fputs (rptr, stdout); if (!DoCliStats && (LineCount++ & 1)) fputs ("\n", stdout); } /*****************************************************************************/ /* Process a single record (line) from the log file. Common: 'client rfc891 authuser date/time request status bytes' Combined: 'client rfc891 authuser date/time request status bytes referer agent' */ BOOL ProcessLogRecord (char *Line) { BOOL RejectRecord; int ByteCount; uchar Ip4Addr [4], Ip6Addr [16]; char *cptr, *sptr, *AuthUserPtr, *ClientPtr, *BytesPtr, *DateTimePtr, *QueryStringPtr, *ProtocolPtr, *RefererPtr, *RemoteIdentPtr, *RequestPtr, *StatusPtr, *UserAgentPtr; char ch; char EmptyString [1], Scratch [MAX_LINE_LENGTH]; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "ProcessLogRecord()\n"); EmptyString[0] = '\0'; cptr = Line; /* client */ ClientPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; if (*cptr) *cptr++ = '\0'; /* remote ident (RFC891) */ while (*cptr && isspace(*cptr)) cptr++; RemoteIdentPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; if (*cptr) *cptr++ = '\0'; /* authorized user */ while (*cptr && isspace(*cptr)) cptr++; AuthUserPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; if (*cptr) *cptr++ = '\0'; /* date/time */ while (*cptr && *cptr != '[') cptr++; if (*cptr) cptr++; DateTimePtr = cptr; while (*cptr && *cptr != ']') cptr++; if (*cptr) *cptr++ = '\0'; /* request ("method path?query protocol") */ while (*cptr && *cptr != '\"') cptr++; if (*cptr) cptr++; RequestPtr = cptr; while (*cptr && *cptr != '\"') cptr++; if (*cptr) *cptr++ = '\0'; /* HTTP response status */ while (*cptr && isspace(*cptr)) cptr++; StatusPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; if (*cptr) *cptr++ = '\0'; /* bytes transmitted */ while (*cptr && isspace(*cptr)) cptr++; BytesPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; if (*cptr) *cptr++ = '\0'; /* referer (only for COMBINED) */ RefererPtr = NULL; while (*cptr && isspace(*cptr)) cptr++; if (*cptr == '\"') { cptr++; RefererPtr = cptr; while (*cptr && *cptr != '\"') cptr++; } else if (*cptr) { RefererPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; } if (*cptr) *cptr++ = '\0'; /* user agent (only for COMBINED) */ UserAgentPtr = NULL; while (*cptr && isspace(*cptr)) cptr++; if (*cptr == '\"') { cptr++; UserAgentPtr = cptr; while (*cptr && *cptr != '\"') cptr++; } else if (*cptr) { UserAgentPtr = cptr; while (*cptr && !isspace(*cptr)) cptr++; } if (*cptr) *cptr++ = '\0'; if (!ClientPtr[0] || !DateTimePtr[0] || !RequestPtr[0] || !StatusPtr[0] || !BytesPtr) { SuspectRecordCountTotal++; if (ViewRecords != VIEW_SUSPECT) return (false); } else if (ViewRecords == VIEW_SUSPECT) return (false); if (!RefererPtr && !UserAgentPtr) CommonLogRecordCount++; else CombinedLogRecordCount++; /* reset the pattern counter */ FilterThisOut (NULL, NULL); RejectRecord = false; if (ClientFilterPtr) { if (LookupRetry) { cptr = TcpIpLookup (ClientPtr, NULL, NULL, NULL); if (cptr) ClientPtr = cptr; } if (FilterThisOut (ClientPtr, ClientFilterPtr)) { ClientFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else ClientFilterAcceptCount++; } if (IpVersionFilter) { /* check if it looks like an IPv4 address */ for (cptr = ClientPtr; isdigit(*cptr) || *cptr == '.'; cptr++); if (!*cptr) { /* looks like IPv4 */ if (IpVersionFilter == 4) Ip4FilterAcceptCount++; else { Ip4FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } } else { /* check if it looks like an IPv6 address */ cptr = ClientPtr; if (*(unsigned long*)cptr == '::FF' && !memcmp (cptr, "::FFFF:", 7)) cptr = ""; else if (*(unsigned long*)cptr == '::ff' && !memcmp (cptr, "::ffff:", 7)) cptr = ""; else while (isxdigit(*cptr) || *cptr == ':' || *cptr == '-') cptr++; if (!*cptr) { /* looks like IPv6 */ if (IpVersionFilter == 6) Ip6FilterAcceptCount++; else { Ip6FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } } else { /* assume it's a host name */ cptr = TcpIpLookup (ClientPtr, NULL, Ip4Addr, Ip6Addr); if (IpVersionFilter == 4) { if (cptr) { if (MATCH4 (Ip4Addr, Ip4AddrZero)) Ip4FilterAcceptCount++; else { Ip4FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } } else { Ip4FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } } else if (IpVersionFilter == 6) { if (cptr) { if (MATCH0 (Ip6Addr, Ip6AddrZero, 16)) { Ip6FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else Ip6FilterAcceptCount++; } else { Ip6FilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } } } } } if (RemoteIdentFilterPtr) { if (FilterThisOut (RemoteIdentPtr, RemoteIdentFilterPtr)) { RemoteIdentFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else RemoteIdentFilterAcceptCount++; } if (AuthUserFilterPtr) { if (FilterThisOut (AuthUserPtr, AuthUserFilterPtr)) { AuthUserFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else AuthUserFilterAcceptCount++; } if (DtBeforeFilterBinTime[0] || DtSinceFilterBinTime[0]) { if (FilterBinDateTime (DateTimePtr)) { DateTimeFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else DateTimeFilterAcceptCount++; } else if (DateTimeFilterPtr) { /* binary time takes precedence over this pattern matching */ if (FilterThisOut (DateTimePtr, DateTimeFilterPtr)) { DateTimeFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else DateTimeFilterAcceptCount++; } if (MethodFilterPtr) { for (cptr = RequestPtr; *cptr && *cptr != ' '; cptr++); /* terminate path at query string (should be necessary) */ if (!*cptr) cptr = NULL; if (cptr) *cptr = '\0'; if (*MethodFilterPtr == '?') { if (MATCH4 (RequestPtr, "GET") || MATCH5 (RequestPtr, "HEAD") || MATCH5 (RequestPtr, "POST") || /* now the less frequently occuring methods */ MATCH8 (RequestPtr, "CONNECT") || MATCH5 (RequestPtr, "COPY") || MATCH7 (RequestPtr, "DELETE") || MATCH5 (RequestPtr, "LOCK") || MATCH6 (RequestPtr, "MKCOL") || MATCH5 (RequestPtr, "MOVE") || MATCH8 (RequestPtr, "OPTIONS") || MATCH8 (RequestPtr, "PROPFIND") || MATCH8 (RequestPtr, "PROPATCH") || MATCH4 (RequestPtr, "PUT") || MATCH6 (RequestPtr, "TRACE") || MATCH7 (RequestPtr, "UNLOCK")) { MethodFilterRejectCount++; RejectRecord = true; } else MethodFilterAcceptCount++; } else if (*MethodFilterPtr == '(') { /* WebDAV-specific methods */ if (MATCH5 (RequestPtr, "COPY") || MATCH7 (RequestPtr, "DELETE") || MATCH5 (RequestPtr, "LOCK") || MATCH6 (RequestPtr, "MKCOL") || MATCH5 (RequestPtr, "MOVE") || MATCH8 (RequestPtr, "PROPFIND") || MATCH8 (RequestPtr, "PROPATCH") || MATCH7 (RequestPtr, "UNLOCK")) MethodFilterAcceptCount++; else { MethodFilterRejectCount++; RejectRecord = true; } } else { if (FilterThisOut (RequestPtr, MethodFilterPtr)) { MethodFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else MethodFilterAcceptCount++; } if (cptr) *cptr = ' '; } QueryStringPtr = ProtocolPtr = NULL; if (NoWasdEntries) { /* include the server method (POST) */ cptr = RequestPtr; while (*cptr && *cptr != ' ') cptr++; while (*cptr && *cptr == ' ') cptr++; /* find the start of any query string */ for (sptr = cptr; *sptr && *sptr != '?' && *sptr != ' '; sptr++); /* terminate path at query string */ ch = *sptr; *sptr = '\0'; /* If it matches something like: "POST /KLAATU::WASD:80-BEGIN-00000000" "POST /KLAATU::WASD:80-TIMESTAMP-00000001" "POST /KLAATU::WASD:80-END-00000002" */ if (!FilterThisOut (RequestPtr, "POST /*::WASD:*-*-*") || !FilterThisOut (RequestPtr, "POST /*::WASD%:*-*-*") || !FilterThisOut (RequestPtr, "POST /*::HTTP%:*-*-*")) RejectRecord = true; *sptr = ch; } if (PathFilterPtr) { /* skip over the method */ for (cptr = RequestPtr; *cptr && *cptr != ' '; cptr++); while (*cptr && *cptr == ' ') cptr++; /* find the start of any query string */ for (sptr = cptr; *sptr && *sptr != '?' && *sptr != ' '; sptr++); /* terminate path at query string */ if (*sptr == '?') QueryStringPtr = sptr + 1; else { QueryStringPtr = EmptyString; if (*sptr == ' ') ProtocolPtr = sptr + 1; } ch = *sptr; *sptr = '\0'; if (UrlDecodePath && *cptr) { strcpy (Scratch, cptr); cptr = Scratch; CgiLibUrlDecode (cptr); } if (FilterThisOut (cptr, PathFilterPtr)) { PathFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else PathFilterAcceptCount++; *sptr = ch; } if (QueryFilterPtr) { if (QueryStringPtr) cptr = QueryStringPtr; else { /* find the start of any query string */ for (cptr = RequestPtr; *cptr && *cptr != '?'; cptr++); if (*cptr) cptr++; QueryStringPtr = cptr; } for (sptr = cptr; *sptr && *sptr != ' '; sptr++); ch = *sptr; *sptr = '\0'; if (UrlDecodeQuery && *cptr) { strcpy (Scratch, cptr); cptr = Scratch; CgiLibUrlDecode (cptr); } if (FilterThisOut (cptr, QueryFilterPtr)) { QueryFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else QueryFilterAcceptCount++; *sptr = ch; if (*sptr == ' ') ProtocolPtr = sptr + 1; else ProtocolPtr = EmptyString; } if (ProtocolFilterPtr) { if (ProtocolPtr) cptr = ProtocolPtr; else { if (QueryStringPtr) cptr = QueryStringPtr; else { /* find the start of any query string */ for (cptr = RequestPtr; *cptr && *cptr != ' '; cptr++); while (*cptr && *cptr == ' ') cptr++; while (*cptr && *cptr != ' ' && *cptr != '?') cptr++; if (*cptr == '?') cptr++; } /* find the end of the query string and the start of the protocol */ while (*cptr && *cptr != ' ') cptr++; } if (*cptr == ' ') cptr++; if (FilterThisOut (cptr, ProtocolFilterPtr)) { ProtocolFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else ProtocolFilterAcceptCount++; } if (RefererFilterPtr) { /* if the record did not have a COMBINED referer field then reject */ if ((cptr = RefererPtr) && UrlDecodeReferer) { strcpy (Scratch, cptr); cptr = Scratch; CgiLibUrlDecode (cptr); } if (!cptr || FilterThisOut (cptr, RefererFilterPtr)) { RefererFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else RefererFilterAcceptCount++; } if (ResponseFilterPtr) { if (!cptr || FilterThisOut (StatusPtr, ResponseFilterPtr)) { StatusFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else StatusFilterAcceptCount++; } if (UserAgentFilterPtr) { /* if the record did not have a COMBINED user agent field then reject */ if (!UserAgentPtr || FilterThisOut (UserAgentPtr, UserAgentFilterPtr)) { UserAgentFilterRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else UserAgentFilterAcceptCount++; } ByteCount = atoi(BytesPtr); if (ResponseMinSize) { if (ByteCount < ResponseMinSize || !isdigit(*BytesPtr)) { ResponseMinSizeRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else ResponseMinSizeAcceptCount++; } if (ResponseMaxSize) { if (ByteCount > ResponseMaxSize || !isdigit(*BytesPtr)) { ResponseMaxSizeRejectCount++; if (!FilterOnAll) return (false); RejectRecord = true; } else ResponseMaxSizeAcceptCount++; } if (ViewRecords == VIEW_UNIQUE) RejectRecord = !UniqueClient (ClientPtr); if (RejectRecord) return (false); RequestCountTotal++; ByteCountTotal += (float)ByteCount; switch (StatusPtr[0]) { case '1' : StatusCodeCount[1]++; break; case '2' : StatusCodeCount[2]++; break; case '3' : StatusCodeCount[3]++; break; case '4' : StatusCodeCount[4]++; break; case '5' : StatusCodeCount[5]++; break; default : StatusCodeCount[0]++; } for (cptr = RequestPtr; *cptr && *cptr != ' '; cptr++); /* terminate path at query string (should be necessary) */ if (!*cptr) cptr = NULL; if (cptr) *cptr = '\0'; if (MATCH4 (RequestPtr, "GET")) MethodGetCount++; else if (MATCH5 (RequestPtr, "HEAD")) MethodHeadCount++; else if (MATCH5 (RequestPtr, "POST")) MethodPostCount++; else /* now the less frequently occuring methods */ if (MATCH8 (RequestPtr, "CONNECT")) MethodConnectCount++; else if (MATCH5 (RequestPtr, "COPY")) MethodCopyCount++; else if (MATCH7 (RequestPtr, "DELETE")) MethodDeleteCount++; else if (MATCH5 (RequestPtr, "LOCK")) MethodLockCount++; else if (MATCH6 (RequestPtr, "MKCOL")) MethodMkColCount++; else if (MATCH5 (RequestPtr, "MOVE")) MethodMoveCount++; else if (MATCH8 (RequestPtr, "OPTIONS")) MethodOptionsCount++; else if (MATCH8 (RequestPtr, "PROPFIND")) MethodPropFindCount++; else if (MATCH8 (RequestPtr, "PROPPATCH")) MethodPropPatchCount++; else if (MATCH4 (RequestPtr, "PUT")) MethodPutCount++; else if (MATCH6 (RequestPtr, "TRACE")) MethodTraceCount++; else if (MATCH7 (RequestPtr, "UNLOCK")) MethodUnLockCount++; else MethodUnknownCount++; if (cptr) *cptr = ' '; return (true); } /*****************************************************************************/ /* Pre-compile regular expressions. Report the first error by return false. Return true if all required compile ok. */ BOOL CompileFilters () { /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "CompileFilters()\n"); FilterThisOut (NULL, NULL); if (ClientFilterPtr) FilterThisOut (NULL, ClientFilterPtr); if (RemoteIdentFilterPtr) FilterThisOut (NULL, RemoteIdentFilterPtr); if (AuthUserFilterPtr) FilterThisOut (NULL, AuthUserFilterPtr); if (!DtBeforeFilterBinTime[0] && !DtSinceFilterBinTime[0]) if (DateTimeFilterPtr) FilterThisOut (NULL, DateTimeFilterPtr); if (MethodFilterPtr) if (*MethodFilterPtr != '?') FilterThisOut (NULL, MethodFilterPtr); if (PathFilterPtr) FilterThisOut (NULL, PathFilterPtr); if (QueryFilterPtr) FilterThisOut (NULL, QueryFilterPtr); if (ProtocolFilterPtr) FilterThisOut (NULL, ProtocolFilterPtr); if (RefererFilterPtr) FilterThisOut (NULL, RefererFilterPtr); if (ResponseFilterPtr) FilterThisOut (NULL, ResponseFilterPtr); if (UserAgentFilterPtr) FilterThisOut (NULL, UserAgentFilterPtr); if (RegCompPatternPtr) return (false); return (true); } /*****************************************************************************/ /* Return false if the pattern matches the string, true if it doesn't. Keep an array of pre-compiled regular expressions. */ BOOL FilterThisOut ( char *StringPtr, char *PatternPtr ) { static int PatternCount; static regex_t CompiledPattern[REGEX_PATTERN_MAX]; BOOL NegateResult; int retval; char *pptr, *sptr, *tptr, *zptr; char Scratch [256]; regex_t *pregptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "FilterThisOut() |%s|%s|\n", StringPtr ? StringPtr : "", PatternPtr ? PatternPtr : ""); if (!StringPtr && !PatternPtr) { /* reset the pattern count */ PatternCount = 0; return (true); } if (PatternCount >= REGEX_PATTERN_MAX) exit (SS$_BUGCHECK); pregptr = &CompiledPattern[PatternCount++]; if (*PatternPtr == '!') { PatternPtr++; /* if we're going to say *not* this, there must be *something* there */ if (StringPtr && !*StringPtr) return (true); NegateResult = true; } else NegateResult = false; if (!pregptr->buffer) { /* check for simple wildcard match */ if (*PatternPtr != REGEX_CHAR) { /* simple wildcard match, create regex equivalent */ zptr = (sptr = Scratch) + sizeof(Scratch)-2; /* anchor the start */ *sptr++ = '^'; for (pptr = PatternPtr; *pptr && sptr < zptr; pptr++) { switch (*pptr) { case '*' : /* match any zero or more characters */ *sptr++ = '.'; if (sptr < zptr) *sptr++ = *pptr; break; case '%' : /* match any single character */ *sptr++ = '.'; break; case '?' : /* match zero or any single character */ *sptr++ = '.'; if (sptr < zptr) *sptr++ = *pptr; break; case '\\' : case '^' : case '$' : case '.' : case '+' : case '|' : case '{' : case '[' : case '(' : /* meta-character, quote the character */ *sptr++ = '\\'; if (sptr < zptr) *sptr++ = *pptr; break; default : *sptr++ = *pptr; } } /* anchor the end */ *sptr++ = '$'; *sptr = '\0'; PatternPtr = Scratch; if (Debug) fprintf (stdout, "|%s|\n", PatternPtr); } else PatternPtr++; /* compile the pattern */ retval = regcomp (pregptr, PatternPtr, REGEX_C_FLAGS); if (retval) { /* compilation error */ if (!RegCompPatternPtr) { /* note the first regex compilation error details */ RegCompPatternPtr = PatternPtr; regerror (retval, pregptr, RegCompErrorString, sizeof(RegCompErrorString)); if (Debug) fprintf (stdout, "|%s|%s|\n", RegCompPatternPtr, RegCompErrorString); RegCompErrorString[0] = tolower(RegCompErrorString[0]); } return (true); } if (!StringPtr) return (true); } /* Start with a light-weight character match. Even if regex is eventually required this will eliminate many strings on simple character comparison before more heavy-weight pattern matching needs to be called into play. */ sptr = StringPtr; pptr = PatternPtr; if (*pptr == '^') { /* must be a regex pattern */ pptr++; /* step over any start-of-line anchor seeing we're there already */ if (*pptr == '^') { pptr++; /* if regex for empty string, then it's a match */ if (*(USHORTPTR)pptr == '$\0' && !*sptr) return (NegateResult ? true : false); } } while (*pptr && *sptr) { if (*(USHORTPTR)pptr == '*\0') break; switch (*pptr) { /* "significant" characters when pattern matching */ case '*' : case '^' : case '$' : case '.' : case '+' : case '?' : case '|' : case '{' : case '[' : case '(' : case '\\' : /* meta-character, quit now and perform a regex match */ retval = regexec (pregptr, StringPtr, 0, NULL, REGEX_E_FLAGS); if (retval) return (NegateResult ? false : true); return (NegateResult ? true : false); } if (tolower(*pptr) != tolower(*sptr) && *pptr != '%') break; pptr++; sptr++; } /* if matched exactly then don't filter it out */ if (!*pptr && !*sptr) return (NegateResult ? true : false); /* if the pattern ended in a trailing wildcard it matches, don't filter */ if (*(USHORTPTR)pptr == '*\0') return (NegateResult ? true : false); /* doesn't match! */ return (NegateResult ? false : true); } /*****************************************************************************/ /* Return true if the supplied client string has not been encountered before. Maintain a simple linked list of each client as it is encountered. */ typedef struct ClientRecordStruct { struct ClientRecordStruct *NextPtr; char ClientString[]; } ClientRecord; BOOL UniqueClient (char *ClientPtr) { static struct ClientRecordStruct *ClientList; struct ClientRecordStruct *crptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "UniqueClient() |%s|\n", ClientPtr); for (crptr = ClientList; crptr != NULL; crptr = crptr->NextPtr) { if (!MATCH8 (ClientPtr, crptr->ClientString)) continue; if (!strcmp (ClientPtr, crptr->ClientString)) return (false); } crptr = calloc (1, sizeof(struct ClientRecordStruct) + strlen(ClientPtr) + 1); if (!crptr) exit (vaxc$errno); strcpy (crptr->ClientString, ClientPtr); crptr->NextPtr = ClientList; ClientList = crptr; return (true); } /*****************************************************************************/ /* If |Name| is non-NULL lookup the IP address using the host name. If |Address| is non-NULL lookup the host name using the address. If the Ip4AddtrPtr and/or Ip6AddrPtr are non-NULL populate them. Return either a pointer to the resolved host name or IP address string or an error message between square brackets. */ char* TcpIpLookup ( char *Name, char *Address, uchar *Ip4Addr, uchar *Ip6Addr ) { #define CACHE_MAX 16 static aCacheIdx, nCacheIdx; static char abuf [CACHE_MAX][256], ares [CACHE_MAX][256], nbuf [CACHE_MAX][256], nres [CACHE_MAX][256]; int idx, retry, retval; void *addptr; struct sockaddr_in addr4; struct sockaddr_in6 addr6; struct addrinfo hints; struct addrinfo *aiptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "TcpIpLookup() |%s|%s| %d %d\n", Name, Address, Ip4Addr, Ip6Addr); if (Ip4Addr) memset (Ip4Addr, 0, 4); if (Ip6Addr) memset (Ip6Addr, 0, 16); if (Name) { for (idx = 0; idx < CACHE_MAX; idx++) if (!strcmp (nbuf[idx], Name)) return (nres[idx]); idx = nCacheIdx++ % CACHE_MAX; strcpy (nbuf[idx], Name); aiptr = NULL; memset (&hints, 0, sizeof(hints)); hints.ai_flags |= AI_CANONNAME; retval = 0; for (retry = TCPIP_LOOKUP_HOST_NAME_RETRY; retry; retry--) { retval = getaddrinfo (Name, NULL, &hints, &aiptr); if (retval != EINTR && retval != EAI_AGAIN) break; sleep (1); } if (retval) { if (retval == EAI_NONAME) sprintf (nres[idx], "[unknown]"); else if (retval == EAI_FAIL || retval == EAI_AGAIN) sprintf (nres[idx], "[failed]"); else sprintf (nres[idx], "[%s]", gai_strerror(retval)); return (nres[idx]); } else { if (aiptr->ai_family == AF_INET) { /* IPv4 */ addptr = &((struct sockaddr_in *)aiptr->ai_addr)->sin_addr; if (Ip4Addr) memcpy (Ip4Addr, addptr, 4); } else { /* must be IPv6 */ addptr = &((struct sockaddr_in6 *)aiptr->ai_addr)->sin6_addr; if (Ip6Addr) memcpy (Ip6Addr, addptr, 16); } if (!inet_ntop (aiptr->ai_family, addptr, nres[idx], sizeof(nres[0]))) sprintf (nres[idx], "[%s]", strerror(errno)); } /* free the addrinfo */ freeaddrinfo(aiptr); return (nres[idx]); } if (Address) { for (idx = 0; idx < CACHE_MAX; idx++) if (!strcmp (abuf[idx], Address)) return (ares[idx]); idx = aCacheIdx++ % CACHE_MAX; strcpy (abuf[idx], Address); retval = 0; memset (&addr4, 0, sizeof(addr4)); if (inet_pton (AF_INET, Address, &addr4.sin_addr) > 0) { /* MultiNet does not support BSD 4.4 AF_INET addresses */ #ifdef NO_SOCKADDR_LEN_4 /* this kludge seems to work for both! */ *(USHORTPTR)&addr4 = AF_INET; #else addr4.sin_len = sizeof(struct sockaddr_in); addr4.sin_family = AF_INET; #endif if (Ip4Addr) memcpy (Ip4Addr, &addr4.sin_addr, sizeof(addr4.sin_addr)); for (retry = TCPIP_LOOKUP_HOST_NAME_RETRY; retry; retry--) { retval = getnameinfo ((struct sockaddr*)&addr4, sizeof(addr4), ares[idx], sizeof(ares[0]), NULL, 0, NI_NAMEREQD); if (retval != EINTR && retval != EAI_AGAIN) break; sleep (1); } if (retval) { if (retval == EAI_NONAME) sprintf (ares[idx], "[unknown]"); else if (retval == EAI_FAIL || retval == EAI_AGAIN) sprintf (ares[idx], "[failed]"); else sprintf (ares[idx], "[%s]", gai_strerror(retval)); } return (ares[idx]); } else { memset (&addr6, 0, sizeof(addr6)); if (inet_pton (AF_INET6, Address, &addr6.sin6_addr) > 0) { addr6.sin6_len = sizeof(struct sockaddr_in6); addr6.sin6_family = AF_INET6; if (Ip6Addr) memcpy (Ip6Addr, &addr6.sin6_addr, sizeof(addr6.sin6_addr)); for (retry = TCPIP_LOOKUP_HOST_NAME_RETRY; retry; retry--) { retval = getnameinfo ((struct sockaddr*)&addr6, addr6.sin6_len, ares[idx], sizeof(ares[0]), NULL, 0, NI_NAMEREQD); if (retval != EINTR && retval != EAI_AGAIN) break; sleep (1); } if (retval) { if (retval == EAI_NONAME) sprintf (ares[idx], "[unknown]"); else if (retval == EAI_FAIL || retval == EAI_AGAIN) sprintf (ares[idx], "[failed]"); else sprintf (ares[idx], "[%s]", gai_strerror(retval)); } } else sprintf (ares[aCacheIdx], "[%s]", strerror(errno)); return (ares[idx]); } } return ("[bugcheck]"); } /*****************************************************************************/ /* Return true if it looks like an IPv4 or IPv6 address. */ BOOL TcpIpIsAddress (char *string) { char *cptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "TcpIpIsAddress() |%s|\n", string); if (!string) return (false); for (cptr = string; *cptr && isspace(*cptr); cptr++); if (!*cptr) return (false); cptr = string; if (*(unsigned long*)cptr == '::FF' && !memcmp (cptr, "::FFFF:", 7)) cptr += 7; else if (*(unsigned long*)cptr == '::ff' && !memcmp (cptr, "::ffff:", 7)) cptr += 7; while (*cptr && (isdigit(*cptr) || *cptr == '.')) cptr++; if (!*cptr) return (true); for (cptr = string; *cptr && (isxdigit(*cptr) || *cptr == ':' || *cptr == '-'); cptr++); if (!*cptr) return (true); return (false); } /*****************************************************************************/ /* Initialise (first call) and then output (second call) some statistics. */ char* StatTimer () { static $DESCRIPTOR (StatFaoDsc, "time:!%T cpu:!UL.!UL dio:!UL bio:!UL faults:!UL rec/sec:!AZ\0"); static unsigned long LibStatTimerReal = 1, LibStatTimerCpu = 2, LibStatTimerBio = 3, LibStatTimerDio = 4, LibStatTimerFaults = 5; static char StatString [128]; static $DESCRIPTOR (StatStringDsc, StatString); int status; unsigned long CpuBinTime, CountBio, CountDio, CountFaults; unsigned long RealBinTime [2]; float fSeconds; char *cptr, *sptr; char RealTime[32], RecordsPerSecond [32]; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "StatTimer()\n"); if (!StatString[0]) { lib$init_timer (0); StatString[0] = '*'; return (NULL); } lib$stat_timer (&LibStatTimerReal, &RealBinTime, 0); lib$stat_timer (&LibStatTimerCpu, &CpuBinTime, 0); lib$stat_timer (&LibStatTimerBio, &CountBio, 0); lib$stat_timer (&LibStatTimerDio, &CountDio, 0); lib$stat_timer (&LibStatTimerFaults, &CountFaults, 0); fSeconds = (float)RealBinTime[0]; fSeconds += (float)((signed int)RealBinTime[1]) * 4294967296.0; fSeconds *= -0.0000001; if ((float)RecordCountTotal / fSeconds > 100.0) sprintf (RecordsPerSecond, "%d", (int)((float)RecordCountTotal / fSeconds)); else sprintf (RecordsPerSecond, "%g", (float)RecordCountTotal / fSeconds); sys$fao (&StatFaoDsc, 0, &StatStringDsc, &RealBinTime, CpuBinTime/100, CpuBinTime%100, CountDio, CountBio, CountFaults, RecordsPerSecond); /* compress the real time string */ for (sptr = cptr = StatString; cptr < StatString+5; *sptr++ = *cptr++); while ((*cptr == '0' || *cptr == ':') && cptr < StatString+12) cptr++; while (*cptr) *sptr++ = *cptr++; *sptr = '\0'; return (StatString); } /*****************************************************************************/ /* Get "command-line" parameters, whether from the command-line or from a configuration symbol or logical containing the equivalent. (This code is *derived* from a similar functionality in [SRC.HTTPD]TCPIP.C) */ GetParameters () { static char CommandLine [256]; static unsigned long Flags = 0; int status, SkipParameters; unsigned short Length; char ch; char *aptr, *cptr, *clptr, *sptr; $DESCRIPTOR (CommandLineDsc, CommandLine); $DESCRIPTOR (DateTimeDsc, ""); /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "GetParameters()\n"); if (!(clptr = getenv ("QDLOGSTATS$PARAM"))) { /* get the entire command line following the verb */ if (VMSnok (status = lib$get_foreign (&CommandLineDsc, 0, &Length, &Flags))) exit (status); (clptr = CommandLine)[Length] = '\0'; } /* if [C]SWS (VMS Apache) */ if (CgiLibEnvironmentIsApache()) { /* CSWS 1.2/3 look for something non-space outside of quotes */ for (cptr = clptr; *cptr; cptr++) { if (isspace(*cptr)) continue; if (*cptr != '\"') break; cptr++; while (*cptr && *cptr != '\"') cptr++; if (*cptr) cptr++; } /* CSWS 1.2/3 if nothing outside of quotes then ignore command line */ if (!*cptr) return; /* SWS 2.0 doesn't begin with /APACHE from DCL procedure wrapper */ if (!strsame (cptr, "/APACHE", 7)) return; } /* if OSU environment then skip P1, P2, P3 */ if (CgiLibEnvironmentIsOsu()) SkipParameters = 3; else SkipParameters = 0; aptr = NULL; ch = *clptr; for (;;) { if (aptr && *aptr == '/') *aptr = '\0'; if (!ch) break; *clptr = ch; if (Debug) fprintf (stdout, "clptr |%s|\n", clptr); while (*clptr && isspace(*clptr)) *clptr++ = '\0'; aptr = clptr; if (*clptr == '/') clptr++; while (*clptr && !isspace (*clptr) && *clptr != '/') { if (*clptr != '\"') { clptr++; continue; } cptr = clptr; clptr++; while (*clptr) { if (*clptr == '\"') if (*(clptr+1) == '\"') clptr++; else break; *cptr++ = *clptr++; } *cptr = '\0'; if (*clptr) clptr++; } ch = *clptr; if (*clptr) *clptr = '\0'; if (Debug) fprintf (stdout, "aptr |%s|\n", aptr); if (!*aptr) continue; if (SkipParameters) { SkipParameters--; continue; } if (strsame (aptr, "/RESOLVE", 6)) { Addr2Name = true; continue; } if (strsame (aptr, "/APACHE", 4)) { /* just skip this marker for command-line parameters under SWS 2.0 */ continue; } if (strsame (aptr, "/ALL", -1)) { FilterOnAll = true; continue; } if (strsame (aptr, "/AUTHUSER=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; AuthUserFilterPtr = cptr; continue; } if (strsame (aptr, "/BEFORE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; DateTimeDsc.dsc$a_pointer = cptr; DateTimeDsc.dsc$w_length = strlen(cptr); if (DateTimeDsc.dsc$w_length >= 11 && !isalnum(cptr[11])) cptr[11] = ' '; status = lib$convert_date_string (&DateTimeDsc, &LmBeforeBinTime, 0, 0, 0, 0); if (VMSnok (status)) exit (status); continue; } if (strsame (aptr, "/CLIENT=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; ClientFilterPtr = cptr; continue; } if (strsame (aptr, "/DBUG", -1)) { Debug = true; continue; } if (strsame (aptr, "/DATETIME=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; DateTimeFilterPtr = DateTimeFilter (cptr); continue; } if (strsame (aptr, "/DECODE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; if (!*cptr) { UrlDecodePath = UrlDecodeQuery = UrlDecodeReferer = true; continue; } if (*cptr == '(') cptr++; while (*cptr && *cptr != ')') { while (*cptr == ',') cptr++; if (*cptr) cptr++; if (toupper(*cptr) == 'P') UrlDecodePath = true; else if (toupper(*cptr) == 'Q') UrlDecodeQuery = true; else if (toupper(*cptr) == 'R') UrlDecodeReferer = true; else { for (sptr = cptr; *sptr && *sptr != ',' && *sptr != ')'; sptr++); *sptr = '\0'; fprintf (stdout, "%%%s-E-INVKEYW, invalid keyword\n \\%s\\\n", Utility, cptr); exit (STS$K_ERROR | STS$M_INHIB_MSG); } while (*cptr && *cptr != ',' && *cptr != ')') cptr++; } continue; } if (strsame (aptr, "/DTBEFORE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; DateTimeDsc.dsc$a_pointer = cptr; DateTimeDsc.dsc$w_length = strlen(cptr); if (DateTimeDsc.dsc$w_length >= 11 && !isalnum(cptr[11])) cptr[11] = ' '; status = lib$convert_date_string (&DateTimeDsc, &DtBeforeFilterBinTime, 0, 0, 0, 0); if (VMSnok (status)) exit (status); continue; } if (strsame (aptr, "/DTSINCE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; DateTimeDsc.dsc$a_pointer = cptr; DateTimeDsc.dsc$w_length = strlen(cptr); if (DateTimeDsc.dsc$w_length >= 11 && !isalnum(cptr[11])) cptr[11] = ' '; status = lib$convert_date_string (&DateTimeDsc, &DtSinceFilterBinTime, 0, 0, 0, 0); if (VMSnok (status)) exit (status); continue; } if (strsame (aptr, "/IP=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; if (isdigit(*cptr)) IpVersionFilter = atoi(cptr); else { fprintf (stdout, "%%%s-E-IVKEYW, unrecognized keyword\n \\%s\\\n", Utility, cptr); exit (STS$K_ERROR | STS$M_INHIB_MSG); } continue; } if (strsame (aptr, "/LOG", -1)) { ShowLogFile = true; continue; } if (strsame (aptr, "/LOOKUP=", 5)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; if (isdigit(*cptr)) LookupRetry = atoi(cptr); else LookupRetry = 0; continue; } if (strsame (aptr, "/METHOD=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; MethodFilterPtr = cptr; continue; } if (strsame (aptr, "/NOWASD", -1)) { NoWasdEntries = true; continue; } if (strsame (aptr, "/OUTPUT=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; OutputPtr = cptr; continue; } if (strsame (aptr, "/PATH=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; PathFilterPtr = cptr; continue; } if (strsame (aptr, "/PROGRESS=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; ShowProgress = atoi(cptr); if (ShowProgress <= 0) ShowProgress = PROGRESS_RECORDS; continue; } if (strsame (aptr, "/QUERY=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; QueryFilterPtr = cptr; continue; } if (strsame (aptr, "/REFERER=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; RefererFilterPtr = cptr; continue; } if (strsame (aptr, "/REMOTEID=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; RemoteIdentFilterPtr = cptr; continue; } if (strsame (aptr, "/RESPONSE", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; ResponseFilterPtr = cptr; continue; } if (strsame (aptr, "/SINCE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; DateTimeDsc.dsc$a_pointer = cptr; DateTimeDsc.dsc$w_length = strlen(cptr); if (DateTimeDsc.dsc$w_length >= 11 && !isalnum(cptr[11])) cptr[11] = ' '; status = lib$convert_date_string (&DateTimeDsc, &LmSinceBinTime, 0, 0, 0, 0); if (VMSnok (status)) exit (status); continue; } if (strsame (aptr, "/SIZE=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); while (*cptr == '=') cptr++; while (*cptr == '(') cptr++; while (*cptr) { if (strsame (cptr, "MIN=", 4)) { cptr += 4; ResponseMinSize = atoi(cptr); while (isdigit(*cptr)) cptr++; if (tolower(*cptr) == 'k') ResponseMinSize *= 1000; if (toupper(*cptr) == 'M') ResponseMinSize *= 1000000; } else if (strsame (cptr, "MAX=", 4)) { cptr += 4; ResponseMaxSize = atoi(cptr); while (isdigit(*cptr)) cptr++; if (tolower(*cptr) == 'k') ResponseMaxSize *= 1000; if (toupper(*cptr) == 'M') ResponseMaxSize *= 1000000; } else { fprintf (stdout, "%%%s-E-IVKEYW, unrecognized keyword\n \\%s\\\n", Utility, cptr); exit (STS$K_ERROR | STS$M_INHIB_MSG); } while (*cptr && *cptr != ',' && *cptr != ')') cptr++; while (*cptr == ',' || *cptr == ')') cptr++; } continue; } if (strsame (aptr, "/SOFTWAREID", 4) || strsame (aptr, "/VERSION", 4)) { fprintf (stdout, "%%%s-I-SOFTWAREID, %s\n%s\n", Utility, SoftwareID, CopyrightInfo); exit (SS$_NORMAL); } if (strsame (aptr, "/USERAGENT=", 4)) { for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (*cptr) cptr++; UserAgentFilterPtr = cptr; continue; } if (strsame (aptr, "/VIEW=", 4)) { ViewRecords = VIEW_MATCH; for (cptr = aptr; *cptr && *cptr != '='; cptr++); if (!*cptr) continue; cptr++; if (strsame (cptr, "ALL", 3)) ViewRecords = VIEW_ALL; else if (strsame (cptr, "LOG", 3)) ViewRecords = VIEW_LOG; else if (strsame (cptr, "MATCH", 3)) ViewRecords = VIEW_MATCH; else if (strsame (cptr, "NOMATCH", 3)) ViewRecords = VIEW_NOMATCH; else if (strsame (cptr, "SUSPECT", 7)) ViewRecords = VIEW_SUSPECT; else if (strsame (cptr, "UNIQUE", 6)) ViewRecords = VIEW_UNIQUE; else { fprintf (stdout, "%%%s-E-IVKEYW, unrecognized keyword\n \\%s\\\n", Utility, cptr); exit (STS$K_ERROR | STS$M_INHIB_MSG); } continue; } if (*aptr == '/') { fprintf (stdout, "%%%s-E-IVQUAL, unrecognized qualifier\n \\%s\\\n", Utility, aptr+1); exit (STS$K_ERROR | STS$M_INHIB_MSG); } if (!LogFileSpec[0]) { sptr = LogFileSpec; for (cptr = aptr; *cptr; *sptr++ = toupper(*cptr++)); *sptr = '\0'; continue; } fprintf (stdout, "%%%s-E-MAXPARM, too many parameters\n \\%s\\\n", Utility, aptr); exit (STS$K_ERROR | STS$M_INHIB_MSG); } } /*****************************************************************************/ /* Get the value of a system-level (only) logical name. Returns a pointer to dynamic storage containing the null-terminated string, or NULL if a problem. A *little* like 'getenv()' but only from the system logical table - serves much the same purpose anyway. */ char* SysTrnLnmSystem (char *LogicalName) { static unsigned short Length; static char LogicalValue [256]; static $DESCRIPTOR (LogicalNameDsc, ""); static $DESCRIPTOR (LnmSystemDsc, "LNM$SYSTEM"); static struct { short int buf_len; short int item; void *buf_addr; unsigned short *ret_len; } LnmItems [] = { { sizeof(LogicalValue)-1, LNM$_STRING, LogicalValue, &Length }, { 0,0,0,0 } }; int status; char *sptr; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "SysTrnLnmSystem() |%s|\n", LogicalName); LogicalNameDsc.dsc$w_length = strlen(LogicalName); LogicalNameDsc.dsc$a_pointer = LogicalName; status = sys$trnlnm (0, &LnmSystemDsc, &LogicalNameDsc, 0, &LnmItems); if (Debug) fprintf (stdout, "sys$trnlnm() %%X%08.08X\n", status); if (VMSnok (status)) return (NULL); LogicalValue[Length] = '\0'; if (Debug) fprintf (stdout, "|%s|\n", LogicalValue); if (!(sptr = calloc (1, Length+1))) exit (vaxc$errno); memcpy (sptr, LogicalValue, Length+1); return (sptr); } /*****************************************************************************/ /* Return a pointer to the message string corresponding to the supplied VMS status value. */ char* SysGetMsg (int VmsStatus) { static char Message [256]; int status; short int Length; $DESCRIPTOR (MessageDsc, Message); /*********/ /* begin */ /*********/ Message[0] = '\0'; if (VmsStatus) { /* text component only */ sys$getmsg (VmsStatus, &Length, &MessageDsc, 1, 0); Message[Length] = '\0'; } if (Message[0]) return (Message); else return ("(internal error)"); } /*****************************************************************************/ /* Because it can be installed with privileges (for CLI usage) ... */ void NeedsPrivilegedAccount () { static unsigned long PrivAcctMask [2] = { PRV$M_SETPRV | PRV$M_SYSPRV, 0 }; static long Pid = -1; static unsigned long JpiAuthPriv [2]; static struct { unsigned short buf_len; unsigned short item; void *buf_addr; void *ret_len; } JpiItems [] = { { sizeof(JpiAuthPriv), JPI$_AUTHPRIV, &JpiAuthPriv, 0 }, {0,0,0,0} }; int status; /*********/ /* begin */ /*********/ if (Debug) fprintf (stdout, "NeedsPrivilegedAccount()\n"); status = sys$getjpiw (0, &Pid, 0, &JpiItems, 0, 0, 0); if (VMSnok (status)) exit (status); if (!(JpiAuthPriv[0] & PrivAcctMask[0])) exit (SS$_NOSYSPRV); } /****************************************************************************/ /* Does a case-insensitive, character-by-character string compare and returns true if two strings are the same, or false if not. If a maximum number of characters are specified only those will be compared, if the entire strings should be compared then specify the number of characters as 0. */ BOOL strsame ( char *sptr1, char *sptr2, int count ) { /*********/ /* begin */ /*********/ while (*sptr1 && *sptr2) { if (toupper (*sptr1++) != toupper (*sptr2++)) return (false); if (count) if (!--count) return (true); } if (*sptr1 || *sptr2) return (false); else return (true); } /*****************************************************************************/ /* */ int ShowHelp () { fputs ( "Usage for Quick and Dirty LOG STATisticS\n\ \n\ $ QDLOGSTATS [] []\n\ \n\ Utility to extract very elementary statistics from Web server common/combined\n\ format log files. A number of filters allow subsets of the log contents to be\n\ selected using simple wildcard expressions. Strings are NOT case-sensitive.\n\ Optionally, log file records can be viewed as processed, or a simple progress\n\ indicator can be displayed (\"+\" for each file, \".\" per 1000 records thereof).\n\ \n\ /ALL /AUTHUSER=filter /BEFORE=time /CLIENT=filter /DATETIME=filter\n\ /DECODE[=keyword] /DTBEFORE=time /DTSINCE=time /IP=[4|6] /LOG\n\ /LOOKUP[=integer] /METHOD=filter /NOWASD /OUTPUT=file /PATH=filter\n\ /PROGRESS[=integer] /PROTOCOL=filter /QUERY=filter /REFERER=filter\n\ /RESOLVE /RESPONSE=filter /SINCE=time /SIZE=[MIN=,MAX=] /SOFTWAREID\n\ SERAGENT=filter /VERSION /VIEW[=MATCH(D)|ALL|NOMATCH|LOG|SUSPECT]\n\ \n\ $ QDLOGSTATS == \"$dir:QDLOGSTATS\"\n\ $ QDLOGSTATS WASD_LOGS:*.LOG /VIEW /PATH=\"/wasd/*.zip\"\n\ $ QDLOGSTATS WASD_LOGS:*NOV*ACCESS* /PATH=\"/CGI-BIN/*\" /QUERY=\"-{-}\"\n\ $ QDLOGSTATS WASD_LOGS:*ACCESS*.LOG /METHOD=POST /DTSINCE=YESTERDAY\n\ $ QDLOGSTATS WASD_LOGS:*.LOG /SINCE=01-FEB-2002 /USERAGENT=*MOZILLA*X11*\n\ \n", stdout); return (SS$_NORMAL); } /*****************************************************************************/