OpenVMS Demos: Programmer's Corner

This entire site is a private effort of free information.
There are no cookies, no advertisements, and nothing is for sale.
non-code OpenVMS resources can be found here
Caveat:
  1. The information presented here is intended for educational use by qualified OpenVMS programmers.
  2. The information presented here is provided free of charge, as-is, with no warranty of any kind.
  3. To the best of my knowledge, the programs presented here compile, link and run properly. Please send me an email if they do not (I only support this stuff in my spare time).
Updated: 2024-02-03

OpenVMS Source Code (presented in HTML format)

Presented in HTML format so:
  • Google can index them
    • you can search them with "Google proper"
    • you can search them with "Google site-search" like so: site:neilrieck.net  lib$spawn (just drop this red text into a Google search to see what I mean)
  • you can display them with a browser (tab characters are rendered properly with modern browsers)
  • you can drag a cursor over source text then copy-paste into an EDT editor (provided you are using a good terminal emulator)

How my source code pages were webified (is that a real word?)

These demo programs were converted to HTML pages by running the source code through this VMS-BASIC utility ( tool_webify_source_code.bas ) which converts certain HTML-sensitive characters into HTML entities then generates a complete web page

OpenVMS Python demos

  • Why use Python? Read my introduction to learn how Python bypasses usual 64-bit limitations
    • here I am talking about native math support languages like BASIC, C, C++ etc. Many developers get around these limitations by writing, or purchasing, a math library
  • Logical Name Demo.py (also shows how to access STARLET definitions)
  • Python JIT Compiler (on OpenVMS)

OpenVMS BASIC demos

People new to VMS-BASIC might wish to read these articles:

File I/O Demos

RMS - Record Management Services for OpenVMS

comments:
  1. Back in the day, many hardware vendors also sold their own proprietary operating systems. IBM (International Business Machines) and DEC (Digital Equipment Corporation) both incorporated support for sequential and relative data records into their respective operating systems in order to facilitate COBOL-74.
  2. Then they both included support for sorted indexed records in order to support COBOL-80 which was finally released as COBOL-85. IBM invented the name ISAM which contains all three technologies.
  3. On DEC systems, like VMS and OpenVMS, indexed ISAM can be seen in action by examining the system account file.
    Type this DCL command dir sys$system:SYSUAF.DAT /full to see what I mean
  4. COBOL uses PIC statements to layout record structure (PIC is short for PICTURE). VMS-BASIC employs MAP statements

program files

  • rms_sequential_demo.bas
    • shows how to manipulate sequential RMS files (text files are one example)
  • rms_relative_demo.bas
    • shows how to manipulate relative RMS files
  • rms_indexed_demo.bas
    • shows how to manipulate indexed RMS files (ISAM = Indexed Sequential Access Method)
    • this is a vanilla green screen demo
    • comment: many software manufacturers, including IBM and DEC incorporated ISAM into their operating systems in order to support COBOL
  • rms_indexed_demo_fms.bas
    • shows how to manipulate indexed RMS files (ISAM = Indexed Sequential Access Method)
    • this is an FMS-based green screen demo
      • you will need this FMS form: rms_indexed_demo_fms.zip
      • you will need this FMS include: fms_fdvdef.inc
      • you will need this other include: device_controls.inc
      • FMS (Forms Management System) Notes:
        • during the VAX/VMS era, developers were encouraged to insert their various FMS forms (name1.frm) into a developer-created "forms library" (name.flb). Just after image activation, your program would call FDV$LOPEN to open the library, then FDV$LOAD to fetch the forms into workspace buffers in your program, then call FDV$LCLOS to close the library channel. In those days I went to great lengths to reduce the cost of image activations on our underpowered while overworked VAX platforms. One solution was to bypass the form library by creating a binary version of each form at compile-time then linking them to the executable. The FMS demo shows you how to do this.
        • There are ~ 66 FMS functions and you can read more about them by downloading manuals from here:
        • caveat: many FMS-based solutions were migrated to a newer API called DECForms which has fewer functions with more parameters
  • rms_test_fsp.bas
    • a little hacking with the FSP$ function (originally provided to do a trial file-open on BASIC-PLUS-2 programs)
    • newer programs should only use USEROPEN (see next demo)
  • rms_test_useropen.bas
    • do a trial file-open using a USEROPEN function
      • an intermediate version includes a hack to allow inclusion of XABSUM and XABKEY (this last one is for inspecting keys a.k.a. indexes)
      • recently added code to include XAMPRO (to inspect protection bits)
      • this program works in two modes:
        • simple-and-safe:
          • simply tack my pointer onto the existing FAB:XAB pointer (overwriting what was their previously)
          • do my thing
          • restore original pointer before exiting USEROPEN
        • complex-and-dangerous:
          • start at FAB:XAB then scan the linked list until you hit a zero. Then use a home-grown POKE routine to tack my list onto the end of the existing list
          • do my thing
          • restore original pointer before exiting USEROPEN
  • sys_file_io_demo.bas
    • doing file i/o in BASIC without using BASIC's file i/o routines
  • read_sysuaf.bas
    • directly reading SYSUAF records for audit purposes only (you need lots of privs just to open SYSUAF.DAT)
    • however, I also used this code to move client records from my VMS system into a relational database (I wanted to support application-level authentication without allowing access to VMS proper; information here allowed me to move active VMS accounts into an application-level database without needed to reset passwords)

RDB - Oracle-Rdb (Relational Data Base)

  • DEC intended RDB to be the successor to RMS which they sold to Oracle Corporation in the 1990s
  • you can find the original 43-file zipped package here
  • More information: OpenVMS Notes: Oracle-Rdb

MariaDB - An alternate fork of MySQL

  • Click here to view several VMS-BASIC demos where I call the MariaDB client via the C-API. People wanting to use MySQL would only need to slightly modify the build script.

Tools: RMS Export - MySQL Import

  • mysql_import_helper.bas
  • mysql_import_helper_basic_template_part1.bas
  • mysql_import_helper_basic_template_part2.bas
  • mysql_import_helper_basic_template_part3.bas
  • In 2014, our primary business system consisted of ~ 70 BASIC programs reading/writing ~ 150 RMS indexed files. In order for us to do a parallel operations (RMS=production, MySQL=daily snapshot for ad-hoc queries) over the next 12 months, we needed a semi-autonomous method of tool generation. These four BASIC programs do the following:
    • generate 99% complete BASIC programs to extract data from RMS files then write it to CSV files
      • you only need to edit one line before compiling (the input file name)
      • generic open/read of your RMS files (bypasses indexes/keys)
      • this tool is preferable to opening up your source code to insert customized export routines
      • once built, there is no need to do this again unless the RMS file format changes
    • generate 99% complete SQL scripts to create the tables then import the data from CSV files
      • since my tool does not (yet) analyze the OPEN statements, you will need to insert SQL statements to create constraints, indexes and keys, etc.
      • CAVEAT: you could modify the table creation statements before the import but this would cause the import to be hundreds of times slower (lots of recursion while it maintains each index AND you could still end up with a lop sided index if the inserted data was already ordered). All large imports should always be indexless and constraint-less; you tack them on as a finishing step.
      • I placed all my post-import SQL statements in a second script which won't be overwritten by the tool
  • 2015 Update: after 12 months or so, many of our programs are bypassing RMS and writing production data directly into MariaDB. This means that these transfer tools are taking a back seat to our daily operations. We expect to have everything moved out of RMS by the end of 2019.
  • 2020 Update: we still have five RMS-ISAM files which have not yet been permanently moved to MariaDB so I updated the four programs above to do a better conversion from mixed to latin1 (cp1252)

I/P Stack Demos (written entirely in VMS-BASIC)

  • click here to see my c-demos farther down this page

TCPIP Services for OpenVMS

  • TCPIP Services for OpenVMS is HPE's official IP stack for OpenVMS
    • IIRC, there was never a version of "TCPIP Services for VMS on VAX" which is why most VMS sites ran TCPware from Process Software
    • versions 4.x and lower produce logical names prefixed with "UCX" ("UNIX Communications Extensions" or "Ultrix Communications Extensions")
    • versions 5.0 and higher produce logical names prefixed with "TCPIP"
      • later versions of 5.x support both IPv4 and IPv6
  • Notes:
    • the demo programs just below where written in 2014 when I was preparing to move from TCPware on Alpha to MultiNet on Itanium
      • Why change? In 2014, PSC told us that TCPware only supported IPv4 and never would support IPv6
      • Much of our home-grown apps were dependent upon a TCPware API known as the TELNET Library (nothing equivalent in any other stack)
      • These demos were developed on TCPIP Services (the machine I was working on at the time did not have have a MultiNet license)
      • these programs employ VMS System Services rather than the Sockets API so will work with any stack
  • tcpip$tcp_client_qio_2014a.bas
    • Synchronous (qiow) TCP client demo - employs http to retrieve a web page
  • tcpip$tcp_client_qio_2014b.bas
    • Synchronous (qiow) TCP client demo - adds support for gethostbyname
  • tcpip$tcp_client_qio_2014c.bas
    • Asynchronous (qio) TCP client demo - employs programmable timers
  • tcpip$tcp_client_qio_2014d.bas
  • tcpip$tcp_client_qio_2014d.inc
    • Same as "2014c" except my TCP routines have been moved into external functions
  • tcpip$tcp_client_qio_2014e.bas
  • tcpip$tcp_client_qio_2014e.inc
  • nvt_definitions.inc
    • Same as "2014d" but with NVT support for TELNET (previously handled in TCPware by the TELNET Library)
  • tcpip$tcp_client_qio_2014f_bas
    • Same as "2014e" but a bit more polished
  • tcpip$tcp_sa_client_qio_basic.bas
    • stand-alone client demo (telnet to another system, log in, run a program, log out)
      this demo program works but could use a little polishing; it probably will not be developed any further
  • get_host_by_name_qio.bas
    • DNS lookup from BASIC (also contains a few hacking tools)
  • sff_demo.bas (caveat: SEND FROM FILE is supported by "TCPIP Services")
    • demos function tcpip$send_from_file() which is a nifty way to send SMTP messages to your own stack without opening a connection to port #25

MultiNet

  1. a third-party stack from Process Software (supports IPv4 and IPv6)
  2. more feature-rich than TCPIP Services for OpenVMS from HP
  3. No stack-specific demos here since my "TCPIP Services" demos work with this stack

TCPware

  1. a third-party stack from Process Software (only supports IPv4)
  2. this was the only IP Stack for VMS in the 1980s and first half of the 1990s
  3. the Telnet API and FTP API provide a solution midway between Sockets and System Services
  • TCPware_telnet_sample.bas
    • Employs the TELNET API to demo CHARGEN, ECHO, HTTP, and TELNET. Also contains a minimal NVT handshake necessary to telnet to UNIX systems (caveat: Solaris will not let you connect unless you can provide a TERM TYPE via IAC WILL-WONT-DO-DONT; makes me wonder if this isn't some sort of simplistic anti-hacking effort)
  • TCPware_pop3_demo.bas
    • Employs the TELNET API to demo connecting to a POP3 server to retrieve email
  • TCPware_smtp_demo.bas
    • Employs the TELNET API to demo connecting to an SMTP server to send email
  • TCPware_ftp_sample.bas
    • Employs the FTP API to demo doing FTP from within a BASIC executable

Web Demos

Apache Glue Example

  • read_html_apache.com
    1. drop this DCL script into any one of your apache user-accessible scripts directories (I use "[.scripts]")
    2. rename the file to match the URL you wish to use. For example, if the link is "/demo123" then rename the file to "demo123.com"
    3. modify the script to run this next program called "read_html_apache.exe". It will read/decode the received HTML page then create DCL symbols representing the contents of HTML objects on the web page.
    4. if the program ran properly, now run your desired program which will read the DCL symbols then do whatever you want
    5. use a browser to test your application like so:
          http://node-spec/scripts/demo123
  • read_html_apache.bas
    • Have the script above run this program which will:
      1. read data sent by the browser to the web server
      2. decode the HTML page
      3.  create DCL symbols representing HTML data contents of HTML objects on the web page

Apache No-Glue Example

  • apache_demo.bas
    • This stand-alone application does not require any glue (DCL scripts). You could do one of the following:
      1. create a one-line DCL script which does nothing other than run your executable program
      2. drop the executable program directly into the Apache scripts directory

Password Change Demo

  • www_password_change.bas
    • CGI Glue
      • can be used standalone (does not use CGI glue)
      • can be used not standalone (see Apache Glued Example above)
    • can hook into an external database (RMS in my case) to validate users
    • demos how a small amount of JavaScript (embedded in the BASIC source) can be used to pre-validate the HTML form then enabling/disabling the submit button
    • be sure to see my SYSUAF hack on this same page

Upload Demo

  • coming soon
    • Got this working in production but it needs to be cleaned up before publishing
    • update: it is so ugly that you might be better off using this C function which can be called from BASIC

SOAP Demo

AJAX Demo

  • coming soon
    • but here is a little taste of only the JavaScript file here: ajax_vdsl_port_mtce.js
      • AJAX (Asynchronous JavaScript and XML) is a neat, although poorly understood, client-server technology used in conjunction with DOM-compliant browsers.
      • AJAX allows a browser to fire JavaScript function XMLHttpRequest to issue an HTTP request on a secondary channel behind the current web page. More JavaScript is then employed to inject data from HTTP response into the DOM.
      • AJAX can send back ANY text document. Plain, XML, HTML, JavaScript, JSON, whatever.
      • Back in the day, I wrote many VMS-BASIC applications which employed FMS (forms management system) forms on character cell terminals like so:
        Desired Action BASIC Code Notes
        display a form call fdv$disp( fname$ )  
        populate a form (field-by-field) call fdv$put( last_name$, "f$last_name" )  
        populate a form (all at once) call fdv$putal( whole_form$ ) must be a mapped string
        read a form (field-by-field) call fdv$get( last_name$, terminator%, "f$last_name" )  

Data conversion / manipulation

  • caveats:
    • programmers in 2016 are still confusing UTF-8 with Unicode and this is causing huge problems
      facts:
      • UTF-8 is a Unicode encoding
      • Microsoft states that all written languages can be represented by 16-bit Unicode but this will require 3-octet UTF-8 (the spec prefers octet to byte)
      • Both MySQL and MariaDB will set aside 24-bits if you declare the data to be UTF8
      • Both MySQL and MariaDB will set aside 32-bits if you declare the data to be UTF8mb4
    • programmers in 2016 are still interchanging iso-8859-1 with cp1252 (code-page-1252 a.k.a. Windows-1252)
      facts:
      • cp1252 is a 32-character superset of iso-8859-1 so browsers treat both declarations as cp1252 (no visible errors when any of the additional 32-characters accompany a page declared to be iso-8859-1) 
      • Both MySQL and MariaDB require you to declare the character set as latin1 which is an alias for cp1252
    • some older programs below were developed in 2002 when we were attempting to translate Microsoft-generated emails -AND- incorrectly assumed we will only see 2-octet UTF-8. If you want to represent all European characters (both East and West) then you will need to support 3-octect UTF-8. It you have any contact with Asian supply chains (think Alibaba) then you will need to support 4-octet UTF-8
  • Unicode and UTF-8
    • overview:
      1. When you think about it, a string is nothing more than an array of bytes. This next routine (mixed-to-unicode) assumes the input data is mixed (ASCII, ISO-8859-1, cp1252, or UTF-8) then performs a best-effort conversion into an array of 32-bit integers representing raw Unicode (you can set an option to throw away anything that doesn't appear to be legal).
      2. After that, you can call another routine to get the data you wish
    • mixed_to_unicode_array.fun
      • this external BASIC function assumes the input data contains a mixture of UTF-8 and CP1252 (which is a superset of ISO-8859-1)
        • comment: this should not happen but I have seen UTF-8 mixed with cp-1252 coming from Microsoft database engines like SQL Server
      • anything between x00 and x7F is converted to 32-bit Unicode as-is (obviously)
      • anything between x80 and xFF is assumed to be a mixture of UTF-8 and CP1252
        • everything which appears to be a valid UTF-8 sequence is converted to 32-bit Unicode
        • any remaining stuff between x80 and x9F is assumed to be CP1252 which is mapped back to equivalent 32-bit Unicode code points
        • any remaining stuff is assumed to be ISO-8859-1 and so is converted to 32-bit Unicode
    • unicode_array_to_cp1252.fun
    • unicode_array_to_utf8.fun
      • this external BASIC function converts from 32-bit Unicode to UTF-8
      • use this function to send mixed data to any jQuery widget which almost always requires utf-8 even though the hosting web page desires character set "windows-1252"
    • unicode_testing_bas.html
      • this demo program shows how to use the functions just described
  • ebcdic_ascii.bas
    • convert EBCDIC to ASCII
  • base64 conversions:
    code description
    base64_demo_bas this demo program invokes the next two stand-alone functions
    base64_encode_fun function: base64 encode
    base64_decode_fun function: base64 decode
  • sort_demo.bas
    • calling VMS sort routines (sometimes it is best to defer to the experts)

Miscellaneous Stuff

Command Line

  • foreign_cmd.bas
    • allows a BASIC program to read DCL command-line parameters like C programs do on UNIX

DCL symbols

DECnet Communications

Device Scan

Diffie-Hellman Key Exchange Demo

  • diffie_hellman_demo.bas
    • Diffie-Hellman Key Exchange Demo (limited to 64-bit math)
    • note: this happens at the start of every encrypted data transmission on the net

Displaying System errors

  • display_system_error.bas
    • can translate any system error number into the associated text message. This program is no longer required since modern OpenVMS versions can do this for you from the DCL prompt.

Health Monitor

Lock Demos (CEF - Common Event Flag)

Lock Demos (DLM - Distributed Lock Manager)

  • dlm_demo_100.bas
    • creates a null lock then synchronously converts it to more restrictive forms
  • dlm_demo_101.bas
    • creates a null lock then synchronously converts it to more restrictive forms
  • dlm_demo_102.bas
    • asynchronously requests an exclusive lock then polls DLM to see if it was granted
  • dlm_demo_103.bas
    • asynchronously requests an exclusive lock, then arms a timer, then waits for one of two event flags

Linked Lists and Memory Queues

  • mem_que_demo.bas
    • demos: lib$insqhi, lib$insqti, lib$remqhi, lib$remqti but these is better done in C/C++

Mailbox Demo (Unix people use the word 'pipe')

  • advocate_client.bas
    • client program to send DCL requests to a server program via a VMS mailbox; note: a VMS mailbox is similar to, but not identical to, a UNIX PIPE
  • advocate_server.bas
    • server program to receive DCL requests from a client program (via a VMS mailbox)
  • advocate_DCL_scripts
    • scripts to start the programs above

When you need larger DCL symbols

  • multi-line_logicals_demo.bas
    • Some apps like Apache save session data (like HTTP_COOKIE) as a DCL symbol. When the data is greater than 930 bytes, Apache will save the data as multi-line logical names. This program demos how you would do something similar from BASIC

      Overview:
      1. all operating systems support shell variables known as symbols
      2. VMS (and OpenVMS) also support a different data-passing method known as logical names
      3. programs wishing to create or modify logical names will need to call lib$set_logical or sys$crelnm
        Caveat: lib$spawn will allow you to make changes on copied data which is discarded upon exit of that spawned task
      4. While everyone knows how to create single line logical names from BASIC, multi-line logical names are a little more difficult

Password Stuff

Peek

  • peek_demo.bas
    • Unlike most other BASIC implementations, VMS-BASIC doesn't have a PEEK function but sometimes you need it
  • peek_demo_test_run.bas
    • see how variables are stored in an OpenVMS system
    • see how the Alpha compiler is able to achieve such high levels of performance relative to VAX
  • demo_decimal_hack.bas
    • see how fixed-point decimals are stored in OpenVMS and RMS (no difference by the way)

Population Simulation

System Calls you can't live without

  • sys$getjpi
    • getjpi_demo.bas
      • demos how to properly call sys$getjpi (get job process information)
  • sys$setddir
    • show_default_demo.bas
      • demos how a program can do the equivalent of the DCL command "$show default"
      • comment: anyone coming from RSX-11 or CP/M will remember that using SET without any parameters is the same as SHOW
  • sys$getrmi
  • sys$qio
    • qio-demo.bas
    • calling sys$qio from BASIC allows more control than PRINT and INPUT

Source Code Tools

  • source_code_reporter.bas
    • generate "code change" reports (from change-area comments) to justify your existence
  • source_code_formatter.bas
    • This horribly written quick-hack has proved invaluable as a tool for locating BASIC coding problems introduced by other authors.
      • Anyone who has inherited source code from another organization knows what I am typing about.
    • Caveat: I've been meaning to rewrite this tool for a decade now but can never find the time. But this tool can format itself so maybe it's good enough for occasional use

Sequencer

  • sequencer.bas
    • When run from a DCL script, this program goes great lengths to ensure that your process, and only your process, gets the next number in a numerical sequence. I use this program so that an intercepted email account can temporarily access one of twenty subdirectories before I call MUNPACK to extract MIME attachments (if any) from cell phone messages. Of course, I need to run a different program to read the plain-text email if nothing was extracted

Terminal Characteristics

Timer Demos

VMS Mail

Watchdog

  • watchdog.bas
    • my own watchdog process (demos lots of systems calls)

Webification

OpenVMS BASIC standalone functions (file extension ".fun")

Some BASIC demos may require these standalone (mostly BASIC) functions:

  • wcsm_calc_easter_n_good_friday.fun
    • compute Easter or Good Friday for any date after 1582. Necessary for writing an annual holiday generator.
  • wcsm_dt_stamp.fun
    • returns the current date-time in ccyymmddhhmmss format
  • wcsm_get_mime_time.fun
    • mime time function for smtp use
  • wcsm_get_symbol.fun
    • returns the value of a desired DCL symbol
  • wcsm_peek_quad.fun
    • returns the 64-bit data work found at the specified 32-bit address
  • wcsm_set_symbol_or_logical.fun
    • this function will attempt to save your data as a DCL symbol. If it can't (usually because the data is too large for your machine) then it will shift modes and save your data as multi-line logical names (emulates functionality found in the CGI interface of Apache for OpenVMS)
  • wcsm_submit_to_batch.fun
    • submitting to batch from within BASIC
  • wcsm_trnlnm.fun
    • translates a logical name
      caveat: we used this function in the VAX days because it was much more efficient than calling lib$get_logical and easier to set up than sys$trnlnm. But there seems to be no performance hit when using lib$get_logical on Alpha so this  function is only included here incase I've included a few demos still calling it.
  • wcsm_url_decode.fun
    • undo so-called percent encoding (as well as some other stuff)

Question: So just what does the prefix WCSM stand for?
Answer: Waterloo Computer System Maintenance. That department name morphed into something else but the prefix remains smile

OpenVMS BASIC includes (file extension ".inc")

Some BASIC demos may require these standalone includes:

  • vms_structures.inc
    • data structures required for item lists in system calls, etc.
  • vms_externals.inc
    • system declarations required to do system calls from BASIC (should not be used for new code development)
  • device_controls.inc
    • escape sequences required to embed VT terminal and LA printer inside your programs. (written for use on small VAX systems where it was not practical to control devices by system calls to the terminal driver)
  • fms_fdvdef.inc
    • FMS (forms management system) definitions: functions and constants
  • tcpware_ccb_definitions.inc
    • TCPware control block definitions

My BASIC Development Environment

A snapshot of my development environment. It will help you understand my how I refer to includes and standalone functions

Directory File
Extensions
Notes
[.dvlp] .bas
.com
basic source code
DCL scripts for building some applications
[.dvlp.flb] .frm
.flb
FMS forms (created by dcl command: $fms yada.frm)
FMS form libraries
[.dvlp.fil] .rec
.opn
.fdl
mapped string declarations (records) for relative and indexed files
open statements for relative and indexed files
file description language files for tuning rms files
[.dvlp.fun] .fun basic functions
[.dvlp.inc] .inc basic include files
[.dvlp.mar] .mar
.m64
macro-32 files
macro-64 files
[.dvlp.sql] .sql MySQL or MariaDB scripts

BASIC Compiler bugs

Overview: this all started when I was trying to write a limited Diffie-Helman demo in BASIC. Why limited? I knew before I began that I wouldn't have access to the whole key space because BASIC does not natively support unsigned integers. But VMS-BASIC-1.7 does support a QUAD datatype which weighs in at 64-bits so that should have gotten me part way (a complete Diffie-Helman demo would require 512-bits, 1024-bits or more). But then I found the following problems with quad.

basic_quad_mod_bug.bas

  1. I just (2012-08-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
  2. Although you can divide large QUADWORD integers, the modulus function fails
  3. Here is a "C" version which works properly: basic_quad_mod_bug.c
  4. I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem
    Version Notes-1 Notes-2
    1.7-000 test fails at 2^32 bug
    1.7-001 test works up to 2^62 fixed
  5. CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support contract and want it fixed then you must input a support request"

basic_decimal_mod_bug.bas

  1. I just (2012-09-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
  2. Although you can divide large DECIMAL integers, the modulus function fails
  3. I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem.
    Version Declaration Notes-1 Notes-2
    1.7-000 decimal(16,0) test fails at 2^32 bug
    1.7-001 decimal(16,0) test works up to 2^53 fixed
      decimal(17,0) test works up to 2^56 with compile warnings safer to use quad
      decimal(18,0) test works up to 2^59 with compile warnings safer to use quad
      decimal(19,0) test works up to 2^62 with compile warnings safer to use quad
  4. CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support contract and want it fixed then you must input a support request"

BASIC RTL (run-time library) bug

  • This affects OpenVMS-8.4 ECO 900 (a.k.a. consolidated patch kit 900 and earlier)
  • In BASIC, you can use the open statement to create an indexed file with one or more keys.
  • If any one of the keys is a QUAD (signed 64-bit integer) then any attempt to reopen the file will fail
  • I placed a support request with HP (OpenVMS Engineering) who provided me with an experimental BASRTL 10-days later.
  • Not sure if this will be released to the public so if you need it, PLACE A SUPPORT REQUEST with HP.

p.s. I was attempting to write code which would directly open SYSUAF which contains a key (a.k.a. index) based upon an unsigned quadword. Since VMS-BASIC has no unsigned data types an updated BASRTL would be of no use. Even still, I am surprised to learn that this bug was hanging around since 2005 when HP added the quad data type to the language (up until then you needed to use basic$quadword defined in the STARLET library)

BASIC bug or oversight?

I just (2020-09-03) noticed this problem with VMS BASIC-1.7 (Itanium) but with no support contract this is the only place I can complain
1000	option type=explicit
	declare string	i$	,&
		long	i%
	input "array size? ";i$
	when error in
	    i% = integer(i$)
	use
	    i% = 10
	when
2000	dim single		s(i%)
	dim double		d(i%)
	dim string		s$(i%)
	dim decimal(14,2)	de(i%)
	!
	!	use the matrix command to init these arrays
	!
3000	mat s = zer	!
	mat d = zer	!
	mat s$ = nul$	!
	mat de = zer	! this line will not compile
	!
4000	s(0) = 12.3	!

DEC-C  VMS-C  DEC-CXX VMS-C++  VMS-CXX

General

Caveats for developers:
  1. both C and C++ languages "appear" to be more complicated on OpenVMS than what you will see on UNIX, Linux or Windows so be sure to read OpenVMS specific documentation.
    1. C on OpenVMS
    2. C++ on OpenVMS
      • http://h41379.www4.hpe.com/commercial/cplus/cplus_index.html
      • just what the heck is "cxx"? This is another example of Dilbert humor where the plus signs of "c++" are rotated 45 degrees. This was necessary because the VMS operating system did not allow plus signs in file names. This means that ".cpp" is also supported.
    3. DEC-C Run-Time-Library Reference Manual For OpenVMS which is used by both C and C++
      •  http://h41379.www4.hpe.com/doc/732final/5763/5763pro_001.html
      • tells you where your reference libraries are located
      • compares UNIX I/O to RMS I/O
      • introduces optional library directives which will allow UNIX-dependent c programs to run on OpenVMS
        e.g. automatically translate a full filename specification from "//one/two/three/four.txt" to "[one.two.three]four.txt"
    4. Many people today only purchase a C++ license then use it to compile C programs
    5. One definition of Sensei (a  Black Belt  in Karate) is "one who has gone before". Many times you will attempt to solve a problem which has already been solved by someone else long ago but you didn't know where to look. Some of those solutions can be seen here:
  2. Two compilers? Many programmers claim since c++ was developed to be a superset of c then will point to numerous articles where programmers use a c++ compiler to develop better c programs. That said, I can show you examples where original developers tell third-party programmers to not use c++ compilers when building their APIs. OpenSSL is one example that springs to mind.
    • some features of c++ have been back-ported to c (using "//" to delimit comments is one obvious example)
    • however, a lot of new and interesting features added to c++ have not been back-ported to c for fear of breaking something.
    • both c and c++ compilers from HP/HPE look for these two meta data files...
      __DECC_INCLUDE_PROLOGUE.H
      __DECC_INCLUDE_EPILOGUE.H
      ...when loading language header files on a per directory basis. These files can modify many things including pragma statements (including case sensitivity of exported symbols) so please read chapter 1.7 of the C User's guide or chapter 9.5 of the CXX User's guide.
       
  3. When calling OpenVMS routines defined in the STARLET library, do yourself a favor and enable strict type-checking. This is done by inserting this statement before the library references you wish to be strict (for new programs it should be before all library includes).
    #define __NEW_STARLET 1
    Doing so will endure your code will be in better shape to be ported to another OpenVMS hardware platform. In one of the programs below (sys_create_uid_101.c) I was forced to add a few more lines (resorted to a union) to satisfy the compiler but the optimizer was able to generate a program which was byte-for-byte identical to an earlier relaxed version.
  4. Some large projects (especially code originally written for UNIX/Linux) may require you to build your program with a so-called "make utility". MMS (module management system) is the name of the official licensed make tool. Developers without an MMS license are advised to use MMK which is the open source freeware version available here:
    1. http://github.com/endlesssoftware/mmk
    2. http://www.decuslib.com/freeware/freewarev80/make-mmk/
    Additional Advice:
    1. many old-school VMS programmers prefer to invoke the compiler and linker from a DCL prompt. When their DCL commands become too long they will often move them into a DCL build script. While there is nothing wrong with this, older applications like "$LINK" do not support DCL symbol substitution after the "sys$input/opt" directive which then forces the developer to resort to temporary "logical names" or generating-then-executing temporary scripts. Even then, you will have some difficulty squelching a noisy link operation (lots of warnings but still a successful build). All these problems can be handled by moving to a make tool like MMK
    2. We had been using MMK-4.1 for more than a decade on Alpha with no difficulties. When we moved from Alpha to Itanium in 2014, I acquired an Itanium version of MMK-4.1 which did not always work properly with some open source offerings like mtools. Upgrading to MMK-5.0 fixed those problems
      comment: years ago, one of our developers attempted to use MMK-5.0 on Alpha but it proved problematic so we stayed with MMK-4. Do not jump to a new version of MMK unless forced to do so.  
  5. GNV (Gnu Not VMS) hosts a Unix-like environment on OpenVMS: gnv.sourceforge.net
     
  6. Lots of open-source software has already been ported to OpenVMS: vms-ports.sourceforge.net

C Demo Programs

File Notes
Hello World
hello_world.c every website has one of these
hello_openvms_world.c introductory "hello world" program for OpenVMS
hello_openvms_world_more.c added a few system calls to the previous program
 Where the rubber meets the road
chmod_demo.c changing VMS file protection bits
credit_card_number_test.c employs the Luhn Algorithm to pre-validate credit card numbers
(as well as some other stuff including Canadian Social Insurance Numbers, etc.)
cmd_line.c reads command line arguments (and named logicals and/or DCL symbols)
descriptor_macro.c the $DESCRIPTOR MACRO is dumber than most people realize so use it with caution or not at all
diffie_hellman_demo.c Diffie-Hellman Key Exchange Demo (limited to 64-bit integer math)
dns_lookup_demo.c A very-cool LINUX socket programming to do Name Server lookups. It was not written by me but it
works as-is on OpenVMS-8.4 Alpha with "TCPIP Services for OpenVMS v 5.7". It won't work from
behind a firewall (without a few mods) but you'll get the basic idea.
Caveat: be sure to see RES_SEARCH_DEMO.c below which may be a bit more useful. Why? Library functions res_search() and res_query() work properly with IPv4 -and- I assume they also work properly with IPv6
do_100k.c This little test file came from OpenVMS Engineering (I kept it as an homage to them) when we
reported a 100k problem with CSWS 2.0 (Apache for OpenVMS). Drop this file into the scripts
directory of your Apache server then trigger it from a browser like so: http://nodename.ext/scripts/do_100k It will help you test your Apache server as well as assist in confidence tests with your network
admin people.
ebcdic-ascii-demo.c convert between EBCDIC and ASCII character sets
file_upload_neil_c3.html this is the webpage used to start the file-picker
flock_demo.c
  • POSIX C provides lockf
  • BSD/Linux C provides flock
  • the CRTL library published for OpenVMS by HP in 2005 contains neither function
    so you may need to rely upon the ANSI functions flockfile and funlockfile
    -OR-
    do specific OpenVMS system calls to the distributed lock manager.
    comment:
    in a VMS Cluster the distributed lock manager is the only way to go.
    I haven't worked in a clustered environment for more than a decade -AND-
    found the flock method so easy to use on Linux that I hope VSI (VMS Software Inc)
    adds support for it ASAP. Heck, it may already exist in their x86-64 product.
    comment2: I've produced a working demo for OpenVMS which employs flockfile
    but is is so ugly that I won't post it here.
getuai.c demos use of getuai (get user authorization information)
http-check.c HTTP connection test-tool from Hunter Goatley (Process Software)
http_file_upload_31.c
(broken when uploading
certain binary files; see caveat)
this is a c version of my BASIC code (it can be built as either "a standalone executable
program" or "a function which can be called from any so-called DEC language including BASIC)
caveat: on 2016-03-xx I realized this program contains a small bug when saving uploaded binary files to OpenVMS. With regard to RMS files of the "512-byte fixed" variety, this broken program appends nulls after the last data byte in the last block then places an EOF at the end of the block. Most programmers already know that many files (with extensions like ".exe" and ".xslx" etc.) expect to see an EOF after the last data byte in the last block. I fear the only way around this is to switch from "standard-i/o functions like fprintf" to "OpenVMS functions like sys$put and sys$write". If anyone is aware of a simpler way only using c then please send me an email. Anyway, a solution is just around the corner.
solution: I now have a working version (this is just a spare time project) but I have not gotten around to publishing it
ldap.h
ldap_example.c
ldap_example_hack.c
ldap_example_hack_authenticated.c
Lightweight Directory Access Protocol
lib_mysqludf_build_vms.com DCL script to build lib_mysqludf_sys on OpenVMS (user defined function to do OpenVMS system calls from within Stored Procedures) on MySQL and MariaDB
note: this script shows how to force the OpenVMS linker to produce both uppercase and
lowercase symbols
mysql_demo01.c a "hello world" demo for MySQL and MariaDB
mysql_demo02.c
mysql_demo.ini
doing some real world work with SQL
an optional parameter file for use with demo02 and higher
mysql_demo03.c connecting via SSL
mysql_demo04.c demonstrates how to import data from RMS to MySQL/MariaDB
caveat: since I wrote this program I've had better luck working with the client CLI
mysql_demo05.sql playing with SQL (creating tables, displaying execution plans, etc.)
mysql_demo.com simple build script (produces the fastest executable)
mysql_demo_alt.com complicated build script (can link to a mixed-case shareable SSL library)
pipe_demo_parent.c this program creates 3 pipes before calling vfork
pipe_demo_child.c this C program is executed by the vfork
pipe_demo_child.com this DCL script is optionally executed by the vfork
popen_demo.c
system_demo.c
two more ways to execute a DCL script from c
pointer_demo.c  
quad_mod_demo.c playing with quads (long long) in "C". This program was used to verify some
problems with HP Alpha BASIC 1.7-000
res_search_demo.c As of October-2014 I think this is the only reasonably complete OpenVMS demo
on the internet showing how to use
res_search() and res_query() then extracting A, MX and NS records. If anyone knows a better/easier way to to
extract this information on OpenVMS (perhaps using library functions I don't know
about) then please send me the details.
Comments:
  1. Many library functions are available for Linux (and I was tempted for a short while to begin porting a half dozen or so but why reinvent the wheel if others have gone before me? I certainly couldn't be the first OpenVMS application programmer standing before this chasm, right? Also, the responsibility of keeping the C/C++ libraries up-to-date squarely lays with HP
  2. Anyone who has viewed the official ISC source code for nslookup.c or dig.c might be wondering WTF? I'm certain they are kept remark-poor to thwart modifications by non-specialists. Meanwhile, I am certain that professional spammers already know how to extract MX records.
rms_c_ug_example.c This program is an annotated version of RMSEXP.C found in chapter 2 of the HP C User's Guide.
Any newbie wishing to use RMS from C or C++ should first look at this program before you write a single line of code
sequencer.c a tiny little app for selecting working folders on OpenVMS (coming soon)
smtp-check.c SMTP connection test-tool from Hunter Goatley (Process Software)
sys_sndjbcw.c sndjbc - submitting a job to a batch queue (this will enable you to submit a job on behalf of another user for the express purpose of having that user send a file to another system via SFTP or SCP)
sff_demo.c demos function tcpip$send_from_file() which is a nifty way to send SMTP messages to your own stack without opening a connection to port #25
sudoku.c to benchmark hardware/software performance
sys_create_uid_100.c generate a UID (with a tip of the hat to Jim Duff) caveat: need a 1 ms delay between successive calls for true uniqueness
sys_create_uid_101.c generate a GUID (with a tip of the hat to Stephen Hoffman) caveat: need a 1 ms delay between successive calls for true uniqueness
tcpip$tcp_client_qio.c my presentation of a client demo program found in vms folder sys$examples (copyright by HP)
notes:
  1. this program uses sys$qiow rather than Berkeley sockets and is synchronous
  2. a synchronous version (based on sys$qio) will be published soon
timezone_demo.c display the current time translated to any other time zone
unix_hater.c does the "C" example in the UNIX Hater's Handbook actually compile? (yes)


vt340-color-demo.c display colored text on real VT terminals (or any good emulator) without SMG$
dec-device-control-codes.c this file contains escape sequences used to control popular DEC peripheral devices
vt-smg-demo.c note: SMG$ is one of five OpenVMS RTLs. This limited demo employs SMG$ routines to display colored text on a virtual terminal
vms_lock_dlm.c Accessing the OpenVMS DLM (Distributed Lock Manager)
utf8_decode.c no matter how you configure them, most popular browsers will push Unicode via UTF-8 back to your programs. This will cause a problem if your databases are configured to store single characters so you will want to convert from UTF-8 to ANSI/Windows-1252 (which is a superset of iso-8859-1). But what happens when you are unable to convert everything? Do you throw away anything over 255 or do you map to something else?
utf8_encode.c this program can convert from ANSI/Windows-1252 back to UTF-8
internationalization_demo_100.c demos converting from "iso-8859-1" to "ucs-4" (calls: setlocale(), iconv_open(), icon_close(), and iconv())
internationalization_demo_101.c demos converting from "iso-8859-1-euro" to "utf-8" okay for ISO-8859-1 to UTF-8; not okay for Windows-1252 (also called ANSI) for this reason alone, I will keep these Internationalization demos close to my UTF-8 routines

C++ Demo Programs

File Description
hello_world.cpp every site has one
string_demo.cpp This string demo is very interesting. Why?
  1. at first glance these C++ strings appear much more BASIC-like than those found in traditional C. But this syntax seems closer to what you see in Java, JavaScript, or C#
  2. Many traditional third generation languages, like HP-BASIC for OpenVMS, limit string size to 32,767 bytes. I have tested this C++ demo up to a string size of 1,999,000 bytes
    (I could have gone higher but became bored) and found that it worked equally well with:
    1. "HP C++ V7.3-009 for OpenVMS Alpha V8.4" (complied and linked in 32-bit mode)
    2. "CodeBlocks 12.11 on Windows 7"

Apache Specific (c-only)

Apache Module Authentication

Overview
  • In 2014 we transitioned 100 users from green-screen to the web. Our web applications do their own authentication by hooking into SYSUAF then setting an encrypted session cookie to be used for the next hour:
    • we record every transaction and can "log out" inactive web users by deleting their specific cookie from our "current sessions" database
    • this database is also used to implement an administrative dashboard
  • The negative here is that old-school access to documents stored in protected directories still require a second authentication challenge from mod_auth_openvms which cannot be checked for inactivity -AND- doesn't set a cookie. Our clients are becoming really annoyed at this.
  • The success of our transition from green-screen to the web has resulted in the creation of new business opportunities resulting in ~1300 new users. However, the usual account churn has become somewhat of a burden so we would like to transition to self-serve account maintenance (think Amazon or eBay). This could done in a simple secure way if all user accounts were moved from SYSUAF to an application-level database (which is an indexed RMS file in our case). This also means that only five people (developers and sys-admins) would ever have SYSUAF accounts. Very cool!
  • So I copied the password hashes of our user accounts from SYSUAF to our user profile database but still needed to keep ~100 accounts in SYSUAF to be authenticated via mod_auth_openvms. This means doing password changes in two places -AND- not allowing full self-serve account maintenance on these. (well I could but then how would I sleep at night knowing someone might stumble on a way to hack the system?)
  • So we need to replace mod_auth_openvms with something more useful; something that supports session cookies
Hacks
  • I started with mod_auth_openvms.c as my template but first needed to learn how to build it or there would be no point in continuing
  • I downloaded a UNIX distribution of httpd-2.0.63 from Apache with the intent of only building part of it (the authorization plugins) on OpenVMS
  • Then I realized APR (Apache Portable Runtime) was a prerequisite which needed to be downloaded then built separately. I wondered if this would be best done with a MAKE file under GNV or by some other method
  • So I bounced the MAKE/GNV questions off people at comp.os.vms who suggested downloading CSWS sources from HP then linking against the Apache run-time which is already present (who knew?)
  • CSWS source kits can be downloaded from HP's website via anonymous FTP
    • ftp://ftp.hp.com/pub/openvms/apache/ (works from IE-11)
      1. note that CSWS-V20-2 is newer and larger while CSWS-V21-1 is older and smaller (???)
      2. my examples use CSWS-V21-1 (older and smaller)
    • note: the official build instructions in chapter 5 of manual csws_install_22.pdf are broken; see my scripts below for one possible solution
  • caveat: my original programs were pure hacks based upon peeking into the ".h files" included in the kit. Since then, I have learned my time could have been reduced by 80% had I first purchased a copy of Apache: The Definitive Guide published by O'Reilly. A complete list of API routines (like ap_get_basic_auth_pw) can be found in the appendix. Professional module writers may also want to check out Writing Apache Modules with Perl and C
Source Code
  • mod_auth_openvms.c
    • this module was published by Compaq before the merger with HP (now HPE)
    • This 2015 production file hasn't changed since 2003-01-15 (presented here for reference purposes)
    • Comments: with the benefit of 12 years of hindsight
      • Parts appear to be derived from a Solaris program written before y2k (keep in mind that Solaris usernames are case sensitive)
      • as written, this plugin is very inefficient. It contains two hooks:
        • hook-1 will fire for every transaction "requiring Apache authentication" in the whole system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been defaulted to DECLINE. When enabled, will access SYSUAF
        • hook-2 will fire for every transaction "requiring an Apache authorization" in the whole system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been defaulted to DECLINE. When enabled, will search for the current USERNAME in an external GROUP FILE.
      • It might have been a better idea for Compaq to operate both routines under a single hook which would make the plugin slightly less parasitic. Alternatively, they could have published two plugin modules: "authentication only" and "authentication and authorization"
      • Another problem with the code under hook-2 is that the username test is case-sensitive. OpenVMS usernames are up-cased before going through SYSUAF authentication. This means that the GROUP FILE associated with hook-2 needs to contain both upper and lower versions of every username but someone logging in with a mixed-case username will still be bounced.  
      • the first two "c language" tests near the top of hook-1 are in the wrong order. This plugin could be made a tiny bit less parasitic if the very first line of code was the test of fUserEnable rather than a call to ap_get_basic_auth_pw (why test for a password in places where the module is disabled?)
    • my DCL build script: build_mod_auth_openvms.com
  • mod_auth_vms_ext.c
    • it works with cookies, and/or username+passwords
    • it can (optionally) engage an indexed RMS-based "session cache" so the external authenticator is called less often (I do not think it would be too difficult to move this into a "global section" -- also called a shared memory object)
    • it spawns an external authenticator (which can connect to any kind of database)
    • it can command a browser to raise the username-password dialog
    • it can (optionally) command the browser to store a COOKIE (cookie data must be provided by your authenticator)
    • it can (optionally) do GROUP access restriction tests (group data must be provided by your authenticator)
    • DCL build script: build_mod_auth_vms_ext.com
  • mod_auth_vms_rms.c
    • opens an indexed RMS file to perform application-level authentication (can use characters not allowed in SYSUAF)
    • essentially identical to mod_auth_vms_ext.c (above) except that lib$spawn was replaced by directly accessing an RMS-file
    • DCL build script: build_mod_auth_vms_rms.com

gSOAP Specific (c-only)

When you check out the sample programs delivered with the gSOAP stack they seem fairly straight forward (right now I'm thinking about "calc") but most were created using the WSDL-last method (a.k.a. code-first method). When someone hands you a WSDL then says "build client and server apps which will be part of a two way messaging system" things get complicated fast. You will be forced to use the WSDL-first method, and the tool wsdl2h (read as: wsdl-to-h) will stuff your SOAP variables deep inside complicated structures and unions then pass the whole mess to you with a pointer. If the WSDL contains more than 100 fields like the one I saw a few months ago, you might want to tackle a smaller project first just to develop skills.

Last year I used the WSDL-last method to create a Java-based web service under Tomcat/AXIS2 called "SimpleService" which can be viewed here. This service only had three methods: ssADD, ssSubtract, and ssEcho. Then I used AXIS2 to generate a WSDL which would be used to build this gSOAP service with the same functionality.

gSOAP demo web service
simple_service.wsdl
simple_service_client_100.c
simple_service_server_100.c
simple_service_server_101.c
simple_service_wrapper.c
build_simple_service_step1_common.com
build_simple_service_step2_server.com
build_simple_service_step3_client.com

this WSDL was generated by AXIS2 (using: http://www.site.com/service?wsdl)
this is a simple gSOAP client (add, subtract, echo)
this is a simple gSOAP service (add, subtract, echo)
this is a simple gSOAP service with a few more bells and whistles
the service requires this wrapper if you want to access it from Apache
common build script (DCL)
service build script (DCL)
client build script (DCL)
gSOAP demos (other)
calcclient.c
dom2_hack.c
this gSOAP sample client has been tweaked to support OpenSSL (demonstrates three different SSL client contexts) doesn't do anything useful other than traversing a DOM-loaded SOAP packet
(needs more work; would like to produce a better version in C++)
gSOAP reference
mod_gsoap.c
plugin code to allow Apache access to gSOAP (problems highlighted in yellow) note: do not use this with OpenVMS (Brett Cameron has rewritten this plugin for his gSOAP distro)

gSOAP Specific (c++)

Brett Cameron and John Apps have provided a really good set of gSOAP sample programs written in VMS-C but nothing (as of 2012) written in VMS-C++. So here are my unofficial offerings of sample programs tweaked for use with VMS-C++ for OpenVMS.

Notes:
  1. Large gSOAP programs written in C++ seem clearer than the same ones written in C. Consider using C++ if you you want your gSOAP code to me more easily maintainable by other programmers.
  2. When the gSOAP preprocessor "soapcpp2" generates C++ code it will use a ".CPP" extension
  3. VMS-C++ defaults to a file extension of ".CXX" while competing compilers default to ".CPP" so be sure to include a fully qualified file name.
gSOAP Calc Samples
calcClient.cpp
calcServer.cpp
build_calc_cpp.com
CalcSslClient.cpp

gSOAP sample Client program from //gsoap/samples/calc++/calcClient.cpp
gSOAP sample Server program from //gsoap/samples/calc++/calcServer.cpp
DCL script to build these two gSOAP sample programs on OpenVMS
gSOAP sample Client program (w/OpenSSL) from //gsoap/samples/calc++/calcClient.cpp
(this last  demo has only been tested on Windows-7)

Porting OpenSSL to OpenVMS

While it is possible to build OpenSSL from public sources, at least one of the demonstration applications will not work without modifications. The main reason for this involves use of the UNIX I/O select() statement to test for I/O readiness. While this is perfectly acceptable in the UNIX/Linux world, select() can only be used to wait for network I/O in the VMS and Windows worlds. There are two possible fixes for this:

  1. use a different technique to detect the start of keyboard I/O
    • see other platform examples in latest versions of s_client.c where a home-brewed function called kbhit (keyboard hit) is used to detect when the "C" program should begin a keyboard read operation in Windows
    • this is the el-cheapo solution which should be okay for s_client.c since it is not production quality code.
       
  2. use a VMS-recommended technique to use a socket to connect to SYS$COMMAND
    • this solution requires a socket interface at the SSL end and a SYS$QIO at the keyboard end
    • this gold-plated solution can be seen in the program term_sock.c (terminal socket) shown just below
Programs Notes
s_client.c This program was found in kit CPQ-OpenSSL-1.0A (OpenSSL 0.9.6b) published July 2001 by Compaq
It shows how to create a socket (TerminalSocket) to communicate with stdin (your keyboard)
No Compaq/HP copyright was included with this package.
While this program is copyrighted by Eric Young, the VMS changes are not copyrighted by anyone.
"I think" it was modified by someone at Compaq Computer Corporation
term_sock.c This program was found in kit CPQ-OpenSSL-1.0A and is used to create a socket (TerminalSocket) to communicate with stdin (your keyboard). No Compaq/HP copyright was included with this package or this file. "I think" it was written by someone at Compaq Computer Corporation. It allows the programmer to get around the "bad select 38" problem
term_sock.h This c header supports term_sock.c No Compaq/HP copyright was included with this package or this file. "I think" it was written by someone at Compaq Computer Corporation
select_demo.c A demo program (derived from HP's UNIX to OpenVMS porting guide) to show how to use select() 

Webpage Hit Counters

We have always been told to never use hit-counters because they consume precious resources. But many rinky-dink sites, especially some on corporate intranets sitting behind a firewall, need them for good P.R. with other departments.

Image-based Counters

These seem to be the gold-standard in counters. Between 1995 and 2000 (when web servers were powerful platforms serving up mostly static webpages to underpowered desktop PCs sporting Pentium-2 or Pentium-3 processors) most sites used a bit of freeware written by Muhammad Muquit named "count". Needless to say I am envious of his programming skills. But this solution places a more-than-trivial computational burden on the server. Why? Updating a counter file is the easy part. Using the digits to reference a graphics library of individual digits then assemble the binary pieces scan-line by scan-line into a resultant GIF is the hard part.

Text-based Counters

I'm running an overworked decade-old server (AlphaServer DS20e installed in 2002) and I think the time has come to shift the computational burden from the server to the client. I have a little "C" program ready which increments the counter file then sends back the plain-text result. The calling webpage uses a small amount of AJAX (~20 lines) to accomplish the task then the browser is responsible for displaying the count. The browser may not have access to all the cool image sets seen in Muhammad's offering but that may not be as big a deal as you might think. I will publish it soon.

documents:

  • here is the HTML-JavaScript Code to demonstrate my AJAX-based hit counter
  • here is the c-based hit counter code. It requires polishing. it also requires the addition of calls to the OpenVMS distributed lock manager (DLM) to provide better file locking than is possible with vanilla C routines for UNIX. This should be easy to do and I will publish that as soon as time permits.

Web Page Hit Counter (2.4 + 2.6)

  • Muhammad Muquit is the author of a very cool multi-platform 'Hit Counter' which is currently run by many large ISPs in the world (if you don't believe me, just intercept a generated GIF then look inside where you'll see his name and URL in the comments area; his name continually flies around the internet but this is a small prices to pay for the efforts of such a gifted programmer)
  •  Phil Ottewell's code to work with Process Software Corp's Purveyor 2.x Encrypt Web Server and the code is available at here).
    BTW,  I also inserted calls to the OpenVMS Distributed Lock Manager to prevent the counter data file from becoming locked out due to multiple process requiring exclusive write access.
    • If you don't believe this can happen, set up a demo web page with 20 image references to the same counter like so:
      <img src="http://www.your-site.org/cgi-bin/Count.cgi?dd=D&df=hack.dat&sh=1&incr=1">
    • now publish the web page to your ISP so you've got maximum CPU power and network bandwidth
    • now access the web page and watch the counters get displayed in your browser
    • now add another 10 images references and repeat the whole process. I've discovered that the counter data file will become permanently locked out by the ISP when the number of images exceeds 32 (according to www.netcraft.com one site was running Netscape-Enterprise/4.1 over Solaris while another was Microsoft-IIS/4.0 over Windows-NT4). BTW, before my patch, you would never see a permanent lockout on an OpenVMS system; just the odd counter GIF not being displayed. My Distributed Lock Manager routines make sure that every request gets honored so the odd GIF will just pop in a few milliseconds late. No big deal but that is what is so neat about OpenVMS.
  • 2003-05-23: I've got a pre-release of Hit Counter v2.6 for Compaq Secure Web Server version 1.3.26 (a.k.a. CSWS, or Apache for OpenVMS) working on my AlphaServer-DS20
    Files Description
    wwwcount2_4.zip original 2.4 distribution
    wwwcount2_4p.zip includes mods for use with WASD includes Phil's mods for use with OSU includes my mods for use with Purveyor
    wwwcount2_6.zip original 2.6 distribution
    wwwcount2_6c.zip my mods for use with CSWS (Apache)
Technical Observations (2013-11-xx)
  • As currently implemented on OpenVMS under Apache, count.exe will produce two files. File-1 contains "the MIME header text" while file-2 contains "the binary graphic data"
  • In 2013 I shifted my webserver to Ruthless Browser Caching which caused the web-counter image to stop updating (and we still need it for departmental PR)
  • Since I hadn't worked on this code for ten years, I started by rewriting the DCL script to make it smaller and more efficient
  • Next, I replaced the copying of file-1 (the MIME header) with output from DCL. To do this I replaced one image execution (APACHE$DCL_BIN) with another (APACHE$FLIP_CCL) which is still less expensive since we do not need to locate, then transfer, file-1.
  • After that I added a second MIME line ("Cache-Control: no-cache, no-store") to deal with servers implementing Ruthless Browser Caching.
  • It should be possible to modify the C sources to do away with creating intermediate files. The result would be a two-line DCL script. I publish this solution as soon as possible
    Updated DCL script for use with CSWS (aka Apache on OpenVMS)
    
    $ on error then $ continue
    $ t = f$cvtime(,,"TIME") -":"-":"-"."
    $ p = f$getjpi("","pid")
    $ f = f$trnlnm("csws_server_root") + "count_" + p +"_" + t +".gif"
    $ set prot=(S:rwed,o:rwed,g:rwed,W:rwed)/default
    $ define/user/nolog sys$output 'f'
    $ mcr www_root:[scripts]count.exe
    $ m=1 ! DCL
    $ if (m.eq.0)
    $ then ! MIME from a file
    $	APACHE$DCL_BIN 'f';1 ! MIME header
    $ else ! MIME from DCL
    $	APACHE$FLIP_CCL
    $!	write sys$output f$fao("!AS!/!/","Content-type: image/gif")
    $	write sys$output f$fao("!AS!/!AS!/!/","Content-type: image/gif","Cache-Control: no-cache, no-store")
    $ endif 
    $ APACHE$DCL_BIN 'f';2 ! GIF data
    $ del/nolog 'f';*
    $ exit

Hybrid Programming (VMS-BASIC calling VMS-C)

All so-called "DEC languages" (VMS-COBOL, VMS-FORTRAN, VMS-Pascal, VMS-BASIC, etc) can easily pass variables (including strings and arrays) to each other. Passing variables between "DEC languages" and code written in C, C++, or MACRO is a little more difficult but not impossible.

Some BASIC solutions can be anywhere between impractical to impossible (LDAP springs to mind). Here are your options:

  1. "rewrite C programs into BASIC"
    • this will be time consuming if not error prone
    • VMS does not have access to some c-language primitives like unsigned variables
  2. BASIC programs could call C programs indirectly which means passing data  via "shell variables" or "logical names"
  3. BASIC programs could be linked to C programs. For the last method you need to know how to pass variables back and forth

Here are some examples

  • Hybrid Demo 1
    • this BASIC program basic_calling_c_demo1.bas calls C functions defined here: basic_calling_c_demo1_part2.c
    • demonstrates passing variables (long, word, byte, and VMS string) to C; one function returns a result while one does not
    • also demonstrates how C can directly access global variables declared in BASIC (ugh!)
    • caveat: the string can only be accessed in C as read only; see demo-4 below to learn how to properly modify the BASIC string from C
    • here is the DCL build script for demos 1-5: basic_calling_c_demo.com
  • Hybrid Demo 2
  • Hybrid Demo 3
  • Hybrid Demo 4
    • this BASIC program basic_calling_c_demo4.bas calls a C function defined here: basic_calling_c_demo4_part2.c
    • a variable length VMS string is passed from BASIC to C where the string is extended then sent back
    • teaser: new string storage space must be allocated via str$get1_dx rather than malloc
    • hacking: write a small BASIC program which only manipulates a few strings then compile it with the "/list/machine" command-line parameters. For the most part the BASIC program will be making calls to the STR$ run-time library
  • Hybrid Demo 5
    • if you have ever wondered about the differences between "passing an array by ref" vs. "passing an array by desc" then this is the demo for you.
      BTW...
      • when you pass an array by desc you are passing the address of the descriptor.
      • when you pass an array by ref you are passing the address of the array data (the receiving routine may need some additional information like array bounds, etc.)
    • this BASIC program basic_calling_c_demo5.bas calls C functions defined here: basic_calling_c_demo5_part2.c
    • in the first test, a 1-dimesion array of strings is created at run-time then passed to C (by ref) for processing
      • the C routines lengthen all strings for use by BASIC
    • in the second test, a 2-dimension array of strings is created at run-time then passed to C (by ref) for processing
      • the C routines lengthen all strings for use by BASIC
    • the third and fourth tests pass the arrays to C (by desc) so you can peek at internals (how many dimensions? what are the upper and lower bounds of each dimension? what is the striding parameter?)
    • food for thought:
      http://h30266.www3.hpe.com/odl/axplp/progtool/cpqc73/6180profile_011.html#variable_length_array_sec
  • Other Hybrid Demos
    Title VMS-BASIC source VMS-C Source Purpose Notes
    MySQL/MariaDB demo 11 mysql_demo11.bas mysql_api_demo11.c a crude demo to prove BASIC can link to MySQL client routines  
    MySQL/MariaDB demo 12 mysql_demo12.bas mysql_api_demo12.c c copies row results into an array  
    MySQL/MariaDB demo 13 mysql_demo13.bas mysql_api_demo13.c c copies field results into an array  
    MySQL/MariaDB demo 14 mysql_demo14.bas mysql_api_demo14.c c copies field results into a VMS-BASIC array  
    MySQL/MariaDB demo 15 mysql_demo15.bas mysql_api_demo15.c c copies field results into a VMS-BASIC array  
    MySQL/MariaDB demo 16 mysql_demo16.bas mysql_api_demo16.c BASIC instructs c to change the character set
    c copies field results into a VMS-BASIC array
    okay
    mysql/maria api 100   mysql_api_100.c derived from mysql_api_demo16.c good
    mysql/maria api 200   mysql_api_200.c corrects some run-time problems in mysql_api_100.c better
    mysql_demo.com     DCL build script for all MySql demos (new)  
    mysql_demo1x.com     DCL build script for MySql demos 11-16 (retired)  
             
    ssl aes demo 01 (needs polishing) ssl_aes_demo01.bas
    (needs polishing)
    ssl_aes_api_demo01.c (needs polishing) encrypt BASIC string data via AES (advanced encryption standard) (needs polishing)  
    ssl_aes_demo.com     DCL build script  
        base64.h cool include file required by this demo  

Java and AXIS2 Web Services

Category Title Description
Java Program HelloWorld.java a hello world to test the javac command
Axis2/Java SOAP Web Services SimpleService.java a very simple service with three methods
  SimpleService2.java.html like above but a tiny more complicated

OpenVMS DCL scripts

caveat: the ".com" file extension in VMS/OpenVMS predates MS-DOS and MS-Windows. Here it means "commands" 

Category DCL Scripts Notes
Calc Time Difference calc_date_time_diff_001.com calc_date_time_diff_002.com quick hack
much better
Create Certificates create_certs_alt.com a hack to create SSL certificates (for development use only)
1) create a "CA key" and "CA cert"
2) create a "server key" and "server csr" (certificate signing request)
3) use the "CA cert" to sign the "server csr"
DECnet communication decnet_demo_c.com decnet_demo_s.com client (start a remote server then send it some messages)
server (echo messages back to client)
Internet decode_sys_rem_id.com logical name SYS$REM_ID contains a hex encoded representation of the connected TCP/IP address. This demo shows how to display it in standard IPv4 format.
Intercepting an
FTP/SFTP login
systartup_vms_ftp_stub.com
login.com
login_helper.com
Example DCL script for properly setting up FTP/SFTP
DCL script to intercept an FTP/SFTP login then submit a login helper job
DCL script which will wait for the network partner to exit then process received file(s)
(similar to what happens at the end of an XCOM transaction) More notes can be found here: OpenVMS Notes: TCPware
FTP Scripts ftp_multi_stack_scripting.com a script which can FTP from three different stacks: MultNet, TCPware, and TCPIP Services
uppercase - lowercase switcheroo sense_nonsense.com downcase_filenames.com a tool to easily toggle your account between two modes: case-sensitive and normal
a tool to easily down-case filenames. Intended for SSH key files.
Apache Support rotate_apache_logs.com a little batch job to rotate apache client log files every 24 hours
also will (optionally) do monthly maintenance of apache process logs
Environmental
Monitoring (AXP only)
env_check_100.com env_check_101.com original script
tweaked for calling from your LOGIN.COM
MSA-RAID Monitor check_raid.com this self-submitting script can check your RAID controller, and drives, as often as you like

OpenVMS Macro32 source code

Programs Notes
helloworld.mar a little demo from Stephen Hoffman (works on Alpha and Itanium)
peek_byte_vax.mar Code to peek at bytes
peek_byte_function.mar Code to peek at bytes (function)
peek_any_int_vax.mar Code to peek at bytes, words, and longs

OpenVMS Programming

Introduction to OpenVMS programming

It was a lot tougher before y2k

It was a lot tougher learning how to write VMS applications in the 1980s. Why? DEC charged big-bucks for everything including hardware, software, training and documentation (including pocket programming cards!) Many IT people wondered if that was why DEC used a dollar sign ($) for the VMS/DCL command-line prompt.

First off, you needed the 32-binder  Orange Wall of VMS documentation  which I remember being around $5k for the initial purchase. Then, even if you already knew how to program in one of the many DEC-supported programming languages, you still needed to attend DEC language-specific courses to learn how to do system calls. This meant you needed to work for a company with deep pockets, or needed to get VMS-specific training in college or university.

In the early 1990s, DEC renamed the product line from VMS to OpenVMS at the same time they introduced the 64-bit Alpha hardware line as a successor to 32-bit VAX. During this time the  Orange Wall  became the  Gray Wall  

Then Things Changed

In 1998, DEC was purchased by "PC manufacturer" Compaq.

  • The first thing Compaq did was to convert the 32-binder Gray Wall (with a single CD-ROM.  GOOD  This allowed them to shut down their document publishing business)
     
  • Next, they put all their OpenVMS documents online so that anyone could access them for free.  GOOD 
     
  • Next, they shut down their educational services division to save money.  BAD  Technically speaking, they didn't just shut it down cold, they attempted to pass the business to third-party educators like they did with their PC products. But the scheme flopped. (hey, what would you expect from a shoot-from-the-hip PC company?)

In 2002, HP purchased (err, merged with) Compaq. HP restored some in-house training but now OpenVMS was just one of many software offerings from HP's, and I think OpenVMS has been lost in the mix.


Although my employer did provide us with the 32-binder Gray Wall, we had to learn how to interface VMS BASIC to VMS the good old-fashioned hard way: by hacking. In 1993 I fortunately stumbled across a copy of Writing VAX/VMS Applications Using Pascal (Author: Theo De Klerk, Published: 1991 by Digital Press) in DEC's student retail store in Bedford, Massachusetts. I had learned Apple Pascal in late 1970s so found it relativity easy to convert many of these programs into VMS-BASIC and VMS-C. This book contained many descriptions of VMS subsystems and was full of "system calls" on how to access them. These books pop up every so often on www.bookfinder.com and I think every serious OpenVMS programmer should own a copy.

Okay so that was then and this is now. To the best of my knowledge, you can only get the OpenVMS Documentation CD-ROM by purchasing it from HP, buying a used copy on eBay, or begging a friend to make you a copy. But if you use HP's online documentation along with examples from this book, you will learn how to become a VMS BASIC developer much faster than me.

Online HP Manuals + Documents

Advanced hacking (system services)

Many programmers do not know about this technique to trace "system service calls" originating from your process (DCL or otherwise). So if you have the necessary privs, and are on an Alpha or Itanium, then try this experiment:
Legend:
<sr> = system response <ur> = user response -------------------------------------------------------- <sr> $ ! this is my DCL prompt
<ur> set process /sslog=(state=on,count=4,flag=arg) ! enable tracing on YOUR process
<sr> $
<ur> run your-program.exe <sr> ...run the program then exit... $ <ur> set process /sslog=(state=off) ! disable tracing <sr> $ <ur> set process /sslog=(state=unload) ! this closes the file <sr> $ <ur> ANALYZE/SSLOG/FULL/WIDE/OUTPUT=ss_trace.txt ! human readable data goes to this file <sr> $ <ur> type/page ss_trace.txt

Official Documents:

Noteworthy C/C++ demos at other sites

Noteworthy C/C++ articles and manuals

How to recover from FTP-induced saveset corruption

$set file /attr=(RFM:FIX,MRS:512,LRL=512,ORG=SEQ,RAT=NONE)	yourfile.sav

Back toHome
Neil Rieck
Waterloo, Ontario, Canada.