OpenVMS Demos: Programmer's Corner
This entire site is a private effort of free information.
There are no cookies, no advertisements, and nothing is for sale.
non-code OpenVMS resources can be found
here
Caveat:
- The information presented here is intended for educational use by qualified OpenVMS programmers.
- The information presented here is provided free of charge, as-is, with no warranty of any kind.
- To the best of my knowledge, the programs presented here compile, link and run properly. Please send me an email if they
do not (I only support this stuff in my spare time).
Updated: 2024-09-07
OpenVMS Source Code (presented in HTML format)
Presented in HTML format so:
- Google can index them
- you can search them with "Google proper"
- you can search them with "Google site-search" like so: site:neilrieck.net lib$spawn
(just drop this red text into a Google search to see what I mean)
- you can display them with a browser (tab characters are rendered properly with modern browsers)
- you can drag a cursor over source text then copy-paste into an EDT editor (provided you are using a good terminal emulator)
How my source code pages were webified (is that a real word?)
These demo programs were converted to HTML pages by running the source code through this VMS-BASIC utility ( tool_webify_source_code.bas
) which converts certain HTML-sensitive characters into HTML entities
then generates a complete web page
OpenVMS Python demos
- Why use Python? Read my introduction to learn how Python bypasses
usual 64-bit limitations
- here I am talking about native math support languages like BASIC, C, C++ etc. Many developers get around these
limitations by writing, or purchasing, a math library
- Logical Name Demo.py (also shows how to access STARLET definitions)
- Python JIT Compiler (on OpenVMS)
OpenVMS BASIC demos
People new to VMS-BASIC might wish to read these articles:
File I/O Demos
RMS - Record Management Services for OpenVMS
comments:
- Back in the day, many hardware vendors also sold their own proprietary operating systems. IBM (International Business
Machines) and DEC (Digital Equipment Corporation) both incorporated support for sequential and relative data records into their respective operating systems in order to facilitate COBOL-74.
- Then they both included support for sorted indexed records in order to support COBOL-80 which was
finally released as COBOL-85. IBM invented the name ISAM
which contains all three technologies.
- On DEC systems, like VMS and OpenVMS, indexed ISAM can be seen in action by examining the system
account file.
Type this DCL command dir sys$system:SYSUAF.DAT /full to see what I mean
- COBOL uses PIC statements to layout record
structure (PIC is short for PICTURE). VMS-BASIC employs MAP statements
program files
- rms_sequential_demo.bas
- shows how to manipulate sequential RMS files (text files are one example)
- rms_relative_demo.bas
- shows how to manipulate relative RMS files
- rms_indexed_demo.bas
- shows how to manipulate indexed RMS files (ISAM = Indexed Sequential Access Method)
- this is a vanilla green screen demo
- comment: many software manufacturers, including IBM and DEC incorporated ISAM into their operating systems in order to
support COBOL
- rms_indexed_demo_fms.bas
- shows how to manipulate indexed RMS files (ISAM = Indexed Sequential Access Method)
- this is an FMS-based green screen demo
- you will need this FMS form: rms_indexed_demo_fms.zip
- you will need this FMS include: fms_fdvdef.inc
- you will need this other include: device_controls.inc
- FMS (Forms Management System) Notes:
- during the VAX/VMS era, developers were encouraged to insert their various FMS forms (name1.frm)
into a developer-created "forms library" (name.flb). Just after image activation, your
program would call FDV$LOPEN to open the library, then call FDV$LOAD to fetch
the forms into workspace buffers in your program, then call FDV$LCLOS to close the library
channel. In those days I went to great lengths to reduce the cost of disk i/o on our under-powered while
overworked VAX platforms. One solution was to bypass the form library by creating a binary version of each form at
compile-time, then linking them to the executable. The FMS demo shows you how to do this.
- There are ~ 66 FMS functions and you can read more about them by downloading manuals from here: https://docs.vmssoftware.com/
- caveat1: many FMS-based solutions were migrated to a newer API called
DECForms which has fewer functions with more parameters
- caveat2: back in the day, we would present these small skeleton programs to new programmers entering our group
(the might have been familiar with BASIC, but not DEC-BASIC or DEC-RMS or DEC-FMS). I recently used them to debug
a new implementation of FMS on OpenVMS-9.2-3 on x86-64. You may find them useful if you are doing something
similar: https://neilrieck.net/misc/zip/openvms/basic-fms-demos2.zip
- rms_test_fsp.bas
- a little hacking with the FSP$ function (originally provided to do a trial file-open on BASIC-PLUS-2
programs)
- newer programs should only use USEROPEN (see next demo)
- rms_test_useropen.bas
- do a trial file-open using a USEROPEN function
- an intermediate version includes a hack to allow inclusion of XABSUM and XABKEY (this last one is for inspecting
keys a.k.a. indexes)
- recently added code to include XAMPRO (to inspect protection bits)
- this program works in two modes:
- simple-and-safe:
- simply tack my pointer onto the existing FAB:XAB pointer (overwriting what was their previously)
- do my thing
- restore original pointer before exiting USEROPEN
- complex-and-dangerous:
- start at FAB:XAB then scan the linked list until you hit a zero. Then use a home-grown POKE routine to tack
my list onto the end of the existing list
- do my thing
- restore original pointer before exiting USEROPEN
- sys_file_io_demo.bas
- doing file i/o in BASIC without using BASIC's file i/o routines
- read_sysuaf.bas
- directly reading SYSUAF records for audit purposes only (you need lots of privs just to open SYSUAF.DAT)
- however, I also used this code to move client records from my VMS system into a relational database (I wanted to support
application-level authentication without allowing access to VMS proper; information here allowed me to move active VMS
accounts into an application-level database without needed to reset passwords)
RDB - Oracle-Rdb (Relational Data Base)
- DEC intended RDB to be the successor to RMS which they sold to Oracle Corporation in the 1990s
- you can find the original 43-file zipped package here
- More information: OpenVMS Notes: Oracle-Rdb
MariaDB - An alternate fork of MySQL
- Click here to view several VMS-BASIC demos where I call the MariaDB client via the
C-API. People wanting to use MySQL would only need to slightly modify the build script.
Tools: RMS Export - MySQL Import
- mysql_import_helper.bas
- mysql_import_helper_basic_template_part1.bas
- mysql_import_helper_basic_template_part2.bas
- mysql_import_helper_basic_template_part3.bas
- In 2014, our primary business system consisted of ~ 70 BASIC programs reading/writing ~ 150 RMS indexed files. In order for
us to do a parallel operations (RMS=production, MySQL=daily snapshot for ad-hoc queries) over the next 12 months, we needed a
semi-autonomous method of tool generation. These four BASIC programs do the following:
- generate 99% complete BASIC programs to extract data from RMS files then write it to CSV files
- you only need to edit one line before compiling (the input file name)
- generic open/read of your RMS files (bypasses indexes/keys)
- this tool is preferable to opening up your source code to insert customized export routines
- once built, there is no need to do this again unless the RMS file format changes
- generate 99% complete SQL scripts to create the tables then import the data from CSV files
- since my tool does not (yet) analyze the OPEN statements, you will need to insert SQL statements to create
constraints, indexes and keys, etc.
- CAVEAT: you could modify the table creation statements before the import but this would cause the import to be
hundreds of times slower (lots of recursion while it maintains each index AND you could still end up with a lop sided
index if the inserted data was already ordered). All large imports should always be indexless and constraint-less; you
tack them on as a finishing step.
- I placed all my post-import SQL statements in a second script which won't be overwritten by the tool
- 2015 Update: after 12 months or so, many of our programs are bypassing RMS and writing production data
directly into MariaDB. This means that these transfer tools are taking a back seat to our daily operations. We expect to have
everything moved out of RMS by the end of 2019.
- 2020 Update: we still have five RMS-ISAM files which have not yet been permanently moved to MariaDB so I
updated the four programs above to do a better conversion from mixed to latin1 (cp1252)
I/P Stack Demos (written entirely in VMS-BASIC)
- click here to see my c-demos farther down this page
TCPIP Services for OpenVMS
- TCPIP Services for OpenVMS is HPE's official IP stack for OpenVMS
- IIRC, there was never a version of "TCPIP Services for VMS on VAX" which is why most VMS sites ran TCPware from Process
Software
- versions 4.x and lower produce logical names prefixed with "UCX" ("UNIX Communications Extensions" or "Ultrix
Communications Extensions")
- versions 5.0 and higher produce logical names prefixed with "TCPIP"
- later versions of 5.x support both IPv4 and IPv6
- Notes:
- the demo programs just below where written in 2014 when I was preparing to move from TCPware on
Alpha to MultiNet on Itanium
- Why change? In 2014, PSC told us that TCPware only supported IPv4 and never would support IPv6
- Much of our home-grown apps were dependent upon a TCPware API known as the TELNET Library (nothing equivalent in any
other stack)
- These demos were developed on TCPIP Services (the machine I was working on at the time did
not have have a MultiNet license)
- these programs employ VMS System Services rather than the Sockets API
so will work with any stack
- tcpip$tcp_client_qio_2014a.bas
- Synchronous (qiow) TCP client demo - employs http to retrieve a web page
- tcpip$tcp_client_qio_2014b.bas
- Synchronous (qiow) TCP client demo - adds support for gethostbyname
- tcpip$tcp_client_qio_2014c.bas
- Asynchronous (qio) TCP client demo - employs programmable timers
- tcpip$tcp_client_qio_2014d.bas
- tcpip$tcp_client_qio_2014d.inc
- Same as "2014c" except my TCP routines have been moved into external functions
- tcpip$tcp_client_qio_2014e.bas
- tcpip$tcp_client_qio_2014e.inc
- nvt_definitions.inc
- Same as "2014d" but with NVT support for TELNET (previously handled in TCPware by the TELNET Library)
- tcpip$tcp_client_qio_2014f_bas
- Same as "2014e" but a bit more polished
- tcpip$tcp_sa_client_qio_basic.bas
- stand-alone client demo (telnet to another system, log in, run a program, log out)
this demo program works but could use a little polishing; it probably will not be developed any further
- get_host_by_name_qio.bas
- DNS lookup from BASIC (also contains a few hacking tools)
- sff_demo.bas (caveat: SEND FROM FILE is supported by "TCPIP Services")
- demos function tcpip$send_from_file() which is a nifty way to send SMTP messages to your own stack
without opening a connection to port #25
MultiNet
- a third-party stack from Process Software (supports IPv4 and IPv6)
- more feature-rich than TCPIP Services for OpenVMS from HP
- No stack-specific demos here since my "TCPIP Services" demos work with this stack
TCPware
- a third-party stack from Process Software (only supports IPv4)
- this was the only IP Stack for VMS in the 1980s and first half of the 1990s
- the Telnet API and FTP API provide a solution midway between Sockets
and System Services
- TCPware_telnet_sample.bas
- Employs the TELNET API to demo CHARGEN, ECHO, HTTP, and TELNET. Also contains a minimal NVT handshake necessary to
telnet to UNIX systems (caveat: Solaris will not let you connect unless you can provide a TERM
TYPE via IAC WILL-WONT-DO-DONT; makes me wonder if this isn't some sort of simplistic
anti-hacking effort)
- TCPware_pop3_demo.bas
- Employs the TELNET API to demo connecting to a POP3 server to retrieve email
- TCPware_smtp_demo.bas
- Employs the TELNET API to demo connecting to an SMTP server to send email
- TCPware_ftp_sample.bas
- Employs the FTP API to demo doing FTP from within a BASIC executable
Web Demos
Apache Glue Example
- read_html_apache.com
- drop this DCL script into any one of your apache user-accessible scripts directories (I use "[.scripts]")
- rename the file to match the URL you wish to use. For example, if the link is "/demo123" then rename the file to "demo123.com"
- modify the script to run this next program called "read_html_apache.exe". It will read/decode the
received HTML page then create DCL symbols representing the contents of HTML objects on the web page.
- if the program ran properly, now run your desired program which will read the DCL symbols then do whatever you want
- use a browser to test your application like so:
http://node-spec/scripts/demo123
- read_html_apache.bas
- Have the script above run this program which will:
- read data sent by the browser to the web server
- decode the HTML page
- create DCL symbols representing HTML data contents of HTML objects on the web page
Apache No-Glue Example
- apache_demo.bas
- This stand-alone application does not require any glue (DCL scripts). You could do one of the following:
- create a one-line DCL script which does nothing other than run your executable program
- drop the executable program directly into the Apache scripts directory
Password Change Demo
- www_password_change.bas
- CGI Glue
- can be used standalone (does not use CGI glue)
- can be used not standalone (see Apache Glued Example above)
- can hook into an external database (RMS in my case) to validate users
- demos how a small amount of JavaScript (embedded in the BASIC source) can be used to pre-validate the HTML form then
enabling/disabling the submit button
- be sure to see my SYSUAF hack on this same page
Upload Demo
- coming soon
- Got this working in production but it needs to be cleaned up before publishing
- update: it is so ugly that you might be better off using this C function which can be
called from BASIC
SOAP Demo
- axis2_soap_test_tool.bas
- A little client-side tool used to test my SOAP services are written in C (via gSOAP) or Java (via AXIS2 running on
Apache Tomcat)
- The main program requires these three files:
AJAX Demo
- coming soon
- but here is a little taste of only the JavaScript file here: ajax_vdsl_port_mtce.js
- AJAX (Asynchronous JavaScript and XML) is a
neat, although poorly understood, client-server technology used in conjunction with DOM-compliant browsers.
- AJAX allows a browser to fire JavaScript function XMLHttpRequest
to issue an HTTP request on a secondary channel behind the current web page. More JavaScript is then employed to
inject data from HTTP response into the DOM.
- AJAX can send back ANY text document. Plain, XML, HTML, JavaScript, JSON, whatever.
- Back in the day, I wrote many VMS-BASIC applications which employed FMS (forms management system) forms on character
cell terminals like so:
Desired Action |
BASIC Code |
Notes |
display a form |
call fdv$disp( fname$ ) |
|
populate a form (field-by-field) |
call fdv$put( last_name$, "f$last_name" ) |
|
populate a form (all at once) |
call fdv$putal( whole_form$ ) |
must be a mapped string |
read a form (field-by-field) |
call fdv$get( last_name$, terminator%, "f$last_name" ) |
|
Data conversion / manipulation
- caveats:
- programmers in 2016 are still confusing UTF-8 with Unicode
and this is causing huge problems
facts:
- UTF-8 is a Unicode encoding
- Microsoft states that all written languages can be represented by 16-bit Unicode but this will require 3-octet UTF-8
(the spec prefers octet to byte)
- Both MySQL and MariaDB will set aside 24-bits if you declare the data to be UTF8
- Both MySQL and MariaDB will set aside 32-bits if you declare the data to be UTF8mb4
- programmers in 2016 are still interchanging iso-8859-1 with cp1252
(code-page-1252 a.k.a. Windows-1252)
facts:
- cp1252 is a 32-character superset of iso-8859-1 so browsers treat both declarations as cp1252 (no visible errors when any of the additional 32-characters accompany a page declared to be
iso-8859-1)
- Both MySQL and MariaDB require you to declare the character set as latin1 which is an alias
for cp1252
- some older programs below were developed in 2002 when we were attempting to translate Microsoft-generated emails -AND-
incorrectly assumed we will only see 2-octet UTF-8. If you want to represent all European
characters (both East and West) then you will need to support 3-octect UTF-8. It you have any
contact with Asian supply chains (think Alibaba) then you will need to support 4-octet UTF-8
- Unicode and UTF-8
- overview:
- When you think about it, a string is nothing more than an array of bytes. This next routine (mixed-to-unicode)
assumes the input data is mixed (ASCII, ISO-8859-1, cp1252, or UTF-8) then performs a best-effort conversion into an
array of 32-bit integers representing raw Unicode (you can set an option to throw away anything that doesn't appear to
be legal).
- After that, you can call another routine to get the data you wish
- mixed_to_unicode_array.fun
- this external BASIC function assumes the input data contains a mixture of UTF-8 and CP1252 (which is a superset of
ISO-8859-1)
- comment: this should not happen but I have seen UTF-8 mixed with cp-1252 coming from Microsoft database engines
like SQL Server
- anything between x00 and x7F is converted to 32-bit Unicode as-is (obviously)
- anything between x80 and xFF is assumed to be a mixture of UTF-8 and CP1252
- everything which appears to be a valid UTF-8 sequence is converted to 32-bit Unicode
- any remaining stuff between x80 and x9F is assumed to be CP1252 which is mapped back to equivalent 32-bit
Unicode code points
- any remaining stuff is assumed to be ISO-8859-1 and so is converted to 32-bit Unicode
- unicode_array_to_cp1252.fun
- also called windows-1252 (or latin1 in MySQL/MariaDB)
- other converters:
- unicode_array_to_utf8.fun
- this external BASIC function converts from 32-bit Unicode to UTF-8
- use this function to send mixed data to any jQuery widget which almost always requires utf-8 even though the hosting
web page desires character set "windows-1252"
- unicode_testing_bas.html
- this demo program shows how to use the functions just described
- ebcdic_ascii.bas
- base64 conversions:
- sort_demo.bas
- calling VMS sort routines (sometimes it is best to defer to the experts)
Miscellaneous Stuff
Command Line
- foreign_cmd.bas
- allows a BASIC program to read DCL command-line parameters like C programs do on UNIX
DCL symbols
DECnet Communications
Device Scan
Diffie-Hellman Key Exchange Demo
- diffie_hellman_demo.bas
- Diffie-Hellman Key Exchange Demo (limited to 64-bit math)
- note: this happens at the start of every encrypted data transmission on the net
Displaying System errors
- display_system_error.bas
- can translate any system error number into the associated text message. This program is no longer required since modern
OpenVMS versions can do this for you from the DCL prompt.
Health Monitor
Lock Demos (CEF - Common Event Flag)
Lock Demos (DLM - Distributed Lock Manager)
- dlm_demo_100.bas
- creates a null lock then synchronously converts it to more restrictive forms
- dlm_demo_101.bas
- creates a null lock then synchronously converts it to more restrictive forms
- dlm_demo_102.bas
- asynchronously requests an exclusive lock then polls DLM to see if it was granted
- dlm_demo_103.bas
- asynchronously requests an exclusive lock, then arms a timer, then waits for one of two event flags
Linked Lists and Memory Queues
- mem_que_demo.bas
- demos: lib$insqhi, lib$insqti, lib$remqhi, lib$remqti but these is better done in C/C++
Mailbox Demo (Unix people use the word 'pipe')
- advocate_client.bas
- client program to send DCL requests to a server program via a VMS mailbox; note: a VMS mailbox is similar to, but not
identical to, a UNIX PIPE
- advocate_server.bas
- server program to receive DCL requests from a client program (via a VMS mailbox)
- advocate_DCL_scripts
- scripts to start the programs above
When you need larger DCL symbols
- multi-line_logicals_demo.bas
- Some apps like Apache save session data (like HTTP_COOKIE) as a DCL symbol. When the data is greater than 930 bytes,
Apache will save the data as multi-line logical names. This program demos how you would do something similar from BASIC
Overview:
- all operating systems support shell variables known as symbols
- VMS (and OpenVMS) also support a different data-passing method known as logical names
- programs wishing to create or modify logical names will need to call lib$set_logical or sys$crelnm
Caveat: lib$spawn will allow you to make changes on copied data which is discarded upon exit of that spawned task
- While everyone knows how to create single line logical names from BASIC, multi-line logical names are a little more
difficult
Password Stuff
Peek
- peek_demo.bas
- Unlike most other BASIC implementations, VMS-BASIC doesn't have a PEEK function but sometimes you need it
- peek_demo_test_run.bas
- see how variables are stored in an OpenVMS system
- see how the Alpha compiler is able to achieve such high levels of performance relative to VAX
- demo_decimal_hack.bas
- see how fixed-point decimals are stored in OpenVMS and RMS (no difference by the way)
Population Simulation
System Calls you can't live without
- sys$getjpi
- getjpi_demo.bas
- demos how to properly call sys$getjpi (get job process information)
- sys$setddir
- show_default_demo.bas
- demos how a program can do the equivalent of the DCL command "$show default"
- comment: anyone coming from RSX-11 or CP/M will remember that using SET without any parameters is the same as SHOW
- sys$getrmi
- sys$qio
- qio-demo.bas
- calling sys$qio from BASIC allows more control than PRINT and INPUT
Source Code Tools
- source_code_reporter.bas
- generate "code change" reports (from change-area comments) to justify your existence
- source_code_formatter.bas
- This horribly written quick-hack has proved invaluable as a tool for locating BASIC coding problems introduced by other
authors.
- Anyone who has inherited source code from another organization knows what I am typing about.
- Caveat: I've been meaning to rewrite this tool for a decade now but can never find the time. But this tool can format
itself so maybe it's good enough for occasional use
Sequencer
- sequencer.bas
- When run from a DCL script, this program goes great lengths to ensure that your process, and only your process, gets the
next number in a numerical sequence. I use this program so that an intercepted email account can temporarily access one of
twenty subdirectories before I call MUNPACK to extract MIME attachments (if any) from cell phone messages. Of course, I
need to run a different program to read the plain-text email if nothing was extracted
Terminal Characteristics
Timer Demos
VMS Mail
Watchdog
- watchdog.bas
- my own watchdog process (demos lots of systems calls)
Webification
OpenVMS BASIC standalone functions (file extension ".fun")
Some BASIC demos may require these standalone (mostly BASIC) functions:
- wcsm_calc_easter_n_good_friday.fun
- compute Easter or Good Friday for any date after 1582. Necessary for writing an annual holiday generator.
- wcsm_dt_stamp.fun
- returns the current date-time in ccyymmddhhmmss format
- wcsm_get_mime_time.fun
- mime time function for smtp use
- wcsm_get_symbol.fun
- returns the value of a desired DCL symbol
- wcsm_peek_quad.fun
- returns the 64-bit data work found at the specified 32-bit address
- wcsm_set_symbol_or_logical.fun
- this function will attempt to save your data as a DCL symbol. If it can't (usually because the data is too large for
your machine) then it will shift modes and save your data as multi-line logical names (emulates functionality found in the
CGI interface of Apache for OpenVMS)
- wcsm_submit_to_batch.fun
- submitting to batch from within BASIC
- wcsm_trnlnm.fun
- translates a logical name
caveat: we used this function in the VAX days because it was much more efficient than calling
lib$get_logical and easier to set up than sys$trnlnm. But there seems to be no performance
hit when using lib$get_logical on Alpha so this function is only included here incase I've
included a few demos still calling it.
- wcsm_url_decode.fun
- undo so-called percent encoding (as well as some other stuff)
Question: So just what does the prefix WCSM stand for?
Answer: Waterloo Computer System Maintenance. That department name morphed into something else but the
prefix remains
OpenVMS BASIC includes (file extension ".inc")
Some BASIC demos may require these standalone includes:
- vms_structures.inc
- data structures required for item lists in system calls, etc.
- vms_externals.inc
- system declarations required to do system calls from BASIC (should not be used for new code development)
- device_controls.inc
- escape sequences required to embed VT terminal and LA printer inside your programs. (written for use on small VAX
systems where it was not practical to control devices by system calls to the terminal driver)
- fms_fdvdef.inc
- FMS (forms management system) definitions: functions and constants
- tcpware_ccb_definitions.inc
- TCPware control block definitions
My BASIC Development Environment
A snapshot of my development environment. It will help you understand my how I refer to includes and standalone functions
Directory |
File
Extensions |
Notes |
[.dvlp] |
.bas
.com |
basic source code
DCL scripts for building some applications |
[.dvlp.flb] |
.frm
.flb |
FMS forms (created by dcl command: $fms yada.frm)
FMS form libraries |
[.dvlp.fil] |
.rec
.opn
.fdl |
mapped string declarations (records) for relative and indexed files
open statements for relative and indexed files
file description language files for tuning rms files |
[.dvlp.fun] |
.fun |
basic functions |
[.dvlp.inc] |
.inc |
basic include files |
[.dvlp.mar] |
.mar
.m64 |
macro-32 files
macro-64 files |
[.dvlp.sql] |
.sql |
MySQL or MariaDB scripts |
BASIC Compiler bugs
Overview: this all started when I was trying to write a limited Diffie-Helman demo in BASIC. Why
limited? I knew before I began that I wouldn't have access to the whole key space because BASIC does not natively support
unsigned integers. But VMS-BASIC-1.7 does support a QUAD datatype which weighs in at 64-bits so that should have gotten me part
way (a complete Diffie-Helman demo would require 512-bits, 1024-bits or more). But then I found the following problems with
quad.
basic_quad_mod_bug.bas
- I just (2012-08-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
- Although you can divide large QUADWORD integers, the modulus function fails
- Here is a "C" version which works properly: basic_quad_mod_bug.c
- I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem
Version |
Notes-1 |
Notes-2 |
1.7-000 |
test fails at 2^32 |
bug |
1.7-001 |
test works up to 2^62 |
fixed |
- CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support
contract and want it fixed then you must input a support request"
basic_decimal_mod_bug.bas
- I just (2012-09-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
- Although you can divide large DECIMAL integers, the modulus function fails
- I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem.
Version |
Declaration |
Notes-1 |
Notes-2 |
1.7-000 |
decimal(16,0) |
test fails at 2^32 |
bug |
1.7-001 |
decimal(16,0) |
test works up to 2^53 |
fixed |
|
decimal(17,0) |
test works up to 2^56 with compile warnings |
safer to use quad |
|
decimal(18,0) |
test works up to 2^59 with compile warnings |
safer to use quad |
|
decimal(19,0) |
test works up to 2^62 with compile warnings |
safer to use quad |
- CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support
contract and want it fixed then you must input a support request"
BASIC RTL (run-time library) bug
- This affects OpenVMS-8.4 ECO 900 (a.k.a. consolidated patch kit 900 and earlier)
- In BASIC, you can use the open statement to create an indexed file with one or more keys.
- If any one of the keys is a QUAD (signed 64-bit integer) then any attempt to reopen the file will
fail
- I placed a support request with HP (OpenVMS Engineering) who provided me with an experimental BASRTL 10-days later.
- Not sure if this will be released to the public so if you need it, PLACE A SUPPORT REQUEST with HP.
p.s. I was attempting to write code which would directly open SYSUAF which contains a key (a.k.a. index)
based upon an unsigned quadword. Since VMS-BASIC has no unsigned data types an
updated BASRTL would be of no use. Even still, I am surprised to learn that this bug was hanging around since 2005 when HP added
the quad data type to the language (up until then you needed to use basic$quadword
defined in the STARLET library)
BASIC bug or oversight?
I just (2020-09-03) noticed this problem with VMS BASIC-1.7 (Itanium) but with no support contract this is the only place I can
complain
1000 option type=explicit
declare string i$ ,&
long i%
input "array size? ";i$
when error in
i% = integer(i$)
use
i% = 10
when
2000 dim single s(i%)
dim double d(i%)
dim string s$(i%)
dim decimal(14,2) de(i%)
!
! use the matrix command to init these arrays
!
3000 mat s = zer !
mat d = zer !
mat s$ = nul$ !
mat de = zer ! this line will not compile
!
4000 s(0) = 12.3 !
DEC-C (a.k.a. VMS-C)
DEC-C++ (a.k.a. VMS-C++ DEC-CXX , VMS-CXX)
General
Caveats for developers:
- both C and C++ languages "appear" to be more complicated on OpenVMS than what you will see on UNIX, Linux or Windows so be
sure to read OpenVMS specific documentation.
- C on OpenVMS
- C++ on OpenVMS
- http://h41379.www4.hpe.com/commercial/cplus/cplus_index.html
- just what the heck is "cxx"? This is another example of Dilbert humor where the plus signs of "c++" are rotated 45
degrees. This was necessary because the VMS operating system did not allow plus signs in file names. This means that
".cpp" is also supported.
- DEC-C Run-Time-Library Reference Manual For OpenVMS which is used by both C and C++
- http://h41379.www4.hpe.com/doc/732final/5763/5763pro_001.html
- tells you where your reference libraries are located
- compares UNIX I/O to RMS I/O
- introduces optional library directives which will allow UNIX-dependent c programs to run on OpenVMS
e.g. automatically translate a full filename specification from "//one/two/three/four.txt" to
"[one.two.three]four.txt"
- Many people today only purchase a C++ license then use it to compile C programs
- One definition of Sensei (a Black
Belt in Karate) is "one who has gone before". Many times you will attempt to solve a
problem which has already been solved by someone else long ago but you didn't know where to look. Some of those solutions
can be seen here:
- Two compilers? Many programmers claim since c++ was developed to be a superset of c then will point to numerous articles
where programmers use a c++ compiler to develop better c programs. That said, I can show you examples where original
developers tell third-party programmers to not use c++ compilers when building their APIs. OpenSSL is one example that springs
to mind.
- When calling OpenVMS routines defined in the STARLET library, do yourself a favor and enable strict type-checking. This is
done by inserting this statement before the library references you wish to be strict (for new programs it should be before all
library includes).
#define __NEW_STARLET 1
Doing so will endure your code will be in better shape to be ported to another OpenVMS hardware platform. In one of the
programs below (sys_create_uid_101.c) I was forced to add a few more lines (resorted to a union) to satisfy the compiler but
the optimizer was able to generate a program which was byte-for-byte identical to an earlier relaxed version.
- Some large projects (especially code originally written for UNIX/Linux) may require you to build your program with a
so-called "make utility". MMS (module management system) is the name of the official licensed make tool.
Developers without an MMS license are advised to use MMK which is the open source freeware version available
here:
- http://github.com/endlesssoftware/mmk
- http://www.decuslib.com/freeware/freewarev80/make-mmk/
Additional Advice:
- many old-school VMS programmers prefer to invoke the compiler and linker from a DCL prompt. When their DCL commands
become too long they will often move them into a DCL build script. While there is nothing wrong with this, older
applications like "$LINK" do not support DCL symbol substitution after the "sys$input/opt"
directive which then forces the developer to resort to temporary "logical names" or generating-then-executing temporary
scripts. Even then, you will have some difficulty squelching a noisy link operation (lots of warnings but still a
successful build). All these problems can be handled by moving to a make tool like MMK
- We had been using MMK-4.1 for more than a decade on Alpha with no difficulties. When we moved from
Alpha to Itanium in 2014, I acquired an Itanium version of MMK-4.1 which did not always work properly with some open
source offerings like mtools. Upgrading to MMK-5.0 fixed those problems
comment: years ago, one of our developers attempted to use MMK-5.0 on Alpha but it proved
problematic so we stayed with MMK-4. Do not jump to a new version of MMK unless forced to do so.
- GNV (Gnu Not VMS) hosts a Unix-like environment on OpenVMS: gnv.sourceforge.net
- Lots of open-source software has already been ported to OpenVMS: vms-ports.sourceforge.net
C Demo Programs
File |
Notes |
Hello World |
hello_world.c |
every website has one of these |
hello_openvms_world.c |
introductory "hello world" program for OpenVMS |
hello_openvms_world_more.c |
added a few system calls to the previous program |
Where the rubber meets the road |
chmod_demo.c |
changing VMS file protection bits |
credit_card_number_test.c |
employs the Luhn Algorithm to pre-validate credit card numbers
(as well as some other stuff including Canadian Social Insurance Numbers, etc.) |
cmd_line.c |
reads command line arguments (and named logicals and/or DCL symbols) |
descriptor_macro.c |
the $DESCRIPTOR MACRO is dumber than most people realize so use it with caution or not at all |
diffie_hellman_demo.c |
Diffie-Hellman Key Exchange Demo (limited to 64-bit integer math) |
dns_lookup_demo.c |
A very-cool LINUX socket programming to do Name Server lookups. It was not written by me but it
works as-is on OpenVMS-8.4 Alpha with "TCPIP Services for OpenVMS v 5.7". It won't work from
behind a firewall (without a few mods) but you'll get the basic idea.
Caveat: be sure to see RES_SEARCH_DEMO.c below which may be a bit more
useful. Why? Library functions res_search() and res_query()
work properly with IPv4 -and- I assume they also work properly with IPv6 |
do_100k.c |
This little test file came from OpenVMS Engineering (I kept it as an homage to them) when we
reported a 100k problem with CSWS 2.0 (Apache for OpenVMS). Drop this file into the scripts
directory of your Apache server then trigger it from a browser like so: http://nodename.ext/scripts/do_100k
It will help you test your Apache server as well as assist in confidence tests with your network
admin people. |
ebcdic-ascii-demo.c |
convert between EBCDIC and ASCII character sets |
file_upload_neil_c3.html |
this is the webpage used to start the file-picker |
flock_demo.c |
- POSIX C provides lockf
- BSD/Linux C provides flock
- the CRTL library published for OpenVMS by HP in 2005 contains neither function
so you may need to rely upon the ANSI functions flockfile and funlockfile
-OR-
do specific OpenVMS system calls to the distributed lock manager.
comment: in a VMS Cluster the distributed lock manager is the only way to go.
I haven't worked in a clustered environment for more than a decade -AND-
found the flock method so easy to use on Linux that I hope VSI
(VMS Software Inc)
adds support for it ASAP. Heck, it may already exist in their x86-64 product.
comment2: I've produced a working demo for OpenVMS which employs flockfile
but is is so ugly that I won't post it here.
|
getuai.c |
demos use of getuai (get user authorization information) |
http-check.c |
HTTP connection test-tool from Hunter Goatley (Process Software) |
http_file_upload_31.c
(broken when uploading
certain binary files; see caveat) |
this is a c version of my BASIC code (it can be built as either "a standalone executable
program" or "a function which can be called from any so-called DEC language including BASIC)
caveat: on 2016-03-xx I realized this program contains a small bug when saving uploaded binary
files to OpenVMS. With regard to RMS files of the "512-byte fixed" variety, this broken program appends nulls after the
last data byte in the last block then places an EOF at the end of the block. Most programmers already know that many
files (with extensions like ".exe" and ".xslx" etc.) expect to see an EOF after the last data byte in the last block. I
fear the only way around this is to switch from "standard-i/o functions like fprintf" to "OpenVMS functions like sys$put
and sys$write". If anyone is aware of a simpler way only using c then please send me an email. Anyway, a solution is
just around the corner.
solution: I now have a working version (this is just a spare time project) but I have not gotten
around to publishing it |
ldap.h
ldap_example.c
ldap_example_hack.c
ldap_example_hack_authenticated.c |
Lightweight Directory Access Protocol |
lib_mysqludf_build_vms.com |
DCL script to build lib_mysqludf_sys on OpenVMS (user defined function to do
OpenVMS system calls from within Stored Procedures) on MySQL and MariaDB
note: this script shows how to force the OpenVMS linker to produce both uppercase and
lowercase symbols |
mysql_demo01.c |
a "hello world" demo for MySQL and MariaDB |
mysql_demo02.c
mysql_demo.ini
mysql_demo2_build.com |
doing some real world work with SQL
an optional parameter file for use with mysql_demo02.c and higher
Three build scripts for three OpenVMS environments (Berryman MariaDB, VSI MariaDB, VSI Libmariadb) |
mysql_demo03.c |
connecting via SSL |
mysql_demo04.c |
demonstrates how to import data from RMS to MySQL/MariaDB
caveat: since I wrote this program I've had better luck working with the mysql-admin client CLI
(so do not use this code) |
mysql_demo05.sql |
playing with SQL (creating tables, displaying execution plans, etc.) |
mysql_demo.com |
simple build script (produces the fastest executable) |
mysql_demo_alt.com |
complicated build script (can link to a mixed-case shareable SSL library) |
pipe_demo_parent.c |
this program creates 3 pipes before calling vfork |
pipe_demo_child.c |
this C program is executed by the vfork |
pipe_demo_child.com |
this DCL script is optionally executed by the vfork |
popen_demo.c
system_demo.c |
two more ways to execute a DCL script from c |
pointer_demo.c |
|
quad_mod_demo.c |
playing with quads (long long) in "C". This program was used to verify some
problems with HP Alpha BASIC 1.7-000 |
res_search_demo.c |
As of October-2014 I think this is the only reasonably complete OpenVMS demo
on the internet showing how to use res_search() and res_query()
then extracting A, MX and NS records. If anyone knows a better/easier way to to
extract this information on OpenVMS (perhaps using library functions I don't know
about) then please send me the details.
Comments:
- Many library functions are available for Linux (and I was tempted for a short while to begin porting a half dozen
or so but why reinvent the wheel if others have gone before me? I certainly couldn't be the first OpenVMS
application programmer standing before this chasm, right? Also, the responsibility of keeping the C/C++ libraries
up-to-date squarely lays with HP
- Anyone who has viewed the official ISC source code for nslookup.c
or dig.c might be wondering WTF?
I'm certain they are kept remark-poor to thwart modifications by non-specialists. Meanwhile, I am certain that
professional spammers already know how to extract MX records.
|
rms_c_ug_example.c |
This program is an annotated version of RMSEXP.C found in chapter 2 of the HP C User's Guide.
Any newbie wishing to use RMS from C or C++ should first look at this program before you write a single line of code |
sequencer.c |
a tiny little app for selecting working folders on OpenVMS (coming soon) |
smtp-check.c |
SMTP connection test-tool from Hunter Goatley (Process Software) |
sys_sndjbcw.c |
sndjbc - submitting a job to a batch queue (this will enable you to submit a job on behalf of another user for the
express purpose of having that user send a file to another system via SFTP or
SCP) |
sff_demo.c |
demos function tcpip$send_from_file() which is a nifty way to send SMTP messages to your own stack
without opening a connection to port #25 |
sudoku.c |
to benchmark hardware/software performance |
sys_create_uid_100.c |
generate a UID (with a tip of the hat to Jim Duff) caveat:
need a 1 ms delay between successive calls for true uniqueness |
sys_create_uid_101.c |
generate a GUID (with a tip of the hat to Stephen Hoffman) caveat:
need a 1 ms delay between successive calls for true uniqueness |
tcpip$tcp_client_qio.c |
my presentation of a client demo program found in vms folder sys$examples (copyright by HP)
notes:
- this program uses sys$qiow rather than Berkeley sockets and is synchronous
- a synchronous version (based on sys$qio) will be published soon
|
timezone_demo.c |
display the current time translated to any other time zone |
unix_hater.c |
does the "C" example in the UNIX Hater's Handbook actually compile? (yes) |
|
|
vt340-color-demo.c |
display colored text on real VT terminals (or any good emulator) without SMG$ |
dec-device-control-codes.c |
this file contains escape sequences used to control popular DEC peripheral devices |
vt-smg-demo.c |
note: SMG$ is one of five OpenVMS RTLs. This limited demo employs SMG$ routines to display colored text on a virtual
terminal |
vms_lock_dlm.c |
Accessing the OpenVMS DLM (Distributed Lock Manager) |
utf8_decode.c |
no matter how you configure them, most popular browsers will push Unicode via UTF-8 back to your programs. This will
cause a problem if your databases are configured to store single characters so you will want to convert from UTF-8 to
ANSI/Windows-1252 (which is a superset of iso-8859-1). But what happens when you are unable to convert everything? Do
you throw away anything over 255 or do you map to something else? |
utf8_encode.c |
this program can convert from ANSI/Windows-1252 back to UTF-8 |
internationalization_demo_100.c |
demos converting from "iso-8859-1" to "ucs-4" (calls: setlocale(), iconv_open(), icon_close(), and iconv()) |
internationalization_demo_101.c |
demos converting from "iso-8859-1-euro" to "utf-8" okay for ISO-8859-1 to UTF-8; not okay for Windows-1252 (also
called ANSI) for this reason alone, I will keep these Internationalization demos close to my UTF-8 routines |
C++ Demo Programs
File |
Description |
hello_world.cpp |
every site has one |
string_demo.cpp |
This string demo is very interesting. Why?
- at first glance these C++ strings appear much more BASIC-like than those found in traditional C. But this syntax
seems closer to what you see in Java, JavaScript, or C#
- Many traditional third generation languages, like HP-BASIC for OpenVMS, limit string size to
32,767 bytes. I have tested this C++ demo up to a string size of 1,999,000 bytes
(I could have gone higher but became bored) and found that it worked equally well with:
- "HP C++ V7.3-009 for OpenVMS Alpha V8.4" (complied and linked in 32-bit mode)
- "CodeBlocks 12.11 on Windows 7"
|
Apache Specific (c-only)
Apache Module Authentication
Overview
- In 2014 we transitioned 100 users from green-screen to the web. Our web applications do their own authentication by hooking
into SYSUAF then setting an encrypted session cookie to be used for the next
hour:
- we record every transaction and can "log out" inactive web users by deleting their specific cookie from our "current
sessions" database
- this database is also used to implement an administrative dashboard
- The negative here is that old-school access to documents stored in protected directories still require a second
authentication challenge from mod_auth_openvms which cannot be checked for inactivity -AND- doesn't
set a cookie. Our clients are becoming really annoyed at this.
- The success of our transition from green-screen to the web has resulted in the creation of new business opportunities
resulting in ~1300 new users. However, the usual account churn has become somewhat of a burden so we would like to transition
to self-serve account maintenance (think Amazon or eBay). This could done in a simple secure way if all user
accounts were moved from SYSUAF to an application-level database (which is an indexed RMS file in our case). This
also means that only five people (developers and sys-admins) would ever have SYSUAF accounts. Very cool!
- So I copied the password hashes of our user accounts from SYSUAF to our user profile database but
still needed to keep ~100 accounts in SYSUAF to be authenticated via mod_auth_openvms. This means doing
password changes in two places -AND- not allowing full self-serve account maintenance on these. (well I could but then how
would I sleep at night knowing someone might stumble on a way to hack the system?)
- So we need to replace mod_auth_openvms with something more useful; something that supports session cookies
Hacks
- I started with mod_auth_openvms.c as my template but first needed to learn how to
build it or there would be no point in continuing
- I downloaded a UNIX distribution of httpd-2.0.63 from Apache with the
intent of only building part of it (the authorization plugins) on OpenVMS
- Then I realized APR (Apache Portable Runtime) was a prerequisite which needed to be downloaded
then built separately. I wondered if this would be best done with a MAKE file under GNV or by some other method
- So I bounced the MAKE/GNV questions off people at comp.os.vms
who suggested downloading CSWS sources from HP then linking against the Apache run-time which is already present (who knew?)
- CSWS source kits can be downloaded from HP's website via anonymous FTP
- ftp://ftp.hp.com/pub/openvms/apache/ (works from IE-11)
- note that CSWS-V20-2 is newer and larger while CSWS-V21-1 is older and smaller (???)
- my examples use CSWS-V21-1 (older and smaller)
- note: the official build instructions in chapter 5 of manual
csws_install_22.pdf are broken; see my scripts below for one possible solution
- caveat: my original programs were pure hacks based upon peeking into the ".h files" included in the
kit. Since then, I have learned my time could have been reduced by 80% had I first purchased a copy of
Apache: The Definitive Guide published by O'Reilly. A complete list of API routines (like ap_get_basic_auth_pw) can be
found in the appendix. Professional module writers may also want to check out
Writing Apache Modules with Perl and C
Source Code
- mod_auth_openvms.c
- this module was published by Compaq before the merger with HP (now HPE)
- This 2015 production file hasn't changed since 2003-01-15 (presented here for reference purposes)
- Comments: with the benefit of 12 years of hindsight
- Parts appear to be derived from a Solaris program written before y2k (keep in mind that Solaris
usernames are case sensitive)
- as written, this plugin is very inefficient. It contains two hooks:
- hook-1 will fire for every transaction "requiring Apache authentication" in the whole
system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been
defaulted to DECLINE. When enabled, will access SYSUAF
- hook-2 will fire for every transaction "requiring an Apache authorization" in the whole
system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been
defaulted to DECLINE. When enabled, will search for the current USERNAME in an external GROUP FILE.
- It might have been a better idea for Compaq to operate both routines under a single hook which would make the plugin
slightly less parasitic. Alternatively, they could have published two plugin modules: "authentication only" and
"authentication and authorization"
- Another problem with the code under hook-2 is that the username test is case-sensitive. OpenVMS usernames are
up-cased before going through SYSUAF authentication. This means that the GROUP FILE associated with hook-2 needs to
contain both upper and lower versions of every username but someone logging in with a mixed-case username will still
be bounced.
- the first two "c language" tests near the top of hook-1 are in the wrong order. This plugin could be made a tiny bit
less parasitic if the very first line of code was the test of fUserEnable rather than a call to
ap_get_basic_auth_pw (why test for a password in places where the module is disabled?)
- my DCL build script: build_mod_auth_openvms.com
- mod_auth_vms_ext.c
- it works with cookies, and/or username+passwords
- it can (optionally) engage an indexed RMS-based "session cache" so the external authenticator is called less often (I do
not think it would be too difficult to move this into a "global section" -- also called a shared memory object)
- it spawns an external authenticator (which can connect to any kind of database)
- it can command a browser to raise the username-password dialog
- it can (optionally) command the browser to store a COOKIE (cookie data must be provided by your authenticator)
- it can (optionally) do GROUP access restriction tests (group data must be provided by your authenticator)
- DCL build script: build_mod_auth_vms_ext.com
- mod_auth_vms_rms.c
- opens an indexed RMS file to perform application-level authentication (can use characters not allowed in SYSUAF)
- essentially identical to mod_auth_vms_ext.c (above) except that lib$spawn was replaced by directly accessing an RMS-file
- DCL build script: build_mod_auth_vms_rms.com
gSOAP Specific (c-only)
When you check out the sample programs delivered with the gSOAP stack they seem fairly straight forward (right now I'm thinking
about "calc") but most were created using the WSDL-last method (a.k.a. code-first method).
When someone hands you a WSDL then says "build client and server apps which will be part of a two way messaging system" things
get complicated fast. You will be forced to use the WSDL-first method, and the tool wsdl2h
(read as: wsdl-to-h) will stuff your SOAP variables deep inside complicated structures and unions then pass the whole mess to
you with a pointer. If the WSDL contains more than 100 fields like the one I saw a few months ago, you might want to tackle a
smaller project first just to develop skills.
Last year I used the WSDL-last method to create a Java-based web service under Tomcat/AXIS2 called "SimpleService" which can be
viewed here. This service only had three methods: ssADD, ssSubtract, and ssEcho. Then I used AXIS2 to
generate a WSDL which would be used to build this gSOAP service with the same functionality.
gSOAP Specific (c++)
Brett Cameron and John Apps have provided a really good set of gSOAP sample programs written in VMS-C but nothing (as of 2012)
written in VMS-C++. So here are my unofficial offerings of sample programs tweaked for use with VMS-C++ for OpenVMS.
Notes:
- Large gSOAP programs written in C++ seem clearer than the same ones written in C. Consider using C++ if you you want your
gSOAP code to me more easily maintainable by other programmers.
- When the gSOAP preprocessor "soapcpp2" generates C++ code it will use a ".CPP" extension
- VMS-C++ defaults to a file extension of ".CXX" while competing compilers default to ".CPP" so be sure to include a fully
qualified file name.
gSOAP Calc Samples
calcClient.cpp
calcServer.cpp
build_calc_cpp.com
CalcSslClient.cpp |
gSOAP sample Client program from //gsoap/samples/calc++/calcClient.cpp
gSOAP sample Server program from //gsoap/samples/calc++/calcServer.cpp
DCL script to build these two gSOAP sample programs on OpenVMS
gSOAP sample Client program (w/OpenSSL) from //gsoap/samples/calc++/calcClient.cpp
(this last demo has only been tested on Windows-7) |
Porting OpenSSL to OpenVMS
While it is possible to build OpenSSL from public sources, at least one of the demonstration applications will not work without
modifications. The main reason for this involves use of the UNIX I/O select()
statement to test for I/O readiness. While this is perfectly acceptable in the UNIX/Linux world, select()
can only be used to wait for network I/O in the VMS and Windows worlds. There are two possible fixes for
this:
- use a different technique to detect the start of keyboard I/O
- see other platform examples in latest versions of s_client.c where a home-brewed function called kbhit
(keyboard hit) is used to detect when the "C" program should begin a keyboard read operation in Windows
- this is the el-cheapo solution which should be okay for s_client.c since it is not production quality
code.
- use a VMS-recommended technique to use a socket to connect to SYS$COMMAND
- this solution requires a socket interface at the SSL end and a SYS$QIO at the keyboard end
- this gold-plated solution can be seen in the program term_sock.c (terminal socket) shown just
below
Programs |
Notes |
s_client.c |
This program was found in kit CPQ-OpenSSL-1.0A (OpenSSL 0.9.6b) published July 2001 by Compaq
It shows how to create a socket (TerminalSocket) to communicate with stdin (your keyboard)
No Compaq/HP copyright was included with this package.
While this program is copyrighted by Eric Young, the VMS changes are not copyrighted by anyone.
"I think" it was modified by someone at Compaq Computer Corporation |
term_sock.c |
This program was found in kit CPQ-OpenSSL-1.0A and is used to create a socket (TerminalSocket) to
communicate with stdin (your keyboard). No Compaq/HP copyright was included with this package or this file. "I think" it
was written by someone at Compaq Computer Corporation. It allows the programmer to get around the "bad select 38"
problem |
term_sock.h |
This c header supports term_sock.c No Compaq/HP copyright was included with this package or this
file. "I think" it was written by someone at Compaq Computer Corporation |
select_demo.c |
A demo program (derived from HP's UNIX to OpenVMS porting guide) to show how
to use select() |
Webpage Hit Counters
We have always been told to never use hit-counters because they consume precious resources. But many rinky-dink sites, especially
some on corporate intranets sitting behind a firewall, need them for good P.R. with other departments.
Image-based Counters
These seem to be the gold-standard in counters. Between 1995 and 2000 (when web servers were powerful platforms serving up
mostly static webpages to underpowered desktop PCs sporting Pentium-2 or Pentium-3
processors) most sites used a bit of freeware written by Muhammad Muquit named "count".
Needless to say I am envious of his programming skills. But this solution places a more-than-trivial computational burden on the
server. Why? Updating a counter file is the easy part. Using the digits to reference a graphics library of individual digits
then assemble the binary pieces scan-line by scan-line into a resultant GIF is the hard part.
Text-based Counters
I'm running an overworked decade-old server (AlphaServer DS20e installed in 2002) and I think the time has come to shift the
computational burden from the server to the client. I have a little "C" program ready which increments the counter file then
sends back the plain-text result. The calling webpage uses a small amount of AJAX (~20 lines) to accomplish the task then the
browser is responsible for displaying the count. The browser may not have access to all the cool image sets seen in Muhammad's
offering but that may not be as big a deal as you might think. I will publish it soon.
documents:
- here is the HTML-JavaScript Code to demonstrate my AJAX-based hit counter
- here is the c-based hit counter code. It requires polishing. it also requires the addition
of calls to the OpenVMS distributed lock manager (DLM) to provide better file locking than is possible with
vanilla C routines for UNIX. This should be easy to do and I will publish that as soon as time permits.
Web Page Hit Counter (2.4 + 2.6)
- Muhammad Muquit is the author of a very cool multi-platform
'Hit Counter' which is currently run by many large ISPs in the world (if you don't believe me, just intercept a
generated GIF then look inside where you'll see his name and URL in the comments area; his name continually flies around the
internet but this is a small prices to pay for the efforts of such a gifted programmer)
- Phil Ottewell's code to work with Process Software Corp's Purveyor 2.x Encrypt Web Server
and the code is available at here).
BTW, I also inserted calls to the OpenVMS Distributed Lock Manager to prevent the counter data
file from becoming locked out due to multiple process requiring exclusive write access.
- 2003-05-23: I've got a pre-release of Hit Counter v2.6 for Compaq Secure Web Server version 1.3.26 (a.k.a. CSWS, or Apache
for OpenVMS) working on my AlphaServer-DS20
Technical Observations (2013-11-xx)
- As currently implemented on OpenVMS under Apache, count.exe will produce two files. File-1 contains "the
MIME header text" while file-2 contains "the binary graphic data"
- In 2013 I shifted my webserver to Ruthless Browser Caching which caused the web-counter image to
stop updating (and we still need it for departmental PR)
- Since I hadn't worked on this code for ten years, I started by rewriting the DCL script to make it smaller and more
efficient
- Next, I replaced the copying of file-1 (the MIME header) with output from DCL. To do this I replaced one image execution
(APACHE$DCL_BIN) with another (APACHE$FLIP_CCL) which is still less expensive since we do not need to locate, then transfer,
file-1.
- After that I added a second MIME line ("Cache-Control: no-cache, no-store") to deal with servers implementing Ruthless
Browser Caching.
- It should be possible to modify the C sources to do away with creating intermediate files. The result would be a two-line
DCL script. I publish this solution as soon as possible
Updated DCL script for use with CSWS (aka Apache on OpenVMS)
$ on error then $ continue
$ t = f$cvtime(,,"TIME") -":"-":"-"."
$ p = f$getjpi("","pid")
$ f = f$trnlnm("csws_server_root") + "count_" + p +"_" + t +".gif"
$ set prot=(S:rwed,o:rwed,g:rwed,W:rwed)/default
$ define/user/nolog sys$output 'f'
$ mcr www_root:[scripts]count.exe
$ m=1 ! DCL
$ if (m.eq.0)
$ then ! MIME from a file
$ APACHE$DCL_BIN 'f';1 ! MIME header
$ else ! MIME from DCL
$ APACHE$FLIP_CCL
$! write sys$output f$fao("!AS!/!/","Content-type: image/gif")
$ write sys$output f$fao("!AS!/!AS!/!/","Content-type: image/gif","Cache-Control: no-cache, no-store")
$ endif
$ APACHE$DCL_BIN 'f';2 ! GIF data
$ del/nolog 'f';*
$ exit
Hybrid Programming (VMS-BASIC calling VMS-C)
All so-called "DEC languages" (VMS-COBOL, VMS-FORTRAN, VMS-Pascal, VMS-BASIC, etc) can easily pass variables (including strings
and arrays) to each other. Passing variables between "DEC languages" and code written in C, C++, or MACRO is a little more
difficult but not impossible.
Some BASIC solutions can be anywhere between impractical to impossible (LDAP springs to mind). Here are your options:
- "rewrite C programs into BASIC"
- this will be time consuming if not error prone
- VMS does not have access to some c-language primitives like unsigned variables
- BASIC programs could call C programs indirectly which means passing data via "shell variables" or "logical names"
- BASIC programs could be linked to C programs. For the last method you need to know how to pass variables back and forth
Here are some examples
- Hybrid Demo 1
- this BASIC program basic_calling_c_demo1.bas calls C functions defined
here: basic_calling_c_demo1_part2.c
- demonstrates passing variables (long, word, byte, and VMS string) to C; one function returns a result while one does not
- also demonstrates how C can directly access global variables declared in BASIC (ugh!)
- caveat: the string can only be accessed in C as read only; see demo-4 below to learn how to
properly modify the BASIC string from C
- here is the DCL build script for demos 1-5: basic_calling_c_demo.com
- Hybrid Demo 2
- Hybrid Demo 3
- Hybrid Demo 4
- this BASIC program basic_calling_c_demo4.bas calls a C function defined
here: basic_calling_c_demo4_part2.c
- a variable length VMS string is passed from BASIC to C where the string is extended then sent back
- teaser: new string storage space must be allocated via str$get1_dx
rather than malloc
- hacking: write a small BASIC program which only manipulates a few strings then compile it with
the "/list/machine" command-line parameters. For the most part the BASIC program will be making calls to the STR$ run-time
library
- Hybrid Demo 5
- if you have ever wondered about the differences between "passing an array by ref" vs. "passing
an array by desc" then this is the demo for you.
BTW...
- when you pass an array by desc you are passing the address of the descriptor.
- when you pass an array by ref you are passing the address of the array data (the receiving
routine may need some additional information like array bounds, etc.)
- this BASIC program basic_calling_c_demo5.bas calls C functions defined
here: basic_calling_c_demo5_part2.c
- in the first test, a 1-dimesion array of strings is created at run-time then passed to C (by ref) for processing
- the C routines lengthen all strings for use by BASIC
- in the second test, a 2-dimension array of strings is created at run-time then passed to C (by ref) for processing
- the C routines lengthen all strings for use by BASIC
- the third and fourth tests pass the arrays to C (by desc) so you can peek at internals (how many
dimensions? what are the upper and lower bounds of each dimension? what is the striding parameter?)
- food for thought:
http://h30266.www3.hpe.com/odl/axplp/progtool/cpqc73/6180profile_011.html#variable_length_array_sec
- Other Hybrid Demos
Executive
Summary: for mysql_demo16.bas + mysql_api_200.c:
(1) Suppose you needed to open a table that would return 35 fields with a maximum of 500 records. Just create a
2-dimensional array of size 500 x 35 in BASIC, then copy the array details into a common, then call the routines defined in
"mysql_api_200.c". That c program will make the call to the database, then will parse the results, then will update the
2-dimensional array. When you exit back to BASIC, the array will already be sitting there ready to be processed in any way
you wish.
(2) this program allows you to have up to 10 tables open at once.
Java and AXIS2 Web Services
OpenVMS DCL scripts
caveat: the ".com" file extension in VMS/OpenVMS predates MS-DOS and MS-Windows. Here it means "commands"
Category |
DCL Scripts |
Notes |
Calc Time Difference |
calc_date_time_diff_001.com calc_date_time_diff_002.com |
quick hack
much better |
Create Certificates |
create_certs_alt.com |
a hack to create SSL certificates (for development use only)
1) create a "CA key" and "CA cert"
2) create a "server key" and "server csr" (certificate signing request)
3) use the "CA cert" to sign the "server csr" |
DECnet communication |
decnet_demo_c.com decnet_demo_s.com |
client (start a remote server then send it some messages)
server (echo messages back to client) |
Internet |
decode_sys_rem_id.com |
logical name SYS$REM_ID contains a hex encoded representation of the connected TCP/IP address. This
demo shows how to display it in standard IPv4 format. |
Intercepting an
FTP/SFTP login |
systartup_vms_ftp_stub.com
login.com
login_helper.com |
Example DCL script for properly setting up FTP/SFTP
DCL script to intercept an FTP/SFTP login then submit a login helper job
DCL script which will wait for the network partner to exit then process received file(s)
(similar to what happens at the end of an XCOM transaction) More notes can be found here: OpenVMS
Notes: TCPware |
FTP Scripts |
ftp_multi_stack_scripting.com |
a script which can FTP from three different stacks: MultNet, TCPware, and TCPIP Services |
uppercase - lowercase switcheroo |
sense_nonsense.com downcase_filenames.com |
a tool to easily toggle your account between two modes: case-sensitive and normal
a tool to easily down-case filenames. Intended for SSH key files. |
Apache Support |
rotate_apache_logs.com |
a little batch job to rotate apache client log files every 24 hours
also will (optionally) do monthly maintenance of apache process logs |
Environmental
Monitoring (AXP only) |
env_check_100.com env_check_101.com |
original script
tweaked for calling from your LOGIN.COM |
MSA-RAID Monitor |
check_raid.com |
this self-submitting script can check your RAID controller, and drives, as often as you like |
OpenVMS Macro32 source code
OpenVMS Programming
Introduction to OpenVMS programming
It was a lot tougher before y2k
It was a lot tougher learning how to write VMS applications in the 1980s. Why? DEC charged big-bucks for everything including
hardware, software, training and documentation (including pocket programming cards!) Many IT people wondered if that was why DEC
used a dollar sign ($) for the VMS/DCL command-line prompt.
First off, you needed the 32-binder Orange Wall of VMS documentation which
I remember being around $5k for the initial purchase. Then, even if you already knew how to program in one of the many
DEC-supported programming languages, you still needed to attend DEC language-specific courses to learn how to do system calls.
This meant you needed to work for a company with deep pockets, or needed to get VMS-specific training in college or university.
In the early 1990s, DEC renamed the product line from VMS to OpenVMS at the same time they introduced the 64-bit Alpha hardware
line as a successor to 32-bit VAX. During this time the Orange Wall became
the Gray Wall
Then Things Changed
In 1998, DEC was purchased by "PC manufacturer" Compaq.
- The first thing Compaq did was to convert the 32-binder Gray Wall (with a single CD-ROM. GOOD This
allowed them to shut down their document publishing business)
- Next, they put all their OpenVMS documents online so that anyone could access them for free. GOOD
- Next, they shut down their educational services division to save money. BAD Technically
speaking, they didn't just shut it down cold, they attempted to pass the business to third-party educators like they did with
their PC products. But the scheme flopped. (hey, what would you expect from a shoot-from-the-hip PC company?)
In 2002, HP purchased (err, merged with) Compaq. HP restored some in-house training but now OpenVMS was just one of many
software offerings from HP's, and I think OpenVMS has been lost in the mix.
Although my employer did provide us with the 32-binder Gray Wall, we had to learn how to interface VMS BASIC to VMS the
good old-fashioned hard way: by hacking. In 1993 I fortunately stumbled across a copy
of Writing VAX/VMS Applications Using Pascal (Author: Theo De Klerk, Published: 1991 by Digital Press) in DEC's
student retail store in Bedford, Massachusetts. I had learned Apple Pascal in late 1970s so found it
relativity easy to convert many of these programs into VMS-BASIC and VMS-C. This book contained many descriptions of VMS
subsystems and was full of "system calls" on how to access them. These books pop up every so often on www.bookfinder.com
and I think every serious OpenVMS programmer should own a copy.
Okay so that was then and this is now. To the best of my knowledge, you can only get the OpenVMS Documentation CD-ROM
by purchasing it from HP, buying a used copy on eBay, or begging a friend to make you a copy. But if you use HP's online
documentation along with examples from this book, you will learn how to become a VMS BASIC developer much faster than me.
Online HP Manuals + Documents
Advanced hacking (system services)
Many programmers do not know about this technique to trace "system service calls" originating from your process (DCL or
otherwise). So if you have the necessary privs, and are on an Alpha or Itanium, then try this experiment:
Legend:
<sr> = system response
<ur> = user response
--------------------------------------------------------
<sr> $ ! this is my DCL prompt
<ur> set process /sslog=(state=on,count=4,flag=arg) ! enable tracing on YOUR process
<sr> $
<ur> run your-program.exe
<sr> ...run the program then exit...
$
<ur> set process /sslog=(state=off) ! disable tracing
<sr> $
<ur> set process /sslog=(state=unload) ! this closes the file
<sr> $
<ur> ANALYZE/SSLOG/FULL/WIDE/OUTPUT=ss_trace.txt ! human readable data goes to this file
<sr> $
<ur> type/page ss_trace.txt
Official Documents:
Noteworthy C/C++ demos at other sites
Noteworthy C/C++ articles and manuals
How to recover from FTP-induced saveset corruption
$set file /attr=(RFM:FIX,MRS:512,LRL=512,ORG=SEQ,RAT=NONE) yourfile.sav
Back to
Home
Neil Rieck
Waterloo, Ontario, Canada.