OpenVMS Notes: ZIP and UNZIP

  1. The information presented here is intended for educational use by qualified OpenVMS technologists.
  2. The information presented here is provided free of charge, as-is, with no warranty of any kind.
Edit: 2016-08-31 (added gzip examples)

Introduction

There are lots of better websites documenting problems using UNZIP on VMS so I will not attempt to reproduce that work here. Like most of my other published stuff, I am going to explain something that just (2012-07-09) happened to me. Maybe this information can help one other poor devil who ended up here via Google :-)

Strange things after downloading a zipped file from HP

Experiment:

legend: <sr> = system response
        <ur> = user response
-------------------------------------------------
<sr> $
<ur> unzip DEC-AXPVMS-GNV-V0201-003-1.ZIP
<sr> bla...bla...bla...
     $
<ur> unzip DEC-AXPVMS-GNV-V0300-001-1.ZIP
<sr> bla...bla...bla...
     $
<ur> product list gnv
<sr>
	1 - DEC AXPVMS GNV V3.0-1               Layered Product
	2 - DEC AXPVMS GNV V2.1-3               Layered Product
	? - Help
	E - Exit
 
     Choose one or more items from the menu:
<ur> 1
<sr> Performing product kit validation of signed kits ...
 
     %PCSI-E-VALFAILED, validation of KAWC99$DRA1:[CSMIS.USR.][ADMCSM.NEIL]DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED;1 failed
     -PCSI-E-HPC_TEXT, validate_finalize: Verification of signed file failed
     %PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition
     $
<ur> product list gnv
<sr>
	1 - DEC AXPVMS GNV V3.0-1               Layered Product
	2 - DEC AXPVMS GNV V2.1-3               Layered Product
	? - Help
	E - Exit
 
     Choose one or more items from the menu:
<ur> 2
<sr> Performing product kit validation of signed kits ...
     %PCSI-I-ESWVALPASSED, validation of KAWC99$DRA1:[CSMIS.USR.][ADMCSM.NEIL]dec-axpvms-gnv-v0201-003-1.pcsi$compressed;1 succeeded
 
     The following product has been selected:
	DEC AXPVMS GNV V2.1-3                  Layered Product
 
	Do you want to continue? [YES]
<ur> <enter>
<sr> KAWC99$DRA1:[CSMIS.USR.][ADMCSM.NEIL]dec-axpvms-gnv-v0201-003-1.pcsi$compressed;1
	 Format:   Compressed
	 Kit Size: 86272 blocks and 144128 blocks when decompressed
	 Manifest: Kit indicates that a manifest file was created for it
		   Associated manifest file was used to successfully validate kit
 
     --------------------------------------------------------------------------------
     CONTENTS OF KIT USING RELATIVE FILE SPECIFICATION
     --------------------------------------------------------------------------------
     [000000]DEC-AXPVMS-GNV-v0201-003-1.PCSI$TLB
     [...snip...]
     $
  • Okay so the folks at OpenVMS Software Support say these files are supposed to be pure binary so let's look at them with "dir/full"
  • It looks like "record format" and "record attributes" are mangled on both files even only one of them fails kit validation (this means GNV 2.1-3 passed by sheer luck)
  • On top of that they both fail file analysis using "ana/rms"
<sr> $
<ur> dir *.pcsi$compressed;*/full
<sr> Directory CSMIS$USER3:[ADMCSM.NEIL]

     dec-axpvms-gnv-v0201-003-1.pcsi$compressed;1                  File ID:  (13852,26,0)          
     Size:        86272/86272      Owner:    [NEIL]
     Created:     5-NOV-2009 15:54:08.00
     Revised:     5-NOV-2009 15:54:08.00 (0)
     Expires:    <None specified>
     Backup:     <No backup recorded>
     Effective:  <None specified>
     Recording:  <None specified>
     Accessed:    9-JUL-2012 18:43:27.51
     Attributes:  5-NOV-2009 15:54:08.00
     Modified:    5-NOV-2009 15:54:08.00
     Linkcount:  1
     File organization:  Sequential
     Shelved state:      Online 
     Caching attribute:  Writethrough
     File attributes:    Allocation: 86272, Extend: 16384, Global buffer count: 0, No version limit
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     RMS attributes:     None
     Journaling enabled: None
     File protection:    System:RWED, Owner:RWED, Group:, World:RWED
     Access Cntrl List:  None
     Client attributes:  None

     DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED;1                  File ID:  (745,717,0)           
     Size:       393450/393456     Owner:    [NEIL]
     Created:    18-NOV-2011 15:10:00.00
     Revised:    18-NOV-2011 15:10:00.00 (0)
     Expires:    <None specified>
     Backup:     <No backup recorded>
     Effective:  <None specified>
     Recording:  <None specified>
     Accessed:    9-JUL-2012 18:42:07.72
     Attributes: 18-NOV-2011 15:10:00.00
     Modified:   18-NOV-2011 15:10:00.00
     Linkcount:  1
     File organization:  Sequential
     Shelved state:      Online 
     Caching attribute:  Writethrough
     File attributes:    Allocation: 393456, Extend: 16384, Global buffer count: 0, No version limit
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     RMS attributes:     None
     Journaling enabled: None
     File protection:    System:RWED, Owner:RWED, Group:RE, World:
     Access Cntrl List:  None
     Client attributes:  None
	
     Total of 2 files, 479722/479728 blocks.
     $
  • We now have three alternatives:
     
    1. A little hacking proves these files can be installed as-is but without kit validation (not recommended)
      prod inst gnv /option=novalid
    2. HP says the files can be repaired like so:
      set file/attribute=(rfm:fix,lrl:512,mrs:512,org:seq,rat:none) DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED
    3. But what about changes to how we UNZIP?

How UNZIP is supposed to be used on VMS (and OpenVMS)

  • When someone ZIPs a VMS file they are supposed to use the following syntax which will leave "VMS file attribute info" in the zip archive (I believe HP is aware of this)
    <sr> $					!
    <ur> zip "-V" yada.zip *.* -x *.zip	! zip whole directory (preserving VMS file attributes)
    <sr> adding: .BASHRC (deflated 52%)
         [...SNIP...]
         $ 

    Caveat: when UNZIP is properly installed on VMS systems you will find a second DCL symbol called ZIPINFO which runs UNZIP with switch "-Z"

  • People preparing for an UNZIP session on VMS should first look into the file to see if any attributes were saved (I was not aware of this until now)
    <sr> $							!
    <ur> unzip "-Z" DEC-AXPVMS-GNV-V0201-003-1.ZIP		! 
    <sr> Archive:  CSMIS$USER3:[ADMCSM.NEIL]DEC-AXPVMS-GNV-V0201-003-1.ZIP;1
         Zip file size: 36339064 bytes, number of entries: 2
         RWED,,RWED  2.3 vms 44171264 bx defN 09-Nov-05 10:54 dec-axpvms-gnv-v0201-003-1.pcsi$compressed
         RWED,,RWED  2.3 vms     8841 bx defN 09-Nov-05 10:54 dec-axpvms-gnv-v0201-003-1.pcsi$compressed_esw
         2 files, 44180105 bytes uncompressed, 36338678 bytes compressed:  17.7%
         $
    <ur> unzip "-Z" DEC-AXPVMS-GNV-V0300-001-1.ZIP
    <sr> Archive:  CSMIS$USER3:[ADMCSM.NEIL]DEC-AXPVMS-GNV-V0300-001-1.ZIP;3
         Zip file size: 165666597 bytes, number of entries: 3
         RWED,RE,    2.3 vms 201446400 bx defN 11-Nov-18 04:39 DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED
         RWED,RE,    2.3 vms     8841 bx defN 11-Nov-18 04:56 DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED_ESW
         RWED,RE,    2.3 vms     5732 tx defN 11-Nov-18 04:43 DEC-AXPVMS-GNV-V0300-001-1.PCSI$COMPRESSED_HPC
         3 files, 201460973 bytes uncompressed, 165666025 bytes compressed:  17.8%
         $ 
  • In the examples just shown:
    • field 1: file protection bits
    • field 2: zip version level
    • field 3: source OS (provides a context to field 1)
    • field 4: uncompressed file size in bytes
    • field 5a: (data type field)
      • "t" means text file (upper case "T" means encrypted text)
      • "b" means binary files (upper case "B" means encrypted binary)
    • field 5b (extra field):
      Character literal extended local header extra field
      - dash N N
      l lower case L Y N
      x lower case X N Y
      X upper case X Y Y
      note: to see the contents of the extra field, use this command: unzip "-Zv" DEC-AXPVMS-GNV-V0300-001-1.ZIP
    • field 6: compression method (defN = Deflate Normal)
    • reference: hhttp://www.info-zip.org/mans/zipinfo.html
       
  • Once we notice there is extra data, we are supposed to unzip with the auto-binary mode like so:
    <ur> unzip -b DEC-AXPVMS-GNV-V0201-003-1.ZIP		!
  • Hey, switch "-b" is never mentioned in the online help for UNZIP-5.5.2

    So upgrade to UNZIP 6.0 then you will find it in the extended help:
    <sr> $
    <ur> unzip -? 						! get regular help from UNZIP 6.0
    <sr> UnZip 6.00 of 20 April 2009, by Info-ZIP.  For more details see: unzip -v.
    
         Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]
           Default action is to extract files in list, except those in xlist, to exdir;
           file[.zip] may be a wildcard.  -Z => ZipInfo mode ("unzip -Z" for usage).
         
           -p  extract files to pipe, no messages     -l  list files (short format)
           -f  freshen existing files, create none    -t  test compressed archive data
           -u  update files, create if necessary      -z  display archive comment only
           -v  list verbosely/show version info       -T  timestamp archive to latest
           -x  exclude files that follow (in xlist)   -d  extract files into exdir
         modifiers:
           -n  never overwrite or make a new version of an existing file
           -o  always make a new version (-oo: overwrite original) of an existing file
           -q  quiet mode (-qq => quieter)            -a  auto-convert any text files
           -j  junk paths (do not make directories)   -aa treat ALL files as text
           -C  match filenames case-insensitively     -L  make (some) names lowercase
           -X  restore owner/ACL protection info      -V  retain VMS version numbers
           -Y  treat ".nnn" as ";nnn" version         -2  force ODS2 names
           --D restore dir (-D: no) timestamps        -M  pipe through "more" pager
           (Must quote upper-case options, like "-V", unless SET PROC/PARSE=EXTEND.)
    
         See "unzip -hh" or unzip.txt for more help.  Examples:
           unzip data1 -x joe   => extract all files except joe from zipfile data1.zip
           unzip "-V" foo "Bar" (Quote names to preserve case, unless SET PROC/PARS=EXT)
           unzip -fo foo vms.c  => quietly replace existing vms.c if archive file newer
         $
    <ur> unzip -hh						! get extended help from UNZIP 6.0
    <sr> Extended Help for UnZip
    
         See the UnZip Manual for more detailed help
    
         UnZip lists and extracts files in zip archives.  The default action is to
         extract zipfile entries to the current directory, creating directories as
         needed.  With appropriate options, UnZip lists the contents of archives
         instead.
    
         Basic unzip command line:
           unzip [-Z] options archive[.zip] [file ...] [-x xfile ...] [-d exdir]
    
         Some examples:
           unzip -l foo.zip        - list files in short format in archive foo.zip
    
           unzip -t foo            - test the files in archive foo
    
           unzip -Z foo            - list files using more detailed zipinfo format
    
           unzip foo               - unzip the contents of foo in current dir
    
           unzip -a foo            - unzip foo and convert text files to local OS
    
         If unzip is run in zipinfo mode, a more detailed list of archive contents
         is provided.  The -Z option sets zipinfo mode and changes the available
         options.
    
         Basic zipinfo command line:
           zipinfo options archive[.zip] [file ...] [-x xfile ...]
           unzip -Z options archive[.zip] [file ...] [-x xfile ...]
    
         Below, Mac OS refers to Mac OS before Mac OS X.  Mac OS X is a Unix based
         port and is referred to as Unix Apple.
    
         unzip options:
           -Z   Switch to zipinfo mode.  Must be first option.
           -hh  Display extended help.
           -A   [OS/2, Unix DLL] Print extended help for DLL.
           -c   Extract files to stdout/screen.  As -p but include names.  Also,
                  -a allowed and EBCDIC conversions done if needed.
           -f   Freshen by extracting only if older file on disk.
           -l   List files using short form.
           -p   Extract files to pipe (stdout).  Only file data is output and all
                  files extracted in binary mode (as stored).
           -t   Test archive files.
           -T   Set timestamp on archive(s) to that of newest file.  Similar to
                zip -o but faster.
           -u   Update existing older files on disk as -f and extract new files.
           -v   Use verbose list format.  If given alone as unzip -v show version
                  information.  Also can be added to other list commands for more
                  verbose output.
           -z   Display only archive comment.
    
         unzip modifiers:
           -a   Convert text files to local OS format.  Convert line ends, EOF
                  marker, and from or to EBCDIC character set as needed.
           -b   Treat all files as binary.  [Tandem] Force filecode 180 ('C').
                  [VMS] Autoconvert binary files.  -bb forces convert of all files.
           -B   [UNIXBACKUP compile option enabled] Save a backup copy of each
                  overwritten file in foo~ or foo~99999 format.
           -C   Use case-insensitive matching.
           -D   Skip restoration of timestamps for extracted directories.  On VMS this
                  is on by default and -D essentially becames -DD.
           -DD  Skip restoration of timestamps for all entries.
           -E   [MacOS (not Unix Apple)]  Display contents of MacOS extra field during
                  restore.
           -F   [Acorn] Suppress removal of NFS filetype extension.  [Non-Acorn if
                  ACORN_FTYPE_NFS] Translate filetype and append to name.
           -i   [MacOS] Ignore filenames in MacOS extra field.  Instead, use name in
                  standard header.
           -j   Junk paths and deposit all files in extraction directory.
           -J   [BeOS] Junk file attributes.  [MacOS] Ignore MacOS specific info.
           -K   [AtheOS, BeOS, Unix] Restore SUID/SGID/Tacky file attributes.
           -L   Convert to lowercase any names from uppercase only file system.
           -LL  Convert all files to lowercase.
           -M   Pipe all output through internal pager similar to Unix more(1).
           -n   Never overwrite existing files.  Skip extracting that file, no prompt.
           -N   [Amiga] Extract file comments as Amiga filenotes.
           -o   Overwrite existing files without prompting.  Useful with -f.  Use with
                  care.
           -P p Use password p to decrypt files.  THIS IS INSECURE!  Some OS show
                  command line to other users.
           -q   Perform operations quietly.  The more q (as in -qq) the quieter.
           -s   [OS/2, NT, MS-DOS] Convert spaces in filenames to underscores.
           -S   [VMS] Convert text files (-a, -aa) into Stream_LF format.
           -U   [UNICODE enabled] Show non-local characters as #Uxxxx or #Lxxxxxx ASCII
                  text escapes where x is hex digit.  [Old] -U used to leave names
                  uppercase if created on MS-DOS, VMS, etc.  See -L.
           -UU  [UNICODE enabled] Disable use of stored UTF-8 paths.  Note that UTF-8
                  paths stored as native local paths are still processed as Unicode.
           -V   Retain VMS file version numbers.
           -W   [Only if WILD_STOP_AT_DIR] Modify pattern matching so ? and * do not
                  match directory separator /, but ** does.  Allows matching at specific
                  directory levels.
           -X   [VMS, Unix, OS/2, NT, Tandem] Restore UICs and ACL entries under VMS,
                  or UIDs/GIDs under Unix, or ACLs under certain network-enabled
                  versions of OS/2, or security ACLs under Windows NT.  Can require
                  user privileges.
           -XX  [NT] Extract NT security ACLs after trying to enable additional
                  system privileges.
           -Y   [VMS] Treat archived name endings of .nnn as VMS version numbers.
           -$   [MS-DOS, OS/2, NT] Restore volume label if extraction medium is
                  removable.  -$$ allows fixed media (hard drives) to be labeled.
           -/ e [Acorn] Use e as extension list.
           -:   [All but Acorn, VM/CMS, MVS, Tandem] Allow extract archive members into
                  locations outside of current extraction root folder.  This allows
                  paths such as ../foo to be extracted above the current extraction
                  directory, which can be a security problem.
           -^   [Unix] Allow control characters in names of extracted entries.  Usually
                  this is not a good thing and should be avoided.
           -2   [VMS] Force unconditional conversion of names to ODS-compatible names.
                  Default is to exploit destination file system, preserving cases and
                  extended name characters on ODS5 and applying ODS2 filtering on ODS2.
    
         Wildcards:
           Internally unzip supports the following wildcards:
             ?       (or %% or #, depending on OS) matches any single character
             *       matches any number of characters, including zero
             [list]  matches char in list (regex), can do range [ac-f], all but [!bf]
           If port supports [], must escape [ as [[]
           For shells that expand wildcards, escape (\* or "*") so unzip can recurse.
    
         Include and Exclude:
           -i pattern pattern ...   include files that match a pattern
           -x pattern pattern ...   exclude files that match a pattern
           Patterns are paths with optional wildcards and match paths as stored in
           archive.  Exclude and include lists end at next option or end of line.
             unzip archive -x pattern pattern ...
    
         Multi-part (split) archives (archives created as a set of split files):
           Currently split archives are not readable by unzip.  A workaround is
           to use zip to convert the split archive to a single-file archive and
           use unzip on that.  See the manual page for Zip 3.0 or later.
    
         Streaming (piping into unzip):
           Currently unzip does not support streaming.  The funzip utility can be
           used to process the first entry in a stream.
             cat archive | funzip
    
         Testing archives:
           -t        test contents of archive
           This can be modified using -q for quieter operation, and -qq for even
           quieter operation.
    
         Unicode:
           If compiled with Unicode support, unzip automatically handles archives
           with Unicode entries.  Currently Unicode on Win32 systems is limited.
           Characters not in the current character set are shown as ASCII escapes
           in the form #Uxxxx where the Unicode character number fits in 16 bits,
           or #Lxxxxxx where it doesn't, where x is the ASCII character for a hex
           digit.
    
    
         zipinfo options (these are used in zipinfo mode (unzip -Z ...)):
           -1  List names only, one per line.  No headers/trailers.  Good for scripts.
           -2  List names only as -1, but include headers, trailers, and comments.
           -s  List archive entries in short Unix ls -l format.  Default list format.
           -m  List in long Unix ls -l format.  As -s, but includes compression %.
           -l  List in long Unix ls -l format.  As -m, but compression in bytes.
           -v  List zipfile information in verbose, multi-page format.
           -h  List header line.  Includes archive name, actual size, total files.
           -M  Pipe all output through internal pager similar to Unix more(1) command.
           -t  List totals for files listed or for all files.  Includes uncompressed
                 and compressed sizes, and compression factors.
           -T  Print file dates and times in a sortable decimal format (yymmdd.hhmmss)
                 Default date and time format is a more human-readable version.
           -U  [UNICODE] If entry has a UTF-8 Unicode path, display any characters
                 not in current character set as text #Uxxxx and #Lxxxxxx escapes
                 representing the Unicode character number of the character in hex.
           -UU [UNICODE]  Disable use of any UTF-8 path information.
           -z  Include archive comment if any in listing.
    
         funzip stream extractor:
           funzip extracts the first member in an archive to stdout.  Typically
           used to unzip the first member of a stream or pipe.  If a file argument
           is given, read from that file instead of stdin.
    
         funzip command line:
           funzip [-password] [input[.zip|.gz]]
    
         unzipsfx self extractor:
           Self-extracting archives made with unzipsfx are no more (or less)
           portable across different operating systems than unzip executables.
           In general, a self-extracting archive made on a particular Unix system,
           for example, will only self-extract under the same flavor of Unix.
           Regular unzip may still be used to extract embedded archive however.
    
         unzipsfx command line:
           <unzipsfx+archive_filename>  [-options] [file(s) ... [-x xfile(s) ...]]
    
         unzipsfx options:
           -c, -p - Output to pipe.  (See above for unzip.)
           -f, -u - Freshen and Update, as for unzip.
           -t     - Test embedded archive.  (Can be used to list contents.)
           -z     - Print archive comment.  (See unzip above.)
    
         unzipsfx modifiers:
           Most unzip modifiers are supported.  These include
           -a     - Convert text files.
           -n     - Never overwrite.
           -o     - Overwrite without prompting.
           -q     - Quiet operation.
           -C     - Match names case-insensitively.
           -j     - Junk paths.
           -V     - Keep version numbers.
           -s     - Convert spaces to underscores.
           -$     - Restore volume label.
    
         If unzipsfx compiled with SFX_EXDIR defined, -d option also available:
           -d exd - Extract to directory exd.
         By default, all files extracted to current directory.  This option
         forces extraction to specified directory.
    
         See unzipsfx manual page for more information.
    
         $

Real-world Experiments

VMS Rules:

  • always zip using switch "-V"
    • you are guaranteed ZIP induced changes if you do not
  • always unzip using switch "-b"
    • but you can sometimes get away without this switch (not recommended)
Consider the following examples:

Analyze (via the DIR/FULL command) all the BASIC (text) source files here
<sr> $
<ur> dir *.*/col=1							! display all files here
<sr> Directory CSMIS$ROOT3:[DVLP.YADA]

     float_demo_a.bas;1 
     float_demo_b.bas;1 
     folding_time_100.bas;1
     olivaw_100.bas;1   
     olivaw_101.bas;1   
     olivaw_102.bas;1   

     Total of 6 files.
     $
<ur> pipe dir/ful *.* | search sys$input "record ",".bas"		! see how they are encoded
<sr> float_demo_a.bas;1            File ID:  (1106,10,0)           
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     float_demo_b.bas;1            File ID:  (1387,684,0)          
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     folding_time_100.bas;1                    File ID:  (1419,560,0)          
     Record format:      Variable length, maximum 255 bytes, longest 122 bytes
     Record attributes:  Carriage return carriage control
     olivaw_100.bas;1              File ID:  (1495,319,0)          
     Record format:      Variable length, maximum 255 bytes, longest 73 bytes
     Record attributes:  Carriage return carriage control
     olivaw_101.bas;1              File ID:  (1561,996,0)          
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     olivaw_102.bas;1              File ID:  (1655,247,0)          
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     $

Create two (slightly different) zip files
 
<ur> zip archive1.zip *.bas						! create a zip (using no switches)
<sr>   adding: FLOAT_DEMO_A.BAS (deflated 68%)
       adding: FLOAT_DEMO_B.BAS (deflated 68%)
       adding: FOLDING_TIME_100.BAS (deflated 68%)
       adding: OLIVAW_100.BAS (deflated 66%)
       adding: OLIVAW_101.BAS (deflated 71%)
       adding: OLIVAW_102.BAS (deflated 71%)
     $
<ur> zip "-V" archive2.zip *.bas					! create a zip using the -V switch
<sr>   adding: FLOAT_DEMO_A.BAS (deflated 66%)
       adding: FLOAT_DEMO_B.BAS (deflated 66%)
       adding: FOLDING_TIME_100.BAS (deflated 65%)
       adding: OLIVAW_100.BAS (deflated 63%)
       adding: OLIVAW_101.BAS (deflated 68%)
       adding: OLIVAW_102.BAS (deflated 68%)
     $

See how the data was archived (twice)
<ur> unzip "-Z" archive1.zip     					! get details of archive1
<sr> Archive:  CSMIS$ROOT3:[DVLP.YADA]ARCHIVE1.ZIP;1
     Zip file size: 6849 bytes, number of entries: 6
     RWED,RWED,RWED  vms     1522 t- defN 11-Feb-01 08:03 float_demo_a.bas
     RWED,RWED,RWED  vms     1602 t- defN 11-Feb-02 08:34 float_demo_b.bas
     RWED,RWED,RWED  vms     1663 t- defN 10-Sep-28 18:59 folding_time_100.bas
     RWED,RWED,RWED  vms     3466 t- defN 10-Sep-22 11:27 olivaw_100.bas
     RWED,RWED,RWED  vms     6045 t- defN 10-Sep-22 12:17 olivaw_101.bas
     RWED,RWED,RWED  vms     6141 t- defN 10-Sep-22 12:22 olivaw_102.bas
     6 files, 20439 bytes uncompressed, 6183 bytes compressed:  69.7%
     $
<ur> unzip "-Z" archive2.zip						! get details of archive2 
<sr> Archive:  CSMIS$ROOT3:[DVLP.YADA]ARCHIVE2.ZIP;1
     Zip file size: 9406 bytes, number of entries: 6
     RWED,RWED,RWED  vms     1582 bx defN 11-Feb-01 08:03 float_demo_a.bas
     RWED,RWED,RWED  vms     1662 bx defN 11-Feb-02 08:34 float_demo_b.bas
     RWED,RWED,RWED  vms     1754 bx defN 10-Sep-28 18:59 folding_time_100.bas
     RWED,RWED,RWED  vms     3660 bx defN 10-Sep-22 11:27 olivaw_100.bas
     RWED,RWED,RWED  vms     6320 bx defN 10-Sep-22 12:17 olivaw_101.bas
     RWED,RWED,RWED  vms     6420 bx defN 10-Sep-22 12:22 olivaw_102.bas
     6 files, 21398 bytes uncompressed, 7168 bytes compressed:  66.5%
     $
  • all the files in archive1 are marked text "t-"
  • all the files in archive2 are marked provisionally binary "bx" (so processing the extra field is mandatory)
Now unzip archive #1 then inspect the resultant files
<ur> unzip      archive1.zip						! unzip archive1
<sr>	Archive:  CSMIS$ROOT3:[DVLP.YADA]ARCHIVE1.ZIP;1
	new version of float_demo_a.bas? [y]es, [n]o, [A]ll, [N]one, [r]ename:
<ur> A
<sr>	  inflating: float_demo_a.bas        
	  inflating: float_demo_b.bas        
	  inflating: folding_time_100.bas    
	  inflating: olivaw_100.bas          
	  inflating: olivaw_101.bas          
	  inflating: olivaw_102.bas          
	$
<ur> pipe dir/ful *.*;/exc=*.zip | search sys$input "record ",".bas"	! view the results
<sr> float_demo_a.bas;2            File ID:  (1238,1233,0)         
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     float_demo_b.bas;2            File ID:  (1770,173,0)          
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     folding_time_100.bas;2                    File ID:  (1935,558,0)          
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     olivaw_100.bas;2              File ID:  (2217,1397,0)         
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     olivaw_101.bas;2              File ID:  (2238,30,0)           
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     olivaw_102.bas;2              File ID:  (2239,29,0)           
     Record format:      Stream_LF, maximum 0 bytes, longest 0 bytes
     Record attributes:  Carriage return carriage control
     $
  • notice that these files all contain the wrong record format
Now unzip archive #2 (twice) inspect the resultant files each time
<ur> unzip      archive2.zip						! unzip archive 2
<sr> Archive:  CSMIS$ROOT3:[DVLP.YADA]ARCHIVE2.ZIP;1
     new version of float_demo_a.bas? [y]es, [n]o, [A]ll, [N]one, [r]ename:
<ur> A
<sr>   inflating: float_demo_a.bas        
       inflating: float_demo_b.bas        
       inflating: folding_time_100.bas    
       inflating: olivaw_100.bas          
       inflating: olivaw_101.bas          
       inflating: olivaw_102.bas          
     $
<ur> pipe dir/ful *.*;/exc=*.zip | search sys$input "record ",".bas"	! view the results
<sr> float_demo_a.bas;3            File ID:  (2240,17,0)           
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     float_demo_b.bas;3            File ID:  (2247,24,0)           
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     folding_time_100.bas;3                    File ID:  (2253,17,0)           
     Record format:      Variable length, maximum 255 bytes, longest 122 bytes
     Record attributes:  Carriage return carriage control
     olivaw_100.bas;3              File ID:  (2255,455,0)          
     Record format:      Variable length, maximum 255 bytes, longest 73 bytes
     Record attributes:  Carriage return carriage control
     olivaw_101.bas;3              File ID:  (2260,18,0)           
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     olivaw_102.bas;3              File ID:  (2270,16,0)           
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     $
<ur> unzip   -b archive2.zip						! unzip archive2 (again) using -b
<sr> Archive:  CSMIS$ROOT3:[DVLP.YADA]ARCHIVE2.ZIP;1
     new version of float_demo_a.bas? [y]es, [n]o, [A]ll, [N]one, [r]ename:
<ur> A
<sr>   inflating: float_demo_a.bas        
       inflating: float_demo_b.bas        
       inflating: folding_time_100.bas    
       inflating: olivaw_100.bas          
       inflating: olivaw_101.bas          
       inflating: olivaw_102.bas          
     $
<ur> pipe dir/ful *.*;/exc=*.zip | search sys$input "record ",".bas"	! view the results
<sr> float_demo_a.bas;4            File ID:  (2271,18,0)           
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     float_demo_b.bas;4            File ID:  (2277,27,0)           
     Record format:      Variable length, maximum 255 bytes, longest 75 bytes
     Record attributes:  Carriage return carriage control
     folding_time_100.bas;4                    File ID:  (2282,17,0)           
     Record format:      Variable length, maximum 255 bytes, longest 122 bytes
     Record attributes:  Carriage return carriage control
     olivaw_100.bas;4              File ID:  (2283,25,0)           
     Record format:      Variable length, maximum 255 bytes, longest 73 bytes
     Record attributes:  Carriage return carriage control
     olivaw_101.bas;4              File ID:  (2292,39,0)           
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     olivaw_102.bas;4              File ID:  (2303,22,0)           
     Record format:      Variable length, maximum 255 bytes, longest 123 bytes
     Record attributes:  Carriage return carriage control
     $ 
  • the first unzip (no switches) of archive #2 produced the correct results in this case
     
  • the second unzip (-b) of archive #2 produced identical results in this case.
     
    QUESTION: so why use "-b"?
     
    ANSWER: if any of these files were some other format (like PCSI) then "-b" would have ensured they were restored as binary.

Advice for VMS users

  • always zip with switch "-V" (to preserve VMS file attributes)
    • ZIP "-V" archive.zip
       
  • before you unzip, you should inspect the archive with either one of these two commands
    • UNZIP "-Z" archive.zip
    • ZIPINFO archive.zip
       
  • always unzip with switch "-b-" (to recover VMS file attributes under all conditions)
    • UNZIP -b archive.zip
       
  • make sure you are running the latest version of ZIP and UNZIP

Compression of large save-sets

  • many sites like ours run BACKUP nightly to produce savesets of each disk volume. Then we zip them before writing out to tape for offsite archival (we have a 30-day rotation)
  • We are moving from an Alpha DS20e with 24 disks configured as 6-RAID volumes to an Itanium rx2800-i2 with 4 disks configured as 2-RAID volumes
  • This means the save sets are much larger on Itanium and won't zip due to the 2GB and 4GB limits of the current zip tools
  • Our only options are:
    • shift to incremental backups
    • change our backup scripts to be path specific
    • use newer versions of BPZIP, BZIP2, and GZIP which are able to compress files larger than 4GB
      note: of these three, GZIP is much faster. Download an OpenVMS copy from here: http://antinode.info/dec/sw/gzip.html

gzip examples for OpenVMS

compression example #1

	$ gzip onuadmin.sql

		1. creates onuadmin.sql-gz
		2. deletes the original file (might not be what you want)

compression example #2

	$ def/user sys$output onuadmin.sql-gz
	$ gzip -c onuadmin.sql

		1. creates onuadmin.sql-gz
		2. does not delete the original file

decompress example #1

	$ gzip -d onuadmin.sql-gz

		1. creates onuadmin.sql
		2. deletes the original file (might not be what you want)

decompress example #2

	$ def/user sys$output whatever.sql
	$ gzip -cd onuadmin.sql-gz

		1. creates whatever.sql
		2. does not delete the original file

Links


Back to Home
Neil Rieck
Waterloo, Ontario, Canada.