Files-11 is thefile system used in theRSX-11 andOpenVMSoperating systems fromDigital Equipment Corporation. It supportsrecord-orientedI/O, remotenetwork access, andfile versioning. The original ODS-1 layer is aflat file system; the ODS-2 version is ahierarchical file system, with support foraccess control lists,.
Files-11 is similar to, but significantly more advanced than, the file systems used in previousDigital Equipment Corporation operating systems such asTOPS-20 andRSTS/E.
The native OpenVMS file system is descended from older DEC operating systems and is similar in many ways, both having been designed byDave Cutler. A major difference is the layout of directories. These file systems all provided some form of rudimentary non-hierarchical directory structure, typically based on assigning one directory per user account. Under RSTS/E, each user account was represented by two numbers, a[project,programmer]
pair, and had one associated directory. Special system files, such as program executables and the OS itself, were stored in the directory of a reserved system account.
While this was suitable forPDP-11 systems, which possessed limited permanent storage capacity,VAX systems with much largerhard drives required a more flexible method of file storage: hierarchical directory layout in particular, the most notable improvement in ODS-2.
"Files-11" is the general term for five separate file systems, known as on-disk structure (ODS) levels 1 through 5.
ODS-1 is theflat file system used by the RSX-11 OS, supported by olderVMS systems for RSX compatibility, but never used to support VMS itself; it has been largely superseded by ODS-2 and ODS-5.
ODS-2 is the original VMS file system. Compared with ODS-1, it is ahierarchical file system.
Although seldom referred to by their ODS level designations,ODS-3 andODS-4 are the Files-11 support for the CD-ROMISO 9660 andHigh Sierra Format file systems, respectively.
ODS-5 is an extended version of ODS-2 available onAlpha,IA-64 andx86-64 platforms which adds support forcase-preserving filenames with non-ASCII characters and improvements to the hierarchical directory support. It was originally intended for file serving toMicrosoft Windows or other non-VMS systems as part of the "NT Affinity" project, but is also used on user disks andInternet servers.
All files and directories in a Files-11 file system are contained inside one or moreparent directories, and eventually under the root directory, themaster file directory (see below). The file system is therefore organised in a directed acyclic graph (DAG) structure.
In this example (see right),File 2 has a directory entry under bothDir 2 andDir 3; it is "in" both directories simultaneously. Even if removed from one, it would still exist in the other directory until removed from there also. This is similar to the concept ofhard links inUNIX, although care must be taken that the file is not actually deleted on disks that are not set up for hard links (only available on ODS-5 disks, and then only if the disk has hard links enabled).
An operational VMS system has access to one or more online disks, each of which contains a complete, independent file system. These are either local storage or, in the case of a cluster, storage shared with remote systems.
In an OpenVMS cluster configuration, non-private disks are shared between all nodes in the cluster(see figure 1). In this configuration, the two system disks are accessible to both nodes via the network, but the private disk is not shared: it is mounted for use only by a particular user or process on that machine. Access to files across a cluster is managed by the OpenVMS Distributed Lock Manager, an integral part of the file system.
Multiple disks can be combined to form a single large logical disk, orvolume set. Disks can also be automatically replicated intoshadow sets for data security or faster read performance.
A disk is identified by either its physical name or (more often) by a user-defined logical name. For example, the boot device (system disk) may have the physical name$3$DKA100, but it is generally referred to by thelogical nameSYS$SYSDEVICE.
File systems on each disk (with the exception of ODS-1) are hierarchical. A fully specifiedfilename consists of a nodename, a username and password, a device name, directory, filename, file type, and a version number, in the format:
NODE"accountname password"::device:[directory.subdirectory]filename.type;ver
For example,[DIR1.DIR2.DIR3]FILE.EXT refers to the latest version ofFILE.EXT, on the current default disk, in directory[DIR1.DIR2.DIR3].
DIR1 is a subdirectory of the master file directory (MFD), orroot directory, andDIR2 is a subdirectory ofDIR1. A disk's MFD is identified by[000000].
Most parts of the filename can be omitted, in which case they are taken from the currentdefault file specification. The default file specification replaces the concept of "current directory" in other operating systems by providing a set of defaults for node, device name and directory. All processes have a default file specification which includes disk name and directory, and most VMS file systemroutines accept a default file specification which can also include the file type; theTYPE command, for example, defaults to ".LIS" as the file type, so the commandTYPE F, with no extension, attempts to open the fileF.LIS.
Every file has a version number, which defaults to 1 if no other versions of the same filename are present (otherwise one higher than the greatest version). Every time a file is saved, rather than overwriting the existing version, a new file with the same name but an incremented version number is created. Old versions can be deleted explicitly, with theDELETE or thePURGE command, or optionally, older versions of a file can be deleted automatically when the file'sversion limit is reached (set bySET FILE/VERSION_LIMIT). Old versions are thus not overwritten, but are kept on disk and may be retrieved at any time. The architectural limit on version numbers is 32767. The versioning behavior is easily overridden if it is unwanted. In particular, files which are directly updated, such as databases, do not create new versions unless explicitly programmed.
ODS-2 is limited to eight levels of subdirectories, and only uppercase, alphanumeric names (plus the underscore, dash, and dollar sign) up to 39.39 characters (39 for the filename and another 39 for the extension). ODS-5 expands the character set to lowercase letters and most other printable ASCII characters, as well asISOLatin-1 andUnicode characters, increases the maximum filename length and allows unlimited levels of subdirectories. When constructing a pathname for an ODS-5 file which uses characters not allowed under ODS-2, a special "^" syntax is used to preserve backwards compatibility; the file "file.tar.gz;1" on an ODS-5 disk, for example, would be referred to as "file^.tar.gz"—the file's name is "file.tar", and the extension is ".gz".
VMS file security is defined by two mechanisms, UIC-based access control andACL-based access control. UIC access control is based on the owner of the file and the UIC, or user, accessing the file. Access is determined by four groups of permissions:
And four permission bits:
The "system" access applies to any user whose UIC group code is less than or equal to theSYSGEN parameterMAXSYSGROUP (typically 8, or 10octal) (for example theSYSTEM user); "owner" and "group" apply to the owner of the file and that user's user group, and "world" applies to any other user. There is also a fifth permission bit, "Control", which is used to determine access to change file metadata such as protection. This group cannot be set explicitly; it is always set for System and Owner, and never for Group or World.
UIC-based access control is also affected by four systemprivileges, which allow users holding them to override access controls:
ACLs allow additional privileges to be assigned on a user– or group–specific basis; for example, a web server's UIC could be granted read access to all files in a particular directory. ACLs can be marked asinherited, where a directory file's ACL applies to all files underneath it. ACLs are modified using theEDIT/ACL command, and take the form of identifier/access pairs. For example, the ACL entry
(IDENTIFIER=HTTP$SERVER,ACCESS=READ+EXECUTE)
would allow the userHTTP$SERVER to read and execute the file.
Alogical name is a system variable which may reference a disk, directory or file, or contain other program-specific information. For example, the logicalSYS$SYSDEVICE contains the system's boot device. A logical name normally refers to a single directory or disk,e.g.SYS$LOGIN: which is the user's login (home) directory (or directories); these logicals cannot be used as true disk names—SYS$LOGIN:[DIR]FILE is not a valid file specification. However,concealed logical names, defined byDEFINE/TRANSLATION=CONCEALED, can be used in that way; theserooted directories are defined with a trailing "." on the directory specification, hence
$ DEFINE/TRANS=CONCEAL HOME DISK$USERS:[username.]
would allowHOME:[DIR]FILE to be used. More common are simple logicals which point to specific directories associated with some application software which may be located in on any disk or any directory. Hence logical ABC_EXE may point to a directory of executable programs for application ABC and ABC_TEMP may point to a directory of temporary files for that same application and this directory may be on the same disk and in the same directory tree as ABC_EXE or could be somewhere on another disk (and in a different directory tree).
In a manner similar to Unix, VMS defines severalstandard input and output channels which are accessed through the logical namesSYS$INPUT,SYS$OUTPUT,SYS$ERROR andSYS$COMMAND.[1]
Logical names do not have a close equivalent in POSIX operating systems. They resemble Unixenvironment variables, except they are expanded by the file system, instead of the command shell or application program. They must be defined before use, so it is common for many logical names to be defined in the system startup command file, as well as user login command files.In VMS, logical names may reference other logical names (up to a predefined nesting limit of 10), and may contain lists of names to search for an existing filename. Some frequently referenced logical names are:
logical name | meaning |
---|---|
SYS$INPUT | standard input - used interactively, this represents the terminal keyboard. Used in a batch file, it reads batch file lines not preceded with a $ symbol, or specified as an input deck using theDECK command. |
SYS$OUTPUT | standard output - it will output to the terminal display or the batch log file depending on whether the process is interactive or not. |
SYS$ERROR | standard error - it will output to the terminal display or the batch error log file depending on whether the process is interactive or not. |
SYS$COMMAND | source of batch file commands. It will read from the terminal or the SYS$INPUT stream depending on whether the process is interactive or not. |
TT | the terminal associated with the process |
SYS$PRINT | the default printer or print queue |
SYS$LOGIN | home directory for each user |
SYS$SCRATCH | temporary folder, directory for temporary files |
SYS$SYSTEM | directory containing most system programs and a few vital data files, such as the system authorization file (accounts and passwords) |
SYS$SHARE | shared runtime libraries, executables, etc. |
SYS$LIBRARY | system and added libraries |
The closest non-DEC operating system to support the concept of logical names isAmigaOS, through theASSIGN command. AmigaOS's disk operating system,AmigaDOS, which is a port ofTRIPOS, bears some resemblance to DEC operating systems. For example, physical device names follow a pattern like DF0: for the first floppy disk, CDROM2: for the 3rd CD-ROM drive, etc. However, since the system can boot from any attached drive, the operating system creates the SYS: assignment to automatically reference the boot device used. Other assignments, LIBS:, PREFS:, C:, S:, et al. are also made, themselves referenced off SYS:. Users are, of course, allowed to create and destroy their own assignments too.
Record Management Services is the structuredI/O layer of the VMS operating system. RMS provides comprehensive program support for managingstructuredfiles, such as record-based and indexeddatabase files. The VMS file system, in conjunction with RMS, extends files access past simplebyte-streams and allows OS-level support for a variety of rich files types. Each file in the VMS file system may be thought of as adatabase, containing a series ofrecords, each of which has one of more individualfields. A text file, for example, is a list of records (lines) separated by a newline character. RMS is an example of arecord-oriented filesystem.
There are fourrecord formats defined by RMS:
There are fourrecord access methods, or methods to retrieve extant records from files:
At the disk level, ODS represents the file system as an array ofblocks, a block being 512 contiguous bytes on one physical disk (volume). Disk blocks are assigned inclusters (originally 3 contiguous blocks but later increased with larger disk sizes). A file on the disk will ideally be entirely contiguous, i.e. the blocks which contain the file will be sequential, but disk fragmentation will sometimes require the file to be located in discontiguous clusters in which case the fragments are calledextents. Disks may be combined with other disks to form avolume set and files stored anywhere across that set of disks, but larger disk sizes have reduced the use of volume sets because management of a single physical disk is simpler.
Every file on a Files-11 disk (or volume set) has a uniquefile identification (FID), composed of three numbers: thefile number (NUM), thefile sequence number (SEQ), and therelative volume number (RVN). The NUM indicates where in theINDEXF.SYS file (see below) the metadata for the file is located; the SEQ is a generation number which incremented when the file is deleted and another file is created reusing the same INDEXF.SYS entry (so any dangling references to the old file do not accidentally point to the new one); and the RVN indicates the volume number on which the file is stored when using a volume set.
The structural support of an ODS volume is provided by adirectory file—a special file containing a list of file names, file version numbers and their associated FIDs, similar to VSAM catalogs onMVS and directories onUnix file systems andNTFS. At the root of the directory structure is themaster file directory (MFD), the root directory which contains (directly or indirectly) every file on the volume.
This diagram shows an example directory containing 3 files, and the way each filename is mapped to theINDEXF.SYS entry (each INDEXF entry contains more information; only the first few items are shown here).
At the top level of an ODS file system is themaster file directory (MFD), which contains all top-level directory files (including itself), and several system files used to store file system information. On ODS-1 volumes, a two-level directory structure is used: eachuser identification code (UIC) has an associateduser file directory (UFD), of the form[GROUP.USER]. On ODS-2 and later volumes, the layout of directories under the MFD is free-form, subject to a limit on the nesting of directories (8 levels on ODS-2 and unlimited on ODS-5). On multi-volume sets, the MFD is always stored on the first volume, and contains the subdirectories of all volumes.
The following system files are present in the ODS MFD:
Note that the file system implementation itself does not refer to these files by name, but by their file IDs, which always have the same values. Thus, INDEXF.SYS is always the file with NUM = 1 and SEQ = 1.
The index file contains the most basic information about a Files-11 volume set.
There are two organizations of INDEXF.SYS, the traditional organization and the organization used on disks with GPT.SYS; with the GUID Partition Table (GPT) structures.
With the traditional organization, block 1 is theboot block, which contains the location of theprimary bootstrap image, used to load the VMS operating system. This is always located at logical block 0 on the disk, so that the hardware firmware can read it. This block is always present, even on non-system (non-bootable) volumes.
After the boot block is theprimary home block. This contains thevolume name, the location of the extents comprising the remainder of the index file, the volume owner's UIC, and thevolume protection information. There are normally several additional copies of the home block, known as thesecondary home blocks, to allow recovery of the volume if it is lost or damaged.
On disks with GPT.SYS, GPT.SYS contains the equivalent of the boot block (known as the Master Boot Record (MBR)), and there is no primary home block. All home blocks present on a GPT-based disk are alternate home blocks. These structures are not included in INDEXF.SYS, and the blocks of the INDEXF.SYS file are unused.
The rest of the index file is composed offile headers, which describe the extents allocated to the files residing on the volume, and file metadata such as the owner UIC, ACLs and protection information. Each file is described by one or more file headers—more than one can be required when a file has a large number of extents. The file header is a fixed-length block, but contains both fixed– and variable–length sections:
If possible, the map and ACL sections of the header are contained completely in theprimary header. However, if the ACL is too long, or the file contains too many extents, there will not be enough space in the primary header to store them. In this case, anextension header is allocated to store the overflow information.
The file header begins with 4 offsets (IDOFFSET,MPOFFSET,ACOFFSET andROFFSET). Since the size of the areas after the fixed-length header may vary (such as the map and ACL areas), the offsets are required to locate these additional areas. Each offset is the number of 16-bit words from the beginning of the file header to the beginning of that area.
If the file requires multiple headers, theextension segment number (SEGNUM) contains the sequence number of this header, beginning at 0 in the first entry in INDEXF.SYS.
STRUCLEV contains the current structure level (in the high byte) and version (in the low byte) of the file system; ODS-2 being structure level 2. An increase in the version number indicates a backwards-compatible change that older software may ignore; changes in the structure level itself are incompatible.
W_FID (containing three values:FID_NUM,FID_SEQ andFID_RVN, corresponding to the file, sequence, and relative volume number) contains the ID of this file;EXT_FID (again composed of three values) holds the location of the next extension header, if any. In both of these values, the RVN is specified as 0 to represent the "current" volume (0 is not normally a valid RVN).
FILECHAR contains several flags which affect how the file is handled or organised:
ACCMODE describes theprivilege level at which a process must be running in order to access the file. VMS defines four privilege levels: user, supervisor, exec, and kernel. Each type of access - read, write, execute and delete - is encoded as a 2-bit integer.
FILEPROT contains thediscretionary access control information for the file. It is divided into 4 groups of 4 bits each: system, owner, group and world. Bit 0 corresponds to read access, 1 to write, 2 to execute and 3 to delete. Setting a bit denies a particular access to a group; clearing it allows it.
If the file header is an extension header,BACKLINK contains the file ID of the primary header; otherwise, it contains the file ID of the directory file containing the primary entry for the file.