Movatterモバイル変換

RFC 683, NIC 32251FTPSRV - TENEX FTP EXTENSIONS FOR PAGED FILES                        R. Clements - BBN - 3 April 751   Introduction    In response to a long-known need for the ability to transfer TENEX pagedfiles over the net via FTP, the TENEX FTP implementation has been extended.    This implementation is an extension to the "OLD" protocol (RFC 354).  Itwas built after useful discussions with Postel, Neigus, et al.  I do not meanto imply that they agreed that this implementation is correct, nor for thatmatter do I feel it is correct.  A "correct" implementation will be negotiatedand implemented in the "NEW" protocol (RFC 542), if funding ever appears forthat task.2   The Problem(s)    This extension attacks two separate problems: Network reliability andTENEX disk file format's incompatibility with FTP.  A checksummed andblock-sequence-numbered transmission mode is seriously needed, in my opinion.This mode should also allow data compression.    It is also necessary to handle paged, holey TENEX files.  This latterproblem, seriously needed for NLS, is the motivation for the currentextension.    The former problem requires a new MODE command, if done correctly;probably two MODEs, to allow data compression in addition to checksumming.Actually, I think that is the tip of an iceberg which grows as 2**N foradditional sorts of modes, so maybe some mode combination system needs to bedreamed up.  Cf the AN, AT, AC, EN, ET, EC TYPEs.  Also, one should be able touse MODE B and MODE C together (NEW protocol) to gain both the compression andrestart facilities if one wanted.    The second problem, TENEX files, are probably a new kind of STRUcture.However, it should be possible to send a paper tape to a disk file, or viceversa, with the transfer looking like a paged file; so perhaps we are dealingwith a data representation TYPE.  This argument is a bit strained, though, soa paged STRUcture is quite likely correct.  I admit to feeling very unsureabout what is a MODE, what is a TYPE and what is a STRUcture.3   The (Incorrect) choices made    Having decided that new MODEs and STRUctures were needed, I insteadimplemented the whole thing as a single new TYPE.  After all, I rationalize,checksumming the data on the network (MODE) and representing the data in theprocessing system as a checksummed TYPE are really just a matter of where youdraw the imaginary line between the net and the data.  Also, a single new TYPEcommand reduced the size of the surgery required on the FTP user and serverprograms.4   Implementation details    The name of the new TYPE is "XTP".  I propose this as a standard for allthe Key Letter class of FTP commands: the "X" stands for "experimental" --agreed on between cooperating sites.  The letter after the "X" is signed outfrom the protocol deity by an implementor for a given system.  In this case,"T" is for TENEX.  Subsequent letter(s) distinguish among possibly multipleprivate values of the FTP command.  Here "P" is "Paged" type.   TYPE XTP is only implemented for STRU F, BYTE 36, and MODE S.    Information of TYPE XTP is transfered in chunks (I intentionally avoid thewords RECORD and BLOCK) which consist of a header and some data.  The data ina chunk may be part of the data portion of the file being transferred, or itmay be the FDB (File Descriptor Block) associated with the file.5   Diversion: the TENEX Disk File    For those not familiar with the TENEX file system, a brief dissertation isincluded here to make the rest of the implementation meaningful.    A TENEX disk file consists of four things: a pathname, a page table, a(possibly empty) set of pages, and a set of attributes.    The pathname is specified in the RETR or STOR verb.  It includes thedirectory name, file name, file name extension, and version number.    The page table contains up to 2**18 entries.  Each entry may be EMPTY, ormay point to a page.  If it is not empty, there are also some page-specificaccess bits; not all pages of a file need have the same access protection.    A page is a contiguous set of 512 words of 36 bits each.    The attributes of the file, in the FDB, contain such things as creationtime, write time, read time, writer's byte-size, end of file pointer, count ofreads and writes, backup system tape numbers, etc.    NOTE: there is NO requirement that pages in the page table be contiguous.There may be empty page table slots between occupied ones.  Also, the end offile pointer is simply a number.  There is no requirement that it in factpoint at the "last" datum in the file. Ordinary sequential I/O calls in TENEXwill cause the end of file pointer to be left after the last datum written,but other operations may cause it not to be so, if a particular programmingsystem so requires.    In fact both of these special cases, "holey" files and end-of-filepointers not at the end of the file, occur with NLS data files.  These fileswere the motivation for the new TYPE.6   Meanwhile, back at the implementation,...    Each chunk of information has a header.  The first byte, which is thefirst word (since TYPE XTP is only implemented for BYTE 36) of the chunk, is asmall number, currently 6, which is the number of following words which arestill in the header.  Next come those six words, and then come some datawords.   The six header words are:      Word 1: a checksum.         This is a one's complement sum (magnitude and end-around carry) of         the six header words and the following data words (but not the         leading "6" itself).  The sum of all words including the checksum         must come out + or - zero.      Word 2: A sequence number.         The first chunk is number 1, the second is number 2, etc.      Word 3: NDW,         the number of data words in this chunk, following the header.  Thus         the total length of the chunk is 1 (the word containing NHEAD) +         NHEAD +NDW.  The checksum checks all but the first of these.      Word 4: Page number.         If the data is a disk file page, this is the number of that page in         the file's page map.  Empty pages (holes) in the file are simply         not sent.  Note that a hole is NOT the same as a page of zeroes.      Word 5: ACCESS.         The access bits associated with the page in the file's page map.         (This full word quantity is put into AC2 of an SPACS by the program         reading from net to disk.)      Word 6: TYPE.         A code for what type of chunk this is. Currently, only type zero         for a data page, and type -3 for an FDB are sent.    After the header are NDW data words.  NDW is currently either 1000 octalfor a data page or 25 octal for an FDB.  Trailing zeroes in a disk file pagewill soon be discarded, making NDW less than 1000 in that case.  The receivingportions of FTP server and user will accept these shortened pages.  The senderdoesn't happen to send them that way yet.Verification is performed such that an error is reported if either:   The checksum fails,   The sequence number is not correct,   NDW is unreasonable for the given chunk type, or   The network file ends at some point other than immediately following thedata portion of an FDB chunk.7   Closing comments    This FTP server and user are in operation on all the BBN systems and atsome other sites -- the user being more widely distributed since fewer siteshave made local modifications to the user process.    I believe the issues of checksumming and sequencing should be addressedfor the "NEW" protocol.  I hope the dissertation on TENEX files has beenuseful to users of other systems.  It may explain my lack of comprehension ofthe "record" concept, for example.  A TENEX file is just a bunch of wordspointed to by a page table.  If those words contain CRLF's, fine -- but thatdoesn't mean "record" to TENEX. I think this RFC also points out clearly thatnet data transfers are implemented like the layers of an onion: somecharacters are packaged into a line.  Some lines are packaged into a file.The file is broken into other managable units for transmission.  Those unitshave compression applied to them.  The units may be flagged by restart markers(has anyone actually done that?).  The compressed units may be checksummed,sequence numbered, date-and-time stamped, and flagged special delivery.  Onthe other end, the process is reversed.  Perhaps MODE, TYPE, and STRU don'treally adequately describe the situation. This RFC was written to allowimplementors to interface with the new FTP server at TENEX sites which installit.  It is also really a request for comments on some of these other issues.