1014 lines
36 KiB
Plaintext
1014 lines
36 KiB
Plaintext
|
|
|
|
|
|
|
|
|
|
XMODEM/YMODEM PROTOCOL REFERENCE
|
|
|
|
A compendium of documents describing the
|
|
|
|
XMODEM and YMODEM
|
|
|
|
File Transfer Protocols
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Edited by Chuck Forsberg
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Please distribute as widely as possible.
|
|
|
|
Questions to Chuck Forsberg
|
|
|
|
|
|
|
|
|
|
|
|
Omen Technology Inc
|
|
17505-V Sauvie Island Road
|
|
Portland Oregon 97231
|
|
Voice : 503-621-3406
|
|
Modem (Telegodzilla): 503-621-3746 Speed 1200,300
|
|
Compuserve : 70715,131
|
|
UUCP : ...!tektronix!reed!omen!caf
|
|
|
|
|
|
CONTENTS
|
|
|
|
1. TOWER OF BABEL ??............................................ 1
|
|
1.1 Some Messages from the Pioneer.......................... 1
|
|
|
|
2. XMODEM PROTOCOL ENHANCEMENTS................................. 3
|
|
2.1 CAN-CAN Abort........................................... 3
|
|
2.2 CRC-16 Option........................................... 4
|
|
2.3 1024 Byte Packet Option................................. 4
|
|
|
|
3. YMODEM Batch File Transmission............................... 5
|
|
|
|
4. XMODEM PROTOCOL OVERVIEW..................................... 10
|
|
4.1 Definitions............................................. 10
|
|
4.2 Transmission Medium Level Protocol...................... 10
|
|
4.3 File Level Protocol..................................... 11
|
|
4.4 Programming Tips........................................ 13
|
|
|
|
5. XMODEM/CRC Overview.......................................... 13
|
|
5.1 CRC Calculation......................................... 14
|
|
5.2 CRC File Level Protocol Changes......................... 16
|
|
5.3 Data Flow Examples with CRC Option...................... 17
|
|
|
|
|
|
LIST OF FIGURES
|
|
|
|
Figure 1. 1024 byte Packets.................................... 5
|
|
|
|
Figure 2. Mixed 1024 and 128 byte Packets...................... 5
|
|
|
|
Figure 3. Batch Transmission Session........................... 9
|
|
|
|
Figure 4. Filename packet transmitted by sb.................... 9
|
|
|
|
Figure 5. XMODEM Message Block Level Protocol.................. 11
|
|
|
|
Figure 6. Data flow including Error Recovery................... 12
|
|
|
|
Figure 7. Message Block Level Protocol, CRC mode............... 14
|
|
|
|
Figure 8. Example of CRC Calculation written in C.............. 15
|
|
|
|
Figure 9. Data Flow: Receiver has CRC Option, Sender Doesn't... 17
|
|
|
|
Figure 10. Receiver and Sender Both have CRC Option............. 19
|
|
|
|
|
|
Chapter 1
|
|
|
|
TOWER OF BABEL ??
|
|
|
|
|
|
In the interest of fostering compatibility among communications
|
|
programs, part of of the Professional-YAM manual is reproduced here to
|
|
minimize the Electronic Tower of Babel.
|
|
|
|
The YMODEM Protocol is supported by the public domain programs YAM
|
|
(CP/M), YAM(CP/M-86), YAM(CCPM-86), rb/sb (Unix, Berkeley Unix, Venix,
|
|
Xenix, Coherent, IDRIS, Regulus) as well as Professional-YAM. These
|
|
programs have been in use since 1981.
|
|
|
|
The protocols described below are enhancements to Ward Christensen's
|
|
XMODEM protocol, and are public domain.
|
|
|
|
The 1k packet length capability described below may be used in
|
|
conjunction with the Batch Protocol, or with single file transfers
|
|
identical to the XMODEM/CRC protocol except for minimal changes to
|
|
support 1k packets.
|
|
|
|
To complete this tome, Ward Christensen's original protocol document
|
|
and John Byrns's CRC-16 document are included for reference.
|
|
|
|
References to the MODEM or MODEM7 protocol have been changed to XMODEM
|
|
to accommodate the vernacular.
|
|
|
|
Watch for an article describing the YMODEM protocol in a more coherent
|
|
fashion later this year. This article will include some interesting
|
|
history on the development of file microcomputer file transfers.
|
|
|
|
|
|
1.1 Some Messages from the Pioneer
|
|
|
|
#: 130940 S0/Communications 25-Apr-85 18:38:47
|
|
Sb: my protocol
|
|
Fm: Ward Christensen 76703,302 (EDITED)
|
|
To: all
|
|
|
|
|
|
Be aware the article[1] DID quote me correctly in terms of the phrases
|
|
like "not robust", etc.
|
|
|
|
It was a quick hack I threw together, very unplanned (like everything
|
|
I do), to satisfy a personal need to communicate with "some other"
|
|
people.
|
|
|
|
ONLY the fact that it was done in 8/77, and that I put it in the
|
|
public domain immediately, made it become the standard that it is.
|
|
-------
|
|
1. Infoworld April 29 p. 16
|
|
|
|
|
|
Chapter 1 (cont)
|
|
|
|
|
|
I think its time for me to
|
|
|
|
1. document it; (people call me and say "my product is going to
|
|
include it - what can I 'reference'", or "I'm writing a paper
|
|
on it, what do I put in the bibliography") and
|
|
|
|
2. propose an "incremental extension" to it, which might take
|
|
"exactly" the form of Chuck Forsberg's YAM protocol. He
|
|
wrote YAM in C for CP/M and put it in the public domain, and
|
|
wrote a batch protocol for Unix called rb and sb (receive
|
|
batch, send batch), which was basically XMODEM with
|
|
|
|
a. a record 0 containing filename date time and size
|
|
b. a 1K block size option
|
|
c. CRC-16.
|
|
|
|
He did some clever programming to detect false ACK or EOT, but
|
|
basically left them the same.
|
|
|
|
People who suggest I make SIGNIFICANT changes to the protocol, such as
|
|
"full duplex", "multiple outstanding blocks", "multiple destinations",
|
|
etc etc don't understand that the incredible simplicity of the
|
|
protocol is one of the reasons it survived to this day in as many
|
|
machines and programs as it may be found in!
|
|
|
|
Consider the PC-NET group back in '77 or so - documenting to beat the
|
|
band - THEY had a protocol, but it was "extremely complex", because it
|
|
tried to be "all things to all people" - i.e. send binary files on a
|
|
7-bit system, etc. I was not that "benevolent". I (emphasize > I < )
|
|
had an 8-bit UART, so "my protocol was an 8-bit protocol", and I would
|
|
just say "sorry" to people who were held back by 7-bit limitations.
|
|
...
|
|
Block size: Chuck Forsberg created an extension of my protocol, called
|
|
YAM, which is also supported via his public domain programs for UNIX
|
|
called rb and sb - receive batch and send batch. They cleverly send a
|
|
"block 0" which contains the filename, date, time, and size.
|
|
Unfortunately, its UNIX style, and is a bit weird[1] - octal numbers,
|
|
etc. BUT, it is a nice way to overcome the kludgy "echo the chars of
|
|
the name" introduced with MODEM7. Further, chuck uses CRC-16 and
|
|
optional 1K blocks. Thus the record 0, 1K, and CRC, make it a "pretty
|
|
slick new protocol" which is not significantly different from my own.
|
|
-------
|
|
1. The Unix style stuff (time, file mode) is optional. The pathname
|
|
and file length may be sent alone if desired.
|
|
|
|
|
|
Chapter 1 (cont)
|
|
|
|
|
|
Also, there is a catchy name - YMODEM. That means to some that it is
|
|
the "next thing after XMODEM", and to others that it is the Y(am)MODEM
|
|
protocol. I don't want to emphasize that too much - out of fear that
|
|
other mfgrs might think it is a "competitive" protocol, rather than an
|
|
"unaffiliated" protocol. Chuck is currently selling a much-enhanced
|
|
version of his CP/M-80 C program YAM, calling it Professional Yam, and
|
|
its for the PC - I'm using it right now. VERY slick! 32K capture
|
|
buffer, script, scrolling, previously captured text search, plus
|
|
built-in commands for just about everything - directory (sorted every
|
|
which way), XMODEM, YMODEM, KERMIT, and ASCII file upload/download,
|
|
etc. You can program it to "behave" with most any system - for
|
|
example when trying a number for CIS it detects the "busy" string back
|
|
from the modem and substitutes a diff phone # into the dialing string
|
|
and branches back to try it.
|
|
|
|
|
|
Chapter 2
|
|
|
|
XMODEM Protocol Enhancements
|
|
|
|
|
|
Professional-YAM uses several compatible extensions and logic
|
|
enhancements to the widely used Ward Christensen XMODEM protocol.
|
|
|
|
This chapter discusses the protocol extensions to Ward Christensen's
|
|
1982 XMODEM protocol description document.
|
|
|
|
YAM does not ask the operator whether he wishes to keep retrying after
|
|
10 attempts have failed to correctly transfer a packet. Virtually all
|
|
correctable errors are corrected within the first few retransmissions.
|
|
If the line is so bad that ten attempts are insufficient, there is a
|
|
significant danger of undetected errors. In that case, it's better to
|
|
redial for a better connection.
|
|
|
|
|
|
2.1 CAN-CAN Abort
|
|
|
|
YAM recognizes a sequence of two consecutive CAN (Hex 18) characters
|
|
without modem errors (overrun, framing, etc.) as a transfer abort
|
|
command[1]. The check for two consecutive CAN characters virtually
|
|
eliminates the possibility of a line hit aborting the transfer. YAM
|
|
sends five CAN characters when it aborts a XMODEM protocol file
|
|
transfer, followed by five backspaces to delete the CAN characters
|
|
from the remote's keyboard input buffer (in case the remote had
|
|
already aborted the transfer).
|
|
-------
|
|
1. This is recognized when YAM is waiting for the beginning of a
|
|
packet or for an acknowledge to one that has been sent.
|
|
|
|
|
|
Chapter 2 (cont)
|
|
|
|
|
|
2.2 CRC-16 Option
|
|
|
|
The XMODEM protocol uses an optional two character CRC-16 instead of
|
|
the one character arithmetic checksum used by the original protocol
|
|
and by most commercial implementations. CRC-16 guarantees detection
|
|
of all single and double bit errors, all errors with an odd number of
|
|
error bits, all burst errors of length 16 or less, 99.9969% of all
|
|
17-bit error bursts, and 99.9984 per cent of all possible longer error
|
|
bursts. By contrast, a double bit error, or a burst error of 9 bits
|
|
or more can sneak past the XMODEM protocol arithmetic checksum.
|
|
|
|
The XMODEM/CRC protocol is similar to the XMODEM protocol, except that
|
|
the receiver specifies CRC-16 by sending C (Hex 43) instead of NAK
|
|
when requesting the FIRST packet. A two byte CRC is sent in place of
|
|
the one byte arithmetic checksum. YAM's c option to the r command
|
|
enables CRC-16 in single file reception, corresponding to the original
|
|
implementation in the MODEM7 series programs. Many commercial
|
|
communications programs and bulletin board systems still do not
|
|
support CRC-16, especially those written in Basic or Pascal.
|
|
|
|
XMODEM protocol with CRC is accurate provided both sender and receiver
|
|
both report a successful transmission. The protocol is robust in the
|
|
presence of characters lost by buffer overloading on timesharing
|
|
systems.
|
|
|
|
Professional-YAM add several proprietary logic enhancements to
|
|
XMODEM's error detection and recovery. These compatible enhancements
|
|
eliminate most of the bad file transfers other programs make when
|
|
using the XMODEM protocol under less than ideal conditions.
|
|
|
|
|
|
2.3 1024 Byte Packet Option
|
|
|
|
If YAM is sending with the k option, the transmitted packet contains
|
|
1024 bytes of data. An STX (02) replaces the SOH (01) at the
|
|
beginning of the transmitted block to notify the receiver of the
|
|
longer packet length. The receiver should be able to accept any
|
|
mixture of 128 and 1024 byte packets. The packet number is
|
|
incremented by one for each packet[1].
|
|
-------
|
|
1. The displayed sector number should be the number of bytes divided
|
|
by 128. This maintains a familiar unit of measurement for the
|
|
progress report and avoids ambuguity caused by rollowver of the
|
|
sector number.
|
|
|
|
|
|
Chapter 2 (cont)
|
|
|
|
|
|
If 1024 byte packets are being used, it is possible for a 128 byte
|
|
file to "grow" to 1024 bytes on CP/M. This does not waste disk space
|
|
on CP/M because the allocation granularity is 1k. When 1024 byte
|
|
packets are used with batch transmission, the file length transmitted
|
|
in the file name packet allows the receiver to discard the padding.
|
|
|
|
CRC-16 should be used with the k option to preserve data integrity
|
|
over phone lines. The k option may be used with batch file
|
|
transmission, or with single file transmission.
|
|
|
|
|
|
Chapter 3
|
|
|
|
XMODEM Protocol Enhancements
|
|
|
|
|
|
Figure 1. 1024 byte Packets
|
|
|
|
SENDER RECEIVER
|
|
"s -k foo.bar"
|
|
"foo.bar open x.x minutes"
|
|
C
|
|
STX 01 FE Data[1024] CRC CRC
|
|
ACK
|
|
STX 02 FD Data[1024] CRC CRC
|
|
ACK
|
|
STX 03 FC Data[1000] CPMEOF[24] CRC CRC
|
|
ACK
|
|
EOT
|
|
ACK
|
|
|
|
Figure 2. Mixed 1024 and 128 byte Packets
|
|
|
|
SENDER RECEIVER
|
|
"s -k foo.bar"
|
|
"foo.bar open x.x minutes"
|
|
C
|
|
STX 01 FE Data[1024] CRC CRC
|
|
ACK
|
|
STX 02 FD Data[1024] CRC CRC
|
|
ACK
|
|
SOH 03 FC Data[128] CRC CRC
|
|
ACK
|
|
SOH 04 FB Data[100] CPMEOF[28] CRC CRC
|
|
ACK
|
|
EOT
|
|
ACK
|
|
|
|
|
|
Chapter 3 (cont)
|
|
|
|
|
|
YMODEM Batch File Transmission
|
|
|
|
The YMODEM Batch protocol is an extension to the XMODEM/CRC protocol
|
|
that allows 0 or more files to be transmitted with a single command.
|
|
(Zero files may be sent if none of the requested files is accessible.)
|
|
The design approach of the YAM Batch protocol is to use the normal
|
|
routines for sending and receiving XMODEM packets in a layered fashion
|
|
similar to packet switching methods.
|
|
|
|
Why was it necessary to design a new batch protocol when one already
|
|
existed in MODEM7? The batch file mode used by MODEM7 is unsuitable
|
|
because it does not permit full pathnames, file length, file date, or
|
|
any other attribute information to be transmitted. Such a restrictive
|
|
design, hastily implemented with only CP/M in mind, would not have
|
|
permitted extensions to current areas of personal computing such as
|
|
Unix, DOS, and object oriented systems.
|
|
|
|
As in the case of single a file transfer, the receiver initiates batch
|
|
file transmission by sending a "C" character (for CRC-16).
|
|
|
|
The sender opens the first file and sends packet number 0 with the
|
|
following information[1].
|
|
|
|
Only the pathname (file name) part is required for batch transfers.
|
|
|
|
To maintain upwards compatibility, all unused bytes in packet 0 must
|
|
be set to null.
|
|
|
|
Pathname The pathname (conventionally, the file name) is sent as a
|
|
null terminated ASCII string. This is the filename format used by the
|
|
handle oriented MSDOS(TM) functions and C library fopen functions. An
|
|
assembly language example follows:
|
|
|
|
DB 'foo.bar',0
|
|
|
|
No spaces are included in the pathname. Normally only the file name
|
|
stem (no directory prefix) is transmitted unless the sender has
|
|
selected YAM's f option to send the full pathname. The source drive
|
|
(A:, B:, etc.) is not sent.
|
|
-------
|
|
1. Only the data part of the packet is described here.
|
|
|
|
|
|
Chapter 3 (cont)
|
|
|
|
|
|
Filename Considerations:
|
|
|
|
+ File names should be translated to lower case unless the
|
|
sending system supports upper/lower case file names. This is
|
|
a convenience for users of systems (such as Unix) which store
|
|
filenames in upper and lower case.
|
|
|
|
+ The receiver should accommodate file names in lower and upper
|
|
case.
|
|
|
|
+ The rb(1) program on Unix systems normally translates the
|
|
filename to lower case unless one or more letters in the
|
|
filename are already in lower case.
|
|
|
|
+ When transmitting files between different operating systems,
|
|
file names must be acceptable to both the sender and receiving
|
|
operating systems. If directories are included, they are
|
|
delimited by /; i.e., "subdir/foo" is acceptable, "subdir\foo"
|
|
is not.
|
|
|
|
Length The file length and each of the succeeding fields are
|
|
optional[1]. The length field is stored in the packet as a decimal
|
|
string. The file length does not include any CPMEOF (^Z) characters
|
|
used to pad the last packet.
|
|
|
|
If the file being transmitted is growing during transmission, the
|
|
length field should be set to at least the final expected file length,
|
|
or not sent.
|
|
|
|
The receiver stores the specified number of characters, discarding any
|
|
padding added by the sender to fill up the last packet.
|
|
|
|
Mod Date A single space separates the modification date from the file
|
|
length.
|
|
|
|
The mod date is optional, and the filename and length may be sent
|
|
without requiring the mod date to be sent.
|
|
|
|
The mod date is sent as an octal number giving the time the contents
|
|
of the file were last changed measured in seconds from Jan 1 1970
|
|
Universal Coordinated Time (GMT). A date of 0 implies the
|
|
modification date is unknown and should be left as the date the file
|
|
is received.
|
|
|
|
This standard format was chosen to eliminate ambiguities arising from
|
|
transfers between different time zones.
|
|
-------
|
|
1. Fields may not be skipped.
|
|
|
|
|
|
Chapter 3 (cont)
|
|
|
|
|
|
Two Microsoft blunders complicate the use of modification dates in
|
|
file transfers with MSDOS(TM) systems. The first is the lack of
|
|
timezone standardization in MS-DOS. A file's creation time can not be
|
|
known unless the timezone of the system that happened to write the
|
|
file[1] is known. Unix solved this problem (for planet Earth, anyway)
|
|
by stamping files with Universal Time (GMT). Microsoft would have to
|
|
include the timezone of origin in the directory entries, but does not.
|
|
YAM gets around this problem by using the z parameter which is set to
|
|
the number of minutes local time lags GMT. For files known to
|
|
originate from a different timezone, the -zT option may be used use T
|
|
as the timezone for an individual file transfer.
|
|
|
|
The second problem is the lack of a separate file creation date in
|
|
DOS. Since some backup schemes used with DOS rely on the file
|
|
creation date to select files to be copied to the archive, backdating
|
|
the file modification date could interfere with the safety of the
|
|
transferred files. For this reason, Professional-YAM does not modify
|
|
the date of received files with the header information unless the d
|
|
parameter is non zero.
|
|
|
|
|
|
Mode A single space separates the file mode from the modification
|
|
date. The file mode is stored as an octal string. Unless the file
|
|
originated from a Unix system, the file mode is set to 0. rb(1)
|
|
checks the file mode for the 0x8000 bit which indicates a Unix type
|
|
regular file. Files with the 0x8000 bit set are assumed to have been
|
|
sent from another Unix (or similar) system which uses the same file
|
|
conventions. Such files are not translated in any way.
|
|
|
|
Serial Number A single space separates the serial number from the file
|
|
mode. The serial number of the transmitting program is stored as an
|
|
octal string. Programs which do not have a serial number should omit
|
|
this field, or set it to 0. The receiver's use of this field is
|
|
optional.
|
|
|
|
The rest of the packet is set to nulls. This is essential to preserve
|
|
upward compatibility[2]. After the filename packet has been received,
|
|
it is ACK'ed if the write open is successful. The receiver then
|
|
initiates transfer of the file contents according to the standard
|
|
XMODEM/CRC protocol. If the file cannot be opened for writing, the
|
|
receiver cancels the transfer with CAN characters as described above.
|
|
-------
|
|
1. Not necessarily that of the transmitting system!
|
|
2. If, perchance, this information extends beyond 128 bytes (possible
|
|
with Unix 4.2 BSD extended file names), the packet should be sent
|
|
as a 1k packet as described above.
|
|
|
|
|
|
Chapter 3 (cont)
|
|
|
|
|
|
After the file contents have been transmitted, the receiver again asks
|
|
for the next pathname. Transmission of a null pathname terminates
|
|
batch file transmission. Note that transmission of no files is not
|
|
necessarily an error. This is possible if none of the files requested
|
|
of the sender could be opened for reading.
|
|
|
|
In batch transmission, the receiver automatically requests CRC-16.
|
|
|
|
The Unix programs sb(1) and rb(1) included in the source code file
|
|
RBSB.QQQ (rbsb.sh) should answer questions about YAM's batch protocol.
|
|
|
|
Figure 3. Batch Transmission Session
|
|
|
|
SENDER RECEIVER
|
|
"sb foo.*"
|
|
"sending in batch mode etc."
|
|
C
|
|
SOH 00 FF foo.c NUL[123] CRC CRC
|
|
ACK
|
|
C
|
|
SOH 01 FE Data[128] CRC CRC
|
|
ACK
|
|
SOH 02 FD Data[1024] CRC CRC
|
|
ACK
|
|
SOH 03 FC Data[128] CRC CRC
|
|
ACK
|
|
SOH 04 FB Data[100] CPMEOF[28] CRC CRC
|
|
ACK
|
|
EOT
|
|
ACK
|
|
C
|
|
SOH 00 FF NUL[128] CRC CRC
|
|
ACK
|
|
|
|
Figure 4. Filename packet transmitted by sb
|
|
|
|
-rw-r--r-- 6347 Jun 17 1984 20:34 bbcsched.txt
|
|
|
|
00 0100FF62 62637363 6865642E 74787400 |...bbcsched.txt.|
|
|
10 36333437 20333331 34373432 35313320 |6347 3314742513 |
|
|
20 31303036 34340000 00000000 00000000 |100644..........|
|
|
30 00000000 00000000 00000000 00000000
|
|
80 000000CA 56
|
|
|
|
|
|
Chapter 4
|
|
|
|
XMODEM PROTOCOL OVERVIEW
|
|
|
|
|
|
8/9/82 by Ward Christensen.
|
|
|
|
I will maintain a master copy of this. Please pass on changes or
|
|
suggestions via CBBS/Chicago at (312) 545-8086, CBBS/CPMUG (312)
|
|
849-1132 or by voice at (312) 849-6279.
|
|
|
|
4.1 Definitions
|
|
|
|
<soh> 01H
|
|
<eot> 04H
|
|
<ack> 06H
|
|
<nak> 15H
|
|
<can> 18H
|
|
<C> 43H
|
|
|
|
4.2 Transmission Medium Level Protocol
|
|
|
|
Asynchronous, 8 data bits, no parity, one stop bit.
|
|
|
|
The protocol imposes no restrictions on the contents of the data being
|
|
transmitted. No control characters are looked for in the 128-byte
|
|
data messages. Absolutely any kind of data may be sent - binary,
|
|
ASCII, etc. The protocol has not formally been adopted to a 7-bit
|
|
environment for the transmission of ASCII-only (or unpacked-hex) data
|
|
, although it could be simply by having both ends agree to AND the
|
|
protocol-dependent data with 7F hex before validating it. I
|
|
specifically am referring to the checksum, and the block numbers and
|
|
their ones- complement.
|
|
|
|
Those wishing to maintain compatibility of the CP/M file structure,
|
|
i.e. to allow modemming ASCII files to or from CP/M systems should
|
|
follow this data format:
|
|
|
|
+ ASCII tabs used (09H); tabs set every 8.
|
|
|
|
+ Lines terminated by CR/LF (0DH 0AH)
|
|
|
|
+ End-of-file indicated by ^Z, 1AH. (one or more)
|
|
|
|
+ Data is variable length, i.e. should be considered a
|
|
continuous stream of data bytes, broken into 128-byte chunks
|
|
purely for the purpose of transmission.
|
|
|
|
+ A CP/M "peculiarity": If the data ends exactly on a 128-byte
|
|
boundary, i.e. CR in 127, and LF in 128, a subsequent sector
|
|
containing the ^Z EOF character(s) is optional, but is
|
|
preferred. Some utilities or user programs still do not
|
|
handle EOF without ^Zs.
|
|
|
|
|
|
Chapter 4 (cont)
|
|
|
|
|
|
+ The last block sent is no different from others, i.e. there
|
|
is no "short block".
|
|
|
|
Figure 5. XMODEM Message Block Level Protocol
|
|
|
|
Each block of the transfer looks like:
|
|
|
|
<SOH><blk #><255-blk #><--128 data bytes--><cksum>
|
|
in which:
|
|
<SOH> = 01 hex
|
|
<blk #> = binary number, starts at 01 increments by 1,
|
|
and wraps 0FFH to 00H (not to 01)
|
|
<255-blk #> = blk # after going thru 8080 "CMA" instr, i.e.
|
|
each bit complemented in the 8-bit block
|
|
number. Formally, this is the "ones
|
|
complement".
|
|
<cksum> = the sum of the data bytes only. Toss any
|
|
carry.
|
|
|
|
|
|
4.3 File Level Protocol
|
|
|
|
4.3.1 Common_to_Both_Sender_and_Receiver
|
|
|
|
All errors are retried 10 times. For versions running with an
|
|
operator (i.e. NOT with XMODEM), a message is typed after 10 errors
|
|
asking the operator whether to "retry or quit".
|
|
|
|
Some versions of the protocol use <can>, ASCII ^X, to cancel
|
|
transmission. This was never adopted as a standard, as having a
|
|
single "abort" character makes the transmission susceptible to false
|
|
termination due to an <ack> <nak> or <soh> being corrupted into a
|
|
<can> and cancelling transmission.
|
|
|
|
The protocol may be considered "receiver driven", that is, the sender
|
|
need not automatically re-transmit, although it does in the current
|
|
implementations.
|
|
|
|
|
|
4.3.2 Receive_Program_Considerations
|
|
|
|
The receiver has a 10-second timeout. It sends a <nak> every time it
|
|
times out. The receiver's first timeout, which sends a <nak>, signals
|
|
the transmitter to start. Optionally, the receiver could send a <nak>
|
|
immediately, in case the sender was ready. This would save the
|
|
initial 10 second timeout. However, the receiver MUST continue to
|
|
timeout every 10 seconds in case the sender wasn't ready.
|
|
|
|
|
|
Chapter 4 (cont)
|
|
|
|
|
|
Once into a receiving a block, the receiver goes into a one-second
|
|
timeout for each character and the checksum. If the receiver wishes
|
|
to <nak> a block for any reason (invalid header, timeout receiving
|
|
data), it must wait for the line to clear. See "programming tips" for
|
|
ideas.
|
|
|
|
Synchronizing: If a valid block number is received, it will be:
|
|
|
|
1. the expected one, in which case everything is fine; or
|
|
2. a repeat of the previously received block. This should be
|
|
considered OK, and only indicates that the receivers <ack>
|
|
got glitched, and the sender re-transmitted
|
|
3. any other block number indicates a fatal loss of
|
|
synchronization, such as the rare case of the sender getting
|
|
a line-glitch that looked like an <ack>. Abort the
|
|
transmission, sending a <can>
|
|
|
|
|
|
4.3.3 Sending_program_considerations
|
|
|
|
While waiting for transmission to begin, the sender has only a single
|
|
very long timeout, say one minute. In the current protocol, the
|
|
sender has a 10 second timeout before retrying. I suggest NOT doing
|
|
this, and letting the protocol be completely receiver-driven. This
|
|
will be compatible with existing programs.
|
|
|
|
When the sender has no more data, it sends an <eot>, and awaits an
|
|
<ack>, resending the <eot> if it doesn't get one. Again, the protocol
|
|
could be receiver-driven, with the sender only having the high-level
|
|
1-minute timeout to abort.
|
|
|
|
Here is a sample of the data flow, sending a 3-block message. It
|
|
includes the two most common line hits - a garbaged block, and an
|
|
<ack> reply getting garbaged. <xx> represents the checksum byte.
|
|
|
|
|
|
Figure 6. Data flow including Error Recovery
|
|
|
|
SENDER RECEIVER
|
|
times out after 10 seconds,
|
|
<--- <nak>
|
|
<soh> 01 FE -data- <xx> --->
|
|
<--- <ack>
|
|
<soh> 02 FD -data- xx ---> (data gets line hit)
|
|
<--- <nak>
|
|
<soh> 02 FD -data- xx --->
|
|
<--- <ack>
|
|
<soh> 03 FC -data- xx --->
|
|
(ack gets garbaged) <--- <ack>
|
|
<soh> 03 FC -data- xx ---> <ack>
|
|
<eot> --->
|
|
<--- <ack>
|
|
|
|
|
|
Chapter 4 (cont)
|
|
|
|
|
|
4.4 Programming Tips
|
|
|
|
+ The character-receive subroutine should be called with a
|
|
parameter specifying the number of seconds to wait. The
|
|
receiver should first call it with a time of 10, then <nak>
|
|
and try again, 10 times.
|
|
|
|
After receiving the <soh>, the receiver should call the
|
|
character receive subroutine with a 1-second timeout, for the
|
|
remainder of the message and the <cksum>. Since they are sent
|
|
as a continuous stream, timing out of this implies a serious
|
|
like glitch that caused, say, 127 characters to be seen
|
|
instead of 128.
|
|
|
|
+ When the receiver wishes to <nak>, it should call a "PURGE"
|
|
subroutine, to wait for the line to clear. Recall the sender
|
|
tosses any characters in its UART buffer immediately upon
|
|
completing sending a block, to ensure no glitches were mis-
|
|
interpreted.
|
|
|
|
The most common technique is for "PURGE" to call the character
|
|
receive subroutine, specifying a 1-second timeout[1], and
|
|
looping back to PURGE until a timeout occurs. The <nak> is
|
|
then sent, ensuring the other end will see it.
|
|
|
|
+ You may wish to add code recommended by John Mahr to your
|
|
character receive routine - to set an error flag if the UART
|
|
shows framing error, or overrun. This will help catch a few
|
|
more glitches - the most common of which is a hit in the high
|
|
bits of the byte in two consecutive bytes. The <cksum> comes
|
|
out OK since counting in 1-byte produces the same result of
|
|
adding 80H + 80H as with adding 00H + 00H.
|
|
|
|
|
|
Chapter 5
|
|
|
|
XMODEM/CRC Overview
|
|
|
|
|
|
1/13/85 by John Byrns -- CRC option.
|
|
|
|
Please pass on any reports of errors in this document or suggestions
|
|
for improvement to me via Ward's/CBBS at (312) 849-1132, or by voice
|
|
at (312) 885-1105.
|
|
-------
|
|
1. These times should be adjusted for use with timesharing systems.
|
|
|
|
|
|
Chapter 5 (cont)
|
|
|
|
|
|
The CRC used in the Modem Protocol is an alternate form of block check
|
|
which provides more robust error detection than the original checksum.
|
|
Andrew S. Tanenbaum says in his book, Computer Networks, that the
|
|
CRC-CCITT used by the Modem Protocol will detect all single and double
|
|
bit errors, all errors with an odd number of bits, all burst errors of
|
|
length 16 or less, 99.997% of 17-bit error bursts, and 99.998% of
|
|
18-bit and longer bursts.
|
|
|
|
The changes to the Modem Protocol to replace the checksum with the CRC
|
|
are straight forward. If that were all that we did we would not be
|
|
able to communicate between a program using the old checksum protocol
|
|
and one using the new CRC protocol. An initial handshake was added to
|
|
solve this problem. The handshake allows a receiving program with CRC
|
|
capability to determine whether the sending program supports the CRC
|
|
option, and to switch it to CRC mode if it does. This handshake is
|
|
designed so that it will work properly with programs which implement
|
|
only the original protocol. A description of this handshake is
|
|
presented in section 10.
|
|
|
|
Figure 7. Message Block Level Protocol, CRC mode
|
|
|
|
Each block of the transfer in CRC mode looks like:
|
|
|
|
<SOH><blk #><255-blk #><--128 data bytes--><CRC hi><CRC lo>
|
|
in which:
|
|
<SOH> = 01 hex
|
|
<blk #> = binary number, starts at 01 increments by 1,
|
|
and wraps 0FFH to 00H (not to 01)
|
|
<255-blk #> = ones complement of blk #.
|
|
<CRC hi> = byte containing the 8 hi order coefficients of
|
|
the CRC.
|
|
<CRC lo> = byte containing the 8 lo order coefficients of
|
|
the CRC. See the next section for CRC
|
|
calculation.
|
|
|
|
5.1 CRC Calculation
|
|
|
|
5.1.1 Formal_Definition
|
|
|
|
To calculate the 16 bit CRC the message bits are considered to be the
|
|
coefficients of a polynomial. This message polynomial is first
|
|
multiplied by X^16 and then divided by the generator polynomial (X^16
|
|
+ X^12 + X^5 + 1) using modulo two arithemetic. The remainder left
|
|
after the division is the desired CRC. Since a message block in the
|
|
Modem Protocol is 128 bytes or 1024 bits, the message polynomial will
|
|
be of order X^1023. The hi order bit of the first byte of the message
|
|
block is the coefficient of X^1023 in the message polynomial. The lo
|
|
order bit of the last byte of the message block is the coefficient of
|
|
X^0 in the message polynomial.
|
|
|
|
|
|
Chapter 5 (cont)
|
|
|
|
|
|
Figure 8. Example of CRC Calculation written in C
|
|
|
|
/*
|
|
* This function calculates the CRC used by the XMODEM/CRC
|
|
* Protocol.
|
|
* The first argument is a pointer to the message block.
|
|
* The second argument is the number of bytes in the message
|
|
* block.
|
|
* The function returns an integer which contains the CRC.
|
|
* The low order 16 bits are the coefficients of the CRC.
|
|
*/
|
|
|
|
int calcrc(ptr, count)
|
|
char *ptr;
|
|
int count;
|
|
{
|
|
int crc, i;
|
|
|
|
crc = 0;
|
|
while(--count >= 0) {
|
|
crc = crc ^ (int)*ptr++ << 8;
|
|
for(i = 0; i < 8; ++i)
|
|
if(crc & 0x8000)
|
|
crc = crc << 1 ^ 0x1021;
|
|
else
|
|
crc = crc << 1;
|
|
}
|
|
return (crc & 0xFFFF);
|
|
}
|
|
|
|
|
|
Chapter 5 (cont)
|
|
|
|
|
|
5.2 CRC File Level Protocol Changes
|
|
|
|
5.2.1 Common_to_Both_Sender_and_Receiver
|
|
|
|
The only change to the File Level Protocol for the CRC option is the
|
|
initial handshake which is used to determine if both the sending and
|
|
the receiving programs support the CRC mode. All Modem Programs should
|
|
support the checksum mode for compatibility with older versions. A
|
|
receiving program that wishes to receive in CRC mode implements the
|
|
mode setting handshake by sending a <C> in place of the initial <nak>.
|
|
If the sending program supports CRC mode it will recognize the <C> and
|
|
will set itself into CRC mode, and respond by sending the first block
|
|
as if a <nak> had been received. If the sending program does not
|
|
support CRC mode it will not respond to the <C> at all. After the
|
|
receiver has sent the <C> it will wait up to 3 seconds for the <soh>
|
|
that starts the first block. If it receives a <soh> within 3 seconds
|
|
it will assume the sender supports CRC mode and will proceed with the
|
|
file exchange in CRC mode. If no <soh> is received within 3 seconds
|
|
the receiver will switch to checksum mode, send a <nak>, and proceed
|
|
in checksum mode. If the receiver wishes to use checksum mode it
|
|
should send an initial <nak> and the sending program should respond to
|
|
the <nak> as defined in the original Modem Protocol.
|
|
|
|
After the mode has been set by the initial <C> or <nak> the protoco
|
|
follows the original Modem Protocol and is identical whether the
|
|
checksum or CRC is being used.
|
|
|
|
|
|
5.2.2 Receive_Program_Considerations
|
|
|
|
There are at least 4 things that can go wrong with the mode setting
|
|
handshake.
|
|
|
|
1. the initial <C> can be garbled or lost.
|
|
|
|
2. the initial <soh> can be garbled.
|
|
|
|
3. the initial <C> can be changed to a <nak>.
|
|
|
|
4. the initial <nak> from a receiver which wants to receive in
|
|
checksum can be changed to a <C>.
|
|
|
|
The first problem can be solved if the receiver sends a second <C>
|
|
after it times out the first time. This process can be repeated
|
|
several times. It must not be repeated too many times before sending
|
|
a <nak> and switching to checksum mode or a sending program without
|
|
CRC support may time out and abort. Repeating the <C> will also fix
|
|
the second problem if the sending program cooperates by responding as
|
|
if a <nak> were received instead of ignoring the extra <C>.
|
|
|
|
|
|
Chapter 5 (cont)
|
|
|
|
|
|
It is possible to fix problems 3 and 4 but probably not worth the
|
|
trouble since they will occur very infrequently. They could be fixed
|
|
by switching modes in either the sending or the receiving program
|
|
after a large number of successive <nak>s. This solution would risk
|
|
other problems however.
|
|
|
|
|
|
5.2.3 Sending_Program_Considerations
|
|
|
|
The sending program should start in the checksum mode. This will
|
|
insure compatibility with checksum only receiving programs. Anytime a
|
|
<C> is received before the first <nak> or <ack> the sending program
|
|
should set itself into CRC mode and respond as if a <nak> were
|
|
received. The sender should respond to additional <C>s as if they were
|
|
<nak>s until the first <ack> is received. This will assist the
|
|
receiving program in determining the correct mode when the <soh> is
|
|
lost or garbled. After the first <ack> is received the sending program
|
|
should ignore <C>s.
|
|
|
|
|
|
5.3 Data Flow Examples with CRC Option
|
|
|
|
Here is a data flow example for the case where the receiver requests
|
|
transmission in the CRC mode but the sender does not support the CRC
|
|
option. This example also includes various transmission errors. <xx>
|
|
represents the checksum byte.
|
|
|
|
|
|
Figure 9. Data Flow: Receiver has CRC Option, Sender Doesn't
|
|
|
|
SENDER RECEIVER
|
|
<--- <C>
|
|
times out after 3 seconds,
|
|
<--- <nak>
|
|
<soh> 01 FE -data- <xx> --->
|
|
<--- <ack>
|
|
<soh> 02 FD -data- <xx> ---> (data gets line hit)
|
|
<--- <nak>
|
|
<soh> 02 FD -data- <xx> --->
|
|
<--- <ack>
|
|
<soh> 03 FC -data- <xx> --->
|
|
(ack gets garbaged) <--- <ack>
|
|
times out after 10 seconds,
|
|
<--- <nak>
|
|
<soh> 03 FC -data- <xx> --->
|
|
<--- <ack>
|
|
<eot> --->
|
|
<--- <ack>
|
|
|
|
|
|
Chapter 5 (cont)
|
|
|
|
|
|
Here is a data flow example for the case where the receiver requests
|
|
transmission in the CRC mode and the sender supports the CRC option.
|
|
This example also includes various transmission errors. <xxxx>
|
|
represents the 2 CRC bytes.
|
|
|
|
Figure 10. Receiver and Sender Both have CRC Option
|
|
|
|
SENDER RECEIVER
|
|
<--- <C>
|
|
<soh> 01 FE -data- <xxxx> --->
|
|
<--- <ack>
|
|
<soh> 02 FD -data- <xxxx> ---> (data gets line hit)
|
|
<--- <nak>
|
|
<soh> 02 FD -data- <xxxx> --->
|
|
<--- <ack>
|
|
<soh> 03 FC -data- <xxxx> --->
|
|
(ack gets garbaged) <--- <ack>
|
|
times out after 10
|
|
seconds,
|
|
<--- <nak>
|
|
<soh> 03 FC -data- <xxxx> --->
|
|
<--- <ack>
|
|
<eot> --->
|
|
<--- <ack>
|
|
|
|
|
|
Technical questions about Professional-YAM and requests for evaluation
|
|
copies for magazine reviews may be directed to:
|
|
|
|
Chuck Forsberg
|
|
Omen Technology Inc
|
|
17505-V Sauvie Island Road
|
|
Portland Oregon 97231
|
|
Voice: 503-621-3406
|
|
Modem: 503-621-3746 Speed: 1200,300
|
|
Compuserve: 70715,131
|
|
Source: TCE022
|
|
|
|
More information about Professional-YAM may be obtained by calling
|
|
Telegodzilla at 503-621-3746. Hit RETURNs for baud rate detection.
|
|
|
|
A version of this file with boldface, underlining, and superscripts
|
|
for printing on Epson or Gemini printers is available on Telegodzilla
|
|
as "YMODEME.DOC" or "YMODEME.DQC".
|