Help Info for MUSIC/SP FTP Server File: $tcp:ftphelp.txt
================================= Author: Dave Edwards
Updated: Aug 6, 2007
(c) Copyright Dave Edwards 2000-2007. All rights reserved.
Contents:
1. Introduction
2. General Notes on the MUSIC/SP File System
3. Commands Supported by MUSIC/SP's FTP Server
4. Using MUSIC/SP's FTP Client Program
5. Using the FTP Client Program Provided by Windows 95/98/NT/2000/XP
6. Using the WS_FTP Client
7. Using MUSIC/SP as a Repository for PC Files
8. Using the PC as a Repository for MUSIC/SP Files
See also: $tcp:ftpd_info.txt
1. Introduction
---------------
This file has information for users using an FTP (File Transfer
Protocol) client program to connect to the FTP Server (FTPD)
running on a MUSIC/SP system. Examples of FTP client programs
are (1) the FTP command of MUSIC/SP; (2) the ftp.exe program provided
with Windows 95/98/NT/2000/XP (enter the command "ftp xxx" from a DOS
session, where xxx is the server host name or numeric IP address);
(3) the excellent WS_FTP product available from www.ipswitch.com;
(4) a Web browser, using a URL of the form ftp://site; (5) various
FTP clients available on Unix and Linux systems.
The FTP Protocol is defined in RFC 959 (and its later versions).
This file can be obtained by the command "get ftphelp.txt" after
connecting to a MUSIC/SP FTP server and logging in. FTPHELP.TXT is
a special public file that can be got while in any directory.
MUSIC/SP stands for Multi-User System for Interactive Computing /
System Product, an operating system originally developed by McGill
University. For info on it, visit the Web sites:
http://canpub.com/teammpg/de/
http://canpub.com/teammpg/de/sim390
2. General Notes on the MUSIC/SP File System
--------------------------------------------
MUSIC/SP is an operating system that runs on IBM S/370-S/390
mainframe computers, usually as a virtual machine under VM (VM/SP
or VM/ESA). TCP/IP services are provided to MUSIC/SP by the
TCPIP virtual machine running under VM. MUSIC/SP can also run
under the Sim390 and Hercules mainframe emulators on a PC, in which
case TCP/IP services are provided by the emulator and the
underlying operating system (such as Windows). Currently, MUSIC/SP
under Hercules cannot use TCP/IP; TCP/IP is ok under Sim390.
The character set encoding for text is EBCDIC, not ASCII. Text
files (i.e. non-binary files) are translated by the FTP server from
EBCDIC to ASCII before being sent to the FTP client, and from ASCII
to EBCDIC when received from the client. See the TYPE, AUTOTYPE,
and BINARY BLOCK commands below. Normally, in a text file in Ascii
form, record ends are indicated by CR/LF (the 2 Ascii control
characters CR = hex 0D and LF = hex 0A), or a single LF character.
The MUSIC/SP file system has a directory structure similar to that
of DOS and Unix. The separator character between directory names
is a backslash (\), but the FTP client can use either a slash (/)
or a backslash (\). Directory names can be up to 17 characters.
File names within a directory can be up to 17 characters. The total
name, including directory names on the front, can be up to 50
characters. Upper/lower case is not significant in file names.
Letters (A to Z), digits (0 to 9), and some special characters
(including $ # @ _ + - . % & !) can be used, but the first character
of each part must not be a digit or + - . % & !.
An example of a full MUSIC file name is abcde:test\source\prog1.s .
The prefix "abcde:" indicates that the file's owner is the MUSIC
userid ABCDE. Often the owner prefix can be omitted.
MUSIC files have a record format that is one of F (fixed-length
records), FC (fixed-length, compressed), V (variable-length records),
VC (variable-length, compressed), or U (undefined). A U format file
is treated as a sequence of 512-byte blocks. F and FC format files
must have a logical record length (RSIZE or LRECL) of 1 to 32767
bytes specified when they are created. The logical record length
of a V or VC file is the length of its longest record; on creation,
the logical record length of a V or VC file is ignored, and can
be specified as 0 (but, for a V or VC binary file created by MUSIC's
FTP client, the logical record length determines the length of all the
records except the last one, which may be shorter).
The maximum logical record length of a MUSIC file is 32767 (=32K-1),
but, for compatibility with other systems, record lengths > 32760
should be avoided. Some versions of MUSIC FTP client or server
have a lower limit on the record length supported for exchange of
files (in non-binary mode) with non-MUSIC systems; record lengths
up to 16384 (=16K) are usually safe.
The maximum size of a file is about 57 megabytes (or less, if
insufficient unfragmented free disk space is available). Logical
records are determined by internal control bytes and counts; a
logical record can contain any of the 256 possible byte values;
logical records are NOT separated by CR or CR/LF characters as in
DOS and Unix. However, when MUSIC's FTP server sends a text file
to an FTP client, it inserts CR/LF at the end of each logical
record sent to the client.
3. Commands Supported by MUSIC/SP's FTP Server
----------------------------------------------
These are the commands accepted by the FTP server (FTPD). They are
generated by the FTP client program, in response to the commands or
operations done by the user (GET, PUT, etc.)
Most FTP clients have a mechanism, such as the QUOTE command, for
sending any desired command to the server. This is useful when the
user needs to send server-specific commands, such as ones for setting
special server options. The FTP client user enters the command
"quote xxx", where xxx is any command supported by the server, and
the client program then sends the command "xxx" to the server to be
executed, and displays the response from the server. Examples:
"quote help", "quote autotype off", "quote rsize 100".
Some FTP clients use the SITE command instead of QUOTE.
When using a non-MUSIC FTP client, the QUOTE or SITE command must
be used for MUSIC FTP server commands which are non-standard,
such as AUTOTYPE, BINARY BLOCK, DIRFORMAT, IF, RECFM, LRECL, RSIZE,
and may also be needed for other commands such as SYST, ALLO.
For example, to send command AUTOTYPE OFF to the server, the
user types "quote autotype off" in the non-MUSIC FTP client.
This may also be needed even in the MUSIC FTP client, since it
does not recognize all of the server commands.
Behaviour of the MUSIC FTP server can be customized by the system
administrator, by setting various parameters in the file $TCP:FTPD .
The port number (normally 21) on which the server listens for
connections can be specified in file $TCP:INETD.PORTS .
In the command descriptions below, aliases for command names and
option keywords are stacked vertically. Alternatives (e.g. ON or
OFF) for operand values are also shown stacked vertically. Words
shown in upper case should be entered as given. Variable parts of
the command are shown in lower case; "filename" stands for a server
file name; "dir" stands for a server directory name. Upper/lower
case is not significant in the commands (except in the password on
the PASS command if the userid has the PWCASENS Profile option set).
Optional items are shown in square brackets [ ]; do not enter the
square brackets.
Commands accepted by MUSIC/SP's FTP server:
USER userid
Specifies the MUSIC/SP userid for FTP log-in.
This command is not valid AFTER a successful log-in.
NOTE: Parameters in file $TCP:FTPD can be set up to
allow anonymous (unauthenticated) access, in which
case a special userid such as anonymous or ftp
is accepted, which is not a real MUSIC/SP userid.
PASS password
Specifies the password for the userid. It must
immediately follow the USER command, if a password
is required. Some special userids, such as anonymous
of ftp (if allowed), do not require a password.
NOTES:
- If your MUSIC userid requires an extra password for
sign-on, enter xxx!!yyy as the password field to
FTP, where xxx is your normal password and yyy is
the extra password. If you do not know the extra
password number required, just enter xxx, and you
will get a message giving the number. Then log in
again.
- MUSIC allows a challenge/response protocol for
FTP login, which avoids transmitting the password
in clear text over the network. The 331 message
for the USER command includes a random challenge
string. The user calculates the MD5 digest (as 32
printable hex characters) of the string xxxppp where
xxx is the challenge and ppp is the password. Then
the user sends the 32-char MD5 digest instead of the
password. The Windows command-line program md5pw.exe
can be used to calculate the MD5 digest. It can be
downloaded from web sites mentioned in section 1.
The FTP server accepts both the password and the MD5
digest string, therefore use of the challenge is
optional, but is recommended for better security.
- In the Windows ftp.exe client, the maximum length
of the password field on the PASS command is 32
characters. The MUSIC FTP server allows longer
password fields.
QUIT
Closes the TCP/IP connection and exits from the FTP
server program.
HELP
Gives help information about the FTP server.
It displays a short message, referring to this file
(FTPHELP.TXT) for more information.
SYST
Identifies the server system as MUSIC/SP, and also may
give the virtual machine name ("system name") if any.
Notes:
- The QUOTE or SITE command may be needed in some
non-MUSIC FTP clients. For example: quote syst
- The first word of the text response is supposed to
indicate the general type of server system.
For a MUSIC/SP system, the first word may depend
on the current setting of the DIRFORMAT command.
If DIRFORMAT UNIX is in effect, the first word is
normally UNIX. Experiment to find the actual
output for your MUSIC server.
CWD dir
Changes to a new current working directory on the
server. dir can be / or \ to go to the root, or
.. to go up one level.
CDUP
Changes to the parent directory, i.e. one level higher,
on the server. It is equivalent to "cwd ..".
PWD
XPWD
Prints (i.e. displays) the name of the current working
directory on the server.
PORT a1,a2,a3,a4,p1,p2
Specifies the numeric IP address and port number to be
used for a subsequent data transfer. This command is
generated automatically by the FTP client, as needed.
This command is not used in Passive Mode (see the PASV
command). The client listens on the specified port,
and then the server connects to it, thus establishing
the TCP data connection for the transfer. After the
file (or directory listing data) has been transferred,
the data connection is closed. The IP address on the
PORT command is normally that of the FTP client (and
it must be, unless the OK3RDP=T parameter is specified
in file $TCP:FTPD for the server). The port number
is p1*256+p2.
Example: for client IP addr 130.12.34.156 and port 4321,
the command is: port 130,12,34,156,16,225
PASV
This command tells the server to use Passive Mode
for the next file transfer (or directory listing).
In Passive Mode, the server creates a socket and
listens on it, and tells the client the port via the
227 response to the PASV cmd. Then the client
connects to the port, which establishes the data
connection for the transfer. The form of the response
to the PASV command is:
227 Entering Passive Mode (a1,a2,a3,a4,p1,p2)
The IP address is that of the server (same format as
in the PORT command), and the server port number is
p1*256+p2.
The alternative to Passive Mode is "active" or
"normal" mode, in which the client creates a socket
and listens on it, tells the server the port # via
the PORT cmd, and the server connects to the client's
port, in order to establish the data connection.
Most modern FTP clients try the PASV cmd, and use
Passive Mode if possible. Passive Mode is friendlier
to firewalls and routers, especially when NAT (Network
Address Translation) is done for clients behind the
router or firewall. Passive Mode can be disabled in
the server by the parameter PASVOK=F in file $TCP:FTPD .
Some earlier versions of MUSIC FTPD do not support
Passive Mode. The Windows client ftp.exe does not
support Passive Mode. Recent versions of the MUSIC/SP
FTP client support Passive Mode, via the PASSIVE ON/OFF
client command. Most web browsers support Passive mode.
In MS Internet Explorer, it is turned on by Tools -->
Internet Options --> Advanced --> "Use Passive FTP".
LIST [dir]
This command is generated by the DIR client command.
The server sends a list of the file names in the
current (or specified) directory, with attribute
information for each file. The first line is a
header line with column titles. Attribute information
includes record length, record format, file size,
last read date, last write date, userid of last
writer, access control attributes, and number of
records. Subdirectory names are indicated by
.
Note: If DIRFRM=1 is specified in the configuration
of the MUSIC/SP FTP server (in file $tcp:ftpd),
the server gives the directory listing in the standard
Unix format, rather than in the native MUSIC format.
This can also be controlled by the DIRFORMAT command.
DIRFORMAT MUSIC
UNIX
Specifies the desired format for the output of the
LIST (directory list) command. MUSIC (or MUS)
specifies MUSIC native format (as in the MUSIC
command LIBR * X). UNIX specifies the format as in
the Unix "ls -l" command. The default is as specified
by the DIRFRM=n parameter in server file $tcp:ftpd;
DIRFRM=0 uses MUS format, DIRFRM=1 uses UNIX format.
(The default is usually UNIX format.)
Abbreviations: DIRFORM MUS
Note: Since this is a non-standard server command,
the QUOTE or SITE command is normally needed in a
non-MUSIC client. For example: quote dirform mus
NLST [dir]
This command is generated by the LS client command.
Same as the LIST command, but only the file names
are displayed, without the header line and attributes.
TYPE A[SCII]
I[MAGE]
E[BCDIC]
Specifies the type of data to be transferred:
(1) ASCII means text (printable) characters. The server
translates from EBCDIC encoding to ASCII before
sending data, and translates from ASCII to EBCDIC
when receiving data. Client logical records are
assumed to be separated by special control characters
such as CR/LF (unless MODE BLOCK is in effect - see
the MODE command below). When the server sends a
text file to the client, it removes trailing blanks
from each logical record (if the MUSIC file is record
format F or FC) and sends CR/LF after each record.
(2) IMAGE means binary (non-text) data, where each byte
may contain any of the possible 256 bit combinations.
No character set translation is done. Client logical
records are not assumed to be separated by control
characters such as CR/LF. When the server sends a
MUSIC file to the client, each logical record is sent
as is, with no translation, and with no CR/LF added;
in the case of a record format U file, each 512-byte
block of the file is sent as is.
(3) EBCDIC means text (like ASCII), but no translation
between ASCII and EBCDIC character sets is done,
since the data is assumed to be in EBCDIC on both the
server and client systems.
See also the AUTOTYPE, MODE, and BINARY BLOCK commands
below.
AUTOTYPE [ON ]
[OFF]
Specifies whether the AUTOTYPE option should be used or
not in the FTP server program. When AUTOTYPE is ON,
the server determines the TYPE of data transfer (see
the TYPE command above) from the file name extension,
regardless of the current setting of the TYPE
command. The extension is the ".xxx" part at the end
of the MUSIC/SP file name. Some extensions (.BIN, .EXE,
.COM, .GIF, .ARC, .OBJ, .LMOD, .DOC, .XLS, .PPT, .MDB,
.JPEG, .JPG, .ZIP, etc.) imply binary data (TYPE
IMAGE), while others (.TXT, .TEXT, .HTML, .HTM, .S,
.C, .BAT, etc.) imply text data (TYPE ASCII). The
extensions are defined in file $TCP:FTPD.FILETYPES .
An unknown extension is usually assumed to be text
data. However, the AUTOTYPE option does not affect
the TYPE when TYPE EBCDIC is in effect, or when
MUSIC-to-MUSIC mode is in effect (see the BINARY
BLOCK command). When AUTOTYPE is OFF, the TYPE
command setting is honoured and the file extension is
ignored. AUTOTYPE is normally ON by default (but the
MUSIC/SP system administrator may change that by the
AUTOTY=F parameter in $TCP:FTPD). If neither ON nor
OFF is specified on the AUTOTYPE command, ON is
assumed.
Note: Since this is a non-standard server command,
the QUOTE or SITE command is normally needed in a
non-MUSIC client. For example: quote autotype off
Examples: (1) If you want to transfer a binary file called
MYFILE.XYZ, you should enter the commands
"type image" and "quote autotype off" in the
FTP client, before using the GET or PUT command.
Otherwise the MUSIC/SP FTP server may assume the
file is a text file (since the extension .XYZ is
unknown), even though you used "type image".
(2) If you want to transfer a text file called
MYNOTES.DOC, you should enter the commands
"type ascii" and "quote autotype off" in the FTP
client, before using the GET or PUT command.
Otherwise the MUSIC/SP FTP server may assume the
file is a binary file (MS Word document), even
though you used "type ascii". For this reason, it
is best not to use .DOC when naming a text file.
MODE S[TREAM]
B[LOCK]
This defines the "transmission mode" for transferring
text (TYPE ASCII or TYPE EBCDIC) data. In STREAM mode,
which is the default, logical records on the client
system are assumed to separated by special control
characters, such as CR/LF. In BLOCK mode, each logical
record is transmitted with preceding header bytes,
which contain the length of the record. The MODE
command does not affect binary transfers (TYPE IMAGE).
BLOCK mode is rarely used.
BINARY BLOCK [ON ]
BIN [OFF]
[MUS512]
This command sets binary data transfer (TYPE IMAGE),
regardless of whether ON or OFF is specified.
In addition, if ON is specified (or if no option is
specified after BLOCK), a special block transfer mode
is set. In that mode, data is transferred in 512-byte
blocks, exactly matching the internal 512-byte
block structure of the MUSIC/SP file. When both the
FTP client and server are MUSIC systems, this allows
an efficient, exact file transfer in all cases, and
also preserves the file's attributes (record format,
record length, tag, space values, etc.) When the
MUSIC client detects that the server is also a MUSIC
system, BINARY BLOCK ON is automatically set for all
file transfers; this is called "MUSIC-to-MUSIC" mode
or "MUSIC File Copy" mode; you do not have to worry
about using the correct TYPE, MODE, RECFM, RSIZE, or
ALLO commands in that case.
To turn off MUSIC-to-MUSIC mode, use the command
"binary block off", followed by the appropriate TYPE
and AUTOTYPE commands.
Note: Since this is a non-standard server command,
the QUOTE or SITE command is normally needed in a
non-MUSIC client. For example: quote binary block on
Even in the MUSIC client, the quote command form is
needed when the MUS512 option is used.
Additional notes on "binary block on" (MUSIC-to-MUSIC) mode:
- In this mode, when sending a file to the FTP client,
MUSIC's FTPD server first sends 94 bytes of binary
file info, containing: MFINFO (INFIN) block (20 bytes),
EOFPT block (10 bytes), file TAG (64 bytes). Refer to
the "File System" chapter in the MUSIC/SP Admin Ref. manual.
Then MUSIC's FTPD server sends the 512-byte data blocks
of the file, unchanged, as they appear on MUSIC's disk.
The last 512-byte block may be short (less than 512 bytes)
in some cases. Source file reference: $TCP:FTPD.PUTBIN.S .
- In this mode, when receiving a file from the FTP client,
MUSIC's FTPD server interprets the first 94 bytes as the
file info header, and uses that info to set record format,
record length, file size, etc. when creating the target
MUSIC file.
- The client system does not have to be a MUSIC system in
order for MUSIC's FTPD server to use this mode. But then
the 94-byte file info header appears as extra data at
the start of the file on the FTP client side.
In this case, the client sets MUSIC-to-MUSIC mode in the
MUSIC server by the command "quote binary block on".
- This mode can also be used with a MUSIC client and a
non-MUSIC server. The user sets MUSIC-to-MUSIC mode in
the client by the command "binary block on" (without using
"quote"). Then the client sends the extra 94-byte header
on all binary uploads, and expects binary download data to
start with the header. NOTE: MUSIC's FTP client supports
the BINARY BLOCK ON command, but not the BINARY BLOCK MUS512
command. [The 12MAR2003 update to MUSIC's FTP client is
required in order to use the BINARY BLOCK ON command with a
non-MUSIC server.]
- The option MUS512 is very similar to the ON option,
except that the info header is 512 bytes instead of 94
bytes, and the final data block is always a full 512 bytes
(not a short block). The format of the 512-byte header
info is the same as that used by MUSIC/SP's MUS512DMP and
MUS512RST utilities (see $pgm:mus512dmp.s, $pgm:mus512rst.s).
It contains extra info, including the full name of the
original MUSIC/SP file. The FTP client should use the
command "quote binary block mus512" to set this mode in
the MUSIC/SP server. The client should also use the
command "type binary". In this mode, the MUSIC/SP server
sends all files to the client in the format described
(512-byte info header followed by 0 or more 512-byte data
blocks), and expects files from the client to be in the
same format. This mode is useful for storing MUSIC/SP files
(of any type) on a PC client machine, as binary files that
can later be restored to a MUSIC/SP system.
ALLO n
Specifies the size n (number of bytes) to be allocated
initially when the server creates a MUSIC file (for the
STOR or STOU command). This command may be needed
for very large files. Example of command entered in
the FTP client: "quote allo 2000000". This command is
not required for MUSIC-to-MUSIC mode (see BINARY BLOCK).
Note: The QUOTE or SITE command may be needed in a
non-MUSIC FTP client. For example: quote allo 500000
RECFM F
REC FC
V
VC
U
Specifies the record format to be used when the server
creates a MUSIC file (for the STOR or STOU command).
The default record format is VC for text transfers
and U for binary transfers. This command is not needed
in MUSIC-to-MUSIC mode (see BINARY BLOCK).
Note: The FTP client user enters a command such as
"recfm fc" or "lrecl 80" to specify the attributes of
files to be created by the MUSIC client (i.e. downloads).
The user enters a command such as "quote recfm fc" or
"quote lrecl 80" to specify the attributes of files
to be created by the MUSIC server (i.e. uploads).
RSIZE n
LRECL
Specifies the logical record length (0 or more), in
bytes, to be used when the server creates a MUSIC file
(for the STOR or STOU command). The default is 0, which
causes a record length of 512 or 4096 to be used when
a file is transferred to MUSIC. How the record length
is used depends on the TYPE of transfer (text or binary)
and on the record format (RECFM) in effect:
Text, F or FC: The record length should be greater than
or equal to the length of the longest logical record
to be transferred to MUSIC. Otherwise long records
are truncated.
Text, V or VC: The record length is ignored. Any size
record, up to the MUSIC maximum of 32760, is handled.
Binary, U: The record length is ignored.
Binary, F, FC, V, or VC: The data is divided into
logical records of the specified length. If the
total number of bytes transferred is not an exact
multiple of the record length, the final record is
shorter (V or VC), or is extended with zeros (F
or FC). If the record length is 0 (not specified),
512 is used for F or FC, and 4096 is used for V or VC,
on a download; on an upload, the default is 512 for
F, FC, V, or VC. ("Upload" refers to transferring a
file to the server.)
See the examples in the sections below. This command is
not needed in MUSIC-to-MUSIC mode (see BINARY BLOCK).
NOTE: Although the record length of a MUSIC file
may be up to 32767, the maximum for files created by
the FTP server is the data transfer buffer size,
normally 20480 ( = the BS parameter in file $TCP:FTPD).
The value n on the command must not exceed BS = 20480.
Note: The FTP client user enters a command such as
"recfm fc" or "lrecl 80" to specify the attributes of
files to be created by the MUSIC client (i.e. downloads).
The user enters a command such as "quote recfm fc" or
"quote lrecl 80" to specify the attributes of files
to be created by the MUSIC server (i.e. uploads).
BINARY FIXED n
BIN
This is equivalent to the commands RECFM F and RSIZE n.
NOTE: This command does NOT set the data TYPE (text or
binary).
BINARY VARIABLE
BIN
This is equivalent to the commands RECFM VC and RSIZE 0.
NOTE: This command does NOT set the data TYPE (text or
binary).
RETR filename
This command is generated by the FTP client when a GET
command is entered. It tells the server to send the
specified MUSIC file to the client. The current
settings from the previous TYPE, AUTOTYPE, MODE,
BINARY, RECFM, RSIZE, and ALLO server commands are
used for the file transfer.
STOR filename
This command is generated by the FTP client when a PUT
command is entered. It tells the server to receive
file data from the client, and store it into a new
MUSIC file, whose name is specified. If the MUSIC file
already exists, it is deleted. The current settings
from the previous TYPE, AUTOTYPE, MODE, BINARY,
RECFM, RSIZE, and ALLO server commands are used for
the file transfer.
STOU filename
This command (Store Unique) is the same as STOR,
except that the target MUSIC file is not replaced if
it already exists. Instead, if the file already
exists, a dollar sign character ($) is added at the
end of the MUSIC file name and that name is tried.
The server continues to add $'s until a name is found
that does not already exist.
APPE filename
(Append) This command is not supported by the server.
DELE filename
Deletes the specified MUSIC file.
RNFR filename1
Specifies a MUSIC file to be renamed. The RNFR and
RNTO commands are generated by the FTP client when a
RENAME command is entered.
RNTO filename2
Specifies the new MUSIC file name for a rename
operation. This command must immediately follow RNFR.
MKDIR dir
MKD
MD
XMKD
Creates a MUSIC subdirectory.
RMDIR dir
RMD
RD
XRMD
Deletes a MUSIC subdirectory.
IF FILE filename EXIT
IF NOTFILE filename EXIT
This command, which is a nonstandard command not in
the FTP protocol, can be used to test some condition
on the server and take an action based on that test.
Currently, the only condition that can be tested is
whether a specified file on the server is present and
accessible for downloading (GET) to the client.
The only action is EXIT, which terminates the FTP
session on the server and closes the control connection.
QUIT can be used as an alias for EXIT. The client must
use the QUOTE or SITE command to send the IF command
to the server. For example, if a client (perhaps in
an automated script of commands) wants the connection
to be closed (and therefore most subsequent commands
in the script to fail as "not logged in") if file
myfile.txt is not GETable, it could use the command:
quote if notfile myfile.txt exit
If the file exists and is GETable, the script continues
normally.
Notes on how FTP clients can cause a file to be transferred
between 2 servers ("3rd-party" file transfer):
Most FTP clients are not programmed to support this, but the
FTP server commands for doing it are part of the FTP specification
(see RFC 959 and its later versions). The QUOTE (or SITE)
command in the client can be used to send the required commands
to the servers.
Suppose server 1 (SRV1) is at IP address s1.s2.s3.s4, and
server 2 (SRV2) is at IP address t1.t2.t3.t4. We wish to
transfer a file fn1 on SRV1 to file fn2 on SRV2 (assume fn2
does not already exist on SRV2). SRV1 must be able to operate
in Passive Mode (i.e. PASVOK=T in $tcp:ftpd on MUSIC).
Both servers must allow "3rd-party" transfers (i.e. OK3RDP=T
in $tcp:ftpd). We require 2 client sessions (CLN1 and CLN2).
The procedure is:
- CLN1: Log-in to SRV1. Issue any cmds needed to set the type
of file transfer (text or binary).
- CLN2: Log-in to SRV2. Issue any cmds needed to set the type
of file transfer (text or binary).
- CLN1: quote pasv
- SRV1: Listens on port p1*256+p2 and sends response
227 Entering Passive Mode (s1,s2,s3,s4,p1,p2)
- CLN2: quote port s1,s2,s3,s4,p1,p2
- SRV2: Connects to port p1*256+p2 on SRV1, thus establishing
the data connection for the file transfer.
- CLN1: quote retr fn1
- CLN2: quote stor fn2
- At this point, the file transfer starts. SRV1 writes fn1 to
the data socket, and SRV2 reads from the socket and creates
file fn2. The clients should wait a suitable time for the
transfer to finish.
- Each client, if it does not automatically display a 226 "transfer
ended" message from its server, should issue a command such as
quote noop or quote syst, which should cause the message to be
displayed. (Some clients seem to have trouble with this, and
get out-of-synch between commands and responses, since the
QUOTE command is being used. The MUSIC client seems to have less
trouble than Windows ftp.exe.)
4. Using MUSIC/SP's FTP Client Program
--------------------------------------
MUSIC's FTP client program is started by the MUSIC command FTP.
For detailed help information on it, type the MUSIC command
"/help ftp". When you are in the FTP client, you can use the
command help to display a list of the client commands, or the
command lochelp to invoke more detailed full-screen help, or
the command remhelp (or remotehelp) to request help from the
server.
When the MUSIC FTP client connects to a MUSIC FTP server,
MUSIC-to-MUSIC file transfer mode is automatically set. In this
mode, all file characteristics and attributes are preserved when
they are transferred. You do not have to worry about using the
correct TYPE, MODE, BINARY, RSIZE, RECFM, or ALLO commands. All
file types (text and binary) are transferred correctly. You can
turn off MUSIC-to-MUSIC mode by the command "binary block off".
You can turn it on by the command "binary block on".
NOTE: Currently, MUSIC's FTP client can operate in "binary block on"
mode, even when talking to a non-MUSIC server. But it cannot
operate in "binary block mus512" mode.
When you transfer a file to MUSIC (by the GET command), you can
precede the GET command by RECFM, RSIZE, and SPACE commands, to
specify the file attributes to be used when MUSIC creates the file.
(These settings are ignored in MUSIC-to-MUSIC mode.)
For example:
recfm fc <-- other values: f, v, vc, u
rsize 100 <-- logical rec len of MUSIC file (omit if recfm u)
space 500 <-- initial file allocation, in K
get data1.txt
These are local settings (not sent to the server).
The specified values are not used when in MUSIC-to-MUSIC mode.
For a PUT command to a MUSIC server, you use the commands
QUOTE RECFM, QUOTE RSIZE, and QUOTE ALLO to specify the file
attributes for the files created by the MUSIC FTPD server.
Note that the operand of QUOTE ALLO is a file size in bytes,
while the operand of SPACE is a file size in K.
In the MUSIC client, "quote" can be omitted for ALLO.
ALLO may be needed if the file is large. Example: allo 5000000
Transferring recfm U MUSIC files:
When using MUSIC's FTP client to transfer a recfm U file (e.g.
the result of encrypting a MUSIC file by the CRYPT3 utility) to
a non-MUSIC FTP server, use the command TYPE BINARY. The binary
data sent is the n*512 bytes (n = # of 512-byte data blocks in
the MUSIC file) from the 512-byte blocks of the MUSIC file.
When GET'ing the file back to MUSIC, use the commands TYPE BINARY
and RECFM U. If the file is large, also use the command SPACE m,
where m is an estimate of the size of the file in K. The SPACE
command is entered when using the MUSIC FTP client.
MUSIC's FTP client can operate in either of two user interface
modes: line mode or full-screen (3270) mode. You may find line mode
easier in some cases. To start FTP in line mode, use the option -l
on the command, for example: ftp mysite.mcgill.ca -l
When in the client, you can switch modes by the commands line
and fs.
You can specify a TCP/IP server port to connect to by using the -p
option on the command, for example: ftp mysite.mcgill.ca -l -p1021
The default port number is 21. In the full-screen interface,
you must start FTP with the -p option in order to use a non-21
server port number, for example: ftp -p200
Most of the normal FTP client commands are supported by MUSIC/SP's
FTP client: open, close, user, pass, cd, dir, pwd, get, put, mget,
mput, dele, rename, etc. To exit, use any of the commands exit,
quit, end, bye. Case is not significant in the first word of the
command. For more details on MUSIC FTP client commands and
command-line options and parameters, type help ftp in *Go, or
type lochelp while in FTP.
For quick transfer of a file, try the *Go-mode command QFTP (Quick
FTP). It is a Rexx front-end to the FTP client. Press F1 for help.
The QFTP file is public and you can look at it. Similar Rexx
programs can be written to automate FTP connections and transfers.
Nonstandard commands supported by MUSIC's FTP client:
HELP Displays a list of the FTP client commands. You can
view this list by: /view ftpshorthelp.txt
LOCHELP Invokes MUSIC's Help Facility for the FTP client.
The Help Facility has more info on FTP commands.
REMHELP Sends a HELP command to the server, which may cause it
to display help info about the FTP server.
LOCSTAT Displays the current state of the FTP client, including
settings for various options. Abbreviation: ?
PASSIVE ON/OFF Specifes whether the client should attempt to use
Passive Mode for directory listings and file transfers.
Passive Mode sends a PASV command to the server, which
(if the server supports it) causes the server to listen
on a particular port number, and the client connects to
that port for the next data connection. The client sends
a new PASV command for each directory listing or file
transfer. The server chooses the port number, and tells it
to the client in the 227 response message to the PASV
command. "Passive" refers to the fact that the server
is passive (listens for a data connection from the client).
The opposite of Passive Mode is "normal" or "port" mode,
in which the roles are reversed and the client is passive.
When Passive Mode is ON, which is the default, the client
sends a PASV command, and Passive Mode is used if supported
by the server. If the server rejects the PASV command,
the client turns off its Passive Mode option, and uses
normal mode for subsequent data connections. Most servers
support Passive Mode, and it is usually better for cases
where the client machine is behind a router or firewall,
especially when NAT (Network Address Translation) is used.
Use of Passive Mode can also be controlled by the options
-pasv and -nopasv on the command that starts FTP.
The current setting of Passive Mode (on or off) can be
displayed by a PASSIVE command with no operand, or by
the ? command (LOCSTAT).
RECFM xx Record format (U, F, FC, V, or VC) to be used when the
client creates a local MUSIC file (for the GET command).
LRECL n
RSIZE n Logical record length to be used when the client creates
a local MUSIC file. Does not apply to RECFM U, V, or VC
files.
SPACE n Initial space allocation, in K, when the client creates
a local MUSIC file. Note: Any unused space is freed when
the transfer ends.
SECSP n Secondary space allocation, in K, when the client creates
a local MUSIC file. A negative value -n means n% of the
current space. For example, SECSP -150 means a file with
current size 100K will grow to 100K+150K=250K the next
time it needs more space.
ALLO n Specifies the space allocation (as number of bytes) to be
used by the server when it creates a file (PUT). This may
be needed for a MUSIC/SP server when the file is large,
to avoid exceeding the limit on the number of extents.
Example: allo 5000000
The value only needs to be approximate, since additional
space is allocated if needed and unused space is released
when the new file is closed. This command is not needed
in MUSIC-to-MUSIC (BINARY BLOCK ON/MUS512) mode.
QUOTE RECFM xx Specifies the MUSIC record format to be used by the
server when it creates a file (PUT).
QUOTE LRECL n Specifies the MUSIC logical record length to be used
by the server when it creates a file (PUT).
DELIM xxx Specifies the type of record ending delimiter the FTP
client looks for when receiving a text file.
xxx is one of:
ANYDELIM Any of CRLF, LF, CR, NL (NL is for EBCDIC only).
CRLFONLY
CRONLY
LFONLY
NLONLY Applies only to EBCDIC mode. (NL=hex 15)
The default is usually ANYDELIM.
DISPLAY filename Same as a GET command, except the file's data
DISP is displayed instead of written to a MUSIC file.
This is useful for displaying small text files.
For larger text files, use GET xxx then
/VIEW xxx
LINE or NOFS Change to line mode for the user interface.
SCREEN or FS Change to full-screen mode for the user interface.
TRACE Toggle trace mode (for debugging).
GET and MGET commands: Optional parameter -repl or -r can be used
to replace the target MUSIC file if it already exists.
/xxxx Executes the MUSIC command xxxx on the local system.
Examples: /attrib filename
/purge filename
/edit filename
/view filename
/lib abc*.txt x
/dir abc*.txt
Note: Since multi-session is used to execute the MUSIC
command, commands such as /cd (change directory) are not
effective, since they are not done on the current session.
Use the command LCD (change local directory) instead.
5. Using the FTP Client Program Provided by Windows 95/98/NT/2000/XP
--------------------------------------------------------------------
Windows includes an FTP client program, FTP.EXE in the WINDOWS
directory. It, along with other utilities such as NETSTAT, PING,
and TELNET, are installed automatically when you install the TCP/IP
protocol. These utilities are described in the book "Inside
Windows 95" by Jim Boyce et al (New Riders Publishing, 1995, ISBN
1-56205-375-2). Another good description is in the book "Windows 95
in a nutshell" by Tim O'Reilly (O'Reilly, 1998, ISBN 1565923162),
McGill Engineering Library call # QA76.76 W56. Windows Help also has
a good description of the commands and options.
To start the FTP client, create a DOS session and enter the command
"ftp" or "ftp servername" at the DOS prompt. To display help info
for the command, enter "ftp -?". The FTP program is interactive;
you enter commands (get, put, etc.) to it at the "ftp>" prompt.
To see what commands are available, type "help". To get help on a
particular command xxx, type "help xxx". To close the FTP connection
and exit from the program, enter "quit" or "bye".
Notes:
- To specify a non-standard FTP port number to connect to on the
server, do the following. Do not specify the server name on
the FTP command. At the ftp> prompt, enter:
open servername n
where n is the port number. Example: open vm1.mcgill.ca 1021
The default port number is 21 for FTP. When more than one MUSIC
system is running on the same computer under VM, each may require
a different FTP port number. For example, McGill University ran
two production MUSIC systems called MUSICA and MUSICB, under the
VM/ESA system whose host name is vm1.mcgill.ca. The FTP server
port numbers were 1021 for MUSICA and 1022 for MUSICB.
- To send a specific server command to the FTP server you are
connected to, enter "quote xxx", where xxx is the server command.
The command "literal xxx" is equivalent.
Example: quote autotype off
quote rsize 80
quote syst
- To get help information about the FTP server you are connected
to, enter "remotehelp" or "quote help". You can also use the
command "quote syst" to get server system information.
- To see the commands the FTP client sends to the server, enter
the command "debug".
- To get a file from the server: get serverfile [localfile]
- To send a file to the server: put localfile [serverfile]
- To get or send multiple files by a single command:
mget filepattern or mput filepattern
where filepattern is a file name pattern, containing "wild"
characters * and/or ?. * matches any 0 or more chars, and ?
matches any single char. For example, mget proj*.txt gets
all files (in the current remote directory) whose names start
with proj and end with .txt . By default, you will be prompted
before each file is transferred; reply y to transfer it, or n
to skip it. You can turn off this prompting by the prompt
command. The prompt command toggles prompting on/off.
- To set file transfer type to ascii, enter "ascii" or "type ascii"
before the transfer. NOTE: You may also have to enter
"quote autotype off" to turn off MUSIC's AUTOTYPE mode.
See the AUTOTYPE server command above.
- To set file transfer type to binary, enter "binary" or "type binary"
or "type i" before the transfer. NOTE: You may also have to enter
"quote autotype off" to turn off MUSIC's AUTOTYPE mode.
See the AUTOTYPE server command above.
- To change to another directory on the server machine: "cd dir".
- To change to another local (DOS) directory, enter "lcd dir".
- A problem is that long directory listings scroll off the screen
before you can read them. In that case, use the command
"mdir dir localfile" (listing with file attributes) or
"mls dir localfile" (listing of file names only), which stores
the directory listing into the specified local file. Then use
another DOS session under Windows to edit or view the listing file.
Examples: mdir * temp1
mls ab* temp2
- To read a file that you have downloaded from the server (by a
get command), start another DOS session under Windows and use
it to edit or view the file. Your FTP session continues, and
you can switch back and forth between the DOS windows.
- FTP.EXE can be automated by setting up a script file, which
contains the commands and input for the program. For example,
if the script file is auto1.scr, you would execute it by the
DOS command "ftp -s:auto1.scr". Here is a sample script file:
open mysite.mcgill.ca
jsmith
mypasswd
quote syst
get doc1.txt
bye
6. Using the WS_FTP Client
--------------------------
The FTP client program, WS_FTP, available in various versions from
www.ipswitch.com, is a popular and excellent product. It works well
with MUSIC's FTP server, and uses an intuitive graphical user
interface. A freeware/shareware version of WS_FTP is available
for academic/personal use.
To allow for MUSIC servers that may be set up to give directory
listings in the standard Unix format (instead of the native MUSIC
format), specify the Host Type as "Automatic detect" (not "Music"),
in WS_FTP's connection Session Properties screen.
To send a specific command xxx to the server, right-click anywhere
in the WS_FTP window, select "FTP Commands", then "QUOTE", then
enter the command "quote xxx" (without the " quotation marks).
To transfer multiple files in a single operation, you can select
multiple files, via the mouse, in either the local or remote
list of file names. To select several files: click on the first
one, then Ctrl+click (i.e. hold down the Ctrl key while you click
with the mouse) on other names to be added. To select a set of
consecutive files: click on the first one, then Shift+click on the
last one. All selected files are highlighted.
7. Using MUSIC/SP as a Repository for PC Files
----------------------------------------------
One example of using FTP is to store PC files (from DOS or Windows)
on MUSIC/SP. This may be for backup or distribution purposes, for
example. You run an FTP client program on the PC, and enter PUT or
MPUT commands to send PC files to MUSIC. At a later time, you use
GET or MGET commands to return the files to the PC.
It is very important to make sure the correct transfer type (text or
binary) is used for each file transfer. You should use the command
"type binary" (for binary files) or "type ascii" (for text files)
to tell the FTP client the type of file, before using the GET or PUT
commands. Also, if the MUSIC file name extension is not a common
one (such as .TXT, .HTML, .HTM, .EXE, .GIF, .ARC), and the file is
binary, you should turn off the MUSIC FTP server's AUTOTYPE feature
by the command "quote autotype off". See the AUTOTYPE command above.
When sending the PC files to MUSIC, it is best to tell MUSIC to use
record format VC (variable-length compressed) in all cases. This is
done by entering the command "quote recfm vc" in the FTP client,
before using the PUT or MPUT commands. VC is the default for text
files, but not for binary files. For text files, VC ensures that
long records are not truncated. For binary files, VC preserves the
exact size of the file, since the last record on MUSIC can be
shorter than the others. See the description of the RECFM and RSIZE
commands above. "quote recfm v" can also be used, if data compression
is not wanted on the MUSIC side. In the case of a binary file,
"quote lrecl n" should also be specified, where n is (for example)
8192. This means that records in the MUSIC file will all be n bytes
long (before compression, if any), except for the last record, which
may be shorter than n. The default is n=512, but n=8192 is more
efficient in most cases. n must not exceed 20480 (or whatever the
BS parameter is in the server, in file $tcp:ftpd).
Here is an example, using the FTP.EXE client of Windows 95:
(1) Upload files to MUSIC:
quote autotype off
quote recfm vc
quote allo 500000 <-- initial file allocation, in bytes
(may be needed if the file is large)
type ascii
put file1.txt
put another.txt
put myfile.htm
...
type binary
quote recfm v
quote lrecl 8192
put prog1.exe
put another.exe
put myfile.gif
...
(2) Download the files back to the PC:
quote autotype off
type ascii
get file1.txt
get another.txt
get myfile.htm
...
type binary
get prog1.exe
get another.exe
get myfile.gif
...
8. Using the PC as a Repository for MUSIC/SP Files
--------------------------------------------------
Another example of using FTP is to store MUSIC files on the PC, as
DOS or Windows files. This may be for backup or distribution
purposes, for example. You run an FTP client program on the PC, and
enter GET or MGET commands to transfer MUSIC files to the PC. At a
later time, you may use PUT or MPUT commands to transfer the files
(or some of them) back to MUSIC.
Storage of MUSIC archive files on a removable PC disk, such as a CD
or 100 MB ZIP disk, is a practical alternative to mainframe tape,
when off-site backup is required. This works best when there is
a high-speed network connection between the PC and the MUSIC mainframe.
It is important to ensure that, when the file is transferred back to
MUSIC, it is an exact duplicate of the original MUSIC file, and has
the same attributes such as record format and record size. This is
especially important for binary files. It is also important to make
sure the correct transfer type (text or binary) is used for each
file transfer. MUSIC load modules (.LMOD), object modules (.OBJ)
and archive files (.ARC) are binary files. It is best to use the
command "quote autotype off", to make sure MUSIC's FTP server
honours the transfer type you specify; see the AUTOTYPE command above.
For text files, you normally do not have to worry about specifying
the record format or record length. Files will have record format VC
(variable-length compressed) when they are returned to MUSIC.
However, if you want a returned file to have a specific format, such
as FC and record length 80, you can use the commands "quote recfm fc"
and "quote rsize 80" before the PUT or MPUT command. See the
descriptions of the RECFM and RSIZE commands above. For fixed-length
(record format F or FC) on MUSIC, be sure you specify a large enough
record length, to avoid truncating records.
Binary files are more problematic when it comes to preserving the
original record format and record lengths. For record formats F,
FC, and U, take a note of the original format and record length for
each file, and specify those same values when returning the binary
file to MUSIC. Use the commands "quote recfm x" and "quote rsize
n", where x is F, FC, or U and n is the record length. You can use
the ATTRIB or LIB or DIR command on MUSIC to see the record formats
and lengths (use the X or F options on the LIB and DIR commands).
For example, most archive (.ARC) files on MUSIC are FC/80, so you
you would specify "quote recfm fc" and "quote rsize 80" when
uploading the file back to MUSIC.
For binary files with record formats V or VC, the records may have
different lengths within the same file, and this detailed record
length information is not retained when the file is transferred to
the PC. One solution for this problem is to archive the MUSIC file
or files to a single .ARC file, using the FILARC or MFARC2 utility.
Make the .ARC file record format FC, record length 80. Then transfer
the binary .ARC file to the PC. When sending the .ARC file back to
MUSIC, use "quote recfm fc" and "quote rsize 80". Finally, use the
FILRST or MFREST utility to de-archive the original file(s) from the
archive file. This method also has the advantage of preserving
other file attributes such as the tag and access control. This
method can also be used for any types of MUSIC files, and is useful
when dealing with a large number of MUSIC files.
Here is an example:
(1) Run the following FILARC job on MUSIC to archive the files
into the single file GROUP.ARC. To run this job, use the
Editor to type the lines into a file, then execute the file
at the terminal. Note that wild characters * and ? can be
used in the file name records, to specify groups of files.
Here, PROG1.* means all file names starting with "PROG1.".
/FILE 1 NAME(GROUP.ARC) NEW RECFM(FC) LRECL(80) SPACE(200)
/INCLUDE *COM:FILARC
OUTPUT=1
ANYFILE1.TXT
ANYFILE2.OBJ
ANYFILE3.LMOD
ANYFILE4.XX
PROG1.*
Note: Since FTP deals only with MUSIC Save Library files, and
such files are limited to a maximum size of about 57 MBytes,
it may be necessary to run more than one FILARC job and
create several .ARC files.
(2) Using the FTP.EXE client of Windows 95, transfer the binary
archive file to the PC. (Note that the AUTOTYPE OFF command
is not really required here and in (3) below, since the
extension .ARC is normally defined as a binary file in
$tcp:ftpd.filetypes, but it is best to use it anyway.)
type binary
quote autotype off
get group.arc
As an alternative, see also (2a) below.
(3) At some later time, transfer the archive file back to MUSIC.
Here, it is important to specify the MUSIC record format (FC)
and record length (80) matching the original MUSIC archive
file. Note that the PUT command automatically replaces the
target MUSIC file if it still exists.
type binary
quote autotype off
quote recfm fc
quote rsize 80
quote allo n <-- allocation size in bytes (may be needed
if file is large)
put group.arc
As an alternative, see also (3a) below.
(4) Restore the files from the archive file, by running the
following FILRST job on MUSIC. JSMITH is your file ownership
id (your userid without the subcode part, if any) on MUSIC.
The asterisk (*) indicates all files in the archive.
/FILE 1 NAME(GROUP.ARC)
/INCLUDE *COM:FILRST
NAME='JSMITH:*',TO='JSMITH:*',REPL=T
As a variation of the above procedure, the "binary block on" command
of MUSIC's FTP server can be used. This command can be issued via
the QUOTE command from the PC FTP client. It causes the transfer to
be done in MUSIC 512-byte file blocks, and file info (attributes
such as RECFM, LRECL, EOFPT, etc.) is also transmitted. This gives
a smaller file on the PC, since RECFM(FC) compression is not lost.
The first 94 bytes of the PC file contain file attribute info;
the remaining bytes of the PC file contain the data (unconverted)
from the 512-byte blocks of the MUSIC file (last block may be short).
The archive file on MUSIC is FC/80. On transfer back to MUSIC, the
file size and attributes are restored exactly. Using this method,
the replacements for steps (2) and (3) above are:
(2a) type binary
quote autotype off
quote binary block on
get group.arc
(3a) type binary
quote autotype off
quote binary block on
put group.arc
In (2a) and (3a), the option "mus512" could be used instead of "on".
"quote binary block mus512" uses a 512-byte file info header (instead
of 94 bytes), and the header includes extra info such as the original
MUSIC/SP file name, and date and time info.
Note: Once the file is on the PC, it is also possible to apply a
compression utility such as WinZip to it and/or dump it to some
other medium such as a PC tape cartridge or removable disk or
writable CD.