Help Info for MUSIC/SP FTP Server File: $tcp:ftphelp.txt ================================= Author: Dave Edwards Updated: Aug 6, 2007 (c) Copyright Dave Edwards 2000-2007. All rights reserved. Contents: 1. Introduction 2. General Notes on the MUSIC/SP File System 3. Commands Supported by MUSIC/SP's FTP Server 4. Using MUSIC/SP's FTP Client Program 5. Using the FTP Client Program Provided by Windows 95/98/NT/2000/XP 6. Using the WS_FTP Client 7. Using MUSIC/SP as a Repository for PC Files 8. Using the PC as a Repository for MUSIC/SP Files See also: $tcp:ftpd_info.txt 1. Introduction --------------- This file has information for users using an FTP (File Transfer Protocol) client program to connect to the FTP Server (FTPD) running on a MUSIC/SP system. Examples of FTP client programs are (1) the FTP command of MUSIC/SP; (2) the ftp.exe program provided with Windows 95/98/NT/2000/XP (enter the command "ftp xxx" from a DOS session, where xxx is the server host name or numeric IP address); (3) the excellent WS_FTP product available from www.ipswitch.com; (4) a Web browser, using a URL of the form ftp://site; (5) various FTP clients available on Unix and Linux systems. The FTP Protocol is defined in RFC 959 (and its later versions). This file can be obtained by the command "get ftphelp.txt" after connecting to a MUSIC/SP FTP server and logging in. FTPHELP.TXT is a special public file that can be got while in any directory. MUSIC/SP stands for Multi-User System for Interactive Computing / System Product, an operating system originally developed by McGill University. For info on it, visit the Web sites: http://canpub.com/teammpg/de/ http://canpub.com/teammpg/de/sim390 2. General Notes on the MUSIC/SP File System -------------------------------------------- MUSIC/SP is an operating system that runs on IBM S/370-S/390 mainframe computers, usually as a virtual machine under VM (VM/SP or VM/ESA). TCP/IP services are provided to MUSIC/SP by the TCPIP virtual machine running under VM. MUSIC/SP can also run under the Sim390 and Hercules mainframe emulators on a PC, in which case TCP/IP services are provided by the emulator and the underlying operating system (such as Windows). Currently, MUSIC/SP under Hercules cannot use TCP/IP; TCP/IP is ok under Sim390. The character set encoding for text is EBCDIC, not ASCII. Text files (i.e. non-binary files) are translated by the FTP server from EBCDIC to ASCII before being sent to the FTP client, and from ASCII to EBCDIC when received from the client. See the TYPE, AUTOTYPE, and BINARY BLOCK commands below. Normally, in a text file in Ascii form, record ends are indicated by CR/LF (the 2 Ascii control characters CR = hex 0D and LF = hex 0A), or a single LF character. The MUSIC/SP file system has a directory structure similar to that of DOS and Unix. The separator character between directory names is a backslash (\), but the FTP client can use either a slash (/) or a backslash (\). Directory names can be up to 17 characters. File names within a directory can be up to 17 characters. The total name, including directory names on the front, can be up to 50 characters. Upper/lower case is not significant in file names. Letters (A to Z), digits (0 to 9), and some special characters (including $ # @ _ + - . % & !) can be used, but the first character of each part must not be a digit or + - . % & !. An example of a full MUSIC file name is abcde:test\source\prog1.s . The prefix "abcde:" indicates that the file's owner is the MUSIC userid ABCDE. Often the owner prefix can be omitted. MUSIC files have a record format that is one of F (fixed-length records), FC (fixed-length, compressed), V (variable-length records), VC (variable-length, compressed), or U (undefined). A U format file is treated as a sequence of 512-byte blocks. F and FC format files must have a logical record length (RSIZE or LRECL) of 1 to 32767 bytes specified when they are created. The logical record length of a V or VC file is the length of its longest record; on creation, the logical record length of a V or VC file is ignored, and can be specified as 0 (but, for a V or VC binary file created by MUSIC's FTP client, the logical record length determines the length of all the records except the last one, which may be shorter). The maximum logical record length of a MUSIC file is 32767 (=32K-1), but, for compatibility with other systems, record lengths > 32760 should be avoided. Some versions of MUSIC FTP client or server have a lower limit on the record length supported for exchange of files (in non-binary mode) with non-MUSIC systems; record lengths up to 16384 (=16K) are usually safe. The maximum size of a file is about 57 megabytes (or less, if insufficient unfragmented free disk space is available). Logical records are determined by internal control bytes and counts; a logical record can contain any of the 256 possible byte values; logical records are NOT separated by CR or CR/LF characters as in DOS and Unix. However, when MUSIC's FTP server sends a text file to an FTP client, it inserts CR/LF at the end of each logical record sent to the client. 3. Commands Supported by MUSIC/SP's FTP Server ---------------------------------------------- These are the commands accepted by the FTP server (FTPD). They are generated by the FTP client program, in response to the commands or operations done by the user (GET, PUT, etc.) Most FTP clients have a mechanism, such as the QUOTE command, for sending any desired command to the server. This is useful when the user needs to send server-specific commands, such as ones for setting special server options. The FTP client user enters the command "quote xxx", where xxx is any command supported by the server, and the client program then sends the command "xxx" to the server to be executed, and displays the response from the server. Examples: "quote help", "quote autotype off", "quote rsize 100". Some FTP clients use the SITE command instead of QUOTE. When using a non-MUSIC FTP client, the QUOTE or SITE command must be used for MUSIC FTP server commands which are non-standard, such as AUTOTYPE, BINARY BLOCK, DIRFORMAT, IF, RECFM, LRECL, RSIZE, and may also be needed for other commands such as SYST, ALLO. For example, to send command AUTOTYPE OFF to the server, the user types "quote autotype off" in the non-MUSIC FTP client. This may also be needed even in the MUSIC FTP client, since it does not recognize all of the server commands. Behaviour of the MUSIC FTP server can be customized by the system administrator, by setting various parameters in the file $TCP:FTPD . The port number (normally 21) on which the server listens for connections can be specified in file $TCP:INETD.PORTS . In the command descriptions below, aliases for command names and option keywords are stacked vertically. Alternatives (e.g. ON or OFF) for operand values are also shown stacked vertically. Words shown in upper case should be entered as given. Variable parts of the command are shown in lower case; "filename" stands for a server file name; "dir" stands for a server directory name. Upper/lower case is not significant in the commands (except in the password on the PASS command if the userid has the PWCASENS Profile option set). Optional items are shown in square brackets [ ]; do not enter the square brackets. Commands accepted by MUSIC/SP's FTP server: USER userid Specifies the MUSIC/SP userid for FTP log-in. This command is not valid AFTER a successful log-in. NOTE: Parameters in file $TCP:FTPD can be set up to allow anonymous (unauthenticated) access, in which case a special userid such as anonymous or ftp is accepted, which is not a real MUSIC/SP userid. PASS password Specifies the password for the userid. It must immediately follow the USER command, if a password is required. Some special userids, such as anonymous of ftp (if allowed), do not require a password. NOTES: - If your MUSIC userid requires an extra password for sign-on, enter xxx!!yyy as the password field to FTP, where xxx is your normal password and yyy is the extra password. If you do not know the extra password number required, just enter xxx, and you will get a message giving the number. Then log in again. - MUSIC allows a challenge/response protocol for FTP login, which avoids transmitting the password in clear text over the network. The 331 message for the USER command includes a random challenge string. The user calculates the MD5 digest (as 32 printable hex characters) of the string xxxppp where xxx is the challenge and ppp is the password. Then the user sends the 32-char MD5 digest instead of the password. The Windows command-line program md5pw.exe can be used to calculate the MD5 digest. It can be downloaded from web sites mentioned in section 1. The FTP server accepts both the password and the MD5 digest string, therefore use of the challenge is optional, but is recommended for better security. - In the Windows ftp.exe client, the maximum length of the password field on the PASS command is 32 characters. The MUSIC FTP server allows longer password fields. QUIT Closes the TCP/IP connection and exits from the FTP server program. HELP Gives help information about the FTP server. It displays a short message, referring to this file (FTPHELP.TXT) for more information. SYST Identifies the server system as MUSIC/SP, and also may give the virtual machine name ("system name") if any. Notes: - The QUOTE or SITE command may be needed in some non-MUSIC FTP clients. For example: quote syst - The first word of the text response is supposed to indicate the general type of server system. For a MUSIC/SP system, the first word may depend on the current setting of the DIRFORMAT command. If DIRFORMAT UNIX is in effect, the first word is normally UNIX. Experiment to find the actual output for your MUSIC server. CWD dir Changes to a new current working directory on the server. dir can be / or \ to go to the root, or .. to go up one level. CDUP Changes to the parent directory, i.e. one level higher, on the server. It is equivalent to "cwd ..". PWD XPWD Prints (i.e. displays) the name of the current working directory on the server. PORT a1,a2,a3,a4,p1,p2 Specifies the numeric IP address and port number to be used for a subsequent data transfer. This command is generated automatically by the FTP client, as needed. This command is not used in Passive Mode (see the PASV command). The client listens on the specified port, and then the server connects to it, thus establishing the TCP data connection for the transfer. After the file (or directory listing data) has been transferred, the data connection is closed. The IP address on the PORT command is normally that of the FTP client (and it must be, unless the OK3RDP=T parameter is specified in file $TCP:FTPD for the server). The port number is p1*256+p2. Example: for client IP addr 130.12.34.156 and port 4321, the command is: port 130,12,34,156,16,225 PASV This command tells the server to use Passive Mode for the next file transfer (or directory listing). In Passive Mode, the server creates a socket and listens on it, and tells the client the port via the 227 response to the PASV cmd. Then the client connects to the port, which establishes the data connection for the transfer. The form of the response to the PASV command is: 227 Entering Passive Mode (a1,a2,a3,a4,p1,p2) The IP address is that of the server (same format as in the PORT command), and the server port number is p1*256+p2. The alternative to Passive Mode is "active" or "normal" mode, in which the client creates a socket and listens on it, tells the server the port # via the PORT cmd, and the server connects to the client's port, in order to establish the data connection. Most modern FTP clients try the PASV cmd, and use Passive Mode if possible. Passive Mode is friendlier to firewalls and routers, especially when NAT (Network Address Translation) is done for clients behind the router or firewall. Passive Mode can be disabled in the server by the parameter PASVOK=F in file $TCP:FTPD . Some earlier versions of MUSIC FTPD do not support Passive Mode. The Windows client ftp.exe does not support Passive Mode. Recent versions of the MUSIC/SP FTP client support Passive Mode, via the PASSIVE ON/OFF client command. Most web browsers support Passive mode. In MS Internet Explorer, it is turned on by Tools --> Internet Options --> Advanced --> "Use Passive FTP". LIST [dir] This command is generated by the DIR client command. The server sends a list of the file names in the current (or specified) directory, with attribute information for each file. The first line is a header line with column titles. Attribute information includes record length, record format, file size, last read date, last write date, userid of last writer, access control attributes, and number of records. Subdirectory names are indicated by . Note: If DIRFRM=1 is specified in the configuration of the MUSIC/SP FTP server (in file $tcp:ftpd), the server gives the directory listing in the standard Unix format, rather than in the native MUSIC format. This can also be controlled by the DIRFORMAT command. DIRFORMAT MUSIC UNIX Specifies the desired format for the output of the LIST (directory list) command. MUSIC (or MUS) specifies MUSIC native format (as in the MUSIC command LIBR * X). UNIX specifies the format as in the Unix "ls -l" command. The default is as specified by the DIRFRM=n parameter in server file $tcp:ftpd; DIRFRM=0 uses MUS format, DIRFRM=1 uses UNIX format. (The default is usually UNIX format.) Abbreviations: DIRFORM MUS Note: Since this is a non-standard server command, the QUOTE or SITE command is normally needed in a non-MUSIC client. For example: quote dirform mus NLST [dir] This command is generated by the LS client command. Same as the LIST command, but only the file names are displayed, without the header line and attributes. TYPE A[SCII] I[MAGE] E[BCDIC] Specifies the type of data to be transferred: (1) ASCII means text (printable) characters. The server translates from EBCDIC encoding to ASCII before sending data, and translates from ASCII to EBCDIC when receiving data. Client logical records are assumed to be separated by special control characters such as CR/LF (unless MODE BLOCK is in effect - see the MODE command below). When the server sends a text file to the client, it removes trailing blanks from each logical record (if the MUSIC file is record format F or FC) and sends CR/LF after each record. (2) IMAGE means binary (non-text) data, where each byte may contain any of the possible 256 bit combinations. No character set translation is done. Client logical records are not assumed to be separated by control characters such as CR/LF. When the server sends a MUSIC file to the client, each logical record is sent as is, with no translation, and with no CR/LF added; in the case of a record format U file, each 512-byte block of the file is sent as is. (3) EBCDIC means text (like ASCII), but no translation between ASCII and EBCDIC character sets is done, since the data is assumed to be in EBCDIC on both the server and client systems. See also the AUTOTYPE, MODE, and BINARY BLOCK commands below. AUTOTYPE [ON ] [OFF] Specifies whether the AUTOTYPE option should be used or not in the FTP server program. When AUTOTYPE is ON, the server determines the TYPE of data transfer (see the TYPE command above) from the file name extension, regardless of the current setting of the TYPE command. The extension is the ".xxx" part at the end of the MUSIC/SP file name. Some extensions (.BIN, .EXE, .COM, .GIF, .ARC, .OBJ, .LMOD, .DOC, .XLS, .PPT, .MDB, .JPEG, .JPG, .ZIP, etc.) imply binary data (TYPE IMAGE), while others (.TXT, .TEXT, .HTML, .HTM, .S, .C, .BAT, etc.) imply text data (TYPE ASCII). The extensions are defined in file $TCP:FTPD.FILETYPES . An unknown extension is usually assumed to be text data. However, the AUTOTYPE option does not affect the TYPE when TYPE EBCDIC is in effect, or when MUSIC-to-MUSIC mode is in effect (see the BINARY BLOCK command). When AUTOTYPE is OFF, the TYPE command setting is honoured and the file extension is ignored. AUTOTYPE is normally ON by default (but the MUSIC/SP system administrator may change that by the AUTOTY=F parameter in $TCP:FTPD). If neither ON nor OFF is specified on the AUTOTYPE command, ON is assumed. Note: Since this is a non-standard server command, the QUOTE or SITE command is normally needed in a non-MUSIC client. For example: quote autotype off Examples: (1) If you want to transfer a binary file called MYFILE.XYZ, you should enter the commands "type image" and "quote autotype off" in the FTP client, before using the GET or PUT command. Otherwise the MUSIC/SP FTP server may assume the file is a text file (since the extension .XYZ is unknown), even though you used "type image". (2) If you want to transfer a text file called MYNOTES.DOC, you should enter the commands "type ascii" and "quote autotype off" in the FTP client, before using the GET or PUT command. Otherwise the MUSIC/SP FTP server may assume the file is a binary file (MS Word document), even though you used "type ascii". For this reason, it is best not to use .DOC when naming a text file. MODE S[TREAM] B[LOCK] This defines the "transmission mode" for transferring text (TYPE ASCII or TYPE EBCDIC) data. In STREAM mode, which is the default, logical records on the client system are assumed to separated by special control characters, such as CR/LF. In BLOCK mode, each logical record is transmitted with preceding header bytes, which contain the length of the record. The MODE command does not affect binary transfers (TYPE IMAGE). BLOCK mode is rarely used. BINARY BLOCK [ON ] BIN [OFF] [MUS512] This command sets binary data transfer (TYPE IMAGE), regardless of whether ON or OFF is specified. In addition, if ON is specified (or if no option is specified after BLOCK), a special block transfer mode is set. In that mode, data is transferred in 512-byte blocks, exactly matching the internal 512-byte block structure of the MUSIC/SP file. When both the FTP client and server are MUSIC systems, this allows an efficient, exact file transfer in all cases, and also preserves the file's attributes (record format, record length, tag, space values, etc.) When the MUSIC client detects that the server is also a MUSIC system, BINARY BLOCK ON is automatically set for all file transfers; this is called "MUSIC-to-MUSIC" mode or "MUSIC File Copy" mode; you do not have to worry about using the correct TYPE, MODE, RECFM, RSIZE, or ALLO commands in that case. To turn off MUSIC-to-MUSIC mode, use the command "binary block off", followed by the appropriate TYPE and AUTOTYPE commands. Note: Since this is a non-standard server command, the QUOTE or SITE command is normally needed in a non-MUSIC client. For example: quote binary block on Even in the MUSIC client, the quote command form is needed when the MUS512 option is used. Additional notes on "binary block on" (MUSIC-to-MUSIC) mode: - In this mode, when sending a file to the FTP client, MUSIC's FTPD server first sends 94 bytes of binary file info, containing: MFINFO (INFIN) block (20 bytes), EOFPT block (10 bytes), file TAG (64 bytes). Refer to the "File System" chapter in the MUSIC/SP Admin Ref. manual. Then MUSIC's FTPD server sends the 512-byte data blocks of the file, unchanged, as they appear on MUSIC's disk. The last 512-byte block may be short (less than 512 bytes) in some cases. Source file reference: $TCP:FTPD.PUTBIN.S . - In this mode, when receiving a file from the FTP client, MUSIC's FTPD server interprets the first 94 bytes as the file info header, and uses that info to set record format, record length, file size, etc. when creating the target MUSIC file. - The client system does not have to be a MUSIC system in order for MUSIC's FTPD server to use this mode. But then the 94-byte file info header appears as extra data at the start of the file on the FTP client side. In this case, the client sets MUSIC-to-MUSIC mode in the MUSIC server by the command "quote binary block on". - This mode can also be used with a MUSIC client and a non-MUSIC server. The user sets MUSIC-to-MUSIC mode in the client by the command "binary block on" (without using "quote"). Then the client sends the extra 94-byte header on all binary uploads, and expects binary download data to start with the header. NOTE: MUSIC's FTP client supports the BINARY BLOCK ON command, but not the BINARY BLOCK MUS512 command. [The 12MAR2003 update to MUSIC's FTP client is required in order to use the BINARY BLOCK ON command with a non-MUSIC server.] - The option MUS512 is very similar to the ON option, except that the info header is 512 bytes instead of 94 bytes, and the final data block is always a full 512 bytes (not a short block). The format of the 512-byte header info is the same as that used by MUSIC/SP's MUS512DMP and MUS512RST utilities (see $pgm:mus512dmp.s, $pgm:mus512rst.s). It contains extra info, including the full name of the original MUSIC/SP file. The FTP client should use the command "quote binary block mus512" to set this mode in the MUSIC/SP server. The client should also use the command "type binary". In this mode, the MUSIC/SP server sends all files to the client in the format described (512-byte info header followed by 0 or more 512-byte data blocks), and expects files from the client to be in the same format. This mode is useful for storing MUSIC/SP files (of any type) on a PC client machine, as binary files that can later be restored to a MUSIC/SP system. ALLO n Specifies the size n (number of bytes) to be allocated initially when the server creates a MUSIC file (for the STOR or STOU command). This command may be needed for very large files. Example of command entered in the FTP client: "quote allo 2000000". This command is not required for MUSIC-to-MUSIC mode (see BINARY BLOCK). Note: The QUOTE or SITE command may be needed in a non-MUSIC FTP client. For example: quote allo 500000 RECFM F REC FC V VC U Specifies the record format to be used when the server creates a MUSIC file (for the STOR or STOU command). The default record format is VC for text transfers and U for binary transfers. This command is not needed in MUSIC-to-MUSIC mode (see BINARY BLOCK). Note: The FTP client user enters a command such as "recfm fc" or "lrecl 80" to specify the attributes of files to be created by the MUSIC client (i.e. downloads). The user enters a command such as "quote recfm fc" or "quote lrecl 80" to specify the attributes of files to be created by the MUSIC server (i.e. uploads). RSIZE n LRECL Specifies the logical record length (0 or more), in bytes, to be used when the server creates a MUSIC file (for the STOR or STOU command). The default is 0, which causes a record length of 512 or 4096 to be used when a file is transferred to MUSIC. How the record length is used depends on the TYPE of transfer (text or binary) and on the record format (RECFM) in effect: Text, F or FC: The record length should be greater than or equal to the length of the longest logical record to be transferred to MUSIC. Otherwise long records are truncated. Text, V or VC: The record length is ignored. Any size record, up to the MUSIC maximum of 32760, is handled. Binary, U: The record length is ignored. Binary, F, FC, V, or VC: The data is divided into logical records of the specified length. If the total number of bytes transferred is not an exact multiple of the record length, the final record is shorter (V or VC), or is extended with zeros (F or FC). If the record length is 0 (not specified), 512 is used for F or FC, and 4096 is used for V or VC, on a download; on an upload, the default is 512 for F, FC, V, or VC. ("Upload" refers to transferring a file to the server.) See the examples in the sections below. This command is not needed in MUSIC-to-MUSIC mode (see BINARY BLOCK). NOTE: Although the record length of a MUSIC file may be up to 32767, the maximum for files created by the FTP server is the data transfer buffer size, normally 20480 ( = the BS parameter in file $TCP:FTPD). The value n on the command must not exceed BS = 20480. Note: The FTP client user enters a command such as "recfm fc" or "lrecl 80" to specify the attributes of files to be created by the MUSIC client (i.e. downloads). The user enters a command such as "quote recfm fc" or "quote lrecl 80" to specify the attributes of files to be created by the MUSIC server (i.e. uploads). BINARY FIXED n BIN This is equivalent to the commands RECFM F and RSIZE n. NOTE: This command does NOT set the data TYPE (text or binary). BINARY VARIABLE BIN This is equivalent to the commands RECFM VC and RSIZE 0. NOTE: This command does NOT set the data TYPE (text or binary). RETR filename This command is generated by the FTP client when a GET command is entered. It tells the server to send the specified MUSIC file to the client. The current settings from the previous TYPE, AUTOTYPE, MODE, BINARY, RECFM, RSIZE, and ALLO server commands are used for the file transfer. STOR filename This command is generated by the FTP client when a PUT command is entered. It tells the server to receive file data from the client, and store it into a new MUSIC file, whose name is specified. If the MUSIC file already exists, it is deleted. The current settings from the previous TYPE, AUTOTYPE, MODE, BINARY, RECFM, RSIZE, and ALLO server commands are used for the file transfer. STOU filename This command (Store Unique) is the same as STOR, except that the target MUSIC file is not replaced if it already exists. Instead, if the file already exists, a dollar sign character ($) is added at the end of the MUSIC file name and that name is tried. The server continues to add $'s until a name is found that does not already exist. APPE filename (Append) This command is not supported by the server. DELE filename Deletes the specified MUSIC file. RNFR filename1 Specifies a MUSIC file to be renamed. The RNFR and RNTO commands are generated by the FTP client when a RENAME command is entered. RNTO filename2 Specifies the new MUSIC file name for a rename operation. This command must immediately follow RNFR. MKDIR dir MKD MD XMKD Creates a MUSIC subdirectory. RMDIR dir RMD RD XRMD Deletes a MUSIC subdirectory. IF FILE filename EXIT IF NOTFILE filename EXIT This command, which is a nonstandard command not in the FTP protocol, can be used to test some condition on the server and take an action based on that test. Currently, the only condition that can be tested is whether a specified file on the server is present and accessible for downloading (GET) to the client. The only action is EXIT, which terminates the FTP session on the server and closes the control connection. QUIT can be used as an alias for EXIT. The client must use the QUOTE or SITE command to send the IF command to the server. For example, if a client (perhaps in an automated script of commands) wants the connection to be closed (and therefore most subsequent commands in the script to fail as "not logged in") if file myfile.txt is not GETable, it could use the command: quote if notfile myfile.txt exit If the file exists and is GETable, the script continues normally. Notes on how FTP clients can cause a file to be transferred between 2 servers ("3rd-party" file transfer): Most FTP clients are not programmed to support this, but the FTP server commands for doing it are part of the FTP specification (see RFC 959 and its later versions). The QUOTE (or SITE) command in the client can be used to send the required commands to the servers. Suppose server 1 (SRV1) is at IP address s1.s2.s3.s4, and server 2 (SRV2) is at IP address t1.t2.t3.t4. We wish to transfer a file fn1 on SRV1 to file fn2 on SRV2 (assume fn2 does not already exist on SRV2). SRV1 must be able to operate in Passive Mode (i.e. PASVOK=T in $tcp:ftpd on MUSIC). Both servers must allow "3rd-party" transfers (i.e. OK3RDP=T in $tcp:ftpd). We require 2 client sessions (CLN1 and CLN2). The procedure is: - CLN1: Log-in to SRV1. Issue any cmds needed to set the type of file transfer (text or binary). - CLN2: Log-in to SRV2. Issue any cmds needed to set the type of file transfer (text or binary). - CLN1: quote pasv - SRV1: Listens on port p1*256+p2 and sends response 227 Entering Passive Mode (s1,s2,s3,s4,p1,p2) - CLN2: quote port s1,s2,s3,s4,p1,p2 - SRV2: Connects to port p1*256+p2 on SRV1, thus establishing the data connection for the file transfer. - CLN1: quote retr fn1 - CLN2: quote stor fn2 - At this point, the file transfer starts. SRV1 writes fn1 to the data socket, and SRV2 reads from the socket and creates file fn2. The clients should wait a suitable time for the transfer to finish. - Each client, if it does not automatically display a 226 "transfer ended" message from its server, should issue a command such as quote noop or quote syst, which should cause the message to be displayed. (Some clients seem to have trouble with this, and get out-of-synch between commands and responses, since the QUOTE command is being used. The MUSIC client seems to have less trouble than Windows ftp.exe.) 4. Using MUSIC/SP's FTP Client Program -------------------------------------- MUSIC's FTP client program is started by the MUSIC command FTP. For detailed help information on it, type the MUSIC command "/help ftp". When you are in the FTP client, you can use the command help to display a list of the client commands, or the command lochelp to invoke more detailed full-screen help, or the command remhelp (or remotehelp) to request help from the server. When the MUSIC FTP client connects to a MUSIC FTP server, MUSIC-to-MUSIC file transfer mode is automatically set. In this mode, all file characteristics and attributes are preserved when they are transferred. You do not have to worry about using the correct TYPE, MODE, BINARY, RSIZE, RECFM, or ALLO commands. All file types (text and binary) are transferred correctly. You can turn off MUSIC-to-MUSIC mode by the command "binary block off". You can turn it on by the command "binary block on". NOTE: Currently, MUSIC's FTP client can operate in "binary block on" mode, even when talking to a non-MUSIC server. But it cannot operate in "binary block mus512" mode. When you transfer a file to MUSIC (by the GET command), you can precede the GET command by RECFM, RSIZE, and SPACE commands, to specify the file attributes to be used when MUSIC creates the file. (These settings are ignored in MUSIC-to-MUSIC mode.) For example: recfm fc <-- other values: f, v, vc, u rsize 100 <-- logical rec len of MUSIC file (omit if recfm u) space 500 <-- initial file allocation, in K get data1.txt These are local settings (not sent to the server). The specified values are not used when in MUSIC-to-MUSIC mode. For a PUT command to a MUSIC server, you use the commands QUOTE RECFM, QUOTE RSIZE, and QUOTE ALLO to specify the file attributes for the files created by the MUSIC FTPD server. Note that the operand of QUOTE ALLO is a file size in bytes, while the operand of SPACE is a file size in K. In the MUSIC client, "quote" can be omitted for ALLO. ALLO may be needed if the file is large. Example: allo 5000000 Transferring recfm U MUSIC files: When using MUSIC's FTP client to transfer a recfm U file (e.g. the result of encrypting a MUSIC file by the CRYPT3 utility) to a non-MUSIC FTP server, use the command TYPE BINARY. The binary data sent is the n*512 bytes (n = # of 512-byte data blocks in the MUSIC file) from the 512-byte blocks of the MUSIC file. When GET'ing the file back to MUSIC, use the commands TYPE BINARY and RECFM U. If the file is large, also use the command SPACE m, where m is an estimate of the size of the file in K. The SPACE command is entered when using the MUSIC FTP client. MUSIC's FTP client can operate in either of two user interface modes: line mode or full-screen (3270) mode. You may find line mode easier in some cases. To start FTP in line mode, use the option -l on the command, for example: ftp mysite.mcgill.ca -l When in the client, you can switch modes by the commands line and fs. You can specify a TCP/IP server port to connect to by using the -p option on the command, for example: ftp mysite.mcgill.ca -l -p1021 The default port number is 21. In the full-screen interface, you must start FTP with the -p option in order to use a non-21 server port number, for example: ftp -p200 Most of the normal FTP client commands are supported by MUSIC/SP's FTP client: open, close, user, pass, cd, dir, pwd, get, put, mget, mput, dele, rename, etc. To exit, use any of the commands exit, quit, end, bye. Case is not significant in the first word of the command. For more details on MUSIC FTP client commands and command-line options and parameters, type help ftp in *Go, or type lochelp while in FTP. For quick transfer of a file, try the *Go-mode command QFTP (Quick FTP). It is a Rexx front-end to the FTP client. Press F1 for help. The QFTP file is public and you can look at it. Similar Rexx programs can be written to automate FTP connections and transfers. Nonstandard commands supported by MUSIC's FTP client: HELP Displays a list of the FTP client commands. You can view this list by: /view ftpshorthelp.txt LOCHELP Invokes MUSIC's Help Facility for the FTP client. The Help Facility has more info on FTP commands. REMHELP Sends a HELP command to the server, which may cause it to display help info about the FTP server. LOCSTAT Displays the current state of the FTP client, including settings for various options. Abbreviation: ? PASSIVE ON/OFF Specifes whether the client should attempt to use Passive Mode for directory listings and file transfers. Passive Mode sends a PASV command to the server, which (if the server supports it) causes the server to listen on a particular port number, and the client connects to that port for the next data connection. The client sends a new PASV command for each directory listing or file transfer. The server chooses the port number, and tells it to the client in the 227 response message to the PASV command. "Passive" refers to the fact that the server is passive (listens for a data connection from the client). The opposite of Passive Mode is "normal" or "port" mode, in which the roles are reversed and the client is passive. When Passive Mode is ON, which is the default, the client sends a PASV command, and Passive Mode is used if supported by the server. If the server rejects the PASV command, the client turns off its Passive Mode option, and uses normal mode for subsequent data connections. Most servers support Passive Mode, and it is usually better for cases where the client machine is behind a router or firewall, especially when NAT (Network Address Translation) is used. Use of Passive Mode can also be controlled by the options -pasv and -nopasv on the command that starts FTP. The current setting of Passive Mode (on or off) can be displayed by a PASSIVE command with no operand, or by the ? command (LOCSTAT). RECFM xx Record format (U, F, FC, V, or VC) to be used when the client creates a local MUSIC file (for the GET command). LRECL n RSIZE n Logical record length to be used when the client creates a local MUSIC file. Does not apply to RECFM U, V, or VC files. SPACE n Initial space allocation, in K, when the client creates a local MUSIC file. Note: Any unused space is freed when the transfer ends. SECSP n Secondary space allocation, in K, when the client creates a local MUSIC file. A negative value -n means n% of the current space. For example, SECSP -150 means a file with current size 100K will grow to 100K+150K=250K the next time it needs more space. ALLO n Specifies the space allocation (as number of bytes) to be used by the server when it creates a file (PUT). This may be needed for a MUSIC/SP server when the file is large, to avoid exceeding the limit on the number of extents. Example: allo 5000000 The value only needs to be approximate, since additional space is allocated if needed and unused space is released when the new file is closed. This command is not needed in MUSIC-to-MUSIC (BINARY BLOCK ON/MUS512) mode. QUOTE RECFM xx Specifies the MUSIC record format to be used by the server when it creates a file (PUT). QUOTE LRECL n Specifies the MUSIC logical record length to be used by the server when it creates a file (PUT). DELIM xxx Specifies the type of record ending delimiter the FTP client looks for when receiving a text file. xxx is one of: ANYDELIM Any of CRLF, LF, CR, NL (NL is for EBCDIC only). CRLFONLY CRONLY LFONLY NLONLY Applies only to EBCDIC mode. (NL=hex 15) The default is usually ANYDELIM. DISPLAY filename Same as a GET command, except the file's data DISP is displayed instead of written to a MUSIC file. This is useful for displaying small text files. For larger text files, use GET xxx then /VIEW xxx LINE or NOFS Change to line mode for the user interface. SCREEN or FS Change to full-screen mode for the user interface. TRACE Toggle trace mode (for debugging). GET and MGET commands: Optional parameter -repl or -r can be used to replace the target MUSIC file if it already exists. /xxxx Executes the MUSIC command xxxx on the local system. Examples: /attrib filename /purge filename /edit filename /view filename /lib abc*.txt x /dir abc*.txt Note: Since multi-session is used to execute the MUSIC command, commands such as /cd (change directory) are not effective, since they are not done on the current session. Use the command LCD (change local directory) instead. 5. Using the FTP Client Program Provided by Windows 95/98/NT/2000/XP -------------------------------------------------------------------- Windows includes an FTP client program, FTP.EXE in the WINDOWS directory. It, along with other utilities such as NETSTAT, PING, and TELNET, are installed automatically when you install the TCP/IP protocol. These utilities are described in the book "Inside Windows 95" by Jim Boyce et al (New Riders Publishing, 1995, ISBN 1-56205-375-2). Another good description is in the book "Windows 95 in a nutshell" by Tim O'Reilly (O'Reilly, 1998, ISBN 1565923162), McGill Engineering Library call # QA76.76 W56. Windows Help also has a good description of the commands and options. To start the FTP client, create a DOS session and enter the command "ftp" or "ftp servername" at the DOS prompt. To display help info for the command, enter "ftp -?". The FTP program is interactive; you enter commands (get, put, etc.) to it at the "ftp>" prompt. To see what commands are available, type "help". To get help on a particular command xxx, type "help xxx". To close the FTP connection and exit from the program, enter "quit" or "bye". Notes: - To specify a non-standard FTP port number to connect to on the server, do the following. Do not specify the server name on the FTP command. At the ftp> prompt, enter: open servername n where n is the port number. Example: open vm1.mcgill.ca 1021 The default port number is 21 for FTP. When more than one MUSIC system is running on the same computer under VM, each may require a different FTP port number. For example, McGill University ran two production MUSIC systems called MUSICA and MUSICB, under the VM/ESA system whose host name is vm1.mcgill.ca. The FTP server port numbers were 1021 for MUSICA and 1022 for MUSICB. - To send a specific server command to the FTP server you are connected to, enter "quote xxx", where xxx is the server command. The command "literal xxx" is equivalent. Example: quote autotype off quote rsize 80 quote syst - To get help information about the FTP server you are connected to, enter "remotehelp" or "quote help". You can also use the command "quote syst" to get server system information. - To see the commands the FTP client sends to the server, enter the command "debug". - To get a file from the server: get serverfile [localfile] - To send a file to the server: put localfile [serverfile] - To get or send multiple files by a single command: mget filepattern or mput filepattern where filepattern is a file name pattern, containing "wild" characters * and/or ?. * matches any 0 or more chars, and ? matches any single char. For example, mget proj*.txt gets all files (in the current remote directory) whose names start with proj and end with .txt . By default, you will be prompted before each file is transferred; reply y to transfer it, or n to skip it. You can turn off this prompting by the prompt command. The prompt command toggles prompting on/off. - To set file transfer type to ascii, enter "ascii" or "type ascii" before the transfer. NOTE: You may also have to enter "quote autotype off" to turn off MUSIC's AUTOTYPE mode. See the AUTOTYPE server command above. - To set file transfer type to binary, enter "binary" or "type binary" or "type i" before the transfer. NOTE: You may also have to enter "quote autotype off" to turn off MUSIC's AUTOTYPE mode. See the AUTOTYPE server command above. - To change to another directory on the server machine: "cd dir". - To change to another local (DOS) directory, enter "lcd dir". - A problem is that long directory listings scroll off the screen before you can read them. In that case, use the command "mdir dir localfile" (listing with file attributes) or "mls dir localfile" (listing of file names only), which stores the directory listing into the specified local file. Then use another DOS session under Windows to edit or view the listing file. Examples: mdir * temp1 mls ab* temp2 - To read a file that you have downloaded from the server (by a get command), start another DOS session under Windows and use it to edit or view the file. Your FTP session continues, and you can switch back and forth between the DOS windows. - FTP.EXE can be automated by setting up a script file, which contains the commands and input for the program. For example, if the script file is auto1.scr, you would execute it by the DOS command "ftp -s:auto1.scr". Here is a sample script file: open mysite.mcgill.ca jsmith mypasswd quote syst get doc1.txt bye 6. Using the WS_FTP Client -------------------------- The FTP client program, WS_FTP, available in various versions from www.ipswitch.com, is a popular and excellent product. It works well with MUSIC's FTP server, and uses an intuitive graphical user interface. A freeware/shareware version of WS_FTP is available for academic/personal use. To allow for MUSIC servers that may be set up to give directory listings in the standard Unix format (instead of the native MUSIC format), specify the Host Type as "Automatic detect" (not "Music"), in WS_FTP's connection Session Properties screen. To send a specific command xxx to the server, right-click anywhere in the WS_FTP window, select "FTP Commands", then "QUOTE", then enter the command "quote xxx" (without the " quotation marks). To transfer multiple files in a single operation, you can select multiple files, via the mouse, in either the local or remote list of file names. To select several files: click on the first one, then Ctrl+click (i.e. hold down the Ctrl key while you click with the mouse) on other names to be added. To select a set of consecutive files: click on the first one, then Shift+click on the last one. All selected files are highlighted. 7. Using MUSIC/SP as a Repository for PC Files ---------------------------------------------- One example of using FTP is to store PC files (from DOS or Windows) on MUSIC/SP. This may be for backup or distribution purposes, for example. You run an FTP client program on the PC, and enter PUT or MPUT commands to send PC files to MUSIC. At a later time, you use GET or MGET commands to return the files to the PC. It is very important to make sure the correct transfer type (text or binary) is used for each file transfer. You should use the command "type binary" (for binary files) or "type ascii" (for text files) to tell the FTP client the type of file, before using the GET or PUT commands. Also, if the MUSIC file name extension is not a common one (such as .TXT, .HTML, .HTM, .EXE, .GIF, .ARC), and the file is binary, you should turn off the MUSIC FTP server's AUTOTYPE feature by the command "quote autotype off". See the AUTOTYPE command above. When sending the PC files to MUSIC, it is best to tell MUSIC to use record format VC (variable-length compressed) in all cases. This is done by entering the command "quote recfm vc" in the FTP client, before using the PUT or MPUT commands. VC is the default for text files, but not for binary files. For text files, VC ensures that long records are not truncated. For binary files, VC preserves the exact size of the file, since the last record on MUSIC can be shorter than the others. See the description of the RECFM and RSIZE commands above. "quote recfm v" can also be used, if data compression is not wanted on the MUSIC side. In the case of a binary file, "quote lrecl n" should also be specified, where n is (for example) 8192. This means that records in the MUSIC file will all be n bytes long (before compression, if any), except for the last record, which may be shorter than n. The default is n=512, but n=8192 is more efficient in most cases. n must not exceed 20480 (or whatever the BS parameter is in the server, in file $tcp:ftpd). Here is an example, using the FTP.EXE client of Windows 95: (1) Upload files to MUSIC: quote autotype off quote recfm vc quote allo 500000 <-- initial file allocation, in bytes (may be needed if the file is large) type ascii put file1.txt put another.txt put myfile.htm ... type binary quote recfm v quote lrecl 8192 put prog1.exe put another.exe put myfile.gif ... (2) Download the files back to the PC: quote autotype off type ascii get file1.txt get another.txt get myfile.htm ... type binary get prog1.exe get another.exe get myfile.gif ... 8. Using the PC as a Repository for MUSIC/SP Files -------------------------------------------------- Another example of using FTP is to store MUSIC files on the PC, as DOS or Windows files. This may be for backup or distribution purposes, for example. You run an FTP client program on the PC, and enter GET or MGET commands to transfer MUSIC files to the PC. At a later time, you may use PUT or MPUT commands to transfer the files (or some of them) back to MUSIC. Storage of MUSIC archive files on a removable PC disk, such as a CD or 100 MB ZIP disk, is a practical alternative to mainframe tape, when off-site backup is required. This works best when there is a high-speed network connection between the PC and the MUSIC mainframe. It is important to ensure that, when the file is transferred back to MUSIC, it is an exact duplicate of the original MUSIC file, and has the same attributes such as record format and record size. This is especially important for binary files. It is also important to make sure the correct transfer type (text or binary) is used for each file transfer. MUSIC load modules (.LMOD), object modules (.OBJ) and archive files (.ARC) are binary files. It is best to use the command "quote autotype off", to make sure MUSIC's FTP server honours the transfer type you specify; see the AUTOTYPE command above. For text files, you normally do not have to worry about specifying the record format or record length. Files will have record format VC (variable-length compressed) when they are returned to MUSIC. However, if you want a returned file to have a specific format, such as FC and record length 80, you can use the commands "quote recfm fc" and "quote rsize 80" before the PUT or MPUT command. See the descriptions of the RECFM and RSIZE commands above. For fixed-length (record format F or FC) on MUSIC, be sure you specify a large enough record length, to avoid truncating records. Binary files are more problematic when it comes to preserving the original record format and record lengths. For record formats F, FC, and U, take a note of the original format and record length for each file, and specify those same values when returning the binary file to MUSIC. Use the commands "quote recfm x" and "quote rsize n", where x is F, FC, or U and n is the record length. You can use the ATTRIB or LIB or DIR command on MUSIC to see the record formats and lengths (use the X or F options on the LIB and DIR commands). For example, most archive (.ARC) files on MUSIC are FC/80, so you you would specify "quote recfm fc" and "quote rsize 80" when uploading the file back to MUSIC. For binary files with record formats V or VC, the records may have different lengths within the same file, and this detailed record length information is not retained when the file is transferred to the PC. One solution for this problem is to archive the MUSIC file or files to a single .ARC file, using the FILARC or MFARC2 utility. Make the .ARC file record format FC, record length 80. Then transfer the binary .ARC file to the PC. When sending the .ARC file back to MUSIC, use "quote recfm fc" and "quote rsize 80". Finally, use the FILRST or MFREST utility to de-archive the original file(s) from the archive file. This method also has the advantage of preserving other file attributes such as the tag and access control. This method can also be used for any types of MUSIC files, and is useful when dealing with a large number of MUSIC files. Here is an example: (1) Run the following FILARC job on MUSIC to archive the files into the single file GROUP.ARC. To run this job, use the Editor to type the lines into a file, then execute the file at the terminal. Note that wild characters * and ? can be used in the file name records, to specify groups of files. Here, PROG1.* means all file names starting with "PROG1.". /FILE 1 NAME(GROUP.ARC) NEW RECFM(FC) LRECL(80) SPACE(200) /INCLUDE *COM:FILARC OUTPUT=1 ANYFILE1.TXT ANYFILE2.OBJ ANYFILE3.LMOD ANYFILE4.XX PROG1.* Note: Since FTP deals only with MUSIC Save Library files, and such files are limited to a maximum size of about 57 MBytes, it may be necessary to run more than one FILARC job and create several .ARC files. (2) Using the FTP.EXE client of Windows 95, transfer the binary archive file to the PC. (Note that the AUTOTYPE OFF command is not really required here and in (3) below, since the extension .ARC is normally defined as a binary file in $tcp:ftpd.filetypes, but it is best to use it anyway.) type binary quote autotype off get group.arc As an alternative, see also (2a) below. (3) At some later time, transfer the archive file back to MUSIC. Here, it is important to specify the MUSIC record format (FC) and record length (80) matching the original MUSIC archive file. Note that the PUT command automatically replaces the target MUSIC file if it still exists. type binary quote autotype off quote recfm fc quote rsize 80 quote allo n <-- allocation size in bytes (may be needed if file is large) put group.arc As an alternative, see also (3a) below. (4) Restore the files from the archive file, by running the following FILRST job on MUSIC. JSMITH is your file ownership id (your userid without the subcode part, if any) on MUSIC. The asterisk (*) indicates all files in the archive. /FILE 1 NAME(GROUP.ARC) /INCLUDE *COM:FILRST NAME='JSMITH:*',TO='JSMITH:*',REPL=T As a variation of the above procedure, the "binary block on" command of MUSIC's FTP server can be used. This command can be issued via the QUOTE command from the PC FTP client. It causes the transfer to be done in MUSIC 512-byte file blocks, and file info (attributes such as RECFM, LRECL, EOFPT, etc.) is also transmitted. This gives a smaller file on the PC, since RECFM(FC) compression is not lost. The first 94 bytes of the PC file contain file attribute info; the remaining bytes of the PC file contain the data (unconverted) from the 512-byte blocks of the MUSIC file (last block may be short). The archive file on MUSIC is FC/80. On transfer back to MUSIC, the file size and attributes are restored exactly. Using this method, the replacements for steps (2) and (3) above are: (2a) type binary quote autotype off quote binary block on get group.arc (3a) type binary quote autotype off quote binary block on put group.arc In (2a) and (3a), the option "mus512" could be used instead of "on". "quote binary block mus512" uses a 512-byte file info header (instead of 94 bytes), and the header includes extra info such as the original MUSIC/SP file name, and date and time info. Note: Once the file is on the PC, it is also possible to apply a compression utility such as WinZip to it and/or dump it to some other medium such as a PC tape cartridge or removable disk or writable CD.