Transcript Chapter 3 Traditional Applications

Computer Networks with
Internet Technology
William Stallings
Chapter 03
Traditional Applications
Terminal Access – Telnet
History
• Oldest Internet application
• Demonstrated on four-node ARPANET deployed in 1969
• Two years to expand protocol sufficiently to make it
useful and to work out bugs
• First published version RFC 97
— "First Cut at a Proposed Telnet Protocol," February 1971
• 1983 final form issued as RFC 854 and RFC 855
— (Get and study these RFCs – see last slide)
• Still useful Internet application
• Also pioneering effort application-level protocol design
• Basis of many newer protocols
— HTTP
Remote Terminal Access
• Early motivation for networks was remote access to
interactive systems
• Dumb terminals
• Keyboard and screen with primitive comms hardware
• Stream of character data transmitted in each direction
• Local host computer or terminal controller establish
connection to remote host
• Local user can use remote host
• Hosts handle particular set of characteristics
Figure 3.1 (a) Telnet Operational
Environment on Arpanet
Network Virtual Terminals
• Virtual terminal protocol (VTP)
• Transform characteristics of real terminal into
standardized form
—Network virtual terminal (NVT)
• Imaginary device
—Well defined set of characteristics
• Both sides generate data and control signals in
native language
• Each side translates these to NVT and translates
incoming NVT to native signals
Figure 3.2 Network Virtual
Terminal Concept
Phases of operation
• Connection management
— Connection request and termination
— Telnet uses TCP
• Negotiation
— To determine mutually agreeable set of characteristics
— NVT has range of capabilities and features
— Real terminal more limited
— NVT has options, such as line length
• Control
— Exchange of control information and commands
— e.g., end of line, interrupt process
• Data
— Transfer of data between two correspondents
• For Telnet, control and data conveyed in single stream
Current Use of Telnet
• Original environment for Telnet little relevance
today
• Still used and included in the TCP/IP suite
• Availableon PCs for use over the Internet
—PC includes Telnet software
—Telnet protocol and translation between PC
keyboard/display and NVT
—Not GUI
• Services available include United States Library
of Congress
—locis.loc.gov
Figure 3.1(b) Telnet Operational
Environment on Internet
Figure 3.3
(Incomplete) A Telnet Session
telnet locis.loc.gov
Trying 140.147.254.3...
Connected to locis.loc.gov.
Escape character is '^]'.
L O C I S: LIBRARY OF CONGRESS INFORMATION SYSTEM
To make a choice: type a number, then press ENTER
1
Copyright Information
-- files available and up-to-date
2
Braille and Audio
-- files frozen mid-August 1999
3
Federal Legislation
-- files frozen December 1998
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
The LC Catalog Files are available at:
http://lcweb.loc.gov/catalog/
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
8
Searching Hours and Basic Search Commands
9
Library of Congress General Information
10
Library of Congress Fast Facts
12
Comments and Logoff
Choice:
Telnet NVT
• Lowest common denominator of existing systems (at the
time)
• Includes options and negotiation for more advanced
equipment
• Between two terminals, two processes, or terminal and
process
• Server module and user module map characteristics of
terminal to NVT
• NVT bi-directional character device with display and
keyboard
• Keyboard can generate all 128 ASCII codes plus user
control signals
Telnet Transfer Protocol
•
•
•
•
Data sent half-duplex
Terminal-to-process, newline signifies end of user input
Process-to-terminal, Telnet Go Ahead used
Underlying TCP full duplex
— Control signals sent any time regardless of current data
direction
• Data sent as stream of 8-bit bytes
— No other formatting
• Control signals and other non-data information sent as
Telnet commands
— Byte strings embedded in data stream
— User control signals, commands between Telnet processes as
part of protocol and option negotiation and sub-negotiation
Interpret as Command (IAC)
• Delimited by Interpret as Command (IAC) char (all 1s, FF, 255)
— If bit pattern 255 is part of the data stream, it must also be preceded
by an IAC. i.e. IAC 255 or 255 255
• Commands 2 bytes long
• Option negotiation commands 3 bytes long
— Third byte option identifier
— Option sub-negotiation commands vary in length
• Begin 3-byte sequence (IAC SB option-id)
• End 2-byte sequence (IAC SE)
• Protocol minimizes transmission overhead
— No message headers
• Processing overhead is high
• Must process stream one character at a time
— Data translation (between NVT and native)
— Scan for Telnet commands
• Classic tradeoff between message and stream protocols
TCP Urgent Data Pointer
• Tells receiver there is urgent data further along
data stream
• TCP does not define what user about this
• Receiving process will usually process urgent
data quickly
• Prevents TCP buffering holding up urgent data
Telnet Synch Mechanism
• Allows user to communicate urgent command to server
process
— E.g. Interrupt Process (IP) or Abort Output (AO) command
• Partially alleviates problems caused by time delays over
network connections
• Consists of Telnet command Data Mark (DM) in TCP
segment with TCP urgent notification
• Telnet sends urgent command such as AO, or IP,
followed by DM sequence (IAC DM) as urgent data
• Receiver should immediately scan data stream for Telnet
commands as normal
— Discard all data
— Telnet commands handled normally
— Continues until DM found
— Then processing returns to normal
Telnet Options
• Enable two sides connection to use capabilities
beyond default NVT
• Negotiation allows one side to request an option
—Otherside may accept or reject
—If accepted, effective immediately
—Negotiation any time after connection established
—Usually as soon as connected
• Options not part of Telnet protocol specification
• Published in RFCs
Telnet Options Categories
• Three major categories
— Change, enhance, or refine NVT characteristics
• E.g. further definition of printer characteristics
— Change transfer protocol
• E.g. Suppress Go Ahead option (3), making data transfer protocol
full duplex instead of half duplex
— Allow other information to be defined and passed over
connection
• E.g. Status option (5) requests remote side to send status of all
options negotiated
• Most options may be single sided
— Two negotiations required for both sides to use such options
Option Negotiation
• Either side may initiate negotiation
• Can ask that option be enabled or that currently
enabled option be disabled
—You may always reject a request to enable an option
—You must always accept a request to disable an
option
—Options are not enabled until the negotiation is
complete
—Never negotiate (either request or respond) about
something that is already true; that is, do not initiate
or respond to a request to initiate an option that is
already in effect
WILL WONT DO DONT
• Interpretation of commands depends on context (see
next slide)
• Unambiguous if both sides make same request and
messages pass on network
— A wishes B to implement option
• A issues DO
• If B agrees, it responds with WILL
— B wishes to implement option on its (B’s) side, it issues WILL
• If A agrees, it responds with DO
— Both A and B request B to implement option at about same time
• A issues DO and B issues WILL
• Commands cross in network
• Both sides receive response to their command
• May be sub-negotiation to define details
Figure 3.4 Negotiation
Messages in Telnet
The Longevity of Telnet
• Telnet is older than most of its users
— (But not most lecturers!)
• Telnet is simple
— RFC 854 is 15 pages
— HTTP (see later lecture) is 176 pages
— Simple job done by simple protocol
• Telnet can evolve
— Option negotiation was brilliant
— Not common in IETF protocol designs until late 1980s
— Enables Telnet to evolve to meet new demands without endless
new versions of basic protocol
• Currently over 100 RFCs on Telnet and its options
— 3% of the entire body of RFCs
— Most recent RFC 2953, Telnet Encryption, September 2000
FILE TRANSFER—FTP
• FTP evolved from an era of radically diverse systems
• Has obsolete commands, transfer modes, and data representations
• Objectives:
—
—
—
—
Promote sharing of files (computer programs and/or data)
Encourage indirect or implicit (via programs) use of remote computers
Shield user from variations in file storage systems among hosts
Transfer data reliably and efficiently
• File systems, rather than just files
• Single file viewed as set of bits with name
—
—
—
—
Trivial File Transfer Protocol (TFTP) does this
Send request header to read or write file with some name
Stream bits across
11 pages to define
• FTP deals with metadata such as file pathnames, file organization,
access control, and data representation
— Accordingly, RFC 959 is 69 pages long
FTP Model
• User FTP entity and Server FTP entity
• Initiating host is user
— Chooses file name and options
• Server accepts or rejects request
— Based on its file system protection and options requested
— If accepted, server responsible for establishing and managing
transfer
• Operates on two levels (Figure 3.5)
• Establish TCP connection
— Exchange control information (commands and replies)
— Second TCP connection established for data transfer
• FTP user interface enables human user or program to
access User FTP
FTP Commands
• Specify parameters for data connection
— Data port, transfer mode, representation type, and structure
— Nature of file system operation
• Store, retrieve, append, delete
• User data transfer protocol should "listen" on specified
data port
• Server initiates data connection and data transfer
• FTP uses Telnet protocol on control connection
— FTP user protocol or FTP server protocol may implement Telnet
rules directly
— FTP user protocol or FTP server protocol may use existing Telnet
module
Figure 3.5
FTP Model
Figure 3.6
Overview
of an FTP
Transfer
Options
• FTP assumes files are objects in mass storage
— Share some properties regardless of machine
— Files uniquely identified by symbolic names
— Files have owners and protection against unauthorized access
— Files may be created, read from (copied from), written into, or
deleted (within protection rules)
• To support specific computers and operating systems,
FTP can negotiate options in three dimensions
— Datatype, file type, and transfer mode
• Systems programmer on each system determines
— How particular file can be mapped to standard file type
— Using one of standard data types
— Transferred using standard mode
— Such that it is useful at destination
Data Types
• ASCII, EBCDIC, image, and logical byte size
• Text files normally stored as character string
— 8-bit ASCII on most machines
— If ASCII option used, no character code conversion required at either
end in most cases
— EBCDIC appropriate if both machines IBM hosts
• ASCII or EBCDIC files may have further line or page printer
specification
— Nonprint: Suitable for files not destined for a line printer
— Telnet formatting: Embedded control characters
— Character control formatting: Formatting conventions from FORTRAN
• Image transfer is bit-by-bit replication of file from the source
machine on the target machine
• Logical byte size type used when data unit size must be preserved
— Specifies byte size (need not be 8 bits)
File Types
• File structure, record structure, and page structure
• To promote convenient, efficient interface to file system
• Not possible to address idiosyncrasies of all operating
systems
• File structure
— String of bytes (defined by data type option) that terminates in
an end of file marker
— Most transfers use this type
• Record structure
— Sequence of records
— Causes transmission of individual records, separated by
standard End of Record marker for specified data type
• Page structure
— For files not stored contiguously on disk
— Page structure needs to be preserved
Figure 3.7 FTP File Types
Transmission Modes –
Stream Mode
• Optimise use of network
• Stream mode (default)
—Raw data sent
—Least computational burden on user and server
systems
—No restriction on file type
• Record-structure files, 2-byte control code for EOR and EOF
Transmission Modes –
Block Mode
• Provides for restarting failed or interrupted transfer
— Source encapsulates data into blocks (see Figure 3.8a)
• Block begins with two field header
• Descriptor may indicate zero of more of:
— Last block in record: If record structure used each record
consists of one or more blocks
— Last block in file
— Suspect data: data may contain errors
• Not intended for error control within FTP
• Allows sites to exchanging data (e.g., seismic or weather) to send
and receive all data despite local errors (such as "magnetic tape
read errors"), but showcertain portions are suspect
— Restart marker: marks checkpoint in data stream
• Receiver marks corresponding position and returns this
• May restart from last correctly received marker
• Count field indicates total length of data block in bytes
Transmission Modes –
Compressed Mode
• Allows source to squeeze sequences of same
character into a shorter coded sequence
—Uncompressed data
—Replicated byte: Up to 63 of specified bytes
—Filler string: Up to 63 of filler characters inserted at
destination
—Escape sequence: Byte of all zeros followed by
descriptor code byte, as in block mode
Figure 3.8 FTP Transmission
Mode Formats
Electronic Mail
• Most heavily used application on any network
• Simple Mail Transfer Protocol (SMTP)
—TCP/IP
—Delivery of simple text messages
• Multi-purpose Internet Mail Extension (MIME)
—Delivery of other types of data
—Voice, images, video clips
SMTP
• RFC 821
• Not concerned with format of messages or data
— Covered in RFC 822 (see later)
• SMTP uses info written on envelope of mail
— Message header
• Does not look at contents
— Message body
• Except:
— Standardize message character set to 7 bit ASCII
— Add log info to start of message
• Shows path taken
Basic Operation
• Mail created by user agent program (mail client)
—Message consists of:
• Header containing recipient’s address and other info
• Body containing user data
• Messages queued and sent as input to SMTP
sender program
—Typically a server process (daemon on UNIX)
Mail Message Contents
• Each queued message has:
—Message text
• RFC 822 header with message envelope and list of recipients
• Message body, composed by user
—A list of mail destinations
•
•
•
•
Derived by user agent from header
May be listed in header
May require expansion of mailing lists
May need replacement of mnemonic names with mailbox
names
• If BCCs indicated, user agent needs to prepare
correct message format
SMTP Sender
• Takes message from queue
• Transmits to proper destination host
— Via SMTP transaction
— Over one or more TCP connections to port 25
• Host may have multiple senders active
• Host should be able to create receivers on demand
• When delivery complete, sender deletes destination
from list for that message
• When all destinations processed, message is deleted
Optimization
• If message destined for multiple users on a
given host, it is sent only once
—Delivery to users handled at destination host
• If multiple messages ready for given host, a
single TCP connection can be used
—Saves overhead of setting up and dropping
connection
Possible Errors
•
•
•
•
Host unreachable
Host out of operation
TCP connection fail during transfer
Sender can re-queue mail
—Give up after a period
• Faulty destination address
—User error
—Target user changed address
—Redirect if possible
—Inform user if not
SMTP Protocol - Reliability
• Used to transfer messages from sender to
receiver over TCP connection
• Attempts to provide reliable service
• No guarantee to recover lost messages
• No end to end acknowledgement to originator
• Error indication delivery not guaranteed
• Generally considered reliable
SMTP Receiver
• Accepts arriving message
• Places in user mailbox or copies to outgoing
queue for forwarding
• Receiver must:
—Verify local mail destinations
—Deal with errors
• Transmission
• Lack of disk space
• Sender responsible for message until receiver
confirm complete transfer
—Indicates mail has arrived at host, not user
SMTP Forwarding
• Mostly direct transfer from sender host to
receiver host
• May go through intermediate machine via
forwarding capability
—Sender can specify route
—Target user may have moved
Conversation
• SMTP limited to conversation between sender
and receiver
• Main function is to transfer messages
• Rest of mail handling beyond scope of SMTP
—May differ between systems
Figure 3.9 SMTP Mail Flow
SMTP System Overview
• Commands and responses between sender and
receiver
• Initiative with sender
—Establishes TCP connection
•
•
•
•
Sender sends commands to receiver
e.g. HELO<SP><domain><CRLF>
Each command generates exactly one reply
e.g. 250 requested mail action ok; completed
SMTP Replies
• Leading digit indicates category
—Positive completion reply (2xx)
—Positive intermediate reply (3xx)
—Transient negative completion reply (4xx)
—Permanent negative completion reply (5xx)
Operation Phases
• Connection setup
• Exchange of command-response pairs
• Connection termination
Connection Setup
• Sender opens TCP connection with receiver
• Once connected, receiver identifies itself
—220 <domain> service ready
• Sender identifies itself
—HELO
• Receiver accepts sender’s identification
—250 OK
• If mail service not available, step 2 above
becomes:
—421 service not available
Mail Transfer
• Sender may send one or more messages to receiver
• MAIL command identifies originator
— Gives reverse path to used for error reporting
— Receiver returns 250 OK or appropriate fail/error message
• One or more RCPT commands identifies recipients for
the message
— Separate reply for each recipient
• DATA command transfers message text
— End of message indicated by line containing just period (.)
Closing Connection
•
•
•
•
Two steps
Sender sends QUIT and waits for reply
Then initiate TCP close operation
Receiver initiates TCP close after sending reply
to QUIT
Format for Text Messages
RFC 882
• Message viewed as having envelope and
contents
• Envelope contains information required to
transmit and deliver message
• Message is sequence of lines of text
—Uses general memo framework
—Header usually keyword followed by colon followed
by arguments
Example Message
Date:Tue, 16 Jan 1996 10:37:17 (EST)
From: “William Stallings” <[email protected]>
Subject:The syntax of RFC 822
To: [email protected]
Cc: Jones@Yet-another_host.com
This is the main text, delimited from the header by
a blank line.
Multipurpose Internet Mail
Extension (MIME)
• Extension to RFC822
• SMTP can not transmit executables
— Uuencode and other schemes are available
• Not standardized
• Can not transmit text including international characters
(e.g. â, å, ä, è, é, ê, ë)
— Need 8 bit ASCII
• Servers may reject mail over certain size
• Translation between ASCII and EBCDIC not standard
• SMTP gateways to X.400 can not handle none text data
in X.400 messages
• Some SMTP implementations do not adhere to standard
— CRLF, truncate or wrap long lines, removal of white space, etc.
Overview of MIME
• Five new message header fields
—MIME version
—Content type
—Content transfer encoding
—Content Id
—Content Description
• Number of content formats defines
• Transfer encoding defined
Content Types
• Text body
• Multipart
— Mixed, Parallel, Alternative, Digest
• Message
— RFC 822, Partial, External-body
• Image
— jpeg, gif
• Video
— mpeg
• Audio
— Basic
• Application
— Postscript
— octet stream
MIME Transfer Encodings
• Reliable delivery across wide largest range of
environments
• Content transfer encoding field
— Six values
— Three (7bit, 8bit, binary) no encoding done
• Provide info about nature of data
• Quoted-printable
— Data largely printable ASCII characters
— Non-printing characters represented by hex code
• Base64
— Maps arbitrary binary input onto printable output
• X-token
— Named nonstandard encoding
Figure 3.10 Printable Encoding of
Binary Data into Radix-64 Format
Required Reading
• Stallings chapter 03
• RFCs from www.rfc-editor.org