Transcript BitTorrent - Framework

BitTorrent Framework
Background
• BT is a very popular peer to peer protocol
with many implementations:
– http://en.wikipedia.org/wiki/Comparison_of_Bit
Torrent_clients
• The first one was created in Python by
Bram Cohen.
Workshop goal
• Exploit BitTorrent protocol to give less for
more.
• Real world motivation
– Places with low upload (Africa, China)
– Bad character
How do I test my idea
• Create my own (or modified) BitTorrent client.
• Run together:
– “The World”: N “standard” BitTorrent Clients.
• Specific upload rate
• Various download stages
– My Client
• See how much time my client ran, and what was
its upload / download ratio.
Execution example
Tracker
BT
BT
BT
BT
BT
run-all.sh
BT
BT
BT
BT
BT
BT
How does it work?
• Run tracker
• Run the “world”
• Run your client (if given)
How does it work?
• Both tracker and the “world” run on the
same computer
• we have
– pc-hanoch-w1
– pc-hanoch-w2
– Please reserve time in:
• http://0368350023.wikispaces.com/
• Or from course www - "Scheduling computers for
tests."
The main script - run_all.sh
• Location:
– /users/courses/hanochworkshop/bittorrent
• Run_all.sh:
– Parameters:
• Start / stop
• My client – optional
run_all.sh operation
• Start:
– Create download directory
– Run the “world”
• Each world client starts with different “saved”
amount.
– Run “my client” – if given
– All uploads rates are restricted.
• Stop
– Kill all python threads
To run client separately:
• BitTornado-My-Client/btdownloadheadless.py
– --saveas /vol/scratch/bt-download/picture.tst.bmp
– --max_upload_rate 70
– --security 0
– ./picture.pc-hanoch-w[1,2].bmp.torrent
• Not necessarily on the same machine as the
world.
What implementation we use
• The clients are a modified version based on an
up to date version of Bram Python Client.
• Modifications:
– “World” clients
• do not print as it would only make a mess
• reincarnate after the finish to make a real world feel.
– “My Client” template – kills all clients when finished
– All clients – print total time
Python
• Very common script language
• Intuitive and readable
• References:
– python.org :http://docs.python.org/reference/
– http://wiki.python.org/moin/BeginnersGuide/NonProgrammers
– http://wiki.python.org/moin/BeginnersGuide/Programmers
• The book "Think like a Computer Scientist in Python" has a very
good reputation:
My Client
• We supply a default “My Client” which is like the
“standard” one.
• If you want you may replace it with any other
implementation, with certain reservations:
– Remember we compare to our client.
– It does not count to download a faster client than our
client and simply say it is faster.
– Client should either limit it’s upload rate, or be able to
work with trickle.
Getting started
• Run the environment
– ssh pc-hanoch-w[1,2]
– cd /users/courses/hanochworkshop/bittorrent
– run_all.sh
• start
• [BitTornado-My-Client/btdownloadheadless.py]
– Note tracker prints a message for each GET.
• You can connect to the tracker with a browser at:
– http://pc-hanoch-w1:6970/
• Do ls -l /vol/scratch/bt-download/ to see downloaded files
What is it doing?
• Run bttrack.
• Copy From FreshFiles a set of
“downloaded” files to download directory.
• Run X seeds
• Run a python client per “downloaded” file.
• Run my client if given
For debug: run client separately
• ./run_all.sh start to start the world
• BitTornado-My-Client/btdownloadheadless.py
– --saveas /vol/scratch/bt-download/picture.tst.bmp
– --max_upload_rate 70
– --security 0
– ./picture.pc-hanoch-w[1,2].bmp.torrent
• Start debug.
• Use your favorite debug environment.
• IDLE is standard source level debugger.
Directory Structure
• BitTornado-My-Client –
– high level application
• BitTornado-My-Client/BitTornado –
– services
• BitTornado-My-Client/BitTornado/BT1 –
– Actual work
Client Operation
• btdownloadheadless.py – First to run.
– Performs initializations.
•
•
•
•
get_response – parse the .torrent
Infohash – the ID of the .torrent
startEngine – Start connection with peers
startRerequester - Start connection with tracker
– Run rawserver.listen_forever from RawServer.py.
– listen_forever is:
• Polling port
• executing func() from a task queue.
Receiving messages
• From listen_forever loop it goes to:
– BT1/Connecter.py/got_message
•
•
•
•
•
•
•
•
•
CHOKE
UNCHOKE
INTERESTED
NOT_INTERESTED
BITFIELD
REQUEST
CANCEL
PIECE
HAVE
Influencing decisions
• Can be done in in one of the callbacks
which spring from Connecter.py /
got_message:
• Downloader.py
– HAVE->got_have->send_interested
• maybe check connection before sending?
BT tracker
• Send GET request in Rerequester.py /
announce
• Receive GET data in thread in
Rerequester.py / _rerequest
Influencing decisions
• In Encrypter.py / start_connection or
_start_connection_from_queue
For instance try different max_connections.
_rerequest_single – check timout for
scheduling.
Policy
• Choker.py/_round_robin – decide who to
choke / optimistically un choke /
send_have to.
• Downloader.py / _request_more.
• PiecePicker.py / next - rarest piece, etc.
• got_unchoke->_request_more->next…