BitTorrentWishList
From TheoryOrg
Jump to navigationJump to searchBitTorrent Wish List
These ideas do not express the determined future ofBitTorrent. They are ideas made by people in theBitTorrent community who are requesting comments. See related documents below for specifications.
Identification
BitTorrent is a peer-to-peer file sharing protocol designed by Bram Cohen. Visit his pages at[1]. BitTorrent is designed to facilitate file transfers among multiple peers across unreliable networks.
Purpose
This document is intended to be maintained and used by the BitTorrent development community. Everyone is invited to contribute to this document. This is the place to capture detailed descriptions of feature requests, behavorial changes, and other deviations from version 1.0 of the BitTorrent protocol specification.
Related Documents
BitTorrentSpecification - Version 1.0 of the BitTorrent protocol specification.
[2] - The 'official protocol specification.
Contributions
Template Header
I'd like to propose a rough template header, so that we come out of this with some semblance of structure and/or organization. Please use this template header for each proposal that you add.
* Title: Brief (one-line) title for your proposal* Submitter: Your Yahoo! group handle or name* Email: Your email address, if you care to share it.* Category: Category of your proposal. For example, this could be "Feature Request", "Enhancement", etc. If nothing fits, make something up.* Compatibilty: Yes/No, yes if your proposal is backwards compatible with the v1.0 spec, no if it's not. For instance, message additions are backwards compatible if you don't rely on them for basic functionality in the v2.0 client. Changing the handshake message format, however, would not be a compatible change.* Thread: If you've described this request on the BitTorrent group, specify a link to the thread here. This will provide an easy reference for others to do some background reading.* Description: Put as much detail in here as you can. Edit what you have as details become apparent from the discussions on the BitTorrent group.* Pros: The author and others in the community may add to this section. Basically, this is a place to summarize the good things about this proposal.* Cons: The author is likely to not have much to say here, so this section is primarily for the community. As downsides are identified on the ~Bittorrent group, they should be briefly summarized here, possibly with references to the threads themselves. This will help to preserve the final conclusions (if any) for the proposal.
Proposals
Please populate this section.
* Title: Neighbour bitfield, Neighbour HAVE* Submitter: Vaste* Email: vaste at vaste mine nu* Category: Enhancement, Client to client* Compatibilty: Yes, can be ignored; merely provides better information* Description: Ability to request/send a histogram summarizing the bitfields of neighbours. From the perspective of the peer sending the histogram neighbours would be directly connected peers other than the requesting peer (also excluding self). The histogram could be encoded in a similar way to a bitfield except the radix of the histogram would be the maximum value of the histogram + 1, instead of always 2 (binary). (E.g. bitfields 110 + 011 >> histogram 121 >> 16 0x10 encoded, radix 3; bitfield 101 >> histogram 101 >> 5 0x5 encoded, radix 2.) Received (normal) HAVEs would be forwarded as Neighbour HAVEs to other peers. Since change of availability in rare pieces is more important information than in non-rare pieces, HAVEs of rare-pieces could be forwarded more frequently. Could (should?) be batched. Should be rationalized, see proposals regarding HAVE management.* Pros: Greatly reduces the local blindness of rarity described inAvailability, where a rare piece seems less rare to the peer(s) that can (and likely should) download it. This should optimize seeding (esp. initial). Gives a slightly better view on availability.* Cons: Easy to lie. Requires exponential space, time, and bandwidth (highly complex graphs to correctly model network). Withoutextreme' care, highly likely to cause broadcast storms over entire swarm (well known networking problem). Minimal performance gains with avery* high cost.* Title: Compact Scrape* Submitter: BORGET S颡stien* Email: sqrtboy@phpsoft.org* Category: Tracker - Bandwidth reduction* Compatibilty: Yes* Thread:http://www.phpsoft.org/trackermodes.php* Description: the main idea is to implement a compact mode for scrape data too. in the thread, I suggest a way to do it. it might need further improvement, but results are already here !* Pros: saves more bandwith on tracker.
* Cons: need updating bt clients, need more computing power for the bt client
* Title: Get Info / Info* Submitter: Olaf van der Spek* Email* : OvdSpek@LIACS.NL* Category: : Feature Request* Compatibilty: Yes* Thread:http://groups.yahoo.com/group/BitTorrent/message/3889* Description: Add a pair of messages to allow the exchange of the info section of the metainfo file. With these messages, the metainfo file may substitute an infohash key (with the SHA1 hash of the info key) instead of the info key itself, thus reducing the size of the metainfo file. Also, URIs could be formed which contain the SHA1 infohash, which can be used to request peer lists from the tracker and initiate the getinfo message. Seehttp://sourceforge.net/tracker /?funcdetail&aid842089&groupid84122&atid575157 for details.* Pros: This will relieve much of the bandwidth load on trackers, by allowing the peer network to exchange the info section of the metainfo file.* Cons: This increases the percieved vulnerability of the p2p network to malicious attacks, in the form of erroneous data being introduced to peers requesting the info data from other peers.Olaf: Because the peer data can be verified with the info_hash you get from the tracker, such attacks will fail.* Note: this is implemented as theut_metainfoextension (BEP 9).
* Title: Get Peers / Peers* Submitter:EnergizerRabbit1* Email: dpmott@sep.com* Category: Feature Request* Compatibilty: Yes* Thread:http://groups.yahoo.com/group/BitTorrent/message/3987* Description: Add a pair of messages to allow the exchange of peer lists between peers. With these messages, peers may learn of other peers, even if the tracker goes down or disavows the torrent.* Pros: This will relieve some of the bandwidth load on trackers, by allowing the peer network to exchange peer lists. Also, clients could enter the peer network with only one other peer IP address.* Cons: This will have a serious effect on the way the network is connected. Now, the peers you connect to are (uniformly) randomly selected, resulting in a network without cliques of highly connected peers in which some peers are acting as gateways, or bottlenecks, to other cliques. With this feature such cliques and bottlenecks could be created, resulting in e.g. rare pieces being distributed slowly to/from some parts of the network. (A distributed random peer selection algorithm could be used as a substitute, but that is generally expensive and orders of magnitude more complicated.)
It has also been suggested that this feature could be used to initiateDoS attacks against a specific target, and although the likelihood that simultaneous or sustained attacks seems nil, the possibility cannot be ignored.* Note: this is already implemented, at least twice (ut_pex and Azureux PEX).
* Title: Piece compression* Submitter:* Email:* Category: Feature Request* Compatibilty: Yes, by using options bitfield and not compressing for 1.0 peers* Thread:* Description: Provide compression for pieces that are sent on the wire.* Pros: Peers actually get more than you send, after decompression, effectively increasing you upload bandwidth.* Pros: Seeding from offline media like a CDROM ISO would benefit from this. Compression using different OS and archive application is highly unlikely to yield the exact binary image needed for seeding, while an ISO rip will.* Cons: Peers need to be able to tell your client that they can support decompression. Also, compression takes processing power, which may reduce your overall upload throughput.* Cons: Highly unlikely to yield useful results. Files being transfered are quite likely to already be highly compressed with payload-specific compression. The general schemes BT would use cannot* provide useful additional compression (basic fact of compression methods). * Re:Cons: Send the piece compressed only if compression saved space, otherwise send uncompressed, Emule does this. Not sure about CPU/resouce usage.* Pros: as an *optional* extension, this can actually be pretty useful. BitTorrent is often used for transferring audio CD images, and those can compress quite well if you use the right predictor. Most BitTorrent clients do not use much CPU (disk is usually the limiting factor), and at any rate compression can be disabled when CPU usage is high. The main difficulty is defining the right set of compression algorithms/predictors, and detecting incompressible data without actually attempting to compress it.
* Title: Better HAVE message management* Submitter:* Email:* Category: Client Feature Implementation* Compatibilty: Yes (no protocol change what-so-ever)* Thread:* Description: Don't send HAVE messages to peers that already have the piece. (dubbed HAVE Suppressionhttp://lists.ibiblio.org/pipermail/bittorrent/2005-June/001513.html )* Pros: Saves your client's upload bandwidth, saves everyone elses download bandwidth. Guarenteed to provide 50% reduction in number of HAVE messages, roughly 25-33% decrease inBitTorrent overhead.* Cons: Weakens the ability of peers to model peer behavior.* Note: The protocol doesn't strictly require sending HAVE messages when pieces are obtained. Unless a peer is prepared to verify the peer has the piece by downloading it, you cannot be certain that the peer has in fact obtained the piece. As such it is worthless to rely on this information being accurate (smart peers will save bandwdith, malicious peers will lie).
* Title: Get Bitfield / Bitfield, HAVE message management* Submitter:EnergizerRabbit1* Email: dpmott@sep.com* Category: Feature Request* Compatibilty: No. Sending request is okay (can be ignored by peers), but depending on the response for proper operation is not backwards compatible.* Thread :* Description: Add a pair of messages to allow the exchange of the bitfield which describes available pieces. This will also involve modification to the HAVE message management, notably there needs to be a way to tell the peer to not send the HAVE messages.* Pros: Allows asking for a list of pieces that are available from a peer, instead of receiving HAVE messages.* Cons: Not backwards compatible. Real improvement?
* Title: MULTIHAVE message* Submitter: Bob* Email:* Category: Feature Request* Compatibilty: Yes, via the options bitfield.* Thread:* Description: Instead of sending a HAVE message per piece to everyone, send a MULTIHAVE message periodically. If too many pieces have been downloaded, then send the bitfield instead. This feature could also benefit from a way to tell a peer to not send HAVE, MULTIHAVE, or Bitfield messages.* Pros: Reduces bandwidth* Cons: With a well behaving client and reasonable piece size back-to-back HAVE messages will be uncommon, resulting in trivial bandwidth reduction.
* Note: Most likely implementation, flag in options field. Use length on HAVE message to indicate advertisement of multiple pieces.
* Title: Done message* Submitter:EnergizerRabbit1* Email: dpmott@sep.com* Category: Feature Request* Compatibilty: Yes, can always be sent* Thread:* Description: Send a DONE message when download completes. This will allow peers to update their piece bitfields that represent what you have (i.e. everything), and they can start downloading pieces from you.* Pros: Coupled with other 2.0 features, such as the ability to choke HAVE messages from peers, this is a good way to save bandwidth and still get the information you need. Also, this particular enhancement is backwards compatible.* Cons: Useless. Increases protocol complexity. At time of completion you'll of already of advertised almost all messages, therefore advertising multiple pieces at once will be of minimal value. Smart peers can already supress sending HAVE messages for pieces you already have.
* Title: have_ctl (1|0) message* Submitter : EnergizerRabbit1* Email* : dpmott@sep.com* Category: Feature Request* Compatibilty : Yes, can always be sent* Thread:* Description: Provide a message to enable/disable HAVE messages from peers.* Pros: Allows a peer to manage the (otherwise) unsolicited HAVE messages from peers.* Cons: How can this possibly enhance performance?
* Title: Restructure metadata info file* Submitter: EnergizerRabbit1* Email: dpmott@sep.com* Category: Feature Request* Compatibilty: No. The original metadata would need to be preserved for 1.0 clients.* Thread:* Description: Restructure the metadata file to make it possible to add files to it, without actually having the other media there. In this way, you could take a dozen or so individual metadata files, and make a batch torrent out of them, without having the media on your hard drive. This would involve reorganizing the metadata info to be file-centric, instead of piece-centric, i.e. each file would have a list of SHA1 hashes, one for each piece.* Pros: allows creation of batch torrents (i.e. bundles) w/o the original media* Cons: not backwards compatible with the 1.0 metainfo spec. Also, it can become inefficient if many of the files are smaller than the chunk size.
* Title: Brief (one-line) No peer ID extension* Submitter: Olaf van der Spek* Email: OvdSpek@LIACS.NL* Category: Tracker - Bandwidth reduction* Compatibilty: Yes* Thread: -* Description: Each client adds &nopeerid=1& to the request URI it sends to the tracker. The tracker will exclude peer IDs from the output it sends to the client.* Pros: Tracker bandwidth usage is reduced* Cons: -Olaf: The compact tracker protocol already performs this bandwidth reduction, and more. (add compact=1 to the tracker query URI.)
* Title: Multi Hash Info* Submitter: Mirco Romanato* Email: painlord2k@yahoo.it* Category: Feature - Request* Compatibilty: Yes* Thread: -* Description: Over the usual metadata, insert in the .torrent file the metadata about the single files (SHA-1,TigerTree, CMD4, MD5). Would be better if this will be accomplished using magnet links, that can be extended, if needed, without changing the standard.* Pros: Sources of data can be located out from the torrent if the application is enabled (Gnutella/eDonkey200/etc.). The program could detect if files with the same hashes are present in shared/checked folders. This enable to build "virtual" torrent, where there is not real seeder (one or more of the leechers have only a part of the torrent or the original data will come from outside of the torrent), but leechers will cluster for the same data, will give priority to the torrent data exchange and will be able to build a strong request for the missing data with the external sources (they will ask for the same data in the same time, so there will be a major chance that the data will leack from the sources and will be served to the other leechers) . This is like the "horde" system in Overnet. Trackers are needed only as entry points of the torrent, this will reduce greatly the BW needing.* Cons: The last features is dungerous, because could produce a DDoS versus the data sources, if the applications behaviour is wrong.
* Title: listen_port-message* Submitter: Arvid Norberg* Email: c99ang@cs.umu.se* Category: Feature request* Compatibilty: Yes* Thread: -* Description: When a client connects to another peer and has a green light (a working listen-port) it tells that peer which port it is listening on. If the client doesn't have a grren light, or if it just get's an incoming connection, it will not send this information.* Pros: When a client is stopped, it can save all peers to which it can reconnect to a fast-resume data file. Without this extension, only 'local' connections can be saved. With this extension, some 'remote' connections can be saved too. The point of saving the peers ips and listen-ports is to be able to resume without the need of a tracker, in case it's down. It will also increase the number of peers you can excange in the presence of "Get Peers / Peers" extension.* Cons: a small overhead of a few bytes when connections are opened.* Note: already implemented as part of LTEP (the 'p' entry).
* Title: Connect To More Seeds Before Connecting To Peers* Submitter: /dev/null* Email: devnull2@ownmail.net* Category: Feature Request/Enhancement* Compatibilty: Yes, it should possibly be backwords compatible with the v1.0 spec* Thread: Have not joined the mailing list.* Description: It seems you always connect to more Peers than Seeds when you start downloading from the BT network, I believe that it would be better if you were connected to more Seeds at first and slowly connect to the Peers as the client searches the network for pieces you need and sends out the pieces you have to trade. Although, the tracker should connect you to more of the Seeds than it does at the moment, it seems that you always connect to more Peers than you do Seeds, even if there are more Seeds in the swarm than Peers. I don't think I am making myself very clear, but someone could probably figure out what I am trying to explain.* Pros: None detected.* Cons: Pointless and wrongheaded. To optimize the overall swarm transfer rate, downloading should be spread across as many hosts as possible. A swarm of clients preferentially concentrating their load onto seeds would overload the seeds and underutilize other peers. As long as the peers your client is connected to hold pieces you don't have, it doesn't matter whether or not they have a complete set; the client does not have to "search the network" for pieces it doesn't have, since peers continuously share their piece availability information.
* Title:BitTorrentAsProtocol* Submitter: Doug Ransom* Email: doug.ransom@alumni.uvic.ca* Category: Tracker -* Compatibilty: No* Thread:* Description: address a resource available via bittorrent with a"URI" such as torrent://tracker:port/announce* Pros: replaces ftp with bittorrent when appropriate. In spirit and purpose of adding new URI schemes.* Cons: Requires some standard way to express all the data normally present in a .torrent file in a form short enough to place in a URI. Likely requires significant protocol changes in order to do so.* Note: already implemented asmagnet:btih: URIs.
* Title: user-agent message* Submitter: WhitSlack* Email: mwhitlock@whitsoftdev.com* Category: Enhancement* Compatibilty: Yes, informational message only, could safely be ignored by non-supporting clients.* Thread: -* Description: Add a new message to the protocol to allow a client to identify itself after handshake and bitfield are sent.* Pros: Remove dependence on the encoding/decoding of the peerid field for determining remote peer client software and version. User-agent message would remove space constraints on client id strings; clients could identify themselves more like HTTP clients (e.g. "Azureus/2.1.0.4" rather than "-AZ2104-..."). This would simplify coding of clients as the user agent string could be displayed to the user directly rather than having to parse the peerid and look up a string to display to the user.* Cons:Slightly increased overhead. May confuse some poorly written 1.0 clients that don't ignore messages they don't understand.* Note: already implemented as part of LTEP (the 'v' entry).
* Title: proxy mode and proxy discovery, akaTorrentSquid* Submitter: DavidNicol* Email: wikitheoryorg@davidnicol.com* Category: Extension* Compatibilty: Yes, Extension, could safely be ignored by non-supporting clients.* Thread: -* Description: Provide a standard way to set up a proxying configuration, in which torrents are all downloaded to a shared torrent server which then serves the files more directly to clients on an internal network, who cannot offer uploading. Users behind impenetrable firewalls would be able to access torrent content, provided their network admins have been kind enough to set such a thing up. Not really a protocol extension, could be handled easily enough with some new software, the proxy server would have a torrent client on it and save all torrents somewhere where the inside systems could get the files via HTTP, and clients would have stub clients that inform the proxy which torrent they are interested in, and the stub waits for the proxy to get the torrent if it doesn't have it already, thenpulls it in from the beginning. The protocol extension would involve a protocol for automatic discovery of a torrent proxy on your network if one exists, which could be as simple as declaring a preferred host name. Everyone using helsinki.example.net for their internet service might look to _torrentproxy.helsinki.example.net for instance, at start-up, before trying to connect to provided tracker URLs directly. Of course a client would remember that no proxy was found and not check again until a week or two had passed.
* Pros: # reduces redundant upstream/downstream torrent traffic within an ISP # zero configuration through reserving a reccomended DNS name # definition ofProxyClient -TorrentSquid interface allows interoperability betweenProxyClients andTorrentSquids from different vendors, as well as unchoked download from localTorrentSquid into standard future torrent client which can then upload as currently, in non-firewalled environments.* Cons: # allows censorship at service provider level # advertises usage of client to ISP via DNS lookup (privacy violation)
* Title: Allow clients to act as "bandwidth donators" (please come up with a better name)* Submitter: poiutrew* Category: Enhancement* Compatibilty: Yes* Description: In order to improve their upload to download ratio users sometimes download files that have only a few seeders and many downloading peers, without actually wanting the file. This is a good thing. But it could be automated. What if users could select one or more trackers for the client to help when it has unused bandwidth. The client would then contact the tracker and ask for files that are in need of such help. The client would download pieces of the file and upload it to other peers. Only when it cannnot use all of its upload bandwith, would it try to download another piece.* Pros: # improves download speeds* Cons:
* Title: A BitTorrent Private Tracker Client "Accounting Error Message" Protocol Alert * Submitter: Max Power* Category: Enhancement* Compatibility: Can be made compatible with existing clients* Description:Abstract Private BitTorrent tracker websites have to obsess over what clients they choose to connect with. One of the primary reasons those that run private BitTorrent protocol tracker websites must obsess over what clients they choose to support is this : so many BT protocol clients don't correctly provide proper upload or download information to the tracker. Without proper accounting data from the client application proper upload and download ratios can be provided to their (paying) users. A protocol fix to this unpleasant situation is long overdue.Scope This protocol modification is not designed to be used with public trackers -- nor should it ever be used with public trackers, as public trackers are open to anyone to use and only engage in limited accounting of the activity of each Torrent. Public trackers most importantly do not store any user specific upload or download accounting data.http://hireme.geek.nz/bt-protocol-fix.html * Pros: Acts as a debugging system for BT clients* Cons: Increases protocol complexity
Part of