New Protocol work


#1

Hi All,

We are working on new protocol that we’re calling the “Asset Transfer Protocol” (atp://). This will be a system to store and serve content over UDP that we are currently only able to serve via HTTP/HTTPS. We hope to make it easier for domain owners to quickly serve their content files to incoming viewers and to be able ask for help from extra assignment-clients to more quickly serve those files to more viewers.

To reliably and quickly serve files over UDP we’re adopting the UDT protocol (http://udt.sourceforge.net/). Although UDT has a library, we unfortunately can’t use it because it prefers to assume total control of a network socket and we need our hole punched socket to carry all network communication. We’ll be building what UDT does for reliable congestion controlled packet sending as an option into the network library - the first use of this will likely be for the asset-server but we imagine there are other packet types we would pretty quickly want to move over to UDT.

We’re aiming to have our first mergeable pull request well tested and ready to go Wednesday. So far we have been refactoring the existing networking code (packet creation and packet transfer) to create a good foundation to add UDT on top of. We believe the new API will make it a lot easier to send and receive all types of packets (including unreliable packets and packets that break over MTU boundaries).

I will talk more about this, tomorrow at the Alpha meetup.

Chris


#2

Cool, will we get more details on this as well written after the meeting tommorrow?
Would be cool to understand where exactly would we put the assets in:

As in would there be any change the data hosting at all?

With a distributed system, there is also the nature of cache. How can we make sure everyone has latest?

Also The distributed nature would help with the bandwidth issue, but one of the other concerns over the current http(s) method is that it is very easy just to clients sniff out the urls of the entities.

I know a few content creators that are simply put off by end users being able to copy stuff easily (just get the URL) when they’d expect such to require model capture tools / hacked clients to be able to do so.


#3

A distributed, congestion controlled, and reliable protocol like UDT seems like a very nice fit for asset transfer. It doesn’t deal with attestation and access control, which it should not. I think, though, that it is time to start seriously thinking about access control soonest, though understandably not in this topic.


#4

http://learnchile.cl/wp-content/uploads/2012/08/udp.png

seems straightforward enough to not require further explanation