Subject: Re: Enhanced buffering made faster uploads!

Re: Enhanced buffering made faster uploads!

From: Aris Adamantiadis <aris_at_0xbadc0de.be>
Date: Sun, 24 Oct 2010 20:10:27 +0200

Le 24/10/10 19:43, Daniel Stenberg a écrit :
> - The multiple outgoing packets thing. I think I'll proceed and do some
> experiments with a SFTP write function that (assuming that more than a
> certain amount of data is sent) sends the data in two SFTP packets,
> and as
> soon as the first packet is ACKed the function will return that amount.
>
> When the function is called again with a data pointer pointing to the
> second chunk, the SFTP write function is already waiting for the ACK for
> that so it sends off the second part only and returns as soon as the
> first
> part is ACKed etc.
>
> The idea here being that while waiting for an ACK we are better kept
> busy
> by sending the next part than by just waiting.
Hi Daniel

Your implementation gave good results (only 50% slower than openssh)
because you were running in on localhost. Doing it on a high-latency
connection (like a 100mbps path between europe and US) would give
catastrophic results.

Your bandwidth is limited by one chuck of data for each RTT, which
gives a upper bound of 35KB/RTT ~= 350KB/s on a 100ms latency link.

The only way to avoid this is to use asynchronous SFTP read and writes
(that is, you send several requests without waiting for the responses).
So clearly it's the way to go.

libssh does this (despite its supposed terrible performances) since
0.4. You can have a look at the current API being used here:

http://api.libssh.org/master/group__libssh__sftp.html#ga4c2cca19b38c54d4b04bb3584b0517cd

I'm not that happy of this API (difficult to use) but it exists and I
know at least one application that uses it.

Kr,

Aris
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2010-10-24