Subject: Re: libssh2_sftp_read reads a number of bytes smaller than both the file size and the specified buffer size

Re: libssh2_sftp_read reads a number of bytes smaller than both the file size and the specified buffer size

From: Alexander Lamaison <swish_at_lammy.co.uk>
Date: Thu, 29 Mar 2012 13:52:49 +0100

On 29 March 2012 01:19, Adam Craig <adamgcraig_at_accessnoexcuse.org> wrote:
> Hello Libssh2 Dev Mailing List,
>
> I am working on a cocoa application that uses libssh2 to SFTP files to
> and from a server.

... snip

> I have been testing this method with both small text files of a few kB
> each and Quicktime .mov files of up to 1MB. I am running the app on
> 64-bit Intel OS X 10.6.8 on the client side and communicating with a
> server running the standard linux sftp-server on Redhat Enterprise
> Linux Server 5.6. The version of libssh2 I am using is 1.4.0-20120229.
>
> For all files I have tested, the method downloads and returns the
> correct data, but it always returns it in chunks no larger than 2K,
> regardless of how large I make the buffer and regardless of the size
> of the file. More precisely, if the file is smaller than 2K, the first
> call downloads the complete file and returns the number of bytes in
> the file, but, if the file is larger than 2K, libssh2_sftp_read
> returns a value of 2000 after every call except for the second-to-last
> call, which returns (file size)%2000, and the last call, which returns
> 0, indicating we have read all the bytes in the file. At first, I
> thought it might be a peculiarity of the server's configuration, but I
> do not see any hard limit on sent packet size in sftp-server's
> documentation. Is this an inherent limitation of the libssh2_sftp_read
> method?

So just to summarise, you're not reporting that the transfer is
incomplete? Just that you wish the transfer used bigger chunks at a
time? This is a perfectly valid thing to report, I just want to check
I'm understanding you correctly.

... snip

> I am testing this with the same client macbook and server and the same
> version of libssh2. If I upload a file smaller than 30K,
> libssh2_sftp_write, the program tries to upload it all in one call,
> succeeds, and returns the correct number of bytes. If I upload a file
> larger than 30K but smaller than packet_size, it attempts to upload
> all the data in one call. The complete file appears on the server, but
> libssh2_sftp_write only returns 30000. If I upload a file larger than
> the packet size, each full-packet call returns 30000, causing the loop
> to slide the window forward by 30000. Once it gets to a point where
> fewer than packet_size bytes remains, it jumps to the last call and
> tries to upload all the remaining bytes. This call returns up to
> 30000. When I check the server, it has the complete file.
> It appears that the return value and the amount of data uploaded do not match.

This sounds like the expected behaviour of libssh2's pipelined SFTP
architecture. The return value doesn't have to correspond to the
amount uploaded on _that_ call as long as the overall total is
correct. The last call that doesn't appear to do anything allows
libssh2 to do some housekeeping.

> I would like to be able to download larger packets

I'm not sure why yours are limited to 2K but I believe this is
negotiated with the server (jump in here guys if I'm wrong) so it's
possible your server is refusing to send bigger packets.

Alex

-- 
Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2012-03-29