Subject: Re: large file support in sftp

Re: large file support in sftp

From: Alexander Lamaison <swish_at_lammy.co.uk>
Date: Tue, 1 Jun 2010 16:01:26 +0100

On 1 June 2010 15:30, <Martin.Dommermuth_at_inovasec.de> wrote:
>
> I use the following script to get the size of a remote file:
>
> use Net::SSH2;
> use Data::Dumper;
>
> my $ssh2 = Net::SSH2->new();
>
> $ssh2->connect('172.1.1.1', 222) or die $!;
>
> if ($ssh2->auth_publickey('dummy', '/tmp/key.pub', '/tmp/key.priv' ))
> {
>       my $sftp = $ssh2->sftp();
>       my $fileinfo = $sftp->stat('/tmp/file.big') or die;
>       print Dumper($fileinfo);
> }
>
> I tried this script again with Ubuntu 10.04. versions:
> Kernel 2.6.32-22-generic #33-Ubuntu SMP
> libssh2  1.2.2.
> OpenSSH_5.3p1 Debian-3ubuntu3
> OpenSSL 0.9.8k
> Perl Net:SSH2 0.28
>
> My test before failed when the file size exceeded 2047MB. Now I hit a limit
> at 4095MB.
>
> 4095MB:
> # perl test.pl
> $VAR1 = {
>           'uid' => 0,
>           'mtime' => 1275400507,
>           'mode' => 33279,
>           'name' => '/tmp/file.big',
>           'atime' => 1275400080,
>           'gid' => 0,
>           'size' => 4293918720
>         };
>
> 4096MB:
> # perl test.pl
> $VAR1 = {
>           'uid' => 0,
>           'mtime' => 1275400990,
>           'mode' => 33279,
>           'name' => '/tmp/file.big',
>           'atime' => 1275400982,
>           'gid' => 0,
>           'size' => 0
>         };

I believe the problem is in the Perl wrapper. It transforms the
LIBSSH2_SFTP_ATTRIBUTES structure into a Perl object with this
function:

static HV* hv_from_attrs(LIBSSH2_SFTP_ATTRIBUTES* attrs) {
    HV* hv = newHV();
    debug("hv_from_attrs: attrs->flags = %d\n", attrs->flags);
    if (attrs->flags & LIBSSH2_SFTP_ATTR_SIZE)
        hv_store(hv, "size", 4, newSVuv(attrs->filesize), 0/*hash*/);
    if (attrs->flags & LIBSSH2_SFTP_ATTR_UIDGID) {
        hv_store(hv, "uid", 3, newSVuv(attrs->uid), 0/*hash*/);
        hv_store(hv, "gid", 3, newSVuv(attrs->gid), 0/*hash*/);
    }
    if (attrs->flags & LIBSSH2_SFTP_ATTR_PERMISSIONS)
        hv_store(hv, "mode", 4, newSVuv(attrs->permissions), 0/*hash*/);
    if (attrs->flags & LIBSSH2_SFTP_ATTR_ACMODTIME) {
        hv_store(hv, "atime", 5, newSVuv(attrs->atime), 0/*hash*/);
        hv_store(hv, "mtime", 5, newSVuv(attrs->mtime), 0/*hash*/);
    }
    return hv;
}

See how the file size is passed to newSVuv(). This is why it fails.
newSVuv takes a UV as its argument which is defined as being the size
of a pointer. When running on x86, as I assume you are, this is only
32-bits - not enough to hold a 64-bit file size.

I'm afraid I don't know who is maintaining NET::SSH2.

Alex
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2010-06-01