Subject: Re: Packet sizes [was: ANNOUNCE: libssh2 version 1.2.6]

Re: Packet sizes [was: ANNOUNCE: libssh2 version 1.2.6]

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 9 Aug 2010 15:53:49 +0200 (CEST)

On Mon, 9 Aug 2010, Peter Stuge wrote:

>>> Out of curiosity, where does this 32,500 byte limit come from? Is it a
>>> protocol limit?
>>
>> It was a bit taken out of the air
>
> The number came from me. I suggested to try it as buffer size to get as
> large packets at possible yet avoid another libssh2 bug, once the problems
> with >1024 had been fixed.

Perhaps you suggested that limit - I honestly don't remember, but my use of it
was not to avoid any bugs. libssh2 will simply not send any SSH packets that
are much larger than so, so passing on larger data sizes to those functions
are pointless.

>> All servers SHOULD support packets of at least 34000 bytes (where the
>
> That's SFTP, but the bug I'm talking about is in the transport layer and not
> SFTP specific.

Right. Let's take this step by step and see what the specs say!

> The length field that the abouve quote refers to is defined as:
>
> uint32 length

Yes, the protocol uses 32 bits for the length field.

> So the protocol limit is 2^32.

I strongly disagree. The length field has room for such a size, yes, but the
plain text is very clear in the spec I quoted: "This should allow for reads
and writes of at most 32768 bytes."

*at most* it says. In what situations are "at most" not the max?

> filexfer draft 13 (yes, later than 02) reads as follows, immediately before
> your quote:
>
> The maximum size of a packet is in practice determined by the
> client (the maximum size of read or write requests that it sends,
> plus a few bytes of packet overhead).

... and even draft 13 says (in section 4):

       This should allow for reads and writes of at most 32768 bytes.

Again: *at most*. If 32768 is the most, how can we send more? The text also
says that servers only need to support receving 34000 bytes so when we can we
know that it the server support larger packets? I don't know of any
negotiation field where it tells its maximum packet size limit.

> I think it's important for SFTP to support large packets but I think the
> transport layer is even more fundamental, and whatever solution is found for
> that will allow SFTP to follow suite.

I don't see how larger SFTP packets is allowed by the spec. I trust you will
tell me (us?) in what way my reading of this is wrong.

The packet size limit for the transport layer is similarly limited due to what
the spec says. RFC 4253 section 6.1 says:

    All implementations MUST be able to process packets with an
    uncompressed payload length of 32768 bytes or less and a total packet
    size of 35000 bytes or less (including 'packet_length',
    'padding_length', 'payload', 'random padding', and 'mac'). The
    maximum of 35000 bytes is an arbitrarily chosen value that is larger
    than the uncompressed length noted above.

This means that implementations are not required to support larger packets
than 32768 bytes uncompressed payload. When can libssh2 use larger packets
than what the other end is required to support? Should/can we probe it by
trying larger packets?

>> 64K is too large for a single packet
>
> No. What gives you this impression?

The above mentioned quote. How can we know that 64K works when it isn't
mandated by the spec?

-- 
  / daniel.haxx.se
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2010-08-09