From libssh2-devel-bounces@cool.haxx.se Sat Jun 7 23:17:10 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s57LGaDx015084; Sat, 7 Jun 2014 23:17:03 +0200 Received: from nm33.bullet.mail.sg3.yahoo.com (nm33.bullet.mail.sg3.yahoo.com [106.10.151.28]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s57LGVVC014979 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sat, 7 Jun 2014 23:16:33 +0200 Received: from [106.10.166.122] by nm33.bullet.mail.sg3.yahoo.com with NNFMP; 07 Jun 2014 21:16:27 -0000 Received: from [106.10.151.234] by tm11.bullet.mail.sg3.yahoo.com with NNFMP; 07 Jun 2014 21:16:27 -0000 Received: from [127.0.0.1] by omp1018.mail.sg3.yahoo.com with NNFMP; 07 Jun 2014 21:16:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 367951.37603.bm@omp1018.mail.sg3.yahoo.com Received: (qmail 7003 invoked by uid 60001); 7 Jun 2014 21:16:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ymail.com; s=s1024; t=1402175787; bh=JSyinbXWLdLBKV4RG3Y6c7qovWorXF8+5XiJIOYTHQ4=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=GWvzLVb5P08Lftmuvp7FAo7iN2ILpwjZQn+EH2RafXjCXX8ImQ0SLsWl/MYEVKcIjSLgtxm0nPIw5wx8sJjHIIMN0xGeL1WkCPXbOdVjtu4eV+WAasAg7U1TJ7u9g0MWU/epxaP7iWMx32qJB020SQXl16xDWxlaBPyWaIM8+ww= X-YMail-OSG: AUwi9hcVM1nz5ERiR4ka8nGbDuJSM.e0sBvmHy4GksTvju9 1.z.gyuYXoP_vlDeypXiar8hZQ0iYRmE3Q0fbX928tPfWIfbUcZtHScSPKeC CJuQy2T3TjD1OsvzCX1je5HR.ISWijg3Je_ExPPyI8OS8pCUvCjZBy9IJmDJ GQbmmKzl3XggAEj5lR8wOCiQkUoZ4T32ct3RKfkh7mNPHIFur4XxxGWLZSSS rK9_85urvTz8jBQyASgVAzvB0CCEU.xsPRwlCn.zQoimbHZT0NsOCsdwyOJR JKJk1nRxKEPLl4Ykptt0gbP.VB2bdbGMj7Z8MTAZvSk48aEDgsX9ykNGOfhw rT8cZAbkXyCFnGI5iBJh1_W3WoqtpKUSNCKMg4g_BWBVnHaX5FF_2aQGi2Hl cWOqOzMEvwm3sbtHLNtDwQFIPVTYV3T08mFNrRQaZGRzrdHfXpAlCMp86sJd U2GR.3vt.5fehVPhvRVVg4BOkOYhntsleburBQhncsAWzZFLij7TvPjQPuiS woGPCmCTGvh9aezepAXEloaJSyYlQ8ImZCkX4v4w_Zp3tU_qN1S1WGxQKyKY 3huV2jjkEpAJPKxObRsnrZsXHjbc3aEPLXPJ7fSI7VrFL61F0wyM8FQQXtXu .ebLEG82qFqYHtBbfDFLg1J4i2SKJuBssJx_dn.i2FEm.9N9jlDwT1exMFV1 08Au3Ce1WYlAg9Djs05qX0tpEWqIiGY6gIZYpv9elmoer7DdcbuFhLmP5z8N RQ9ElHC.jAUqecg-- Received: from [216.240.30.4] by web192706.mail.sg3.yahoo.com via HTTP; Sun, 08 Jun 2014 05:16:27 SGT X-Rocket-MIMEInfo: 002.001, SGkswqAKSSdtIHRyeWluZyB0byB3cml0ZSBhIGV2ZW50IGRyaXZlbiBwcm9ncmFtbWluZyAod3JpdHRlbiBpbiBwZXJsIHRocm91Z2ggTmV0OjpTU0gyIGFuZCBFVikgd2hlcmUgaWYgc29tZSBkYXRhIGlzIGF2YWlsYWJsZSBvbiBzb2NrZXQgSSBnbyBvdmVyIGFsbCB0aGUgY2hhbm5lbHMgaW4gbm9uLWJsb2NraW5nIHdheSB0byBzZWUgaWYgdGhlcmUgaXMgc29tZSBkYXRhIHRoYXQgY2FuIGJlIHJlYWQuwqAKCkknbSBzZWVpbmcgaXNzdWVzIHdpdGggYWJvdmUgYXBwcm9hY2ggaW4gc29tZSBjYXNlcyB3aGUBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 Message-ID: <1402175787.63370.YahooMailNeo@web192706.mail.sg3.yahoo.com> Date: Sun, 8 Jun 2014 05:16:27 +0800 (SGT) From: sudheer kumar Subject: libssh2 and select on multiple channels To: "libssh2-devel@cool.haxx.se" MIME-Version: 1.0 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: sudheer kumar , libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0746942300==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============0746942300== Content-Type: multipart/alternative; boundary="-897285659-781705372-1402175787=:63370" ---897285659-781705372-1402175787=:63370 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi,=A0=0AI'm trying to write a event driven programming (written in perl th= rough Net::SSH2 and EV) where if some data is available on socket I go over= all the channels in non-blocking way to see if there is some data that can= be read.=A0=0A=0AI'm seeing issues with above approach in some cases where= socket is not shown as readable with select but when tried to read channel= , it is available (I expected select to return before i did read on channel= ).=A0=0A=0AWhile browsing archives I found below one discussed back in 2010= with related topic:=A0=0Ahttp://www.libssh2.org/mail/libssh2-devel-archive= -2010-06/0040.shtml=A0=0A=0A=0ADid anyone try other approach that works whe= re select always says when something can be read from channel?=A0=0A=A0=0A= =0ADoes above approach work at all, as we have only one socket and multiple= channels of it. And each read from channel should read from socket, and th= at is affecting select because if we have 'n' channels in a session, trying= read on nth channel could have read data from socket which belongs to 1st = channel and is stored in buffer and that affecting select to say socket is = readable later?=0A=0AAny leads on this topic if at all discussed before is = greatly appreciated.=A0=0A=0A=0AThanks,=0ASudheer=A0=0A ---897285659-781705372-1402175787=:63370 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi, 
I'm trying to write a event= driven programming (written in perl through Net::SSH2 and EV) where if some dat= a is available on socket I go over all the channels in non-blocking way to = see if there is some data that can be read. 

I'm seeing issues with above approa= ch in some cases where socket is = not shown as readable with select but when tried to read channel, it is ava= ilable (I expected select to return before i did read on channel). 

While browsing archives I= found below one discussed back in 2010 with related topic: 
http://www.libssh2.org/mail/libssh2-devel-archive-2010-06/0040.shtm= l 

 <= /span>
Does above approach work at all, as = we have only one socket and multiple channels of it. And each read from cha= nnel should read from socket, and that is affecting select because if we have '= n' channels in a session, trying read on nth channel could have read data f= rom socket which belongs to 1st channel and is stored in buffer and that af= fecting select to say socket is readable later?

Any le= ads on this topic if at all discussed before is greatly appreciated. <= /span>

Tha= nks,
Sudheer 
---897285659-781705372-1402175787=:63370-- --===============0746942300== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============0746942300==-- From libssh2-devel-bounces@cool.haxx.se Sat Jun 14 03:07:43 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s5E17B8D010818; Sat, 14 Jun 2014 03:07:36 +0200 Received: from mail-qc0-f179.google.com (mail-qc0-f179.google.com [209.85.216.179]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s5E177F1010603 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Sat, 14 Jun 2014 03:07:08 +0200 Received: by mail-qc0-f179.google.com with SMTP id x3so3841275qcv.38 for ; Fri, 13 Jun 2014 18:07:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:mime-version:thread-index:date:message-id :subject:to:cc:content-type; bh=fCc1FVxUPaAvE3n9BroKPkLC0vbEzrhluUJ76bHgIXU=; b=AlD+yTF+mhdebV3I5KsAFIGwca6wmlW5hO45Q8jIAljgdA6B97pr65oQVIJDiqATj7 D1HCVEgXeTnUD67bg/xH4ifGj3EwfINyw98KTZ+D2bx2VjRs/vYypGiKYZbEKSMCmSG/ 5bw0FCicgSKPOU2Kmhd3MKF1xUkn5L5rSgzYm4vEcriQWgf8Zlegt2h8hDbA9RqqKIJN ABJ7f+R6pjx+hbmrZJo/x+n6RyZ0YJ+T4N9BG7wwI7QPFrjuf+HaWUx+DKtsex/FAUJs a3WRdO1Tjadpn/qrhBMOhoIywQQYRyav4qye+6BtwfvW+bA3IpsIdjBXvGL3ZfGyXzha 39fg== X-Gm-Message-State: ALoCoQlsS5qoRRkqoeleddBnpjJJzDOxrq5DX01OHDd8PTrmXMpTD9QvScT1objsvOBOkhxeb1BR X-Received: by 10.140.30.195 with SMTP id d61mr1039357qgd.62.1402708023721; Fri, 13 Jun 2014 18:07:03 -0700 (PDT) From: Nitin Deokate MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac+HareMEg1+LTv+RhqIbwMAnKp4Lw== Date: Sat, 14 Jun 2014 06:21:03 +0530 Message-ID: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> Subject: Questions about libssh2_sftp_read() To: libssh2-devel@cool.haxx.se Cc: Nitin Deokate X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2005851356==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============2005851356== Content-Type: multipart/alternative; boundary=001a113a6e867a846504fbc1692f --001a113a6e867a846504fbc1692f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Guys, I have couple questions to the libssh2 developers: 1. I have an application, where I use libssh2_sftp_read(), and I pass larger buffer(say 8K to 16MB) to same function, What I expect is, data of same bytes, but all I get is 2000Bytes. What could help me to get as equal to the buffer size I passed and not 2000 bytes? 2. Is it any significant reason for selecting value for #define MAX_SFTP_READ_SIZE 2000 Why it can=E2=80=99t have more bytes than that? Has anybody faced this scenario before, please revert as early as possible. Thanks, Nitin --001a113a6e867a846504fbc1692f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Guys,

I have couple questions to the libssh2 developers:

1.=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 I have an application, where I= use l= ibssh2_sftp_read(), and I pass larger buffer(say 8K to 16MB) to same= function,

What I expect is, data of same bytes, but all= I get is 2000Bytes.

What could help me to= get as equal to the buffer size I passed and not 2000 bytes?

=C2=A0=

2.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 Is it any significant reason for selecting value for

#define MAX_SFTP_READ_SIZE 2000

Why it can=E2=80=99t have more bytes than that?

=C2=A0

Has anybody faced this scenario before, pl= ease revert as early as possible.

Thanks,

Nitin

--001a113a6e867a846504fbc1692f-- --===============2005851356== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============2005851356==-- From libssh2-devel-bounces@cool.haxx.se Sat Jun 14 23:04:41 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s5EL4Car025719; Sat, 14 Jun 2014 23:04:36 +0200 Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-4.1) with ESMTP id s5EL49ON025631 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 14 Jun 2014 23:04:09 +0200 Received: from localhost (dast@localhost) by giant.haxx.se (8.14.4/8.14.4/Submit) with ESMTP id s5EL49LT025627; Sat, 14 Jun 2014 23:04:09 +0200 X-Authentication-Warning: giant.haxx.se: dast owned process doing -bs Date: Sat, 14 Jun 2014 23:04:09 +0200 (CEST) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Questions about libssh2_sftp_read() In-Reply-To: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> Message-ID: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="1129329158-1786761946-1402779840=:321" Content-ID: Cc: Nitin Deokate X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --1129329158-1786761946-1402779840=:321 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-7; FORMAT=flowed Content-Transfer-Encoding: 8BIT Content-ID: On Sat, 14 Jun 2014, Nitin Deokate wrote: > 1. I have an application, where I use libssh2_sftp_read(), and I pass > larger buffer(say 8K to 16MB) to same function, > > What I expect is, data of same bytes, but all I get is 2000Bytes. No, that is probably all you get in the first read call. That's quite a difference. In subsequent reads you are likely to get larger pieces. > What could help me to get as equal to the buffer size I passed and not 2000 > bytes? If you have less latency to the server you may get more, but the first call is likely to always just give you a small piece. > 2. Is it any significant reason for selecting value for > > #define MAX_SFTP_READ_SIZE 2000 > > Why it can¢t have more bytes than that? It can, just bump it. But you will not get the amount you ask for at once, you can up to that amount, then you call the function over and over again until you're done. > Has anybody faced this scenario before, please revert as early as possible. I don't even understand your scenario. We have users downloading insane amounts of data over SFTP with no problems. This is not a known problem you're talking about. The 2K number is simply the "block size". libssh2 sends read MANY requests with that size and as soon as one comes back it can return data. When you call read again, more packets might already have arrived and can be returned and so on. I blogged about this technique when I made this change: http://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers-fast/ -- / daniel.haxx.se --1129329158-1786761946-1402779840=:321 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --1129329158-1786761946-1402779840=:321-- From libssh2-devel-bounces@cool.haxx.se Sun Jun 15 02:35:26 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5F0YZka018746; Sun, 15 Jun 2014 02:35:19 +0200 Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5F0YVFV018721 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Sun, 15 Jun 2014 02:34:32 +0200 Received: by mail-qa0-f41.google.com with SMTP id cm18so5627847qab.28 for ; Sat, 14 Jun 2014 17:34:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=51O9QqsJFUtNLUT4wd6XT8boLHZ07LSPYya4ZFz1ZLk=; b=EtH0NJbI8DdOOcronrfFPASXT4H402oWvc1Skka9VuRtiOEqFpRmFLQpuh/EpMfU9c fNCp4kGE/guGfh+qW0yAGTY9B8iJ5s5N7OgP5u0V+SP3xNH3IeqQqtqD58D58TkQ2ExB 8ydy3WbphW1T00qAO1Ug7dHx+jxsHliTW89P7amFFmLOuiFSvpZQkMbVdiWAvttnKga8 wsRMQAKAkNb1s2N/RxnnlBu+6prGIwAmfPfCQLrJRNb84lPwBffBKm5FXOfPLaBK7Pzr oY/ieLAtn4Z97Iilo4UpNn46sNr+WauHQUBHnR53FbON7ilXsKX373wLNF3+3Fo2ZFRi Cl1g== X-Gm-Message-State: ALoCoQlyQHSUye8vjZ8MF/Z88rqE/raFMHSYOzijim0XUmdfPuVmjxt4yHJQOQqr7/pOrUSLE7LX MIME-Version: 1.0 X-Received: by 10.140.50.168 with SMTP id s37mr14564142qga.36.1402792472409; Sat, 14 Jun 2014 17:34:32 -0700 (PDT) Received: by 10.229.128.7 with HTTP; Sat, 14 Jun 2014 17:34:32 -0700 (PDT) In-Reply-To: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> Date: Sun, 15 Jun 2014 06:04:32 +0530 Message-ID: Subject: Re: Questions about libssh2_sftp_read() From: Nitin Deokate To: Daniel Stenberg Cc: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1098870503==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============1098870503== Content-Type: multipart/alternative; boundary=001a1135247203143204fbd5138a --001a1135247203143204fbd5138a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for the reply. In subsequent reads too, I get smaller chunk. Could you please suggest what can be done in my code to have more subsequent calls? any test case you have ? On Sun, Jun 15, 2014 at 2:34 AM, Daniel Stenberg wrote: > On Sat, 14 Jun 2014, Nitin Deokate wrote: > > 1. I have an application, where I use libssh2_sftp_read(), and I >> pass >> larger buffer(say 8K to 16MB) to same function, >> >> What I expect is, data of same bytes, but all I get is 2000Bytes. >> > > No, that is probably all you get in the first read call. That's quite a > difference. In subsequent reads you are likely to get larger pieces. > > > What could help me to get as equal to the buffer size I passed and not >> 2000 bytes? >> > > If you have less latency to the server you may get more, but the first > call is likely to always just give you a small piece. > > > 2. Is it any significant reason for selecting value for >> >> #define MAX_SFTP_READ_SIZE 2000 >> >> Why it can=E2=80=99t have more bytes than that? >> > > It can, just bump it. But you will not get the amount you ask for at once= , > you can up to that amount, then you call the function over and over again > until you're done. > > > Has anybody faced this scenario before, please revert as early as >> possible. >> > > I don't even understand your scenario. We have users downloading insane > amounts of data over SFTP with no problems. This is not a known problem > you're talking about. > > The 2K number is simply the "block size". libssh2 sends read MANY request= s > with that size and as soon as one comes back it can return data. When you > call read again, more packets might already have arrived and can be > returned and so on. I blogged about this technique when I made this chang= e: > > http://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers-fast/ > > -- > > / daniel.haxx.se --001a1135247203143204fbd5138a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for the reply.
In subsequent reads = too, I get smaller chunk.
Could you please suggest what can be done in my code to have more
subsequent calls? any test case you have ?


On Sun, Jun 15, 2014 at= 2:34 AM, Daniel Stenberg <daniel@haxx.se> wrote:
On Sat, 14 Jun 2014, Nitin D= eokate wrote:

1. =C2=A0 =C2=A0 =C2=A0 I have an application, where I use libssh2_sftp_rea= d(), and I pass
larger buffer(say 8K to 16MB) to same function,

What I expect is, data of same bytes, but all I get is 2000Bytes.

No, that is probably all you get in the first read call. That's quite a= difference. In subsequent reads you are likely to get larger pieces.


What could help me to get as equal to the buffer size I passed and not 2000= bytes?

If you have less latency to the server you may get more, but the first call= is likely to always just give you a small piece.


2. =C2=A0 =C2=A0 =C2=A0 Is it any significant reason for selecting value fo= r

#define MAX_SFTP_READ_SIZE 2000

Why it can=E2=80=99t have more bytes than that?

It can, just bump it. But you will not get the amount you ask for at once, = you can up to that amount, then you call the function over and over again u= ntil you're done.


Has anybody faced this scenario before, please revert as early as possible.=

I don't even understand your scenario. We have users downloading insane= amounts of data over SFTP with no problems. This is not a known problem yo= u're talking about.

The 2K number is simply the "block size". libssh2 sends read MANY= requests with that size and as soon as one comes back it can return data. = When you call read again, more packets might already have arrived and can b= e returned and so on. I blogged about this technique when I made this chang= e:

=C2=A0 http://daniel.haxx.se/blog/2010/12/08/ma= king-sftp-transfers-fast/

--

=C2=A0/ daniel.haxx.se<= /a>

--001a1135247203143204fbd5138a-- --===============1098870503== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============1098870503==-- From libssh2-devel-bounces@cool.haxx.se Thu Jun 19 19:27:47 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5JHRCFJ007683; Thu, 19 Jun 2014 19:27:40 +0200 Received: from mx.uxnr.de (mx.uxnr.de [IPv6:2a00:1828:2000:378:2525:0:59ee:542f]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5JHRBOl007602 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 19 Jun 2014 19:27:11 +0200 Received: from [10.2.2.10] (MH01.ma01.uxnr.net [10.2.2.10]) by mx.uxnr.de (Postfix) with ESMTPSA id C817B1C5A27C for ; Thu, 19 Jun 2014 19:27:01 +0200 (CEST) X-DKIM: OpenDKIM Filter v2.6.8 mx.uxnr.de C817B1C5A27C DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=marc-hoersken.de; s=picard; t=1403198821; bh=1D0sA6gV3CF4YqNU83eT2hxTwhdvDFJ4vevULccKAVs=; h=Date:From:To:Subject:References:In-Reply-To:From; b=OiK3Bb3m9qSE/F8lE9qzAngOHAPcbBPqv+4cyAnPESFI2mDiYIgZQU0n6HEN1rzMQ iIlhXSthBhxNGP/fOgirQJMnPh7148w1Qx2SO0WTLA8uuJGqcUvOzWHABC8ON6xiOm BscH1PnWnk0ms8OO0Ah1Uvv8i3EdwIFxKiSKD2uY= Message-ID: <53A31D64.2020808@marc-hoersken.de> Date: Thu, 19 Jun 2014 19:27:00 +0200 From: Marc Hoersken User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Re: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend References: <5325F5FF.2060703@marc-hoersken.de> <5378B273.3040500@marc-hoersken.de> <5378E9FA.4000309@marc-hoersken.de> In-Reply-To: <5378E9FA.4000309@marc-hoersken.de> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------080602030409000906020601" X-Spam-Status: No, score=-1.1 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on picard.vpn.uxnr.de X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This is a multi-part message in MIME format. --------------080602030409000906020601 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hello everyone, attached you will find a new patch which takes Daniel's feedback into account: - Replaced random overwrite with secure zeroing using SecureZeroMemory [2] - Option renamed from --disable-memory-overwrite to --disable-clear-memory - Define renamed from LIBSSH2_MEMORY_OVERWRITE to LIBSSH2_CLEAR_MEMORY On 18.05.2014 19:12, Marc Hoersken wrote: > On 18.05.2014 19:02, Daniel Stenberg wrote: >> This option only disables the random fill of the free data, it still >> overwrites memory - only with zeros instead. So it doesn't disable >> memory overwrite at all. > You are right, [snip] > >> A question though: is there really anyone who suggests that it is >> safer to fill the data with random data rather than just zeros? I just >> can't see the point with doing such a slow operation and waste random >> seed on this. > I don't have specific expertise in this area, but I think a reason could > be that a compiler might be tempted to optimize memset(buf, 0, len) out. > > Looking at the memory erasure procedure of the Tails operating system > [1], it seems like overwriting with zeros is enough. The feature now overwrites the data in internal memory buffers with zeros using the secure functionality provided by the OS. If this feature is ever expanded to other backends and of course different operating systems, such a function would need to be provided and used. Thanks, Daniel! Please review the new patch. Any feedback is welcome. I guess the patch should also include some warning about it only being available for Windows with WinCNG for now before being merged at the current stage of the implementation. Best regards, Marc [1] https://tails.boum.org/contribute/design/memory_erasure/ [2] http://msdn.microsoft.com/en-us/library/windows/desktop/aa366877.aspx --------------080602030409000906020601 Content-Type: text/plain; charset=windows-1252; name="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.pa"; filename*1="tch" From 137b97b84137d1aee7849a31a15e1e4786dfb0f6 Mon Sep 17 00:00:00 2001 From: Marc Hoersken Date: Thu, 19 Jun 2014 18:54:10 +0200 Subject: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend This re-introduces the original feature proposed during the development of the WinCNG crypto backend. It still needs to be added to libssh2 itself and probably other backends. Memory is cleared using the function SecureZeroMemory which is available on Windows systems, just like the WinCNG backend. --- configure.ac | 7 +++ src/wincng.c | 146 +++++++++++++++++++++++++++++++++-------------------------- 2 files changed, 88 insertions(+), 65 deletions(-) diff --git a/configure.ac b/configure.ac index ba4dd7a..cbc2312 100644 --- a/configure.ac +++ b/configure.ac @@ -197,6 +197,13 @@ if test "$GEX_NEW" != "no"; then AC_DEFINE(LIBSSH2_DH_GEX_NEW, 1, [Enable newer diffie-hellman-group-exchange-sha1 syntax]) fi +AC_ARG_ENABLE(clear-memory, + AC_HELP_STRING([--disable-clear-memory],[Disable clearing of memory before being freed]), + [CLEAR_MEMORY=$enableval]) +if test "$CLEAR_MEMORY" != "no"; then + AC_DEFINE(LIBSSH2_CLEAR_MEMORY, 1, [Enable clearing of memory before being freed]) +fi + dnl ************************************************************ dnl option to switch on compiler debug options dnl diff --git a/src/wincng.c b/src/wincng.c index b575350..5dfca52 100644 --- a/src/wincng.c +++ b/src/wincng.c @@ -280,6 +280,20 @@ _libssh2_wincng_random(void *buf, int len) return BCRYPT_SUCCESS(ret) ? 0 : -1; } +static void +_libssh2_wincng_mfree(void *buf, int len) +{ + if (!buf) + return; + +#ifdef LIBSSH2_CLEAR_MEMORY + if (len > 0) + SecureZeroMemory(buf, len); +#endif + + free(buf); +} + /*******************************************************************/ /* @@ -322,7 +336,7 @@ _libssh2_wincng_hash_init(_libssh2_wincng_hash_ctx *ctx, pbHashObject, dwHashObject, key, keylen, 0); if (!BCRYPT_SUCCESS(ret)) { - free(pbHashObject); + _libssh2_wincng_mfree(pbHashObject, dwHashObject); return -1; } @@ -355,11 +369,11 @@ _libssh2_wincng_hash_final(_libssh2_wincng_hash_ctx *ctx, ret = BCryptFinishHash(ctx->hHash, hash, ctx->cbHash, 0); BCryptDestroyHash(ctx->hHash); + ctx->hHash = NULL; - if (ctx->pbHashObject) - free(ctx->pbHashObject); - - memset(ctx, 0, sizeof(_libssh2_wincng_hash_ctx)); + _libssh2_wincng_mfree(ctx->pbHashObject, ctx->dwHashObject); + ctx->pbHashObject = NULL; + ctx->dwHashObject = 0; return ret; } @@ -403,11 +417,11 @@ void _libssh2_wincng_hmac_cleanup(_libssh2_wincng_hash_ctx *ctx) { BCryptDestroyHash(ctx->hHash); + ctx->hHash = NULL; - if (ctx->pbHashObject) - free(ctx->pbHashObject); - - memset(ctx, 0, sizeof(_libssh2_wincng_hash_ctx)); + _libssh2_wincng_mfree(ctx->pbHashObject, ctx->dwHashObject); + ctx->pbHashObject = NULL; + ctx->dwHashObject = 0; } @@ -449,17 +463,17 @@ _libssh2_wincng_key_sha1_verify(_libssh2_wincng_key_ctx *ctx, _libssh2_wincng.hAlgHashSHA1, hash, hashlen); - free(data); + _libssh2_wincng_mfree(data, datalen); if (ret) { - free(hash); + _libssh2_wincng_mfree(hash, hashlen); return -1; } datalen = sig_len; data = malloc(datalen); if (!data) { - free(hash); + _libssh2_wincng_mfree(hash, hashlen); return -1; } @@ -474,8 +488,8 @@ _libssh2_wincng_key_sha1_verify(_libssh2_wincng_key_ctx *ctx, ret = BCryptVerifySignature(ctx->hKey, pPaddingInfo, hash, hashlen, data, datalen, flags); - free(hash); - free(data); + _libssh2_wincng_mfree(hash, hashlen); + _libssh2_wincng_mfree(data, datalen); return BCRYPT_SUCCESS(ret) ? 0 : -1; } @@ -568,7 +582,7 @@ _libssh2_wincng_asn_decode(unsigned char *pbEncoded, pbEncoded, cbEncoded, 0, NULL, pbDecoded, &cbDecoded); if (!ret) { - free(pbDecoded); + _libssh2_wincng_mfree(pbDecoded, cbDecoded); return -1; } @@ -638,7 +652,7 @@ _libssh2_wincng_asn_decode_bn(unsigned char *pbEncoded, *ppbDecoded = pbDecoded; *pcbDecoded = cbDecoded; } - free(pbInteger); + _libssh2_wincng_mfree(pbInteger, cbInteger); } return ret; @@ -683,10 +697,10 @@ _libssh2_wincng_asn_decode_bns(unsigned char *pbEncoded, *pcbCount = length; } else { for (length = 0; length < index; length++) { - if (rpbDecoded[length]) { - free(rpbDecoded[length]); - rpbDecoded[length] = NULL; - } + _libssh2_wincng_mfree(rpbDecoded[length], + rcbDecoded[length]); + rpbDecoded[length] = NULL; + rcbDecoded[length] = 0; } free(rpbDecoded); free(rcbDecoded); @@ -699,7 +713,7 @@ _libssh2_wincng_asn_decode_bns(unsigned char *pbEncoded, ret = -1; } - free(pbDecoded); + _libssh2_wincng_mfree(pbDecoded, cbDecoded); } return ret; @@ -845,7 +859,7 @@ _libssh2_wincng_rsa_new(libssh2_rsa_ctx **rsa, ret = BCryptImportKeyPair(_libssh2_wincng.hAlgRSA, NULL, lpszBlobType, &hKey, key, keylen, 0); if (!BCRYPT_SUCCESS(ret)) { - free(key); + _libssh2_wincng_mfree(key, keylen); return -1; } @@ -853,7 +867,7 @@ _libssh2_wincng_rsa_new(libssh2_rsa_ctx **rsa, *rsa = malloc(sizeof(libssh2_rsa_ctx)); if (!(*rsa)) { BCryptDestroyKey(hKey); - free(key); + _libssh2_wincng_mfree(key, keylen); return -1; } @@ -889,7 +903,7 @@ _libssh2_wincng_rsa_new_private(libssh2_rsa_ctx **rsa, PKCS_RSA_PRIVATE_KEY, &pbStructInfo, &cbStructInfo); - free(pbEncoded); + _libssh2_wincng_mfree(pbEncoded, cbEncoded); if (ret) { return -1; @@ -900,7 +914,7 @@ _libssh2_wincng_rsa_new_private(libssh2_rsa_ctx **rsa, LEGACY_RSAPRIVATE_BLOB, &hKey, pbStructInfo, cbStructInfo, 0); if (!BCRYPT_SUCCESS(ret)) { - free(pbStructInfo); + _libssh2_wincng_mfree(pbStructInfo, cbStructInfo); return -1; } @@ -908,7 +922,7 @@ _libssh2_wincng_rsa_new_private(libssh2_rsa_ctx **rsa, *rsa = malloc(sizeof(libssh2_rsa_ctx)); if (!(*rsa)) { BCryptDestroyKey(hKey); - free(pbStructInfo); + _libssh2_wincng_mfree(pbStructInfo, cbStructInfo); return -1; } @@ -982,7 +996,7 @@ _libssh2_wincng_rsa_sha1_sign(LIBSSH2_SESSION *session, ret = STATUS_NO_MEMORY; } - free(data); + _libssh2_wincng_mfree(data, datalen); return BCRYPT_SUCCESS(ret) ? 0 : -1; } @@ -994,12 +1008,10 @@ _libssh2_wincng_rsa_free(libssh2_rsa_ctx *rsa) return; BCryptDestroyKey(rsa->hKey); + rsa->hKey = NULL; - if (rsa->pbKeyObject) - free(rsa->pbKeyObject); - - memset(rsa, 0, sizeof(libssh2_rsa_ctx)); - free(rsa); + _libssh2_wincng_mfree(rsa->pbKeyObject, rsa->cbKeyObject); + _libssh2_wincng_mfree(rsa, sizeof(libssh2_rsa_ctx)); } @@ -1093,7 +1105,7 @@ _libssh2_wincng_dsa_new(libssh2_dsa_ctx **dsa, ret = BCryptImportKeyPair(_libssh2_wincng.hAlgDSA, NULL, lpszBlobType, &hKey, key, keylen, 0); if (!BCRYPT_SUCCESS(ret)) { - free(key); + _libssh2_wincng_mfree(key, keylen); return -1; } @@ -1101,7 +1113,7 @@ _libssh2_wincng_dsa_new(libssh2_dsa_ctx **dsa, *dsa = malloc(sizeof(libssh2_dsa_ctx)); if (!(*dsa)) { BCryptDestroyKey(hKey); - free(key); + _libssh2_wincng_mfree(key, keylen); return -1; } @@ -1135,7 +1147,7 @@ _libssh2_wincng_dsa_new_private(libssh2_dsa_ctx **dsa, ret = _libssh2_wincng_asn_decode_bns(pbEncoded, cbEncoded, &rpbDecoded, &rcbDecoded, &length); - free(pbEncoded); + _libssh2_wincng_mfree(pbEncoded, cbEncoded); if (ret) { return -1; @@ -1154,10 +1166,9 @@ _libssh2_wincng_dsa_new_private(libssh2_dsa_ctx **dsa, } for (index = 0; index < length; index++) { - if (rpbDecoded[index]) { - free(rpbDecoded[index]); - rpbDecoded[index] = NULL; - } + _libssh2_wincng_mfree(rpbDecoded[index], rcbDecoded[index]); + rpbDecoded[index] = NULL; + rcbDecoded[index] = 0; } free(rpbDecoded); @@ -1215,14 +1226,14 @@ _libssh2_wincng_dsa_sha1_sign(libssh2_dsa_ctx *dsa, memcpy(sig_fixed, sig, siglen); } - free(sig); + _libssh2_wincng_mfree(sig, siglen); } else ret = STATUS_NO_MEMORY; } else ret = STATUS_NO_MEMORY; } - free(data); + _libssh2_wincng_mfree(data, datalen); return BCRYPT_SUCCESS(ret) ? 0 : -1; } @@ -1234,12 +1245,10 @@ _libssh2_wincng_dsa_free(libssh2_dsa_ctx *dsa) return; BCryptDestroyKey(dsa->hKey); + dsa->hKey = NULL; - if (dsa->pbKeyObject) - free(dsa->pbKeyObject); - - memset(dsa, 0, sizeof(libssh2_dsa_ctx)); - free(dsa); + _libssh2_wincng_mfree(dsa->pbKeyObject, dsa->cbKeyObject); + _libssh2_wincng_mfree(dsa, sizeof(libssh2_dsa_ctx)); } #endif @@ -1289,7 +1298,7 @@ _libssh2_wincng_pub_priv_keyfile(LIBSSH2_SESSION *session, ret = _libssh2_wincng_asn_decode_bns(pbEncoded, cbEncoded, &rpbDecoded, &rcbDecoded, &length); - free(pbEncoded); + _libssh2_wincng_mfree(pbEncoded, cbEncoded); if (ret) { return -1; @@ -1362,10 +1371,9 @@ _libssh2_wincng_pub_priv_keyfile(LIBSSH2_SESSION *session, for (index = 0; index < length; index++) { - if (rpbDecoded[index]) { - free(rpbDecoded[index]); - rpbDecoded[index] = NULL; - } + _libssh2_wincng_mfree(rpbDecoded[index], rcbDecoded[index]); + rpbDecoded[index] = NULL; + rcbDecoded[index] = 0; } free(rpbDecoded); @@ -1461,10 +1469,10 @@ _libssh2_wincng_cipher_init(_libssh2_cipher_ctx *ctx, ret = BCryptImportKey(*type.phAlg, NULL, BCRYPT_KEY_DATA_BLOB, &hKey, pbKeyObject, dwKeyObject, key, keylen, 0); - free(key); + _libssh2_wincng_mfree(key, keylen); if (!BCRYPT_SUCCESS(ret)) { - free(pbKeyObject); + _libssh2_wincng_mfree(pbKeyObject, dwKeyObject); return -1; } @@ -1472,7 +1480,7 @@ _libssh2_wincng_cipher_init(_libssh2_cipher_ctx *ctx, pbIV = malloc(dwBlockLength); if (!pbIV) { BCryptDestroyKey(hKey); - free(pbKeyObject); + _libssh2_wincng_mfree(pbKeyObject, dwKeyObject); return -1; } dwIV = dwBlockLength; @@ -1531,7 +1539,7 @@ _libssh2_wincng_cipher_crypt(_libssh2_cipher_ctx *ctx, memcpy(block, pbOutput, cbOutput); } - free(pbOutput); + _libssh2_wincng_mfree(pbOutput, cbOutput); } else ret = STATUS_NO_MEMORY; } @@ -1543,13 +1551,11 @@ void _libssh2_wincng_cipher_dtor(_libssh2_cipher_ctx *ctx) { BCryptDestroyKey(ctx->hKey); + ctx->hKey = NULL; - if (ctx->pbKeyObject) { - free(ctx->pbKeyObject); - ctx->pbKeyObject = NULL; - } - - memset(ctx, 0, sizeof(_libssh2_cipher_ctx)); + _libssh2_wincng_mfree(ctx->pbKeyObject, ctx->dwKeyObject); + ctx->pbKeyObject = NULL; + ctx->dwKeyObject = 0; } @@ -1581,6 +1587,12 @@ _libssh2_wincng_bignum_resize(_libssh2_bn *bn, unsigned long length) if (length == bn->length) return 0; +#ifdef LIBSSH2_CLEAR_MEMORY + if (bn->bignum && bn->length > 0 && length < bn->length) { + SecureZeroMemory(bn->bignum + length, bn->length - length); + } +#endif + bignum = realloc(bn->bignum, length); if (!bignum) return -1; @@ -1688,7 +1700,7 @@ _libssh2_wincng_bignum_mod_exp(_libssh2_bn *r, r->bignum, r->length, &offset, BCRYPT_PAD_NONE); - free(bignum); + _libssh2_wincng_mfree(bignum, length); if (BCRYPT_SUCCESS(ret)) { _libssh2_wincng_bignum_resize(r, offset); @@ -1702,7 +1714,7 @@ _libssh2_wincng_bignum_mod_exp(_libssh2_bn *r, BCryptDestroyKey(hKey); } - free(key); + _libssh2_wincng_mfree(key, keylen); return BCRYPT_SUCCESS(ret) ? 0 : -1; } @@ -1780,6 +1792,10 @@ _libssh2_wincng_bignum_from_bin(_libssh2_bn *bn, unsigned long len, if (offset > 0) { memmove(bn->bignum, bn->bignum + offset, length); +#ifdef LIBSSH2_CLEAR_MEMORY + SecureZeroMemory(bn->bignum + length, offset); +#endif + bignum = realloc(bn->bignum, length); if (bignum) { bn->bignum = bignum; @@ -1801,11 +1817,11 @@ _libssh2_wincng_bignum_free(_libssh2_bn *bn) { if (bn) { if (bn->bignum) { - free(bn->bignum); + _libssh2_wincng_mfree(bn->bignum, bn->length); bn->bignum = NULL; } bn->length = 0; - free(bn); + _libssh2_wincng_mfree(bn, sizeof(_libssh2_bn)); } } -- 1.9.2.msysgit.0 --------------080602030409000906020601 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --------------080602030409000906020601-- From libssh2-devel-bounces@cool.haxx.se Thu Jun 19 19:57:18 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5JHv9t5032644; Thu, 19 Jun 2014 19:57:16 +0200 Received: from foo.stuge.se (qmailr@foo.stuge.se [212.116.89.98]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5JHv7pf032634 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 19 Jun 2014 19:57:07 +0200 Received: (qmail 15734 invoked by uid 501); 19 Jun 2014 17:57:08 -0000 Message-ID: <20140619175708.15733.qmail@stuge.se> Date: Thu, 19 Jun 2014 19:57:08 +0200 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend Mail-Followup-To: libssh2-devel@cool.haxx.se References: <5325F5FF.2060703@marc-hoersken.de> <5378B273.3040500@marc-hoersken.de> <5378E9FA.4000309@marc-hoersken.de> <53A31D64.2020808@marc-hoersken.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <53A31D64.2020808@marc-hoersken.de> X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5JHv9t5032644 Marc Hoersken wrote: > Please review the new patch. Any feedback is welcome. I guess the patch > should also include some warning about it only being available for > Windows with WinCNG for now before being merged at the current stage of > the implementation. The configure switch should only be available when configuring with wincng crypto. If that is not possible (autoconf limitations) then enabling the option should throw an error when this functionality is not available in code. Failing silently (ie. not securely zeroing memory) after a successful configure of the library with the option enabled isn't really acceptable IMO. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Jun 20 15:38:38 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5KDc6OD017335; Fri, 20 Jun 2014 15:38:32 +0200 Received: from mx.uxnr.de (mx.uxnr.de [IPv6:2a00:1828:2000:378:2525:0:59ee:542f]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5KDc48o017189 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 20 Jun 2014 15:38:04 +0200 Received: from [10.10.0.3] (mh02.marc.uxnr.eu [10.10.0.3]) by mx.uxnr.de (Postfix) with ESMTPSA id 3B2C01C5A311 for ; Fri, 20 Jun 2014 15:37:55 +0200 (CEST) X-DKIM: OpenDKIM Filter v2.6.8 mx.uxnr.de 3B2C01C5A311 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=marc-hoersken.de; s=picard; t=1403271475; bh=djZHjmnIbzcyab3Mq3Z59gI+llmbk0vFGVSn+ujk0AU=; h=Date:From:To:Subject:References:In-Reply-To:From; b=vi71zoGTBuu34dcZReu3qvZ23vB0KqvX3RtEuD1J11FsT/w7Lhq3/z5GSY4jiDfSo JCjmXIlbq4h47NCIEqCIqkwO7KB18ttkGTYwaTdzkMZKUZvzZdL7MDl7kpYMLhkUpD 3SHNo7baHuMi2YLAmXfnT+Ho0K2kNOxHURu4MqyE= Message-ID: <53A43915.5090607@marc-hoersken.de> Date: Fri, 20 Jun 2014 15:37:25 +0200 From: Marc Hoersken User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Re: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend References: <5325F5FF.2060703@marc-hoersken.de> <5378B273.3040500@marc-hoersken.de> <5378E9FA.4000309@marc-hoersken.de> <53A31D64.2020808@marc-hoersken.de> <20140619175708.15733.qmail@stuge.se> In-Reply-To: <20140619175708.15733.qmail@stuge.se> X-Enigmail-Version: 1.6 X-Spam-Status: No, score=-1.1 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on picard.vpn.uxnr.de X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0177031237==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============0177031237== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PhXswnxf3hpKCTNH0TXPO5rVqwDVeeNI8" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --PhXswnxf3hpKCTNH0TXPO5rVqwDVeeNI8 Content-Type: multipart/mixed; boundary="------------020304000002060809080905" This is a multi-part message in MIME format. --------------020304000002060809080905 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Peter, On 19.06.2014 19:57, Peter Stuge wrote: > The configure switch should only be available when configuring with > wincng crypto. > > If that is not possible (autoconf limitations) then enabling the > option should throw an error when this functionality is not available > in code. > > Failing silently (ie. not securely zeroing memory) after a successful > configure of the library with the option enabled isn't really > acceptable IMO. thanks for the feedback. I updated configure.ac to produce a warning if secure clearing/zeroing of memory is unsupported / not available and expanded the configure summary to look like the following, as an example for the OpenSSL backend: configure: summary of build options: version: 1.4.4_DEV Host type: x86_64-unknown-linux-gnu Install prefix: /usr/local Compiler: gcc Compiler flags: -g -O2 Library types: Shared=3Dyes, Static=3Dyes Crypto library: OpenSSL (AES-CTR: yes) Clear memory: unsupported Debug build: no Build examples: yes Path to sshd: /usr/sbin/sshd (only for self-tests) zlib compression: yes Clear memory shows either "yes" (enabled and available), "no" (disabled) or "unsupported" (unavailable). Please find the updated patch attached to this email. Best regards, Marc --------------020304000002060809080905 Content-Type: text/plain; charset=windows-1252; name="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*0="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.pa"; filename*1="tch" RnJvbSAxY2U4NDZjY2ZmMWE2MDMxNjdhMmZhNDhkMzk0ZDM1ODUzNjk2ZTQ4IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBNYXJjIEhvZXJza2VuIDxpbmZvQG1hcmMtaG9lcnNr ZW4uZGU+CkRhdGU6IEZyaSwgMjAgSnVuIDIwMTQgMTU6MzI6MTEgKzAyMDAKU3ViamVjdDog W1BBVENIXSB3aW5jbmc6IEFkZGVkIGV4cGxpY2l0IGNsZWFyIG1lbW9yeSBmZWF0dXJlIHRv IFdpbkNORyBiYWNrZW5kCgpUaGlzIHJlLWludHJvZHVjZXMgdGhlIG9yaWdpbmFsIGZlYXR1 cmUgcHJvcG9zZWQgZHVyaW5nCnRoZSBkZXZlbG9wbWVudCBvZiB0aGUgV2luQ05HIGNyeXB0 byBiYWNrZW5kLiBJdCBzdGlsbCBuZWVkcwp0byBiZSBhZGRlZCB0byBsaWJzc2gyIGl0c2Vs ZiBhbmQgcHJvYmFibHkgb3RoZXIgYmFja2VuZHMuCgpNZW1vcnkgaXMgY2xlYXJlZCB1c2lu ZyB0aGUgZnVuY3Rpb24gU2VjdXJlWmVyb01lbW9yeSB3aGljaCBpcwphdmFpbGFibGUgb24g V2luZG93cyBzeXN0ZW1zLCBqdXN0IGxpa2UgdGhlIFdpbkNORyBiYWNrZW5kLgotLS0KIGNv bmZpZ3VyZS5hYyB8ICAxNiArKysrKysrCiBzcmMvd2luY25nLmMgfCAxNDYgKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIDIg ZmlsZXMgY2hhbmdlZCwgOTcgaW5zZXJ0aW9ucygrKSwgNjUgZGVsZXRpb25zKC0pCgpkaWZm IC0tZ2l0IGEvY29uZmlndXJlLmFjIGIvY29uZmlndXJlLmFjCmluZGV4IGJhNGRkN2EuLmEy MzY3YjggMTAwNjQ0Ci0tLSBhL2NvbmZpZ3VyZS5hYworKysgYi9jb25maWd1cmUuYWMKQEAg LTE5Nyw2ICsxOTcsMjEgQEAgaWYgdGVzdCAiJEdFWF9ORVciICE9ICJubyI7IHRoZW4KICAg QUNfREVGSU5FKExJQlNTSDJfREhfR0VYX05FVywgMSwgW0VuYWJsZSBuZXdlciBkaWZmaWUt aGVsbG1hbi1ncm91cC1leGNoYW5nZS1zaGExIHN5bnRheF0pCiBmaQogCitBQ19BUkdfRU5B QkxFKGNsZWFyLW1lbW9yeSwKKyAgQUNfSEVMUF9TVFJJTkcoWy0tZGlzYWJsZS1jbGVhci1t ZW1vcnldLFtEaXNhYmxlIGNsZWFyaW5nIG9mIG1lbW9yeSBiZWZvcmUgYmVpbmcgZnJlZWRd KSwKKyAgW0NMRUFSX01FTU9SWT0kZW5hYmxldmFsXSkKK2lmIHRlc3QgIiRhY19jdl9saWJi Y3J5cHQiID0gInllcyI7IHRoZW4KKyAgaWYgdGVzdCAiJENMRUFSX01FTU9SWSIgIT0gIm5v IjsgdGhlbgorICAgIEFDX0RFRklORShMSUJTU0gyX0NMRUFSX01FTU9SWSwgMSwgW0VuYWJs ZSBjbGVhcmluZyBvZiBtZW1vcnkgYmVmb3JlIGJlaW5nIGZyZWVkXSkKKyAgICBlbmFibGVf Y2xlYXJfbWVtb3J5PXllcworICBlbHNlCisgICAgZW5hYmxlX2NsZWFyX21lbW9yeT1ubwor ICBmaQorZWxzZQorICBBQ19NU0dfV0FSTihbc2VjdXJlIGNsZWFyaW5nL3plcm9pbmcgb2Yg bWVtb3J5IGlzIG9ubHkgc3VwcG9ydGVkIGJ5IHRoZSBXaW5DTkcgYmFja2VuZF0pCisgIGVu YWJsZV9jbGVhcl9tZW1vcnk9dW5zdXBwb3J0ZWQKK2ZpCisKIGRubCAqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioKIGRubCBv cHRpb24gdG8gc3dpdGNoIG9uIGNvbXBpbGVyIGRlYnVnIG9wdGlvbnMKIGRubApAQCAtMzYy LDYgKzM3Nyw3IEBAIEFDX01TR19OT1RJQ0UoW3N1bW1hcnkgb2YgYnVpbGQgb3B0aW9uczoK ICAgQ29tcGlsZXIgZmxhZ3M6ICAgJHtDRkxBR1N9CiAgIExpYnJhcnkgdHlwZXM6ICAgIFNo YXJlZD0ke2VuYWJsZV9zaGFyZWR9LCBTdGF0aWM9JHtlbmFibGVfc3RhdGljfQogICBDcnlw dG8gbGlicmFyeTogICAke2ZvdW5kX2NyeXB0b30KKyAgQ2xlYXIgbWVtb3J5OiAgICAgJGVu YWJsZV9jbGVhcl9tZW1vcnkKICAgRGVidWcgYnVpbGQ6ICAgICAgJGVuYWJsZV9kZWJ1Zwog ICBCdWlsZCBleGFtcGxlczogICAkYnVpbGRfZXhhbXBsZXMKICAgUGF0aCB0byBzc2hkOiAg ICAgJGFjX2N2X3BhdGhfU1NIRCAob25seSBmb3Igc2VsZi10ZXN0cykKZGlmZiAtLWdpdCBh L3NyYy93aW5jbmcuYyBiL3NyYy93aW5jbmcuYwppbmRleCBiNTc1MzUwLi41ZGZjYTUyIDEw MDY0NAotLS0gYS9zcmMvd2luY25nLmMKKysrIGIvc3JjL3dpbmNuZy5jCkBAIC0yODAsNiAr MjgwLDIwIEBAIF9saWJzc2gyX3dpbmNuZ19yYW5kb20odm9pZCAqYnVmLCBpbnQgbGVuKQog ICAgIHJldHVybiBCQ1JZUFRfU1VDQ0VTUyhyZXQpID8gMCA6IC0xOwogfQogCitzdGF0aWMg dm9pZAorX2xpYnNzaDJfd2luY25nX21mcmVlKHZvaWQgKmJ1ZiwgaW50IGxlbikKK3sKKyAg ICBpZiAoIWJ1ZikKKyAgICAgICAgcmV0dXJuOworCisjaWZkZWYgTElCU1NIMl9DTEVBUl9N RU1PUlkKKyAgICBpZiAobGVuID4gMCkKKyAgICAgICAgU2VjdXJlWmVyb01lbW9yeShidWYs IGxlbik7CisjZW5kaWYKKworICAgIGZyZWUoYnVmKTsKK30KKwogCiAvKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq Ki8KIC8qCkBAIC0zMjIsNyArMzM2LDcgQEAgX2xpYnNzaDJfd2luY25nX2hhc2hfaW5pdChf bGlic3NoMl93aW5jbmdfaGFzaF9jdHggKmN0eCwKICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHBiSGFzaE9iamVjdCwgZHdIYXNoT2JqZWN0LAogICAgICAgICAgICAgICAgICAgICAg ICAgICAga2V5LCBrZXlsZW4sIDApOwogICAgIGlmICghQkNSWVBUX1NVQ0NFU1MocmV0KSkg ewotICAgICAgICBmcmVlKHBiSGFzaE9iamVjdCk7CisgICAgICAgIF9saWJzc2gyX3dpbmNu Z19tZnJlZShwYkhhc2hPYmplY3QsIGR3SGFzaE9iamVjdCk7CiAgICAgICAgIHJldHVybiAt MTsKICAgICB9CiAKQEAgLTM1NSwxMSArMzY5LDExIEBAIF9saWJzc2gyX3dpbmNuZ19oYXNo X2ZpbmFsKF9saWJzc2gyX3dpbmNuZ19oYXNoX2N0eCAqY3R4LAogICAgIHJldCA9IEJDcnlw dEZpbmlzaEhhc2goY3R4LT5oSGFzaCwgaGFzaCwgY3R4LT5jYkhhc2gsIDApOwogCiAgICAg QkNyeXB0RGVzdHJveUhhc2goY3R4LT5oSGFzaCk7CisgICAgY3R4LT5oSGFzaCA9IE5VTEw7 CiAKLSAgICBpZiAoY3R4LT5wYkhhc2hPYmplY3QpCi0gICAgICAgIGZyZWUoY3R4LT5wYkhh c2hPYmplY3QpOwotCi0gICAgbWVtc2V0KGN0eCwgMCwgc2l6ZW9mKF9saWJzc2gyX3dpbmNu Z19oYXNoX2N0eCkpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShjdHgtPnBiSGFzaE9i amVjdCwgY3R4LT5kd0hhc2hPYmplY3QpOworICAgIGN0eC0+cGJIYXNoT2JqZWN0ID0gTlVM TDsKKyAgICBjdHgtPmR3SGFzaE9iamVjdCA9IDA7CiAKICAgICByZXR1cm4gcmV0OwogfQpA QCAtNDAzLDExICs0MTcsMTEgQEAgdm9pZAogX2xpYnNzaDJfd2luY25nX2htYWNfY2xlYW51 cChfbGlic3NoMl93aW5jbmdfaGFzaF9jdHggKmN0eCkKIHsKICAgICBCQ3J5cHREZXN0cm95 SGFzaChjdHgtPmhIYXNoKTsKKyAgICBjdHgtPmhIYXNoID0gTlVMTDsKIAotICAgIGlmIChj dHgtPnBiSGFzaE9iamVjdCkKLSAgICAgICAgZnJlZShjdHgtPnBiSGFzaE9iamVjdCk7Ci0K LSAgICBtZW1zZXQoY3R4LCAwLCBzaXplb2YoX2xpYnNzaDJfd2luY25nX2hhc2hfY3R4KSk7 CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGN0eC0+cGJIYXNoT2JqZWN0LCBjdHgtPmR3 SGFzaE9iamVjdCk7CisgICAgY3R4LT5wYkhhc2hPYmplY3QgPSBOVUxMOworICAgIGN0eC0+ ZHdIYXNoT2JqZWN0ID0gMDsKIH0KIAogCkBAIC00NDksMTcgKzQ2MywxNyBAQCBfbGlic3No Ml93aW5jbmdfa2V5X3NoYTFfdmVyaWZ5KF9saWJzc2gyX3dpbmNuZ19rZXlfY3R4ICpjdHgs CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nLmhBbGdI YXNoU0hBMSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoYXNoLCBoYXNobGVu KTsKIAotICAgIGZyZWUoZGF0YSk7CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGRhdGEs IGRhdGFsZW4pOwogCiAgICAgaWYgKHJldCkgewotICAgICAgICBmcmVlKGhhc2gpOworICAg ICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoaGFzaCwgaGFzaGxlbik7CiAgICAgICAgIHJl dHVybiAtMTsKICAgICB9CiAKICAgICBkYXRhbGVuID0gc2lnX2xlbjsKICAgICBkYXRhID0g bWFsbG9jKGRhdGFsZW4pOwogICAgIGlmICghZGF0YSkgewotICAgICAgICBmcmVlKGhhc2gp OworICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoaGFzaCwgaGFzaGxlbik7CiAgICAg ICAgIHJldHVybiAtMTsKICAgICB9CiAKQEAgLTQ3NCw4ICs0ODgsOCBAQCBfbGlic3NoMl93 aW5jbmdfa2V5X3NoYTFfdmVyaWZ5KF9saWJzc2gyX3dpbmNuZ19rZXlfY3R4ICpjdHgsCiAg ICAgcmV0ID0gQkNyeXB0VmVyaWZ5U2lnbmF0dXJlKGN0eC0+aEtleSwgcFBhZGRpbmdJbmZv LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoYXNoLCBoYXNobGVuLCBkYXRh LCBkYXRhbGVuLCBmbGFncyk7CiAKLSAgICBmcmVlKGhhc2gpOwotICAgIGZyZWUoZGF0YSk7 CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGhhc2gsIGhhc2hsZW4pOworICAgIF9saWJz c2gyX3dpbmNuZ19tZnJlZShkYXRhLCBkYXRhbGVuKTsKIAogICAgIHJldHVybiBCQ1JZUFRf U1VDQ0VTUyhyZXQpID8gMCA6IC0xOwogfQpAQCAtNTY4LDcgKzU4Miw3IEBAIF9saWJzc2gy X3dpbmNuZ19hc25fZGVjb2RlKHVuc2lnbmVkIGNoYXIgKnBiRW5jb2RlZCwKICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIHBiRW5jb2RlZCwgY2JFbmNvZGVkLCAwLCBOVUxMLAog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGJEZWNvZGVkLCAmY2JEZWNvZGVkKTsK ICAgICBpZiAoIXJldCkgewotICAgICAgICBmcmVlKHBiRGVjb2RlZCk7CisgICAgICAgIF9s aWJzc2gyX3dpbmNuZ19tZnJlZShwYkRlY29kZWQsIGNiRGVjb2RlZCk7CiAgICAgICAgIHJl dHVybiAtMTsKICAgICB9CiAKQEAgLTYzOCw3ICs2NTIsNyBAQCBfbGlic3NoMl93aW5jbmdf YXNuX2RlY29kZV9ibih1bnNpZ25lZCBjaGFyICpwYkVuY29kZWQsCiAgICAgICAgICAgICAq cHBiRGVjb2RlZCA9IHBiRGVjb2RlZDsKICAgICAgICAgICAgICpwY2JEZWNvZGVkID0gY2JE ZWNvZGVkOwogICAgICAgICB9Ci0gICAgICAgIGZyZWUocGJJbnRlZ2VyKTsKKyAgICAgICAg X2xpYnNzaDJfd2luY25nX21mcmVlKHBiSW50ZWdlciwgY2JJbnRlZ2VyKTsKICAgICB9CiAK ICAgICByZXR1cm4gcmV0OwpAQCAtNjgzLDEwICs2OTcsMTAgQEAgX2xpYnNzaDJfd2luY25n X2Fzbl9kZWNvZGVfYm5zKHVuc2lnbmVkIGNoYXIgKnBiRW5jb2RlZCwKICAgICAgICAgICAg ICAgICAgICAgKnBjYkNvdW50ID0gbGVuZ3RoOwogICAgICAgICAgICAgICAgIH0gZWxzZSB7 CiAgICAgICAgICAgICAgICAgICAgIGZvciAobGVuZ3RoID0gMDsgbGVuZ3RoIDwgaW5kZXg7 IGxlbmd0aCsrKSB7Ci0gICAgICAgICAgICAgICAgICAgICAgICBpZiAocnBiRGVjb2RlZFts ZW5ndGhdKSB7Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgZnJlZShycGJEZWNvZGVk W2xlbmd0aF0pOwotICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJwYkRlY29kZWRbbGVu Z3RoXSA9IE5VTEw7Ci0gICAgICAgICAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICAg ICAgICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtsZW5ndGhdLAor ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJjYkRlY29k ZWRbbGVuZ3RoXSk7CisgICAgICAgICAgICAgICAgICAgICAgICBycGJEZWNvZGVkW2xlbmd0 aF0gPSBOVUxMOworICAgICAgICAgICAgICAgICAgICAgICAgcmNiRGVjb2RlZFtsZW5ndGhd ID0gMDsKICAgICAgICAgICAgICAgICAgICAgfQogICAgICAgICAgICAgICAgICAgICBmcmVl KHJwYkRlY29kZWQpOwogICAgICAgICAgICAgICAgICAgICBmcmVlKHJjYkRlY29kZWQpOwpA QCAtNjk5LDcgKzcxMyw3IEBAIF9saWJzc2gyX3dpbmNuZ19hc25fZGVjb2RlX2Jucyh1bnNp Z25lZCBjaGFyICpwYkVuY29kZWQsCiAgICAgICAgICAgICByZXQgPSAtMTsKICAgICAgICAg fQogCi0gICAgICAgIGZyZWUocGJEZWNvZGVkKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKHBiRGVjb2RlZCwgY2JEZWNvZGVkKTsKICAgICB9CiAKICAgICByZXR1cm4gcmV0 OwpAQCAtODQ1LDcgKzg1OSw3IEBAIF9saWJzc2gyX3dpbmNuZ19yc2FfbmV3KGxpYnNzaDJf cnNhX2N0eCAqKnJzYSwKICAgICByZXQgPSBCQ3J5cHRJbXBvcnRLZXlQYWlyKF9saWJzc2gy X3dpbmNuZy5oQWxnUlNBLCBOVUxMLCBscHN6QmxvYlR5cGUsCiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAmaEtleSwga2V5LCBrZXlsZW4sIDApOwogICAgIGlmICghQkNSWVBU X1NVQ0NFU1MocmV0KSkgewotICAgICAgICBmcmVlKGtleSk7CisgICAgICAgIF9saWJzc2gy X3dpbmNuZ19tZnJlZShrZXksIGtleWxlbik7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9 CiAKQEAgLTg1Myw3ICs4NjcsNyBAQCBfbGlic3NoMl93aW5jbmdfcnNhX25ldyhsaWJzc2gy X3JzYV9jdHggKipyc2EsCiAgICAgKnJzYSA9IG1hbGxvYyhzaXplb2YobGlic3NoMl9yc2Ff Y3R4KSk7CiAgICAgaWYgKCEoKnJzYSkpIHsKICAgICAgICAgQkNyeXB0RGVzdHJveUtleSho S2V5KTsKLSAgICAgICAgZnJlZShrZXkpOworICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZy ZWUoa2V5LCBrZXlsZW4pOwogICAgICAgICByZXR1cm4gLTE7CiAgICAgfQogCkBAIC04ODks NyArOTAzLDcgQEAgX2xpYnNzaDJfd2luY25nX3JzYV9uZXdfcHJpdmF0ZShsaWJzc2gyX3Jz YV9jdHggKipyc2EsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgUEtD U19SU0FfUFJJVkFURV9LRVksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgJnBiU3RydWN0SW5mbywgJmNiU3RydWN0SW5mbyk7CiAKLSAgICBmcmVlKHBiRW5jb2Rl ZCk7CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiRW5jb2RlZCwgY2JFbmNvZGVkKTsK IAogICAgIGlmIChyZXQpIHsKICAgICAgICAgcmV0dXJuIC0xOwpAQCAtOTAwLDcgKzkxNCw3 IEBAIF9saWJzc2gyX3dpbmNuZ19yc2FfbmV3X3ByaXZhdGUobGlic3NoMl9yc2FfY3R4ICoq cnNhLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgTEVHQUNZX1JTQVBSSVZBVEVf QkxPQiwgJmhLZXksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYlN0cnVjdElu Zm8sIGNiU3RydWN0SW5mbywgMCk7CiAgICAgaWYgKCFCQ1JZUFRfU1VDQ0VTUyhyZXQpKSB7 Ci0gICAgICAgIGZyZWUocGJTdHJ1Y3RJbmZvKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKHBiU3RydWN0SW5mbywgY2JTdHJ1Y3RJbmZvKTsKICAgICAgICAgcmV0dXJuIC0x OwogICAgIH0KIApAQCAtOTA4LDcgKzkyMiw3IEBAIF9saWJzc2gyX3dpbmNuZ19yc2FfbmV3 X3ByaXZhdGUobGlic3NoMl9yc2FfY3R4ICoqcnNhLAogICAgICpyc2EgPSBtYWxsb2Moc2l6 ZW9mKGxpYnNzaDJfcnNhX2N0eCkpOwogICAgIGlmICghKCpyc2EpKSB7CiAgICAgICAgIEJD cnlwdERlc3Ryb3lLZXkoaEtleSk7Ci0gICAgICAgIGZyZWUocGJTdHJ1Y3RJbmZvKTsKKyAg ICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiU3RydWN0SW5mbywgY2JTdHJ1Y3RJbmZv KTsKICAgICAgICAgcmV0dXJuIC0xOwogICAgIH0KIApAQCAtOTgyLDcgKzk5Niw3IEBAIF9s aWJzc2gyX3dpbmNuZ19yc2Ffc2hhMV9zaWduKExJQlNTSDJfU0VTU0lPTiAqc2Vzc2lvbiwK ICAgICAgICAgICAgIHJldCA9IFNUQVRVU19OT19NRU1PUlk7CiAgICAgfQogCi0gICAgZnJl ZShkYXRhKTsKKyAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoZGF0YSwgZGF0YWxlbik7CiAK ICAgICByZXR1cm4gQkNSWVBUX1NVQ0NFU1MocmV0KSA/IDAgOiAtMTsKIH0KQEAgLTk5NCwx MiArMTAwOCwxMCBAQCBfbGlic3NoMl93aW5jbmdfcnNhX2ZyZWUobGlic3NoMl9yc2FfY3R4 ICpyc2EpCiAgICAgICAgIHJldHVybjsKIAogICAgIEJDcnlwdERlc3Ryb3lLZXkocnNhLT5o S2V5KTsKKyAgICByc2EtPmhLZXkgPSBOVUxMOwogCi0gICAgaWYgKHJzYS0+cGJLZXlPYmpl Y3QpCi0gICAgICAgIGZyZWUocnNhLT5wYktleU9iamVjdCk7Ci0KLSAgICBtZW1zZXQocnNh LCAwLCBzaXplb2YobGlic3NoMl9yc2FfY3R4KSk7Ci0gICAgZnJlZShyc2EpOworICAgIF9s aWJzc2gyX3dpbmNuZ19tZnJlZShyc2EtPnBiS2V5T2JqZWN0LCByc2EtPmNiS2V5T2JqZWN0 KTsKKyAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUocnNhLCBzaXplb2YobGlic3NoMl9yc2Ff Y3R4KSk7CiB9CiAKIApAQCAtMTA5Myw3ICsxMTA1LDcgQEAgX2xpYnNzaDJfd2luY25nX2Rz YV9uZXcobGlic3NoMl9kc2FfY3R4ICoqZHNhLAogICAgIHJldCA9IEJDcnlwdEltcG9ydEtl eVBhaXIoX2xpYnNzaDJfd2luY25nLmhBbGdEU0EsIE5VTEwsIGxwc3pCbG9iVHlwZSwKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICZoS2V5LCBrZXksIGtleWxlbiwgMCk7CiAg ICAgaWYgKCFCQ1JZUFRfU1VDQ0VTUyhyZXQpKSB7Ci0gICAgICAgIGZyZWUoa2V5KTsKKyAg ICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGtleSwga2V5bGVuKTsKICAgICAgICAgcmV0 dXJuIC0xOwogICAgIH0KIApAQCAtMTEwMSw3ICsxMTEzLDcgQEAgX2xpYnNzaDJfd2luY25n X2RzYV9uZXcobGlic3NoMl9kc2FfY3R4ICoqZHNhLAogICAgICpkc2EgPSBtYWxsb2Moc2l6 ZW9mKGxpYnNzaDJfZHNhX2N0eCkpOwogICAgIGlmICghKCpkc2EpKSB7CiAgICAgICAgIEJD cnlwdERlc3Ryb3lLZXkoaEtleSk7Ci0gICAgICAgIGZyZWUoa2V5KTsKKyAgICAgICAgX2xp YnNzaDJfd2luY25nX21mcmVlKGtleSwga2V5bGVuKTsKICAgICAgICAgcmV0dXJuIC0xOwog ICAgIH0KIApAQCAtMTEzNSw3ICsxMTQ3LDcgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9uZXdf cHJpdmF0ZShsaWJzc2gyX2RzYV9jdHggKipkc2EsCiAgICAgcmV0ID0gX2xpYnNzaDJfd2lu Y25nX2Fzbl9kZWNvZGVfYm5zKHBiRW5jb2RlZCwgY2JFbmNvZGVkLAogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmcnBiRGVjb2RlZCwgJnJjYkRlY29kZWQs ICZsZW5ndGgpOwogCi0gICAgZnJlZShwYkVuY29kZWQpOworICAgIF9saWJzc2gyX3dpbmNu Z19tZnJlZShwYkVuY29kZWQsIGNiRW5jb2RlZCk7CiAKICAgICBpZiAocmV0KSB7CiAgICAg ICAgIHJldHVybiAtMTsKQEAgLTExNTQsMTAgKzExNjYsOSBAQCBfbGlic3NoMl93aW5jbmdf ZHNhX25ld19wcml2YXRlKGxpYnNzaDJfZHNhX2N0eCAqKmRzYSwKICAgICB9CiAKICAgICBm b3IgKGluZGV4ID0gMDsgaW5kZXggPCBsZW5ndGg7IGluZGV4KyspIHsKLSAgICAgICAgaWYg KHJwYkRlY29kZWRbaW5kZXhdKSB7Ci0gICAgICAgICAgICBmcmVlKHJwYkRlY29kZWRbaW5k ZXhdKTsKLSAgICAgICAgICAgIHJwYkRlY29kZWRbaW5kZXhdID0gTlVMTDsKLSAgICAgICAg fQorICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtpbmRleF0sIHJj YkRlY29kZWRbaW5kZXhdKTsKKyAgICAgICAgcnBiRGVjb2RlZFtpbmRleF0gPSBOVUxMOwor ICAgICAgICByY2JEZWNvZGVkW2luZGV4XSA9IDA7CiAgICAgfQogCiAgICAgZnJlZShycGJE ZWNvZGVkKTsKQEAgLTEyMTUsMTQgKzEyMjYsMTQgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9z aGExX3NpZ24obGlic3NoMl9kc2FfY3R4ICpkc2EsCiAgICAgICAgICAgICAgICAgICAgIG1l bWNweShzaWdfZml4ZWQsIHNpZywgc2lnbGVuKTsKICAgICAgICAgICAgICAgICB9CiAKLSAg ICAgICAgICAgICAgICBmcmVlKHNpZyk7CisgICAgICAgICAgICAgICAgX2xpYnNzaDJfd2lu Y25nX21mcmVlKHNpZywgc2lnbGVuKTsKICAgICAgICAgICAgIH0gZWxzZQogICAgICAgICAg ICAgICAgIHJldCA9IFNUQVRVU19OT19NRU1PUlk7CiAgICAgICAgIH0gZWxzZQogICAgICAg ICAgICAgcmV0ID0gU1RBVFVTX05PX01FTU9SWTsKICAgICB9CiAKLSAgICBmcmVlKGRhdGEp OworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShkYXRhLCBkYXRhbGVuKTsKIAogICAgIHJl dHVybiBCQ1JZUFRfU1VDQ0VTUyhyZXQpID8gMCA6IC0xOwogfQpAQCAtMTIzNCwxMiArMTI0 NSwxMCBAQCBfbGlic3NoMl93aW5jbmdfZHNhX2ZyZWUobGlic3NoMl9kc2FfY3R4ICpkc2Ep CiAgICAgICAgIHJldHVybjsKIAogICAgIEJDcnlwdERlc3Ryb3lLZXkoZHNhLT5oS2V5KTsK KyAgICBkc2EtPmhLZXkgPSBOVUxMOwogCi0gICAgaWYgKGRzYS0+cGJLZXlPYmplY3QpCi0g ICAgICAgIGZyZWUoZHNhLT5wYktleU9iamVjdCk7Ci0KLSAgICBtZW1zZXQoZHNhLCAwLCBz aXplb2YobGlic3NoMl9kc2FfY3R4KSk7Ci0gICAgZnJlZShkc2EpOworICAgIF9saWJzc2gy X3dpbmNuZ19tZnJlZShkc2EtPnBiS2V5T2JqZWN0LCBkc2EtPmNiS2V5T2JqZWN0KTsKKyAg ICBfbGlic3NoMl93aW5jbmdfbWZyZWUoZHNhLCBzaXplb2YobGlic3NoMl9kc2FfY3R4KSk7 CiB9CiAjZW5kaWYKIApAQCAtMTI4OSw3ICsxMjk4LDcgQEAgX2xpYnNzaDJfd2luY25nX3B1 Yl9wcml2X2tleWZpbGUoTElCU1NIMl9TRVNTSU9OICpzZXNzaW9uLAogICAgIHJldCA9IF9s aWJzc2gyX3dpbmNuZ19hc25fZGVjb2RlX2JucyhwYkVuY29kZWQsIGNiRW5jb2RlZCwKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJnJwYkRlY29kZWQsICZy Y2JEZWNvZGVkLCAmbGVuZ3RoKTsKIAotICAgIGZyZWUocGJFbmNvZGVkKTsKKyAgICBfbGli c3NoMl93aW5jbmdfbWZyZWUocGJFbmNvZGVkLCBjYkVuY29kZWQpOwogCiAgICAgaWYgKHJl dCkgewogICAgICAgICByZXR1cm4gLTE7CkBAIC0xMzYyLDEwICsxMzcxLDkgQEAgX2xpYnNz aDJfd2luY25nX3B1Yl9wcml2X2tleWZpbGUoTElCU1NIMl9TRVNTSU9OICpzZXNzaW9uLAog CiAKICAgICBmb3IgKGluZGV4ID0gMDsgaW5kZXggPCBsZW5ndGg7IGluZGV4KyspIHsKLSAg ICAgICAgaWYgKHJwYkRlY29kZWRbaW5kZXhdKSB7Ci0gICAgICAgICAgICBmcmVlKHJwYkRl Y29kZWRbaW5kZXhdKTsKLSAgICAgICAgICAgIHJwYkRlY29kZWRbaW5kZXhdID0gTlVMTDsK LSAgICAgICAgfQorICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtp bmRleF0sIHJjYkRlY29kZWRbaW5kZXhdKTsKKyAgICAgICAgcnBiRGVjb2RlZFtpbmRleF0g PSBOVUxMOworICAgICAgICByY2JEZWNvZGVkW2luZGV4XSA9IDA7CiAgICAgfQogCiAgICAg ZnJlZShycGJEZWNvZGVkKTsKQEAgLTE0NjEsMTAgKzE0NjksMTAgQEAgX2xpYnNzaDJfd2lu Y25nX2NpcGhlcl9pbml0KF9saWJzc2gyX2NpcGhlcl9jdHggKmN0eCwKICAgICByZXQgPSBC Q3J5cHRJbXBvcnRLZXkoKnR5cGUucGhBbGcsIE5VTEwsIEJDUllQVF9LRVlfREFUQV9CTE9C LCAmaEtleSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgcGJLZXlPYmplY3QsIGR3S2V5 T2JqZWN0LCBrZXksIGtleWxlbiwgMCk7CiAKLSAgICBmcmVlKGtleSk7CisgICAgX2xpYnNz aDJfd2luY25nX21mcmVlKGtleSwga2V5bGVuKTsKIAogICAgIGlmICghQkNSWVBUX1NVQ0NF U1MocmV0KSkgewotICAgICAgICBmcmVlKHBiS2V5T2JqZWN0KTsKKyAgICAgICAgX2xpYnNz aDJfd2luY25nX21mcmVlKHBiS2V5T2JqZWN0LCBkd0tleU9iamVjdCk7CiAgICAgICAgIHJl dHVybiAtMTsKICAgICB9CiAKQEAgLTE0NzIsNyArMTQ4MCw3IEBAIF9saWJzc2gyX3dpbmNu Z19jaXBoZXJfaW5pdChfbGlic3NoMl9jaXBoZXJfY3R4ICpjdHgsCiAgICAgICAgIHBiSVYg PSBtYWxsb2MoZHdCbG9ja0xlbmd0aCk7CiAgICAgICAgIGlmICghcGJJVikgewogICAgICAg ICAgICAgQkNyeXB0RGVzdHJveUtleShoS2V5KTsKLSAgICAgICAgICAgIGZyZWUocGJLZXlP YmplY3QpOworICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiS2V5T2JqZWN0 LCBkd0tleU9iamVjdCk7CiAgICAgICAgICAgICByZXR1cm4gLTE7CiAgICAgICAgIH0KICAg ICAgICAgZHdJViA9IGR3QmxvY2tMZW5ndGg7CkBAIC0xNTMxLDcgKzE1MzksNyBAQCBfbGli c3NoMl93aW5jbmdfY2lwaGVyX2NyeXB0KF9saWJzc2gyX2NpcGhlcl9jdHggKmN0eCwKICAg ICAgICAgICAgICAgICBtZW1jcHkoYmxvY2ssIHBiT3V0cHV0LCBjYk91dHB1dCk7CiAgICAg ICAgICAgICB9CiAKLSAgICAgICAgICAgIGZyZWUocGJPdXRwdXQpOworICAgICAgICAgICAg X2xpYnNzaDJfd2luY25nX21mcmVlKHBiT3V0cHV0LCBjYk91dHB1dCk7CiAgICAgICAgIH0g ZWxzZQogICAgICAgICAgICAgcmV0ID0gU1RBVFVTX05PX01FTU9SWTsKICAgICB9CkBAIC0x NTQzLDEzICsxNTUxLDExIEBAIHZvaWQKIF9saWJzc2gyX3dpbmNuZ19jaXBoZXJfZHRvcihf bGlic3NoMl9jaXBoZXJfY3R4ICpjdHgpCiB7CiAgICAgQkNyeXB0RGVzdHJveUtleShjdHgt PmhLZXkpOworICAgIGN0eC0+aEtleSA9IE5VTEw7CiAKLSAgICBpZiAoY3R4LT5wYktleU9i amVjdCkgewotICAgICAgICBmcmVlKGN0eC0+cGJLZXlPYmplY3QpOwotICAgICAgICBjdHgt PnBiS2V5T2JqZWN0ID0gTlVMTDsKLSAgICB9Ci0KLSAgICBtZW1zZXQoY3R4LCAwLCBzaXpl b2YoX2xpYnNzaDJfY2lwaGVyX2N0eCkpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShj dHgtPnBiS2V5T2JqZWN0LCBjdHgtPmR3S2V5T2JqZWN0KTsKKyAgICBjdHgtPnBiS2V5T2Jq ZWN0ID0gTlVMTDsKKyAgICBjdHgtPmR3S2V5T2JqZWN0ID0gMDsKIH0KIAogCkBAIC0xNTgx LDYgKzE1ODcsMTIgQEAgX2xpYnNzaDJfd2luY25nX2JpZ251bV9yZXNpemUoX2xpYnNzaDJf Ym4gKmJuLCB1bnNpZ25lZCBsb25nIGxlbmd0aCkKICAgICBpZiAobGVuZ3RoID09IGJuLT5s ZW5ndGgpCiAgICAgICAgIHJldHVybiAwOwogCisjaWZkZWYgTElCU1NIMl9DTEVBUl9NRU1P UlkKKyAgICBpZiAoYm4tPmJpZ251bSAmJiBibi0+bGVuZ3RoID4gMCAmJiBsZW5ndGggPCBi bi0+bGVuZ3RoKSB7CisgICAgICAgIFNlY3VyZVplcm9NZW1vcnkoYm4tPmJpZ251bSArIGxl bmd0aCwgYm4tPmxlbmd0aCAtIGxlbmd0aCk7CisgICAgfQorI2VuZGlmCisKICAgICBiaWdu dW0gPSByZWFsbG9jKGJuLT5iaWdudW0sIGxlbmd0aCk7CiAgICAgaWYgKCFiaWdudW0pCiAg ICAgICAgIHJldHVybiAtMTsKQEAgLTE2ODgsNyArMTcwMCw3IEBAIF9saWJzc2gyX3dpbmNu Z19iaWdudW1fbW9kX2V4cChfbGlic3NoMl9ibiAqciwKICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICByLT5iaWdudW0sIHItPmxlbmd0aCwgJm9mZnNldCwKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBCQ1JZUFRfUEFEX05PTkUp OwogCi0gICAgICAgICAgICAgICAgICAgIGZyZWUoYmlnbnVtKTsKKyAgICAgICAgICAgICAg ICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGJpZ251bSwgbGVuZ3RoKTsKIAogICAgICAg ICAgICAgICAgICAgICBpZiAoQkNSWVBUX1NVQ0NFU1MocmV0KSkgewogICAgICAgICAgICAg ICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nX2JpZ251bV9yZXNpemUociwgb2Zmc2V0KTsK QEAgLTE3MDIsNyArMTcxNCw3IEBAIF9saWJzc2gyX3dpbmNuZ19iaWdudW1fbW9kX2V4cChf bGlic3NoMl9ibiAqciwKICAgICAgICAgQkNyeXB0RGVzdHJveUtleShoS2V5KTsKICAgICB9 CiAKLSAgICBmcmVlKGtleSk7CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGtleSwga2V5 bGVuKTsKIAogICAgIHJldHVybiBCQ1JZUFRfU1VDQ0VTUyhyZXQpID8gMCA6IC0xOwogfQpA QCAtMTc4MCw2ICsxNzkyLDEwIEBAIF9saWJzc2gyX3dpbmNuZ19iaWdudW1fZnJvbV9iaW4o X2xpYnNzaDJfYm4gKmJuLCB1bnNpZ25lZCBsb25nIGxlbiwKICAgICBpZiAob2Zmc2V0ID4g MCkgewogICAgICAgICBtZW1tb3ZlKGJuLT5iaWdudW0sIGJuLT5iaWdudW0gKyBvZmZzZXQs IGxlbmd0aCk7CiAKKyNpZmRlZiBMSUJTU0gyX0NMRUFSX01FTU9SWQorICAgICAgICBTZWN1 cmVaZXJvTWVtb3J5KGJuLT5iaWdudW0gKyBsZW5ndGgsIG9mZnNldCk7CisjZW5kaWYKKwog ICAgICAgICBiaWdudW0gPSByZWFsbG9jKGJuLT5iaWdudW0sIGxlbmd0aCk7CiAgICAgICAg IGlmIChiaWdudW0pIHsKICAgICAgICAgICAgIGJuLT5iaWdudW0gPSBiaWdudW07CkBAIC0x ODAxLDExICsxODE3LDExIEBAIF9saWJzc2gyX3dpbmNuZ19iaWdudW1fZnJlZShfbGlic3No Ml9ibiAqYm4pCiB7CiAgICAgaWYgKGJuKSB7CiAgICAgICAgIGlmIChibi0+YmlnbnVtKSB7 Ci0gICAgICAgICAgICBmcmVlKGJuLT5iaWdudW0pOworICAgICAgICAgICAgX2xpYnNzaDJf d2luY25nX21mcmVlKGJuLT5iaWdudW0sIGJuLT5sZW5ndGgpOwogICAgICAgICAgICAgYm4t PmJpZ251bSA9IE5VTEw7CiAgICAgICAgIH0KICAgICAgICAgYm4tPmxlbmd0aCA9IDA7Ci0g ICAgICAgIGZyZWUoYm4pOworICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoYm4sIHNp emVvZihfbGlic3NoMl9ibikpOwogICAgIH0KIH0KIAotLSAKMS45LjIubXN5c2dpdC4wCgo= --------------020304000002060809080905-- --PhXswnxf3hpKCTNH0TXPO5rVqwDVeeNI8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTpDkaAAoJEPhLrsrSvXftHc8IAL4t1BZ0/Ng2yiPFbjVBa4fb AgMjcCWDLBSzYM4TEyNU3u0C/hM4D7vuLQ0GQpR8uH4m4O2NMFuZwAnl3oUIAEMD CeVqQEH5CnEoDvJJS9PJZolptewWC8eNIjpyg+qT5wXjtoFsxhQe/KTHrhdINVjp mNpXgOn0lAS1yKcPJHf45aaShQlX7nTvYvey1vq4E5SgkvKGng2sFiqR5szWKCgl vaKQ1AFIwJLHUOasaFheSgKjN2lXE8/lTNhwlk0kruRvnwXM/89McnTu7QcXBZ0C 3k1LH7e6tagyLgEtcs15f0NT2w9VSkEx0bA7vZ2HebG39vZd+aWj5kT0TQkM2YM= =oIIR -----END PGP SIGNATURE----- --PhXswnxf3hpKCTNH0TXPO5rVqwDVeeNI8-- --===============0177031237== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============0177031237==-- From libssh2-devel-bounces@cool.haxx.se Fri Jun 20 18:16:38 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5KGFHME015178; Fri, 20 Jun 2014 18:16:34 +0200 Received: from foo.stuge.se (qmailr@foo.stuge.se [212.116.89.98]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5KGFGh3015175 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 20 Jun 2014 18:15:16 +0200 Received: (qmail 12373 invoked by uid 501); 20 Jun 2014 16:15:16 -0000 Message-ID: <20140620161516.12372.qmail@stuge.se> Date: Fri, 20 Jun 2014 18:15:16 +0200 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend Mail-Followup-To: libssh2-devel@cool.haxx.se References: <5325F5FF.2060703@marc-hoersken.de> <5378B273.3040500@marc-hoersken.de> <5378E9FA.4000309@marc-hoersken.de> <53A31D64.2020808@marc-hoersken.de> <20140619175708.15733.qmail@stuge.se> <53A43915.5090607@marc-hoersken.de> MIME-Version: 1.0 In-Reply-To: <53A43915.5090607@marc-hoersken.de> X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1198252410==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============1198252410== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Fba/0zbH8Xs+Fj9o" Content-Disposition: inline --Fba/0zbH8Xs+Fj9o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Marc Hoersken wrote: > > enabling the option should throw an error when this functionality > > is not available >=20 > thanks for the feedback. I updated configure.ac to produce a warning if > secure clearing/zeroing of memory is unsupported / not available I think a warning is appropriate when the --enable option was not explicitly specified. I think an error is neccessary when --enable *was* specified, but support is unavailable. > and expanded the configure summary to look like the following, > as an example for the OpenSSL backend: >=20 > configure: summary of build options: >=20 > version: 1.4.4_DEV > Host type: x86_64-unknown-linux-gnu > Install prefix: /usr/local > Compiler: gcc > Compiler flags: -g -O2 > Library types: Shared=3Dyes, Static=3Dyes > Crypto library: OpenSSL (AES-CTR: yes) > Clear memory: unsupported Looks great! > +++ b/configure.ac > @@ -197,6 +197,21 @@ if test "$GEX_NEW" !=3D "no"; then > AC_DEFINE(LIBSSH2_DH_GEX_NEW, 1, [Enable newer diffie-hellman-group-ex= change-sha1 syntax]) > fi > =20 > +AC_ARG_ENABLE(clear-memory, > + AC_HELP_STRING([--disable-clear-memory],[Disable clearing of memory be= fore being freed]), > + [CLEAR_MEMORY=3D$enableval]) > +if test "$ac_cv_libbcrypt" =3D "yes"; then Please don't add a new list of crypto backends to maintain. I'd suggest to instead introduce an abstraction such as $support_clear_memory which is set to no by default and set to yes by backends supporting this functionality. The above check would then inspect only that variable. Thanks //Peter --Fba/0zbH8Xs+Fj9o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iD8DBQFTpF4UhR3Q0dhIfEgRAjW5AJ9MCTkYVKDe6FOkMv0yVKRNTYSWPQCgxhoC isuG5YgVsWdFptYf5kiESZM= =Nyfq -----END PGP SIGNATURE----- --Fba/0zbH8Xs+Fj9o-- --===============1198252410== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============1198252410==-- From libssh2-devel-bounces@cool.haxx.se Sat Jun 21 15:25:52 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5LDPO7S020361; Sat, 21 Jun 2014 15:25:45 +0200 Received: from mx.uxnr.de (mx.uxnr.de [IPv6:2a00:1828:2000:378:2525:0:59ee:542f]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5LDPNPd020351 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 21 Jun 2014 15:25:23 +0200 Received: from [10.10.0.3] (mh02.marc.uxnr.eu [10.10.0.3]) by mx.uxnr.de (Postfix) with ESMTPSA id E89571C5A27C for ; Sat, 21 Jun 2014 15:25:11 +0200 (CEST) X-DKIM: OpenDKIM Filter v2.6.8 mx.uxnr.de E89571C5A27C DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=marc-hoersken.de; s=picard; t=1403357112; bh=1jWNEJoGNvoboMRsZkqYdggn9wYfQdXREAI8W3UzwiU=; h=Date:From:To:Subject:References:In-Reply-To:From; b=cn1l86uf8kD9KRxMquqyn7qwGSLBZir9zPQ6NAR6wXa29QifO2rlxk3rAd506BPGM KKkmXROwSSuZOwaqEkwx06oTTJ7G9aeDQ+HjxQY406QMimwAmm4IFwacWdRu+ezrfU fTjMSpgCtbSwiHeNGS6ZGyqiSWjGh31zV7NIrfew= Message-ID: <53A5879C.4000405@marc-hoersken.de> Date: Sat, 21 Jun 2014 15:24:44 +0200 From: Marc Hoersken User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Re: [PATCH] wincng: Added explicit clear memory feature to WinCNG backend References: <5325F5FF.2060703@marc-hoersken.de> <5378B273.3040500@marc-hoersken.de> <5378E9FA.4000309@marc-hoersken.de> <53A31D64.2020808@marc-hoersken.de> <20140619175708.15733.qmail@stuge.se> <53A43915.5090607@marc-hoersken.de> <20140620161516.12372.qmail@stuge.se> In-Reply-To: <20140620161516.12372.qmail@stuge.se> X-Enigmail-Version: 1.6 X-Spam-Status: No, score=0.7 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,URIBL_BLACK autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on picard.vpn.uxnr.de X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2108300891==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============2108300891== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uUXdGUT2uI9rHdvE0BV7a97b8I7s4vbMB" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --uUXdGUT2uI9rHdvE0BV7a97b8I7s4vbMB Content-Type: multipart/mixed; boundary="------------070807080507030309000600" This is a multi-part message in MIME format. --------------070807080507030309000600 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Peter, On 20.06.2014 18:15, Peter Stuge wrote: > I think a warning is appropriate when the --enable option was not > explicitly specified. > > I think an error is neccessary when --enable *was* specified, but > support is unavailable. okay, but I am not sure how to differentiate the two scenarios. I tried to implement the warning vs error condition, but now configure will always display an error while configuring against openssl or libgcrypt if --disable-clear-memory is not given. Please take a look at the attached updated patch. > Please don't add a new list of crypto backends to maintain. I'd > suggest to instead introduce an abstraction such as > $support_clear_memory which is set to no by default and set to yes by > backends supporting this functionality. > > The above check would then inspect only that variable. Good idea, implemented. Best regards, Marc --------------070807080507030309000600 Content-Type: text/plain; charset=windows-1252; name="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*0="0001-wincng-Added-explicit-clear-memory-feature-to-WinCNG.pa"; filename*1="tch" RnJvbSBlZmM4ODhkMThjNDlhOGEzZTBhMTA4NWQyZDA1MjY2MGM5ZDFmZjk1IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBNYXJjIEhvZXJza2VuIDxpbmZvQG1hcmMtaG9lcnNr ZW4uZGU+CkRhdGU6IFNhdCwgMjEgSnVuIDIwMTQgMTU6MTQ6MDcgKzAyMDAKU3ViamVjdDog W1BBVENIXSB3aW5jbmc6IEFkZGVkIGV4cGxpY2l0IGNsZWFyIG1lbW9yeSBmZWF0dXJlIHRv IFdpbkNORyBiYWNrZW5kCgpUaGlzIHJlLWludHJvZHVjZXMgdGhlIG9yaWdpbmFsIGZlYXR1 cmUgcHJvcG9zZWQgZHVyaW5nCnRoZSBkZXZlbG9wbWVudCBvZiB0aGUgV2luQ05HIGNyeXB0 byBiYWNrZW5kLiBJdCBzdGlsbCBuZWVkcwp0byBiZSBhZGRlZCB0byBsaWJzc2gyIGl0c2Vs ZiBhbmQgcHJvYmFibHkgb3RoZXIgYmFja2VuZHMuCgpNZW1vcnkgaXMgY2xlYXJlZCB1c2lu ZyB0aGUgZnVuY3Rpb24gU2VjdXJlWmVyb01lbW9yeSB3aGljaCBpcwphdmFpbGFibGUgb24g V2luZG93cyBzeXN0ZW1zLCBqdXN0IGxpa2UgdGhlIFdpbkNORyBiYWNrZW5kLgotLS0KIGNv bmZpZ3VyZS5hYyB8ICAyMyArKysrKysrKysrCiBzcmMvd2luY25nLmMgfCAxNDYgKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0K IDIgZmlsZXMgY2hhbmdlZCwgMTA0IGluc2VydGlvbnMoKyksIDY1IGRlbGV0aW9ucygtKQoK ZGlmZiAtLWdpdCBhL2NvbmZpZ3VyZS5hYyBiL2NvbmZpZ3VyZS5hYwppbmRleCBiYTRkZDdh Li43OThkZWY5IDEwMDY0NAotLS0gYS9jb25maWd1cmUuYWMKKysrIGIvY29uZmlndXJlLmFj CkBAIC05Nyw2ICs5Nyw3IEBAIEFDX0FSR19XSVRIKGxpYnosCiAgIHVzZV9saWJ6PSR3aXRo dmFsLHVzZV9saWJ6PWF1dG8pCiAKIGZvdW5kX2NyeXB0bz1ub25lCitzdXBwb3J0X2NsZWFy X21lbW9yeT1ubwogCiAjIExvb2sgZm9yIE9wZW5TU0wKIGlmIHRlc3QgIiRmb3VuZF9jcnlw dG8iID0gIm5vbmUiICYmIHRlc3QgIiR1c2Vfb3BlbnNzbCIgIT0gIm5vIjsgdGhlbgpAQCAt MTUwLDYgKzE1MSw3IEBAIGlmIHRlc3QgIiRhY19jdl9saWJiY3J5cHQiID0gInllcyI7IHRo ZW4KICAgICBMSUJTPSIkTElCUyAtbGNyeXB0MzIiCiAgIGZpCiAgIGZvdW5kX2NyeXB0bz0i V2luZG93cyBDcnlwdG9ncmFwaHkgQVBJOiBOZXh0IEdlbmVyYXRpb24iCisgIHN1cHBvcnRf Y2xlYXJfbWVtb3J5PXllcwogZmkKIEFNX0NPTkRJVElPTkFMKFdJTkNORywgdGVzdCAiJGFj X2N2X2xpYmJjcnlwdCIgPSAieWVzIikKIApAQCAtMTk3LDYgKzE5OSwyNiBAQCBpZiB0ZXN0 ICIkR0VYX05FVyIgIT0gIm5vIjsgdGhlbgogICBBQ19ERUZJTkUoTElCU1NIMl9ESF9HRVhf TkVXLCAxLCBbRW5hYmxlIG5ld2VyIGRpZmZpZS1oZWxsbWFuLWdyb3VwLWV4Y2hhbmdlLXNo YTEgc3ludGF4XSkKIGZpCiAKK0FDX0FSR19FTkFCTEUoY2xlYXItbWVtb3J5LAorICBBQ19I RUxQX1NUUklORyhbLS1kaXNhYmxlLWNsZWFyLW1lbW9yeV0sW0Rpc2FibGUgY2xlYXJpbmcg b2YgbWVtb3J5IGJlZm9yZSBiZWluZyBmcmVlZF0pLAorICBbQ0xFQVJfTUVNT1JZPSRlbmFi bGV2YWxdKQoraWYgdGVzdCAiJENMRUFSX01FTU9SWSIgIT0gIm5vIjsgdGhlbgorICBpZiB0 ZXN0ICIkc3VwcG9ydF9jbGVhcl9tZW1vcnkiID0gInllcyI7IHRoZW4KKyAgICBBQ19ERUZJ TkUoTElCU1NIMl9DTEVBUl9NRU1PUlksIDEsIFtFbmFibGUgY2xlYXJpbmcgb2YgbWVtb3J5 IGJlZm9yZSBiZWluZyBmcmVlZF0pCisgICAgZW5hYmxlX2NsZWFyX21lbW9yeT15ZXMKKyAg ZWxzZQorICAgIEFDX01TR19FUlJPUihbc2VjdXJlIGNsZWFyaW5nL3plcm9pbmcgb2YgbWVt b3J5IGlzIG5vdCBzdXBwb3J0ZWQgYnkgdGhlIHNlbGVjdGVkIGNyeXB0byBiYWNrZW5kXSkK KyAgICBlbmFibGVfY2xlYXJfbWVtb3J5PXVuc3VwcG9ydGVkCisgIGZpCitlbHNlCisgIGlm IHRlc3QgIiRzdXBwb3J0X2NsZWFyX21lbW9yeSIgPSAieWVzIjsgdGhlbgorICAgIGVuYWJs ZV9jbGVhcl9tZW1vcnk9bm8KKyAgZWxzZQorICAgIEFDX01TR19XQVJOKFtzZWN1cmUgY2xl YXJpbmcvemVyb2luZyBvZiBtZW1vcnkgaXMgbm90IHN1cHBvcnRlZCBieSB0aGUgc2VsZWN0 ZWQgY3J5cHRvIGJhY2tlbmRdKQorICAgIGVuYWJsZV9jbGVhcl9tZW1vcnk9dW5zdXBwb3J0 ZWQKKyAgZmkKK2ZpCisKIGRubCAqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioKIGRubCBvcHRpb24gdG8gc3dpdGNoIG9uIGNv bXBpbGVyIGRlYnVnIG9wdGlvbnMKIGRubApAQCAtMzYyLDYgKzM4NCw3IEBAIEFDX01TR19O T1RJQ0UoW3N1bW1hcnkgb2YgYnVpbGQgb3B0aW9uczoKICAgQ29tcGlsZXIgZmxhZ3M6ICAg JHtDRkxBR1N9CiAgIExpYnJhcnkgdHlwZXM6ICAgIFNoYXJlZD0ke2VuYWJsZV9zaGFyZWR9 LCBTdGF0aWM9JHtlbmFibGVfc3RhdGljfQogICBDcnlwdG8gbGlicmFyeTogICAke2ZvdW5k X2NyeXB0b30KKyAgQ2xlYXIgbWVtb3J5OiAgICAgJGVuYWJsZV9jbGVhcl9tZW1vcnkKICAg RGVidWcgYnVpbGQ6ICAgICAgJGVuYWJsZV9kZWJ1ZwogICBCdWlsZCBleGFtcGxlczogICAk YnVpbGRfZXhhbXBsZXMKICAgUGF0aCB0byBzc2hkOiAgICAgJGFjX2N2X3BhdGhfU1NIRCAo b25seSBmb3Igc2VsZi10ZXN0cykKZGlmZiAtLWdpdCBhL3NyYy93aW5jbmcuYyBiL3NyYy93 aW5jbmcuYwppbmRleCBiNTc1MzUwLi41ZGZjYTUyIDEwMDY0NAotLS0gYS9zcmMvd2luY25n LmMKKysrIGIvc3JjL3dpbmNuZy5jCkBAIC0yODAsNiArMjgwLDIwIEBAIF9saWJzc2gyX3dp bmNuZ19yYW5kb20odm9pZCAqYnVmLCBpbnQgbGVuKQogICAgIHJldHVybiBCQ1JZUFRfU1VD Q0VTUyhyZXQpID8gMCA6IC0xOwogfQogCitzdGF0aWMgdm9pZAorX2xpYnNzaDJfd2luY25n X21mcmVlKHZvaWQgKmJ1ZiwgaW50IGxlbikKK3sKKyAgICBpZiAoIWJ1ZikKKyAgICAgICAg cmV0dXJuOworCisjaWZkZWYgTElCU1NIMl9DTEVBUl9NRU1PUlkKKyAgICBpZiAobGVuID4g MCkKKyAgICAgICAgU2VjdXJlWmVyb01lbW9yeShidWYsIGxlbik7CisjZW5kaWYKKworICAg IGZyZWUoYnVmKTsKK30KKwogCiAvKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKi8KIC8qCkBAIC0zMjIsNyArMzM2 LDcgQEAgX2xpYnNzaDJfd2luY25nX2hhc2hfaW5pdChfbGlic3NoMl93aW5jbmdfaGFzaF9j dHggKmN0eCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBiSGFzaE9iamVjdCwgZHdI YXNoT2JqZWN0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAga2V5LCBrZXlsZW4sIDAp OwogICAgIGlmICghQkNSWVBUX1NVQ0NFU1MocmV0KSkgewotICAgICAgICBmcmVlKHBiSGFz aE9iamVjdCk7CisgICAgICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShwYkhhc2hPYmplY3Qs IGR3SGFzaE9iamVjdCk7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9CiAKQEAgLTM1NSwx MSArMzY5LDExIEBAIF9saWJzc2gyX3dpbmNuZ19oYXNoX2ZpbmFsKF9saWJzc2gyX3dpbmNu Z19oYXNoX2N0eCAqY3R4LAogICAgIHJldCA9IEJDcnlwdEZpbmlzaEhhc2goY3R4LT5oSGFz aCwgaGFzaCwgY3R4LT5jYkhhc2gsIDApOwogCiAgICAgQkNyeXB0RGVzdHJveUhhc2goY3R4 LT5oSGFzaCk7CisgICAgY3R4LT5oSGFzaCA9IE5VTEw7CiAKLSAgICBpZiAoY3R4LT5wYkhh c2hPYmplY3QpCi0gICAgICAgIGZyZWUoY3R4LT5wYkhhc2hPYmplY3QpOwotCi0gICAgbWVt c2V0KGN0eCwgMCwgc2l6ZW9mKF9saWJzc2gyX3dpbmNuZ19oYXNoX2N0eCkpOworICAgIF9s aWJzc2gyX3dpbmNuZ19tZnJlZShjdHgtPnBiSGFzaE9iamVjdCwgY3R4LT5kd0hhc2hPYmpl Y3QpOworICAgIGN0eC0+cGJIYXNoT2JqZWN0ID0gTlVMTDsKKyAgICBjdHgtPmR3SGFzaE9i amVjdCA9IDA7CiAKICAgICByZXR1cm4gcmV0OwogfQpAQCAtNDAzLDExICs0MTcsMTEgQEAg dm9pZAogX2xpYnNzaDJfd2luY25nX2htYWNfY2xlYW51cChfbGlic3NoMl93aW5jbmdfaGFz aF9jdHggKmN0eCkKIHsKICAgICBCQ3J5cHREZXN0cm95SGFzaChjdHgtPmhIYXNoKTsKKyAg ICBjdHgtPmhIYXNoID0gTlVMTDsKIAotICAgIGlmIChjdHgtPnBiSGFzaE9iamVjdCkKLSAg ICAgICAgZnJlZShjdHgtPnBiSGFzaE9iamVjdCk7Ci0KLSAgICBtZW1zZXQoY3R4LCAwLCBz aXplb2YoX2xpYnNzaDJfd2luY25nX2hhc2hfY3R4KSk7CisgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKGN0eC0+cGJIYXNoT2JqZWN0LCBjdHgtPmR3SGFzaE9iamVjdCk7CisgICAgY3R4 LT5wYkhhc2hPYmplY3QgPSBOVUxMOworICAgIGN0eC0+ZHdIYXNoT2JqZWN0ID0gMDsKIH0K IAogCkBAIC00NDksMTcgKzQ2MywxNyBAQCBfbGlic3NoMl93aW5jbmdfa2V5X3NoYTFfdmVy aWZ5KF9saWJzc2gyX3dpbmNuZ19rZXlfY3R4ICpjdHgsCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nLmhBbGdIYXNoU0hBMSwKICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICBoYXNoLCBoYXNobGVuKTsKIAotICAgIGZyZWUoZGF0YSk7 CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGRhdGEsIGRhdGFsZW4pOwogCiAgICAgaWYg KHJldCkgewotICAgICAgICBmcmVlKGhhc2gpOworICAgICAgICBfbGlic3NoMl93aW5jbmdf bWZyZWUoaGFzaCwgaGFzaGxlbik7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9CiAKICAg ICBkYXRhbGVuID0gc2lnX2xlbjsKICAgICBkYXRhID0gbWFsbG9jKGRhdGFsZW4pOwogICAg IGlmICghZGF0YSkgewotICAgICAgICBmcmVlKGhhc2gpOworICAgICAgICBfbGlic3NoMl93 aW5jbmdfbWZyZWUoaGFzaCwgaGFzaGxlbik7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9 CiAKQEAgLTQ3NCw4ICs0ODgsOCBAQCBfbGlic3NoMl93aW5jbmdfa2V5X3NoYTFfdmVyaWZ5 KF9saWJzc2gyX3dpbmNuZ19rZXlfY3R4ICpjdHgsCiAgICAgcmV0ID0gQkNyeXB0VmVyaWZ5 U2lnbmF0dXJlKGN0eC0+aEtleSwgcFBhZGRpbmdJbmZvLAogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBoYXNoLCBoYXNobGVuLCBkYXRhLCBkYXRhbGVuLCBmbGFncyk7CiAK LSAgICBmcmVlKGhhc2gpOwotICAgIGZyZWUoZGF0YSk7CisgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKGhhc2gsIGhhc2hsZW4pOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShkYXRh LCBkYXRhbGVuKTsKIAogICAgIHJldHVybiBCQ1JZUFRfU1VDQ0VTUyhyZXQpID8gMCA6IC0x OwogfQpAQCAtNTY4LDcgKzU4Miw3IEBAIF9saWJzc2gyX3dpbmNuZ19hc25fZGVjb2RlKHVu c2lnbmVkIGNoYXIgKnBiRW5jb2RlZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHBiRW5jb2RlZCwgY2JFbmNvZGVkLCAwLCBOVUxMLAogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgcGJEZWNvZGVkLCAmY2JEZWNvZGVkKTsKICAgICBpZiAoIXJldCkgewotICAg ICAgICBmcmVlKHBiRGVjb2RlZCk7CisgICAgICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShw YkRlY29kZWQsIGNiRGVjb2RlZCk7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9CiAKQEAg LTYzOCw3ICs2NTIsNyBAQCBfbGlic3NoMl93aW5jbmdfYXNuX2RlY29kZV9ibih1bnNpZ25l ZCBjaGFyICpwYkVuY29kZWQsCiAgICAgICAgICAgICAqcHBiRGVjb2RlZCA9IHBiRGVjb2Rl ZDsKICAgICAgICAgICAgICpwY2JEZWNvZGVkID0gY2JEZWNvZGVkOwogICAgICAgICB9Ci0g ICAgICAgIGZyZWUocGJJbnRlZ2VyKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVl KHBiSW50ZWdlciwgY2JJbnRlZ2VyKTsKICAgICB9CiAKICAgICByZXR1cm4gcmV0OwpAQCAt NjgzLDEwICs2OTcsMTAgQEAgX2xpYnNzaDJfd2luY25nX2Fzbl9kZWNvZGVfYm5zKHVuc2ln bmVkIGNoYXIgKnBiRW5jb2RlZCwKICAgICAgICAgICAgICAgICAgICAgKnBjYkNvdW50ID0g bGVuZ3RoOwogICAgICAgICAgICAgICAgIH0gZWxzZSB7CiAgICAgICAgICAgICAgICAgICAg IGZvciAobGVuZ3RoID0gMDsgbGVuZ3RoIDwgaW5kZXg7IGxlbmd0aCsrKSB7Ci0gICAgICAg ICAgICAgICAgICAgICAgICBpZiAocnBiRGVjb2RlZFtsZW5ndGhdKSB7Ci0gICAgICAgICAg ICAgICAgICAgICAgICAgICAgZnJlZShycGJEZWNvZGVkW2xlbmd0aF0pOwotICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHJwYkRlY29kZWRbbGVuZ3RoXSA9IE5VTEw7Ci0gICAgICAg ICAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICAgICAgICAgICAgICBfbGlic3NoMl93 aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtsZW5ndGhdLAorICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHJjYkRlY29kZWRbbGVuZ3RoXSk7CisgICAgICAg ICAgICAgICAgICAgICAgICBycGJEZWNvZGVkW2xlbmd0aF0gPSBOVUxMOworICAgICAgICAg ICAgICAgICAgICAgICAgcmNiRGVjb2RlZFtsZW5ndGhdID0gMDsKICAgICAgICAgICAgICAg ICAgICAgfQogICAgICAgICAgICAgICAgICAgICBmcmVlKHJwYkRlY29kZWQpOwogICAgICAg ICAgICAgICAgICAgICBmcmVlKHJjYkRlY29kZWQpOwpAQCAtNjk5LDcgKzcxMyw3IEBAIF9s aWJzc2gyX3dpbmNuZ19hc25fZGVjb2RlX2Jucyh1bnNpZ25lZCBjaGFyICpwYkVuY29kZWQs CiAgICAgICAgICAgICByZXQgPSAtMTsKICAgICAgICAgfQogCi0gICAgICAgIGZyZWUocGJE ZWNvZGVkKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiRGVjb2RlZCwgY2JE ZWNvZGVkKTsKICAgICB9CiAKICAgICByZXR1cm4gcmV0OwpAQCAtODQ1LDcgKzg1OSw3IEBA IF9saWJzc2gyX3dpbmNuZ19yc2FfbmV3KGxpYnNzaDJfcnNhX2N0eCAqKnJzYSwKICAgICBy ZXQgPSBCQ3J5cHRJbXBvcnRLZXlQYWlyKF9saWJzc2gyX3dpbmNuZy5oQWxnUlNBLCBOVUxM LCBscHN6QmxvYlR5cGUsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmaEtleSwg a2V5LCBrZXlsZW4sIDApOwogICAgIGlmICghQkNSWVBUX1NVQ0NFU1MocmV0KSkgewotICAg ICAgICBmcmVlKGtleSk7CisgICAgICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShrZXksIGtl eWxlbik7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9CiAKQEAgLTg1Myw3ICs4NjcsNyBA QCBfbGlic3NoMl93aW5jbmdfcnNhX25ldyhsaWJzc2gyX3JzYV9jdHggKipyc2EsCiAgICAg KnJzYSA9IG1hbGxvYyhzaXplb2YobGlic3NoMl9yc2FfY3R4KSk7CiAgICAgaWYgKCEoKnJz YSkpIHsKICAgICAgICAgQkNyeXB0RGVzdHJveUtleShoS2V5KTsKLSAgICAgICAgZnJlZShr ZXkpOworICAgICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoa2V5LCBrZXlsZW4pOwogICAg ICAgICByZXR1cm4gLTE7CiAgICAgfQogCkBAIC04ODksNyArOTAzLDcgQEAgX2xpYnNzaDJf d2luY25nX3JzYV9uZXdfcHJpdmF0ZShsaWJzc2gyX3JzYV9jdHggKipyc2EsCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgUEtDU19SU0FfUFJJVkFURV9LRVksCiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJnBiU3RydWN0SW5mbywgJmNi U3RydWN0SW5mbyk7CiAKLSAgICBmcmVlKHBiRW5jb2RlZCk7CisgICAgX2xpYnNzaDJfd2lu Y25nX21mcmVlKHBiRW5jb2RlZCwgY2JFbmNvZGVkKTsKIAogICAgIGlmIChyZXQpIHsKICAg ICAgICAgcmV0dXJuIC0xOwpAQCAtOTAwLDcgKzkxNCw3IEBAIF9saWJzc2gyX3dpbmNuZ19y c2FfbmV3X3ByaXZhdGUobGlic3NoMl9yc2FfY3R4ICoqcnNhLAogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgTEVHQUNZX1JTQVBSSVZBVEVfQkxPQiwgJmhLZXksCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBwYlN0cnVjdEluZm8sIGNiU3RydWN0SW5mbywgMCk7 CiAgICAgaWYgKCFCQ1JZUFRfU1VDQ0VTUyhyZXQpKSB7Ci0gICAgICAgIGZyZWUocGJTdHJ1 Y3RJbmZvKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiU3RydWN0SW5mbywg Y2JTdHJ1Y3RJbmZvKTsKICAgICAgICAgcmV0dXJuIC0xOwogICAgIH0KIApAQCAtOTA4LDcg KzkyMiw3IEBAIF9saWJzc2gyX3dpbmNuZ19yc2FfbmV3X3ByaXZhdGUobGlic3NoMl9yc2Ff Y3R4ICoqcnNhLAogICAgICpyc2EgPSBtYWxsb2Moc2l6ZW9mKGxpYnNzaDJfcnNhX2N0eCkp OwogICAgIGlmICghKCpyc2EpKSB7CiAgICAgICAgIEJDcnlwdERlc3Ryb3lLZXkoaEtleSk7 Ci0gICAgICAgIGZyZWUocGJTdHJ1Y3RJbmZvKTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKHBiU3RydWN0SW5mbywgY2JTdHJ1Y3RJbmZvKTsKICAgICAgICAgcmV0dXJuIC0x OwogICAgIH0KIApAQCAtOTgyLDcgKzk5Niw3IEBAIF9saWJzc2gyX3dpbmNuZ19yc2Ffc2hh MV9zaWduKExJQlNTSDJfU0VTU0lPTiAqc2Vzc2lvbiwKICAgICAgICAgICAgIHJldCA9IFNU QVRVU19OT19NRU1PUlk7CiAgICAgfQogCi0gICAgZnJlZShkYXRhKTsKKyAgICBfbGlic3No Ml93aW5jbmdfbWZyZWUoZGF0YSwgZGF0YWxlbik7CiAKICAgICByZXR1cm4gQkNSWVBUX1NV Q0NFU1MocmV0KSA/IDAgOiAtMTsKIH0KQEAgLTk5NCwxMiArMTAwOCwxMCBAQCBfbGlic3No Ml93aW5jbmdfcnNhX2ZyZWUobGlic3NoMl9yc2FfY3R4ICpyc2EpCiAgICAgICAgIHJldHVy bjsKIAogICAgIEJDcnlwdERlc3Ryb3lLZXkocnNhLT5oS2V5KTsKKyAgICByc2EtPmhLZXkg PSBOVUxMOwogCi0gICAgaWYgKHJzYS0+cGJLZXlPYmplY3QpCi0gICAgICAgIGZyZWUocnNh LT5wYktleU9iamVjdCk7Ci0KLSAgICBtZW1zZXQocnNhLCAwLCBzaXplb2YobGlic3NoMl9y c2FfY3R4KSk7Ci0gICAgZnJlZShyc2EpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShy c2EtPnBiS2V5T2JqZWN0LCByc2EtPmNiS2V5T2JqZWN0KTsKKyAgICBfbGlic3NoMl93aW5j bmdfbWZyZWUocnNhLCBzaXplb2YobGlic3NoMl9yc2FfY3R4KSk7CiB9CiAKIApAQCAtMTA5 Myw3ICsxMTA1LDcgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9uZXcobGlic3NoMl9kc2FfY3R4 ICoqZHNhLAogICAgIHJldCA9IEJDcnlwdEltcG9ydEtleVBhaXIoX2xpYnNzaDJfd2luY25n LmhBbGdEU0EsIE5VTEwsIGxwc3pCbG9iVHlwZSwKICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICZoS2V5LCBrZXksIGtleWxlbiwgMCk7CiAgICAgaWYgKCFCQ1JZUFRfU1VDQ0VT UyhyZXQpKSB7Ci0gICAgICAgIGZyZWUoa2V5KTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKGtleSwga2V5bGVuKTsKICAgICAgICAgcmV0dXJuIC0xOwogICAgIH0KIApAQCAt MTEwMSw3ICsxMTEzLDcgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9uZXcobGlic3NoMl9kc2Ff Y3R4ICoqZHNhLAogICAgICpkc2EgPSBtYWxsb2Moc2l6ZW9mKGxpYnNzaDJfZHNhX2N0eCkp OwogICAgIGlmICghKCpkc2EpKSB7CiAgICAgICAgIEJDcnlwdERlc3Ryb3lLZXkoaEtleSk7 Ci0gICAgICAgIGZyZWUoa2V5KTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGtl eSwga2V5bGVuKTsKICAgICAgICAgcmV0dXJuIC0xOwogICAgIH0KIApAQCAtMTEzNSw3ICsx MTQ3LDcgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9uZXdfcHJpdmF0ZShsaWJzc2gyX2RzYV9j dHggKipkc2EsCiAgICAgcmV0ID0gX2xpYnNzaDJfd2luY25nX2Fzbl9kZWNvZGVfYm5zKHBi RW5jb2RlZCwgY2JFbmNvZGVkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAmcnBiRGVjb2RlZCwgJnJjYkRlY29kZWQsICZsZW5ndGgpOwogCi0gICAgZnJl ZShwYkVuY29kZWQpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShwYkVuY29kZWQsIGNi RW5jb2RlZCk7CiAKICAgICBpZiAocmV0KSB7CiAgICAgICAgIHJldHVybiAtMTsKQEAgLTEx NTQsMTAgKzExNjYsOSBAQCBfbGlic3NoMl93aW5jbmdfZHNhX25ld19wcml2YXRlKGxpYnNz aDJfZHNhX2N0eCAqKmRzYSwKICAgICB9CiAKICAgICBmb3IgKGluZGV4ID0gMDsgaW5kZXgg PCBsZW5ndGg7IGluZGV4KyspIHsKLSAgICAgICAgaWYgKHJwYkRlY29kZWRbaW5kZXhdKSB7 Ci0gICAgICAgICAgICBmcmVlKHJwYkRlY29kZWRbaW5kZXhdKTsKLSAgICAgICAgICAgIHJw YkRlY29kZWRbaW5kZXhdID0gTlVMTDsKLSAgICAgICAgfQorICAgICAgICBfbGlic3NoMl93 aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtpbmRleF0sIHJjYkRlY29kZWRbaW5kZXhdKTsKKyAg ICAgICAgcnBiRGVjb2RlZFtpbmRleF0gPSBOVUxMOworICAgICAgICByY2JEZWNvZGVkW2lu ZGV4XSA9IDA7CiAgICAgfQogCiAgICAgZnJlZShycGJEZWNvZGVkKTsKQEAgLTEyMTUsMTQg KzEyMjYsMTQgQEAgX2xpYnNzaDJfd2luY25nX2RzYV9zaGExX3NpZ24obGlic3NoMl9kc2Ff Y3R4ICpkc2EsCiAgICAgICAgICAgICAgICAgICAgIG1lbWNweShzaWdfZml4ZWQsIHNpZywg c2lnbGVuKTsKICAgICAgICAgICAgICAgICB9CiAKLSAgICAgICAgICAgICAgICBmcmVlKHNp Zyk7CisgICAgICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHNpZywgc2lnbGVu KTsKICAgICAgICAgICAgIH0gZWxzZQogICAgICAgICAgICAgICAgIHJldCA9IFNUQVRVU19O T19NRU1PUlk7CiAgICAgICAgIH0gZWxzZQogICAgICAgICAgICAgcmV0ID0gU1RBVFVTX05P X01FTU9SWTsKICAgICB9CiAKLSAgICBmcmVlKGRhdGEpOworICAgIF9saWJzc2gyX3dpbmNu Z19tZnJlZShkYXRhLCBkYXRhbGVuKTsKIAogICAgIHJldHVybiBCQ1JZUFRfU1VDQ0VTUyhy ZXQpID8gMCA6IC0xOwogfQpAQCAtMTIzNCwxMiArMTI0NSwxMCBAQCBfbGlic3NoMl93aW5j bmdfZHNhX2ZyZWUobGlic3NoMl9kc2FfY3R4ICpkc2EpCiAgICAgICAgIHJldHVybjsKIAog ICAgIEJDcnlwdERlc3Ryb3lLZXkoZHNhLT5oS2V5KTsKKyAgICBkc2EtPmhLZXkgPSBOVUxM OwogCi0gICAgaWYgKGRzYS0+cGJLZXlPYmplY3QpCi0gICAgICAgIGZyZWUoZHNhLT5wYktl eU9iamVjdCk7Ci0KLSAgICBtZW1zZXQoZHNhLCAwLCBzaXplb2YobGlic3NoMl9kc2FfY3R4 KSk7Ci0gICAgZnJlZShkc2EpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShkc2EtPnBi S2V5T2JqZWN0LCBkc2EtPmNiS2V5T2JqZWN0KTsKKyAgICBfbGlic3NoMl93aW5jbmdfbWZy ZWUoZHNhLCBzaXplb2YobGlic3NoMl9kc2FfY3R4KSk7CiB9CiAjZW5kaWYKIApAQCAtMTI4 OSw3ICsxMjk4LDcgQEAgX2xpYnNzaDJfd2luY25nX3B1Yl9wcml2X2tleWZpbGUoTElCU1NI Ml9TRVNTSU9OICpzZXNzaW9uLAogICAgIHJldCA9IF9saWJzc2gyX3dpbmNuZ19hc25fZGVj b2RlX2JucyhwYkVuY29kZWQsIGNiRW5jb2RlZCwKICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgJnJwYkRlY29kZWQsICZyY2JEZWNvZGVkLCAmbGVuZ3RoKTsK IAotICAgIGZyZWUocGJFbmNvZGVkKTsKKyAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUocGJF bmNvZGVkLCBjYkVuY29kZWQpOwogCiAgICAgaWYgKHJldCkgewogICAgICAgICByZXR1cm4g LTE7CkBAIC0xMzYyLDEwICsxMzcxLDkgQEAgX2xpYnNzaDJfd2luY25nX3B1Yl9wcml2X2tl eWZpbGUoTElCU1NIMl9TRVNTSU9OICpzZXNzaW9uLAogCiAKICAgICBmb3IgKGluZGV4ID0g MDsgaW5kZXggPCBsZW5ndGg7IGluZGV4KyspIHsKLSAgICAgICAgaWYgKHJwYkRlY29kZWRb aW5kZXhdKSB7Ci0gICAgICAgICAgICBmcmVlKHJwYkRlY29kZWRbaW5kZXhdKTsKLSAgICAg ICAgICAgIHJwYkRlY29kZWRbaW5kZXhdID0gTlVMTDsKLSAgICAgICAgfQorICAgICAgICBf bGlic3NoMl93aW5jbmdfbWZyZWUocnBiRGVjb2RlZFtpbmRleF0sIHJjYkRlY29kZWRbaW5k ZXhdKTsKKyAgICAgICAgcnBiRGVjb2RlZFtpbmRleF0gPSBOVUxMOworICAgICAgICByY2JE ZWNvZGVkW2luZGV4XSA9IDA7CiAgICAgfQogCiAgICAgZnJlZShycGJEZWNvZGVkKTsKQEAg LTE0NjEsMTAgKzE0NjksMTAgQEAgX2xpYnNzaDJfd2luY25nX2NpcGhlcl9pbml0KF9saWJz c2gyX2NpcGhlcl9jdHggKmN0eCwKICAgICByZXQgPSBCQ3J5cHRJbXBvcnRLZXkoKnR5cGUu cGhBbGcsIE5VTEwsIEJDUllQVF9LRVlfREFUQV9CTE9CLCAmaEtleSwKICAgICAgICAgICAg ICAgICAgICAgICAgICAgcGJLZXlPYmplY3QsIGR3S2V5T2JqZWN0LCBrZXksIGtleWxlbiwg MCk7CiAKLSAgICBmcmVlKGtleSk7CisgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGtleSwg a2V5bGVuKTsKIAogICAgIGlmICghQkNSWVBUX1NVQ0NFU1MocmV0KSkgewotICAgICAgICBm cmVlKHBiS2V5T2JqZWN0KTsKKyAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKHBiS2V5 T2JqZWN0LCBkd0tleU9iamVjdCk7CiAgICAgICAgIHJldHVybiAtMTsKICAgICB9CiAKQEAg LTE0NzIsNyArMTQ4MCw3IEBAIF9saWJzc2gyX3dpbmNuZ19jaXBoZXJfaW5pdChfbGlic3No Ml9jaXBoZXJfY3R4ICpjdHgsCiAgICAgICAgIHBiSVYgPSBtYWxsb2MoZHdCbG9ja0xlbmd0 aCk7CiAgICAgICAgIGlmICghcGJJVikgewogICAgICAgICAgICAgQkNyeXB0RGVzdHJveUtl eShoS2V5KTsKLSAgICAgICAgICAgIGZyZWUocGJLZXlPYmplY3QpOworICAgICAgICAgICAg X2xpYnNzaDJfd2luY25nX21mcmVlKHBiS2V5T2JqZWN0LCBkd0tleU9iamVjdCk7CiAgICAg ICAgICAgICByZXR1cm4gLTE7CiAgICAgICAgIH0KICAgICAgICAgZHdJViA9IGR3QmxvY2tM ZW5ndGg7CkBAIC0xNTMxLDcgKzE1MzksNyBAQCBfbGlic3NoMl93aW5jbmdfY2lwaGVyX2Ny eXB0KF9saWJzc2gyX2NpcGhlcl9jdHggKmN0eCwKICAgICAgICAgICAgICAgICBtZW1jcHko YmxvY2ssIHBiT3V0cHV0LCBjYk91dHB1dCk7CiAgICAgICAgICAgICB9CiAKLSAgICAgICAg ICAgIGZyZWUocGJPdXRwdXQpOworICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVl KHBiT3V0cHV0LCBjYk91dHB1dCk7CiAgICAgICAgIH0gZWxzZQogICAgICAgICAgICAgcmV0 ID0gU1RBVFVTX05PX01FTU9SWTsKICAgICB9CkBAIC0xNTQzLDEzICsxNTUxLDExIEBAIHZv aWQKIF9saWJzc2gyX3dpbmNuZ19jaXBoZXJfZHRvcihfbGlic3NoMl9jaXBoZXJfY3R4ICpj dHgpCiB7CiAgICAgQkNyeXB0RGVzdHJveUtleShjdHgtPmhLZXkpOworICAgIGN0eC0+aEtl eSA9IE5VTEw7CiAKLSAgICBpZiAoY3R4LT5wYktleU9iamVjdCkgewotICAgICAgICBmcmVl KGN0eC0+cGJLZXlPYmplY3QpOwotICAgICAgICBjdHgtPnBiS2V5T2JqZWN0ID0gTlVMTDsK LSAgICB9Ci0KLSAgICBtZW1zZXQoY3R4LCAwLCBzaXplb2YoX2xpYnNzaDJfY2lwaGVyX2N0 eCkpOworICAgIF9saWJzc2gyX3dpbmNuZ19tZnJlZShjdHgtPnBiS2V5T2JqZWN0LCBjdHgt PmR3S2V5T2JqZWN0KTsKKyAgICBjdHgtPnBiS2V5T2JqZWN0ID0gTlVMTDsKKyAgICBjdHgt PmR3S2V5T2JqZWN0ID0gMDsKIH0KIAogCkBAIC0xNTgxLDYgKzE1ODcsMTIgQEAgX2xpYnNz aDJfd2luY25nX2JpZ251bV9yZXNpemUoX2xpYnNzaDJfYm4gKmJuLCB1bnNpZ25lZCBsb25n IGxlbmd0aCkKICAgICBpZiAobGVuZ3RoID09IGJuLT5sZW5ndGgpCiAgICAgICAgIHJldHVy biAwOwogCisjaWZkZWYgTElCU1NIMl9DTEVBUl9NRU1PUlkKKyAgICBpZiAoYm4tPmJpZ251 bSAmJiBibi0+bGVuZ3RoID4gMCAmJiBsZW5ndGggPCBibi0+bGVuZ3RoKSB7CisgICAgICAg IFNlY3VyZVplcm9NZW1vcnkoYm4tPmJpZ251bSArIGxlbmd0aCwgYm4tPmxlbmd0aCAtIGxl bmd0aCk7CisgICAgfQorI2VuZGlmCisKICAgICBiaWdudW0gPSByZWFsbG9jKGJuLT5iaWdu dW0sIGxlbmd0aCk7CiAgICAgaWYgKCFiaWdudW0pCiAgICAgICAgIHJldHVybiAtMTsKQEAg LTE2ODgsNyArMTcwMCw3IEBAIF9saWJzc2gyX3dpbmNuZ19iaWdudW1fbW9kX2V4cChfbGli c3NoMl9ibiAqciwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBy LT5iaWdudW0sIHItPmxlbmd0aCwgJm9mZnNldCwKICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBCQ1JZUFRfUEFEX05PTkUpOwogCi0gICAgICAgICAgICAgICAg ICAgIGZyZWUoYmlnbnVtKTsKKyAgICAgICAgICAgICAgICAgICAgX2xpYnNzaDJfd2luY25n X21mcmVlKGJpZ251bSwgbGVuZ3RoKTsKIAogICAgICAgICAgICAgICAgICAgICBpZiAoQkNS WVBUX1NVQ0NFU1MocmV0KSkgewogICAgICAgICAgICAgICAgICAgICAgICAgX2xpYnNzaDJf d2luY25nX2JpZ251bV9yZXNpemUociwgb2Zmc2V0KTsKQEAgLTE3MDIsNyArMTcxNCw3IEBA IF9saWJzc2gyX3dpbmNuZ19iaWdudW1fbW9kX2V4cChfbGlic3NoMl9ibiAqciwKICAgICAg ICAgQkNyeXB0RGVzdHJveUtleShoS2V5KTsKICAgICB9CiAKLSAgICBmcmVlKGtleSk7Cisg ICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGtleSwga2V5bGVuKTsKIAogICAgIHJldHVybiBC Q1JZUFRfU1VDQ0VTUyhyZXQpID8gMCA6IC0xOwogfQpAQCAtMTc4MCw2ICsxNzkyLDEwIEBA IF9saWJzc2gyX3dpbmNuZ19iaWdudW1fZnJvbV9iaW4oX2xpYnNzaDJfYm4gKmJuLCB1bnNp Z25lZCBsb25nIGxlbiwKICAgICBpZiAob2Zmc2V0ID4gMCkgewogICAgICAgICBtZW1tb3Zl KGJuLT5iaWdudW0sIGJuLT5iaWdudW0gKyBvZmZzZXQsIGxlbmd0aCk7CiAKKyNpZmRlZiBM SUJTU0gyX0NMRUFSX01FTU9SWQorICAgICAgICBTZWN1cmVaZXJvTWVtb3J5KGJuLT5iaWdu dW0gKyBsZW5ndGgsIG9mZnNldCk7CisjZW5kaWYKKwogICAgICAgICBiaWdudW0gPSByZWFs bG9jKGJuLT5iaWdudW0sIGxlbmd0aCk7CiAgICAgICAgIGlmIChiaWdudW0pIHsKICAgICAg ICAgICAgIGJuLT5iaWdudW0gPSBiaWdudW07CkBAIC0xODAxLDExICsxODE3LDExIEBAIF9s aWJzc2gyX3dpbmNuZ19iaWdudW1fZnJlZShfbGlic3NoMl9ibiAqYm4pCiB7CiAgICAgaWYg KGJuKSB7CiAgICAgICAgIGlmIChibi0+YmlnbnVtKSB7Ci0gICAgICAgICAgICBmcmVlKGJu LT5iaWdudW0pOworICAgICAgICAgICAgX2xpYnNzaDJfd2luY25nX21mcmVlKGJuLT5iaWdu dW0sIGJuLT5sZW5ndGgpOwogICAgICAgICAgICAgYm4tPmJpZ251bSA9IE5VTEw7CiAgICAg ICAgIH0KICAgICAgICAgYm4tPmxlbmd0aCA9IDA7Ci0gICAgICAgIGZyZWUoYm4pOworICAg ICAgICBfbGlic3NoMl93aW5jbmdfbWZyZWUoYm4sIHNpemVvZihfbGlic3NoMl9ibikpOwog ICAgIH0KIH0KIAotLSAKMS45LjIubXN5c2dpdC4wCgo= --------------070807080507030309000600-- --uUXdGUT2uI9rHdvE0BV7a97b8I7s4vbMB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTpYekAAoJEPhLrsrSvXft6qsIAM5co/TPfDihwu37e4lNkemV WgWfLmssi1XnCs6jaj0mccqN/UA3AviX+X6ftn0CZem/PRVvB+m9cPqa6aVSy3kG fjlpS8E+i9Q0OeH2H3y36SAcXA6Da8/xetpcLSMSjOJI+Ftxnc08Cx/KAO5XjV59 hVBGgOLb9aXKUYkjyzVZ6QtiTKxkeY3jAfk+t7xaqNuRbXV56Qb/85xoFw0Wt4Yc 5eXPLIm1t94RxqLc0vkrhkUtGXLBFXPGYv6QXwSJwoEOAjvMRY3hUdPxNbKB/oJ/ X/qeEEz+hwEttgfOfTPyvlaETUFwEZ4r5cyKZJn6u4fVbSk7OYlaO4Y6jgA1fqM= =k1YT -----END PGP SIGNATURE----- --uUXdGUT2uI9rHdvE0BV7a97b8I7s4vbMB-- --===============2108300891== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============2108300891==-- From libssh2-devel-bounces@cool.haxx.se Mon Jun 23 22:33:23 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5NKWnkQ004237; Mon, 23 Jun 2014 22:33:15 +0200 Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5NKWlYh004099 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Mon, 23 Jun 2014 22:32:48 +0200 Received: by mail-vc0-f181.google.com with SMTP id il7so6712138vcb.12 for ; Mon, 23 Jun 2014 13:32:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=JRqG/588TOTnwYZm4MAPSd+GjwVWYlgih47mzk6lUGU=; b=BgF9W1Pca9c0eXHMs+XbUBapwdUXt/YYMsmS7kVjybn7e9gQcBt1Preae44ulee81O GkHITR7X4f37K2GKwF5PxZ5EsmCwuKpxpkX9Fh17ffMXrPL/G3ruqwaU2kKjQ2/MeST8 GgATCgCw2PgpYKadUYzHw3GHfYNKgpECmJpJXIq9rgUc+3Dksu2tEJjYJjpkB/ocykXc gSPxFBtkZj6EhKxbfGG7m59mzRogQPOrj2pDjsdUBA9KkS3ZQf5mR1Fqa+6/rzdyL7UB /ysv14+5noC4eBKOyYq5dR4M/gc1TeUNDoXz7+EDVHYiWKUsOS/jSWhHp4H5WYE/CvXn s4lA== X-Received: by 10.58.152.194 with SMTP id va2mr17382veb.54.1403555563233; Mon, 23 Jun 2014 13:32:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Mon, 23 Jun 2014 13:32:23 -0700 (PDT) From: Eduardo Silva Date: Mon, 23 Jun 2014 14:32:23 -0600 Message-ID: Subject: Multiple channels and epoll(7) To: libssh2-devel@cool.haxx.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5NKWnkQ004237 Hi, i was searching around for a mechanism to listen in a local socket for libssh2 session events through epoll and handle properly multiple channels, i ended up on this thread: https://www.mail-archive.com/libssh2-devel@cool.haxx.se/msg03737.html as of today, is there a mechanism to perform specific operations on specific events ?, i am mostly interested into know when to accept a channel request and when to read for channels waiting for data. thanks for your help, -- Eduardo Silva http://edsiper.linuxchile.cl http://monkey-project.com _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Jun 24 02:21:43 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O0LCwT026753; Tue, 24 Jun 2014 02:21:38 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O0L9DZ026709 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 24 Jun 2014 02:21:09 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5O0L93w024722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 24 Jun 2014 02:21:10 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5O0L7s3005384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 24 Jun 2014 02:21:08 +0200 Message-ID: <1403569267.14025.4.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Tue, 24 Jun 2014 02:21:07 +0200 In-Reply-To: References: X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_24_48 autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5O0LCwT026753 mån 2014-06-23 klockan 14:32 -0600 skrev Eduardo Silva: > as of today, is there a mechanism to perform specific operations on > specific events ?, i am mostly interested into know when to accept a > channel request and when to read for channels waiting for data. Not that I can see. You need to try to perform read/write on each active channel & stream to see if there is anything to perform whenever the session fd is active in epoll. How many channels or streams do you need to handle? Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Jun 24 02:39:59 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O0dlu7014898; Tue, 24 Jun 2014 02:39:57 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O0dkiA014893 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 24 Jun 2014 02:39:46 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5O0dkB3024758 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 24 Jun 2014 02:39:48 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5O0djAj005759 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 24 Jun 2014 02:39:45 +0200 Message-ID: <1403570385.14025.12.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Tue, 24 Jun 2014 02:39:45 +0200 In-Reply-To: <1403569267.14025.4.camel@localhost> References: <1403569267.14025.4.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_24_48 autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5O0dlu7014898 tis 2014-06-24 klockan 02:21 +0200 skrev Henrik Nordström: > Not that I can see. You need to try to perform read/write on each active > channel & stream to see if there is anything to perform whenever the > session fd is active in epoll. Thinking on this. Would it make sense to have an API to query what the next packet in the receive queue is about? This would allow multi-channel applications to scale better with number of channels by avoiding to repeatedly loop over all channels (&streams) whenever there is any activity. Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Jun 24 03:30:49 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O1USbH004707; Tue, 24 Jun 2014 03:30:46 +0200 Received: from mail-ve0-x22e.google.com (mail-ve0-x22e.google.com [IPv6:2607:f8b0:400c:c01::22e]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O1UPQV004321 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Tue, 24 Jun 2014 03:30:26 +0200 Received: by mail-ve0-f174.google.com with SMTP id jx11so6863721veb.33 for ; Mon, 23 Jun 2014 18:30:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=QwasUvlVSFICYFvZMCOcRZuxo5cZIUB+Js6Tknq2Wt0=; b=ewEUDvo6S/KtvlPsiOuoWu89qMZVjVyCQCpDOWD4H7HE/v2BPtN35Cv4jIqRVJafAY 73HQQECtUv17iUT3EzpX8R4TbRJlGKyG2Cun9pp4YdNPFrtCCz577RNFigsMpDMEzpav hgt4sBIcXnwk9kdFaglpUAV4+I4i50fb2XgqTJ1mNmGVDGniKKqDOn7gcCbt96UMG1MZ Ksd7J8FCM1ieUm7J64fe3hvQYB8I/TTP9/qnerux7TZk+uYIIj3DVoWsfHVl62bHPQ9g E53Jv9HefSCGBL9aA4OOqeIiaQwmdSwqErsCaJfFBiXWLBFDRBBKKihHSyzC55JO/uJs Oi/A== X-Received: by 10.221.64.80 with SMTP id xh16mr172499vcb.35.1403573421367; Mon, 23 Jun 2014 18:30:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Mon, 23 Jun 2014 18:30:01 -0700 (PDT) In-Reply-To: <1403570385.14025.12.camel@localhost> References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> From: Eduardo Silva Date: Mon, 23 Jun 2014 19:30:01 -0600 Message-ID: Subject: Re: Multiple channels and epoll(7) To: libssh2 development X-MIME-Autoconverted: from quoted-printable to 8bit by giant.haxx.se id s5O1UPQV004321 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5O1USbH004707 On Mon, Jun 23, 2014 at 6:39 PM, Henrik Nordström wrote: > tis 2014-06-24 klockan 02:21 +0200 skrev Henrik Nordström: > >> Not that I can see. You need to try to perform read/write on each active >> channel & stream to see if there is anything to perform whenever the >> session fd is active in epoll. > > Thinking on this. Would it make sense to have an API to query what the > next packet in the receive queue is about? This would allow > multi-channel applications to scale better with number of channels by > avoiding to repeatedly loop over all channels (&streams) whenever there > is any activity. > in response to last two emails (there is one missing in my inbox): - if we could determinate the type of packet would be awesome. - the number of channels to handle may vary between 10 and 20. I do not worry too much about performance at this point. just to clarify, whats the real difference between a channel and a stream in libssh2 ? thanks, _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Jun 24 05:07:47 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O37STV012285; Tue, 24 Jun 2014 05:07:44 +0200 Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5O37QCL012194 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Tue, 24 Jun 2014 05:07:27 +0200 Received: by mail-lb0-f172.google.com with SMTP id c11so5773970lbj.3 for ; Mon, 23 Jun 2014 20:07:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-type; bh=XHDfBHdspqo2OQhA2aJSYP87zvcroC9FAxvFEnOsWNU=; b=k8dqITADHiCbBUyJDXC/j9WvfejNCJPIHJS17uFHsBbfsj7J/UP8XvL3o31XRv+Zv2 K4yBYnqjCMDMaZ18G0+NL8xJ7i8CpOwqIUsTPGUzQGQatbE1Fr0fUpNUpvQjR2s6NQ1V Z3PedRH6dXFfBkYEma8j9YtvPmHsWFov/CsUQfkQHcIGXRz1czF6jX/u0pDWz8UZH/1z x/IJO6WneACmqqt6NUWjedcayOqRf1NoriRMUvVTQJQaUa6njbskK58nc8st/jKSuiPe melpylxsny3mqU6fmtktEzqQn823PQeREg2jsgggAkX8vtOwp3nXFN+DS5RJw00zGBDU e7vg== X-Gm-Message-State: ALoCoQnH+gM/SPYOG4gix3/MlWK8nC/FEv6Jc9PFAMw9t2UpzluOfBfuu0GGbdpCrvNvjrSLT1If X-Received: by 10.112.46.97 with SMTP id u1mr19170730lbm.50.1403579241704; Mon, 23 Jun 2014 20:07:21 -0700 (PDT) From: Nitin Deokate References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQCh8ANGY5tkZe+/B6MGsd0mHP7SDwHImRp1AwIAF0GdtHx60A== Date: Tue, 24 Jun 2014 08:37:21 +0530 Message-ID: Subject: RE: Questions about libssh2_sftp_read() To: libssh2-devel@cool.haxx.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0611771035==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============0611771035== Content-Type: multipart/alternative; boundary=001a113497e61dc86604fc8c4231 --001a113497e61dc86604fc8c4231 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Guys, I have used this test case http://www.libssh2.org/examples/sftp.html I have modified the *char* mem[1024]; to *char* mem[2500]; I am trying to read the 1gb file. First time libssh2_sftp_read() returns 2000 bytes, And even in subsequent read calls too. But last response from Daniel says that I should get bigger chunks in subsequent call, which Is not happening. Do I need to change something to get me bigger chunks. Thanks Nitin *From:* Nitin Deokate [mailto:ndeokate@qualys.com] *Sent:* Sunday, June 15, 2014 6:05 AM *To:* Daniel Stenberg *Cc:* libssh2 development *Subject:* Re: Questions about libssh2_sftp_read() Thanks for the reply. In subsequent reads too, I get smaller chunk. Could you please suggest what can be done in my code to have more subsequent calls? any test case you have ? On Sun, Jun 15, 2014 at 2:34 AM, Daniel Stenberg wrote: On Sat, 14 Jun 2014, Nitin Deokate wrote: 1. I have an application, where I use libssh2_sftp_read(), and I pass larger buffer(say 8K to 16MB) to same function, What I expect is, data of same bytes, but all I get is 2000Bytes. No, that is probably all you get in the first read call. That's quite a difference. In subsequent reads you are likely to get larger pieces. What could help me to get as equal to the buffer size I passed and not 2000 bytes? If you have less latency to the server you may get more, but the first call is likely to always just give you a small piece. 2. Is it any significant reason for selecting value for #define MAX_SFTP_READ_SIZE 2000 Why it can=E2=80=99t have more bytes than that? It can, just bump it. But you will not get the amount you ask for at once, you can up to that amount, then you call the function over and over again until you're done. Has anybody faced this scenario before, please revert as early as possible. I don't even understand your scenario. We have users downloading insane amounts of data over SFTP with no problems. This is not a known problem you're talking about. The 2K number is simply the "block size". libssh2 sends read MANY requests with that size and as soon as one comes back it can return data. When you call read again, more packets might already have arrived and can be returned and so on. I blogged about this technique when I made this change: http://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers-fast/ --=20 / daniel.haxx.se --001a113497e61dc86604fc8c4231 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Gu= ys,

I have used this test cas= e

http://www.libssh2.org/examples/sf= tp.html

I have modified =
the char mem[1024]; to char mem[2500];
I am trying to r=
ead the 1gb file. First time libssh2_sftp_read() returns 2000 bytes, 
And even in subs=
equent read calls too.
=C2=A0
But last respons=
e from Daniel says that I should get bigger chunks in subsequent call, whic=
h 
Is not happening=
. Do I need to change something to get me bigger chunks.
=C2=A0
Thanks
Nitin
=C2=A0


From: Nitin De=
okate [mailto:ndeokate@qualys.com] 

Sent: Sunday, June 15, 2014 6:05 AM
To: Daniel StenbergCc: libssh2 development
Subject: Re: Questions about libs=
sh2_sftp_read()
=C2=A0
=C2=A0
On Sun, Jun 15, 2014 at 2:34 AM, Daniel Stenberg <daniel@haxx.se> wrote:


On Sat, 14 Jun 2=
014, Nitin Deokate wrote:
1. =C2=A0 =C2=A0 =C2=A0=
 I have an application, where I use libssh2_sftp_read(), and I pass
larg=
er buffer(say 8K to 16MB) to same function,


What I expect is, data of same bytes, but all I get is 2000Bytes.
=C2=A0
No, that is pro=
bably all you get in the first read call. That's quite a difference. In=
 subsequent reads you are likely to get larger pieces.


=C2=A0
What c=
ould help me to get as equal to the buffer size I passed and not 2000 bytes=
?


=C2=A0
=
If you have less latency to the server you may get more, but the first call=
 is likely to always just give you a small piece.


=C2=A0
2. =C2=A0 =C2=A0 =C2=A0 Is it any significant reason for select=
ing value for

#define MAX_SFTP_READ_SIZE 2000

Why it can=E2=
=80=99t have more bytes than that?


=C2=A0
=
It can, just bump it. But you will not get the amount you ask for at once, =
you can up to that amount, then you call the function over and over again u=
ntil you're done.


=C2=A0
Has an=
ybody faced this scenario before, please revert as early as possible.


=C2=A0
=
I don't even understand your scenario. We have users downloading insane=
 amounts of data over SFTP with no problems. This is not a known problem yo=
u're talking about.


The 2K number is simply the "block size". libssh2 sends read =
MANY requests with that size and as soon as one comes back it can return da=
ta. When you call read again, more packets might already have arrived and c=
an be returned and so on. I blogged about this technique when I made this c=
hange:


=C2=A0 http://daniel.haxx.se/blog/2010/12/08/makin=
g-sftp-transfers-fast/

-- 


=C2=A0/ daniel.haxx.se
=C2=A0
--001a113497e61dc86604fc8c4231-- --===============0611771035== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============0611771035==-- From libssh2-devel-bounces@cool.haxx.se Tue Jun 24 16:02:30 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5OE21xu016387; Tue, 24 Jun 2014 16:02:22 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5OE1x5V016349 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 24 Jun 2014 16:01:59 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5OE1xoj028764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 24 Jun 2014 16:02:00 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5OE1vC2019913 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 24 Jun 2014 16:01:58 +0200 Message-ID: <1403618517.19230.7.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Tue, 24 Jun 2014 16:01:57 +0200 In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5OE21xu016387 mån 2014-06-23 klockan 19:30 -0600 skrev Eduardo Silva: > On Mon, Jun 23, 2014 at 6:39 PM, Henrik Nordström > wrote: > > tis 2014-06-24 klockan 02:21 +0200 skrev Henrik Nordström: > > > >> Not that I can see. You need to try to perform read/write on each active > >> channel & stream to see if there is anything to perform whenever the > >> session fd is active in epoll. > > > > Thinking on this. Would it make sense to have an API to query what the > > next packet in the receive queue is about? This would allow > > multi-channel applications to scale better with number of channels by > > avoiding to repeatedly loop over all channels (&streams) whenever there > > is any activity. > > > > in response to last two emails (there is one missing in my inbox): Don't know what happened with my first response. I never got it back via the list, but is available in the list archives. > - if we could determinate the type of packet would be awesome. It would be very easy to add an API that tells you which is the next channel / stream that have activity in the packet queue. Question is if it's the right thing or not, and if it is, what should such API look like? How much detail need to be exposed to the application for it to know what libssh2 operation(s) is need to continue with? > - the number of channels to handle may vary between 10 and 20. I do > not worry too much about performance at this point. So for you simple polling of all your channels is not really a problem. > just to clarify, whats the real difference between a channel and a > stream in libssh2 ? Each channel can be divided into N streams. Each stream is processed individually by libssh_channel_read_ex() (without _ex is only a wrapper to process stream #0), but windowing is performed on the channel as a whole if I am not mistaken. Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Jun 25 01:13:59 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5ONDTPI016485; Wed, 25 Jun 2014 01:13:52 +0200 Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5ONDR3i016471 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 25 Jun 2014 01:13:27 +0200 Received: from localhost (dast@localhost) by giant.haxx.se (8.14.4/8.14.4/Submit) with ESMTP id s5ONDR5H016467 for ; Wed, 25 Jun 2014 01:13:27 +0200 X-Authentication-Warning: giant.haxx.se: dast owned process doing -bs Date: Wed, 25 Jun 2014 01:13:27 +0200 (CEST) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: Questions about libssh2_sftp_read() In-Reply-To: Message-ID: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5ONDTPI016485 On Tue, 24 Jun 2014, Nitin Deokate wrote: > I have modified the *char* mem[1024]; to *char* mem[2500]; But what if you make it 100000 or 500000 bytes big instead? > But last response from Daniel says that I should get bigger chunks in > subsequent call, which is not happening. Do I need to change something to > get me bigger chunks. I think 2500 is just too small buffer to get much data with. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Jun 25 05:12:41 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P3CHQW009983; Wed, 25 Jun 2014 05:12:36 +0200 Received: from mail-la0-f42.google.com (mail-la0-f42.google.com [209.85.215.42]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P3CEmS009963 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Wed, 25 Jun 2014 05:12:14 +0200 Received: by mail-la0-f42.google.com with SMTP id pn19so292070lab.15 for ; Tue, 24 Jun 2014 20:12:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=0+hKpOI13wJwHBSDitqFrNXeIPcDm2FpvXkIrh8DQJI=; b=Op5hGJah4eIEC8SbFULsLLT+n5PyDyVozZTsfOmDT8fZXgIeN0+scQE7eihlt0JQ1s bLJeglHNVmv6iNvCvSwvfhdPk4czKWXZHmkMgtnpkxJ9ywQB7zQI8MwmZb2S/ihvNEv/ AWnK7v6Z5CVNG2c3n6sx6+GolVw3k2JZm73VJX3tLHTgxJu+pNVMGJzwc/4Enu5h/Dbg BMYPm32wCK0OuC8sdcwZKh++2jy54zTXibaMvI+1NHjkffyIT2KxCIUfRa17z4g8LIpE gKUH6IMT+t6gQH+ZQJea48VKNZWicKcsRLVOzEJoRmFfv0uRc06UCWZzRNoZP4+sH0D5 e7dw== X-Gm-Message-State: ALoCoQnh/PNYu1/oHyvekUI4aeuRCAFzW5B0lt3KShHsCR26vcXYTVSV6zVnBVP0FUMA5xZD0zR2 MIME-Version: 1.0 X-Received: by 10.112.169.68 with SMTP id ac4mr3357633lbc.43.1403665929867; Tue, 24 Jun 2014 20:12:09 -0700 (PDT) Received: by 10.114.176.162 with HTTP; Tue, 24 Jun 2014 20:12:09 -0700 (PDT) In-Reply-To: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> Date: Wed, 25 Jun 2014 08:42:09 +0530 Message-ID: Subject: Questions about libssh2_sftp_read() From: Nitin Deokate To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0695757444==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============0695757444== Content-Type: multipart/alternative; boundary=001a11c386f0222e7704fca07163 --001a11c386f0222e7704fca07163 Content-Type: text/plain; charset=UTF-8 I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000 bytes in each case. You could also try, if you are developer for the same feature. Thanks Nitin On Wed, Jun 25, 2014 at 4:43 AM, Daniel Stenberg wrote: > > On Tue, 24 Jun 2014, Nitin Deokate wrote: > >> I have modified the *char* mem[1024]; to *char* mem[2500]; > > But what if you make it 100000 or 500000 bytes big instead? > >> But last response from Daniel says that I should get bigger chunks in subsequent call, which is not happening. Do I need to change something to >> get me bigger chunks. > > I think 2500 is just too small buffer to get much data with. > -- > > / daniel.haxx.se > _______________________________________________ > libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --001a11c386f0222e7704fca07163 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000= bytes in each case.
You could also try, if you are developer for the sa= me feature.

Thanks
Nitin

On Wed, Jun 25, 2014 at 4:43 AM, = Daniel Stenberg <daniel@haxx.se>= ; wrote:
>
> On Tue, 24 Jun 2014, Nitin Deokate wrote:
>
>> = I have modified the *char* mem[1024]; to *char* mem[2500];
>
> = But what if you make it 100000 or 500000 bytes big instead?
>
>> But last response from Daniel says that I should get bigger chunks= in subsequent call, which is not happening. Do I need to change something = to
>> get me bigger chunks.
>
> I think 2500 is just t= oo small buffer to get much data with.
> --
>
> =C2=A0/ daniel.ha= xx.se
> _______________________________________________
> l= ibssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel

--001a11c386f0222e7704fca07163-- --===============0695757444== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============0695757444==-- From libssh2-devel-bounces@cool.haxx.se Wed Jun 25 06:18:57 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P4Ibv4020595; Wed, 25 Jun 2014 06:18:54 +0200 Received: from mail-la0-f51.google.com (mail-la0-f51.google.com [209.85.215.51]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P4IZLc020579 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Wed, 25 Jun 2014 06:18:35 +0200 Received: by mail-la0-f51.google.com with SMTP id mc6so323517lab.38 for ; Tue, 24 Jun 2014 21:18:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=wKexNcanNLpbw10coOz0TDmNUW/hpBBhQq16kq1Dv4k=; b=HNEJ9oKKD2CdZrl0JkHyXmLZ4XPPi/xud88E7eflQabHvtW3f3gPNh4o50gy5QDDPm GjoznIeItBkxQfW7FifXoMw801246Dr+krfiL3yssoX7FD0hFo13X3syh/3b6ba+wtW0 aUgdKH2PJnbGzCztTp/lwrgsdlZkQjr+CqbfCeSt4BaLuoiuUs6GTFGiDQpjCisGvFI2 PTb811rKV78wpKX7DCkxB9rOBwroSkNNSrP9zZZn1eIqBd60KIuVO6mmvtc15SmEb1Yd gktXXLRn5w9Syddp6O1hy/3+qqvKrnECWmHbuDSsWerQ3cYwkLbSv//EBJaur1ezud+/ 5rtA== X-Gm-Message-State: ALoCoQm8G8Iq9SpNaO3nbnhnlrv8PEWZztSqK1LsFGtJM3J04WNi51Tms51SVEqB7zXF6RY0Xuq0 MIME-Version: 1.0 X-Received: by 10.152.6.194 with SMTP id d2mr3903467laa.54.1403669910886; Tue, 24 Jun 2014 21:18:30 -0700 (PDT) Received: by 10.114.176.162 with HTTP; Tue, 24 Jun 2014 21:18:30 -0700 (PDT) In-Reply-To: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> Date: Wed, 25 Jun 2014 09:48:30 +0530 Message-ID: Subject: Questions about libssh2_sftp_read() From: Nitin Deokate To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1191737803==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============1191737803== Content-Type: multipart/alternative; boundary=089e013d186a6bb15e04fca15e55 --089e013d186a6bb15e04fca15e55 Content-Type: text/plain; charset=UTF-8 I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000 bytes in each case. You could also try, if you are developer for the same feature. Thanks Nitin On Wed, Jun 25, 2014 at 4:43 AM, Daniel Stenberg wrote: > > On Tue, 24 Jun 2014, Nitin Deokate wrote: > >> I have modified the *char* mem[1024]; to *char* mem[2500]; > > But what if you make it 100000 or 500000 bytes big instead? > >> But last response from Daniel says that I should get bigger chunks in subsequent call, which is not happening. Do I need to change something to >> get me bigger chunks. > > I think 2500 is just too small buffer to get much data with. > -- > > / daniel.haxx.se > _______________________________________________ > libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --089e013d186a6bb15e04fca15e55 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000= bytes in each case.
You could also try, if you are developer for the sa= me feature.

Thanks
Nitin

On Wed, Jun 25, 2014 at 4:43 AM, = Daniel Stenberg <daniel@haxx.se>= ; wrote:
>
> On Tue, 24 Jun 2014, Nitin Deokate wrote:
>
>> = I have modified the *char* mem[1024]; to *char* mem[2500];
>
> = But what if you make it 100000 or 500000 bytes big instead?
>
>> But last response from Daniel says that I should get bigger chunks= in subsequent call, which is not happening. Do I need to change something = to
>> get me bigger chunks.
>
> I think 2500 is just t= oo small buffer to get much data with.
> --
>
> =C2=A0/ daniel.ha= xx.se
> _______________________________________________
> l= ibssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel

--089e013d186a6bb15e04fca15e55-- --===============1191737803== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============1191737803==-- From libssh2-devel-bounces@cool.haxx.se Wed Jun 25 06:29:14 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P4TAUD031893; Wed, 25 Jun 2014 06:29:13 +0200 Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P4T7Se031732 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Wed, 25 Jun 2014 06:29:07 +0200 Received: by mail-vc0-f173.google.com with SMTP id lf12so1339260vcb.18 for ; Tue, 24 Jun 2014 21:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=+O1eBH7Z8QmonTSPS1UtlTrb1XaN6DJfBrkSnR099To=; b=A98//Cs9hZ352nlF5zNzjA4RyXTb2HYn3kolLy/5QpEp44Ugclte5Jl45j3+2zFuKb H+7knTOjZoMboxu/vHTgtAiVrc97hqjfvP8ZD/Wj8YdUS6/h21GYCjqmm0EryGrQArjV +7MUP02Lz5NB01y2cpDPJMZaqs8WSrWjRdlUi8zWmek0FguVrtKnN5WyuwRIeDd9jByR 4Hp5sOAJCLOmmTOFGOQ/Z51axCEqVXXE1Kbgc+4RlORNjWIT5IlzpCuBqq46kDrd5xB8 1tuiFf97E8RnRtcjYIRrEVLoDFE99wqQloWzYotZ8Th739HFUVulVIneUsl4uD/X5OmG IQvQ== X-Received: by 10.58.56.102 with SMTP id z6mr4451890vep.7.1403670541613; Tue, 24 Jun 2014 21:29:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Tue, 24 Jun 2014 21:28:41 -0700 (PDT) In-Reply-To: <1403618517.19230.7.camel@localhost> References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> From: Eduardo Silva Date: Tue, 24 Jun 2014 22:28:41 -0600 Message-ID: Subject: Re: Multiple channels and epoll(7) To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5P4TAUD031893 of all your channels is not really a problem. > >> just to clarify, whats the real difference between a channel and a >> stream in libssh2 ? > > Each channel can be divided into N streams. Each stream is processed > individually by libssh_channel_read_ex() (without _ex is only a wrapper > to process stream #0), but windowing is performed on the channel as a > whole if I am not mistaken. hmm so if my socket is in an event loop and i get notifications, besides to walk through my list of active channels, for each one should i also walk for each stream_id ?, if so, how can i know the number of streams that a channel is handling ? regards, _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Jun 25 11:16:37 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P9G8Wm023599; Wed, 25 Jun 2014 11:16:32 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5P9G7d3023593 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 25 Jun 2014 11:16:07 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5P9G6fh003802 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 25 Jun 2014 11:16:08 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5P9FrPo005552 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 25 Jun 2014 11:15:57 +0200 Message-ID: <1403687749.2512.0.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Wed, 25 Jun 2014 11:15:49 +0200 In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=0.3 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_12_24 autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5P9G8Wm023599 tis 2014-06-24 klockan 22:28 -0600 skrev Eduardo Silva: > hmm so if my socket is in an event loop and i get notifications, > besides to walk through my list of active channels, for each one > should i also walk for each stream_id ?, if so, how can i know the > number of streams that a channel is handling ? By the application you are running. port forwarding -> only the default stream. shell -> default stream + stream 1 (stderr). Should be safe to assume the server do not unexpectedly start using stream numbers you don't know about. Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Jun 26 05:30:28 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q2eR92020568; Thu, 26 Jun 2014 04:40:52 +0200 Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q2ePEJ020559 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 26 Jun 2014 04:40:25 +0200 Received: from localhost (dast@localhost) by giant.haxx.se (8.14.4/8.14.4/Submit) with ESMTP id s5Q2ePwi020541 for ; Thu, 26 Jun 2014 04:40:25 +0200 X-Authentication-Warning: giant.haxx.se: dast owned process doing -bs Date: Thu, 26 Jun 2014 04:40:25 +0200 (CEST) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Questions about libssh2_sftp_read() In-Reply-To: Message-ID: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5Q2eR92020568 On Wed, 25 Jun 2014, Nitin Deokate wrote: > I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000 > bytes in each case. You could also try, if you are developer for the same > feature. I implemented that feature. I've tried it a lot. I don't know why you don't get more. What's the latency/RTT to the server from your client? -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Jun 26 05:55:50 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q3tMWn017861; Thu, 26 Jun 2014 05:55:38 +0200 Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q3tKPN017793 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Thu, 26 Jun 2014 05:55:20 +0200 Received: by mail-ve0-f180.google.com with SMTP id jw12so3067995veb.11 for ; Wed, 25 Jun 2014 20:55:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=VbXo+d6qdQZp9MsWIgk/6JEruQ4dyQYQBc/cE2NN9PU=; b=PM8WTw/4Qdu6tweeSX70D4w4+ekHHTjq0l4xcGVbhDtV4FTcr+ky9WyRSMWhIGDE+Z ZQOFLgtkIhkJrCxvn5UmfTHq/KUqpJwckK29gM0CH81d1whScoyzNsLFfxZ/xL92FdBp 5MdA3uWRirV86NGCaNinry6g+1qCRWISx+ycfkLnIoA2GhzgyXaTDnK6dnlD5tqqob/0 y3jYPHtMMfOl5Y3YG0Q1HDKvsh0TEZ3Gc8ByqvBMCJpFsyCQ45IZqOPvOne2ZJHJd1Hz Jgp/6LvTDbMDNllRD3b6HnhaDt448KHUq+HASMneN0t/tdma0RUiizV1qVhMFfXj0Ias Yy3w== X-Received: by 10.220.163.3 with SMTP id y3mr10809933vcx.7.1403754915052; Wed, 25 Jun 2014 20:55:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Wed, 25 Jun 2014 20:54:55 -0700 (PDT) In-Reply-To: <1403687749.2512.0.camel@localhost> References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> From: Eduardo Silva Date: Wed, 25 Jun 2014 21:54:55 -0600 Message-ID: Subject: Re: Multiple channels and epoll(7) To: libssh2 development X-MIME-Autoconverted: from quoted-printable to 8bit by giant.haxx.se id s5Q3tKPN017793 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5Q3tMWn017861 On Wed, Jun 25, 2014 at 3:15 AM, Henrik Nordström wrote: > > tis 2014-06-24 klockan 22:28 -0600 skrev Eduardo Silva: >> hmm so if my socket is in an event loop and i get notifications, >> besides to walk through my list of active channels, for each one >> should i also walk for each stream_id ?, if so, how can i know the >> number of streams that a channel is handling ? > > > By the application you are running. > > port forwarding -> only the default stream. > > shell -> default stream + stream 1 (stderr). > > > Should be safe to assume the server do not unexpectedly start using > stream numbers you don't know about. > thanks for the feedback. Just to confirm if my approach is correct: 1. i connect to the SSH server and request a tcp-forwarding 2. then in my program i have an epoll(7) loop looking for EPOLLIN (READ) events: 3. for each incoming event i do: 3.a try to accept a channel, if it works create the channel and add it to a list (and jump to 3.c) 3.b if accept a channel failed, try to read for each channel registered in my list 3.c for those channels who succeeded on reading data, write back a response, then close the channel (channel cleanup) but for some reason i see some missing channels notifications when adding a simple concurrency of 2 clients. Should i use a different mechanism to multiplex events on different channels ?, is there something wrong with the proposed model ? any help is appreciated -- Eduardo Silva http://edsiper.linuxchile.cl http://monkey-project.com _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Jun 26 06:04:25 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q44Lsg028166; Thu, 26 Jun 2014 06:04:24 +0200 Received: from mail-lb0-f169.google.com (mail-lb0-f169.google.com [209.85.217.169]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q44JwL028158 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Thu, 26 Jun 2014 06:04:19 +0200 Received: by mail-lb0-f169.google.com with SMTP id l4so2514302lbv.14 for ; Wed, 25 Jun 2014 21:04:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-type; bh=HzX4cj+omNPUIA9l4pFM0XJbxp8N6fMoxwnkpIoVZlE=; b=KFahg85xBWdHO2hi/xAkrIWxDuEYKIOSFAI/mjOGyqblj7x+HqKA6JrqkVdwUcXiBH xzdV2FchLcf2NhWYEp3nEF7UKxE5I5pAnPRnjMAr+AMxoy4akGDpY7R18jaLqdwIgaTl vEuq+oEZSIYxgSHhFGzcWiiVNb9Sy00SqJaUMKY24jpLZO8ear866QTHUgpKSps4e9fR ICs8r1hnt0ZE4y0oM2SJsQ+6JhyxEpoBJSjCAue1LXaSIV6ZgG75z+x8vo7FukHN5zdu 4EnMpNt9f69ejjEQQr/xY5h6FHAbKjqtKGw1pachWypypTWh6Gjiwvu4LKYgkVxoU9+S tXfg== X-Gm-Message-State: ALoCoQlcWjhbytwLr27MHXLQkGb/jT3NvQdl7xV9Hqx0DOZOy4nIdfZdL77w1OS16jgJaykDtsm4 X-Received: by 10.112.24.167 with SMTP id v7mr1920165lbf.19.1403755454564; Wed, 25 Jun 2014 21:04:14 -0700 (PDT) From: Nitin Deokate References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQCh8ANGY5tkZe+/B6MGsd0mHP7SDwHImRp1AwIAF0EBMZ0mAALHHfQ1AMJ6Lsydkdid8A== Date: Thu, 26 Jun 2014 09:34:13 +0530 Message-ID: <9c6913fa9603d4fc9c2c4e64b17703b0@mail.gmail.com> Subject: RE: Questions about libssh2_sftp_read() To: libssh2-devel@cool.haxx.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0239547671==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============0239547671== Content-Type: multipart/alternative; boundary=001a1134997a38a76e04fcb54978 --001a1134997a38a76e04fcb54978 Content-Type: text/plain; charset=UTF-8 Any idea how to get those numbers? -Nitin *> I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000 * *> bytes in each case. You could also try, if you are developer for the same * *> feature. * I implemented that feature. I've tried it a lot. I don't know why you don't get more. What's the latency/RTT to the server from your client? -- / daniel.haxx.se *From:* Nitin Deokate [mailto:ndeokate@qualys.com] *Sent:* Wednesday, June 25, 2014 8:42 AM *To:* libssh2 development *Subject:* Questions about libssh2_sftp_read() I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000 bytes in each case. You could also try, if you are developer for the same feature. Thanks Nitin On Wed, Jun 25, 2014 at 4:43 AM, Daniel Stenberg wrote: > > On Tue, 24 Jun 2014, Nitin Deokate wrote: > >> I have modified the *char* mem[1024]; to *char* mem[2500]; > > But what if you make it 100000 or 500000 bytes big instead? > >> But last response from Daniel says that I should get bigger chunks in subsequent call, which is not happening. Do I need to change something to >> get me bigger chunks. > > I think 2500 is just too small buffer to get much data with. > -- > > / daniel.haxx.se > _______________________________________________ > libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --001a1134997a38a76e04fcb54978 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Any idea how to get those numbers?

= -Nitin

=C2=A0

> I tried with 500000 bytes to almost 10MB as buffer size, what I get= is 2000=C2=A0
> bytes in each case. You could also try, if you are developer for th= e same=C2=A0
> feature.=C2=A0

I implemented that= feature. I've tried it a lot. I don't know why you don't=C2=A0=
get more. What's the latency/RTT to the server from your client?=C2=A0<= /span>

--

= =C2=A0 / daniel.haxx.se

=C2=A0

=C2=A0

From: Nitin De= okate [mailto:ndeokate@qualys.com]
Sent: Wednesday, June 25, 2014 8:42 AM
To: libssh2 develop= ment
Subject: Questions about libssh2_sftp_read()

=C2=A0

I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000= bytes in each case.
You could also try, if you are developer for the sa= me feature.

Thanks
Nitin

On Wed, Jun 25, 2014 at 4:43 AM, = Daniel Stenberg <daniel@haxx.se>= ; wrote:
>
> On Tue, 24 Jun 2014, Nitin Deokate wrote:
>
>> = I have modified the *char* mem[1024]; to *char* mem[2500];
>
> = But what if you make it 100000 or 500000 bytes big instead?
>
>> But last response from Daniel says that I should get bigger chunks= in subsequent call, which is not happening. Do I need to change something = to
>> get me bigger chunks.
>
> I think 2500 is just t= oo small buffer to get much data with.
> --
>
> =C2=A0/ daniel.ha= xx.se
> _______________________________________________
> l= ibssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel

--001a1134997a38a76e04fcb54978-- --===============0239547671== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============0239547671==-- From libssh2-devel-bounces@cool.haxx.se Thu Jun 26 08:53:36 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q6rCGM029671; Thu, 26 Jun 2014 08:53:31 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5Q6rBub029667 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 26 Jun 2014 08:53:11 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5Q6r9Cd005584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 26 Jun 2014 08:53:11 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5Q6r8CR032678 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 26 Jun 2014 08:53:08 +0200 Message-ID: <1403765583.6206.8.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Thu, 26 Jun 2014 08:53:03 +0200 In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_24_48 autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5Q6rCGM029671 ons 2014-06-25 klockan 21:54 -0600 skrev Eduardo Silva: > Just to confirm if my approach is correct: > > 1. i connect to the SSH server and request a tcp-forwarding Which direction? Using libssh2_forward_listen()? > 2. then in my program i have an epoll(7) loop looking for EPOLLIN (READ) events: Ok. > 3. for each incoming event i do: > 3.a try to accept a channel, if it works create the channel and add > it to a list (and jump to 3.c) What exactly do you mean? > 3.b if accept a channel failed, try to read for each channel > registered in my list Ok. > 3.c for those channels who succeeded on reading data, write back a > response, then close the channel (channel cleanup) Ok. > but for some reason i see some missing channels notifications when > adding a simple concurrency of 2 clients. Should i use a different > mechanism to multiplex events on different channels ?, is there > something wrong with the proposed model ? I don't see anything obviously wrong with what you described. The problem is likely in the details somewhere.. How do you know that you loose channel notifications? Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Jun 27 04:28:13 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R2Rp5R031082; Fri, 27 Jun 2014 04:28:09 +0200 Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R2Rntc030936 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Fri, 27 Jun 2014 04:27:49 +0200 Received: by mail-vc0-f169.google.com with SMTP id la4so4510893vcb.28 for ; Thu, 26 Jun 2014 19:27:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=oC9GL8XJtFLAPhUIz+ZgInxfu5IrwKR4y+Qs7a3BMlA=; b=sjwd44D2kZgruHzrjTnzdMBDEE1aj8vhoXgYLt6pVX93aDH9RVBkGfITnlWW8m3VWw Kla/xr6USxsS7ReHOq1rurE1Y3eFsBW1LDvjdBYkrYJqLI1ZysXmr0Z6kXPf/a7VckHn sDqUoy/YkNhkV/JBDbh5JxTJFhs4gB/qyDeNQ98Pv8ZmfOL4h55+GTL1vVRHJf1BhPO2 ZfD8yFqxq5dGLgTabKHqK5CUc0YxbuY9Bs1VR95qqAXfTVS6chjenWee4u8ae0cjao0n Dgm9tslc3xOkm02OxI0UomaN6uniGTfPbs2RJRNOhFxbNkCYOw8eIywCkP8OJwS95I6C MEmA== X-Received: by 10.58.30.15 with SMTP id o15mr17325683veh.34.1403836065230; Thu, 26 Jun 2014 19:27:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Thu, 26 Jun 2014 19:27:25 -0700 (PDT) In-Reply-To: <1403765583.6206.8.camel@localhost> References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> <1403765583.6206.8.camel@localhost> From: Eduardo Silva Date: Thu, 26 Jun 2014 20:27:25 -0600 Message-ID: Subject: Re: Multiple channels and epoll(7) To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5R2Rp5R031082 Hi, > > I don't see anything obviously wrong with what you described. The > problem is likely in the details somewhere.. > > How do you know that you loose channel notifications? In order to simplify the problem explanation, i wrote a test case, any input is welcome: https://github.com/edsiper/libssh2_channels more details on the README file. thanks for your help, -- Eduardo Silva http://edsiper.linuxchile.cl http://monkey-project.com _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Jun 27 07:25:35 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R5PEBr022446; Fri, 27 Jun 2014 07:25:30 +0200 Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R5PCH7022295 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Fri, 27 Jun 2014 07:25:13 +0200 Received: by mail-ve0-f179.google.com with SMTP id sa20so4812591veb.10 for ; Thu, 26 Jun 2014 22:25:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=Jd19KyaandLirWv9is6rfGAEGjvmIXynpESOzvtUKtg=; b=uScEJ3ZgjBB95NEtJdEwlsm/W/RhCI2fngX7GqzZZQdf0PpWspXeGNFJqOhORW8ccS mOdkHTQwf1qQcs3VwZYH/c90+fOx7OYR65nJJAtJbRxcArHZj1cmXlGKwJxFqBM+EHtT Subk5JS7e9uFwg/7hbYVUxIdHH1gNcd3QujsAsdquYILjIHVIgnI9ICSPhSfsybC6f+P 0T9E3QLA0ITsGO0lTN9l3WUll9s2108d4xmfymvDvKMEQk9B75gLPJHEhnYwczSrFq/f 6fIqMVk5A9fIr2OyEYxYhjVfcdUnYfE1vhvgTA+yzFZIJRQnXuKzPhiaf/n6Pzmo4/b7 RgOg== X-Received: by 10.58.228.74 with SMTP id sg10mr17431050vec.6.1403846708437; Thu, 26 Jun 2014 22:25:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.45.196 with HTTP; Thu, 26 Jun 2014 22:24:48 -0700 (PDT) In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> <1403765583.6206.8.camel@localhost> From: Eduardo Silva Date: Thu, 26 Jun 2014 23:24:48 -0600 Message-ID: Subject: Re: Multiple channels and epoll(7) To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5R5PEBr022446 >> I don't see anything obviously wrong with what you described. The >> problem is likely in the details somewhere.. >> >> How do you know that you loose channel notifications? > > In order to simplify the problem explanation, i wrote a test case, any > input is welcome: > > https://github.com/edsiper/libssh2_channels > > more details on the README file. == UPDATE == I fixed the channels problem, i was setting the Listener queue to 1, increased to 64 and fixed the concurrency problem: listener = libssh2_channel_forward_listen_ex(session, "localhost", remote_wantport, &remote_listenport, 64); but there is a remaining problem, when i get the notification that some data is available, after try to accept a channel and then read all channels, i need a timeout in the event loop to re-check if something was pending. Still wondering how to avoid that re-check. -- Eduardo Silva http://edsiper.linuxchile.cl http://monkey-project.com _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Jun 27 07:32:24 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R5WGfh032572; Fri, 27 Jun 2014 07:32:22 +0200 Received: from mail-la0-f51.google.com (mail-la0-f51.google.com [209.85.215.51]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R5WDqt032514 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128 verify=NOT) for ; Fri, 27 Jun 2014 07:32:14 +0200 Received: by mail-la0-f51.google.com with SMTP id mc6so2564305lab.38 for ; Thu, 26 Jun 2014 22:32:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=OgO9vKJ067qvTlSoCxV0E4Xq509O3gY2uWGey5XnAV4=; b=mcrZfmGbFeQpSbygQKJZOdR4xhRKDoCH/SmmCCXocaE8uoNN+PGCFv2M6eh8d5spjJ 3HlAi141eCTFKpA1nsUvXrAGAMWKpl0p1vLjFy1BGPa6HgcQA8G0P/Y4U/eCMTmS+J3N 7P2QrAKCAc2ecihqXdN9fudAvh06yAlq6aXJr6CTaU/PtDYZjvWARFIfW7C19LqOW/7I j3JWP+Ad1Y5i6V5uN64nR2AszE5lnixR4kx2AWj9828ZHy+8qNHB3gAnSH0JoH9WMeAO NN+zcDaCR69rFLl/CMRq8vvzOoZypUdI1Df0swQ5tSjzkNaffd0Z5kbKn7mrPkPRUAF0 AMwg== X-Gm-Message-State: ALoCoQn//Xmy3X470EnPtCHwT0V16HhdsmrDFQp2x4E9dy3D88uAwAMGjbWSuCf09yZGiZkrclKB MIME-Version: 1.0 X-Received: by 10.112.219.65 with SMTP id pm1mr65187lbc.79.1403847130330; Thu, 26 Jun 2014 22:32:10 -0700 (PDT) Received: by 10.114.176.162 with HTTP; Thu, 26 Jun 2014 22:32:10 -0700 (PDT) In-Reply-To: <9c6913fa9603d4fc9c2c4e64b17703b0@mail.gmail.com> References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> <9c6913fa9603d4fc9c2c4e64b17703b0@mail.gmail.com> Date: Fri, 27 Jun 2014 11:02:10 +0530 Message-ID: Subject: Re: Questions about libssh2_sftp_read() From: Nitin Deokate To: libssh2 development X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1273681128==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============1273681128== Content-Type: multipart/alternative; boundary=001a11c31eae85cf7604fccaa1a6 --001a11c31eae85cf7604fccaa1a6 Content-Type: text/plain; charset=UTF-8 Nevertheless, I tried with localhost as well as another machine from same network(no heavy IO,memory operations on both machine. so guessing, it would be low latency) . still I get 2000 bytes Thanks Nitin On Thu, Jun 26, 2014 at 9:34 AM, Nitin Deokate wrote: > Any idea how to get those numbers? > > -Nitin > > > > *> I tried with 500000 bytes to almost 10MB as buffer size, what I get is > 2000 * > *> bytes in each case. You could also try, if you are developer for the > same * > *> feature. * > > I implemented that feature. I've tried it a lot. I don't know why you > don't > get more. What's the latency/RTT to the server from your client? > > -- > > / daniel.haxx.se > > > > > > *From:* Nitin Deokate [mailto:ndeokate@qualys.com] > *Sent:* Wednesday, June 25, 2014 8:42 AM > *To:* libssh2 development > *Subject:* Questions about libssh2_sftp_read() > > > > I tried with 500000 bytes to almost 10MB as buffer size, what I get is > 2000 bytes in each case. > You could also try, if you are developer for the same feature. > > Thanks > Nitin > > On Wed, Jun 25, 2014 at 4:43 AM, Daniel Stenberg wrote: > > > > On Tue, 24 Jun 2014, Nitin Deokate wrote: > > > >> I have modified the *char* mem[1024]; to *char* mem[2500]; > > > > But what if you make it 100000 or 500000 bytes big instead? > > > >> But last response from Daniel says that I should get bigger chunks in > subsequent call, which is not happening. Do I need to change something to > >> get me bigger chunks. > > > > I think 2500 is just too small buffer to get much data with. > > -- > > > > / daniel.haxx.se > > _______________________________________________ > > libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel > --001a11c31eae85cf7604fccaa1a6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Nevertheless, I tried with localhost as well as another ma= chine from same network(no heavy IO,memory operations on both machine. so g= uessing, it would be low latency) . still I get 2000 bytes

Thanks
Nitin


<= div class=3D"gmail_quote">On Thu, Jun 26, 2014 at 9:34 AM, Nitin Deokate <ndeokate@qualys.com> wrote:

Any idea how to get those numbers?

= -Nitin

=C2=A0

> I tried with 500000 bytes to almost 10MB as buffer size, what I get= is 2000=C2=A0
> bytes in each case. You could also try, if you are developer for th= e same=C2=A0
> feature.=C2=A0

I implemente= d that feature. I've tried it a lot. I don't know why you don't= =C2=A0
get more. What's the latency/RTT to the server from your client?=C2=A0<= /span>

--

= =C2=A0 / daniel.haxx.se=

=C2=A0

=C2=A0

From: Nitin De= okate [mailto:ndeo= kate@qualys.com]
Sent: Wednesday, June 25, 2014 8:42 AM
To: libssh2 develop= ment
Subject: Questions about libssh2_sftp_read()

=

=C2=A0

I tried with 500000 bytes to almost 10MB as buffer size, what I get is 2000= bytes in each case.
You could also try, if you are developer for the sa= me feature.

Thanks
Nitin

On Wed, Jun 25, 2014 at 4:43 AM, = Daniel Stenberg <dan= iel@haxx.se> wrote:
>
> On Tue, 24 Jun 2014, Nitin Deokate wrote:
>
>> = I have modified the *char* mem[1024]; to *char* mem[2500];
>
> = But what if you make it 100000 or 500000 bytes big instead?
>
>> But last response from Daniel says that I should get bigger chunks= in subsequent call, which is not happening. Do I need to change something = to
>> get me bigger chunks.
>
> I think 2500 is just t= oo small buffer to get much data with.
> --
>
> =C2=A0/ daniel.haxx.se
> _______________________________________= ________
> libssh2-devel http://cool.haxx.se/cgi-bin/ma= ilman/listinfo/libssh2-devel


--001a11c31eae85cf7604fccaa1a6-- --===============1273681128== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwOi8vY29vbC5oYXh4LnNlL2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9saWJzc2gy LWRldmVsCg== --===============1273681128==-- From libssh2-devel-bounces@cool.haxx.se Fri Jun 27 10:53:48 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R8rMh1004398; Fri, 27 Jun 2014 10:53:42 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R8rKk9004393 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 27 Jun 2014 10:53:20 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5R8rLgl010324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 27 Jun 2014 10:53:22 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5R8rJiu013765 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 27 Jun 2014 10:53:20 +0200 Message-ID: <1403859199.2448.7.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Fri, 27 Jun 2014 10:53:19 +0200 In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> <1403765583.6206.8.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_06_12 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5R8rMh1004398 tor 2014-06-26 klockan 23:24 -0600 skrev Eduardo Silva: > but there is a remaining problem, when i get the notification that > some data is available, after try to accept a channel and then read > all channels, i need a timeout in the event loop to re-check if > something was pending. Still wondering how to avoid that re-check. When do you register the read event interest? You need to register for the read event before looping over all channels. Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Jun 27 11:19:21 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R9JE7N022867; Fri, 27 Jun 2014 11:19:19 +0200 Received: from vps1.hno.se (vps1.hno.se [IPv6:2a02:750:5::f0]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5R9JDnC022827 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 27 Jun 2014 11:19:13 +0200 Received: from home.hno.se (c-fa92e155.1160-1-64736c10.cust.bredbandsbolaget.se [85.225.146.250]) (authenticated bits=128) by vps1.hno.se (8.14.4/8.14.4) with ESMTP id s5R9JDSf010352 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 27 Jun 2014 11:19:15 +0200 Received: from henrik ([127.0.0.1]) (authenticated bits=0) by home.hno.se (8.14.5/8.14.5) with ESMTP id s5R9JBiP015358 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 27 Jun 2014 11:19:11 +0200 Message-ID: <1403860751.2448.24.camel@localhost> Subject: Re: Multiple channels and epoll(7) From: Henrik =?ISO-8859-1?Q?Nordstr=F6m?= To: libssh2 development Date: Fri, 27 Jun 2014 11:19:11 +0200 In-Reply-To: References: <1403569267.14025.4.camel@localhost> <1403570385.14025.12.camel@localhost> <1403618517.19230.7.camel@localhost> <1403687749.2512.0.camel@localhost> <1403765583.6206.8.camel@localhost> X-Mailer: Evolution 3.10.4 (3.10.4-2.fc20) Mime-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_06_12 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on home.hno.se X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5R9JE7N022867 tor 2014-06-26 klockan 23:24 -0600 skrev Eduardo Silva: > but there is a remaining problem, when i get the notification that > some data is available, after try to accept a channel and then read > all channels, i need a timeout in the event loop to re-check if > something was pending. Still wondering how to avoid that re-check. Looking at your code, and thinking and ugh... the libssh2 API is borked when it comes to handling of multiple concurrent channels. What I said earlier might work if you set epoll into edge triggered mode unless kernel cancels events if they get cleared before processed (I do not remember what happens, maybe kernel release dependent even). But current libssh2 API makes any level triggered non-blocking I/O on multiple channels unreliable (including poll/select) and requires a short timeout to avoid blocking due to incoming packets being held in the libssh2 packets queue without the application being aware. Looping over all channels until all report EAGAIN shortens the window where this may happen, but do not eleminate it as packets may still arrive into the packet queue while you process later channels. The earlier discussed API addition of giving access to inspect the packet queue would help solving this. You then loop processing until internal packet queue is drained. When internal packet queue is drained you know that libssh is actually waiting for the socket. Separating I/O from processing would also solve this problem and likely scale better with number of channels, removing the implicit socket read from libssh2_channel_read() and friends, requiring a explicit separate call to read the socket. Another interesting thought would be to aim to rework the libssh2 API to the level that it can support libevent buffered I/O events and similar I/O loops by delegating all actual I/O to the application. Above is likely a first step towards that. Regards Henrik _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Jun 28 05:09:25 2014 Return-Path: Received: from www.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5S390Mj004401; Sat, 28 Jun 2014 05:09:20 +0200 Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.4/8.14.4/Debian-7) with ESMTP id s5S38wMe004393 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 28 Jun 2014 05:08:58 +0200 Received: from localhost (dast@localhost) by giant.haxx.se (8.14.4/8.14.4/Submit) with ESMTP id s5S38wc4004389 for ; Sat, 28 Jun 2014 05:08:58 +0200 X-Authentication-Warning: giant.haxx.se: dast owned process doing -bs Date: Sat, 28 Jun 2014 05:08:58 +0200 (CEST) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Questions about libssh2_sftp_read() In-Reply-To: Message-ID: References: <299fdb986750cc9f580c4db8781e8090@mail.gmail.com> <9c6913fa9603d4fc9c2c4e64b17703b0@mail.gmail.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.16 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id s5S390Mj004401 On Fri, 27 Jun 2014, Nitin Deokate wrote: > Nevertheless, I tried with localhost as well as another machine from same > network(no heavy IO,memory operations on both machine. so guessing, it would > be low latency) . still I get 2000 bytes Then so be it. The point is how fast you transfer data, not how much data you get returned in a single call to libssh2_sftp_read()... -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel