From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 00:57:06 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id o9VNuLdK006835; Mon, 1 Nov 2010 00:57:03 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id o9VNuJGw006770 for ; Mon, 1 Nov 2010 00:56:20 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=kIJvhBlJozokeAIMIhMrNFifej8huCSgXOwAytf9ph29MGHSOZ85VwikPqzLoAP8Wy 7r0WV+aueCrMFKAyJ+qjkpHkM03n0icQaf5HIPBhw8oeYx0hVZHC55GCzFTjR2i1dcKJ njAvhtcTU2oxcSlreQPe+ojPF4MiHEcZix3nM=; bh=84D99hWmkY5t013OFBtFk0jWpVU=; t=1288569358; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 7d8c0006320a349e; Sun, 31 Oct 2010 19:55:57 -0400 From: "Mark Riordan" To: References: In-Reply-To: Subject: Re: libssh2_sftp_write fails with return code -1 Date: Sun, 31 Oct 2010 18:56:08 -0500 Message-ID: <001501cb7957$306ff3b0$914fdb10$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act4GXTXTJeWNHWARFGd86zkOf+9jABO6Ycg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 00:57:06 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 00:56:20 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se >> Yesterday (2010-10-28) I did a "git" from the repository and rebuilt my test >> program with the newest libssh2. Unfortunately, uploads now fail on the very >> first call to libssh2_sftp_write, whereas with 1.2.7, the test program >> generally worked. > Great, we need testing of the current git version to make it run properly! > > Can you please build a debug-enabled libssh2 and call > libssh2_trace(session, ~0); after you've called > session_init() and provide the log output? After checking with git to make sure my libssh2 source was up-to-date (it still was), I recompiled libssh2 with LIBSSH2DEBUG defined. I then enhanced my program to take a debug parameter, which causes it to call libssh2_trace(session, ~0). Alas, when debug is on, the uploads succeed. I tried several times. I have a 649 MB trace log, but I won't upload it, since it doesn't show any failures. With debug off, transfers of my large test file fail, more or less as before. Though with the version of libssh2 compiled with LIBSSH2DEBUG defined, sometimes 100 or more libssh2_sftp_write calls will succeed before one returns -1. The failures do not always occur at the same point, even though I'm using the same test file each time. I've noticed that the average number of bytes sent per call of libssh2_sftp_write is smaller with the LIBSSH2DEBUG version, even if debug is turned off. I put the source to my program, plus Windows executable, at: http://60bits.net/sni/sftpmrr.zip Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 08:40:06 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA17dgAK001151; Mon, 1 Nov 2010 08:40:02 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA17deBp001142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 08:39:40 +0100 Received: (qmail 14582 invoked from network); 1 Nov 2010 07:39:39 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 07:39:39 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 07:39:39 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:2 Message-ID: <059.472817488f5779a1ada51aa77d083846@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 08:40:06 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 08:39:40 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE5NDogU2NwIEJ1ZwotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIFJlcG9ydGVyOiAgcm9ib2NpZGUg IHwgICAgICAgIE93bmVyOiAgICAgICAgDQogICAgICBUeXBlOiAgZGVmZWN0ICAgIHwgICAgICAg U3RhdHVzOiAgY2xvc2VkDQogIFByaW9yaXR5OiAgaGlnaCAgICAgIHwgICAgTWlsZXN0b25lOiAg MS4yLjcgDQogQ29tcG9uZW50OiAgU0NQICAgICAgIHwgICAgICBWZXJzaW9uOiAgMS4yLjcgDQpS ZXNvbHV0aW9uOiAgaW52YWxpZCAgIHwgICAgIEtleXdvcmRzOiAgICAgICAgDQogICAgQmxvY2tz OiAgICAgICAgICAgIHwgICBCbG9ja2VkIEJ5OiAgICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CkNoYW5nZXMgKGJ5IHJvYm9jaWRlKToKCiAqIGNjOiBhdmlzaGF5aGlyc2hA4oCmIChhZGRlZCkN CgoKQ29tbWVudDoKCiBIaSBhbmQgdGhhbmtzLA0KIDFdIGRvZXMgdGhpcyByZXN0cmljdGlvbiBh cHBseXMgdG8gYWNjdWlyaW5nIG1vcmUgdGhhbiAxMCBzZnRwX2hhbmRsZXMgYXMNCiB3ZWxsID8N CiAyXSB3aHkgdGhpcyByZXN0cmljdGlvbiBpcyBub3QgZG9jdW1lbnRlZCBpbiB0aGUgQVBJIHJl ZmVyZW5jZSA/IGkgd291bGQNCiBiZSBoYXBweSB0byBkbyBpdCBhcyAgICB3ZWxsLg0KDQogQXZp c2hheS4KCi0tIApUaWNrZXQgVVJMOiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcvdGlja2V0LzE5 NCNjb21tZW50OjI+CmxpYnNzaDIgPGh0dHA6Ly90cmFjLmxpYnNzaDIub3JnLz4KQyBsaWJyYXJ5 IGZvciB3cml0aW5nIHBvcnRhYmxlIFNTSDIgY2xpZW50cwpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXwpsaWJzc2gyLWRldmVsIGh0dHA6Ly9jb29sLmhheHgu c2UvY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL2xpYnNzaDItZGV2ZWwK From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 09:07:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA187bLU023080; Mon, 1 Nov 2010 09:07:44 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA187ZgQ023017 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 09:07:35 +0100 Received: (qmail 16367 invoked from network); 1 Nov 2010 08:07:34 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 08:07:34 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 08:07:34 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:3 Message-ID: <059.88af16671e12f05a1e2e20dc47f84ba8@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 09:07:44 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 09:07:36 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by bagder): 1) no, SFTP is a subsystem that doesn't use new channels for each new transfer. 2) that restriction is not client-side, it is SERVER-SIDE so what we can document is only that there MIGHT be a restriction in number of concurrent channels. It might of course still be a good idea! libssh2 itself has no limit in number of channels it can handle. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 09:35:51 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18Zhrb008269; Mon, 1 Nov 2010 09:35:50 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18ZfDv008252 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 09:35:41 +0100 Received: (qmail 18728 invoked from network); 1 Nov 2010 08:35:40 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 08:35:40 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 08:35:40 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:4 Message-ID: <059.d506987846d6320353809185b937adde@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 09:35:51 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 09:35:41 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE5NDogU2NwIEJ1ZwotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIFJlcG9ydGVyOiAgcm9ib2NpZGUg IHwgICAgICAgIE93bmVyOiAgICAgICAgDQogICAgICBUeXBlOiAgZGVmZWN0ICAgIHwgICAgICAg U3RhdHVzOiAgY2xvc2VkDQogIFByaW9yaXR5OiAgaGlnaCAgICAgIHwgICAgTWlsZXN0b25lOiAg MS4yLjcgDQogQ29tcG9uZW50OiAgU0NQICAgICAgIHwgICAgICBWZXJzaW9uOiAgMS4yLjcgDQpS ZXNvbHV0aW9uOiAgaW52YWxpZCAgIHwgICAgIEtleXdvcmRzOiAgICAgICAgDQogICAgQmxvY2tz OiAgICAgICAgICAgIHwgICBCbG9ja2VkIEJ5OiAgICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CkNoYW5nZXMgKGJ5IHJvYm9jaWRlKToKCiAqIGNjOiBhdmlzaGF5aGlyc2hA4oCmIChyZW1vdmVk KQ0KCgpDb21tZW50OgoKIEhpICwNCg0KIHJlZ2FyZGluZyBbMV0gLCBpIGFza2VkIGJlY2F1c2Ug aSBlbmNvdW50ZXJkIHRoZSBzYW1lIHByb2JsZW0gd2hpbGUgdHJ5aW5nDQogdG8gYWNxdWlyZSBt b3JlIHRoYW4gMTAgc2Z0cCBoYW5kbGVzICwgaSBnZXQgRVJST1IgY29kZSAtMSB3aGlsZSBhcXVp cmluZw0KIHNmdHAgdGhlIDExdGggaGFuZGxlICwNCiBhbGx0aG9naCBpIGNhbiBzZWUgVENQIGNv bm5lY3Rpb24gaGFzIGJlZW4gZXN0YWJsaXNoZWQuLi4KCi0tIApUaWNrZXQgVVJMOiA8aHR0cDov L3RyYWMubGlic3NoMi5vcmcvdGlja2V0LzE5NCNjb21tZW50OjQ+CmxpYnNzaDIgPGh0dHA6Ly90 cmFjLmxpYnNzaDIub3JnLz4KQyBsaWJyYXJ5IGZvciB3cml0aW5nIHBvcnRhYmxlIFNTSDIgY2xp ZW50cwpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaWJz c2gyLWRldmVsIGh0dHA6Ly9jb29sLmhheHguc2UvY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL2xp YnNzaDItZGV2ZWwK From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 09:41:01 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18eoup011430; Mon, 1 Nov 2010 09:41:00 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18enXS011380 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 09:40:49 +0100 Received: (qmail 19097 invoked from network); 1 Nov 2010 08:40:48 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 08:40:48 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 08:40:48 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:5 Message-ID: <059.e8642597ea1d46f4ba141bf984d2a2d4@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 09:41:01 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 09:40:49 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE5NDogU2NwIEJ1ZwotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIFJlcG9ydGVyOiAgcm9ib2NpZGUg IHwgICAgICAgIE93bmVyOiAgICAgICAgDQogICAgICBUeXBlOiAgZGVmZWN0ICAgIHwgICAgICAg U3RhdHVzOiAgY2xvc2VkDQogIFByaW9yaXR5OiAgaGlnaCAgICAgIHwgICAgTWlsZXN0b25lOiAg MS4yLjcgDQogQ29tcG9uZW50OiAgU0NQICAgICAgIHwgICAgICBWZXJzaW9uOiAgMS4yLjcgDQpS ZXNvbHV0aW9uOiAgaW52YWxpZCAgIHwgICAgIEtleXdvcmRzOiAgICAgICAgDQogICAgQmxvY2tz OiAgICAgICAgICAgIHwgICBCbG9ja2VkIEJ5OiAgICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CkNoYW5nZXMgKGJ5IHJvYm9jaWRlKToKCiAqIGNjOiBhdmlzaGF5aGlyc2hA4oCmIChhZGRlZCkN CgoKLS0gClRpY2tldCBVUkw6IDxodHRwOi8vdHJhYy5saWJzc2gyLm9yZy90aWNrZXQvMTk0I2Nv bW1lbnQ6NT4KbGlic3NoMiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcvPgpDIGxpYnJhcnkgZm9y IHdyaXRpbmcgcG9ydGFibGUgU1NIMiBjbGllbnRzCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCmxpYnNzaDItZGV2ZWwgaHR0cDovL2Nvb2wuaGF4eC5zZS9j Z2ktYmluL21haWxtYW4vbGlzdGluZm8vbGlic3NoMi1kZXZlbAo= From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 09:59:09 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18x0W1021445; Mon, 1 Nov 2010 09:59:07 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18wx1O021438 for ; Mon, 1 Nov 2010 09:58:59 +0100 Date: Mon, 1 Nov 2010 09:58:59 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write fails with return code -1 In-Reply-To: <001501cb7957$306ff3b0$914fdb10$@com> Message-ID: References: <001501cb7957$306ff3b0$914fdb10$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 09:59:09 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 09:58:59 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, 31 Oct 2010, Mark Riordan wrote: > Alas, when debug is on, the uploads succeed. I tried several times. I have > a 649 MB trace log, but I won't upload it, since it doesn't show any > failures. Argh, so it's one of THOSE. Yes, the debug output changes a lot of timing and characteristics so it's not that unusual. > With debug off, transfers of my large test file fail, more or less as > before. Though with the version of libssh2 compiled with LIBSSH2DEBUG > defined, sometimes 100 or more libssh2_sftp_write calls will succeed before > one returns -1. I've worked on getting rid of all the -1 return codes since it is too generic error (that previously was used pretty much all over and didn't really help much). I found some new cases that would cause -1 to get returned just now though, and I've pushed a change that hopefully will make you get a different error code (and error message) that can be useful to better track this down. Please get these changes and re-run. We need to figure out exactly which code that (thinks it) detects a problem and returns error. > The failures do not always occur at the same point, even though I'm > using the same test file each time. Which makes it sound even more like timing and network traffic related. How big file do you send? How's the round-trip delay between your client and the server (roughly)? What server software are you using? > I've noticed that the average number of bytes sent per call of > libssh2_sftp_write is smaller with the LIBSSH2DEBUG version, > even if debug is turned off. Hm, I can only assume that is because a few things more are done so it runs a bit slower. > I put the source to my program, plus Windows executable, at: > http://60bits.net/sni/sftpmrr.zip BTW, for some further testing you could try to extend your upload buffer size from 32500 to something like 100K or so. Current git doesn't use 32500 internally at all, and should even run better with larger buffers. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 10:00:02 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1900Ub021696; Mon, 1 Nov 2010 10:00:02 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA18xwT9021678 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 09:59:58 +0100 Received: (qmail 20813 invoked from network); 1 Nov 2010 08:59:57 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 08:59:57 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 08:59:56 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:6 Message-ID: <059.6e8af1fd72cabf8c25bc3fd18d43a25f@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 10:00:02 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 09:59:58 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by bagder): Well, I assume the server can just as well and likely limit the number of concurrent SFTP transfers that it allows. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 10:03:33 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA193U5m028837; Mon, 1 Nov 2010 10:03:33 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA193TB5028779 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 10:03:29 +0100 Received: (qmail 21178 invoked from network); 1 Nov 2010 09:03:28 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 09:03:28 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 09:03:28 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:7 Message-ID: <059.d163fcf858872e64a00a8936bb4cc837@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 10:03:33 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 10:03:29 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by robocide): Oh Ok i wanted to eliminate the possibility this restriction coming from libssh2.... thanks for your kind help, Have a nice day. Avishay. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 14:47:54 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1DlUKY001526; Mon, 1 Nov 2010 14:47:50 +0100 Received: from nm5-vm0.bullet.mail.sp2.yahoo.com (nm5-vm0.bullet.mail.sp2.yahoo.com [98.139.91.204]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oA1DlRwo001518 for ; Mon, 1 Nov 2010 14:47:28 +0100 Received: from [98.139.91.68] by nm5.bullet.mail.sp2.yahoo.com with NNFMP; 01 Nov 2010 13:47:20 -0000 Received: from [98.139.91.1] by tm8.bullet.mail.sp2.yahoo.com with NNFMP; 01 Nov 2010 13:47:20 -0000 Received: from [127.0.0.1] by omp1001.mail.sp2.yahoo.com with NNFMP; 01 Nov 2010 13:47:20 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 40713.85238.bm@omp1001.mail.sp2.yahoo.com Received: (qmail 89270 invoked by uid 60001); 1 Nov 2010 13:47:19 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1288619239; bh=INyjYYDftG62jriMwu07VsrPe0Pn4ZB/yTsvHnu4+hA=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=LmoQDxY5gv2H+6rY+m6cP43+3D8L6fugbSsm+KWnKf6ubioVyiGxCmB+4hWGrkMqNVz4RXPVvkI1aNWeqwDztzz8e51hZc384TNykgQCxwa8ikLQ53Y6o0a+dQd53V3iOTlm5fEjupkBKqEZ/Tu2/+l6hak7F/5YZ8VWQOIuFvA= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=dwgigXa4e/wxcw1tXBIftiKZTDLyHyYKCMzKWsvUMEuCFYO8tLWelPphuPaUM3Z1UXNWv6GkSKAZw5KbyAXeJN2sXakPqcAnVTsx9cwJbiEyxqlQtmbClz1V0aRlKhEJIy6hqfZFMg/RwEmIyeNfXj6hX0JIL8QBPaa4w+Q/+Js=; Message-ID: <649351.88461.qm@web46207.mail.sp1.yahoo.com> X-YMail-OSG: fu8HjBUVM1nhuRFaFQOvdrkERnCbqOyXtD0SLappjNBVEhU .D9ZMQ407PiDz5BpIPYr6lypc9NB5b5vghG4NIESTDerLsPEZqd9rD.vijyO NICG3MovDVs4EM4mG3_qmMZhVAlb6QEc5GUtK9h.JqZPCXQAFqbX0a3oUl7T Wi71_jTerHiPbzTVrjH7jQSgWHGLUh002KNa_0jDpF09VzNhqrxWnhDgn5WZ X7pA- Received: from [41.100.107.0] by web46207.mail.sp1.yahoo.com via HTTP; Mon, 01 Nov 2010 06:47:19 PDT X-Mailer: YahooMailClassic/11.4.9 YahooMailWebService/0.8.107.284920 Date: Mon, 1 Nov 2010 06:47:19 -0700 (PDT) From: SIFE Abdelrazak Subject: Re: AW: Fail to login with libshh2 To: libssh2 development In-Reply-To: <20101028230351.GA30081@coneharvesters.com> MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 14:47:54 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 14:47:29 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0990309567==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --===============0990309567== Content-Type: multipart/alternative; boundary="0-1976984301-1288619239=:88461" --0-1976984301-1288619239=:88461 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable You can say that=A0 I read all SSH rfc's, even the old one, and to now=A0 I= didn't understand the parameter=A0 specified with kbd_callback function fr= om libssh2 examples.=20 Any way, I use this snippest to authenticate client but this apply only to = a defined password, I wont to accept password from CLI and then authenticat= e=A0 client. static void kbd_callback(const char *name, int name_len, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 co= nst char *instruction, int instruction_len, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 in= t num_prompts, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 co= nst LIBSSH2_USERAUTH_KBDINT_PROMPT *prompts, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 LI= BSSH2_USERAUTH_KBDINT_RESPONSE *responses, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 vo= id **abstract) { =A0=A0=A0 (void)name; =A0=A0=A0 (void)name_len; =A0=A0=A0 (void)instruction; =A0=A0=A0 (void)instruction_len; =A0=A0=A0 if (num_prompts =3D=3D 1) { =A0=A0=A0=A0=A0=A0=A0 responses[0].text =3D strdup(password); =A0=A0=A0=A0=A0=A0=A0 responses[0].length =3D strlen(password); =A0=A0=A0 } =A0=A0=A0 (void)prompts; =A0=A0=A0 (void)abstract; } =0A=0A --0-1976984301-1288619239=:88461 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
You can say that  I read all SSH rfc's, = even the old one, and to now  I didn't understand the parameter  = specified with kbd_callback funct= ion from libssh2 examples.
Any way, I use this snippest to authenticate= client but this apply only to a defined password, I wont to accept passwor= d from CLI and then authenticate  client.

static void kbd_callb= ack(const char *name, int name_len,
      =             &nb= sp;      const char *instruction, int instruction_= len,
           &= nbsp;           &nbs= p; int num_prompts,
         &nbs= p;            &= nbsp;  const LIBSSH2_USERAUTH_KBDINT_PROMPT *prompts,
  &= nbsp;           &nbs= p;          LIBSSH2_USERAUTH_K= BDINT_RESPONSE *responses,
       &nb= sp;            =      void **abstract)
{
    (void)= name;
    (void)name_len;
    (void)ins= truction;
    (void)instruction_len;
   = ; if (num_prompts =3D=3D 1) {
       = responses[0].text =3D strdup(password);
     &= nbsp;  responses[0].length =3D strlen(password);
    }
    (void)prompts;
    (void)abstrac= t;
}


--0-1976984301-1288619239=:88461-- --===============0990309567== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============0990309567==-- From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 15:26:26 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1EQFNT032625; Mon, 1 Nov 2010 15:26:24 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1EQEUc032617 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 15:26:14 +0100 Received: (qmail 14462 invoked from network); 1 Nov 2010 14:26:07 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 14:26:07 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 14:26:07 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:8 Message-ID: <059.ddfff7791cab806bfdc1c9a9cce4b176@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 15:26:26 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 15:26:14 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by stuge): This ticket is about channels in the SSH protocol. Check the protocol RFCs to learn more about those. Replying to [comment:2 robocide]: > 1] does this restriction applys to accuiring more than 10 sftp_handles as well ? libssh2_sftp_init() is the SFTP function that opens a channel, and libssh2_sftp_open() uses that channel to ask the SFTP server running on the remote end to open a file. So if you call _sftp_init() 11 times then the 11th time will fail just like for scp. But _sftp_open() is for talking to the SFTP server that was started by the SSH server for the channel, and it does not open channels. Of course it is possible that the SFTP server also has a limit for 10 open files, just like the SSH server has a limit for 10 open channels, in your case. > 2] why this restriction is not documented in the API reference ? i would be happy to do it as well. As was pointed out this is a server side configuration issue. Your program must always check for errors that libssh2 can return, and your program must always decide how to handle each error that can be returned depending on the internal state of your program. If trying to transfer many files in parallell then there is of course no reason to disconnect from the server or exit the program just because channel or file handle number 11 can not be opened, when there are already some open ones. To study this further for OpenSSH sshd you should look at the function channel_new() in channels.c of the OpenSSH source code. More recent versions of OpenSSH will dynamically allocate extra channels if a client wants to open more than 10, so you could try to use the very latest version of OpenSSH on the server if more than 10 open channels is a requirement. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 15:31:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1EViVs002304; Mon, 1 Nov 2010 15:31:48 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1EV23j002010 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 15:31:02 +0100 Received: (qmail 14798 invoked from network); 1 Nov 2010 14:31:00 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 14:31:00 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 14:31:00 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:9 Message-ID: <059.89584eb32066d0d10a73012bffd5dd79@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 15:31:49 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 15:31:02 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by stuge): Replying to [comment:8 stuge]: > > 1] does this restriction applys to accuiring more than 10 sftp_handles as well ? > > Of course it is possible that the SFTP server also has a limit for 10 open files, To study the OpenSSH SFTP server you should look at the function handle_new() in sftp-server.c in the OpenSSH source code. As you can see it allocates handles dynamically and is thus not limited to 10 open handles. Again maybe this is different in the version you are using, and you could always try to use the latest version of OpenSSH on your server and see if that makes a difference. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 18:10:00 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1H9Gtf011096; Mon, 1 Nov 2010 18:09:55 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1GpnlL031066 for ; Mon, 1 Nov 2010 17:51:50 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=EFhobCjUSCQIIEs1y/tlVuYrVQ6ggBrecJvTqyqTovVGGMg64USlX+7cbfnppGGPyM A8Ut9inm2ni9uiMGibWp7eys9yhxCf9//Hv9o/SICfGF8S9laLyeW9NMPRiYcnwW+E8D Q55xm5fqYkgeIEouGSMN8BWBOq68aSFrwqX68=; bh=mFVs75KoCPHnSHpJeqOUoM4Howg=; t=1288630288; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 93720000832a0934; Mon, 1 Nov 2010 12:51:27 -0400 From: "Mark Riordan" To: References: In-Reply-To: Subject: libssh2_sftp_write now fails with return code -21 Date: Mon, 1 Nov 2010 11:51:43 -0500 Message-ID: <003801cb79e5$1075ac10$31610430$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act5o6qoCtJXJ72mSNibXT/wiRQD3gAQGS9g Content-Language: en-us x-cr-hashedpuzzle: AOH5 Avya B2Hu CPPQ CrgG EYn9 Ez4s FN9h F4LT F68C Gelo GxGI IUxw Ik9u LOgS Ls1v; 1; bABpAGIAcwBzAGgAMgAtAGQAZQB2AGUAbABAAGMAbwBvAGwALgBoAGEAeAB4AC4AcwBlAA==; Sosha1_v1; 7; {78095330-5832-437B-906D-1450926147F3}; bQByAGkAbwByAGQAYQBuAEAAaQBwAHMAdwBpAHQAYwBoAC4AYwBvAG0A; Mon, 01 Nov 2010 16:51:40 GMT; bABpAGIAcwBzAGgAMgBfAHMAZgB0AHAAXwB3AHIAaQB0AGUAIABuAG8AdwAgAGYAYQBpAGwAcwAgAHcAaQB0AGgAIAByAGUAdAB1AHIAbgAgAGMAbwBkAGUAIAAtADIAMQA= x-cr-puzzleid: {78095330-5832-437B-906D-1450926147F3} X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 18:10:00 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 17:51:51 +0100 (CET) X-Mailman-Approved-At: Mon, 01 Nov 2010 18:09:15 +0100 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se > I've worked on getting rid of all the -1 return codes since it is too generic > error (that previously was used pretty much all over and didn't really help > much). I found some new cases that would cause -1 to get returned just now > though, and I've pushed a change that hopefully will make you get a different > error code (and error message) that can be useful to better track this down. > Please get these changes and re-run. Good idea. Now I'm getting error -21. My latest round of testing was done only with Windows 2003R2 as the client platform. > How big file do you send? My test file is 60 MB and it's not compressible. But I don't think the size is important, as the upload fails right away. > How's the round-trip delay between your client and > the server (roughly)? Less than 1 ms, according to ping. > What server software are you using? Sun Solaris, with whatever their standard sshd is. And Ubuntu 10.04 with OpenSSH. Both servers now return -21 with the latest libssh2. Mark R _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 19:44:20 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Ii1UL008322; Mon, 1 Nov 2010 19:44:18 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Ii0Z9008289 for ; Mon, 1 Nov 2010 19:44:00 +0100 Date: Mon, 1 Nov 2010 19:44:00 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write now fails with return code -21 In-Reply-To: <003801cb79e5$1075ac10$31610430$@com> Message-ID: References: <003801cb79e5$1075ac10$31610430$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 19:44:20 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 19:44:00 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Mon, 1 Nov 2010, Mark Riordan wrote: > Now I'm getting error -21. Great, now we're at least narrowing it down. -21 is LIBSSH2_ERROR_CHANNEL_FAILURE and it is only used in a few places. Can you check what error message string libssh2_session_last_error() returns after you've got the error? It should help us to pinpoint exactly which of the channel failures it was. > My test file is 60 MB and it's not compressible. But I don't think the size > is important, as the upload fails right away. Right. If we're lucky you get the error at src/sftp.c:198 (as that's marked as a TODO and I know what the problem with it is and what we should do to fix it....) I think the main reason I haven't seen this problem is that I've only tested on Linux and I figure the TCP stack differences are big enough to be important here. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 22:44:00 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LhX7B023337; Mon, 1 Nov 2010 22:43:56 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LhV6A023283 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 22:43:31 +0100 Received: (qmail 15831 invoked from network); 1 Nov 2010 21:43:27 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 21:43:27 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 21:43:27 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #195: crash during session setup if remote host does not offer valid keys X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/195 Message-ID: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Trac-Ticket-ID: 195 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 22:44:00 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 22:43:31 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #195: crash during session setup if remote host does not offer valid keys ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawlhggg_yplkl7grwwpbbum-omtqud4rmna | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.6 Component: protocol | Version: 1.2.2 Keywords: | Blocks: Blocked By: | ---------------------------------------------------------------------------------------+ While setting up the session, ssh tries to determine the type of encryption method it can use for the session. This requires looking at the keys offered by the remote host and comparing these with the methods supported by libssh2 (rsa & dss). To do this there is an iteration over the array containing the methods supported by libssh2. This array is defined as (in src/hostkey.c) static const LIBSSH2_HOSTKEY_METHOD *hostkey_methods[] = { #if LIBSSH2_RSA &hostkey_method_ssh_rsa, #endif /* LIBSSH2_RSA */ #if LIBSSH2_DSA &hostkey_method_ssh_dss, #endif /* LIBSSH2_DSA */ NULL }; While trying to determine the encryption method there is code in src/kex.c of the form //hostkyp's type is LIBSSH2_HOSTKEY_METHOD* [] while (hostkeyp && (*hostkeyp)->name) { //Examine remote host key to see if we can agree on encryption method. hostkeyp++; } If there is no agreement on the type of encryption we come to the 3rd entry of the hostkeyp array. Here hostkeyp is valid but *hostkep is NULL. Thus when we dereference that in (*hostkeyp)->name there is a crash with the following backtrace. #0 0x00007f0e04103ecc in kex_agree_hostkey (session=0x602040, kex_flags=2, hostkey=0x6079d3 " ", hostkey_len=1) at kex.c:1271 1271 kex.c: No such file or directory. in kex.c (gdb) bt #0 0x00007f0e04103ecc in kex_agree_hostkey (session=0x602040, kex_flags=2, hostkey=0x6079d3 " ", hostkey_len=1) at kex.c:1271 #1 0x00007f0e04104a0b in libssh2_kex_exchange (session=0x602040, reexchange=, key_state=0x606410) at kex.c:1355 #2 0x00007f0e0410bb6e in libssh2_session_startup (session=0x602040, sock=3) at session.c:594 #3 0x0000000000400da0 in main () (gdb) I have fixed this issue in my copy of the library and I am submitting the patch for inclusion here. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 22:55:18 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LtBvB029346; Mon, 1 Nov 2010 22:55:17 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LtAYD029338 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 22:55:10 +0100 Received: (qmail 16472 invoked from network); 1 Nov 2010 21:55:06 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 21:55:06 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 21:55:06 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #186: libssh gets stuck at kex_method_diffie_hellman_groupGP_sha1_key_exchange. CPU utilization goes upto 100% X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/186#comment:1 Message-ID: <123.fa1a2ee02445fa29e88e5ce7dd0884cc@libssh2.stuge.se> References: <108.a5a9892915fbecf282faa78e591a6c63@libssh2.stuge.se> X-Trac-Ticket-ID: 186 In-Reply-To: <108.a5a9892915fbecf282faa78e591a6c63@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 22:55:18 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 22:55:10 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE4NjogbGlic3NoIGdldHMgc3R1Y2sgYXQga2V4X21ldGhvZF9kaWZmaWVfaGVsbG1hbl9ncm91 cEdQX3NoYTFfa2V5X2V4Y2hhbmdlLg0KQ1BVIHV0aWxpemF0aW9uIGdvZXMgdXB0byAxMDAlCi0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSsNCiAgUmVwb3J0ZXI6ICB3d3cuZ29vZ2xlLmNv bS9hY2NvdW50cy9vOC9pZD9pZD1haXRvYXdsaGdnZ195cGxrbDdncnd3cGJidW0tb210cXVkNHJt bmEgIHwgICAgICAgIE93bmVyOiAgICAgICANCiAgICAgIFR5cGU6ICBkZWZlY3QgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHwgICAgICAgU3RhdHVzOiAgbmV3ICANCiAgUHJpb3JpdHk6ICBub3JtYWwgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwg ICAgTWlsZXN0b25lOiAgMS4yLjYNCiBDb21wb25lbnQ6ICBjcnlwdG8gICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAg ICBWZXJzaW9uOiAgMS4yLjINClJlc29sdXRpb246ICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgIEtl eXdvcmRzOiAgICAgICANCiAgICBCbG9ja3M6ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICBCbG9ja2Vk IEJ5OiAgICAgICANCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSsKQ2hhbmdlcyAoYnkg d3d3Lmdvb2dsZS5jb20vYWNjb3VudHMvbzgvaWQ/aWQ9YWl0b2F3bGhnZ2dfeXBsa2w3Z3J3d3Bi YnVtLW9tdHF1ZDRybW5hKToKCiAqIGNjOiBqYXNtZWV0YmFnZ2FA4oCmIChhZGRlZCkNCgoKLS0g ClRpY2tldCBVUkw6IDxodHRwOi8vdHJhYy5saWJzc2gyLm9yZy90aWNrZXQvMTg2I2NvbW1lbnQ6 MT4KbGlic3NoMiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcvPgpDIGxpYnJhcnkgZm9yIHdyaXRp bmcgcG9ydGFibGUgU1NIMiBjbGllbnRzCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmxpYnNzaDItZGV2ZWwgaHR0cDovL2Nvb2wuaGF4eC5zZS9jZ2ktYmlu L21haWxtYW4vbGlzdGluZm8vbGlic3NoMi1kZXZlbAo= From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 22:55:51 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LtnBm029665; Mon, 1 Nov 2010 22:55:51 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1LtmkE029659 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 22:55:48 +0100 Received: (qmail 16497 invoked from network); 1 Nov 2010 21:55:44 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 21:55:44 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 21:55:44 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #195: crash during session setup if remote host does not offer valid keys X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/195#comment:1 Message-ID: <123.d918143719171666036a04dab2bcae64@libssh2.stuge.se> References: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Trac-Ticket-ID: 195 In-Reply-To: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 22:55:51 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 22:55:48 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE5NTogY3Jhc2ggZHVyaW5nIHNlc3Npb24gc2V0dXAgaWYgcmVtb3RlIGhvc3QgZG9lcyBub3Qg b2ZmZXIgdmFsaWQga2V5cwotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rDQogIFJlcG9y dGVyOiAgd3d3Lmdvb2dsZS5jb20vYWNjb3VudHMvbzgvaWQ/aWQ9YWl0b2F3bGhnZ2dfeXBsa2w3 Z3J3d3BiYnVtLW9tdHF1ZDRybW5hICB8ICAgICAgICBPd25lcjogICAgICAgDQogICAgICBUeXBl OiAgZGVmZWN0ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICB8ICAgICAgIFN0YXR1czogIG5ldyAgDQogIFByaW9yaXR5OiAg bm9ybWFsICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8ICAgIE1pbGVzdG9uZTogIDEuMi42DQogQ29tcG9uZW50OiAgcHJv dG9jb2wgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICB8ICAgICAgVmVyc2lvbjogIDEuMi4yDQpSZXNvbHV0aW9uOiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICB8ICAgICBLZXl3b3JkczogICAgICAgDQogICAgQmxvY2tzOiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICB8ICAgQmxvY2tlZCBCeTogICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0rCkNoYW5nZXMgKGJ5IHd3dy5nb29nbGUuY29tL2FjY291bnRzL284L2lkP2lkPWFpdG9h d2xoZ2dnX3lwbGtsN2dyd3dwYmJ1bS1vbXRxdWQ0cm1uYSk6CgogKiBjYzogamFzbWVldGJhZ2dh QOKApiAoYWRkZWQpDQoKCi0tIApUaWNrZXQgVVJMOiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcv dGlja2V0LzE5NSNjb21tZW50OjE+CmxpYnNzaDIgPGh0dHA6Ly90cmFjLmxpYnNzaDIub3JnLz4K QyBsaWJyYXJ5IGZvciB3cml0aW5nIHBvcnRhYmxlIFNTSDIgY2xpZW50cwpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaWJzc2gyLWRldmVsIGh0dHA6Ly9j b29sLmhheHguc2UvY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL2xpYnNzaDItZGV2ZWwK From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:03:12 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1M375b004661; Mon, 1 Nov 2010 23:03:12 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oA1M34O9004619 for ; Mon, 1 Nov 2010 23:03:04 +0100 Received: (qmail 718 invoked by uid 501); 1 Nov 2010 22:03:01 -0000 Message-ID: <20101101220301.717.qmail@stuge.se> Date: Mon, 1 Nov 2010 23:03:01 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: Fail to login with libshh2 Mail-Followup-To: libssh2-devel@cool.haxx.se References: <20101028230351.GA30081@coneharvesters.com> <649351.88461.qm@web46207.mail.sp1.yahoo.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <649351.88461.qm@web46207.mail.sp1.yahoo.com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:03:12 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:03:04 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se U0lGRSBBYmRlbHJhemFrIHdyb3RlOgo+IFlvdSBjYW4gc2F5IHRoYXTCoCBJIHJlYWQgYWxsIFNT SCByZmMncywgZXZlbiB0aGUgb2xkIG9uZSwgYW5kIHRvCj4gbm93IEkgZGlkbid0IHVuZGVyc3Rh bmQgdGhlIHBhcmFtZXRlciBzcGVjaWZpZWQgd2l0aCBrYmRfY2FsbGJhY2sKPiBmdW5jdGlvbiBm cm9tIGxpYnNzaDIgZXhhbXBsZXMuCgpQbGVhc2UgcmVhZCBSRkMgNDI1Ni4KCgovL1BldGVyCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpYnNzaDItZGV2 ZWwgaHR0cDovL2Nvb2wuaGF4eC5zZS9jZ2ktYmluL21haWxtYW4vbGlzdGluZm8vbGlic3NoMi1k ZXZlbAo= From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:26:12 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MQ32o022878; Mon, 1 Nov 2010 23:26:10 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MQ1E9022817 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:26:01 +0100 Received: (qmail 19631 invoked from network); 1 Nov 2010 22:25:57 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:25:57 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:25:57 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #165: size_t conversion X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/165#comment:3 Message-ID: <059.55e8da7a4366f781b3812e578af20592@libssh2.stuge.se> References: <044.0e6545d8a407f07c307985c36b19f9b7@libssh2.stuge.se> X-Trac-Ticket-ID: 165 In-Reply-To: <044.0e6545d8a407f07c307985c36b19f9b7@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:26:12 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:26:01 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #165: size_t conversion --------------------------+------------------------------------------------- Reporter: roadrunn | Owner: Type: enhancement | Status: closed Priority: normal | Milestone: Component: API | Version: 1.2.4 Resolution: outdated | Keywords: Blocks: | Blocked By: --------------------------+------------------------------------------------- Changes (by bagder): * status: new => closed * resolution: => outdated Comment: As there's no work going on here and the existing patch is old and not in a desired shape, I'll just close this. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:30:36 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MUXBw024629; Mon, 1 Nov 2010 23:30:35 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MUVsZ024608 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:30:32 +0100 Received: (qmail 20216 invoked from network); 1 Nov 2010 22:30:27 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:30:27 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:30:27 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #195: crash during session setup if remote host does not offer valid keys X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/195#comment:2 Message-ID: <123.78d4cbc7dc492c34c043362742e1a2d5@libssh2.stuge.se> References: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Trac-Ticket-ID: 195 In-Reply-To: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:30:36 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:30:32 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #195: crash during session setup if remote host does not offer valid keys ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawlhggg_yplkl7grwwpbbum-omtqud4rmna | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.8 Component: protocol | Version: 1.2.2 Resolution: | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by bagder): * milestone: 1.2.6 => 1.2.8 Comment: Thanks a lot. If you give me your name I'll commit this fix using it! -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:33:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MXioW026185; Mon, 1 Nov 2010 23:33:48 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MXgHI026157 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:33:43 +0100 Received: (qmail 20654 invoked from network); 1 Nov 2010 22:33:39 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:33:39 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:33:39 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #187: Unable to connect to SFTP server using SSH-2.0-SSHD X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/187#comment:2 Message-ID: <062.e71ca905ba2c036c667032bf9de86141@libssh2.stuge.se> References: <047.ff473ed169991c7f4a9e3fa703b14621@libssh2.stuge.se> X-Trac-Ticket-ID: 187 In-Reply-To: <047.ff473ed169991c7f4a9e3fa703b14621@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:33:48 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:33:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #187: Unable to connect to SFTP server using SSH-2.0-SSHD --------------------------+------------------------------------------------- Reporter: sriramswamy | Owner: Type: defect | Status: new Priority: blocker | Milestone: Component: SFTP | Version: 1.2.7 Resolution: | Keywords: awaitsinfo Blocks: | Blocked By: --------------------------+------------------------------------------------- Changes (by bagder): * keywords: => awaitsinfo * milestone: 1.2.7 => -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:40:02 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Mdtm8029276; Mon, 1 Nov 2010 23:40:01 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MdrfO029257 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:39:53 +0100 Received: (qmail 21152 invoked from network); 1 Nov 2010 22:39:49 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:39:49 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:39:49 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #170: Test on Solaris 10 amd64 hangs X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/170#comment:5 Message-ID: <123.011bab3baddd533d1dff4c05e74cb1f2@libssh2.stuge.se> References: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Trac-Ticket-ID: 170 In-Reply-To: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:40:02 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:39:53 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #170: Test on Solaris 10 amd64 hangs ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=AItOawmzyrEUkUDbHXFBqjFbBoB5p8Ik5X9sR1A | Owner: stuge Type: defect | Status: assigned Priority: normal | Milestone: Component: tests | Version: 1.2.5 Resolution: | Keywords: select, timeout, hang Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by bagder): * component: API => tests Comment: Does this happen with 1.2.7 or the current git? If so, does it _only_ happen on the test case or do you see this problem in the wild too? Can you build a debug version of the lib and make the test enable full tracing and show that output to us? -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:41:41 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Mfake030435; Mon, 1 Nov 2010 23:41:40 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MfZBk030420 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:41:35 +0100 Received: (qmail 21309 invoked from network); 1 Nov 2010 22:41:31 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:41:31 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:41:31 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #184: Decompression buffers are too small X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/184#comment:2 Message-ID: <067.d311314e26b33be73578c476c167ece6@libssh2.stuge.se> References: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Trac-Ticket-ID: 184 In-Reply-To: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:41:41 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:41:35 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE4NDogRGVjb21wcmVzc2lvbiBidWZmZXJzIGFyZSB0b28gc21hbGwKLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQ0KICBSZXBvcnRlcjogIHRqQOKApiAgICAgICAgICAgICAgfCAgICAgICAgT3duZXI6ICBi YWdkZXIgIA0KICAgICAgVHlwZTogIGRlZmVjdCAgICAgICAgICAgIHwgICAgICAgU3RhdHVzOiAg YXNzaWduZWQNCiAgUHJpb3JpdHk6ICBub3JtYWwgICAgICAgICAgICB8ICAgIE1pbGVzdG9uZTog ICAgICAgICAgDQogQ29tcG9uZW50OiAgcHJvdG9jb2wgICAgICAgICAgfCAgICAgIFZlcnNpb246 ICAxLjIuNyAgIA0KUmVzb2x1dGlvbjogICAgICAgICAgICAgICAgICAgIHwgICAgIEtleXdvcmRz OiAgICAgICAgICANCiAgICBCbG9ja3M6ICAgICAgICAgICAgICAgICAgICB8ICAgQmxvY2tlZCBC eTogICAgICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCkNoYW5nZXMgKGJ5IGJhZ2Rlcik6Cgog ICogb3duZXI6ICA9PiBiYWdkZXINCiAgKiBzdGF0dXM6ICBuZXcgPT4gYXNzaWduZWQNCgoKQ29t bWVudDoKCiBDYW4geW91IHBsZWFzZSByZXN5bmMgdGhpcyBhZ2FpbnN0IHRoZSBjdXJyZW50IGdp dD8gSXQgY3VycmVudGx5DQogY29tcGxldGVseSBmYWlscyB0byBhcHBseS4uLgoKLS0gClRpY2tl dCBVUkw6IDxodHRwOi8vdHJhYy5saWJzc2gyLm9yZy90aWNrZXQvMTg0I2NvbW1lbnQ6Mj4KbGli c3NoMiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcvPgpDIGxpYnJhcnkgZm9yIHdyaXRpbmcgcG9y dGFibGUgU1NIMiBjbGllbnRzCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fCmxpYnNzaDItZGV2ZWwgaHR0cDovL2Nvb2wuaGF4eC5zZS9jZ2ktYmluL21haWxt YW4vbGlzdGluZm8vbGlic3NoMi1kZXZlbAo= From libssh2-devel-bounces@cool.haxx.se Mon Nov 1 23:48:11 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1MlwoS000986; Mon, 1 Nov 2010 23:48:09 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Mlu5k000970 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 1 Nov 2010 23:47:56 +0100 Received: (qmail 21978 invoked from network); 1 Nov 2010 22:47:52 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 22:47:52 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 22:47:52 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #195: crash during session setup if remote host does not offer valid keys X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/195#comment:3 Message-ID: <123.208bfab7c4754ef6705bace713cbcbf3@libssh2.stuge.se> References: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Trac-Ticket-ID: 195 In-Reply-To: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 01 Nov 2010 23:48:11 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 01 Nov 2010 23:47:56 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #195: crash during session setup if remote host does not offer valid keys ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawlhggg_yplkl7grwwpbbum-omtqud4rmna | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.8 Component: protocol | Version: 1.2.2 Resolution: | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Comment (by www.google.com/accounts/o8/id?id=aitoawlhggg_yplkl7grwwpbbum-omtqud4rmna): Replying to [comment:2 bagder]: > Thanks a lot. If you give me your name I'll commit this fix using it! Jasmeet Bagga. Thanks for your quick response. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 00:04:47 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1N4dV2008355; Tue, 2 Nov 2010 00:04:46 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1N4a6Q008291 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 2 Nov 2010 00:04:37 +0100 Received: (qmail 23164 invoked from network); 1 Nov 2010 23:04:33 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 23:04:33 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 23:04:33 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #195: crash during session setup if remote host does not offer valid keys X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/195#comment:4 Message-ID: <123.01faa2ea389613f4f3b007d71a20cc6c@libssh2.stuge.se> References: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Trac-Ticket-ID: 195 In-Reply-To: <108.61c653b6ed5d03101a66821462ff60ea@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 00:04:47 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 00:04:37 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #195: crash during session setup if remote host does not offer valid keys ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawlhggg_yplkl7grwwpbbum-omtqud4rmna | Owner: Type: defect | Status: closed Priority: normal | Milestone: 1.2.8 Component: protocol | Version: 1.2.2 Resolution: fixed | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by bagder): * status: new => closed * resolution: => fixed Comment: Thanks, fixed and pushed, commit 73be9fab04691ab6d1e1cabe39b4c79b9c7421ba -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 00:37:19 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1NbA0u025553; Tue, 2 Nov 2010 00:37:19 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA1Nb8eo025528 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 2 Nov 2010 00:37:08 +0100 Received: (qmail 25893 invoked from network); 1 Nov 2010 23:37:04 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 1 Nov 2010 23:37:04 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 01 Nov 2010 23:37:04 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #184: Decompression buffers are too small X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/184#comment:3 Message-ID: <067.0470d475d90d10f71355b28991d69d13@libssh2.stuge.se> References: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Trac-Ticket-ID: 184 In-Reply-To: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 00:37:19 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 00:37:08 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE4NDogRGVjb21wcmVzc2lvbiBidWZmZXJzIGFyZSB0b28gc21hbGwKLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQ0KICBSZXBvcnRlcjogIHRqQOKApiAgICAgICAgICAgICAgfCAgICAgICAgT3duZXI6ICBi YWdkZXIgIA0KICAgICAgVHlwZTogIGRlZmVjdCAgICAgICAgICAgIHwgICAgICAgU3RhdHVzOiAg YXNzaWduZWQNCiAgUHJpb3JpdHk6ICBub3JtYWwgICAgICAgICAgICB8ICAgIE1pbGVzdG9uZTog ICAgICAgICAgDQogQ29tcG9uZW50OiAgcHJvdG9jb2wgICAgICAgICAgfCAgICAgIFZlcnNpb246 ICAxLjIuNyAgIA0KUmVzb2x1dGlvbjogICAgICAgICAgICAgICAgICAgIHwgICAgIEtleXdvcmRz OiAgICAgICAgICANCiAgICBCbG9ja3M6ICAgICAgICAgICAgICAgICAgICB8ICAgQmxvY2tlZCBC eTogICAgICAgICAgDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgpDb21tZW50IChieSB0akDigKYpOgoK IERvbmUuICBJIGhhdmUgY3JlYXRlZCB0d28gc2VwYXJhdGUgcGF0Y2hlczsgb25lIGZvciBhZGRp bmcganVzdCB0aGUgZGVidWcNCiB0cmFjaW5nIG9mIHpsaWIgZXJyb3JzLCBhbmQgYSBzZXBhcmF0 ZSBvbmUgZm9yIHVzaW5nIGxhcmdlciBidWZmZXIgc2l6ZXMNCiBmb3IgZGVjb21wcmVzc2lvbi4K Ci0tIApUaWNrZXQgVVJMOiA8aHR0cDovL3RyYWMubGlic3NoMi5vcmcvdGlja2V0LzE4NCNjb21t ZW50OjM+CmxpYnNzaDIgPGh0dHA6Ly90cmFjLmxpYnNzaDIub3JnLz4KQyBsaWJyYXJ5IGZvciB3 cml0aW5nIHBvcnRhYmxlIFNTSDIgY2xpZW50cwpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXwpsaWJzc2gyLWRldmVsIGh0dHA6Ly9jb29sLmhheHguc2UvY2dp LWJpbi9tYWlsbWFuL2xpc3RpbmZvL2xpYnNzaDItZGV2ZWwK From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 10:36:34 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA29ZqIr003999; Tue, 2 Nov 2010 10:36:30 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA29Zo9L003983 for ; Tue, 2 Nov 2010 10:35:50 +0100 Date: Tue, 2 Nov 2010 10:35:50 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: SFTP upload compared? Message-ID: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 10:36:34 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 10:35:50 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hi friends, If there are anyone out there who does SFTP uploads with libssh2, I would appreciate if you took the current git version for a spin and compared it with the former version(s) and see how it behaves, both functionailty wise and speed wise. And then tell us about it! (Preferably then tests that are against servers that are bit remote, with larger RTTs.) If we can just establish that this (generally) works and is an improvement, then we can proceed and apply the same concept on SFTP downloads. I just want to wait until the upload part has been polished and proven a bit. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 11:02:28 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2A2HWY024122; Tue, 2 Nov 2010 11:02:26 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2A2FAh024088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 2 Nov 2010 11:02:15 +0100 Received: (qmail 17563 invoked from network); 2 Nov 2010 10:02:14 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 2 Nov 2010 10:02:14 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Tue, 02 Nov 2010 10:02:14 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #170: Test on Solaris 10 amd64 hangs X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/170#comment:6 Message-ID: <123.762290c35fd1a2b35e68fead92d483c4@libssh2.stuge.se> References: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Trac-Ticket-ID: 170 In-Reply-To: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 11:02:28 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 11:02:15 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #170: Test on Solaris 10 amd64 hangs ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=AItOawmzyrEUkUDbHXFBqjFbBoB5p8Ik5X9sR1A | Owner: stuge Type: defect | Status: assigned Priority: normal | Milestone: Component: tests | Version: 1.2.5 Resolution: | Keywords: select, timeout, hang Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Comment (by www.google.com/accounts/o8/id?id=aitoawnwnbfr9defttu_ljqvasu3w7up5yxizmc): I tested with 1.2.7 as I have problems with the autoconf bootstrapping. I have never seen the problem in the wild because the 64 bit version of the library is AFAIK not used by any application yet. The full tracing output looks like this: {{{ current10x% DEBUG=1 gmake check gmake simple gmake[1]: Entering directory `/home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests' gmake[1]: `simple' is up to date. gmake[1]: Leaving directory `/home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests' gmake check-TESTS gmake[1]: Entering directory `/home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests' PASS: simple debug2: load_server_config: filename /dev/null debug2: load_server_config: done config len = 1 debug2: parse_server_config: config /dev/null len 1 debug1: sshd version OpenSSH_5.4p1 debug1: read PEM private key done: type RSA debug1: private host key: #0 type 1 RSA debug1: setgroups() failed: Not owner debug1: rexec_argv[0]='/opt/csw/sbin/sshd' debug1: rexec_argv[1]='-f' debug1: rexec_argv[2]='/dev/null' debug1: rexec_argv[3]='-h' debug1: rexec_argv[4]='/home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386 /build-isa-amd64/libssh2-1.2.7/tests/etc/host' debug1: rexec_argv[5]='-o' debug1: rexec_argv[6]='Port 4711' debug1: rexec_argv[7]='-o' debug1: rexec_argv[8]='Protocol 2' debug1: rexec_argv[9]='-o' debug1: rexec_argv[10]='AuthorizedKeysFile /home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests/etc/user.pub' debug1: rexec_argv[11]='-o' debug1: rexec_argv[12]='UsePrivilegeSeparation no' debug1: rexec_argv[13]='-o' debug1: rexec_argv[14]='StrictModes no' debug1: rexec_argv[15]='-D' debug1: rexec_argv[16]='-d' debug1: rexec_argv[17]='-d' debug2: fd 4 setting O_NONBLOCK debug1: Bind to port 4711 on ::. Server listening on :: port 4711. debug2: fd 5 setting O_NONBLOCK debug1: Bind to port 4711 on 0.0.0.0. Server listening on 0.0.0.0 port 4711. debug1: fd 6 clearing O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug1: rexec start in 6 out 6 newsock 6 pipe -1 sock 11 debug1: inetd sockets after dupping: 4, 4 Connection from 127.0.0.1 port 65077 debug1: Client protocol version 2.0; client software version libssh2_1.2.7 debug1: no match: libssh2_1.2.7 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.4 debug2: fd 4 setting O_NONBLOCK debug1: list_hostkey_types: ssh-rsa debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie- hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman- group1-sha1 debug2: kex_parse_kexinit: ssh-rsa debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc ,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael- cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc ,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael- cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac- ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac- ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group14-sha1,diffie-hellman- group-exchange-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc ,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish- cbc,arcfour128,arcfour,cast128-cbc,3des-cbc debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc ,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish- cbc,arcfour128,arcfour,cast128-cbc,3des-cbc debug2: kex_parse_kexinit: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96 ,hmac-ripemd160,hmac-ripemd160@openssh.com debug2: kex_parse_kexinit: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96 ,hmac-ripemd160,hmac-ripemd160@openssh.com debug2: kex_parse_kexinit: none,zlib debug2: kex_parse_kexinit: none,zlib debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-sha1 debug1: kex: client->server aes128-ctr hmac-sha1 none debug2: mac_setup: found hmac-sha1 debug1: kex: server->client aes128-ctr hmac-sha1 none debug2: dh_gen_key: priv key bits set: 169/320 debug2: bits set: 1077/2048 debug1: expecting SSH2_MSG_KEXDH_INIT debug2: bits set: 1054/2048 debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: KEX done Fingerprint: 86 AD B2 21 33 60 65 3D 9A 29 86 DE 22 99 DA 18 CC BA D3 AC debug1: userauth-request for user dam service ssh-connection method none debug1: attempt 0 failures 0 debug2: parse_server_config: config reprocess config len 1 debug2: input_userauth_request: setting up authctxt for dam debug2: input_userauth_request: try method none Failed none for dam from 127.0.0.1 port 65077 ssh2 Authentication methods: publickey,password,keyboard-interactive debug1: userauth-request for user dam service ssh-connection method publickey debug1: attempt 1 failures 0 debug2: input_userauth_request: try method publickey debug1: test whether pkalg/pkblob are acceptable debug1: temporarily_use_uid: 10000/10000 (e=10000/10000) debug1: trying public key file /home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests/etc/user.pub debug1: fd 5 clearing O_NONBLOCK debug1: matching key found: file /home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests/etc/user.pub, line 1 Found matching RSA key: 48:2e:33:0d:0d:ca:dc:40:08:8b:0f:2c:22:74:7d:37 debug1: restore_uid: (unprivileged) debug2: userauth_pubkey: authenticated 0 pkalg ssh-rsa Postponed publickey for dam from 127.0.0.1 port 65077 ssh2 debug1: userauth-request for user dam service ssh-connection method publickey debug1: attempt 2 failures 0 debug2: input_userauth_request: try method publickey debug1: temporarily_use_uid: 10000/10000 (e=10000/10000) debug1: trying public key file /home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests/etc/user.pub debug1: fd 5 clearing O_NONBLOCK debug1: matching key found: file /home/dam/mgar/pkg/libssh2/trunk/work/solaris9-i386/build-isa- amd64/libssh2-1.2.7/tests/etc/user.pub, line 1 Found matching RSA key: 48:2e:33:0d:0d:ca:dc:40:08:8b:0f:2c:22:74:7d:37 debug1: restore_uid: (unprivileged) debug1: ssh_rsa_verify: signature correct debug2: userauth_pubkey: authenticated 1 pkalg ssh-rsa Accepted publickey for dam from 127.0.0.1 port 65077 ssh2 debug1: Entering interactive session for SSH2. Authentication by public key succeeded. debug2: fd 5 setting O_NONBLOCK debug2: fd 6 setting O_NONBLOCK debug1: server_init_dispatch_20 debug1: server_input_channel_open: ctype session rchan 0 win 65536 max 32768 debug1: input_session_request debug1: channel 0: new [server-session] debug2: session_new: allocate (allocated 0 max 10) debug1: session_new: session 0 debug1: session_open: channel 0 debug1: session_open: session 0: link with channel 0 debug1: server_input_channel_open: confirm session debug1: server_input_channel_req: channel 0 request env reply 1 debug1: session_by_channel: session 0 channel 0 debug1: session_input_channel_req: session 0 req env debug2: Ignoring env request FOO: disallowed name debug1: server_input_channel_req: channel 0 request pty-req reply 1 debug1: session_by_channel: session 0 channel 0 debug1: session_input_channel_req: session 0 req pty-req debug1: Allocating pty. debug1: session_pty_req: session 0 alloc /dev/pts/6 debug1: server_input_channel_req: channel 0 request shell reply 1 debug1: session_by_channel: session 0 channel 0 debug1: session_input_channel_req: session 0 req shell debug2: fd 4 setting TCP_NODELAY debug2: channel 0: rfd 9 isatty debug2: fd 9 setting O_NONBLOCK debug2: channel 0: rcvd eof debug2: channel 0: output open -> drain debug2: channel 0: obuf empty debug2: channel 0: close_write debug2: channel 0: output drain -> closed debug2: channel 0: rcvd close debug2: channel 0: close_read debug2: channel 0: input open -> closed debug2: channel 0: almost dead debug2: channel 0: gc: notify user debug1: session_by_channel: session 0 channel 0 debug1: session_close_by_channel: channel 0 child 17458 debug1: session_close_by_channel: channel 0: has child debug1: session_pty_cleanup: session 0 release /dev/pts/6 Attempt to write login records by non-root user (aborting) }}} Than it hangs. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 11:02:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2A2gku024468; Tue, 2 Nov 2010 11:02:44 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2A2eiZ024438 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 2 Nov 2010 11:02:40 +0100 Received: (qmail 17664 invoked from network); 2 Nov 2010 10:02:39 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 2 Nov 2010 10:02:39 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Tue, 02 Nov 2010 10:02:39 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #170: Test on Solaris 10 amd64 hangs X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/170#comment:7 Message-ID: <123.608fd3fa58b7b23a8e065fff3cb5f175@libssh2.stuge.se> References: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Trac-Ticket-ID: 170 In-Reply-To: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 11:02:44 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 11:02:41 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #170: Test on Solaris 10 amd64 hangs ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=AItOawmzyrEUkUDbHXFBqjFbBoB5p8Ik5X9sR1A | Owner: stuge Type: defect | Status: assigned Priority: normal | Milestone: Component: tests | Version: 1.2.7 Resolution: | Keywords: select, timeout, hang Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by www.google.com/accounts/o8/id?id=aitoawnwnbfr9defttu_ljqvasu3w7up5yxizmc): * version: 1.2.5 => 1.2.7 -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 15:13:25 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2ED24I024660; Tue, 2 Nov 2010 15:13:22 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2ED0It024576 for ; Tue, 2 Nov 2010 15:13:01 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=Qyd6GRq0UBXZmz6HN59XCzBYwqYVPtB6uRyxNPVPJ00nLe2Pq5po8Id9uCS0siUeF1 MaC5xdQaPDAhGAsEcqn44JVIg+C9ybakTO4cvJ/RqN6IWoAxxBTnZcbNWxX6daiFWJ/Z rOsGNb/9HMkAJE0SVgqqMC0lbPk82ZVzKlhdE=; bh=XTIpXMdaOz4po4Y+zUZ0pM2inPo=; t=1288707157; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 8d7200012f0d052b; Tue, 2 Nov 2010 10:12:35 -0400 From: "Mark Riordan" To: Subject: Re: libssh2_sftp_write now fails with return code -21 Date: Tue, 2 Nov 2010 09:13:09 -0500 Message-ID: <000f01cb7a98$14030130$3c090390$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act6mBOVEkuQNpq6S+2zgy/oxaItkg== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 15:13:25 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 15:13:01 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > -21 is LIBSSH2_ERROR_CHANNEL_FAILURE and it is only used in a few places. > > Can you check what error message string libssh2_session_last_error() > returns after you've got the error? It should help us to pinpoint > exactly which of the channel failures it was. libssh2_session_last_error returns "Waiting for SFTP status". (If it's waiting, it's not very patient, as the error occurs immediately.) Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 2 19:20:00 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2IJYdh012167; Tue, 2 Nov 2010 19:19:56 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA2IJXFR012161 for ; Tue, 2 Nov 2010 19:19:33 +0100 Date: Tue, 2 Nov 2010 19:19:33 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write now fails with return code -21 In-Reply-To: <000f01cb7a98$14030130$3c090390$@com> Message-ID: References: <000f01cb7a98$14030130$3c090390$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 02 Nov 2010 19:20:00 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 02 Nov 2010 19:19:33 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Tue, 2 Nov 2010, Mark Riordan wrote: >> Can you check what error message string libssh2_session_last_error() >> returns after you've got the error? It should help us to pinpoint exactly >> which of the channel failures it was. > > libssh2_session_last_error returns "Waiting for SFTP status". Cool, thanks! > (If it's waiting, it's not very patient, as the error occurs immediately.) Well, the error occured when it was in fact waiting. I believe the problem was that the code didn't properly deal with the 32bit length field (that starts each SFTP packet) when the whole field could not be read at once. I've pushed a fix for this, please try again and if it still fails I'm interested in the return code and error message again! -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 3 14:56:13 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3DtWqL018146; Wed, 3 Nov 2010 14:56:10 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3DtU4c018130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 3 Nov 2010 14:55:31 +0100 Received: (qmail 15950 invoked from network); 3 Nov 2010 13:55:28 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 3 Nov 2010 13:55:28 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 03 Nov 2010 13:55:28 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #184: Decompression buffers are too small X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/184#comment:4 Message-ID: <067.530b6fb3261f1eb05e49a12e5d5f16bd@libssh2.stuge.se> References: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Trac-Ticket-ID: 184 In-Reply-To: <052.8b031ed1b041784e6eb93c8278090860@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 03 Nov 2010 14:56:13 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 03 Nov 2010 14:55:31 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se IzE4NDogRGVjb21wcmVzc2lvbiBidWZmZXJzIGFyZSB0b28gc21hbGwKLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQ0KICBSZXBvcnRlcjogIHRqQOKApiAgICAgICAgICAgICAgfCAgICAgICAgT3duZXI6ICBi YWdkZXINCiAgICAgIFR5cGU6ICBkZWZlY3QgICAgICAgICAgICB8ICAgICAgIFN0YXR1czogIGNs b3NlZA0KICBQcmlvcml0eTogIG5vcm1hbCAgICAgICAgICAgIHwgICAgTWlsZXN0b25lOiAgICAg ICAgDQogQ29tcG9uZW50OiAgcHJvdG9jb2wgICAgICAgICAgfCAgICAgIFZlcnNpb246ICAxLjIu NyANClJlc29sdXRpb246ICBmaXhlZCAgICAgICAgICAgICB8ICAgICBLZXl3b3JkczogICAgICAg IA0KICAgIEJsb2NrczogICAgICAgICAgICAgICAgICAgIHwgICBCbG9ja2VkIEJ5OiAgICAgICAg DQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCkNoYW5nZXMgKGJ5IGJhZ2Rlcik6CgogICogc3RhdHVzOiAg YXNzaWduZWQgPT4gY2xvc2VkDQogICogcmVzb2x1dGlvbjogID0+IGZpeGVkDQoKCkNvbW1lbnQ6 CgogbWFueSB0aGFua3MsIGNvbW1pdHRlZCBhcyA4OTc0ZGMwNWFlZGYgYW5kIDc2MDNjMGY4NTRm DQoNCiBDbG9zaW5nIHRoaXMgcmVwb3J0IG5vdyEKCi0tIApUaWNrZXQgVVJMOiA8aHR0cDovL3Ry YWMubGlic3NoMi5vcmcvdGlja2V0LzE4NCNjb21tZW50OjQ+CmxpYnNzaDIgPGh0dHA6Ly90cmFj LmxpYnNzaDIub3JnLz4KQyBsaWJyYXJ5IGZvciB3cml0aW5nIHBvcnRhYmxlIFNTSDIgY2xpZW50 cwpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaWJzc2gy LWRldmVsIGh0dHA6Ly9jb29sLmhheHguc2UvY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL2xpYnNz aDItZGV2ZWwK From libssh2-devel-bounces@cool.haxx.se Wed Nov 3 15:24:54 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3EOlXs010188; Wed, 3 Nov 2010 15:24:53 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3EOjoG010157 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 3 Nov 2010 15:24:45 +0100 Received: (qmail 18573 invoked from network); 3 Nov 2010 14:24:42 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 3 Nov 2010 14:24:42 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 03 Nov 2010 14:24:42 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #170: Test on Solaris 10 amd64 hangs X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/170#comment:8 Message-ID: <123.27107f31d1e268a477824b8096c2e7e1@libssh2.stuge.se> References: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Trac-Ticket-ID: 170 In-Reply-To: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 03 Nov 2010 15:24:54 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 03 Nov 2010 15:24:45 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #170: Test on Solaris 10 amd64 hangs ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=AItOawmzyrEUkUDbHXFBqjFbBoB5p8Ik5X9sR1A | Owner: stuge Type: defect | Status: assigned Priority: normal | Milestone: Component: tests | Version: 1.2.7 Resolution: | Keywords: select, timeout, hang Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Comment (by bagder): I was thinking of enabling the libssh2 debug trace, like if you have built libssh2 with debug enable and then you apply this patch before you run the test: diff --git a/tests/ssh2.c b/tests/ssh2.c index e09ba8a..df70eaa 100644 --- a/tests/ssh2.c +++ b/tests/ssh2.c @@ -86,6 +86,8 @@ int main(int argc, char *argv[]) return -1; } + libssh2_trace(session, ~0); + /* At this point we havn't authenticated, * The first thing to do is check the hostkey's fingerprint against our kno * Your app may have it hard coded, may go to a file, may present it to the -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 3 16:08:58 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3F8ihn010133; Wed, 3 Nov 2010 16:08:57 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3F8epG010057 for ; Wed, 3 Nov 2010 16:08:42 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=dWSvXFHD5CwR9o0LNQDUII8DA2R0Ycndxqp6abnFgfOhdMOZk+249Ek52XsoCT9ZRi 6AQnrYfxEHN2cjElevKJ//tbiZPDIG/egTYcaP2pW8Pc/zXCTrBQtLRstSTQNlySfn2/ OdB7wHrjN4OIL5k3Z0uHyQScXo8wLJiSdItkM=; bh=kwAx3Tbx7/5ow0k9NyRWgI8VD4w=; t=1288796916; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 5080000360b81bec; Wed, 3 Nov 2010 11:08:35 -0400 From: "Mark Riordan" To: Subject: libssh2_sftp_write now hangs eventually Date: Wed, 3 Nov 2010 10:08:31 -0500 Message-ID: <001c01cb7b68$fa997f90$efcc7eb0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act7aPn5Ql7WXG0eS6OzoDA91HSpQw== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 03 Nov 2010 16:08:58 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 03 Nov 2010 16:08:42 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > I believe the problem was that the code didn't properly deal > with the 32bit length field (that starts each SFTP packet) > when the whole field could not be read at once. > > I've pushed a fix for this, please try again and if it still > fails I'm interested in the return code and error message again! Progress, but now it hangs. I've tried both to Solaris x86 and Ubuntu 10.04 x86, with similar results. Eventually the write hangs, usually after only a few writes. (As you know, this is with the client running on Windows 2003R2.) Here's typical output uploading to Ubuntu OpenSSH: Not enabling compression. Fingerprint: 8C D2 47 AC A6 9C 56 DA 60 7F A4 45 1A 4C 77 13 F1 B1 2C 0C Authentication methods: publickey,password libssh2_sftp_init()! libssh2_sftp_open()! libssh2_sftp_open() succeeded. Using output buffer size of 32500 bytes libssh2_sftp_write 1 sent 12000 bytes libssh2_sftp_write 2 sent 4000 bytes libssh2_sftp_write 3 sent 4000 bytes libssh2_sftp_write 4 sent 4000 bytes libssh2_sftp_write 5 sent 4000 bytes libssh2_sftp_write 6 sent 4000 bytes libssh2_sftp_write 7 sent 500 bytes libssh2_sftp_write 8 sent 4000 bytes libssh2_sftp_write 9 sent 8000 bytes libssh2_sftp_write 10 sent 4000 bytes (then it hangs) Sometimes it hangs after write #2. Naturally, if I enable tracing, the upload completes. Thus, my 700+ MB trace logs aren't useful, so I'm not providing any. Here's the tweaked upload code: hand = open(settings.localfile.c_str(), O_BINARY | O_RDONLY, _S_IREAD); if (-1 == hand) { perror("opening local file"); exit(2); } int nwrites = 0; char mem[32500]; printf("Using output buffer size of %d bytes\n", sizeof(mem)); do { /* loop until we fail */ int nbytes = read(hand, mem, sizeof(mem)); if (nbytes > 0) { filesize += nbytes; int offset = 0, bytes_to_send = nbytes; do { rc = libssh2_sftp_write(sftp_handle, mem + offset, bytes_to_send); if (rc < 0) { printf("libssh2_sftp_write failed with code %d\n", rc); char *errmsg=NULL; int errmsg_len=0; rc = libssh2_session_last_error(session, &errmsg, &errmsg_len, 0); printf("libssh2_session_last_error() returned %s\n", errmsg); break; } ++nwrites; if (0 == (nwrites % 100) || settings.debug > 1) { printf("libssh2_sftp_write %d sent %d bytes\n", nwrites, rc); } offset += rc; bytes_to_send -= rc; } while (bytes_to_send > 0); } else { break; } } while (rc >= 0); Thanks, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 3 16:47:21 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3FlAqM005961; Wed, 3 Nov 2010 16:47:20 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3Fl8aO005943 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 3 Nov 2010 16:47:09 +0100 Received: (qmail 24015 invoked from network); 3 Nov 2010 15:47:05 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 3 Nov 2010 15:47:05 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 03 Nov 2010 15:47:05 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #170: Test on Solaris 10 amd64 hangs X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/170#comment:9 Message-ID: <123.d7f392c5f674027125f7532b974b652f@libssh2.stuge.se> References: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Trac-Ticket-ID: 170 In-Reply-To: <108.5c5583c6ac8a532e4766fcbba259f0bc@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 03 Nov 2010 16:47:21 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 03 Nov 2010 16:47:09 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #170: Test on Solaris 10 amd64 hangs ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=AItOawmzyrEUkUDbHXFBqjFbBoB5p8Ik5X9sR1A | Owner: stuge Type: defect | Status: assigned Priority: normal | Milestone: Component: tests | Version: 1.2.7 Resolution: | Keywords: select, timeout, hang Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Comment (by www.google.com/accounts/o8/id?id=aitoawnwnbfr9defttu_ljqvasu3w7up5yxizmc): The full debug log is available at http://buildfarm.opencsw.org/~dam/libssh2-amd64-debug.log -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 3 22:34:10 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3LXmV3014555; Wed, 3 Nov 2010 22:34:06 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA3LXmKo014536 for ; Wed, 3 Nov 2010 22:33:48 +0100 Date: Wed, 3 Nov 2010 22:33:48 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write now hangs eventually In-Reply-To: <001c01cb7b68$fa997f90$efcc7eb0$@com> Message-ID: References: <001c01cb7b68$fa997f90$efcc7eb0$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 03 Nov 2010 22:34:10 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 03 Nov 2010 22:33:48 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Wed, 3 Nov 2010, Mark Riordan wrote: > Progress, but now it hangs. Hangs!? Are you using the libssh2 API set to blocking or non-blocking? > (As you know, this is with the client running on Windows 2003R2.) Uhm, just to be sure: that's not a 64 bit OS, is it? > Using output buffer size of 32500 bytes > libssh2_sftp_write 1 sent 12000 bytes > libssh2_sftp_write 2 sent 4000 bytes > libssh2_sftp_write 3 sent 4000 bytes > libssh2_sftp_write 4 sent 4000 bytes > libssh2_sftp_write 5 sent 4000 bytes > libssh2_sftp_write 6 sent 4000 bytes > libssh2_sftp_write 7 sent 500 bytes > libssh2_sftp_write 8 sent 4000 bytes > libssh2_sftp_write 9 sent 8000 bytes > libssh2_sftp_write 10 sent 4000 bytes > (then it hangs) > > Sometimes it hangs after write #2. When it hangs, can you figure out where it is in the code and what it does? Like attaching a debugger to the running process or possibly run it from a debugger to start with? Or perhaps you have something strace/ltrace-like that can provide further data on what it does when it hangs or even immediately before. > Naturally, if I enable tracing, the upload completes. Yes. It could be an idea to make sure the debugdump() in src/transport.c isn't used to see if you can get the hang with the rest of the debug outputs left. debugdump() is what produces the most output. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 4 17:30:24 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4GTr5X017271; Thu, 4 Nov 2010 17:30:19 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4GTobT017234 for ; Thu, 4 Nov 2010 17:29:52 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=VdB1gqwl0ozrKvz0BarovzwL+TBKLWY84rwbOMrNhAnSCwJbp7b46SpILSrmUk2Wyi h1vMf1Ulewc84pXIMs3FgL4JJ/alfGip4gz20Qw8Thfw4G7mrhGkZ75THoEJatTPJqY7 w3O6pgDokQEugjZftlh+zjvFfDJvwYJgnJsvg=; bh=MdPjiWpJpipLa+RIZFOtEMIGGrE=; t=1288888203; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id a28e0005a9273438; Thu, 4 Nov 2010 12:30:01 -0400 From: "Mark Riordan" To: Subject: Re: libssh2_sftp_write now hangs eventually Date: Thu, 4 Nov 2010 11:29:40 -0500 Message-ID: <000c01cb7c3d$7b171270$71453750$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act8PXqT9sZIPPyhQ3aM1mj85zrueQ== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 04 Nov 2010 17:30:24 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 04 Nov 2010 17:29:52 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > Uhm, just to be sure: that's not a 64 bit OS, is it? Nope, 32-bit. > When it hangs, can you figure out where it is in the code and what it does? > Like attaching a debugger to the running process or possibly > run it from a debugger to start with? When it's hung, I break into the process via the debugger and get: "The process appears to be deadlocked (or is not running any user-mode code). All threads have been stopped." Here's the stack trace: ntdll.dll!7c82860c() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] ntdll.dll!7c827d29() mswsock.dll!71b21af5() mswsock.dll!71b21a03() > sftpmrr.exe!libssh2_keepalive_send(_LIBSSH2_SESSION * session=0x00000000, int * seconds_to_next=0x7ffd4000) Line 98 + 0x1e bytes C Line 98 is the last line of the file - a little odd. This may be all I have time for today. Maybe more another day. Thanks. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 4 18:05:33 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4H5NFM010543; Thu, 4 Nov 2010 18:05:32 +0100 Received: from web29717.mail.ird.yahoo.com (web29717.mail.ird.yahoo.com [77.238.189.184]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oA4H5MpX010515 for ; Thu, 4 Nov 2010 18:05:22 +0100 Received: (qmail 92231 invoked by uid 60001); 4 Nov 2010 17:05:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1288890314; bh=pHXMlhqdeGtti8oxk7vhbD+VT1xE06QjrbwjKMaje8E=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=V1XTYplEmI1PeS8UJFdP59uamDEgRXjJxmtBriKUQwYhO7+pcEa0ZxLLlY8IVFuc/0rtkRxy2CC5KK0PYf6POH6XhiaARQ/X/DenWFwo7R9rB5lEWu5BELPX4+bwNbVYOjqLhNPexUz6o29/2YjU2AN2pHLlJnFZq4EnhWTYkkI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=SArly45diRUg7y8r40OUFHRgxZ4SgoOHxSRUXt8OXKUVylVmkuUcGFvjT2B7VR4HyNfECMQmChfIFawkagZ+hMC5S/081Lz6nZTr0gBLEQNQR+rdhEcD91msYjkxhItMcVCYtEBI6JQxPZFIo1I8WywnNPh0rFrw/QPFUDYM1qw=; Message-ID: <620111.91773.qm@web29717.mail.ird.yahoo.com> X-YMail-OSG: Seb0PFIVM1m.KUDpuEOV.JOvRdNv2vXNh5rWOMW7u0dR_tp urlAItRmqgcPkssxxcN7hMg.Mvk12twtZJLhssQNIs9Ib.a4QqFSNBJTT8lT rbf_fNWDVqOHA3xrhnQY8t_wZcfTAOQmsYrkLRsyLzqNKWQRAwDlV4KKEWIU .Z1_CyJN6qwwRUJrS9aTI43kML0oBApjdbZ2WXnxLO0wTvGFmBEANFPmQAJk PADEKuKc3iFIkzPDnt14xM8yUZkn6gt03avyKbnb4stU0fmNCJupkgIZOWB0 - Received: from [87.189.241.8] by web29717.mail.ird.yahoo.com via HTTP; Thu, 04 Nov 2010 17:05:14 GMT X-Mailer: YahooMailRC/504.5 YahooMailWebService/0.8.107.284920 References: Date: Thu, 4 Nov 2010 17:05:14 +0000 (GMT) From: may nothing Subject: AW: SFTP upload compared? To: libssh2 development In-Reply-To: MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 04 Nov 2010 18:05:33 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 04 Nov 2010 18:05:22 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se >Hi friends, > >If there are anyone out there who does SFTP uploads with libssh2, I would >appreciate if you took the current git version for a spin and compared it with >the former version(s) and see >how it behaves, both functionailty wise and speed >wise. And then tell us about it! (Preferably then tests that are against servers >that are bit remote, with larger RTTs.) > >If we can just establish that this (generally) works and is an improvement, then >we can proceed and apply the same concept on SFTP downloads. I just want to wait >until the upload >part has been polished and proven a bit. Hi Daniel, does the daily snapshot include the improvements ? I just downloaded the libssh2-1.2.8-20101104 and did a download test without any speed gain. cu may _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 4 19:14:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4IEV4b023214; Thu, 4 Nov 2010 19:14:42 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4IEUOt023208 for ; Thu, 4 Nov 2010 19:14:30 +0100 Date: Thu, 4 Nov 2010 19:14:30 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: AW: SFTP upload compared? In-Reply-To: <620111.91773.qm@web29717.mail.ird.yahoo.com> Message-ID: References: <620111.91773.qm@web29717.mail.ird.yahoo.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 04 Nov 2010 19:14:44 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 04 Nov 2010 19:14:30 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Thu, 4 Nov 2010, may nothing wrote: >> If there are anyone out there who does SFTP uploads with libssh2 > > does the daily snapshot include the improvements ? Yes it does. > I just downloaded the libssh2-1.2.8-20101104 and did a download test without > any speed gain. Exactly, which is why I asked about an *upload* test... :-) -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 00:40:06 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA4NdJh5003383; Fri, 5 Nov 2010 00:40:00 +0100 Received: from nm3-vm0.bullet.mail.sp2.yahoo.com (nm3-vm0.bullet.mail.sp2.yahoo.com [98.139.90.230]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oA4NdG5O003371 for ; Fri, 5 Nov 2010 00:39:17 +0100 Received: from [98.139.91.62] by nm3.bullet.mail.sp2.yahoo.com with NNFMP; 04 Nov 2010 23:39:07 -0000 Received: from [98.139.91.58] by tm2.bullet.mail.sp2.yahoo.com with NNFMP; 04 Nov 2010 23:39:07 -0000 Received: from [127.0.0.1] by omp1058.mail.sp2.yahoo.com with NNFMP; 04 Nov 2010 23:39:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 198605.4527.bm@omp1058.mail.sp2.yahoo.com Received: (qmail 70111 invoked by uid 60001); 4 Nov 2010 23:39:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1288913946; bh=XT1I+WAoIea7nOE7Qs/XlJ3Wwec9K572a8BcY7FuwQA=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=TZku23wmOPOYMyBj/NpPkmcXBbrkZSIVW/o+JTeTHwqqvEdXmjWI3nGgyyltewqk8cIHYFW7jBb6reCsp1FJ/TZNRR7D9BZDpNaJdaHHtmzZI2DCFhVfgGf4oa3gD+oTAaetZ34+/ZdNUEcnLmY0vvwwdydAKF22BgfZIEl4Nu8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=GjQFGwZHGeQwiHM2W6Fz8kXGp/FDQjEwLo/ek1i+8i4lzGFZCmSD1Oibr21X3wlk1CQ+TDCnxpnknCZgFm6V3sODqoZZES6A4VB4kPvHuDVVfdLEy5I8O24bbfUSchDwhT9mJZKwM39abd+nhI505+SbYFm6f5NC5PMOmgDXsMs=; Message-ID: <899419.69857.qm@web45307.mail.sp1.yahoo.com> X-YMail-OSG: 3xNvySwVM1mrVdzIXnDeac4_KOuq8MMgohY1Ba_N5bt8XJo 9YsR40lgbqFdrrxXLnepwmHFmsgsW.U8w_CTDVE8Lm2HjBq0kq5YxMWqf0oT 6lKy8z02x33PmMd28.i.UUphI98SI1A3q3mFkKDP6K_bAOOl1MgruyDGyDAv nl150RRhi.FoHMTOCEgHBUJHtQ0gZ2mD9HeqriCylij6oflci54CwBGkKUNX gsAU_5vUKVauKmK8wUqEyE7Zlfb1M4yB9QqrNh5w- Received: from [64.201.246.50] by web45307.mail.sp1.yahoo.com via HTTP; Thu, 04 Nov 2010 16:39:06 PDT X-Mailer: YahooMailClassic/11.4.9 YahooMailWebService/0.8.107.284920 Date: Thu, 4 Nov 2010 16:39:06 -0700 (PDT) From: Thilee Subject: Using libssh2_userauth_publickey_fromfile() when I don't have a public key To: libssh2-devel@cool.haxx.se MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 00:40:06 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 00:39:18 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se One doesn't really need to use a public key for auth, right? In my case, I have only the private key, which technically is sufficient to authenticate me (ssh -i pvtkeyfile ruser@rhost works). However, when I use this library call I don't have a public-key file to supply it with. Giving NULL for pubkeyfile fails with -16 (LIBSSH2_ERROR_FILE). libssh2_userauth_publickey_fromfile_ex(session, username, pubkeyfile, pvtkeyfile, passphrase) Any help? Thanks! _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 10:12:25 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59BjrV025480; Fri, 5 Nov 2010 10:12:22 +0100 Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59BheU025474 for ; Fri, 5 Nov 2010 10:11:44 +0100 Received: by eyb7 with SMTP id 7so1604397eyb.41 for ; Fri, 05 Nov 2010 02:11:42 -0700 (PDT) Received: by 10.14.47.143 with SMTP id t15mr616649eeb.47.1288948302149; Fri, 05 Nov 2010 02:11:42 -0700 (PDT) Received: from [192.168.0.35] (96-160.ftth.onsbrabantnet.nl [88.159.160.96]) by mx.google.com with ESMTPS id v56sm828099eeh.14.2010.11.05.02.11.40 (version=SSLv3 cipher=RC4-MD5); Fri, 05 Nov 2010 02:11:40 -0700 (PDT) Message-ID: <4CD3CA48.4060403@peut.org> Date: Fri, 05 Nov 2010 10:11:36 +0100 From: Jose Baars User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't have a public key References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> In-Reply-To: <899419.69857.qm@web45307.mail.sp1.yahoo.com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 10:12:25 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 10:11:44 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1378543381==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se This is a cryptographically signed message in MIME format. --===============1378543381== Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms080406070207070202010509" This is a cryptographically signed message in MIME format. --------------ms080406070207070202010509 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Op 11/5/2010 12:39 AM, Thilee schreef: > One doesn't really need to use a public key for auth, right? > In my case, I have only the private key, which technically is sufficien= t to authenticate me (ssh -i pvtkeyfile ruser@rhost works). I've been looking at that, too. The public key can be derived=20 (extracted) from the private key, but that requires logic to do so. That logic is not available in=20 libssh2, maybe because it is not SSH, but crypto functionality. As a workaround, some flavours of ssh-keygen or puttygen can extract the = public key from the private key and put it in a public key file to be used with libssh2. I make do with this workaround for the time being, until me, you or=20 someone else adds this functionality to libssh2. Jose --------------ms080406070207070202010509 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIPWjCC BMwwggQ1oAMCAQICEByunWua9OYvIoqj2nRhbB4wDQYJKoZIhvcNAQEFBQAwXzELMAkGA1UE BhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAxIFB1Ymxp YyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA1MTAyODAwMDAwMFoXDTE1 MTAyNzIzNTk1OVowgd0xCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEf MB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNl IGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEgKGMpMDUxHjAcBgNVBAsTFVBlcnNv bmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVyaVNpZ24gQ2xhc3MgMSBJbmRpdmlkdWFs IFN1YnNjcmliZXIgQ0EgLSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMnf rOfq+PgDFMQAktXBfjbCPO98chXLwKuMPRyVzm8eECw/AO2XJua2x+atQx0/pIdHR0w+VPhs +Mf8sZ69MHC8l7EDBeqV8a1AxUR6SwWi8mD81zplYu//EHuiVrvFTnAt1qIfPO2wQuhejVch rKaZ2RHp0hoHwHRHQgv8xTTq/ea6JNEdCBU3otdzzwFBL2OyOj++pRpu9MlKWz2VphW7NQIZ +dTvvI8OcXZZu0u2Ptb8Whb01g6J8kn+bAztFenZiHWcec5gJ925rXXOL3OVekA6hXVJsLjf aLyrzROChRFQo+A8C67AClPN1zBvhTJGG+RJEMJs4q8fef/btLUCAwEAAaOCAYQwggGAMBIG A1UdEwEB/wQIMAYBAf8CAQAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXATAqMCgGCCsGAQUF BwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIBBjARBglghkgB hvhCAQEEBAMCAQYwLgYDVR0RBCcwJaQjMCExHzAdBgNVBAMTFlByaXZhdGVMYWJlbDMtMjA0 OC0xNTUwHQYDVR0OBBYEFBF9Xhl9PATfamzWoooaPzHYO5RSMDEGA1UdHwQqMCgwJqAkoCKG IGh0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTEuY3JsMIGBBgNVHSMEejB4oWOkYTBfMQsw CQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsTLkNsYXNzIDEg UHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHmCEQDNun9W8N/kvFT+Iqyz cqpVMA0GCSqGSIb3DQEBBQUAA4GBALEv2ZbhkqLugWDlyCog++FnLNYAmFOjAhvpkEv4GESf D0b3+qD+0x0Yo9K/HOzWGZ9KTUP4yru+E4BJBd0hczNXwkJavvoAk7LmBDGRTl088HMFN2Pr v4NZmP1m3umGMpqSKTw6rlTaphJRsY/IytNHeObbpR6HBuPRFMDCIfa6MIIFQTCCBCmgAwIB AgIQQkuBnab/mKbwbYynoi4GkzANBgkqhkiG9w0BAQUFADCB3TELMAkGA1UEBhMCVVMxFzAV BgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3Jr MTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3Jw YSAoYykwNTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJp U2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEcyMB4XDTA5MTExMjAw MDAwMFoXDTEwMTExMjIzNTk1OVowggEKMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0G A1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazFGMEQGA1UECxM9d3d3LnZlcmlzaWduLmNv bS9yZXBvc2l0b3J5L1JQQSBJbmNvcnAuIGJ5IFJlZi4sTElBQi5MVEQoYyk5ODEeMBwGA1UE CxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTMwMQYDVQQLEypEaWdpdGFsIElEIENsYXNzIDEg LSBOZXRzY2FwZSBGdWxsIFNlcnZpY2UxEzARBgNVBAMUCkpvc2UgQmFhcnMxHDAaBgkqhkiG 9w0BCQEWDXBldXRAcGV1dC5vcmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCi x8yI3kFLmfYElzZSqsup782ajeuv3TASyn8mUfGrnr2aW1M7jN4vlM8/7/sVL883rH+kWPzV Q4ITYrfW5QFxH+CBqtX0F1xsS1A7AR8xZIs8o7/sMxV2xOSXYEJXlKP4XVWY8oU+TKH5dWYZ 9BJ9pkJ840ayiqqI/P3MHqmu7vS2nPDu8YZFa+46EtWkGbJTjzFdnrU5Jmdudp37fd1btNss RIkqztDOeQOTaZ6pS9RqjhmNuWVtgZucKho6c9Qf2htGz8AL95MqttF2OZYL9AlXZhjJcIMO aZBVjuwLowX/kYL2lo1vyUGt3eZxSkZR/ICfkQwj1BKpFZGFtM5XAgMBAAGjgcwwgckwCQYD VR0TBAIwADBEBgNVHSAEPTA7MDkGC2CGSAGG+EUBBxcBMCowKAYIKwYBBQUHAgEWHGh0dHBz Oi8vd3d3LnZlcmlzaWduLmNvbS9ycGEwCwYDVR0PBAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUF BwMEBggrBgEFBQcDAjBKBgNVHR8EQzBBMD+gPaA7hjlodHRwOi8vSW5kQzFEaWdpdGFsSUQt Y3JsLnZlcmlzaWduLmNvbS9JbmRDMURpZ2l0YWxJRC5jcmwwDQYJKoZIhvcNAQEFBQADggEB ABcuOD4QwMrXJgD8zvCWBRjBqsQLXn0ybqrLZHlcic7+lstAl06fi7fGD3+1Nrcp4VImkyUv JQEvFSvkB+/Mt8kOVSdt+eiYIInKt/lBkSzy0sardsh0u7Ifcgt+nmLZYW+azviXnD4VNMqe Q9QIZPHSUhWDe3IA5NSYE4hfqwXX5c13HllqpUcXRmWIGA9MPr9SQxZ0jg7MFUsZVgG9f5Ts A3yIEmvUgutyX6GRzZQL+wlWymRzh/XA7j2QnYk+IPlNQHJ62MHHlAI48y3ssSXwR8hfbH4e 0sw+8ezuh785OuFMBI3qhxlQX4wT8+v8rKHtiLcddoIohWM0ID+IdOEwggVBMIIEKaADAgEC AhBCS4Gdpv+YpvBtjKeiLgaTMA0GCSqGSIb3DQEBBQUAMIHdMQswCQYDVQQGEwJVUzEXMBUG A1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsx OzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBh IChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlT aWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzIwHhcNMDkxMTEyMDAw MDAwWhcNMTAxMTEyMjM1OTU5WjCCAQoxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29t L3JlcG9zaXRvcnkvUlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4wHAYDVQQL ExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxMzAxBgNVBAsTKkRpZ2l0YWwgSUQgQ2xhc3MgMSAt IE5ldHNjYXBlIEZ1bGwgU2VydmljZTETMBEGA1UEAxQKSm9zZSBCYWFyczEcMBoGCSqGSIb3 DQEJARYNcGV1dEBwZXV0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKLH zIjeQUuZ9gSXNlKqy6nvzZqN66/dMBLKfyZR8auevZpbUzuM3i+Uzz/v+xUvzzesf6RY/NVD ghNit9blAXEf4IGq1fQXXGxLUDsBHzFkizyjv+wzFXbE5JdgQleUo/hdVZjyhT5Mofl1Zhn0 En2mQnzjRrKKqoj8/cweqa7u9Lac8O7xhkVr7joS1aQZslOPMV2etTkmZ252nft93Vu02yxE iSrO0M55A5NpnqlL1GqOGY25ZW2Bm5wqGjpz1B/aG0bPwAv3kyq20XY5lgv0CVdmGMlwgw5p kFWO7AujBf+RgvaWjW/JQa3d5nFKRlH8gJ+RDCPUEqkVkYW0zlcCAwEAAaOBzDCByTAJBgNV HRMEAjAAMEQGA1UdIAQ9MDswOQYLYIZIAYb4RQEHFwEwKjAoBggrBgEFBQcCARYcaHR0cHM6 Ly93d3cudmVyaXNpZ24uY29tL3JwYTALBgNVHQ8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUH AwQGCCsGAQUFBwMCMEoGA1UdHwRDMEEwP6A9oDuGOWh0dHA6Ly9JbmRDMURpZ2l0YWxJRC1j cmwudmVyaXNpZ24uY29tL0luZEMxRGlnaXRhbElELmNybDANBgkqhkiG9w0BAQUFAAOCAQEA Fy44PhDAytcmAPzO8JYFGMGqxAtefTJuqstkeVyJzv6Wy0CXTp+Lt8YPf7U2tynhUiaTJS8l AS8VK+QH78y3yQ5VJ2356Jggicq3+UGRLPLSxqt2yHS7sh9yC36eYtlhb5rO+JecPhU0yp5D 1Ahk8dJSFYN7cgDk1JgTiF+rBdflzXceWWqlRxdGZYgYD0w+v1JDFnSODswVSxlWAb1/lOwD fIgSa9SC63JfoZHNlAv7CVbKZHOH9cDuPZCdiT4g+U1AcnrYwceUAjjzLeyxJfBHyF9sfh7S zD7x7O6Hvzk64UwEjeqHGVBfjBPz6/ysoe2Itx12giiFYzQgP4h04TGCBOwwggToAgEBMIHy MIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZl cmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczov L3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxp ZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVy IENBIC0gRzICEEJLgZ2m/5im8G2Mp6IuBpMwCQYFKw4DAhoFAKCCAs4wGAYJKoZIhvcNAQkD MQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTAxMTA1MDkxMTM2WjAjBgkqhkiG9w0B CQQxFgQUA6yZTEDYFqcboM3bgBPF3lIdOEowXwYJKoZIhvcNAQkPMVIwUDALBglghkgBZQME AQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIH MA0GCCqGSIb3DQMCAgEoMIIBAwYJKwYBBAGCNxAEMYH1MIHyMIHdMQswCQYDVQQGEwJVUzEX MBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdv cmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20v cnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZl cmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzICEEJLgZ2m/5im 8G2Mp6IuBpMwggEFBgsqhkiG9w0BCRACCzGB9aCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNV BAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTsw OQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAo YykwNTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2ln biBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEcyAhBCS4Gdpv+YpvBtjKei LgaTMA0GCSqGSIb3DQEBAQUABIIBAFJaWsiQXVrS0l37e+tdp7OEaRh3N8ej6/HnbzRG1Cmx ji8XpTicrGhu/97H8LHcKdiIWAL1K9ePYh3C6zqPXc4uUkXxV6XMPUEu06O3WAcullZAqE3S hogpST5Rlx+AprMlTDI84WANATC5l/XdfY+/zpfCbbeLjF82P8MgyZgMH3ZFB48OQ3zI+8oI SVSwQ30JNww5Z6p1V9feu9qLm0Zc43pNCrpYxnLMdNIIgeheapw/BSJ2rdUSN7h6S31CYfjE gIlNOrgSoQxBHdriETNVcVwtW8w6579xcjdbrr5JpJOyDYnRRxiWphncO9ZRVyq7Enyzorrf yQIdZpAXsT0AAAAAAAA= --------------ms080406070207070202010509-- --===============1378543381== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============1378543381==-- From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 10:24:05 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59NwK1004379; Fri, 5 Nov 2010 10:24:05 +0100 Received: from mail0.avcosystems.co.uk (mail0.avcosystems.co.uk [195.224.236.86]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59NvMO004262 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 5 Nov 2010 10:23:57 +0100 Received: from lexx.avco ([192.168.0.1] helo=lyta) by mail0.avcosystems.co.uk with esmtp (Exim 4.69 #1 (Debian)) (envelope-from ) id 1PEIWU-0003sM-2y for ; Fri, 05 Nov 2010 09:23:50 +0000 From: "Mark Smith" To: "'libssh2 development'" References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> Subject: RE: Using libssh2_userauth_publickey_fromfile() when I don't havea public key Date: Fri, 5 Nov 2010 09:23:50 -0000 Message-ID: <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_03AD_01CB7CCB.2852EE40" X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <4CD3CA48.4060403@peut.org> Thread-Index: Act8yZYt9QYi4aqZTQKjbJqM7Pv2HgAAT0XA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 X-AVCO-Scan-Signature: a6ec681311ab39c1307876297600bce0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 10:24:05 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 10:23:57 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se This is a multi-part message in MIME format. ------=_NextPart_000_03AD_01CB7CCB.2852EE40 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Jose wrote: > I've been looking at that, too. The public key can be derived > (extracted) from the private key, > but that requires logic to do so. That logic is not available in > libssh2, maybe because > it is not SSH, but crypto functionality. This is the (openssl) patch I use, last applied against 1.2.6, supplied by another user some time ago. It's certainly not in a state to commit, but what I can say is that it seems to work for me. I never found a gcrypt equivalent. Regards, -- Mark Smith ------=_NextPart_000_03AD_01CB7CCB.2852EE40 Content-Type: application/octet-stream; name="libssh2-1.2.6-pubkey.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="libssh2-1.2.6-pubkey.diff" diff -aru libssh2-1.2.6/src/userauth.c libssh2-1.2.6-new/src/userauth.c=0A= --- libssh2-1.2.6/src/userauth.c Sat Mar 28 22:29:45 2009=0A= +++ libssh2-1.2.6-new/src/userauth.c Mon Jun 08 12:45:42 2009=0A= @@ -536,7 +536,281 @@=0A= return 0;=0A= }=0A= =0A= +static unsigned char* write_bn(unsigned char* buf,=0A= + const BIGNUM* bn,=0A= + int bn_bytes)=0A= +{=0A= + unsigned char* p =3D buf;=0A= +=0A= + /* Left space for bn size which will be written below. */=0A= + p +=3D 4;=0A= +=0A= + *p =3D 0;=0A= + BN_bn2bin(bn, p + 1);=0A= + if (!(*(p + 1) & 0x80)) {=0A= + memmove(p, p + 1, --bn_bytes);=0A= + }=0A= + _libssh2_htonu32(p - 4, bn_bytes); /* Post write bn size. */=0A= +=0A= + return p + bn_bytes;=0A= +}=0A= +=0A= +=0A= +static unsigned char* gen_publickey_from_rsa(=0A= + LIBSSH2_SESSION* session,=0A= + RSA* rsa,=0A= + unsigned long* key_len)=0A= +{=0A= + int e_bytes, n_bytes;=0A= + unsigned long len;=0A= + unsigned char* key;=0A= + unsigned char* p;=0A= +=0A= + e_bytes =3D BN_num_bytes(rsa->e) + 1;=0A= + n_bytes =3D BN_num_bytes(rsa->n) + 1;=0A= +=0A= + /* Key form is "ssh-rsa" + e + n. */=0A= + len =3D 4 + 7 + 4 + e_bytes + 4 + n_bytes;=0A= +=0A= + key =3D LIBSSH2_ALLOC(session, len);=0A= + if (key =3D=3D NULL) {=0A= + return NULL;=0A= + }=0A= +=0A= + /* Process key encoding. */=0A= + p =3D key;=0A= +=0A= + _libssh2_htonu32(p, 7); /* Key type. */=0A= + p +=3D 4;=0A= + memcpy(p, "ssh-rsa", 7);=0A= + p +=3D 7;=0A= +=0A= + p =3D write_bn(p, rsa->e, e_bytes);=0A= + p =3D write_bn(p, rsa->n, n_bytes);=0A= +=0A= + *key_len =3D (unsigned long)(p - key);=0A= + return key;=0A= +}=0A= +=0A= +=0A= +static unsigned char* gen_publickey_from_dsa(=0A= + LIBSSH2_SESSION* session,=0A= + DSA* dsa,=0A= + unsigned long* key_len)=0A= +{=0A= + int p_bytes, q_bytes, g_bytes, k_bytes;=0A= + unsigned long len;=0A= + unsigned char* key;=0A= + unsigned char* p;=0A= +=0A= + p_bytes =3D BN_num_bytes(dsa->p) + 1;=0A= + q_bytes =3D BN_num_bytes(dsa->q) + 1;=0A= + g_bytes =3D BN_num_bytes(dsa->g) + 1;=0A= + k_bytes =3D BN_num_bytes(dsa->pub_key) + 1;=0A= +=0A= + /* Key form is "ssh-dss" + p + q + g + pub_key. */=0A= + len =3D 4 + 7 + 4 + p_bytes + 4 + q_bytes + 4 + g_bytes + 4 + = k_bytes;=0A= +=0A= + key =3D LIBSSH2_ALLOC(session, len);=0A= + if (key =3D=3D NULL) {=0A= + return NULL;=0A= + }=0A= +=0A= + /* Process key encoding. */=0A= + p =3D key;=0A= +=0A= + _libssh2_htonu32(p, 7); /* Key type. */=0A= + p +=3D 4;=0A= + memcpy(p, "ssh-dss", 7);=0A= + p +=3D 7;=0A= +=0A= + p =3D write_bn(p, dsa->p, p_bytes);=0A= + p =3D write_bn(p, dsa->q, q_bytes);=0A= + p =3D write_bn(p, dsa->g, g_bytes);=0A= + p =3D write_bn(p, dsa->pub_key, k_bytes);=0A= +=0A= + *key_len =3D (unsigned long)(p - key);=0A= + return key;=0A= +}=0A= +=0A= +=0A= +static int gen_publickey_from_rsa_evp(=0A= + LIBSSH2_SESSION* session,=0A= + unsigned char** method,=0A= + unsigned long* method_len,=0A= + unsigned char** pubkeydata,=0A= + unsigned long* pubkeydata_len,=0A= + EVP_PKEY* pk)=0A= +{=0A= + RSA* rsa =3D NULL;=0A= + unsigned char* key;=0A= + unsigned char* method_buf =3D NULL;=0A= + unsigned long key_len;=0A= +=0A= + _libssh2_debug(session,=0A= + LIBSSH2_TRACE_AUTH,=0A= + "Computing public key from RSA private key envelop");=0A= +=0A= + rsa =3D EVP_PKEY_get1_RSA(pk);=0A= + if (rsa =3D=3D NULL) {=0A= + /* Assume memory allocation error... what else could it be ? */=0A= + goto __alloc_error;=0A= + }=0A= +=0A= + method_buf =3D LIBSSH2_ALLOC(session, 7); /* ssh-rsa. */=0A= + if (method_buf =3D=3D NULL) {=0A= + goto __alloc_error;=0A= + }=0A= +=0A= + key =3D gen_publickey_from_rsa(session, rsa, &key_len);=0A= + if (key =3D=3D NULL) {=0A= + goto __alloc_error;=0A= + }=0A= + RSA_free(rsa);=0A= +=0A= + memcpy(method_buf, "ssh-rsa", 7);=0A= + *method =3D method_buf;=0A= + *method_len =3D 7;=0A= + *pubkeydata =3D key;=0A= + *pubkeydata_len =3D key_len;=0A= + return 0;=0A= +=0A= +__alloc_error:=0A= + if (rsa !=3D NULL) {=0A= + RSA_free(rsa);=0A= + }=0A= + if (method_buf !=3D NULL) {=0A= + LIBSSH2_FREE(session, method_buf);=0A= + }=0A= +=0A= + _libssh2_error(session,=0A= + LIBSSH2_ERROR_ALLOC,=0A= + "Unable to allocate memory for private key data");=0A= + return -1;=0A= +}=0A= +=0A= +=0A= +static int gen_publickey_from_dsa_evp(=0A= + LIBSSH2_SESSION* session,=0A= + unsigned char** method,=0A= + unsigned long* method_len,=0A= + unsigned char** pubkeydata,=0A= + unsigned long* pubkeydata_len,=0A= + EVP_PKEY* pk)=0A= +{=0A= + DSA* dsa =3D NULL;=0A= + unsigned char* key;=0A= + unsigned char* method_buf =3D NULL;=0A= + unsigned long key_len;=0A= +=0A= + _libssh2_debug(session,=0A= + LIBSSH2_TRACE_AUTH,=0A= + "Computing public key from DSA private key envelop");=0A= +=0A= + dsa =3D EVP_PKEY_get1_DSA(pk);=0A= + if (dsa =3D=3D NULL) {=0A= + /* Assume memory allocation error... what else could it be ? */=0A= + goto __alloc_error;=0A= + }=0A= +=0A= + method_buf =3D LIBSSH2_ALLOC(session, 7); /* ssh-dss. */=0A= + if (method_buf =3D=3D NULL) {=0A= + goto __alloc_error;=0A= + }=0A= +=0A= + key =3D gen_publickey_from_dsa(session, dsa, &key_len);=0A= + if (key =3D=3D NULL) {=0A= + goto __alloc_error;=0A= + }=0A= + DSA_free(dsa);=0A= +=0A= + memcpy(method_buf, "ssh-dss", 7);=0A= + *method =3D method_buf;=0A= + *method_len =3D 7;=0A= + *pubkeydata =3D key;=0A= + *pubkeydata_len =3D key_len;=0A= + return 0;=0A= +=0A= +__alloc_error:=0A= + if (dsa !=3D NULL) {=0A= + DSA_free(dsa);=0A= + }=0A= + if (method_buf !=3D NULL) {=0A= + LIBSSH2_FREE(session, method_buf);=0A= + }=0A= +=0A= + _libssh2_error(session,=0A= + LIBSSH2_ERROR_ALLOC,=0A= + "Unable to allocate memory for private key data");=0A= + return -1;=0A= +}=0A= +=0A= +static int gen_publickey_from_private_keyfile(=0A= + LIBSSH2_SESSION* session,=0A= + unsigned char** method,=0A= + unsigned long* method_len,=0A= + unsigned char** pubkeydata,=0A= + unsigned long* pubkeydata_len,=0A= + const char* privatekey,=0A= + const char* passphrase)=0A= +{=0A= + int st;=0A= + BIO* bp;=0A= + EVP_PKEY* pk;=0A= +=0A= + _libssh2_debug(session,=0A= + LIBSSH2_TRACE_AUTH,=0A= + "Computing public key from private key file: %s",=0A= + privatekey);=0A= +=0A= + bp =3D BIO_new_file(privatekey, "r");=0A= + if (bp =3D=3D NULL) {=0A= + _libssh2_error(session,=0A= + LIBSSH2_ERROR_FILE,=0A= + "Unable to open private key file");=0A= + return -1;=0A= + }=0A= + if (!EVP_get_cipherbyname("des")) {=0A= +/* If this cipher isn't loaded it's a pretty good indication that none = are.=0A= + * I have *NO DOUBT* that there's a better way to deal with this = ($#&%#$(%$#(=0A= + * Someone buy me an OpenSSL manual and I'll read up on it.=0A= + */=0A= + OpenSSL_add_all_ciphers();=0A= + }=0A= + BIO_reset(bp);=0A= + pk =3D PEM_read_bio_PrivateKey(bp, NULL, NULL, (void*)passphrase);=0A= + BIO_free(bp);=0A= +=0A= + if (pk =3D=3D NULL) {=0A= + _libssh2_error(session,=0A= + LIBSSH2_ERROR_FILE,=0A= + "Wrong passphrase or invalid/unrecognized "=0A= + "private key file format");=0A= + return -1;=0A= + }=0A= =0A= + switch (pk->type) {=0A= + case EVP_PKEY_RSA :=0A= + st =3D gen_publickey_from_rsa_evp(=0A= + session, method, method_len, pubkeydata, = pubkeydata_len, pk);=0A= + break;=0A= +=0A= + case EVP_PKEY_DSA :=0A= + st =3D gen_publickey_from_dsa_evp(=0A= + session, method, method_len, pubkeydata, = pubkeydata_len, pk);=0A= + break;=0A= +=0A= + default :=0A= + st =3D -1;=0A= + _libssh2_error(session,=0A= + LIBSSH2_ERROR_FILE,=0A= + "Unsupported private key file format");=0A= + break;=0A= + }=0A= +=0A= + EVP_PKEY_free(pk);=0A= + return st;=0A= +}=0A= =0A= /* libssh2_file_read_privatekey=0A= * Read a PEM encoded private key from an id_??? style file=0A= @@ -648,13 +922,22 @@=0A= memset(&session->userauth_host_packet_requirev_state, 0,=0A= sizeof(session->userauth_host_packet_requirev_state));=0A= =0A= - rc =3D file_read_publickey(session, = &session->userauth_host_method,=0A= - &session->userauth_host_method_len,=0A= - &pubkeydata, &pubkeydata_len,=0A= - publickey);=0A= - if(rc)=0A= - /* Note: file_read_publickey() calls _libssh2_error() */=0A= - return rc;=0A= + if (publickey !=3D NULL) {=0A= + rc =3D file_read_publickey(session, = &session->userauth_host_method,=0A= + &session->userauth_host_method_len,=0A= + &pubkeydata, &pubkeydata_len, = publickey);=0A= + if(rc)=0A= + /* Note: file_read_publickey() calls libssh2_error() */=0A= + return rc;=0A= + }=0A= + else {=0A= + /* Compute public key from private key. */=0A= + if (gen_publickey_from_private_keyfile(session, = &session->userauth_host_method,=0A= + &session->userauth_host_method_len,=0A= + &pubkeydata, &pubkeydata_len, privatekey, = passphrase))=0A= + return _libssh2_error(session, LIBSSH2_ERROR_FILE,=0A= + "Unable to extract public key from = private key file");=0A= + }=0A= =0A= /*=0A= * 48 =3D packet_type(1) + username_len(4) + servicename_len(4) = +=0A= @@ -1181,11 +1464,21 @@=0A= privkey_file.passphrase =3D passphrase;=0A= =0A= if (session->userauth_pblc_state =3D=3D libssh2_NB_state_idle) {=0A= - rc =3D file_read_publickey(session, = &session->userauth_pblc_method,=0A= - &session->userauth_pblc_method_len,=0A= - &pubkeydata, &pubkeydata_len, = publickey);=0A= - if(rc)=0A= - return rc;=0A= + if (publickey !=3D NULL) {=0A= + rc =3D file_read_publickey(session, = &session->userauth_pblc_method,=0A= + &session->userauth_pblc_method_len,=0A= + &pubkeydata, = &pubkeydata_len,publickey);=0A= + if(rc)=0A= + return rc;=0A= + }=0A= + else {=0A= + /* Compute public key from private key. */=0A= + if (gen_publickey_from_private_keyfile(session, = &session->userauth_pblc_method,=0A= + &session->userauth_pblc_method_len,=0A= + &pubkeydata, &pubkeydata_len, privatekey, = passphrase))=0A= + return _libssh2_error(session, LIBSSH2_ERROR_FILE,=0A= + "Unable to extract public key from = private key file");=0A= + }=0A= }=0A= =0A= rc =3D _libssh2_userauth_publickey(session, username, username_len,=0A= ------=_NextPart_000_03AD_01CB7CCB.2852EE40 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel ------=_NextPart_000_03AD_01CB7CCB.2852EE40-- From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 10:34:25 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59YGm1009735; Fri, 5 Nov 2010 10:34:24 +0100 Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA59YDIX009652 for ; Fri, 5 Nov 2010 10:34:14 +0100 Received: by qwg8 with SMTP id 8so2679326qwg.41 for ; Fri, 05 Nov 2010 02:34:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=pFoVuUz7cX2hHJrwI9bbPZX4TNwCUrt5qCtuzQ2mgTU=; b=n6TQYyu4n0UpVahWfU+FuiaXCvosoxyvbmqITGHG+h3NDvpkL9FDUtYrFfau5GnHMK XmOTC8GZ2v0ADhHpZrAcDPNujb1Fc6o7kTeMdDMH8qveVoxq6oLkn7K24Pu9pYRt5nPi /s+ZkWeJs7vUVAOEIhvisL+N0PJwsEomVSQrM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=bHUdIT6YU7NFicWICJZc27crRVGXixhxV7SEUGlVXT1msgck5TorIx4JLsSU7x6EBO la7lG9giytVi1no2uEMQvt4D84U0vuI5MzVJywXmCMuh9PBAysTpO5pNfLWuWADjA+ba soy4KtSinOCiTCxDCZSjp72qSXWyoKSlThCoE= MIME-Version: 1.0 Received: by 10.224.209.69 with SMTP id gf5mr833481qab.56.1288949647199; Fri, 05 Nov 2010 02:34:07 -0700 (PDT) Received: by 10.229.80.213 with HTTP; Fri, 5 Nov 2010 02:34:07 -0700 (PDT) In-Reply-To: <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> Date: Fri, 5 Nov 2010 09:34:07 +0000 X-Google-Sender-Auth: 7Y67zZXLeq7qd5ANJ2qXYVnZMSk Message-ID: Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't havea public key From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 10:34:25 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 10:34:14 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 5 November 2010 09:23, Mark Smith wrote: > Jose wrote: >> I've been looking at that, too. =A0The public key can be derived >> (extracted) from the private key, >> but that requires logic to do so. That logic is not available in >> libssh2, maybe because >> it is not SSH, but crypto functionality. > > This is the (openssl) patch I use, last applied against 1.2.6, supplied by > another user some time ago. =A0It's certainly not in a state to commit, b= ut > what I can say is that it seems to work for me. =A0I never found a gcrypt > equivalent. Just skimming it, it looks pretty good. What work does it still need? Alex _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 11:15:19 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5AF4Fa008787; Fri, 5 Nov 2010 11:15:18 +0100 Received: from mail0.avcosystems.co.uk (mail0.avcosystems.co.uk [195.224.236.86]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5AF2lN008444 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 5 Nov 2010 11:15:03 +0100 Received: from lexx.avco ([192.168.0.1] helo=lyta) by mail0.avcosystems.co.uk with esmtp (Exim 4.69 #1 (Debian)) (envelope-from ) id 1PEJJv-0004u1-Mp for ; Fri, 05 Nov 2010 10:14:55 +0000 From: "Mark Smith" To: "'libssh2 development'" References: <899419.69857.qm@web45307.mail.sp1.yahoo.com><4CD3CA48.4060403@peut.org><702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> Subject: RE: Using libssh2_userauth_publickey_fromfile() when I don't haveapublic key Date: Fri, 5 Nov 2010 10:14:55 -0000 Message-ID: <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 11 In-Reply-To: Thread-Index: Act8zKmNkULaJWQJRYuVp5d+KUdUjgAAMIiA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 X-AVCO-Scan-Signature: a84c4160732dc3c00a473efe9b2af42d X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 11:15:19 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 11:15:03 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Alex wrote: > Just skimming it, it looks pretty good. What work does it still need? I seem to remember from the discussion last time I posted a version of this patch was that it was deemed untidy; what it should do instead is move the function calls out of user.c so that they can be implemented in the crypto library interface: put variants of these functions in openssl.c and stubs in gcrypt.c (at least until someone who knows gcrypt well enough can implement equivalents!) This would of course leave gcrypt support lagging even further behind openssl support - last I checked gcrypt couldn't support password-encrypted private keys either - so finding someone with gcrypt knowledge who could at least advise would be a definite plus. I haven't been able to do any of this yet since my requirement at the time was to Just-Make-It-Work [tm] and I haven't had any time to work on the project since; all I did do was make the patch apply cleanly against 1.2 and then 1.2.6 but haven't done anything newer. I'm expecting to be able to update to 1.2.8 "sometime soon" and might be able to clean it up at the same time but I don't have any crypto knowledge and can't provide any kind of timescale. Of course, if the value of not needing the public key appeals to some other kind developer with a bit more time to finish it off, please feel free! Otherwise I certainly plan to post the cleaned patch when I've done it with the hope that it'll get committed, perhaps prompting further additions and enhancements? On a sidenote, since I didn't post this as part of 2WoO: I use SFTP upload both from Win32 and linux for log backup purposes as part of a PCI solution, including realtime extraction of data from IIS, EventLog and MSSQL trace on Win32, although the latter still needs some work. Regards, -- Mark Smith _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 14:11:22 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DAxgc017275; Fri, 5 Nov 2010 14:11:18 +0100 Received: from yxa-v.extundo.com (yxa-v.extundo.com [83.241.177.39]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DAwJH017262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 5 Nov 2010 14:10:58 +0100 Received: from latte.josefsson.org ([213.115.164.118]) (authenticated bits=0) by yxa-v.extundo.com (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id oA5DAotu030597 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Fri, 5 Nov 2010 14:10:53 +0100 From: Simon Josefsson To: libssh2 development Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't haveapublic key References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> OpenPGP: id=B565716F; url=http://josefsson.org/key.txt X-Hashcash: 1:22:101105:libssh2-devel@cool.haxx.se::mGE+yuFtFDeSWujc:IXLM Date: Fri, 05 Nov 2010 14:10:37 +0100 In-Reply-To: <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> (Mark Smith's message of "Fri, 5 Nov 2010 10:14:55 -0000") Message-ID: <87r5ezzzjm.fsf@latte.josefsson.org> User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux) MIME-Version: 1.0 X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_XX, RDNS_NONE,SPF_FAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on yxa-v.extundo.com X-Virus-Scanned: clamav-milter 0.96.3 at yxa-v X-Virus-Status: Clean X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 14:11:22 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 14:10:58 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Just a note here, if you want to make it work with today's libssh2, it is possible to do it through the ssh agent interface. Actually, the ssh agent interface is a much more reliable and secure way to use public-key authentication with libssh2 generally. To solve these particular issues, though, if someone wants to work on it, I think the simplest is to make libssh2 use GnuTLS. Decrypting password-protected private keys isn't a low-level crypto task, and thus doesn't belong in libgcrypt, it is a more high level feature. GnuTLS already supports it, and has the same license as libgcrypt. /Simon _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 14:21:04 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DKwtS022229; Fri, 5 Nov 2010 14:21:03 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DKvGp022148 for ; Fri, 5 Nov 2010 14:20:57 +0100 Date: Fri, 5 Nov 2010 14:20:57 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't haveapublic key In-Reply-To: <87r5ezzzjm.fsf@latte.josefsson.org> Message-ID: References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> <87r5ezzzjm.fsf@latte.josefsson.org> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 14:21:04 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 14:20:57 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Fri, 5 Nov 2010, Simon Josefsson wrote: > I think the simplest is to make libssh2 use GnuTLS. Decrypting > password-protected private keys isn't a low-level crypto task, and thus > doesn't belong in libgcrypt, it is a more high level feature. GnuTLS > already supports it, and has the same license as libgcrypt. I wouldn't mind doing that switch if it brings us that functionality. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 14:25:01 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DOuPG027146; Fri, 5 Nov 2010 14:25:00 +0100 Received: from smtp-gw11.han.skanova.net (smtp-gw11.han.skanova.net [81.236.55.20]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DOsZI027136 for ; Fri, 5 Nov 2010 14:24:54 +0100 Received: from [192.168.20.124] (90.227.111.200) by smtp-gw11.han.skanova.net (8.5.124.10) id 4C7CB7ED02457C76 for libssh2-devel@cool.haxx.se; Fri, 5 Nov 2010 14:24:47 +0100 Message-ID: <4CD40598.2070606@SDlabs.se> Date: Fri, 05 Nov 2010 14:24:40 +0100 From: Lars Nordin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.11) Gecko/20101020 Lanikai/3.1.5 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't haveapublic key References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> <87r5ezzzjm.fsf@latte.josefsson.org> In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 14:25:01 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 14:24:54 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 2010-11-05 14:20, Daniel Stenberg wrote: > On Fri, 5 Nov 2010, Simon Josefsson wrote: > >> I think the simplest is to make libssh2 use GnuTLS. Decrypting >> password-protected private keys isn't a low-level crypto task, and >> thus doesn't belong in libgcrypt, it is a more high level feature. >> GnuTLS already supports it, and has the same license as libgcrypt. > > I wouldn't mind doing that switch if it brings us that functionality. > Why would we switch to GnuTLS instead of just using OpenSSL (ie dropping gcrypt). Any license-issues? _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 14:29:39 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DTYXe029339; Fri, 5 Nov 2010 14:29:38 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5DSSE4028973 for ; Fri, 5 Nov 2010 14:28:28 +0100 Date: Fri, 5 Nov 2010 14:28:28 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Using libssh2_userauth_publickey_fromfile() when I don't haveapublic key In-Reply-To: <4CD40598.2070606@SDlabs.se> Message-ID: References: <899419.69857.qm@web45307.mail.sp1.yahoo.com> <4CD3CA48.4060403@peut.org> <702C2A8AA0DB49DE8D4C4A6456A05A1D@internal.avcosystems.com> <0B0C709AC25D4C4BB7E760234B8591FD@internal.avcosystems.com> <87r5ezzzjm.fsf@latte.josefsson.org> <4CD40598.2070606@SDlabs.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 14:29:39 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 14:28:28 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Fri, 5 Nov 2010, Lars Nordin wrote: >> I wouldn't mind doing that switch if it brings us that functionality. > Why would we switch to GnuTLS instead of just using OpenSSL (ie dropping > gcrypt). Any license-issues? We would still support OpenSSL. The switch would be from using gcrypt to using GnuTLS+gcrypt. And yes, people that opt for GnuTLS/gcrypt instead of OpenSSL often do that due to license reasons. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 5 15:24:34 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5EOFLL002181; Fri, 5 Nov 2010 15:24:32 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA5EOCd3002130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 5 Nov 2010 15:24:13 +0100 Received: (qmail 23927 invoked from network); 5 Nov 2010 14:24:09 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 5 Nov 2010 14:24:09 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Fri, 05 Nov 2010 14:24:09 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #194: Scp Bug X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/194#comment:10 Message-ID: <059.d38a4c290e6c1c0b91040bce0678ea3f@libssh2.stuge.se> References: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Trac-Ticket-ID: 194 In-Reply-To: <044.58182c24cc658b2182f74e694d2f6922@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 05 Nov 2010 15:24:34 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 05 Nov 2010 15:24:13 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #194: Scp Bug -----------------------+---------------------------------------------------- Reporter: robocide | Owner: Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: SCP | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: -----------------------+---------------------------------------------------- Comment (by robocide): Thanks for the information Stuge and Bagder. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 8 23:55:45 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA8MtGnP000372; Mon, 8 Nov 2010 23:55:41 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA8MtDjx000309 for ; Mon, 8 Nov 2010 23:55:14 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=HziJD38OFobYFS5uhOEMlPnE9/dxFEUd90yrQThCuSaYsyjjRNGy/XtzWxyjtLYHOG BL6zGdB0MMm1PCXahHQnTA/KUkFuysxAst/pLXSUYGbx9nTfb+vx26AeUN556Kg+RDg+ EPaN44DipOGbXaWxEJthx/tsmikHOeg22VpNU=; bh=Yl9EtKGRNbKdXpnyohU/bwRv3Wg=; t=1289256914; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id f3c4000481201bb6; Mon, 8 Nov 2010 17:55:13 -0500 From: "Mark Riordan" To: Subject: Re: libssh2_sftp_write now hangs eventually Date: Mon, 8 Nov 2010 16:55:03 -0600 Message-ID: <007201cb7f97$fb048dc0$f10da940$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Act/l/qBYfmTE1A/RASWxRIdqvkoTw== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 08 Nov 2010 23:55:45 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 08 Nov 2010 23:55:14 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se I've gotten a bit further. As you recall, calling libssh2_trace(session, ~0); (all debugging enabled) caused so much output to be generated that the timing changed, and the hang during the SFTP upload went away. So, I disabled various combinations of trace flags until I got the hang to reappear. Now I'm enabling just LIBSSH2_TRACE_ERROR. I changed the last line of sftp_write in sftp.c to: return _libssh2_error(session, LIBSSH2_ERROR_EAGAIN, "Would block at end of sftp_write"); During the last write, the program now gets this: ... libssh2_sftp_write 2 sent 8000 bytes [This is a successful write] [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.245994 Failure Event: -37 - Unable to send channel data [libssh2] 1.245994 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.261619 Failure Event: -37 - Unable to send channel data [libssh2] 1.261619 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.277244 Failure Event: -37 - Unable to send channel data [libssh2] 1.277244 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.292869 Failure Event: -37 - Unable to send channel data [libssh2] 1.292869 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.308494 Failure Event: -37 - Unable to send channel data [libssh2] 1.308494 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.308494 Failure Event: -37 - Unable to send channel data [libssh2] 1.308494 Failure Event: -37 - Would block at end of sftp_write [libssh2] 1.308494 Failure Event: -37 - Would block at end of sftp_write Every time I run it, it logs "Would block at end of sftp_write" twice immediately before hanging. A clue? Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 9 13:15:57 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA9CFDs5017043; Tue, 9 Nov 2010 13:15:54 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA9CFCxB017034 for ; Tue, 9 Nov 2010 13:15:12 +0100 Date: Tue, 9 Nov 2010 13:15:12 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write now hangs eventually In-Reply-To: <007201cb7f97$fb048dc0$f10da940$@com> Message-ID: References: <007201cb7f97$fb048dc0$f10da940$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 09 Nov 2010 13:15:57 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 09 Nov 2010 13:15:12 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Mon, 8 Nov 2010, Mark Riordan wrote: > [libssh2] 1.308494 Failure Event: -37 - Unable to send channel data > [libssh2] 1.308494 Failure Event: -37 - Would block at end of sftp_write > [libssh2] 1.308494 Failure Event: -37 - Would block at end of sftp_write > > Every time I run it, it logs "Would block at end of sftp_write" twice > immediately before hanging. > A clue? Lound and clear, thanks! I've now pushed two additional fixes: one that should prevent _libssh2_wait_socket() from hanging even if there's an internal error that makes it not wait for neither read nor write on the socket. This might be what you see. If this happens now, it should send a debug log output and set a 1 second timeout so it should at least continue. I then followed up and addressed what I believe was the reason for the error above, which is that the sftp_write() function decided on its own that it was a EAGAIN situation and it mustn't since it then doesn't set the correct "direction bits" etc which may confuse internals like I described above. Please pull these two commits and see what happens! -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 9 13:42:15 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA9Cg8Fb002639; Tue, 9 Nov 2010 13:42:14 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA9Cg71t002590; Tue, 9 Nov 2010 13:42:07 +0100 Date: Tue, 9 Nov 2010 13:42:07 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development , libssh@libssh.org Subject: SSH library hacker meetup at FOSDEM? Message-ID: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 09 Nov 2010 13:42:15 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 09 Nov 2010 13:42:07 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hi friends! (This mail is cross-posted to both the libssh and the libssh2 mailing lists, as I believe most people are just subscribed to one of them.) I've previously talked briefly with a few guys in #libssh (on freenode) and I talked with Peter Stuge (of libssh2) this weekend, so this isn't coming completely out of the blue for everyone... How about a little face to face SSH library hacker meetup at FOSDEM? I believe the "Security / hardware crypto" devroom sounds like a perfect place for it. Possibly, we could arrange for one presentation each to present our libraries to each other and to the crowd, for the fun of it and for the education value. I don't have any other intentions behind this suggestion. Meeting in person puts a face to a name and since we're not that many persons actively working on open source SSH libraries it could be cool to get to know each other a bit more. I can only see benefits from that! Of course everyone and anyone would be invited to join. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 9 14:42:34 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oA9DgJqh015627; Tue, 9 Nov 2010 14:42:32 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oA9DgIKx015623 for ; Tue, 9 Nov 2010 14:42:18 +0100 Received: (qmail 25208 invoked by uid 501); 9 Nov 2010 13:42:15 -0000 Message-ID: <20101109134215.25207.qmail@stuge.se> Date: Tue, 9 Nov 2010 14:42:15 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se, libssh@libssh.org Subject: Re: SSH library hacker meetup at FOSDEM? Mail-Followup-To: libssh2-devel@cool.haxx.se, libssh@libssh.org References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 09 Nov 2010 14:42:34 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 09 Nov 2010 14:42:18 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel Stenberg wrote: > How about a little face to face SSH library hacker meetup at FOSDEM? I > believe the "Security / hardware crypto" devroom sounds like a perfect > place for it. I also have a foot in the OpenSC community, which is where the idea for this devroom first came up. We keep a wiki page for the room: https://www.opensc-project.org/opensc/wiki/FOSDEM2011 > Possibly, we could arrange for one presentation each to present our > libraries to each other and to the crowd, for the fun of it and for > the education value. We plan for every project participating in the devroom to do a brief introduction at the start of the day, so that everyone will know what we everyone else does. A note on devrooms at FOSDEM for people who have not visited at all, or just not in a while; The devrooms at FOSDEM tend to always be overcrowded to the point where people can not or are not allowed to enter the room. Please make sure to arrive early before the timeslot if you're involved with one or more participating projects, so that you can be sure to actually get into the room. > I don't have any other intentions behind this suggestion. Meeting > in person puts a face to a name I like it! > Of course everyone and anyone would be invited to join. Very much so. FOSDEM is a developer meeting, no entry fee, Brussels in February was very nice last year. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 16:13:38 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFDCUK005261; Wed, 10 Nov 2010 16:13:34 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFDA4f005255 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 10 Nov 2010 16:13:10 +0100 Received: (qmail 14155 invoked from network); 10 Nov 2010 15:13:08 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 10 Nov 2010 15:13:08 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 10 Nov 2010 15:13:08 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #196: using libssh2_channel_read crushes X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/196 Message-ID: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Trac-Ticket-ID: 196 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 16:13:38 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 16:13:10 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #196: using libssh2_channel_read crushes ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc | Owner: Type: defect | Status: new Priority: high | Milestone: 1.2.7 Component: API | Version: 1.2.7 Keywords: | Blocks: Blocked By: | ---------------------------------------------------------------------------------------+ Core was generated by `/users/ahirsh/libssh2-1.2.7/example/.libs/lt-sftp 10.106.59.20'. Program terminated with signal 11, Segmentation fault. #0 _libssh2_transport_read (session=0xe7c7a88a) at transport.c:294 294 session->socket_block_directions &= ~LIBSSH2_SESSION_BLOCK_INBOUND; while i was trying to use libssh2_channel_read in the proper way : ... int rc; int bytecount; do { char buffer[0x4000]; cout<<"Before Block"< 0 ) { int i; bytecount += rc; fprintf(stdout, "We read:\n"); for( i=0; i < rc; ++i ) fputc( buffer[i], stdout); fprintf(stdout, "\n"); } else { fprintf(stdout, "libssh2_channel_read returned %d\n", rc); } }while( rc > 0); ... i encountered a crush with the below stack trace : Core was generated by `/users/ahirsh/libssh2-1.2.7/example/.libs/lt-sftp 10.106.59.20'. Program terminated with signal 11, Segmentation fault. #0 _libssh2_transport_read (session=0xad8d6e50) at transport.c:294 294 session->socket_block_directions &= ~LIBSSH2_SESSION_BLOCK_INBOUND; (gdb) bt #0 _libssh2_transport_read (session=0xe7c7a88a) at transport.c:294 #1 0x00fc9658 in _libssh2_channel_read (channel=0xb6a00468, stream_id=0, buf=0xb7565fc4 "", buflen=16384) at channel.c:1693 #2 0x00fc98fd in libssh2_channel_read_ex (channel=0xb6a00468, stream_id=0, buf=0xb7565fc4 "", buflen=16384) at channel.c:1844 #3 0x0804bbcb in LibraryCheck (param=0x0) at sftp.c:382 #4 0x00be343b in start_thread () from /lib/libpthread.so.0 #5 0x00348fde in clone () from /lib/libc.so.6 (gdb) quit -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 16:21:48 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFLiQT010276; Wed, 10 Nov 2010 16:21:48 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFLgUa010236 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 10 Nov 2010 16:21:42 +0100 Received: (qmail 14977 invoked from network); 10 Nov 2010 15:21:40 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 10 Nov 2010 15:21:40 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 10 Nov 2010 15:21:40 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #196: using libssh2_channel_read crushes X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/196#comment:1 Message-ID: <123.a68260d422c3dd7cb380516b4ea5d497@libssh2.stuge.se> References: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Trac-Ticket-ID: 196 In-Reply-To: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 16:21:48 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 16:21:42 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #196: using libssh2_channel_read crushes ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc | Owner: bagder Type: defect | Status: accepted Priority: high | Milestone: 1.2.7 Component: API | Version: 1.2.7 Resolution: | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by bagder): * owner: => bagder * status: new => accepted Comment: That would mean that 'session' is a bad pointer in there. Have you tried running your app using valgrind? Also, can you try writing up a small example source that can repeat this? You seem to be using the sftp example code, but that doesn't use channel_read... We've done a lot of SFTP improvements in the code lately, you may want to consider trying a recent daily snapshot/git repo version. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 16:26:35 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFQTrI016451; Wed, 10 Nov 2010 16:26:34 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAFQQ99016413 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 10 Nov 2010 16:26:27 +0100 Received: (qmail 15845 invoked from network); 10 Nov 2010 15:26:24 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 10 Nov 2010 15:26:24 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 10 Nov 2010 15:26:24 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #191: src/packet.c:_libssh2_packet_add problems X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/191#comment:1 Message-ID: <064.90f263adc89487f4f158c2df2a4c7221@libssh2.stuge.se> References: <049.68af1c01c6a3761f21f1486757cc186b@libssh2.stuge.se> X-Trac-Ticket-ID: 191 In-Reply-To: <049.68af1c01c6a3761f21f1486757cc186b@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 16:26:35 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 16:26:27 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #191: src/packet.c:_libssh2_packet_add problems ----------------------------+----------------------------------------------- Reporter: josefsson.org | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: misc | Version: 1.2.7 Resolution: | Keywords: Blocks: | Blocked By: ----------------------------+----------------------------------------------- Comment (by bagder): Lots of these are now addressed. The remaining problem is now that there are a range of packets that packet_add() can receive and add to the packets list that aren't checked at all, but the channel code is later reading from the list and assume the packets to be correct... -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 17:26:10 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAGPrxl018686; Wed, 10 Nov 2010 17:26:08 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAGPp2T018671 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 10 Nov 2010 17:25:52 +0100 Received: (qmail 23617 invoked from network); 10 Nov 2010 16:25:48 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 10 Nov 2010 16:25:48 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 10 Nov 2010 16:25:48 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #189: Error is not correctly reported when authenticated user does not have access to remote_file X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/189#comment:1 Message-ID: <123.200428151e0954d23efc4f8eee2ca3e4@libssh2.stuge.se> References: <108.1fa0ce76573f0cda17767ac0f906d77e@libssh2.stuge.se> X-Trac-Ticket-ID: 189 In-Reply-To: <108.1fa0ce76573f0cda17767ac0f906d77e@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 17:26:10 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 17:25:52 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #189: Error is not correctly reported when authenticated user does not have access to remote_file ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawmvjpzhdvrzdugilqti7ni8qkfjov_hilk | Owner: Type: defect | Status: closed Priority: normal | Milestone: 1.2.6 Component: protocol | Version: 1.2.6 Resolution: worksforme | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by bagder): * status: new => closed * resolution: => worksforme Comment: I just tried this with the git version and I cannot repeat your problems. With SCP, we get LIBSSH2_ERROR_SCP_PROTOCOL returned as there's really no good way to detect "file not found". With SFTP, libssh2_sftp_open() fails and libssh2_sftp_last_error() returns 2, which according to the header file is LIBSSH2_FX_NO_SUCH_FILE. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 19:39:09 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAIck10030633; Wed, 10 Nov 2010 19:39:05 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAIchiw030274 for ; Wed, 10 Nov 2010 19:38:44 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=QbGpx5fEUNszupZ114PIzln9XRDLL2h+0TM0XJHtqY3GXM3rP2oPv6vwfW5DojWdcQ t+cCNtNXTqvAND93whztxavuMK8DVxLxDv0swqgHkUUgbGd2L+WTFeeRzETbj3rPDkWK l4abx6aaKEuS4rYiAQbky6XonsZ2RFkkI0vD8=; bh=F2wScyfQaebpNxRTAPqLeimTf4o=; t=1289414319; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 9ae1000011810184; Wed, 10 Nov 2010 13:38:37 -0500 From: "Mark Riordan" To: Subject: Re: libssh2_sftp_write now hangs eventually Date: Wed, 10 Nov 2010 12:38:32 -0600 Message-ID: <001501cb8106$7adb2440$70916cc0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 thread-index: AcuBBniEfpBbfULOR3SegJOd4lBN9g== Content-Language: en-us x-cr-hashedpuzzle: AFCa Am2t BiQF CJ6C CN9y CthR C6u+ Dc/x Dh4O ECIr EZz8 FHb1 Fu8f GALU HF9O H5bA; 1; bABpAGIAcwBzAGgAMgAtAGQAZQB2AGUAbABAAGMAbwBvAGwALgBoAGEAeAB4AC4AcwBlAA==; Sosha1_v1; 7; {81559A6B-2EE4-454A-8539-62F800EC0F39}; bQByAGkAbwByAGQAYQBuAEAAaQBwAHMAdwBpAHQAYwBoAC4AYwBvAG0A; Wed, 10 Nov 2010 18:38:30 GMT; UgBlADoAIABsAGkAYgBzAHMAaAAyAF8AcwBmAHQAcABfAHcAcgBpAHQAZQAgAG4AbwB3ACAAaABhAG4AZwBzACAAZQB2AGUAbgB0AHUAYQBsAGwAeQA= x-cr-puzzleid: {81559A6B-2EE4-454A-8539-62F800EC0F39} X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 19:39:09 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 19:38:45 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > A clue? >>Lound and clear, thanks! I've now pushed two additional fixes .. >>Please pull these two commits and see what happens! Alas, it still hangs the same way. I did a Wireshark trace and noticed a very large number of full-sized outbound packets before it hung, even though the program thought it had sent only 12000 bytes before it hung. Of course, the SSH Wireshark trace is encrypted, and unlike with SSL, there's no Wireshark plug-in to decrypt it. So I looked a bit further by running mitm-ssh (Man in the Middle SSH), which decrypts the SSH stream. It turned out the Windows client was sending the same data over and over. I put some logs and an explanation here: http://60bits.net/sni/20101110.zip Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 22:58:57 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAALwCQT002176; Wed, 10 Nov 2010 22:58:52 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAALwB27002160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 10 Nov 2010 22:58:11 +0100 Received: (qmail 6912 invoked from network); 10 Nov 2010 21:58:06 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 10 Nov 2010 21:58:06 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 10 Nov 2010 21:58:06 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #187: Unable to connect to SFTP server using SSH-2.0-SSHD X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/187#comment:3 Message-ID: <062.d9a5f161cd68fdd2c40ca463481a70e5@libssh2.stuge.se> References: <047.ff473ed169991c7f4a9e3fa703b14621@libssh2.stuge.se> X-Trac-Ticket-ID: 187 In-Reply-To: <047.ff473ed169991c7f4a9e3fa703b14621@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 22:58:57 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 22:58:11 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #187: Unable to connect to SFTP server using SSH-2.0-SSHD --------------------------+------------------------------------------------- Reporter: sriramswamy | Owner: Type: defect | Status: closed Priority: blocker | Milestone: Component: SFTP | Version: 1.2.7 Resolution: outdated | Keywords: awaitsinfo Blocks: | Blocked By: --------------------------+------------------------------------------------- Changes (by bagder): * status: new => closed * resolution: => outdated -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 10 23:34:32 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAMYOvJ022012; Wed, 10 Nov 2010 23:34:32 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAAMYNxk021977 for ; Wed, 10 Nov 2010 23:34:23 +0100 Date: Wed, 10 Nov 2010 23:34:23 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write now hangs eventually In-Reply-To: <001501cb8106$7adb2440$70916cc0$@com> Message-ID: References: <001501cb8106$7adb2440$70916cc0$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 10 Nov 2010 23:34:32 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 10 Nov 2010 23:34:23 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Wed, 10 Nov 2010, Mark Riordan wrote: >>> Please pull these two commits and see what happens! > > Alas, it still hangs the same way. Ok, so my latest analysis wasn't correct then! :-( The fact that the log ends with two lines saying "Would block sftp_write" should still be some kind of clue I think. Perhaps you can add some debug output next to where that is returned and output the value of the 'eagain' variable there. I'm curious what the value is for the last output. > I did a Wireshark trace and noticed a very large number of full-sized > outbound packets before it hung, even though the program thought it had sent > only 12000 bytes before it hung. Of course, the SSH Wireshark trace is > encrypted, and unlike with SSL, there's no Wireshark plug-in to decrypt it. > So I looked a bit further by running mitm-ssh (Man in the Middle SSH), which > decrypts the SSH stream. It turned out the Windows client was sending the > same data over and over. I'm puzzled and I have no explanation for that right now. I can only think that something in the sftp_write() function isn't handling the 'chunks' properly so that it somehow believes it should send the same chunk again even though it was already sent. I've read through the code a few more times and it made me clean it up somewhat more and do some minor fixes but I don't think any of those are reason for what you experience. I can't understand how the code would resend a block. Anyone else? -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 01:18:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAB0IPMp005108; Thu, 11 Nov 2010 01:18:41 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAB0IMlF005081 for ; Thu, 11 Nov 2010 01:18:23 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=PSCBRlWqoeluqxDRGrXEZWaV8D1xk/7VnUX6T9T+E5eEA76hd/8g2EjxT7vowTajtf V5MivPMCjcmvoLna85pmR01uqxyFSSe9KDzykdUFANehYT82Y1BVK14ITCkX0rRhcqFg vQSXUL4K5xOlbfEGtzp1x8Kb4Ra5/S9Ob8buY=; bh=LLcQxkrfdLI6GEi/GNTTwey4yaA=; t=1289434701; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 6af9000097570c89; Wed, 10 Nov 2010 19:18:20 -0500 From: "Mark Riordan" To: Subject: libssh2_sftp_write doesn't hang anymore Date: Wed, 10 Nov 2010 18:18:08 -0600 Message-ID: <005201cb8135$ed7a5be0$c86f13a0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 thread-index: AcuBNerqucNsCPEJTNKyw7t5CyMmSQ== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 01:18:44 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 01:18:24 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se I saw that Daniel committed a bunch of changes, so I pulled them and tried my test program again. (There was probably an email associated with the changes, but I'm subscribed to the digest rather than individual messages. I fixed that.) My upload test from Windows to Linux now works! But it's very slow, even with debug turned off. I uploaded a 479988 byte file consisting of a bunch of lines like Line 1 Line 2 etc. The 19 MB debug log has 149009 "Would block" messages, which seems excessive - one for every 4 bytes. The log was created by virtue of this code: bitmask = LIBSSH2_TRACE_ERROR; libssh2_trace(session, bitmask); The file is at: http://60bits.net/sni/20101110-479988.zip (This is the first time I've ever gotten zip to report 100% compression!) My application recorded that calls to libssh2_sftp_write generally resulted in 4000 or 8000 bytes written. Never less than 500. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 10:32:07 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAB9Vj2C003348; Thu, 11 Nov 2010 10:32:04 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAB9VhEd003339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 11 Nov 2010 10:31:43 +0100 Received: (qmail 27910 invoked from network); 11 Nov 2010 09:31:39 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 11 Nov 2010 09:31:39 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Thu, 11 Nov 2010 09:31:39 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #196: using libssh2_channel_read crushes X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/196#comment:2 Message-ID: <123.5b95cf33df03083b46c99fb26ebe562b@libssh2.stuge.se> References: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Trac-Ticket-ID: 196 In-Reply-To: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 10:32:07 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 10:31:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #196: using libssh2_channel_read crushes ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc | Owner: bagder Type: defect | Status: accepted Priority: high | Milestone: 1.2.7 Component: API | Version: 1.2.7 Resolution: | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Comment (by www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc): i did not added all the code in order to not confuse with my code, so below is code that relevant to Libssh2, other code initialize 5 Threads, and invokes this below code (note every thread has a seperate connection and libssh2 vars) : .... // Here we are transfering 40 files to a target machine via SFTP for(int i=0;i<40;i++) { cout<<"Transfering File:"<( &(ostringstream() << i) )->str(); //fprintf(stderr,"Line 1 "); buf_size = strlen(buffer_to_send); //fprintf(stderr,"Line 2 "); //file = fopen((file_name+number).c_str(),"rb"); ifstream file; file.open((file_name+number).c_str(),ifstream::out); if(!file.good()) { cout<<"Error , Could not open file"< 0); file.close(); } // Ok we finished transfering 40 files now lets go to invoke some command // and then we will call libssh2_channel_read , NOTE that this code causes the corruption // as described above pthread_mutex_lock(my_mutex); LIBSSH2_CHANNEL *channel = NULL; if((channel = libssh2_channel_open_session(session)) == NULL ) { cout<<"Error Allocating a new channel"< 0 ) { int i; bytecount += rc; fprintf(stdout, "We read:\n"); for( i=0; i < rc; ++i ) fputc( buffer[i], stdout); fprintf(stdout, "\n"); } else { fprintf(stdout, "libssh2_channel_read returned %d\n", rc); } }while( rc > 0); pthread_mutex_unlock(my_mutex); finalize_connection(session,sftp_session,sftp_handle,sock); -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 12:47:07 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABBkqpl027565; Thu, 11 Nov 2010 12:47:06 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABBkpAV027558 for ; Thu, 11 Nov 2010 12:46:51 +0100 Date: Thu, 11 Nov 2010 12:46:51 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_sftp_write doesn't hang anymore In-Reply-To: <005201cb8135$ed7a5be0$c86f13a0$@com> Message-ID: References: <005201cb8135$ed7a5be0$c86f13a0$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 12:47:07 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 12:46:51 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Wed, 10 Nov 2010, Mark Riordan wrote: > My upload test from Windows to Linux now works! Oh. It proves we are at least still making progress... > The 19 MB debug log has 149009 "Would block" messages, which seems > excessive - one for every 4 bytes. (Once for every 4 byte of _your_ payload data, yes, but SFTP, SSH channels and SSH transport will add a notiacble amount of data to send.) It certainly seems a bit excessive yes. I know that we do a few too many "try this just in case" calls so that there can be a few too many per invoke of a the sftp_write function, but it does give me the feeling that we're not doing the waiting for the socket activity correctly. A detail in the log: We get excessive -37 logged, but they are all "lumped up" and get done on *the same microsecond*. See for example at the time 0.414451 where 966 lines are reported on the same timestamp. Then there's a 16 millisecond pause, where presumably the code waits for the socket to turn writable again. Then at 0.430076 there's an additional 1320 lines logged at *the same microsecond*. I figure the GetSystemTimeAsFileTime() function we use for this simply has this low resolution so we get this inexactness in the timings. > My application recorded that calls to libssh2_sftp_write generally resulted > in 4000 or 8000 bytes written. Never less than 500. Right, as you use a 32500 bytes buffer and libssh2 will bite off 4000 bytes at a time from that so the SFTP packets will become 4000*8 + 500*1 (of payload). How many bytes of those that are actually sent off depends on the window size in the SSH channel and how much the TCP layer accepts. You will probably gain an unmeasureably small amount of speed by using even 4000 byte blocks. I've just now pushed yet another change to the sftp_write code that concerns situations where more data is ACKed already than the buffer size passed in to the function as that would cause major weirdness. I don't think you've seen effects of this, as I believe things would go down the drain hard if so. Some ideas on how to move further from here... - Start using a smaller test file. Your problem is probably just as visible with a 100K file. Or perhaps you should even try a much smaller, like a 10K one to see how that behaves. - Since we're now down to "just" being slow, you can try to add more output logs again to see if they reveal anything. - We need to figure out where the EAGAINs come from and make sure libssh2 acts correctly on them, but it a bit hard when the logging slows things down this much. Possibly you should make your example use the libssh2_trace_sethandler() function to get the tracing done by your own function, and then you just keep all the logging in memory until the test run is over and then save it to disk as that is probably gonna be A LOT faster although of course quite memory consuming. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 19:34:07 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIXM91023767; Thu, 11 Nov 2010 19:34:02 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oABIXKdq023753 for ; Thu, 11 Nov 2010 19:33:20 +0100 Received: (qmail 1358 invoked by uid 501); 11 Nov 2010 18:33:14 -0000 Message-ID: <20101111183314.1357.qmail@stuge.se> Date: Thu, 11 Nov 2010 19:33:14 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2 master 7899d28 sftp_write: handle "left over" acked data Mail-Followup-To: libssh2-devel@cool.haxx.se References: <20101111105901.4956.qmail@earth.stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20101111105901.4956.qmail@earth.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 19:34:07 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 19:33:20 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se libssh2@git.stuge.se wrote: > - Log ----------------------------------------------------------------- > commit 7899d28ecbb3f2c0330429ea9c47834167fe2f7f > Author: Daniel Stenberg > Commit: Daniel Stenberg > > sftp_write: handle "left over" acked data > > The SFTP handle struct now buffers number of acked bytes that haven't > yet been returned. The way this is used is as following: > > 1. sftp_write() gets called with a buffer of let say size 32000. We > split 32000 into 8 smaller packets and send them off one by one. One of > them gets acked before the function returns so 4000 is returned. > > 2. sftp_write() gets called again a short while after the previous one, > now with a much smaller size passed in to the function. Lets say 8000. > In the mean-time, all of the remaining packets from the previous call > have been acked (7*4000 = 28000). This function then returns 8000 as all > data passed in are already sent and it can't return any more than what > it got passed in. But we have 28000 bytes acked. We now store the > remaining 20000 in the handle->u.file.acked struct field to add up in > the next call. > > 3. sftp_write() gets called again, and now there's a backlogged 20000 > bytes to return as fine and that will get skipped from the beginning > of the buffer that is passed in. What a mess. :\ Not so much the implementation but the problem.. It would be nice to change at least _sftp_write() so that it can return more bytes as sent than were given to it at this call, but never more than has been given to it total. This would mean a change from the API as it is today, but I think it's the only way to make sense of the mess. If I start by _sftp_write()ing 32000 byte, 4000 get sent, and I then continue _sftp_write()ing less-than-28000 byte on every call, there is a problem. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 19:34:10 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIY7tn023954; Thu, 11 Nov 2010 19:34:10 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIXZHe023786 for ; Thu, 11 Nov 2010 19:33:37 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=F/SAy1vTGBbp2TVUyHX5aWDJEmft0WpZP9C7302JUDOvqZUZe1IMe5j5wW3JgNjpdO /xqN4v/N1fyvIHbkXOGPbS9PHEV2FuN29nVUNS1Nve6Co6SaZtpRPsOztBZ2EjEOEZzr cUNMlCAjHJScmP/U14g6vgT6s29jcCJLcSNJQ=; bh=qsRVLJNXmy6z6d/+Lmcfn7nRkLE=; t=1289500421; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 077a00027150191e; Thu, 11 Nov 2010 13:33:39 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> In-Reply-To: Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Thu, 11 Nov 2010 12:33:26 -0600 Message-ID: <002f01cb81ce$ee226a30$ca673e90$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 thread-index: AcuBlnPqLrWQbCkXQBu1dWsc/TgJ2QAOCQUg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 19:34:10 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 19:33:37 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se > Since we're now down to "just" being slow, you can try to add more > output logs again to see if they reveal anything. > - We need to figure out where the EAGAINs come from and make sure > libssh2 acts correctly on them, but it a bit hard when the logging > slows things down this much. Possibly you should make your example > use the libssh2_trace_sethandler() function... Last time, as I mentioned, uploads were quite slow regardless of whether logging was turned on. Now uploads are faster - 20 times faster - when logging is turned on than when logging is disabled! For that reason, I didn't bother trying to speed up the logging. With the OS buffering writes to disk, I don't think it would have made much of a difference anyway. Without logging turned on, the process uses about 100% of the CPU even though through is about 0.03 MB/sec (compared with about 6 MB/sec expected throughput). With logging turned on, throughput is about 0.7 MB/sec. I have a feeling that a lot of retransmitting is still going on. mitm-ssh reports a massive amount of retransmitting, but I think it's somehow making things worse. The above timings, and below log files, were directly to the SSH server. I did use Wireshark on one of the uploads. I processed the Wireshark dump with a script I wrote, and it indicated that 30517591 bytes were sent for the upload of a 120000-byte file. Granted, there is some overhead to SSH, but not that much overhead. I have a libssh2 trace and corresponding Wireshark capture: DebugFast2010-11-11.zip From a run without tracing turned on (and therefore a run which counter-intuitively was very slow) I have this much larger Wireshark capture, presumably showing a lot of retransmissions (though encrypted, of course): NoDebugSlow2010-11-11.zip I'm running out of space on my personal website, so rather than put them on my website, I will get these files to Daniel via some other way. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 19:56:25 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIuHsw006572; Thu, 11 Nov 2010 19:56:24 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oABIuGtE006568 for ; Thu, 11 Nov 2010 19:56:16 +0100 Received: (qmail 5066 invoked by uid 501); 11 Nov 2010 18:56:14 -0000 Message-ID: <20101111185614.5065.qmail@stuge.se> Date: Thu, 11 Nov 2010 19:56:14 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write doesn't hang anymore Mail-Followup-To: libssh2-devel@cool.haxx.se References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <002f01cb81ce$ee226a30$ca673e90$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 19:56:25 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 19:56:17 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > I'm running out of space on my personal website, so rather than > put them on my website, I will get these files to Daniel > via some other way. Attach them to a ticket on http://trac.libssh2.org/ I just upped the attachment limit to 25MB. Also, the git server supports both git: and http: now, for those behind strict firewalls. Either of these will work: git clone git://git.libssh2.org/libssh2.git git clone http://git.libssh2.org/libssh2.git gitweb remains available at the same address as before: http://git.libssh2.org/?p=libssh2.git //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 19:57:59 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIvuxi007121; Thu, 11 Nov 2010 19:57:59 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABIvtrZ007113 for ; Thu, 11 Nov 2010 19:57:55 +0100 Date: Thu, 11 Nov 2010 19:57:55 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2 master 7899d28 sftp_write: handle "left over" acked data In-Reply-To: <20101111183314.1357.qmail@stuge.se> Message-ID: References: <20101111105901.4956.qmail@earth.stuge.se> <20101111183314.1357.qmail@stuge.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 19:57:59 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 19:57:55 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Thu, 11 Nov 2010, Peter Stuge wrote: >> The SFTP handle struct now buffers number of acked bytes that haven't >> yet been returned. The way this is used is as following: > > What a mess. :\ Not so much the implementation but the problem.. Yes, it is an unfortunate side-effect of our API and the new multi-packet approach of doing SFTP. > It would be nice to change at least _sftp_write() so that it can return more > bytes as sent than were given to it at this call, but never more than has > been given to it total. > > This would mean a change from the API as it is today, but I think it's the > only way to make sense of the mess. That's not a bad idea. There are two drawbacks with it though as I see it: 1) it is not how read() etc works and we mimic their return code systems a bit in other aspects so it might be a bit unexpected to users and 2) it is a bit hard to document and explain in a concise manner. > If I start by _sftp_write()ing 32000 byte, 4000 get sent, and I then > continue _sftp_write()ing less-than-28000 byte on every call, there is a > problem. It's really not that big of a problem. It is mostly a problem to keep things correct in the code and in your head when thinking about it! :-) If you first call it with 32000 bytes and 4000 get sent, and you then switch to call _sftp_write with 4000 from then on, it will take (at best) 7 of those calls to get the backlogged ACKs returned back (and in these calls libssh2 can't send off any new packets since no new data is provided yet), but by then it has adapted and will work "normally" again as it has then dealt with the backlog. Luckily, I don't think this kind of situation is very likely to happen much. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 11 20:14:31 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABJENIn019043; Thu, 11 Nov 2010 20:14:30 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABJELkK019008 for ; Thu, 11 Nov 2010 20:14:21 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=kUg0IDoqCzbD9B1Ig59R0p3BeUzmlvZV+UqCgtaHPr0SI1Fx87t+ciO1YV/Md6ec7W wfmpaxKVxr+KvQYO2DKStJhYXbOy+hthsCcGWX3cagCbRNDKSURwHAjw/Rv6nSyFATxW VPTR6DlLUxK3YsK5+VIG0ffu1K0e3c+5Jdmh0=; bh=XquRtA9a8gLfVwGBWHixN9wcdBo=; t=1289502866; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 126e000283ac1a5f; Thu, 11 Nov 2010 14:14:26 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <20101111185614.5065.qmail@stuge.se> In-Reply-To: <20101111185614.5065.qmail@stuge.se> Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Thu, 11 Nov 2010 13:14:12 -0600 Message-ID: <003501cb81d4$a0309760$e091c620$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 thread-index: AcuB0l/EOQ/pLAdJRSGtTich3dL5YQAAhoqg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 11 Nov 2010 20:14:30 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 11 Nov 2010 20:14:22 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter: > Attach them to a ticket on http://trac.libssh2.org/ > I just upped the attachment limit to 25MB. OK, thanks. Daniel already has the files now - but I'll do that next time. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 00:10:58 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABNAXsX024209; Fri, 12 Nov 2010 00:10:53 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oABNAVsn024200 for ; Fri, 12 Nov 2010 00:10:31 +0100 Date: Fri, 12 Nov 2010 00:10:31 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: libssh2_sftp_write doesn't hang anymore In-Reply-To: <002f01cb81ce$ee226a30$ca673e90$@com> Message-ID: References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 00:10:58 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 00:10:32 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Thu, 11 Nov 2010, Mark Riordan wrote: > Last time, as I mentioned, uploads were quite slow regardless > of whether logging was turned on. > Now uploads are faster - 20 times faster - when logging > is turned on than when logging is disabled! Which also implies that something is seriously broken somwhere. > For that reason, I didn't bother trying to speed up the logging. With the > OS buffering writes to disk, I don't think it would have made much of a > difference anyway. I agree with that reasoning. > Without logging turned on, the process uses about 100% of the CPU even > though through is about 0.03 MB/sec (compared with about 6 MB/sec expected > throughput). That sounds like a busy-loop somewhere. When I do my tests on localhost, I tend to look at the CPU-meter as a good judge of how well it performs. libssh2 1.2.7 used the CPU much less then than what curren git does, simply due to the much higher throughput I get now. BTW, that expected throughput, is that what you get when you upload to the same server from the same machine using OpenSSH's or putty's sftp? 0.03 MB/sec is shockingly slow, but 0.7 MB/sec is not much better... > I have a feeling that a lot of retransmitting is still going on. mitm-ssh > reports a massive amount of retransmitting, but I think it's somehow making > things worse. The above timings, and below log files, were directly to the > SSH server. > > I did use Wireshark on one of the uploads. I processed the Wireshark dump > with a script I wrote, and it indicated that 30517591 bytes were sent for > the upload of a 120000-byte file. Granted, there is some overhead to SSH, > but not that much overhead. I checked your 'DebugFast.log' file. It shows: $ grep "libssh2_transport_write send" DebugFast.log | tr -d '()' | awk '{a+=$4} END {print a}' 541192 Judging from the log, it was a file with "Line 1" to "Line 40000" with CRLF newlines. That makes it a 468894 byte file. I'd say it looks about right. Also, it is interesting to see that your 32500 byte buffer perfectly consistently ends up as send() calls using the sizes 4292 bytes eight times and then a 644 bytes send() and then it loops again until end of file. I cannot explain where that extra data comes from that you see. libssh2 doesn't seem to log any, and I'm quite sure it logs all data it sends. In the 40000 lines case, how much does your script count? > From a run without tracing turned on (and therefore a run which > counter-intuitively was very slow) I have this much larger Wireshark > capture, presumably showing a lot of retransmissions (though encrypted, of > course): NoDebugSlow2010-11-11.zip Right, we can't detect any retransmissions in the SSH layer there but as we don't see any in the logging case I don't think they come from there in the non-log case either. When viewing that PCAP file with a column for delta time, it is obvious that the longest times between packets are always when the server is supposed to ACK a packet. Check out for example from number 11527 and onwards. All packets have less than 100 us between them, and every time the server is about to ACK a TCP seq, it takes 300+ us and then it doesn't send any data but only sends an empty ACK. If you send an SFTP file using another client, do the inter-packet timings look the same? What transfers speeds do other clients get between these two hosts? -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 03:29:53 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAC2T82P007815; Fri, 12 Nov 2010 03:29:48 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAC2T6vm007803 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 12 Nov 2010 03:29:06 +0100 Received: (qmail 32229 invoked from network); 12 Nov 2010 02:29:00 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 12 Nov 2010 02:29:00 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Fri, 12 Nov 2010 02:29:00 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #197: Very slow sftp uploads from Windows X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/197 Message-ID: <045.3abea9eceb7c7df16aff9ed2443543f2@libssh2.stuge.se> X-Trac-Ticket-ID: 197 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 03:29:53 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 03:29:06 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #197: Very slow sftp uploads from Windows ------------------------+--------------------------------------------------- Reporter: riordanmr | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.8 Component: SFTP | Version: 1.2.7 Keywords: | Blocks: Blocked By: | ------------------------+--------------------------------------------------- Uploads to various SSH servers from Windows are very slow. This has been discussed at length on libssh2-devel@cool.haxx.se; this ticket is mostly a place to upload supporting files. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 03:32:07 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAC2W42I016351; Fri, 12 Nov 2010 03:32:07 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAC2W2go016234 for ; Fri, 12 Nov 2010 03:32:02 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=cq69YPrL1daYjXFBPKKTrIobKlyuQH+gg07qfQWEyl3Y7tdj5cPUvxeTiabeIdZY/Y 8yqrowldqC2U7t1+GQm6lE2N5h2z/kwkZ8RwS69IvTPPnIAhC73hgkv3PQzjty3AiQdd bjgqFnV/RC8EwutM1TvqdYYv8S0I+wGbqOx6c=; bh=EzlMiCJ6kKFewIVVPG4UHVwjrac=; t=1289529124; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id e4920003145f21d2; Thu, 11 Nov 2010 21:32:04 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> In-Reply-To: Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Thu, 11 Nov 2010 20:31:51 -0600 Message-ID: <000301cb8211$c3dc4a00$4b94de00$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuB9fxLUGLC3zn4QyKPhv5V6aK+PgAC6SJQ Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 03:32:07 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 03:32:03 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > BTW, that expected throughput, is that what you get when you upload to the > same server from the same machine using OpenSSH's or putty's sftp? That is with both psftp and WS_FTP Pro (in SFTP mode). I get around 6 MB/sec on the very same client and server hardware for these programs. (FYI, I'm back to using a 100 Mbit/sec Ethernet. I have a client with a 1 Gbit/sec link, but it's more difficult to get to and the higher speed isn't yet necessary for our purposes.) > I checked your 'DebugFast.log' file. It shows: > Judging from the log, it was a file with "Line 1" to "Line 40000" with CRLF > newlines. That makes it a 468894 byte file. > I'd say it looks about right. Right you are - my bad. I have two test files, and one is 4 times the size of the other. This was indeed the 479988-byte one. Sorry about that. The DebugFast.txt file in that zip shows that the application thought it sent 479988 bytes. At any rate, the same principle applies. The NoDebugSlow capture showing 30517591 uploaded raw bytes is still much larger than would be necessary for a 479988-byte file. > If you send an SFTP file using another client, do the inter-packet timings > look the same? Well, many of the long delays are when waiting for an ACK, but of course not all. I uploaded a zip file with the capture and some supporting files to http://trac.libssh2.org/ticket/197 I used PSFTP for this upload. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 15:18:34 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACEHnae019639; Fri, 12 Nov 2010 15:18:31 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACEHleE019612 for ; Fri, 12 Nov 2010 15:17:47 +0100 Date: Fri, 12 Nov 2010 15:17:47 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: libssh2_sftp_write doesn't hang anymore In-Reply-To: <000301cb8211$c3dc4a00$4b94de00$@com> Message-ID: References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 15:18:34 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 15:17:47 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Thu, 11 Nov 2010, Mark Riordan wrote: > At any rate, the same principle applies. The NoDebugSlow capture showing > 30517591 uploaded raw bytes is still much larger than would be necessary for > a 479988-byte file. We need to figure out where all that data comes from. I mean which layer that decides it needs to get sent. The fact that it doesn't ruin the file in the server end is funny, which means all the necessary packets are at least sent. For the purpose of debugging this, can you make _libssh2_send() increase a global variable with the amount of data it thinks send() has sent and then output that global variable at the end of the test? That's the raw sender, which then should be ~15-20% larger than the file size. See my suggested patch below. If that counter gets crazy big, we continue and add the similar counter to more upper layer functions. Within channel_write and _libssh2_channel_write and then finally within sftp_write. --- a/src/misc.c +++ b/src/misc.c @@ -119,6 +119,7 @@ _libssh2_recv(libssh2_socket_t socket, void *buffer, size_t * Wrapper around standard send to allow WIN32 systems * to set errno */ +size_t filecounter; ssize_t _libssh2_send(libssh2_socket_t socket, const void *buffer, size_t length, int flags) @@ -127,6 +128,8 @@ _libssh2_send(libssh2_socket_t socket, const void *buffer, s #ifdef WIN32 if (rc < 0 ) errno = wsa2errno(); + else + filecounter += rc; #endif #ifdef VMS if (rc < 0 ){ -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 18:10:30 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACHAAU4029575; Fri, 12 Nov 2010 18:10:26 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oACHA9N9029568 for ; Fri, 12 Nov 2010 18:10:09 +0100 Received: (qmail 31712 invoked by uid 501); 12 Nov 2010 17:10:05 -0000 Message-ID: <20101112171005.31711.qmail@stuge.se> Date: Fri, 12 Nov 2010 18:10:04 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2 master aff9c82 channel_write: move some logic to _libssh2_channel_write Mail-Followup-To: libssh2-devel@cool.haxx.se References: <20101112141704.15893.qmail@earth.stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20101112141704.15893.qmail@earth.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 18:10:30 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 18:10:09 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se libssh2@git.stuge.se wrote: > +++ b/src/channel.c > @@ -1994,27 +1994,6 @@ channel_write(LIBSSH2_CHANNEL *channel, int stream_id, > buflen = 32700; > > if (channel->write_state == libssh2_NB_state_idle) { > - _libssh2_debug(session, LIBSSH2_TRACE_CONN, > - "Writing %d bytes on channel %lu/%lu, stream #%d", > - (int) buflen, channel->local.id, channel->remote.id, > - stream_id); Losing this logging in channel_write() is a bit unfortunate, since there are multiple code paths leading to the function. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 20:17:19 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACJGDIf008833; Fri, 12 Nov 2010 20:17:14 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACJGBwh008827 for ; Fri, 12 Nov 2010 20:16:11 +0100 Date: Fri, 12 Nov 2010 20:16:11 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2 master aff9c82 channel_write: move some logic to _libssh2_channel_write In-Reply-To: <20101112171005.31711.qmail@stuge.se> Message-ID: References: <20101112141704.15893.qmail@earth.stuge.se> <20101112171005.31711.qmail@stuge.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 20:17:19 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 20:16:11 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Fri, 12 Nov 2010, Peter Stuge wrote: >> @@ -1994,27 +1994,6 @@ channel_write(LIBSSH2_CHANNEL *channel, int stream_id, > Losing this logging in channel_write() is a bit unfortunate, since there are > multiple code paths leading to the function. No, I recently introduced an additional function "layer" so _libssh2_channel_write() is the function that is used all over internally and channel_write() is only ever called from within _libssh2_channel_write(). I modified the logging to avoid excessive logging and unnecessary checks when _libssh2_channel_write() loops and does multiple calls to channel_write(). -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 20:19:48 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACJJjVd009786; Fri, 12 Nov 2010 20:19:47 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oACJJiFt009781 for ; Fri, 12 Nov 2010 20:19:44 +0100 Received: (qmail 15548 invoked by uid 501); 12 Nov 2010 19:19:40 -0000 Message-ID: <20101112191940.15547.qmail@stuge.se> Date: Fri, 12 Nov 2010 20:19:40 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2 master aff9c82 channel_write: move some logic to _libssh2_channel_write Mail-Followup-To: libssh2-devel@cool.haxx.se References: <20101112141704.15893.qmail@earth.stuge.se> <20101112171005.31711.qmail@stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 20:19:48 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 20:19:44 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel Stenberg wrote: >> Losing this logging in channel_write() is a bit unfortunate, since >> there are multiple code paths leading to the function. > > No, I recently introduced an additional function "layer" so > _libssh2_channel_write() is the function that is used all over > internally and channel_write() is only ever called from within > _libssh2_channel_write(). Cool! //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 20:55:56 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACJtpWX029288; Fri, 12 Nov 2010 20:55:56 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACJstCE028553 for ; Fri, 12 Nov 2010 20:54:56 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=WZZoG1qClm8/4/BqfKcu4945hp0EsgU+ts8KDkSV2goLNJEvXz+unwNs6TiZguoLPq FXuEqsHN6MqwpJtR9M9E1wT2/2EG7RIOqGj59L3xwuDJD2uP42JBSPzj2VzZi5OCZ54l qhCjX/pM4nLigobxe5ZFM6FdV/RKLsYZ8+n+A=; bh=8aVW5AAw7ZZUpt6yLD+EivgN4hs=; t=1289591697; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 3e60000504fb3361; Fri, 12 Nov 2010 14:54:56 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> In-Reply-To: Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Fri, 12 Nov 2010 13:54:44 -0600 Message-ID: <001f01cb82a3$74feb200$5efc1600$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuCdNt2zzRL3dcrTu6bEAah496awwALi7wQ Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 20:55:56 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 20:54:56 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: >For the purpose of debugging this, can you make _libssh2_send() increase a >global variable with the amount of data it thinks send() has sent and then >output that global variable at the end of the test? That's the raw sender, >which then should be ~15-20% larger than the file size. See my suggested patch >below. > >If that counter gets crazy big, we continue and add the similar counter to >more upper layer functions. Within channel_write and _libssh2_channel_write >and then finally within sftp_write. I did that, and ran it, and put the results here: http://trac.libssh2.org/attachment/ticket/197/ See 2010-11-12.zip I didn't have time to do much analysis, nor to figure out how to attach the Wireshark trace I created. (Trac kept blowing up on the 16 MB attachment.) Hopefully I'll have time later today to look at this more. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 21:14:07 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACKE0vb008675; Fri, 12 Nov 2010 21:14:07 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oACKDxH0008672 for ; Fri, 12 Nov 2010 21:14:00 +0100 Received: (qmail 21609 invoked by uid 501); 12 Nov 2010 20:13:56 -0000 Message-ID: <20101112201356.21608.qmail@stuge.se> Date: Fri, 12 Nov 2010 21:13:56 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write doesn't hang anymore Mail-Followup-To: libssh2-devel@cool.haxx.se References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <001f01cb82a3$74feb200$5efc1600$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 21:14:07 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 21:14:00 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > I did that, and ran it, and put the results here: > http://trac.libssh2.org/attachment/ticket/197/ > See 2010-11-12.zip > > I didn't have time to do much analysis, nor to figure out how to > attach the Wireshark trace I created. (Trac kept blowing up on > the 16 MB attachment.) I changed another part of the server to also accept 25M uploads, please let me know if you run into trouble with this again. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 22:12:24 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACLCCZR007017; Fri, 12 Nov 2010 22:12:23 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACLCBKY006991 for ; Fri, 12 Nov 2010 22:12:11 +0100 Date: Fri, 12 Nov 2010 22:12:11 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: libssh2_sftp_write doesn't hang anymore In-Reply-To: <001f01cb82a3$74feb200$5efc1600$@com> Message-ID: References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 22:12:24 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 22:12:11 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Fri, 12 Nov 2010, Mark Riordan wrote: > I did that, and ran it, and put the results here: Okej, the executive summary is this table: Transferred 0.114 MB Final tot_libssh2_send=15707099 tot_libssh2_transport_send=1558708 tot_send_existing=14148364 tot_libssh2_keepalive_send=0 tot_channel_write=1460245 put took 2.281 seconds = 0.050 MB/sec These are really useful details and they show that things are terribly wrong! I'm first focusing on "tot_libssh2_send=15707099" as it shows the problem is present in the lowest possible layer within libssh2. For this to be able to happen, I'm thinking there's something wrong with the EAGAIN detection for win32 and I've just now remade how it is handled: previously we would convert from the win32 error to the errno value and store that in errno to allow the parent function to read it. Starting now, we only use the _libssh2_recv and _libssh2_send functions and they simply return the negative errno value on errors so that we don't ever have to store anything in errno for win32 but only return the correct value. Commit ca2e81eb1f1bcb3 was just pushed with this change. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 22:43:52 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACLheM5029561; Fri, 12 Nov 2010 22:43:51 +0100 Received: from nona.lunarservers.com (nona.lunarservers.com [74.50.21.220]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACLhbUl029373 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 12 Nov 2010 22:43:39 +0100 Received: from localhost ([127.0.0.1] helo=www.igrspace.com) by nona.lunarservers.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1PH1P4-0000Mg-Ex for libssh2-devel@cool.haxx.se; Fri, 12 Nov 2010 13:43:26 -0800 MIME-Version: 1.0 Date: Fri, 12 Nov 2010 13:43:26 -0800 From: To: Subject: =?UTF-8?Q?ssh=5Ftunnel?= Message-ID: <9d76d0b087568d99cb196facc8f1fa6a@igrspace.com> X-Sender: junk@igrspace.com User-Agent: RoundCube Webmail/0.3.1 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - nona.lunarservers.com X-AntiAbuse: Original Domain - cool.haxx.se X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - igrspace.com X-Source: X-Source-Args: X-Source-Dir: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 22:43:52 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 22:43:39 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hello everyone, I am actually here with a usage question for the libssh2 PHP binding rather than a devel question, feel free to redirect me. I am trying to setup a tunneled ssh connection from a PHP script to an ssh server. It's a two-step connection over a 'security gateway', as firewalls prevent me from reaching the target server directly. Using ssh, i would typically do ssh -NL localhost:localport:desthost:22 & ssh -p localport localhost Now, the PHP binding does provide ssh2_tunnel() which takes remote host and port, but it returns a PHP resource accepting fwrite(), fgets(), etc. Unless there is something major I am missing, I believe this is of any use for another ssh connection through it. Or is there a way to start a libssh2 session on an existing tcp connection using the PHP binding ? Any help is appreciated. Immanuel Gregoire _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 23:08:18 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACM8Anc015550; Fri, 12 Nov 2010 23:08:17 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACM88L8015482 for ; Fri, 12 Nov 2010 23:08:08 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=BCwIyxg8sHn3TPff6MtJNgAsj+7dmSPZCOkGiq4/ZTjwnkldhsblXP+KIhzVUDpAxT g1uiuWIqsmUptcKl9X+Ks9DHIaQntz8FRFSQpn5VhwhvoXQ1HWCB93+7qjgfzEnXolNx 0whEwIXISD+tdX91AI9YRgjeVfbMPcv9QnvF8=; bh=yfbjbucuwipDcAhMJfYzhoxVK7w=; t=1289599689; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id f70e000535a0377e; Fri, 12 Nov 2010 17:08:08 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <20101112201356.21608.qmail@stuge.se> In-Reply-To: <20101112201356.21608.qmail@stuge.se> Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Fri, 12 Nov 2010 16:07:57 -0600 Message-ID: <000301cb82b6$10653cc0$312fb640$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuCpmm39OTu6/TFS5qCibelj/Cl2wAD48vw Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 23:08:18 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 23:08:09 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter: > I changed another part of the server to also accept 25M uploads, > please let me know if you run into trouble with this again. Thanks! Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 23:16:31 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACMGKUZ020134; Fri, 12 Nov 2010 23:16:31 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oACMGIpx020131 for ; Fri, 12 Nov 2010 23:16:18 +0100 Received: (qmail 2222 invoked by uid 501); 12 Nov 2010 22:16:14 -0000 Message-ID: <20101112221614.2221.qmail@stuge.se> Date: Fri, 12 Nov 2010 23:16:14 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: ssh_tunnel Mail-Followup-To: libssh2-devel@cool.haxx.se References: <9d76d0b087568d99cb196facc8f1fa6a@igrspace.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <9d76d0b087568d99cb196facc8f1fa6a@igrspace.com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 23:16:31 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 23:16:18 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se junk@igrspace.com wrote: > I am actually here with a usage question for the libssh2 PHP > binding rather than a devel question, feel free to redirect me. Sorry, we're not really able to help. We only do the C library. > the PHP binding does provide ssh2_tunnel() which takes remote host > and port, but it returns a PHP resource accepting fwrite(), fgets(), > etc. Unless there is something major I am missing, I believe this is > of any use for another ssh connection through it. I don't really know. It depends on what the binding does as far as connection setup goes. > Or is there a way to start a libssh2 session on an existing tcp > connection using the PHP binding ? No idea. Try to find someone who knows what the binding is doing? //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 12 23:53:48 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACMqtCG002143; Fri, 12 Nov 2010 23:53:44 +0100 Received: from ch-smtp03.sth.basefarm.net (ch-smtp03.sth.basefarm.net [80.76.149.214]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACMqrbh002135 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 12 Nov 2010 23:52:53 +0100 Received: from c83-250-116-247.bredband.comhem.se ([83.250.116.247]:40831 helo=reason) by ch-smtp03.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1PH2Tq-0000Co-BN for libssh2-devel@cool.haxx.se; Fri, 12 Nov 2010 23:52:29 +0100 From: Tommy Lindgren To: libssh2 development Subject: libssh2_channel_write_ex may incorrectly return EAGAIN User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux) Date: Fri, 12 Nov 2010 23:52:02 +0100 Message-ID: <87k4kif94d.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Originating-IP: 83.250.116.247 X-Scan-Result: No virus found in message 1PH2Tq-0000Co-BN. X-Scan-Signature: ch-smtp03.sth.basefarm.net 1PH2Tq-0000Co-BN 3779b0dbcfb68d01ac7b4587a6a9b344 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 12 Nov 2010 23:53:48 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 12 Nov 2010 23:52:53 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --=-=-= Hi, I think I have found an issue with libssh2_channel_write_ex. It seems like it can return LIBSSH2_ERROR_EAGAIN when it actually successfully wrote bytes to the socket. I suspect this happens when the send() call returns a value greater than 0 but less than the buffer length. I modified the ssh2_exec.c example to reproduce the problem. The attached example will execute 'cat' and then in a poll loop write 1.5 MB (using a bufsize of 32K) to stdin and read the reply from stdout. If the number bytes read is different from the number of bytes written, it will call exit(1). The problem is not deterministically reproducible, but the following command line while :; do ./ssh2_echo 127.0.0.1 user password || break; done terminates after a few seconds on my machine. I have tested this against the latest git code. After having looked at the code, I suspect the issue exists in 1.2.7 too but is harder to trigger. Regards, Tommy --=-=-= Content-Type: text/x-csrc Content-Disposition: inline; filename=ssh2_echo.c /* * $Id: ssh2_exec.c,v 1.4 2009/05/07 20:30:22 bagder Exp $ * * Sample showing how to use libssh2 to execute a command remotely. * * The sample code has fixed values for host name, user name, password * and command to run. * * Run it like this: * * $ ./ssh2_exec 127.0.0.1 user password "uptime" * */ #include "libssh2_config.h" #include #ifdef HAVE_WINSOCK2_H # include #endif #ifdef HAVE_SYS_SOCKET_H # include #endif #ifdef HAVE_NETINET_IN_H # include #endif #ifdef HAVE_SYS_SELECT_H # include #endif # ifdef HAVE_UNISTD_H #include #endif #ifdef HAVE_ARPA_INET_H # include #endif #include #include #include #include #include #include #include static int waitsocket(int socket_fd, LIBSSH2_SESSION *session) { struct timeval timeout; int rc; fd_set fd; fd_set *writefd = NULL; fd_set *readfd = NULL; int dir; timeout.tv_sec = 10; timeout.tv_usec = 0; FD_ZERO(&fd); FD_SET(socket_fd, &fd); /* now make sure we wait in the correct direction */ dir = libssh2_session_block_directions(session); if(dir & LIBSSH2_SESSION_BLOCK_INBOUND) readfd = &fd; if(dir & LIBSSH2_SESSION_BLOCK_OUTBOUND) writefd = &fd; rc = select(socket_fd + 1, readfd, writefd, NULL, &timeout); return rc; } int main(int argc, char *argv[]) { const char *hostname = "127.0.0.1"; const char *commandline = "cat"; const char *username = "user"; const char *password = "password"; unsigned long hostaddr; int sock; struct sockaddr_in sin; const char *fingerprint; LIBSSH2_SESSION *session; LIBSSH2_CHANNEL *channel; int rc; int exitcode; char *exitsignal=(char *)"none"; int bytecount = 0; size_t len; LIBSSH2_KNOWNHOSTS *nh; int type; #ifdef WIN32 WSADATA wsadata; WSAStartup(MAKEWORD(2,0), &wsadata); #endif if (argc > 1) /* must be ip address only */ hostname = argv[1]; if (argc > 2) { username = argv[2]; } if (argc > 3) { password = argv[3]; } rc = libssh2_init (0); if (rc != 0) { fprintf (stderr, "libssh2 initialization failed (%d)\n", rc); return 1; } hostaddr = inet_addr(hostname); /* Ultra basic "connect to port 22 on localhost" * Your code is responsible for creating the socket establishing the * connection */ sock = socket(AF_INET, SOCK_STREAM, 0); sin.sin_family = AF_INET; sin.sin_port = htons(22); sin.sin_addr.s_addr = hostaddr; if (connect(sock, (struct sockaddr*)(&sin), sizeof(struct sockaddr_in)) != 0) { fprintf(stderr, "failed to connect!\n"); return -1; } /* Create a session instance */ session = libssh2_session_init(); if (!session) return -1; /* tell libssh2 we want it all done non-blocking */ libssh2_session_set_blocking(session, 0); /* ... start it up. This will trade welcome banners, exchange keys, * and setup crypto, compression, and MAC layers */ while ((rc = libssh2_session_startup(session, sock)) == LIBSSH2_ERROR_EAGAIN); if (rc) { fprintf(stderr, "Failure establishing SSH session: %d\n", rc); return -1; } nh = libssh2_knownhost_init(session); if(!nh) { /* eeek, do cleanup here */ return 2; } /* read all hosts from here */ libssh2_knownhost_readfile(nh, "known_hosts", LIBSSH2_KNOWNHOST_FILE_OPENSSH); /* store all known hosts to here */ libssh2_knownhost_writefile(nh, "dumpfile", LIBSSH2_KNOWNHOST_FILE_OPENSSH); fingerprint = libssh2_session_hostkey(session, &len, &type); if(fingerprint) { struct libssh2_knownhost *host; #if LIBSSH2_VERSION_NUM >= 0x010206 /* introduced in 1.2.6 */ int check = libssh2_knownhost_checkp(nh, hostname, 22, fingerprint, len, LIBSSH2_KNOWNHOST_TYPE_PLAIN| LIBSSH2_KNOWNHOST_KEYENC_RAW, &host); #else /* 1.2.5 or older */ int check = libssh2_knownhost_check(nh, hostname, fingerprint, len, LIBSSH2_KNOWNHOST_TYPE_PLAIN| LIBSSH2_KNOWNHOST_KEYENC_RAW, &host); #endif fprintf(stderr, "Host check: %d, key: %s\n", check, (check <= LIBSSH2_KNOWNHOST_CHECK_MISMATCH)? host->key:""); /***** * At this point, we could verify that 'check' tells us the key is * fine or bail out. *****/ } else { /* eeek, do cleanup here */ return 3; } libssh2_knownhost_free(nh); if ( strlen(password) != 0 ) { /* We could authenticate via password */ while ((rc = libssh2_userauth_password(session, username, password)) == LIBSSH2_ERROR_EAGAIN); if (rc) { fprintf(stderr, "Authentication by password failed.\n"); exit(1); } } #if 1 libssh2_trace(session, LIBSSH2_TRACE_SOCKET); #endif /* Exec non-blocking on the remove host */ while( (channel = libssh2_channel_open_session(session)) == NULL && libssh2_session_last_error(session,NULL,NULL,0) == LIBSSH2_ERROR_EAGAIN ) { waitsocket(sock, session); } if( channel == NULL ) { fprintf(stderr,"Error\n"); exit( 1 ); } while( (rc = libssh2_channel_exec(channel, commandline)) == LIBSSH2_ERROR_EAGAIN ) { waitsocket(sock, session); } if( rc != 0 ) { fprintf(stderr,"Error\n"); exit( 1 ); } #if 0 for( ;; ) { /* loop until we block */ int rc; do { char buffer[0x4000]; rc = libssh2_channel_read( channel, buffer, sizeof(buffer) ); if( rc > 0 ) { int i; bytecount += rc; fprintf(stderr, "We read:\n"); for( i=0; i < rc; ++i ) fputc( buffer[i], stderr); fprintf(stderr, "\n"); } else { fprintf(stderr, "libssh2_channel_read returned %d\n", rc); } } while( rc > 0 ); /* this is due to blocking that would occur otherwise so we loop on this condition */ if( rc == LIBSSH2_ERROR_EAGAIN ) { waitsocket(sock, session); } else break; } #endif LIBSSH2_POLLFD *fds = NULL; int running = 1; int bufsize = 32000; char buffer[bufsize]; int totsize = 1500000; int totwritten = 0; int totread = 0; int partials = 0; int rereads = 0; int rewrites = 0; int i; for (i = 0; i < bufsize; i++) buffer[i] = 'A'; if ((fds = malloc (sizeof (LIBSSH2_POLLFD))) == NULL) { fprintf(stderr, "malloc failed\n"); exit(1); } fds[0].type = LIBSSH2_POLLFD_CHANNEL; fds[0].fd.channel = channel; fds[0].events = LIBSSH2_POLLFD_POLLIN | LIBSSH2_POLLFD_POLLOUT; do { int rc = (libssh2_poll(fds, 1, 10)); //fprintf(stderr, "poll: %d\n", rc); if (rc < 1) continue; int act = 0; //fprintf(stderr, "events: %d\n", fds[0].events); //fprintf(stderr, "revents: %d\n", fds[0].revents); if (fds[0].revents & LIBSSH2_POLLFD_POLLIN) { //fprintf(stderr, "CAN READ\n"); act++; int n = libssh2_channel_read(channel, buffer, sizeof(buffer)); if (n == LIBSSH2_ERROR_EAGAIN) { rereads++; fprintf(stderr, "will read again\n"); } else if (n < 0) { fprintf(stderr, "read failed\n"); exit(1); } else { totread += n; fprintf(stderr, "read %d bytes (%d in total)\n", n, totread); } } if (fds[0].revents & LIBSSH2_POLLFD_POLLOUT) { //fprintf(stderr, "CAN WRITE\n"); act++; if (totwritten < totsize) { // we have not written all data yet int left = totsize - totwritten; int size = (left < bufsize) ? left : bufsize; int n = libssh2_channel_write_ex(channel, 0, buffer, size); if (n == LIBSSH2_ERROR_EAGAIN) { rewrites++; fprintf(stderr, "will write again\n"); } else if (n < 0) { fprintf(stderr, "write failed\n"); exit(1); } else { totwritten += n; fprintf(stderr, "wrote %d bytes (%d in total)", n, totwritten); if (left >= bufsize && n != bufsize) { partials++; fprintf(stderr, " PARTIAL"); } fprintf(stderr, "\n"); } } else { // all data written, send EOF rc = libssh2_channel_send_eof(channel); if (rc == LIBSSH2_ERROR_EAGAIN) { fprintf(stderr, "will send eof again\n"); } else if (rc < 0) { fprintf(stderr, "send eof failed\n"); exit(1); } else { fprintf(stderr, "sent eof\n"); // we're done writing, stop listening for OUT events fds[0].events &= ~LIBSSH2_POLLFD_POLLOUT; } } } if (fds[0].revents & LIBSSH2_POLLFD_CHANNEL_CLOSED) { //fprintf(stderr, "CLOSED\n"); if (!act) // don't leave loop until we have read all data running = 0; } } while(running); exitcode = 127; while( (rc = libssh2_channel_close(channel)) == LIBSSH2_ERROR_EAGAIN ) waitsocket(sock, session); if( rc == 0 ) { exitcode = libssh2_channel_get_exit_status( channel ); libssh2_channel_get_exit_signal(channel, &exitsignal, NULL, NULL, NULL, NULL, NULL); } if (exitsignal) printf("\nGot signal: %s\n", exitsignal); libssh2_channel_free(channel); channel = NULL; //fprintf(stderr, "partials: %d\n", partials); fprintf(stderr, "\nrereads: %d rewrites: %d\n", rereads, rewrites); if (totwritten != totread) { fprintf(stderr, "\n*** FAIL bytes written: %d bytes read: %d ***\n", totwritten, totread); exit(1); } shutdown: libssh2_session_disconnect(session, "Normal Shutdown, Thank you for playing"); libssh2_session_free(session); #ifdef WIN32 closesocket(sock); #else close(sock); #endif fprintf(stderr, "all done\n"); libssh2_exit(); return 0; } --=-=-= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --=-=-=-- From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 00:14:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACNEdU4011292; Sat, 13 Nov 2010 00:14:47 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oACNEbv1011283 for ; Sat, 13 Nov 2010 00:14:37 +0100 Received: (qmail 7951 invoked by uid 501); 12 Nov 2010 23:14:33 -0000 Message-ID: <20101112231433.7950.qmail@stuge.se> Date: Sat, 13 Nov 2010 00:14:33 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN Mail-Followup-To: libssh2-devel@cool.haxx.se References: <87k4kif94d.fsf@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87k4kif94d.fsf@gmail.com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 00:14:49 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 00:14:37 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Tommy Lindgren wrote: > I think I have found an issue with libssh2_channel_write_ex. It seems > like it can return LIBSSH2_ERROR_EAGAIN when it actually successfully > wrote bytes to the socket. I suspect this happens when the send() call > returns a value greater than 0 but less than the buffer length. .. > I have tested this against the latest git code. Yeah, channel_write() git now looks weird. rc = _libssh2_transport_send(session, channel->write_packet, .. else if (rc) { channel->write_state = libssh2_NB_state_idle; return _libssh2_error(session, rc, "Unable to send channel data"); } Also, I'm not happy that only one code path leading into channel_write() actually has a debug message in _channel_write(). //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 00:29:37 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACNTN1s021947; Sat, 13 Nov 2010 00:29:35 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oACNTKGW021938 for ; Sat, 13 Nov 2010 00:29:21 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=j9+q9ok7kA2cPT/FJeOGOzWFXyHbsg0pWoIMmPeq4ft4Y26SqwhAwH3kqZ+QUEOjx1 aWrSJVoSJPgk1dHZuaO/AlgDrBSGfwNc3AoakAd9l7rzqq5Lr26hTSHwtxoZ+XDDLLxe FB3bOsYv1Z+lC5gC7Efy2eTflxwHOe4iA+rtc=; bh=uDyhnnryjYn8jztWjN2CqeOnYPU=; t=1289604561; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 382600054e833893; Fri, 12 Nov 2010 18:29:21 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> In-Reply-To: Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Fri, 12 Nov 2010 17:29:10 -0600 Message-ID: <001d01cb82c1$68f0ed70$3ad2c850$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuCrq2ugdNNGPKaRMiSA9923GE/hwAEd4Bg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 00:29:37 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 00:29:21 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel said: > For this to be able to happen, I'm thinking there's something wrong with the > EAGAIN detection for win32 and I've just now remade how it is handled: > previously we would convert from the win32 error to the errno value and store > that in errno to allow the parent function to read it. > > Starting now, we only use the _libssh2_recv and _libssh2_send functions and > they simply return the negative errno value on errors so that we don't ever > have to store anything in errno for win32 but only return the correct value. > > Commit ca2e81eb1f1bcb3 was just pushed with this change. I think we are getting somewhere. The upload fails after the test program outputs this: libssh2_sftp_write 1 sent 32000 bytes; 32000 so far libssh2_sftp_write 2 sent 500 bytes; 32500 so far libssh2_sftp_write 3 sent 32000 bytes; 64500 so far libssh2_sftp_write 4 sent 500 bytes; 65000 so far The trace log ends: [libssh2] 0.724547 SFTP: Requiring packet 101 id 18 [libssh2] 0.724547 SFTP: Asking for 101 packet [libssh2] 0.724547 SFTP: recv packet [libssh2] 0.724547 Conn: channel_read() wants 4 bytes from channel 0/0 stream #0 [libssh2] 0.724547 Socket: Recved 68/16384 bytes to 0099016C+0 [libssh2] 0.724547 Transport: Packet type 94 received, length=37 [libssh2] 0.724547 Conn: 28 bytes packet_add() for 0/0/0 [libssh2] 0.724547 Conn: channel_read() got 4 of data from 0/0/0 [libssh2] 0.724547 SFTP: Data begin - Packet Length: 24 [libssh2] 0.724547 Conn: channel_read() got 24 of data from 0/0/0 [ul] [libssh2] 0.724547 SFTP: Received packet 101 (len 24) [libssh2] 0.724547 SFTP: Asking for 101 packet [libssh2] 0.724547 SFTP: Got 101 [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 [libssh2] 0.740171 Socket: Sent 4292/4292 bytes at 00994194 [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 [libssh2] 0.740171 Socket: Sent 4292/4292 bytes at 00994194 [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 [libssh2] 0.740171 Socket: Error sending 4292 bytes: 11 I put the full trace here: http://trac.libssh2.org/attachment/ticket/197/2010-11-12b-error-11.zip but I doubt you'll need it. Thanks for all your hard work on this. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 01:16:14 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD0G2s3011766; Sat, 13 Nov 2010 01:16:12 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAD0G0u3011736 for ; Sat, 13 Nov 2010 01:16:00 +0100 Received: (qmail 14353 invoked by uid 501); 13 Nov 2010 00:15:55 -0000 Message-ID: <20101113001555.14352.qmail@stuge.se> Date: Sat, 13 Nov 2010 01:15:55 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write doesn't hang anymore Mail-Followup-To: libssh2-devel@cool.haxx.se References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <001d01cb82c1$68f0ed70$3ad2c850$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <001d01cb82c1$68f0ed70$3ad2c850$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 01:16:14 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 01:16:00 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > I think we are getting somewhere. > The upload fails after the test program outputs this: > libssh2_sftp_write 1 sent 32000 bytes; 32000 so far > libssh2_sftp_write 2 sent 500 bytes; 32500 so far > libssh2_sftp_write 3 sent 32000 bytes; 64500 so far > libssh2_sftp_write 4 sent 500 bytes; 65000 so far > > The trace log ends: .. > [libssh2] 0.724547 SFTP: Received packet 101 (len 24) > [libssh2] 0.724547 SFTP: Asking for 101 packet > [libssh2] 0.724547 SFTP: Got 101 > [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 > [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 > [libssh2] 0.740171 Socket: Sent 4292/4292 bytes at 00994194 > [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 > [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 > [libssh2] 0.740171 Socket: Sent 4292/4292 bytes at 00994194 > [libssh2] 0.740171 Conn: Writing 4029 bytes on channel 0/0, stream #0 > [libssh2] 0.740171 Conn: Sending 4029 bytes on channel 0/0, stream_id=0 > [libssh2] 0.740171 Socket: Error sending 4292 bytes: 11 That's WSAEWOULDBLOCK or EAGAIN.. > I put the full trace here: > http://trac.libssh2.org/attachment/ticket/197/2010-11-12b-error-11.zip > but I doubt you'll need it. Let's see.. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 01:26:58 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD0QqEN019706; Sat, 13 Nov 2010 01:26:58 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAD0QocI019702 for ; Sat, 13 Nov 2010 01:26:50 +0100 Received: (qmail 15360 invoked by uid 501); 13 Nov 2010 00:26:46 -0000 Message-ID: <20101113002646.15359.qmail@stuge.se> Date: Sat, 13 Nov 2010 01:26:46 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write doesn't hang anymore Mail-Followup-To: libssh2-devel@cool.haxx.se References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <001d01cb82c1$68f0ed70$3ad2c850$@com> <20101113001555.14352.qmail@stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20101113001555.14352.qmail@stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 01:26:58 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 01:26:50 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter Stuge wrote: > > I think we are getting somewhere. .. > > I put the full trace here: > > http://trac.libssh2.org/attachment/ticket/197/2010-11-12b-error-11.zip > > but I doubt you'll need it. > > Let's see.. There's a buffer overrun error in libssh2 now. Am hunting it down. Also, how did you generate trace.log? It has been corrupted in a few places. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 01:40:00 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD0doqv023906; Sat, 13 Nov 2010 01:39:59 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD0dmH4023889 for ; Sat, 13 Nov 2010 01:39:49 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=mBLChSA22Y0sfcAhrEjq0FLEnLUSzakq1/w07bS5M3tc56VUMVIsi+q9U9LRMdHy3P uostgpUP7h/vEx72ytvat/3TBwQs0V1Fec1S3UF/RqaGzd3FXW31TiJFTCgBiVd32WJg kOyTnFkJokkbioVHjakEfUqdoKypUgzooam+g=; bh=bpCGvb5KiT8eYhhFmOkD1jx0AZs=; t=1289608789; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id e75e000560a338ed; Fri, 12 Nov 2010 19:39:45 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <005201cb8135$ed7a5be0$c86f13a0$@com> <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <001d01cb82c1$68f0ed70$3ad2c850$@com> <20101113001555.14352.qmail@stuge.se> <20101113002646.15359.qmail@stuge.se> In-Reply-To: <20101113002646.15359.qmail@stuge.se> Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Fri, 12 Nov 2010 18:39:32 -0600 Message-ID: <001e01cb82cb$4005b940$c0112bc0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuCybxWcR2SN7aIQ2+cSX/MkfpZ9wAARn4g Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 01:40:00 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 01:39:49 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter: > There's a buffer overrun error in libssh2 now. Am hunting it down. I appreciate it. > Also, how did you generate trace.log? It has been corrupted > in a few places. I did this: int bitmask = ~0; libssh2_trace(session, bitmask); and then I just redirected standard error from the command line. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 02:08:23 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD17tZA013734; Sat, 13 Nov 2010 02:08:22 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAD17rjc013664 for ; Sat, 13 Nov 2010 02:07:53 +0100 Received: (qmail 19355 invoked by uid 501); 13 Nov 2010 01:07:48 -0000 Message-ID: <20101113010748.19354.qmail@stuge.se> Date: Sat, 13 Nov 2010 02:07:48 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write doesn't hang anymore Mail-Followup-To: libssh2-devel@cool.haxx.se References: <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <001d01cb82c1$68f0ed70$3ad2c850$@com> <20101113001555.14352.qmail@stuge.se> <20101113002646.15359.qmail@stuge.se> <001e01cb82cb$4005b940$c0112bc0$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <001e01cb82cb$4005b940$c0112bc0$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 02:08:23 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 02:07:53 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > > There's a buffer overrun error in libssh2 now. Am hunting it down. > > I appreciate it. Have pushed a commit which I think may be sufficient to fix the problem. > > Also, how did you generate trace.log? It has been corrupted > > in a few places. > > I did this: > > int bitmask = ~0; > libssh2_trace(session, bitmask); > > and then I just redirected standard error from the command line. Some of the corruption (and indeed the last 11 error message) was caused by the debug printout running over a buffer, and printing old debug messages. That logic should be fixed, but there was also some more corruption in the log which I don't really understand. Maybe it is related to write() behavior on Windows. Let's see.. I also pushed a patch to use fprintf() instead of write() for debugging. Maybe it helps. Please try latest git source. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 02:38:04 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD1brjl028534; Sat, 13 Nov 2010 02:38:03 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD1bps3028516 for ; Sat, 13 Nov 2010 02:37:51 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=Y6lC3v2Oq7X29s7UK9xNGMy51J/L/21xaockAZNghCk4Qq4hdxAbmSUzbMSSo+HVlc PbQT8FR1q8gVVE540/dhUTXtq+h8CLrT50kqYB5fTUiOXeebckzm2X/egR+qYWDLlkJI kbZcUiVg13mK66rXxLhwivSJR7YENpObrrCEY=; bh=GkJbuWaF1P5lHU6+ZQRfjxnvjjM=; t=1289612272; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id cf42000570513933; Fri, 12 Nov 2010 20:37:51 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <002f01cb81ce$ee226a30$ca673e90$@com> <000301cb8211$c3dc4a00$4b94de00$@com> <001f01cb82a3$74feb200$5efc1600$@com> <001d01cb82c1$68f0ed70$3ad2c850$@com> <20101113001555.14352.qmail@stuge.se> <20101113002646.15359.qmail@stuge.se> <001e01cb82cb$4005b940$c0112bc0$@com> <20101113010748.19354.qmail@stuge.se> In-Reply-To: <20101113010748.19354.qmail@stuge.se> Subject: RE: libssh2_sftp_write doesn't hang anymore Date: Fri, 12 Nov 2010 19:37:36 -0600 Message-ID: <001f01cb82d3$5cc64880$1652d980$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuCz4t5UUGkel++RDul4zc2zDub7AAAipjg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 02:38:04 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 02:37:52 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter: > Have pushed a commit which I think may be sufficient to fix the problem. You fixed it! However, we are now roughly back to where we were: Uploads with trace enabled are pretty fast, and result in reasonably-sized Wireshark captures: C:\Libs\ssh\libssh2-git\tests>ntimer sftpmrr -a:put -h:172.16.22.182 -port:22 -u :mrr -p:secret -r:/home/mrr/tmp/myupload -f:c:\temp\mrr10000.txt -debug:3 2>o ut.log Not enabling compression. Authentication methods: publickey,password Using output buffer size of 32500 bytes libssh2_sftp_write 1 sent 8000 bytes; 8000 so far libssh2_sftp_write 2 sent 24000 bytes; 32000 so far libssh2_sftp_write 3 sent 500 bytes; 32500 so far libssh2_sftp_write 4 sent 8000 bytes; 40500 so far libssh2_sftp_write 5 sent 24000 bytes; 64500 so far libssh2_sftp_write 6 sent 500 bytes; 65000 so far libssh2_sftp_write 7 sent 32000 bytes; 97000 so far libssh2_sftp_write 8 sent 500 bytes; 97500 so far libssh2_sftp_write 9 sent 20000 bytes; 117500 so far libssh2_sftp_write 10 sent 2500 bytes; 120000 so far Transferred 0.114 MB put took 0.063 seconds = 1.817 MB/sec Capture file: 11/12/2010 07:29 PM 210,405 DebugFast2010-11-12.pcap Uploads with no tracing enabled are slow, and result in huge captures: C:\Libs\ssh\libssh2-git\tests>ntimer sftpmrr -a:put -h:172.16.22.182 -port:22 -u :mrr -p:secret -r:/home/mrr/tmp/myupload -f:c:\temp\mrr10000.txt -debug:2 2>o ut2.log Not enabling compression. Authentication methods: publickey,password Using output buffer size of 32500 bytes libssh2_sftp_write 1 sent 4000 bytes; 4000 so far libssh2_sftp_write 2 sent 4000 bytes; 8000 so far libssh2_sftp_write 3 sent 4000 bytes; 12000 so far libssh2_sftp_write 4 sent 4000 bytes; 16000 so far libssh2_sftp_write 5 sent 4000 bytes; 20000 so far libssh2_sftp_write 6 sent 8000 bytes; 28000 so far libssh2_sftp_write 7 sent 4500 bytes; 32500 so far libssh2_sftp_write 8 sent 8000 bytes; 40500 so far libssh2_sftp_write 9 sent 4000 bytes; 44500 so far libssh2_sftp_write 10 sent 8000 bytes; 52500 so far libssh2_sftp_write 11 sent 8000 bytes; 60500 so far libssh2_sftp_write 12 sent 4500 bytes; 65000 so far libssh2_sftp_write 13 sent 8000 bytes; 73000 so far libssh2_sftp_write 14 sent 8000 bytes; 81000 so far libssh2_sftp_write 15 sent 8000 bytes; 89000 so far libssh2_sftp_write 16 sent 4000 bytes; 93000 so far libssh2_sftp_write 17 sent 4500 bytes; 97500 so far libssh2_sftp_write 18 sent 8000 bytes; 105500 so far libssh2_sftp_write 19 sent 8000 bytes; 113500 so far libssh2_sftp_write 20 sent 4000 bytes; 117500 so far libssh2_sftp_write 21 sent 2500 bytes; 120000 so far Transferred 0.114 MB put took 2.656 seconds = 0.043 MB/sec Capture file: 11/12/2010 07:30 PM 20,330,815 NoDebugSlow2010-11-12.pcap Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 10:56:54 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD9t7sH031433; Sat, 13 Nov 2010 10:56:50 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAD9t63W031421 for ; Sat, 13 Nov 2010 10:55:06 +0100 Date: Sat, 13 Nov 2010 10:55:06 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: <20101112231433.7950.qmail@stuge.se> Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 10:56:54 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 10:55:06 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Peter Stuge wrote: > Yeah, channel_write() git now looks weird. > > rc = _libssh2_transport_send(session, channel->write_packet, > .. > else if (rc) { > channel->write_state = libssh2_NB_state_idle; > return _libssh2_error(session, rc, > "Unable to send channel data"); > } I'm sorry, why is that weird? I have a different explanation to the problem and that is quite simply that_libssh2_channel_write() returns EAGAIN instead of a short return when the entire buffer isn't sent in the first case, which can lead to a "loss" of data returned. I'll get to this later today I hope. > Also, I'm not happy that only one code path leading into channel_write() > actually has a debug message in _channel_write(). I'm not sure exactly what you refer to, but feel free to clean up! BTW, thanks a lot for jumping in and helping out! -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 12:09:17 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADB94jT012023; Sat, 13 Nov 2010 12:09:16 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oADB92IF011945 for ; Sat, 13 Nov 2010 12:09:02 +0100 Received: (qmail 18298 invoked by uid 501); 13 Nov 2010 11:09:00 -0000 Message-ID: <20101113110900.18297.qmail@stuge.se> Date: Sat, 13 Nov 2010 12:09:00 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN Mail-Followup-To: libssh2-devel@cool.haxx.se References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 12:09:17 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 12:09:02 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel Stenberg wrote: >> Yeah, channel_write() git now looks weird. >> >> rc = _libssh2_transport_send(session, channel->write_packet, >> .. >> else if (rc) { >> channel->write_state = libssh2_NB_state_idle; >> return _libssh2_error(session, rc, >> "Unable to send channel data"); >> } > > I'm sorry, why is that weird? Yeah, no, looking more closely at how _transport_send() works this is fine. I was confusing it with how _send() works, returning number of bytes rather than 0 for success. > I have a different explanation to the problem and that is quite simply > that_libssh2_channel_write() returns EAGAIN instead of a short return when > the entire buffer isn't sent in the first case, which can lead to a "loss" > of data returned. That's what I thought the problem was above. But maybe it is there in a different place. It fits with the error description. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 12:15:08 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADBF1ha015129; Sat, 13 Nov 2010 12:15:07 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADBExTv015041 for ; Sat, 13 Nov 2010 12:14:59 +0100 Date: Sat, 13 Nov 2010 12:14:59 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: <20101113110900.18297.qmail@stuge.se> Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 12:15:08 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 12:14:59 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Peter Stuge wrote: >> I have a different explanation to the problem and that is quite simply >> that_libssh2_channel_write() returns EAGAIN instead of a short return when >> the entire buffer isn't sent in the first case, which can lead to a "loss" >> of data returned. > > That's what I thought the problem was above. But maybe it is there in a > different place. It fits with the error description. Tommy's example code is an EXCELLENT helper to track this down. I found one bug that reduced the problem but I've still not fixed it completely... -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 12:31:03 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADBUtqO027738; Sat, 13 Nov 2010 12:31:02 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADBUtrZ027733 for ; Sat, 13 Nov 2010 12:30:55 +0100 Date: Sat, 13 Nov 2010 12:30:55 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 12:31:03 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 12:30:55 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Daniel Stenberg wrote: > Tommy's example code is an EXCELLENT helper to track this down. I found one > bug that reduced the problem but I've still not fixed it completely... Okay, I found another bug now. I've fixed both and reran Tommy's code many times and it now seems to be fully functional. I just pushed my changes. I'm sure these flaws also affect Mark Riordan's case and explains some/all of the resends he has seen. Thanks Tommy for the great recipe! I'll make it into an example code so that we can use that for further testing. You ok with that? -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 12:38:08 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADBc5BP031514; Sat, 13 Nov 2010 12:38:08 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oADBc31O031489 for ; Sat, 13 Nov 2010 12:38:03 +0100 Received: (qmail 24503 invoked by uid 501); 13 Nov 2010 11:38:02 -0000 Message-ID: <20101113113802.24502.qmail@stuge.se> Date: Sat, 13 Nov 2010 12:38:02 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN Mail-Followup-To: libssh2-devel@cool.haxx.se References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 12:38:08 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 12:38:03 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel Stenberg wrote: > Okay, I found another bug now. send_existing() strikes again. Has been lots of bugs in there.. > I'm sure these flaws also affect Mark Riordan's case and explains > some/all of the resends he has seen. Yes, I also think this fixes it. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 16:04:35 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADF3VoA020201; Sat, 13 Nov 2010 16:04:31 +0100 Received: from ch-smtp02.sth.basefarm.net (ch-smtp02.sth.basefarm.net [80.76.149.213]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADF3TMg020169 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sat, 13 Nov 2010 16:03:29 +0100 Received: from c83-250-116-247.bredband.comhem.se ([83.250.116.247]:42486 helo=reason) by ch-smtp02.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1PHHcJ-0001HB-81 for libssh2-devel@cool.haxx.se; Sat, 13 Nov 2010 16:02:13 +0100 From: Tommy Lindgren To: libssh2 development Subject: Re: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: (Daniel Stenberg's message of "Sat, 13 Nov 2010 12:30:55 +0100 (CET)") References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux) Date: Sat, 13 Nov 2010 16:01:46 +0100 Message-ID: <87fwv5fesl.fsf@gmail.com> MIME-Version: 1.0 X-Originating-IP: 83.250.116.247 X-Scan-Result: No virus found in message 1PHHcJ-0001HB-81. X-Scan-Signature: ch-smtp02.sth.basefarm.net 1PHHcJ-0001HB-81 881344c9d5f86e48ced969020f474f95 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 16:04:35 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 16:03:29 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel Stenberg writes: > On Sat, 13 Nov 2010, Daniel Stenberg wrote: > >> Tommy's example code is an EXCELLENT helper to track this down. I >> found one bug that reduced the problem but I've still not fixed it >> completely... > > Okay, I found another bug now. I've fixed both and reran Tommy's code > many times and it now seems to be fully functional. I just pushed my > changes. Thanks! Looks fine here too now. > I'm sure these flaws also affect Mark Riordan's case and explains > some/all of the resends he has seen. > > Thanks Tommy for the great recipe! I'll make it into an example code > so that we can use that for further testing. You ok with that? Absolutely, I'm glad it was useful. Regards, Tommy _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 17:45:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADGjQWL005547; Sat, 13 Nov 2010 17:45:41 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADGjLAb005496 for ; Sat, 13 Nov 2010 17:45:23 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=J7jQ+0AXPDkA3QYsSuskbi0l3sxYaENYXq5rWCOnmyzAPCNOe9aK57pHzT0oijAGeL xxjcDOcWyf4+GVTH6Qak6Mtk4mVQeNohirv7vKAA7H4Xej8oVXITOJwR7MSUD0JNr1Eu 1xr5v8bCh46zxPoyp0fjhuWIO/NO3c0AYp/no=; bh=CBIcCX/ANNNygfkCcwfCReIt7gM=; t=1289666723; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 6a7a0006c1c83ca2; Sat, 13 Nov 2010 11:45:21 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> In-Reply-To: Subject: RE: libssh2_channel_write_ex may incorrectly return EAGAIN Date: Sat, 13 Nov 2010 10:45:12 -0600 Message-ID: <000001cb8352$247a8fc0$6d6faf40$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuDJoViMTkYx9F8RZCKFE+ZbuKXvQAId3Hw Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 17:45:44 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 17:45:24 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel: > I'm sure these flaws also affect Mark Riordan's case and explains > some/all of the resends he has seen. My program now terminates with an access violation (segfault to you UNIX types) after a few writes. Access vios occur at various places: msvcr90d.dll!memcpy(unsigned char * dst=0x009c41c2, unsigned char * src=0x009d501e, unsigned long count=32700) Line 271 Asm > sftpmrr.exe!_libssh2_transport_send(_LIBSSH2_SESSION * session=0x009c0068, const unsigned char * data=0x009cef68, unsigned int data_len=9, const unsigned char * data2=0x009d501e, unsigned int data2_len=32700) Line 739 + 0x1b bytes C sftpmrr.exe!channel_write(_LIBSSH2_CHANNEL * channel=0x009ced60, int stream_id=0, const unsigned char * buf=0x009d501e, unsigned int buflen=32700) Line 2050 + 0x2b bytes C sftpmrr.exe!_libssh2_channel_write(_LIBSSH2_CHANNEL * channel=0x009ced60, int stream_id=0, const unsigned char * buf=0x009d501e, unsigned int buflen=4294963267) Line 2140 + 0x15 bytes C sftpmrr.exe!sftp_write(_LIBSSH2_SFTP_HANDLE * handle=0x003bf380, const char * buffer=0x0011f9cc, unsigned int count=0) Line 1504 + 0x20 bytes C sftpmrr.exe!libssh2_sftp_write(_LIBSSH2_SFTP_HANDLE * hnd=0x003bf380, const char * buffer=0x00117ad8, unsigned int count=32500) Line 1607 + 0x11 bytes C sftpmrr.exe!doMain(struct_settings settings={...}) Line 442 + 0x21 bytes C++ sftpmrr.exe!main(int argc=10, char * * argv=0x003b7028) Line 520 + 0x25 bytes C++ This is in a call to memcpy in transport.c line 739, in _libssh2_transport_send: if(data2 && data2_len) memcpy(&p->outbuf[5+data_len], data2, data2_len); Looking at variables: ? data_len 9 ? data2_len 32700 --------------- I noticed this in the trace log in a second run (of a release buld) that simply terminated with no explanation: [libssh2] 0.848116 SFTP: Asking for 101 packet [libssh2] 0.848116 SFTP: Got 101 [libssh2] 0.848116 Conn: Writing -4029 bytes on channel 0/0, stream #0 [libssh2] 0.848116 Conn: Sending 32700 bytes on channel 0/0, stream_id=0 => libssh2_transport_write plain (9 bytes) 0000: 5E 00 00 00 00 00 00 7F BC : ^........ => libssh2_transport_write plain2 (32700 bytes) 0000: 3C 00 28 02 3C 00 28 02 3C 00 30 02 3C 00 30 02 : <.(.<.(.<.0.<.0. ---------------- On a later run with tracing enabled, it blew up at transport.c line 89 in debugdump: if (i + c < size) { buffer[used++] = hex_chars[(ptr[i+c] >> 4) & 0xF]; The error was in reading memory at 0x003c3000. ?used 12 ?i 14400 ?c 2 ?width 16 ?size 32700 ?ptr 0x003bf7be I uploaded the trace log from this last run to: http://trac.libssh2.org/attachment/ticket/197/2010-11-13-access-vio.zip I am going to be busy most of the rest of the day, but I may take a deeper look at this later. Thanks! Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 22:06:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADL65Ob011875; Sat, 13 Nov 2010 22:06:45 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADL64uE011853 for ; Sat, 13 Nov 2010 22:06:04 +0100 Date: Sat, 13 Nov 2010 22:06:04 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: <000001cb8352$247a8fc0$6d6faf40$@com> Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 22:06:49 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 22:06:04 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Mark Riordan wrote: > My program now terminates with an access violation (segfault to you UNIX > types) after a few writes. Access vios occur at various places: Thanks for your patience and I'm sorry we go this back and forth between all sorts of weird bugs and errors, but I do believe we will end up with a better design and better code in the end. > [libssh2] 0.848116 Conn: Writing -4029 bytes on channel 0/0, stream #0 I think this is the key to the problems and I also think I know why this happens. _libssh2_channel_write() can now (since my bug fix from a few hours ago) return more data than what is passed in to it (if a previous call got EAGAIN) and when it does that, it may confuse the caller greatly and might trigger a counter wrap into negative. Tricky. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 13 23:34:56 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADMYeCv008866; Sat, 13 Nov 2010 23:34:54 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oADMYcF6008843 for ; Sat, 13 Nov 2010 23:34:38 +0100 Date: Sat, 13 Nov 2010 23:34:38 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: libssh2_channel_write_ex may incorrectly return EAGAIN In-Reply-To: Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 13 Nov 2010 23:34:56 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 13 Nov 2010 23:34:39 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Daniel Stenberg wrote: > _libssh2_channel_write() can now (since my bug fix from a few hours ago) > return more data than what is passed in to it (if a previous call got EAGAIN) Crap. I've now reverted the approach to have _libssh2_channel_write() split data into smaller chunks and call a sub function to send them out. I think it got a bit faster by doing so, but it turned out too ugly to do and still keep the API working. I hope this fixes Mark's most recent crashes. My tests still run just as good as before at least. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 14 02:38:50 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE1c7uh013914; Sun, 14 Nov 2010 02:38:47 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE1c5Je013843 for ; Sun, 14 Nov 2010 02:38:06 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=UprI1WD6TATtD12o9mLw6UGRIGM8zmmqzkB1bU4wJ8o2z+YlaigiNZHtzllZu5YpB4 Jz3K892HpKnI/bCm/cisRVSYIFwOIBf5f9hEr+t4F8GFie7sazWMAm1iW5rvEHDnYuCT rrR2KqER4vnD7iF6uTo2OwnC2AjQjt/HGObO0=; bh=d37u2Mh1Es+Yixmar+bG4xQ92ag=; t=1289698683; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 49a8000764313e56; Sat, 13 Nov 2010 20:38:02 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> In-Reply-To: Subject: sftp uploads greatly improved Date: Sat, 13 Nov 2010 19:37:54 -0600 Message-ID: <000101cb839c$8f26e9a0$ad74bce0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuDg1y8ed+KYGNMTW+PM88KQ/Pa3AAF9FdA Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 14 Nov 2010 02:38:50 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 14 Nov 2010 02:38:06 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se > Thanks for your patience ... No problemo. Thanks for your admirable persistence. > I've now reverted ... > I hope this fixes Mark's most recent crashes. It did, and now uploads work correctly, and fairly quickly! C:\Libs\ssh\libssh2-git\tests>ntimer sftpmrr -a:put -h:172.16.22.182 -port:22 -u :mrr -p:secret -r:/home/mrr/tmp/myupload -f:c:\temp\DMZ_6001.exe -debug:0 2>t race.log Not enabling compression. Authentication methods: publickey,password Using output buffer size of 32500 bytes libssh2_sftp_write 1000 sent 4000 bytes; 4054000 so far libssh2_sftp_write 2000 sent 4000 bytes; 8254500 so far libssh2_sftp_write 3000 sent 4000 bytes; 12402500 so far libssh2_sftp_write 4000 sent 8000 bytes; 16583000 so far libssh2_sftp_write 5000 sent 4000 bytes; 20726500 so far libssh2_sftp_write 6000 sent 500 bytes; 24862500 so far libssh2_sftp_write 7000 sent 4000 bytes; 29091500 so far libssh2_sftp_write 8000 sent 8000 bytes; 33231000 so far libssh2_sftp_write 9000 sent 4000 bytes; 37496500 so far libssh2_sftp_write 10000 sent 4000 bytes; 41673000 so far libssh2_sftp_write 11000 sent 500 bytes; 45792500 so far libssh2_sftp_write 12000 sent 500 bytes; 49952500 so far libssh2_sftp_write 13000 sent 8000 bytes; 54120500 so far libssh2_sftp_write 14000 sent 4000 bytes; 58325000 so far Transferred 59.247 MB put took 13.765 seconds = 4.304 MB/sec ContextSwitches - 75051 First level fills = 0 Second level fills = 0 ETime( 0:00:14.124 ) UTime( 0:00:02.203 ) KTime( 0:00:00.890 ) ITime( 0:00:12.218 ) Not as fast as most other clients, but much closer than before. I checked the file on the destination with md5sum and it arrived OK. I tried one test with a get and it was faster: Transferred 59.247 MB get took 9.641 seconds = 6.145 MB/sec but I haven't tried with over a network with delays. If I remember correctly, the multiple-packets-pending optimization hasn't yet been applied to downloads. In the next few days, I'm gonna try downloads over the Internet, plus test on Linux and maybe even Solaris instead of Windows. Thanks! Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 14 03:27:57 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE2RmQP032762; Sun, 14 Nov 2010 03:27:57 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAE2RlVo032747 for ; Sun, 14 Nov 2010 03:27:47 +0100 Received: (qmail 1515 invoked by uid 501); 14 Nov 2010 02:27:43 -0000 Message-ID: <20101114022743.1514.qmail@stuge.se> Date: Sun, 14 Nov 2010 03:27:43 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: sftp uploads greatly improved Mail-Followup-To: libssh2-devel@cool.haxx.se References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> <000101cb839c$8f26e9a0$ad74bce0$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <000101cb839c$8f26e9a0$ad74bce0$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 14 Nov 2010 03:27:57 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 14 Nov 2010 03:27:47 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > put took 13.765 seconds = 4.304 MB/sec > get took 9.641 seconds = 6.145 MB/sec These are already pretty nice numbers for libssh2! > but I haven't tried with over a network with delays. Loopback on Windows? > If I remember correctly, the multiple-packets-pending optimization > hasn't yet been applied to downloads. And removed from uploads, because it is tricky to keep the API working like it used to with that optimization. Maybe we should just introduce new API to get the better performance. > In the next few days, I'm gonna try downloads over the Internet, > plus test on Linux and maybe even Solaris instead of Windows. Looking forward to those test results. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 14 03:35:43 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE2Zdbr018560; Sun, 14 Nov 2010 03:35:43 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE2ZaaJ018420 for ; Sun, 14 Nov 2010 03:35:37 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=ZyTsL804O/S5Z5HuSfhoPbP3fl+rh6pT9pKMpmlRK9E0JeMxy+OPzXcbJviPp0cHFQ dNaSm45jIRa0r6Cdf0WDqo8Yy6f1NK8uXROtKmQBoqLSdznVFu92C7i4EN+MSBTz3Av8 vZIbi5UDQKsSFkOPjeyZMsP1NKKRhyAlEtVWw=; bh=aLe0FL6AWwr9kEbD7/jgesgeQuM=; t=1289702135; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 75ca0007718a3e7e; Sat, 13 Nov 2010 21:35:34 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> <000101cb839c$8f26e9a0$ad74bce0$@com> <20101114022743.1514.qmail@stuge.se> In-Reply-To: <20101114022743.1514.qmail@stuge.se> Subject: RE: sftp uploads greatly improved Date: Sat, 13 Nov 2010 20:35:26 -0600 Message-ID: <000201cb83a4$989de170$c9d9a450$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuDo9BuDF9xjYfXRY6wBOcxCwcE9QAAENeQ Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 14 Nov 2010 03:35:43 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 14 Nov 2010 03:35:37 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter said: >> but I haven't tried with over a network with delays. > Loopback on Windows? Watch out - Bill Gates knows where you live. >> If I remember correctly, the multiple-packets-pending optimization >> hasn't yet been applied to downloads. > And removed from uploads, because it is tricky to keep the API > working like it used to with that optimization. Maybe we should > just introduce new API to get the better performance. Oh, I didn't catch on that that is what was removed. Well, I'd certainly be very interested in a new API to address this. I may not be able to use libssh2 without this optimization, due to performance expectations from my users. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 14 09:22:52 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE8M8YC006795; Sun, 14 Nov 2010 09:22:48 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAE8M6JN006764 for ; Sun, 14 Nov 2010 09:22:06 +0100 Date: Sun, 14 Nov 2010 09:22:06 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: RE: sftp uploads greatly improved In-Reply-To: <000201cb83a4$989de170$c9d9a450$@com> Message-ID: References: <87k4kif94d.fsf@gmail.com> <20101112231433.7950.qmail@stuge.se> <20101113110900.18297.qmail@stuge.se> <000001cb8352$247a8fc0$6d6faf40$@com> <000101cb839c$8f26e9a0$ad74bce0$@com> <20101114022743.1514.qmail@stuge.se> <000201cb83a4$989de170$c9d9a450$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 14 Nov 2010 09:22:52 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 14 Nov 2010 09:22:06 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sat, 13 Nov 2010, Mark Riordan wrote: >> And removed from uploads, because it is tricky to keep the API working like >> it used to with that optimization. Maybe we should just introduce new API >> to get the better performance. > > Oh, I didn't catch on that that is what was removed. Well, I'd certainly be > very interested in a new API to address this. I may not be able to use > libssh2 without this optimization, due to performance expectations from my > users. No let me clarify: the multiple outgoing SFTP packets concept is still there and should make SFTP uploads faster than previous libssh2 versions. I want to make the same logic in the SFTP read code later on. What I reverted last night was the channel-sending function's ability to loop over a larger channel packet and send it off in several smaller transport-packets. I tried that as a part of my general libssh2 optimizations. It's GREAT to see that we finally are again working good on Windows again! -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 15 17:50:46 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAFGoIaw006043; Mon, 15 Nov 2010 17:50:42 +0100 Received: from mx2.belnet.be (mx2.belnet.be [193.190.198.13]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAFGoGgL006018 for ; Mon, 15 Nov 2010 17:50:16 +0100 X-IronPort-AV: E=Sophos;i="4.59,200,1288566000"; d="scan'208";a="1176685" Received: from firewall.belnet.be (HELO relay.fw.belnet.be) ([193.190.198.37]) by mx2-out.belnet.be with ESMTP; 15 Nov 2010 17:50:08 +0100 Received: from [192.168.2.3] (unknown [79.132.236.185]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay.fw.belnet.be (Postfix) with ESMTPSA id 311FD49; Mon, 15 Nov 2010 17:50:34 +0100 (CET) Message-ID: <4CE164B9.2040501@0xbadc0de.be> Date: Mon, 15 Nov 2010 17:50:01 +0100 From: Aris Adamantiadis User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Thunderbird/3.0.10 MIME-Version: 1.0 To: libssh@libssh.org Subject: Re: SSH library hacker meetup at FOSDEM? References: In-Reply-To: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 15 Nov 2010 17:50:46 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 15 Nov 2010 17:50:16 +0100 (CET) Cc: libssh2 development , Daniel Stenberg X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hi Daniel, Took me some time to write an answer but I'm at it now. I'm going to be at Fosdem, and maybe Andreas. I think having a little face to face would be a very nice idea. I was planning to present a lightning talk at fosdem, but giving a talk in the devroom is interesting too. I think the presentation would be a little different, since the public would be too (lightning talk has more an advertisement value, you cannot be technical in 15 minutes). I'll keep you posted. Aris Le 09/11/10 13:42, Daniel Stenberg a =E9crit : > Hi friends! > = > (This mail is cross-posted to both the libssh and the libssh2 mailing > lists, as I believe most people are just subscribed to one of them.) > = > I've previously talked briefly with a few guys in #libssh (on freenode) > and I talked with Peter Stuge (of libssh2) this weekend, so this isn't > coming completely out of the blue for everyone... > = > How about a little face to face SSH library hacker meetup at FOSDEM? I > believe the "Security / hardware crypto" devroom sounds like a perfect > place for it. > = > Possibly, we could arrange for one presentation each to present our > libraries to each other and to the crowd, for the fun of it and for the > education value. > = > I don't have any other intentions behind this suggestion. Meeting in > person puts a face to a name and since we're not that many persons > actively working on open source SSH libraries it could be cool to get to > know each other a bit more. I can only see benefits from that! > = > Of course everyone and anyone would be invited to join. > = _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 17 10:51:08 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAH9oiJw021935; Wed, 17 Nov 2010 10:51:04 +0100 Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAH9ofT8021900 for ; Wed, 17 Nov 2010 10:50:42 +0100 Received: by pzk1 with SMTP id 1so379608pzk.41 for ; Wed, 17 Nov 2010 01:50:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:x-enigmail-version:content-type; bh=SAuKbGQkQH+sIaLdynFZVXNtbjIZByepnIkHldGkX5k=; b=aVEbgKI+Xctf5tQaAytvFcirivJ8g81VKo1IDWoWQcus8z1Q19vYADWUKNu4H1NBNg zX9yqz5qiqzQpMdXqeTEM+PI2/xu9e4Wjvli4nvVpleP1hbcv6ZJRyXic1MZSylKD/FV ShbwQZuw7EObhRq3x2SU3AT7wSIoqswmc2f60= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type; b=V0ZWhQW11R3SBI/2ojokfTmBh0is0nGdmLRQHKgTRR3mr8no2x5FkiUUVMwZmmihhM eZ2g0+mHwv2sfKX13kNOPPcxRksbnKCHfV7NDE3RsH+6+A9mbXKRb0WYvkB2vy5Gb1XA muMiD5UNPRGudUPvqiAqGkexs2FeAVA/CLNuU= Received: by 10.142.234.10 with SMTP id g10mr6704180wfh.436.1289987435299; Wed, 17 Nov 2010 01:50:35 -0800 (PST) Received: from [192.168.30.64] ([113.108.186.165]) by mx.google.com with ESMTPS id q13sm2559482wfc.5.2010.11.17.01.50.33 (version=SSLv3 cipher=RC4-MD5); Wed, 17 Nov 2010 01:50:34 -0800 (PST) Message-ID: <4CE3A55C.5050004@gmail.com> Date: Wed, 17 Nov 2010 17:50:20 +0800 From: Dongsheng Song User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: libssh2-devel@cool.haxx.se Subject: Bring up to date windows build environment X-Enigmail-Version: 1.1.1 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 17 Nov 2010 10:51:08 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 17 Nov 2010 10:50:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1310655664==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============1310655664== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig24F8D6DFCC186A89DCFFB002" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig24F8D6DFCC186A89DCFFB002 Content-Type: multipart/mixed; boundary="------------070400010302030903090301" This is a multi-part message in MIME format. --------------070400010302030903090301 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, Windows build environment is not up to date, missing 4 new C source files= , and VC 6.0 was broken now. Tested at VC 6.0, VC 2008 x86 and x64, VC 2010 x86 and x64. Please pull from git://github.com/songdongsheng/libssh2.git Regards, Dongsheng --------------070400010302030903090301 Content-Type: text/plain; name="0002-Bring-up-to-date-windows-build-environment.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0002-Bring-up-to-date-windows-build-environment.patch" RnJvbSBiZjlkY2Q5MmM3YjczZGE2ZDJlMTM0ZTRmZWRhM2NiZmY4MjA0ZmMwIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBEb25nc2hlbmcgU29uZyA8ZG9uZ3NoZW5nLnNvbmdA Z21haWwuY29tPgpEYXRlOiBXZWQsIDE3IE5vdiAyMDEwIDE3OjM1OjQyICswODAwClN1Ympl Y3Q6IFtQQVRDSCAyLzJdIEJyaW5nIHVwIHRvIGRhdGUgd2luZG93cyBidWlsZCBlbnZpcm9u bWVudAoKLS0tCiBOTWFrZWZpbGUgICAgICAgICAgICAgIHwgICAgNSArKy0tLQogc3JjL05N YWtlZmlsZSAgICAgICAgICB8ICAgMTEgKysrKysrKy0tLS0KIHNyYy9rZWVwYWxpdmUuYyAg ICAgICAgfCAgICAyICstCiBzcmMvbGlic3NoMl9wcml2LmggICAgIHwgICAyMSAtLS0tLS0t LS0tLS0tLS0tLS0tLS0KIHdpbjMyL2NvbmZpZy5tayAgICAgICAgfCAgIDE2ICsrKysrKyst LS0tLS0tLS0KIHdpbjMyL2xpYnNzaDIucmMgICAgICAgfCAgICAyICstCiB3aW4zMi9saWJz c2gyX2NvbmZpZy5oIHwgICAgOCArKysrKysrLQogd2luMzIvcnVsZXMubWsgICAgICAgICB8 ICAgIDMgKystCiA4IGZpbGVzIGNoYW5nZWQsIDI3IGluc2VydGlvbnMoKyksIDQxIGRlbGV0 aW9ucygtKQoKZGlmZiAtLWdpdCBhL05NYWtlZmlsZSBiL05NYWtlZmlsZQppbmRleCA1MTk0 OGNjLi4wMjU0ZDUzIDEwMDY0NAotLS0gYS9OTWFrZWZpbGUKKysrIGIvTk1ha2VmaWxlCkBA IC0xMSw5ICsxMSw4IEBAIGNsZWFuOgogCiByZWFsLWNsZWFuOiBjbGVhbgogCS1kZWwgbGli c3NoMi5kbGwKKwktZGVsIGxpYnNzaDIuZGxsLm1hbmlmZXN0CiAJLWRlbCBsaWJzc2gyLmV4 cAogCS1kZWwgbGlic3NoMi5pbGsKIAktZGVsIGxpYnNzaDIubGliCi0JLWRlbCAqLnBkYgot Ci0KKwktZGVsIGxpYnNzaDIucGRiCmRpZmYgLS1naXQgYS9zcmMvTk1ha2VmaWxlIGIvc3Jj L05NYWtlZmlsZQppbmRleCBhYWJjZWFiLi5iNTQ1NDFhIDEwMDY0NAotLS0gYS9zcmMvTk1h a2VmaWxlCisrKyBiL3NyYy9OTWFrZWZpbGUKQEAgLTMsNiArMyw3IEBACiBDRkxBR1M9JChD RkxBR1MpCiAKIE9CSkVDVFMgPSBcCisJJChJTlRESVIpXGFnZW50Lm9iaiBcCiAJJChJTlRE SVIpXGNoYW5uZWwub2JqIFwKIAkkKElOVERJUilcY29tcC5vYmogXAogCSQoSU5URElSKVxj cnlwdC5vYmogXApAQCAtMTAsNiArMTEsOCBAQCBPQkpFQ1RTID0gXAogCSQoSU5URElSKVxo b3N0a2V5Lm9iaiBcCiAJJChJTlRESVIpXGtlZXBhbGl2ZS5vYmogXAogCSQoSU5URElSKVxr ZXgub2JqIFwKKwkkKElOVERJUilca25vd25ob3N0Lm9iaiBcCisJJChJTlRESVIpXGxpYmdj cnlwdC5vYmogXAogCSQoSU5URElSKVxtYWMub2JqIFwKIAkkKElOVERJUilcbWlzYy5vYmog XAogCSQoSU5URElSKVxvcGVuc3NsLm9iaiBcCkBAIC0yMCwxNCArMjMsMTQgQEAgT0JKRUNU UyA9IFwKIAkkKElOVERJUilcc2Vzc2lvbi5vYmogXAogCSQoSU5URElSKVxzZnRwLm9iaiBc CiAJJChJTlRESVIpXHRyYW5zcG9ydC5vYmogXAotCSQoSU5URElSKVx1c2VyYXV0aC5vYmoK KwkkKElOVERJUilcdXNlcmF1dGgub2JqIFwKKwkkKElOVERJUilcdmVyc2lvbi5vYmoKIAog RExMPWxpYnNzaDIkKFNVRkZJWCkuZGxsCiAKLSQoRExMKTogJChPQkpFQ1RTKQotCSQoQ0Mp IC1vICQoRExMKSAkKERMTEZMQUdTKSAkKE9CSkVDVFMpICQoTElCUykKKyQoRExMKTogJChP QkpFQ1RTKSAkKElOVERJUilcbGlic3NoMi5yZXMKKwkkKENDKSAtRmUiJChETEwpIiAkKERM TEZMQUdTKSAkKE9CSkVDVFMpICQoSU5URElSKVxsaWJzc2gyLnJlcyAkKExJQlMpCiAKIGFs bDogJChETEwpCiAKICFpbmNsdWRlICJ3aW4zMi9ydWxlcy5tayIKLQpkaWZmIC0tZ2l0IGEv c3JjL2tlZXBhbGl2ZS5jIGIvc3JjL2tlZXBhbGl2ZS5jCmluZGV4IDI2MDIwNmEuLjE2MGM4 MGUgMTAwNjQ0Ci0tLSBhL3NyYy9rZWVwYWxpdmUuYworKysgYi9zcmMva2VlcGFsaXZlLmMK QEAgLTkxLDcgKzkxLDcgQEAgbGlic3NoMl9rZWVwYWxpdmVfc2VuZCAoTElCU1NIMl9TRVNT SU9OICpzZXNzaW9uLAogICAgICAgICAgICAgKnNlY29uZHNfdG9fbmV4dCA9IHNlc3Npb24t PmtlZXBhbGl2ZV9pbnRlcnZhbDsKICAgICB9IGVsc2UgaWYgKHNlY29uZHNfdG9fbmV4dCkg ewogICAgICAgICAqc2Vjb25kc190b19uZXh0ID0gKGludCkgc2Vzc2lvbi0+a2VlcGFsaXZl X2xhc3Rfc2VudAotICAgICAgICAgICAgKyBzZXNzaW9uLT5rZWVwYWxpdmVfaW50ZXJ2YWwg LSBub3c7CisgICAgICAgICAgICArIHNlc3Npb24tPmtlZXBhbGl2ZV9pbnRlcnZhbCAtIChp bnQpIG5vdzsKICAgICB9CiAKICAgICByZXR1cm4gMDsKZGlmZiAtLWdpdCBhL3NyYy9saWJz c2gyX3ByaXYuaCBiL3NyYy9saWJzc2gyX3ByaXYuaAppbmRleCBkMjgxYzExLi5iOTEyMTEx IDEwMDY0NAotLS0gYS9zcmMvbGlic3NoMl9wcml2LmgKKysrIGIvc3JjL2xpYnNzaDJfcHJp di5oCkBAIC00MywxNCArNDMsNiBAQAogI2RlZmluZSBMSUJTU0gyX0xJQlJBUlkKICNpbmNs dWRlICJsaWJzc2gyX2NvbmZpZy5oIgogCi0jaWZkZWYgSEFWRV9XSU5ET1dTX0gKLSNpbmNs dWRlIDx3aW5kb3dzLmg+Ci0jZW5kaWYKLQotI2lmZGVmIEhBVkVfV1MyVENQSVBfSAotI2lu Y2x1ZGUgPHdzMnRjcGlwLmg+Ci0jZW5kaWYKLQogI2luY2x1ZGUgPHN0ZGlvLmg+CiAjaW5j bHVkZSA8dGltZS5oPgogCkBAIC0xMzIsMTkgKzEyNCw2IEBAIHN0YXRpYyBpbmxpbmUgaW50 IHdyaXRldihpbnQgc29jaywgc3RydWN0IGlvdmVjICppb3YsIGludCBudmVjcykKICNpbmNs dWRlICJvcGVuc3NsLmgiCiAjZW5kaWYKIAotI2lmZGVmIEhBVkVfV0lOU09DSzJfSAotCi0j aW5jbHVkZSA8d2luc29jazIuaD4KLSNpbmNsdWRlIDxtc3dzb2NrLmg+Ci0jaW5jbHVkZSA8 d3MydGNwaXAuaD4KLQotI2lmZGVmIF9NU0NfVkVSCi0vKiAiaW5saW5lIiBrZXl3b3JkIGlz IHZhbGlkIG9ubHkgd2l0aCBDKysgZW5naW5lISAqLwotI2RlZmluZSBpbmxpbmUgX19pbmxp bmUKLSNlbmRpZgotCi0jZW5kaWYKLQogLyogUkZDNDI1MyBzZWN0aW9uIDYuMSBNYXhpbXVt IFBhY2tldCBMZW5ndGggc2F5czoKICAqCiAgKiAiQWxsIGltcGxlbWVudGF0aW9ucyBNVVNU IGJlIGFibGUgdG8gcHJvY2VzcyBwYWNrZXRzIHdpdGgKZGlmZiAtLWdpdCBhL3dpbjMyL2Nv bmZpZy5tayBiL3dpbjMyL2NvbmZpZy5tawppbmRleCA3NDYyN2E0Li4xNWE3YjhhIDEwMDY0 NAotLS0gYS93aW4zMi9jb25maWcubWsKKysrIGIvd2luMzIvY29uZmlnLm1rCkBAIC0xLDEw ICsxLDEwIEBACiAKICMgVHdlYWsgdGhlc2UgZm9yIHlvdXIgc3lzdGVtCi1PUEVOU1NMSU5D PS4uXG9wZW5zc2wtMC45LjhvXGluYzMyCi1PUEVOU1NMTElCPS4uXG9wZW5zc2wtMC45Ljhv XG91dDMyZGxsCitPUEVOU1NMSU5DPS4uXHRoaXJkcGFydHlcb3BlbnNzbFxpbmMzMgorT1BF TlNTTExJQj0uLlx0aGlyZHBhcnR5XG9wZW5zc2xcb3V0MzJkbGwKIAotWkxJQklOQz0tRExJ QlNTSDJfSEFWRV9aTElCPTEgL0kuLlx6bGliLTEuMi41Ci1aTElCTElCPS4uXHpsaWItMS4y LjUKK1pMSUJJTkM9LURMSUJTU0gyX0hBVkVfWkxJQj0xIC9JLi5cdGhpcmRwYXJ0eVx6bGli CitaTElCTElCPS4uXHRoaXJkcGFydHlcemxpYgogCiAhaWYgIiQoVEFSR0VUKSIgPT0gIiIK IFRBUkdFVD1SZWxlYXNlCkBAIC0xNSwxNSArMTUsMTMgQEAgU1VGRklYPV9kZWJ1ZwogQ1BQ RkxBR1M9L09kIC9NRGQKIERMTEZMQUdTPS9ERUJVRyAvTERkCiAhZWxzZQotQ1BQRkxBR1M9 L09nIC9PaSAvTzIgL095IC9HRiAvWS0gL01EIC9ETkRFQlVHCitDUFBGTEFHUz0vTUQgL08y IC9PeS0gL1ppIC9EIl9XSU4zMl9XSU5OVD0weDA1MDIiIC9ETkRFQlVHCiBETExGTEFHUz0v REVCVUcgL0xECiAhZW5kaWYKIAotQ1BQRkxBR1M9L25vbG9nbyAvR0wgL1ppIC9FSHNjICQo Q1BQRkxBR1MpIC9Jd2luMzIgL0lpbmNsdWRlIC9JJChPUEVOU1NMSU5DKSAkKFpMSUJJTkMp IC1ETElCU1NIMl9XSU4zMgorQ1BQRkxBR1M9L25vbG9nbyAvVzMgL0dMIC9FSHNjIC9GZGxp YnNzaDIucGRiICQoQ1BQRkxBR1MpIC9EV0lOMzJfTEVBTl9BTkRfTUVBTiAvRE5PQ1JZUFQg L0l3aW4zMiAvSWluY2x1ZGUgL0kkKE9QRU5TU0xJTkMpICQoWkxJQklOQykgLURMSUJTU0gy X1dJTjMyCiBDRkxBR1M9JChDUFBGTEFHUykKIERMTEZMQUdTPSQoQ0ZMQUdTKSAkKERMTEZM QUdTKQotTElCUz0kKE9QRU5TU0xMSUIpXGxpYmVheTMyLmxpYiAkKE9QRU5TU0xMSUIpXHNz bGVheTMyLmxpYiB3czJfMzIubGliICQoWkxJQkxJQilcemxpYi5saWIKK0xJQlM9d3MyXzMy LmxpYiB1c2VyMzIubGliICQoT1BFTlNTTExJQilcbGliZWF5MzIubGliICQoT1BFTlNTTExJ Qilcc3NsZWF5MzIubGliICQoWkxJQkxJQilcemRsbC5saWIKIAogSU5URElSPSQoVEFSR0VU KVwkKFNVQkRJUikKLQotCmRpZmYgLS1naXQgYS93aW4zMi9saWJzc2gyLnJjIGIvd2luMzIv bGlic3NoMi5yYwppbmRleCAxZmRkNDZjLi4xZDhiNTZmIDEwMDY0NAotLS0gYS93aW4zMi9s aWJzc2gyLnJjCisrKyBiL3dpbjMyL2xpYnNzaDIucmMKQEAgLTMzLDcgKzMzLDcgQEAgQkVH SU4KICAgICAgIFZBTFVFICJPcmlnaW5hbEZpbGVuYW1lIiwgImxpYnNzaDIuZGxsXDAiCiAg ICAgICBWQUxVRSAiUHJvZHVjdE5hbWUiLCAgICAgICJUaGUgbGlic3NoMiBsaWJyYXJ5XDAi CiAgICAgICBWQUxVRSAiUHJvZHVjdFZlcnNpb24iLCAgIExJQlNTSDJfVkVSU0lPTiAiXDAi Ci0gICAgICBWQUxVRSAiTGVnYWxDb3B5cmlnaHQiLCAgICKpICIgTElCU1NIMl9DT1BZUklH SFQgIlwwIgorICAgICAgVkFMVUUgIkxlZ2FsQ29weXJpZ2h0IiwgICAiQ29weXJpZ2h0IChj KSAiIExJQlNTSDJfQ09QWVJJR0hUICJcMCIKICAgICAgIFZBTFVFICJMaWNlbnNlIiwgICAg ICAgICAgImh0dHA6Ly93d3cubGlic3NoMi5vcmcvbGljZW5zZS5odG1sXDAiCiAgICAgRU5E CiAgIEVORApkaWZmIC0tZ2l0IGEvd2luMzIvbGlic3NoMl9jb25maWcuaCBiL3dpbjMyL2xp YnNzaDJfY29uZmlnLmgKaW5kZXggNTZjNjRhNi4uMzI1MjRjZSAxMDA2NDQKLS0tIGEvd2lu MzIvbGlic3NoMl9jb25maWcuaAorKysgYi93aW4zMi9saWJzc2gyX2NvbmZpZy5oCkBAIC00 LDExICs0LDEyIEBACiAjaWZuZGVmIFdJTjMyCiAjZGVmaW5lIFdJTjMyCiAjZW5kaWYKKwog I2lmbmRlZiBfQ1JUX1NFQ1VSRV9OT19ERVBSRUNBVEUKICNkZWZpbmUgX0NSVF9TRUNVUkVf Tk9fREVQUkVDQVRFIDEKICNlbmRpZiAvKiBfQ1JUX1NFQ1VSRV9OT19ERVBSRUNBVEUgKi8K KwogI2luY2x1ZGUgPHdpbnNvY2syLmg+Ci0jaW5jbHVkZSA8bXN3c29jay5oPgogI2luY2x1 ZGUgPHdzMnRjcGlwLmg+CiAKICNpZmRlZiBfX01JTkdXMzJfXwpAQCAtMjIsNyArMjMsMTIg QEAKICNkZWZpbmUgSEFWRV9TRUxFQ1QKIAogI2lmZGVmIF9NU0NfVkVSCisvKiAiaW5saW5l IiBrZXl3b3JkIGlzIHZhbGlkIG9ubHkgd2l0aCBDKysgZW5naW5lISAqLworI2RlZmluZSBp bmxpbmUgX19pbmxpbmUKICNkZWZpbmUgc25wcmludGYgX3NucHJpbnRmCisjaWYgX01TQ19W RVIgPD0gMTIwMAordHlwZWRlZiBpbnQgc29ja2xlbl90OworI2VuZGlmCiAjaWYgX01TQ19W RVIgPCAxNTAwCiAjZGVmaW5lIHZzbnByaW50ZiBfdnNucHJpbnRmCiAjZW5kaWYKZGlmZiAt LWdpdCBhL3dpbjMyL3J1bGVzLm1rIGIvd2luMzIvcnVsZXMubWsKaW5kZXggYmZiNzc4NC4u NDlhMjFlZCAxMDA2NDQKLS0tIGEvd2luMzIvcnVsZXMubWsKKysrIGIvd2luMzIvcnVsZXMu bWsKQEAgLTEyLDQgKzEyLDUgQEAgJChJTlRESVIpOgogeyQoU1VCRElSKX0uY3skKElOVERJ Uil9Lm9iajo6CiAJJChDQykgLWMgJChDRkxBR1MpIC9GbyIkKElOVERJUilcXCIgJDwKIAot CiskKElOVERJUilcbGlic3NoMi5yZXM6IHdpbjMyXGxpYnNzaDIucmMKKwkkKFJDKSAvRm8i JEAiICQ/ClwgTm8gbmV3bGluZSBhdCBlbmQgb2YgZmlsZQotLSAKMS43LjIuMwoK --------------070400010302030903090301 Content-Type: text/plain; name="0001-Typo-fix-s-elsif-elif-g.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-Typo-fix-s-elsif-elif-g.patch" RnJvbSAyYzEzNjgzNTU0ZWU2NGQ4ZmQ2ZjY1OTg3ZGMzNWUyODE5YTk2MDRhIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBEb25nc2hlbmcgU29uZyA8ZG9uZ3NoZW5nLnNvbmdA Z21haWwuY29tPgpEYXRlOiBXZWQsIDE3IE5vdiAyMDEwIDE3OjIzOjE2ICswODAwClN1Ympl Y3Q6IFtQQVRDSCAxLzJdIFR5cG8gZml4LCBzL2Vsc2lmL2VsaWYvZwoKLS0tCiBzcmMvbWlz Yy5jIHwgICAgNiArKystLS0KIDEgZmlsZXMgY2hhbmdlZCwgMyBpbnNlcnRpb25zKCspLCAz IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9taXNjLmMgYi9zcmMvbWlzYy5jCmlu ZGV4IGNiZWEwMzkuLmMxYWMxMjggMTAwNjQ0Ci0tLSBhL3NyYy9taXNjLmMKKysrIGIvc3Jj L21pc2MuYwpAQCAtMTAwLDcgKzEwMCw3IEBAIF9saWJzc2gyX3JlY3YobGlic3NoMl9zb2Nr ZXRfdCBzb2NrLCB2b2lkICpidWZmZXIsIHNpemVfdCBsZW5ndGgsIGludCBmbGFncykKICNp ZmRlZiBXSU4zMgogICAgIGlmIChyYyA8IDAgKQogICAgICAgICByZXR1cm4gLXdzYTJlcnJu bygpOwotI2Vsc2lmIGRlZmluZWQoX19WTVMpCisjZWxpZiBkZWZpbmVkKF9fVk1TKQogICAg IGlmIChyYyA8IDAgKXsKICAgICAgICAgaWYgKCBlcnJubyA9PSBFV09VTERCTE9DSyApCiAg ICAgICAgICAgICByZXR1cm4gLUVBR0FJTjsKQEAgLTEyNiw3ICsxMjYsNyBAQCBfbGlic3No Ml9zZW5kKGxpYnNzaDJfc29ja2V0X3Qgc29jaywgY29uc3Qgdm9pZCAqYnVmZmVyLCBzaXpl X3QgbGVuZ3RoLAogI2lmZGVmIFdJTjMyCiAgICAgaWYgKHJjIDwgMCApCiAgICAgICAgIHJl dHVybiAtd3NhMmVycm5vKCk7Ci0jZWxzaWYgZGVmaW5lZChfX1ZNUykKKyNlbGlmIGRlZmlu ZWQoX19WTVMpCiAgICAgaWYgKHJjIDwgMCApIHsKICAgICAgICAgaWYgKCBlcnJubyA9PSBF V09VTERCTE9DSyApCiAgICAgICAgICAgICByZXR1cm4gLUVBR0FJTjsKQEAgLTU2NCw3ICs1 NjQsNyBAQCB2b2lkIF9saWJzc2gyX2xpc3RfaW5zZXJ0KHN0cnVjdCBsaXN0X25vZGUgKmFm dGVyLCAvKiBpbnNlcnQgYmVmb3JlIHRoaXMgKi8KICAqIFRoZSBPcGVuIEdyb3VwIEJhc2Ug U3BlY2lmaWNhdGlvbnMgSXNzdWUgNgogICogSUVFRSBTdGQgMTAwMy4xLCAyMDA0IEVkaXRp b24KICAqLwotICAKKwogLyoKICAqICBUSElTIFNPRlRXQVJFIElTIE5PVCBDT1BZUklHSFRF RAogICoKLS0gCjEuNy4yLjMKCg== --------------070400010302030903090301-- --------------enig24F8D6DFCC186A89DCFFB002 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (MingW32) iQEcBAEBCAAGBQJM46VjAAoJEAQBqiBG05f/2GcH/jPhAKqR+VHK8olgwuxt44FW Ws80FLjTDe0km6Wn6hGw6TqXLQS280EuiQTwaNZhJIE/W+Filf8NrF0tFoEZlIn7 Bsv3CvyneVLuIJqS+mwPBXlv2iEAIlRkZHCUXGlPZUTHFuBHTkZXFBNXMMeNla7d sRInttqfFgs+3oVyO6srxEwreksiLbMZob8ZmIYxtYw+vljt9sOdZC0GXF20/QON GVtdeSZgo0tUYmd9ORFbPDcq4xtCjIR606xbOOFYeEhZnW6D3YtM1UXj8bzzZ3mb lLTJMB1KU7A2vzzwvvCltuE+6Uau3UIqer6HfnwNjaZs+rEF0d6jMvLHHkWLLTk= =A+cp -----END PGP SIGNATURE----- --------------enig24F8D6DFCC186A89DCFFB002-- --===============1310655664== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============1310655664==-- From libssh2-devel-bounces@cool.haxx.se Wed Nov 17 11:18:51 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAHAIhHU007491; Wed, 17 Nov 2010 11:18:50 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAHAIguU007486 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 17 Nov 2010 11:18:43 +0100 Received: (qmail 22633 invoked from network); 17 Nov 2010 10:18:40 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 17 Nov 2010 10:18:40 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Wed, 17 Nov 2010 10:18:40 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #198: _libssh2_channel_close() may hang X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/198 Message-ID: <040.f267185fd189c3163b83597a7e73540d@libssh2.stuge.se> X-Trac-Ticket-ID: 198 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 17 Nov 2010 11:18:50 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 17 Nov 2010 11:18:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #198: _libssh2_channel_close() may hang ---------------------+------------------------------------------------------ Reporter: fd64 | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.7 Component: API | Version: 1.2.7 Keywords: | Blocks: Blocked By: | ---------------------+------------------------------------------------------ An application is using libssh2 to execute commands remotely. It makes intensive use of libssh2 with many threads to maintain a lot of connections to several test openssh server boxes. We are using iptables to randomly drop packets on the remote box to simulate very bad network conditions. We want to be sure that the application will never hang or crash even when connections are unstable. The socket has been created in non-blocking because we want to be sure that the application will never hang if something wrong happens. When the applications detects a problem with a connection it disconnects using these functions: libssh2_channel_close() + libssh2_channel_free() + libssh2_session_disconnect() + libssh2_session_free(). In one of our test, one thread has been not responding for at least 60 seconds in _libssh2_transport_read() when the application called libssh2_channel_close(). (gdb) thread 6 [Switching to thread 6 (Thread 10476)]#0 0x00000037f6219752 in _libssh2_transport_read (session=) at transport.c:601 601 } (gdb) bt #0 0x00000037f6219752 in _libssh2_transport_read (session=) at transport.c:601 #1 0x00000037f620548e in _libssh2_channel_close (channel=0x7f0124040550) at channel.c:2257 #2 0x00000037f62056f8 in libssh2_channel_close (channel=0x7f0124040550) at channel.c:2292 _libssh2_channel_close() seems to be hanging in the following loop: if (channel->close_state == libssh2_NB_state_sent) { /* We must wait for the remote SSH_MSG_CHANNEL_CLOSE message */ while (!channel->remote.close && !rc) { rc = _libssh2_transport_read(session); } } Maybe this problem only happens in very specific conditions. For instance it may only happen if the connection is still alive when _libssh2_channel_close() starts and is lost in the middle of this function. Anyway, I think it would be great if you could either check that the connection is still alive in this loop, or if you could implement a sort of timeout (give up after X retries/seconds). The program is running on a very up to date Fedora-12-amd64 system with libssh2-1.2.7 (RPM package rebuilt for fedora-12 from official non patched 1.2.7). I can send you the core dump + binary to a private mail address if it can help. Here are the two iptables commands that runs on the openssh client and that makes the connection very unstable to simulate bad network. It's a quite very aggressive configuration, but half of the connections manage to successfully execute a simple command anyway. iptables -A INPUT -i ${IFACE} -s ${SRCIP} -p tcp --dport 22 -m statistic --mode random --probability 0.60 -j breakpkt iptables -A INPUT -i ${IFACE} -s ${SRCIP} -p tcp --dport 22 ! --syn -m statistic --mode nth --every 10 -j breakpkt The test machine is also running Fedora-12-amd64 and the following ssh server: kernel-2.6.32.23-170.fc12.x86_64 openssh-clients-5.3p1-19.fc12.x86_64 openssh-server-5.3p1-19.fc12.x86_64 openssh-5.3p1-19.fc12.x86_64 Many thanks -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 00:14:09 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAHNDjRa027222; Thu, 18 Nov 2010 00:14:05 +0100 Received: from panic.com (posh.panic.com [38.103.165.4]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAHNDgp6027156 for ; Thu, 18 Nov 2010 00:13:43 +0100 Received: from [10.0.0.81] (unknown [38.103.165.36]) by panic.com (Postfix) with ESMTP id B3E7B303FA6C for ; Wed, 17 Nov 2010 15:13:32 -0800 (PST) From: Will Cosgrove Subject: Pipelining and recent sftp upload improvements Date: Wed, 17 Nov 2010 15:13:32 -0800 Message-Id: To: libssh2-devel@cool.haxx.se Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 00:14:08 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 00:13:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hi All, Pardon my newb-ness here, but I've just subscribed to the list after going through recent emails. It would appear in the nightly builds uploading in sftp_write() now does pipelined uploads based on the buffer size. I was impressed and surprised of this development. The reason I'm posting is I recently added my own upload/download pipelining API to libssh2 modeled after libssh's download pipeline API. It pushes a bit more state management onto the user, but it seems to work fairly well and doesn't require a large input buffer to be filled before hand. I was wondering if there was any interest in my additions (which admittedly need to be code-reviewed by someone more familiar to libssh2 than me) or if this current method was going to be applied to downloading at some point and I should just keep my changes to my projects. Thanks, Will _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 04:18:02 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI35uxs017307; Thu, 18 Nov 2010 04:06:16 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI35r1W015074 for ; Thu, 18 Nov 2010 04:05:54 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=US6UXbBLTCyAWodwematvWMcf5UjvmuKte8bRhKDLZJJw3QBfJOPP3+oH46ShU+whz GZmXSUMl4xqD3iq8OUJvTmP1N47/1gtroX3JMaOyJvnUedMyNfbocfK/FWd7OpxQ45vV /bsbTTSGq1VAXpLGhbVF79Ep2iQbcwPpgqYTw=; bh=Ah1j9nvXODrwU2s15+TH9HppLd0=; t=1290049556; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 594c0011e66f8a0a; Wed, 17 Nov 2010 22:05:55 -0500 From: "Mark Riordan" To: "'libssh2 development'" Subject: Updated libssh2 benchmark results Date: Wed, 17 Nov 2010 21:05:45 -0600 Message-ID: <000001cb86cd$7e7f22f0$7b7d68d0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuGzXyrPAnFqoQ6RNqfcvpZgSbtXA== Content-Language: en-us x-cr-hashedpuzzle: a9s= h0M= ARP6 Agjo A4Nq Cg9Q DHjx DVe+ Dwba ESp/ EsLP FHv9 FJZC HLUK IUHB JyK4; 1; bABpAGIAcwBzAGgAMgAtAGQAZQB2AGUAbABAAGMAbwBvAGwALgBoAGEAeAB4AC4AcwBlAA==; Sosha1_v1; 7; {93912CF6-8AD2-4909-A541-D4BE0DB70985}; bQByAGkAbwByAGQAYQBuAEAAaQBwAHMAdwBpAHQAYwBoAC4AYwBvAG0A; Thu, 18 Nov 2010 03:05:42 GMT; VQBwAGQAYQB0AGUAZAAgAGwAaQBiAHMAcwBoADIAIABiAGUAbgBjAGgAbQBhAHIAawAgAHIAZQBzAHUAbAB0AHMA x-cr-puzzleid: {93912CF6-8AD2-4909-A541-D4BE0DB70985} X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 04:06:20 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 04:05:54 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se I ran a bunch more performance tests of the latest libssh2, and some other SSH clients. The results are here: http://60bits.net/sni/libssh2-perf.htm The previous results are here: http://60bits.net/sni/libssh2-perf-0.htm As you can see, libssh2 for some crazy reason is much slower when uploading to Solaris than it used to be. (I tried this many times, and it was repeatable.) However, I'm more concerned about its mediocre upload performance across a WAN. Presumably the difference there is due to packet delays. I would have thought that Daniel's recent changes would have made libssh2 competitive there. I ran my WAN tests from home, where I'm the only one uploading any significant amount of data. At the WAN at work, upload results were even worse, but with many people sharing the same Internet connection, the tests were less repeatable. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 04:49:40 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI3nVcZ008697; Thu, 18 Nov 2010 04:49:40 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAI3nTB3008689 for ; Thu, 18 Nov 2010 04:49:29 +0100 Received: (qmail 32263 invoked by uid 501); 18 Nov 2010 03:49:30 -0000 Message-ID: <20101118034930.32262.qmail@stuge.se> Date: Thu, 18 Nov 2010 04:49:29 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: Updated libssh2 benchmark results Mail-Followup-To: libssh2-devel@cool.haxx.se References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <000001cb86cd$7e7f22f0$7b7d68d0$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 04:49:40 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 04:49:30 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > I ran a bunch more performance tests of the latest libssh2, > and some other SSH clients. > The results are here: > http://60bits.net/sni/libssh2-perf.htm .. > Note: when doing performance testing, I attempted to turn off any > multi-threaded file transfer features; however, I might not have > been successful in Tunnelier and Ubuntu (OpenSSH). OpenSSH never uses threading, but does use segmentation/pipelining. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 04:53:16 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI3rCti010080; Thu, 18 Nov 2010 04:53:16 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAI3rBrX010072 for ; Thu, 18 Nov 2010 04:53:11 +0100 Received: (qmail 31958 invoked by uid 501); 18 Nov 2010 03:46:30 -0000 Message-ID: <20101118034630.31957.qmail@stuge.se> Date: Thu, 18 Nov 2010 04:46:30 +0100 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: Updated libssh2 benchmark results Mail-Followup-To: libssh2-devel@cool.haxx.se References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <000001cb86cd$7e7f22f0$7b7d68d0$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 04:53:16 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 04:53:11 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > I ran a bunch more performance tests of the latest libssh2, > and some other SSH clients. > The results are here: > http://60bits.net/sni/libssh2-perf.htm Quite sad. :( > The previous results are here: > http://60bits.net/sni/libssh2-perf-0.htm > > As you can see, libssh2 for some crazy reason is much slower > when uploading to Solaris than it used to be. > (I tried this many times, and it was repeatable.) Note that every UNIX system ships with a custom variation of OpenSSH, so Solaris and Linux are not running the same code on the server. It would be good to eliminate as many variables as possible, so if you have access to the Linux and Solaris then maybe you can build the vanilla OpenSSH portable code and run it as your user on port 2222 for the purpose of testing. I do not expect it to have even a noticeable impact on test results, but it helps to have just one reference server, and it also allows anyone who wants to work on improvement to set up the exact same environment. > However, I'm more concerned about its mediocre upload performance > across a WAN. That's generous. I wouldn't even call that mediocre. > Presumably the difference there is due to packet delays. Personally I don't really *know* what the problem is, so to profile this will require looking a lot at timing and packets, both from inside the libssh2 app, and from another viewpoint outside the app, e.g. sniffing packets directly from the OS, or even on the wire. > I would have thought that Daniel's recent changes would have made > libssh2 competitive there. If not competitive, then at least decidedly better. For these upload tests with libssh2, where did the data come from that you uploaded? If there is any other I/O in the _sftp_write() loop then please pre-load all data and remove any swap file, so that all data is in fact available directly. Again the purpose is not to significantly improve the numbers, but to create an easily repeatable environment for further testing and development. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 10:43:25 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI9h44Y013001; Thu, 18 Nov 2010 10:43:19 +0100 Received: from mx1.belnet.be (mx1.belnet.be [193.190.198.9]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAI9h3pP012773 for ; Thu, 18 Nov 2010 10:43:03 +0100 X-IronPort-AV: E=Sophos;i="4.59,216,1288566000"; d="scan'208";a="1212279" Received: from firewall.belnet.be (HELO office.belnet.be) ([193.190.198.37]) by mx1-out.belnet.be with ESMTP; 18 Nov 2010 10:42:56 +0100 Received: from localhost (judas.fw.belnet.be [127.0.0.1]) by office.belnet.be (Postfix) with ESMTP id 40E4230293 for ; Thu, 18 Nov 2010 10:42:56 +0100 (CET) Received: from [172.24.185.254] (aris.fw.belnet.be [172.24.185.254]) by office.belnet.be (Postfix) with ESMTP id 33D493028B for ; Thu, 18 Nov 2010 10:42:56 +0100 (CET) Message-ID: <4CE4F520.2050405@0xbadc0de.be> Date: Thu, 18 Nov 2010 10:42:56 +0100 From: Aris Adamantiadis User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Thunderbird/3.0.10 MIME-Version: 1.0 To: libssh2 development Subject: Re: Updated libssh2 benchmark results References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> In-Reply-To: <20101118034630.31957.qmail@stuge.se> X-Virus-Scanned: Belnet virus scanner X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 10:43:25 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 10:43:03 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Le 18/11/10 04:46, Peter Stuge a =E9crit : >> Presumably the difference there is due to packet delays. > = > Personally I don't really *know* what the problem is, so to profile > this will require looking a lot at timing and packets, both from > inside the libssh2 app, and from another viewpoint outside the app, > e.g. sniffing packets directly from the OS, or even on the wire. I'm 100% sure it's due to packet delays. Mark, as a suggestion, could you provide us a little more information, like the RTT between the two hosts (even on the LAN), and the sample of code you used for the benchmark (so it's reproducible). We all know that performance of a lib may be very well depending on the way it's being used. I'm also interested in having libssh's results on your site as well (both 0.4.6 and git master). > For these upload tests with libssh2, where did the data come from > that you uploaded? If there is any other I/O in the _sftp_write() > loop then please pre-load all data and remove any swap file, so that > all data is in fact available directly. Again the purpose is not to > significantly improve the numbers, but to create an easily repeatable > environment for further testing and development. I second this advice. "cat file > /dev/null" before running the test may be enough. On remote side, when possible write to /dev/null or to a SSD disk. Aris _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 12:13:33 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIBDD6N026805; Thu, 18 Nov 2010 12:13:29 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIBDCZn026798 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 18 Nov 2010 12:13:12 +0100 Received: (qmail 28126 invoked from network); 18 Nov 2010 11:13:06 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 18 Nov 2010 11:13:06 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Thu, 18 Nov 2010 11:13:06 -0000 X-URL: http://trac.libssh2.org/ Subject: Re: [libssh2] #196: using libssh2_channel_read crushes X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/196#comment:3 Message-ID: <123.c1470adc23f50033e8f2c67ab1deea25@libssh2.stuge.se> References: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Trac-Ticket-ID: 196 In-Reply-To: <108.e98b3c6004b9bf5f25e4ecb182799720@libssh2.stuge.se> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 12:13:33 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 12:13:12 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #196: using libssh2_channel_read crushes ---------------------------------------------------------------------------------------+ Reporter: www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc | Owner: bagder Type: defect | Status: closed Priority: high | Milestone: 1.2.7 Component: API | Version: 1.2.7 Resolution: invalid | Keywords: Blocks: | Blocked By: ---------------------------------------------------------------------------------------+ Changes (by www.google.com/accounts/o8/id?id=aitoawk50cbnsiuuqgylsi_oyi14_ymtafn1cjc): * status: accepted => closed * resolution: => invalid Comment: I was trying to use a channel after closing it with : libssh2_channel_close(channel); libssh2_channel_free(channel); still it was nice if the API detect a bad channel (NULL) and not crush . -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 18:54:56 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIHsZBg029685; Thu, 18 Nov 2010 18:54:53 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIHsX5Z029672 for ; Thu, 18 Nov 2010 18:54:34 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=HHB7dwaoCHi5WO81tkmPdl7Yz7WO97vox906WXtAgnZ1A7tS2gIZ5pKY+TGRNlUIth YHUPD/dNfuAMcVwcK+Ttp2OcrUB0UYWhOpdZti6NXzZp2Tl1aulGdUCP+clYY6z+Sfq3 PR2ID7tObv7A9hlUG0f3P78YyAivWFfvx8o+U=; bh=TgkiL38MY3GNRKc6G1oRbOTh6mM=; t=1290102879; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id d8f4001327b6953e; Thu, 18 Nov 2010 12:54:38 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> In-Reply-To: <20101118034630.31957.qmail@stuge.se> Subject: RE: Updated libssh2 benchmark results Date: Thu, 18 Nov 2010 11:54:23 -0600 Message-ID: <001701cb8749$a2e1f1f0$e8a5d5d0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuG1Eudi7hLaI9wTYqvrLpBYSvziAAc/QWQ Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 18:54:56 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 18:54:34 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Peter wrote: > It would be good to eliminate as many variables as possible, so if > you have access to the Linux and Solaris then maybe you can build the > vanilla OpenSSH portable code and run it as your user on port 2222 > for the purpose of testing. OK. Hopefully I'll have time to do that. > Personally I don't really *know* what the problem is, so to profile > this will require looking a lot at timing and packets, both from > inside the libssh2 app, and from another viewpoint outside the app, > e.g. sniffing packets directly from the OS, or even on the wire. I'll get Wireshark captures of the WAN uploads. Unfortunately, that will have to wait at least until tonight (US time), because at home is the only place I have an Internet connection that isn't being used sporadically by many other people. I didn't have a Linux environment readily at hand at home last night. I'll have to put one together. > For these upload tests with libssh2, where did the data come from > that you uploaded? If there is any other I/O in the _sftp_write() > loop then please pre-load all data and remove any swap file, so that > all data is in fact available directly. Again the purpose is not to > significantly improve the numbers, but to create an easily repeatable > environment for further testing and development. The data came from the local filesystem. I did not report the results of the first upload, which sometimes was slightly slower than subsequent uploads. I have plenty of RAM in my systems, and I think the various OSes do a very good job of caching recently-used files when they are much smaller than the amount of available memory. The results were quite reproducible after the first run, so I don't think we need to worry about filesystem slowness as an issue here. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 19:12:19 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIIC9XC011107; Thu, 18 Nov 2010 19:12:16 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIIC7Eg010997 for ; Thu, 18 Nov 2010 19:12:07 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=sQbye69XaVr76mezfSEy/A4Qc9sfM86zY8Jb/rBfpAAN756KtEYBqc3bGhNyCpPlin DsrbtFr8EIjSWuNjYZm6Hu8S0JOggznAuSKf1l64GJjBkKhE74ITgJ7FZs6+i/+M52Dy BGLpYw6nzzXkgivRo+eo8m+WO6+IWyWFjcyP0=; bh=+mBV8w4t/eFjIMqk99io0yXabVw=; t=1290103933; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 791800132e4e95be; Thu, 18 Nov 2010 13:12:12 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> <4CE4F520.2050405@0xbadc0de.be> In-Reply-To: <4CE4F520.2050405@0xbadc0de.be> Subject: RE: Updated libssh2 benchmark results Date: Thu, 18 Nov 2010 12:11:58 -0600 Message-ID: <001b01cb874c$17510100$45f30300$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuHBVYkl0NU1I2jSe+xivGo+h8vqwARE47A Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 19:12:19 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 19:12:07 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Aris wrote: > I'm 100% sure it's due to packet delays. Mark, as a suggestion, could > you provide us a little more information, like the RTT between the two > hosts (even on the LAN), and the sample of code you used for the > benchmark (so it's reproducible). We all know that performance of a lib > may be very well depending on the way it's being used. I'll get the round trip time info later--hopefully I'll have enough energy tonight. I put the source (and Windows executable) for my test program here: http://trac.libssh2.org/attachment/ticket/197/sftpmrr.zip > I'm also interested in having libssh's results on your site as well > (both 0.4.6 and git master). I'll see if I can do that today, at least for LAN results. > On remote side, when possible write to /dev/null or to a SSD > disk. Well, I'm looking for real-world performance, not just high numbers in my tests. Since other SSH clients are able to achieve higher numbers consistently on the same client and server hardware, I do not suspect filesystem performance as a factor here. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 20:19:32 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIJJF1c021997; Thu, 18 Nov 2010 20:19:31 +0100 Received: from foo.birdnet.se (qmailr@foo.birdnet.se [213.88.146.6]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with SMTP id oAIJJEGa021982 for ; Thu, 18 Nov 2010 20:19:14 +0100 Received: (qmail 6865 invoked by uid 501); 18 Nov 2010 19:19:10 -0000 Message-ID: <20101118191910.6864.qmail@stuge.se> Date: Thu, 18 Nov 2010 20:19:10 +0100 From: Peter Stuge To: "'libssh2 development'" Subject: Re: Updated libssh2 benchmark results Mail-Followup-To: 'libssh2 development' References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> <4CE4F520.2050405@0xbadc0de.be> <001b01cb874c$17510100$45f30300$@com> <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> <001701cb8749$a2e1f1f0$e8a5d5d0$@com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <001b01cb874c$17510100$45f30300$@com> <001701cb8749$a2e1f1f0$e8a5d5d0$@com> X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 20:19:32 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 20:19:14 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Mark Riordan wrote: > > the purpose is not to significantly improve the numbers, but to > > create an easily repeatable environment for further testing and > > development. .. > I have plenty of RAM in my systems, and I think the various > OSes do a very good job of caching recently-used files I would like to completely eliminate all outside factors in order to have less disinformation to care about and compensate for when working on this. Mark Riordan wrote: > Well, I'm looking for real-world performance, not just high numbers > in my tests. I specifically explained that this is *not* the purpose of creating an artificial environment for testing and for development. The goal is obviously real-world performance, but getting there will be easier if as much noise as is possible can be removed from measurements and data during testing analysis and development. There is obviously a big problem somewhere, which may be easy to find, but it will be easier to also spot other problems with hard reliable reproducible numbers. > Since other SSH clients are able to achieve higher > numbers consistently on the same client and server hardware, > I do not suspect filesystem performance as a factor here. I'll try to make the point again; the purpose of a controlled, artificial environment for testing and development is not to make numbers look better, but to have a common reference for everyone, and to have absolutely minimal interference with timing within the library. //Peter _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 23:15:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIMFS0k016574; Thu, 18 Nov 2010 23:15:45 +0100 Received: from bxwall.lpma.nsw.gov.au (bxwall.lpma.nsw.gov.au [203.57.8.11]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIMFNB4016556 for ; Thu, 18 Nov 2010 23:15:26 +0100 From: Phillip Wu To: "libssh2-devel@cool.haxx.se" Date: Fri, 19 Nov 2010 09:15:00 +1100 Subject: Windows and libssh2 Thread-Topic: Windows and libssh2 Thread-Index: AcuHbgqWtfDqj8qsQzyDtFbrIcyNRA== Message-ID: <137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2E@SRV-QS-MAIL6.lands.nsw> Accept-Language: en-US, en-AU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-AU MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 23:15:49 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 23:15:27 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1990724571==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --===============1990724571== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2ESRVQSMAIL6lan_" --_000_137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2ESRVQSMAIL6lan_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, I am trying to use libssh2 in Visual Studio 2010 Express C++. I am using ss= h2 from the example directory. I managed to get libssh2 (version 1.2.7) for Windows as a dll from http://j= osefsson.org/gnutls4win/. According to MSDN this is the required code to get access to a dll at run t= ime: >> HINSTANCE hinstLib; MYPROC ProcAdd; BOOL fFreeResult, fRunTimeLinkSuccess =3D FALSE; // Get a handle to the DLL module. hinstLib =3D LoadLibrary(TEXT("libssh2-1.dll")); // If the handle is valid, try to get the function address. if (hinstLib !=3D NULL) { ProcAdd =3D (MYPROC) GetProcAddress(hinstLib, "libssh2_session_init= "); // If the function address is valid, call the function. if (NULL !=3D ProcAdd) { fRunTimeLinkSuccess =3D TRUE; (ProcAdd) (); } // Free the DLL module. fFreeResult =3D FreeLibrary(hinstLib); } // If unable to call the DLL function, use an alternative. if (! fRunTimeLinkSuccess) printf("Unable to access Dynamic Library libssh2-1.dll or routine l= ibssh2_session_init\n"); << 1. Has anyone used this approach for libssh2? If so, is there somewhere I c= an get the code stubs for all the libssh2 functions (perhaps as a "define")? 2. When I try to compile ssh2 example it is unable to find: closesocket socket connect htons htonl inet_addr WSAStartup Does anyone know which dll on Windows have these system calls? 3. Would I be better off using MinGW or some other build package for Window= s? *************************************************************** This message is intended for the addressee named and may contain confidenti= al information. If you are not the intended recipient, please delete it and= notify the sender. Views expressed in this message are those of the indivi= dual sender, and are not necessarily the views of the Land and Property Man= agement Authority. This email message has been swept by MIMEsweeper for the= presence of computer viruses. *************************************************************** Please consider the environment before printing this email. --_000_137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2ESRVQSMAIL6lan_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

 

I am trying to use libssh2 in Visual Studio 2010 Expre= ss C++. I am using ssh2 from the example directory.

 

I managed to get libssh2 (version 1.2.7) for Windows a= s a dll from http://josefsson.org/gnutls4win/.

 

According to MSDN this is the required code to get acc= ess to a dll at run time:

>> 

HINSTANCE hinstLib;

MYPROC ProcAdd;

BOOL fFreeResult, fRunTimeLinkSuccess =3D FALSE; =

 

    // Get a handle to the DLL module.<= o:p>

 

    hinstLib =3D LoadLibrary(TEXT("= ;libssh2-1.dll"));

 

    // If the handle is valid, try to g= et the function address.

 

    if (hinstLib !=3D NULL) =

    {

        ProcAdd =3D= (MYPROC) GetProcAddress(hinstLib, "libssh2_session_init");

 

        // If the f= unction address is valid, call the function.

 

        if (NULL != =3D ProcAdd)

        {

         =    fRunTimeLinkSuccess =3D TRUE;

         =    (ProcAdd) ();

        }

        // Free the= DLL module.

 

        fFreeResult= =3D FreeLibrary(hinstLib);

    }

 

    // If unable to call the DLL functi= on, use an alternative.

    if (! fRunTimeLinkSuccess)

        printf(&quo= t;Unable to access Dynamic Library libssh2-1.dll or routine libssh2_session_init\n");

<< 

 

1. Has anyone used this approach for libssh2? If so, is there somewhere I can get the code stubs for

all the libssh2 functions (perhaps as a "define&q= uot;)?

2. When I try to compile ssh2 example it is unable to = find:

closesocket

socket

connect

htons

htonl

inet_addr

WSAStartup

Does anyone know which dll on Windows have these system calls?

3. Would I be better off using MinGW or some other bui= ld package for Windows?

 

This message is intend= ed for the addressee named and may contain confidential information. If you= are not the intended recipient, please delete it and noti= fy the sender. Views expressed in this message are = those of the individual sender, and are not necessa= rily the views of the Land and Property Management = Authority. This email message has been swept by MIM= Esweeper for the presence of computer viruses.

Please consider the environment before printing this email.

 --_000_137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2ESRVQSMAIL6lan_-- --===============1990724571== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============1990724571==-- From libssh2-devel-bounces@cool.haxx.se Thu Nov 18 23:35:00 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIMYqix025311; Thu, 18 Nov 2010 23:35:00 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAIMYott025271 for ; Thu, 18 Nov 2010 23:34:51 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=Sz8g6EjDtfv+YRFrDtJOO0sRdFhq9IQnNyhXuzArwFLmqQPDPWP8RHR0nFoeFTWy6S I7PqgXV/sq2KXY0YLrAdTKELXgEjxBnuCopYWU0fpDsPhqEIBfJOqMvLkRYng2U4Yywx qFp8MI7UABIMQmpIxr3vOhkgzYUChOnN7VNg0=; bh=opeBrjyT2SsRhHhphWnDR8SO/Fs=; t=1290119694; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id e80e001397d19e90; Thu, 18 Nov 2010 17:34:54 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2E@SRV-QS-MAIL6.lands.nsw> In-Reply-To: <137CA4FE5CCDB7449ED3CD4445077AC3050AD1DA2E@SRV-QS-MAIL6.lands.nsw> Subject: RE: Windows and libssh2 Date: Thu, 18 Nov 2010 16:34:40 -0600 Message-ID: <003b01cb8770$ca6dc6f0$5f4954d0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuHbgqWtfDqj8qsQzyDtFbrIcyNRAAAdcgg Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Thu, 18 Nov 2010 23:35:00 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Thu, 18 Nov 2010 23:34:51 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Phillip wrote: > 1. Has anyone used (dynamic linking) for libssh2? > If so, is there somewhere I can get the code stubs for > all the libssh2 functions (perhaps as a "define")? I haven't done it, but it should be reasonably straightforward. I'm assuming you have a specific reason for dynamically linking... > 2. When I try to compile ssh2 example it is unable to find: > closesocket (etc.) > Does anyone know which dll on Windows have these system calls? WSock32.dll > 3. Would I be better off using MinGW or some other > build package for Windows? I've used VC 2008 with libssh2 and it worked well. I've found VC 2010 to be pretty similar, so it should work equally well. MinGW would probably only complicate things. You can take a look at the VC 2008 .vcproj file for my libssh2 sample at: http://trac.libssh2.org/attachment/ticket/197/sftpmrr.zip Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 19 05:21:31 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAJ4L6qW021210; Fri, 19 Nov 2010 05:21:27 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAJ4L3Wf021003 for ; Fri, 19 Nov 2010 05:21:05 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=QoM2izFxgjP/RgA5iuJcXlYwJOYOzieOAV9UqGz3yr8UHi4CJK971ZgJ+IJc4tWZul Q0eUV1Bozm7wcHi6AxyoJ7u3lSRLz/FAbAcpvmnDcucnXFfWEYC9FN1qUtN8VwHTLDB5 6b9vS4/5ErbXu37khltTnt2ChFCKeU7l4OOeU=; bh=6vtbb/x4HcRZcdE5vpKwP8w0JNE=; t=1290140471; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id 9c840013f527a081; Thu, 18 Nov 2010 23:21:10 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> <20101118034630.31957.qmail@stuge.se> <4CE4F520.2050405@0xbadc0de.be> In-Reply-To: <4CE4F520.2050405@0xbadc0de.be> Subject: RE: Updated libssh2 benchmark results Date: Thu, 18 Nov 2010 22:20:56 -0600 Message-ID: <000301cb87a1$2a291150$7e7b33f0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuHBVYkl0NU1I2jSe+xivGo+h8vqwAmbImQ Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 19 Nov 2010 05:21:31 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 19 Nov 2010 05:21:05 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Aris said: > I'm also interested in having libssh's results on your site as well > (both 0.4.6 and git master). I wrote a test program for libssh 0.4.6 and added the results here: http://60bits.net/sni/libssh2-perf.htm The program itself is here: http://trac.libssh2.org/attachment/ticket/197/libsshftpmrr.zip (I couldn't get the lastest git version to work - I had some hangs and access violations.) Briefly, libssh did well at uploading on a LAN, but not so well in the other scenarios. Separately, I also captured a Wireshark trace of the libssh2 test program uploading over a WAN, and put it here: http://trac.libssh2.org/attachment/ticket/197/libssh2-upload-wan.pcap It got a lot of [TCP ACKed lost segment] messages, which I don't understand because I gave it a big capture buffer. Regards, Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 04:35:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAL3ZGsJ019282; Sun, 21 Nov 2010 04:35:40 +0100 Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAL3ZEO6019277 for ; Sun, 21 Nov 2010 04:35:14 +0100 Received: by ewy3 with SMTP id 3so3658463ewy.41 for ; Sat, 20 Nov 2010 19:35:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=7EczwYK2Z7XoTGjLbMvXkI1h8SPHyR18rqVsQsxhruQ=; b=K2nUZd+Np0Ajrd+mO9qWuOYifW6ePnOzmmJbOGyBQWl8ajNLgFCtCoLPkXHXihbfEm 3jq1xlrOFf9Ti2mX+YJR2CB3U4eojrdGAP7QOBbqsr6wBaBCqShcIpOs2FEiBh94BZId UlotiwVAEjTSw1oTAC73XdYNYEEGbFjUB/dNw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=MgFknz1kDwSGfFHcQ5axVIj79QeMAsikzoUadEBsExPUBxGPjYlvQ02Rp9gdmGTRV3 OO66JU6SyO13OiFbQMaQ8RfKVz95AIOg+wBXs1eVjK7M8scyVyT9Hvez0gN5MDPIbwnh T1YpXklaZK7Fq2cjOxwsfzADdASKOe1D2vZy4= MIME-Version: 1.0 Received: by 10.14.11.147 with SMTP id 19mr2816703eex.14.1290310514340; Sat, 20 Nov 2010 19:35:14 -0800 (PST) Received: by 10.14.119.199 with HTTP; Sat, 20 Nov 2010 19:35:14 -0800 (PST) Date: Sat, 20 Nov 2010 19:35:14 -0800 Message-ID: Subject: SUA Interix From: "A. Mark" To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 04:35:44 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 04:35:14 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Hello! Anyone has any hints on how to build libSSH2 using libGCrypt under SUA Interix? I stomped at trying to build libGCrypt but was able to build gpg-error. Or alternately is there any binary builds available form somewhere either for libssh2 or libgcrypt? Thank you! _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 12:35:40 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALBZH4d008244; Sun, 21 Nov 2010 12:35:36 +0100 Received: from yxa-v.extundo.com (yxa-v.extundo.com [83.241.177.39]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALBZFZ8008237 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 21 Nov 2010 12:35:15 +0100 Received: from latte.josefsson.org (c80-216-27-64.bredband.comhem.se [80.216.27.64]) (authenticated bits=0) by yxa-v.extundo.com (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id oALBZ86Y021548 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Sun, 21 Nov 2010 12:35:11 +0100 From: Simon Josefsson To: libssh2 development Subject: Re: SUA Interix References: OpenPGP: id=B565716F; url=http://josefsson.org/key.txt X-Hashcash: 1:22:101121:libssh2-devel@cool.haxx.se::yAYCkWEtjyobxQDi:C0Bo Date: Sun, 21 Nov 2010 12:33:48 +0100 In-Reply-To: (A. Mark's message of "Sat, 20 Nov 2010 19:35:14 -0800") Message-ID: <87y68mhpwj.fsf@latte.josefsson.org> User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux) MIME-Version: 1.0 X-Spam-Status: No, score=0.0 required=5.0 tests=AWL,BAYES_00, DATE_IN_FUTURE_24_48,RDNS_DYNAMIC,SPF_FAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on yxa-v.extundo.com X-Virus-Scanned: clamav-milter 0.96.4 at yxa-v X-Virus-Status: Clean X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 12:35:40 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 12:35:15 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se "A. Mark" writes: > Hello! > > Anyone has any hints on how to build libSSH2 using libGCrypt under SUA Interix? > > I stomped at trying to build libGCrypt but was able to build gpg-error. There are some patches for libgcrypt on interix at: https://bugs.g10code.com/gnupg/issue993 If you quote what errors you get, maybe we could help -- however it is more appropriate to discuss that on the libgcrypt list. /Simon _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 13:46:27 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALCkCSb023011; Sun, 21 Nov 2010 13:46:25 +0100 Received: from rs49.luxsci.com (rs49.luxsci.com [65.61.166.75]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALCk9LK022884 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 21 Nov 2010 13:46:11 +0100 Received: from rs49.luxsci.com (localhost [127.0.0.1]) by rs49.luxsci.com (8.13.1/8.13.7) with ESMTP id oALCk0IY014289 for ; Sun, 21 Nov 2010 06:46:01 -0600 Received: (from root@localhost) by rs49.luxsci.com (8.13.1/8.13.7/Submit) id oALCj2BU013630 for libssh2-devel@cool.haxx.se; Sun, 21 Nov 2010 12:45:02 GMT Message-Id: <201011211245.oALCj2BU013630@rs49.luxsci.com> Received: (from sender 30582) by Lux Scientiae SP; Sun, 21 Nov 2010 12:45:02 +0000 Date: Sun, 21 Nov 2010 07:44:34 -0500 From: Ben Kibbey To: libssh2 development Subject: libssh2_knownhosts_writefile_fp() Content-Type: multipart/mixed; boundary="ibTvN161/egqYuK8" Content-Disposition: inline MIME-Version: 1.0 X-Comment: Lux Scientiae SP Message ID - 1290343502-6405714.25962994 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 13:46:27 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 13:46:11 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --ibTvN161/egqYuK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, I get a warning during linking about tempnam(3) being insecure. Heres a a patch to write the knownhosts to an already open file stream (which I create with tmpfile(3). -- Ben Kibbey [XMPP: bjk AT thiessen DOT im] - [IRC: (bjk) FreeNode/OFTC] --ibTvN161/egqYuK8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-libssh2_knownhost_writefile_fp.patch" From 89a3f8aa853ef8587b1634d7c9f51c3be61f440c Mon Sep 17 00:00:00 2001 From: Ben Kibbey Date: Sun, 21 Nov 2010 07:39:21 -0500 Subject: [PATCH] libssh2_knownhost_writefile_fp(). --- docs/libssh2_knownhost_writefile_fp.3 | 30 ++++++++++++++++++++++++++ include/libssh2.h | 10 ++++++++ src/knownhost.c | 38 ++++++++++++++++++++++---------- 3 files changed, 66 insertions(+), 12 deletions(-) create mode 100644 docs/libssh2_knownhost_writefile_fp.3 diff --git a/docs/libssh2_knownhost_writefile_fp.3 b/docs/libssh2_knownhost_writefile_fp.3 new file mode 100644 index 0000000..b672ae0 --- /dev/null +++ b/docs/libssh2_knownhost_writefile_fp.3 @@ -0,0 +1,30 @@ +.\" +.\" Copyright (c) 2009 by Daniel Stenberg +.\" +.TH libssh2_knownhost_writefile_fp 3 "21 Nov 2010" "libssh2 1.2" "libssh2 manual" +.SH NAME +libssh2_knownhost_writefile_fp - write a collection of known hosts to a file stream +.SH SYNOPSIS +#include + +int libssh2_knownhost_writefile_fp(LIBSSH2_KNOWNHOSTS *hosts, + FILE *fp, int type); +.SH DESCRIPTION +Writes all the known hosts to the specified file stream using the specified +file format. + +\fIfp\fP an open file stream + +\fItype\fP specifies what file type it is, and +\fILIBSSH2_KNOWNHOST_FILE_OPENSSH\fP is the only currently supported +format. +.SH RETURN VALUE +Returns a regular libssh2 error code, where negative values are error codes +and 0 indicates success. +.SH AVAILABILITY +Added in libssh2 1.2 +.SH SEE ALSO +.BR libssh2_knownhost_readfile(3) +.BR libssh2_knownhost_add(3) +.BR fopen(3) + diff --git a/include/libssh2.h b/include/libssh2.h index e011d49..63ab650 100644 --- a/include/libssh2.h +++ b/include/libssh2.h @@ -974,6 +974,16 @@ libssh2_knownhost_writeline(LIBSSH2_KNOWNHOSTS *hosts, char *buffer, size_t buflen, size_t *outlen, /* the amount of written data */ int type); +/* + * libssh2_knownhost_writefile_fp + * + * Write hosts+key pairs to a given file stream. + * + * This implementation currently only knows one 'type' (openssh), all others + * are reserved for future use. + */ +LIBSSH2_API int +libssh2_knownhost_writefile_fp(LIBSSH2_KNOWNHOSTS *hosts, FILE *fp, int type); /* * libssh2_knownhost_writefile diff --git a/src/knownhost.c b/src/knownhost.c index 29f1a1d..9e89f2d 100644 --- a/src/knownhost.c +++ b/src/knownhost.c @@ -1021,16 +1021,15 @@ libssh2_knownhost_writeline(LIBSSH2_KNOWNHOSTS *hosts, } /* - * libssh2_knownhost_writefile() + * libssh2_knownhost_writefile_fp() * - * Write hosts+key pairs to the given file. + * Write hosts+key pairs to the given file stream. */ LIBSSH2_API int -libssh2_knownhost_writefile(LIBSSH2_KNOWNHOSTS *hosts, - const char *filename, int type) +libssh2_knownhost_writefile_fp(LIBSSH2_KNOWNHOSTS *hosts, + FILE *fp, int type) { struct known_host *node; - FILE *file; int rc = LIBSSH2_ERROR_NONE; char buffer[2048]; @@ -1042,11 +1041,6 @@ libssh2_knownhost_writefile(LIBSSH2_KNOWNHOSTS *hosts, "Unsupported type of known-host information " "store"); - file = fopen(filename, "w"); - if(!file) - return _libssh2_error(hosts->session, LIBSSH2_ERROR_FILE, - "Failed to open file"); - for(node = _libssh2_list_first(&hosts->head); node; node= _libssh2_list_next(&node->node) ) { @@ -1057,7 +1051,7 @@ libssh2_knownhost_writefile(LIBSSH2_KNOWNHOSTS *hosts, if(rc) break; - nwrote = fwrite(buffer, 1, wrote, file); + nwrote = fwrite(buffer, 1, wrote, fp); if(nwrote != wrote) { /* failed to write the whole thing, bail out */ rc = _libssh2_error(hosts->session, LIBSSH2_ERROR_FILE, @@ -1065,11 +1059,31 @@ libssh2_knownhost_writefile(LIBSSH2_KNOWNHOSTS *hosts, break; } } - fclose(file); return rc; } +/* + * libssh2_knownhost_writefile() + * + * Write hosts+key pairs to the given file. + */ +LIBSSH2_API int +libssh2_knownhost_writefile(LIBSSH2_KNOWNHOSTS *hosts, + const char *filename, int type) +{ + FILE *file = fopen(filename, "w"); + int rc; + + if(!file) + return _libssh2_error(hosts->session, LIBSSH2_ERROR_FILE, + "Failed to open file"); + + rc = libssh2_knownhost_writefile_fp(hosts, file, type); + fclose(file); + return rc; +} + /* * libssh2_knownhost_get() -- 1.7.2.3 --ibTvN161/egqYuK8 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --ibTvN161/egqYuK8-- From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 13:53:56 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALCrp8c027187; Sun, 21 Nov 2010 13:53:55 +0100 Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALCroIP027149 for ; Sun, 21 Nov 2010 13:53:50 +0100 Received: by qyk35 with SMTP id 35so1138900qyk.20 for ; Sun, 21 Nov 2010 04:53:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=dMH6lEJsbEtfcX644gfVRIFXORocaSVAjaT51iI9lgs=; b=O7/HyQWc7sa8m/9W92r76vfrCE0LbOMKGxeu+aXf+NBAmXuWW+pt49v/k6V/CBf3yj vZIVhsvg1KZhJ8sXwO9DAO8EaXvQmmOH78UMAVKABf7MwQCxogWZx+VQ94E4S/8uOfD9 osEm1/9MZgSPC6bn9OrrQQFUhYbT2duvEacgc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=nTK3yibs6mFrp9Wmj9w1Mjpe7qzcsCx8qmXPvHOuSY3xASlkj9kTgIZ+9P3F4U2Lbo PX5IhMosT8EURDFmzkJtuTQTQUXt53HZEit09Cok7la/igupknaiSW9l0JI1cYDu6uZw asJZxmK1PEOt8mb3uxJcDyT1nXvQY5Hq27lWo= MIME-Version: 1.0 Received: by 10.229.101.197 with SMTP id d5mr3069498qco.143.1290344023435; Sun, 21 Nov 2010 04:53:43 -0800 (PST) Received: by 10.229.80.213 with HTTP; Sun, 21 Nov 2010 04:53:43 -0800 (PST) In-Reply-To: <201011211245.oALCj2BU013630@rs49.luxsci.com> References: <201011211245.oALCj2BU013630@rs49.luxsci.com> Date: Sun, 21 Nov 2010 12:53:43 +0000 X-Google-Sender-Auth: UCv0eM3AzZWIcUFAeL_fKq-Eos4 Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 13:53:56 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 13:53:51 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 21 November 2010 12:44, Ben Kibbey wrote: > > I get a warning during linking about tempnam(3) being insecure. Heres a > a patch to write the knownhosts to an already open file stream (which I > create with tmpfile(3). Passing a FILE* across an API call is a really bad idea. Unless you're linking statically, this can corrupt the C-runtime memory as you're passing an object owned by one runtime instance to another. Although there are a couple of calls in libssh2 that still do this, we're trying to get rid of them. Alex _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 14:36:20 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDaCol030217; Sun, 21 Nov 2010 14:36:19 +0100 Received: from rs49.luxsci.com (rs49.luxsci.com [65.61.166.75]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDa8V4030035 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 21 Nov 2010 14:36:10 +0100 Received: from rs49.luxsci.com (localhost [127.0.0.1]) by rs49.luxsci.com (8.13.1/8.13.7) with ESMTP id oALDa11u020450 for ; Sun, 21 Nov 2010 07:36:01 -0600 Received: (from root@localhost) by rs49.luxsci.com (8.13.1/8.13.7/Submit) id oALDZ2xs019898 for libssh2-devel@cool.haxx.se; Sun, 21 Nov 2010 13:35:02 GMT Message-Id: <201011211335.oALDZ2xs019898@rs49.luxsci.com> Received: (from sender 30582) by Lux Scientiae SP; Sun, 21 Nov 2010 13:35:02 +0000 Date: Sun, 21 Nov 2010 08:34:51 -0500 From: Ben Kibbey To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_knownhosts_writefile_fp() References: <201011211245.oALCj2BU013630@rs49.luxsci.com> Content-Disposition: inline In-Reply-To: MIME-Version: 1.0 X-Comment: Lux Scientiae SP Message ID - 1290346502-8312399.49946692 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 14:36:20 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 14:36:10 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, Nov 21, 2010 at 12:53:43PM +0000, Alexander Lamaison wrote: > On 21 November 2010 12:44, Ben Kibbey wrote: > > > > I get a warning during linking about tempnam(3) being insecure. Heres a > > a patch to write the knownhosts to an already open file stream (which I > > create with tmpfile(3). > > Passing a FILE* across an API call is a really bad idea. Unless > you're linking statically, this can corrupt the C-runtime memory as > you're passing an object owned by one runtime instance to another. > Although there are a couple of calls in libssh2 that still do this, > we're trying to get rid of them. Is it only the FILE* structure? What about the file descriptor of the opened stream obtained from fileno(3)? Is that safe? -- Ben Kibbey [XMPP: bjk AT thiessen DOT im] - [IRC: (bjk) FreeNode/OFTC] _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 14:49:08 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDmuEX019454; Sun, 21 Nov 2010 14:49:06 +0100 Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDmrvg019336 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT) for ; Sun, 21 Nov 2010 14:48:54 +0100 Received: by qyk8 with SMTP id 8so1332962qyk.20 for ; Sun, 21 Nov 2010 05:48:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=iucTMzhQ6qy2guWtsCHEpH/8D6ruJWKa2kEEGuio5xk=; b=BgMpH8TVRzWAwOUahkh5HJu5HTrvQBeDl3ev1bpwovQGBooSznyY9DRd4MmWWqYqpF MRnxyXPRQLKO7183QCb+YegsCzic06Z+d2BWfBqsTK+WEN8Sxx3JY5zwQ+6SGJczsinw IiIhorZwZTT//2cxh9BRAv0SDDgbsc0Mip1z8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=Y81vhazs/QF4MrnSqPRj/x6duwyFCLvVV3kFtsqBT9YTJqW2v2PWi3+yolTp5huhYv WaNPEE8Jlfj/MgijXBeYiOO2+FHQ1gZ+K1EQkchR2B2UCjjiKZAb3yAD9/W5B3UMQSO6 hEGLqOhCxIME1l7ZnwkIyuUhakm91WRExBTMI= MIME-Version: 1.0 Received: by 10.229.184.143 with SMTP id ck15mr3990141qcb.181.1290347325757; Sun, 21 Nov 2010 05:48:45 -0800 (PST) Received: by 10.229.80.213 with HTTP; Sun, 21 Nov 2010 05:48:45 -0800 (PST) In-Reply-To: <201011211335.oALDZ2xs019898@rs49.luxsci.com> References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 13:48:45 +0000 X-Google-Sender-Auth: XiKRPcerPUKUYdv80VnxKvFsbPw Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 14:49:08 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 14:48:54 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 21 November 2010 13:34, Ben Kibbey wrote: > On Sun, Nov 21, 2010 at 12:53:43PM +0000, Alexander Lamaison wrote: >> On 21 November 2010 12:44, Ben Kibbey wrote: >> > >> > I get a warning during linking about tempnam(3) being insecure. Heres a >> > a patch to write the knownhosts to an already open file stream (which I >> > create with tmpfile(3). >> >> Passing a FILE* across an API call is a really bad idea. =A0Unless >> you're linking statically, this can corrupt the C-runtime memory as >> you're passing an object owned by one runtime instance to another. >> Although there are a couple of calls in libssh2 that still do this, >> we're trying to get rid of them. > > Is it only the FILE* structure? What about the file descriptor of the > opened stream obtained from fileno(3)? Is that safe? Also not safe. Any 'object' that belongs to one instance of the C-runtime must not be manipulated by another instance. The file descriptor data is owned by the C-runtime regardless of whether you access it via a fileno or a FILE*. It's a bit like calling malloc() in one instance and calling free() in the other which will also corrupt the runtime memory and most likely crash. While the memory allocated by malloc() is not owned by the C-runtime (you can access the memory anywhere you wish), the table of memory allocations *is*. When you call another instance's free() on the pointer, it is the table of allocations that is changed, not the allocated memory. As each runtime instance has its own table, boo boo ensues. Unfortunately, as far as I'm aware, every file operation manipulates internal tables so you can never safely use any form of opened file descriptor across API calls. Alex _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 14:58:13 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDw5tY024301; Sun, 21 Nov 2010 14:58:12 +0100 Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALDw3Ii024186 for ; Sun, 21 Nov 2010 14:58:03 +0100 Received: by qwf6 with SMTP id 6so651821qwf.41 for ; Sun, 21 Nov 2010 05:57:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=dwJxERIqpjo+G2cYYQP32MeUP64/yT7KT02EQxkT+Kg=; b=QZuMyyFfCNFIgde4hKfszyYT9L9VEfawi0MkvbphkMjqChNCSALkumZY3PLcTEypDX LTt9lvWTk33r0nVZ7izdoYGvOzxQp+LO8tWFDrCmd2sqp6o+z6dEOf6q8xem7yXOGvJc 5+gfVbt2n4zs7YFNsftKR6tCRnL/XEjKSeYxo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=WOw7vZNtqoyLDN5STePVUhh/0xwdgQBPKxkArpOuxCfsvWXBQcxV4UEtBzwmIQt6Yx 0UyeciX/Dz/VTawgbk4mYzk6AQMDO+wOsI46DQQ3UNrL3N2vUHLsXygOcJjN4Anq+Gjy C2axvO7HEI4ot1MxxwZj2WOPoLa97HdfzMQjA= MIME-Version: 1.0 Received: by 10.229.80.2 with SMTP id r2mr3925917qck.170.1290347875043; Sun, 21 Nov 2010 05:57:55 -0800 (PST) Received: by 10.229.80.6 with HTTP; Sun, 21 Nov 2010 05:57:55 -0800 (PST) In-Reply-To: References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 14:57:55 +0100 X-Google-Sender-Auth: r1DcO93s1VKyMuiMfJ_RRfKYXoI Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Tor Arntsen To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 14:58:13 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 14:58:03 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, Nov 21, 2010 at 14:48, Alexander Lamaison wrote: > Also not safe. =A0Any 'object' that belongs to one instance of the > C-runtime must not be manipulated by another instance. =A0The file > descriptor data is owned by the C-runtime regardless of whether you > access it via a fileno or a FILE*. > > It's a bit like calling malloc() in one instance and calling free() in > the other which will also corrupt the runtime memory and most likely > crash. =A0While the memory allocated by malloc() is not owned by the > C-runtime (you can access the memory anywhere you wish), the table of > memory allocations *is*. =A0When you call another instance's free() on > the pointer, it is the table of allocations that is changed, not the > allocated memory. =A0As each runtime instance has its own table, boo boo > ensues. > > Unfortunately, as far as I'm aware, every file operation manipulates > internal tables so you can never safely use any form of opened file > descriptor across API calls. The above is Windows specfic though. On Unix you can program against libA and libB and a malloc from one can safely be free'd by the other, if you so wish. (It's of course possible to manually insert local versions of any libc function and mess this up, but there's only a single instance of the libc runtime even if there are multiple API libs involved. Passing a FILE * from one lib to another is quite common, and safe on Unix. Not so on Windows, from what I understand.) -Tor _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 15:18:14 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALEI6tS007497; Sun, 21 Nov 2010 15:18:12 +0100 Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALEI5Jc007406 for ; Sun, 21 Nov 2010 15:18:05 +0100 Received: by qyk35 with SMTP id 35so1185506qyk.20 for ; Sun, 21 Nov 2010 06:17:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=owS7pZLA9AF/GswxjlKi9f/WTKKGqS9Wxv1OnsItjI8=; b=gixlqvnXU8+RUB1ga9MQ6YkpuKFUaKGSNtiaJuDp5imrH27y8bYJbWT8kfiKF3bdEi 94PYtJs4o75v06qYrqzfY5JazKHyaL2jkxCA478EEpDogdZTQRea6tDV1IS4fVqqcxB5 0s3a0UjFsC0cQ2Q8OXBid6pIK8xSn7GE2nCaw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=uowP+aUsWmrtDQ1+JGBxFK8bifKW2yruQHJFOVvLJycErlRtlDMaYw5ZOz8AWY63pQ n1JM48zccGXjVzsWHlxkifi5OKa4UZ+UFBtCnl8gHrkLCHT9eZg1IOtHYR8o07z8L9sR WYnbx2Dkb8TGwrnxd6LUa8nwelduOJtiUhuwk= MIME-Version: 1.0 Received: by 10.229.101.197 with SMTP id d5mr3134167qco.143.1290349077941; Sun, 21 Nov 2010 06:17:57 -0800 (PST) Received: by 10.229.80.213 with HTTP; Sun, 21 Nov 2010 06:17:57 -0800 (PST) In-Reply-To: References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 14:17:57 +0000 X-Google-Sender-Auth: 5iIly_K5GZk4HrE6DVdMCcTGqN8 Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 15:18:13 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 15:18:05 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 21 November 2010 13:57, Tor Arntsen wrote: > On Sun, Nov 21, 2010 at 14:48, Alexander Lamaison wro= te: > >> Also not safe. =A0Any 'object' that belongs to one instance of the >> C-runtime must not be manipulated by another instance. =A0The file >> descriptor data is owned by the C-runtime regardless of whether you >> access it via a fileno or a FILE*. >> >> It's a bit like calling malloc() in one instance and calling free() in >> the other which will also corrupt the runtime memory and most likely >> crash. =A0While the memory allocated by malloc() is not owned by the >> C-runtime (you can access the memory anywhere you wish), the table of >> memory allocations *is*. =A0When you call another instance's free() on >> the pointer, it is the table of allocations that is changed, not the >> allocated memory. =A0As each runtime instance has its own table, boo boo >> ensues. >> >> Unfortunately, as far as I'm aware, every file operation manipulates >> internal tables so you can never safely use any form of opened file >> descriptor across API calls. > > The above is Windows specfic though. On Unix you can program against > libA and libB and a malloc from one can safely be free'd by the other, > if you so wish. (It's of course possible to manually insert local > versions of any libc function and mess this up, but there's only a > single instance of the libc runtime even if there are multiple API > libs involved. Passing a FILE * from one lib to another is quite > common, and safe on Unix. Not so on Windows, from what I understand.) This is a myth (one I've discussed at length at various times on this list). You're right that it is much more common on Windows than Unix but this is luck rather than design. It relies on the fact that on Unix every .so is typically linked against the same version of libc so only one instance is ever loaded. So, you're lucky, on most Unix systems your software won't crash ... until it does. Passing a FILE* between your code and the _C-runtime that created it_ is safe. Anything else is not. Alex _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 16:27:36 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALFREfs021757; Sun, 21 Nov 2010 16:27:32 +0100 Received: from rs49.luxsci.com (rs49.luxsci.com [65.61.166.75]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALFRA2O021744 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 21 Nov 2010 16:27:12 +0100 Received: from rs49.luxsci.com (localhost [127.0.0.1]) by rs49.luxsci.com (8.13.1/8.13.7) with ESMTP id oALFR2fw012019 for ; Sun, 21 Nov 2010 09:27:02 -0600 Received: (from root@localhost) by rs49.luxsci.com (8.13.1/8.13.7/Submit) id oALFQ2wh011265 for libssh2-devel@cool.haxx.se; Sun, 21 Nov 2010 15:26:02 GMT Message-Id: <201011211526.oALFQ2wh011265@rs49.luxsci.com> Received: (from sender 30582) by Lux Scientiae SP; Sun, 21 Nov 2010 15:26:02 +0000 Date: Sun, 21 Nov 2010 10:25:16 -0500 From: Ben Kibbey To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_knownhosts_writefile_fp() References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Content-Disposition: inline In-Reply-To: MIME-Version: 1.0 X-Comment: Lux Scientiae SP Message ID - 1290353162-3250785.55043881 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 16:27:36 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 16:27:13 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, Nov 21, 2010 at 01:48:45PM +0000, Alexander Lamaison wrote: > > Is it only the FILE* structure? What about the file descriptor of the > > opened stream obtained from fileno(3)? Is that safe? > > Also not safe. Any 'object' that belongs to one instance of the > C-runtime must not be manipulated by another instance. The file > descriptor data is owned by the C-runtime regardless of whether you > access it via a fileno or a FILE*. > > It's a bit like calling malloc() in one instance and calling free() in > the other which will also corrupt the runtime memory and most likely > crash. While the memory allocated by malloc() is not owned by the > C-runtime (you can access the memory anywhere you wish), the table of > memory allocations *is*. When you call another instance's free() on > the pointer, it is the table of allocations that is changed, not the > allocated memory. As each runtime instance has its own table, boo boo > ensues. > > Unfortunately, as far as I'm aware, every file operation manipulates > internal tables so you can never safely use any form of opened file > descriptor across API calls. I'm sorry if I'm asking the same question, but what about a file descriptor returned from open(2)? The data structures rely on the OS rather than libraries don't they? -- Ben Kibbey [XMPP: bjk AT thiessen DOT im] - [IRC: (bjk) FreeNode/OFTC] _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 17:09:36 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALG9Oh2016843; Sun, 21 Nov 2010 17:09:35 +0100 Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALG9McG016831 for ; Sun, 21 Nov 2010 17:09:22 +0100 Received: by qwf6 with SMTP id 6so734772qwf.41 for ; Sun, 21 Nov 2010 08:09:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=RoQ/ouE0rBEGA8R75d1aayt8tydHM40fwdglJma0bMs=; b=jtgxoJhCMQtBwcg7kCMa5GuuAa2eXw9jx34VrHC5csxt4s3B8ZriSBl/pW2QijrCJd oxt25IOulyLWNXlGttICO2CFh9No9tgi/JgSXTbRhS/2ITGR4N8aO3XQ1CY+2sj9qQ6w Mf5JYCCV112/aTNUZNpkeHbLonZOn0kCnYDyE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=kP0Xl7r2djZf6isLvK70UI9Nn06QOTFqQe1/A7s3EpJnOowBkx4GxM86lAU14URf8c gWqyS9z3jPYptvJwrTpxbnzJFzwa0XzeclDH5OGcXAt7pCxkkKhkgLBNZLDFQtbxatOy lEP23idPmXzFEx5KqvBZDtnYyWCFZAHLWTSGk= MIME-Version: 1.0 Received: by 10.229.230.10 with SMTP id jk10mr4069679qcb.105.1290355754610; Sun, 21 Nov 2010 08:09:14 -0800 (PST) Received: by 10.229.80.213 with HTTP; Sun, 21 Nov 2010 08:09:14 -0800 (PST) In-Reply-To: <201011211526.oALFQ2wh011265@rs49.luxsci.com> References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> <201011211526.oALFQ2wh011265@rs49.luxsci.com> Date: Sun, 21 Nov 2010 16:09:14 +0000 X-Google-Sender-Auth: 1Y-nRj-SdZkwUlkiCt4hogbCq38 Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 17:09:36 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 17:09:23 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 21 November 2010 15:25, Ben Kibbey wrote: > On Sun, Nov 21, 2010 at 01:48:45PM +0000, Alexander Lamaison wrote: >> > Is it only the FILE* structure? What about the file descriptor of the >> > opened stream obtained from fileno(3)? Is that safe? >> >> Also not safe. >> >> Unfortunately, as far as I'm aware, every file operation manipulates >> internal tables so you can never safely use any form of opened file >> descriptor across API calls. > > I'm sorry if I'm asking the same question, but what about a file > descriptor returned from open(2)? The data structures rely on the OS > rather than libraries don't they? On Windows, at least, open() is provided by the C-runtime therefore subject to all the earlier caveats. On Linux it appears that open() can be provided by the C-runtime or directly as a system call depending on how you link but I'm not sure about this. Basically, you can't rely on it being a system call on all platforms. I find your problem interesting and it's not one I've considered before: where you have to pass an opened file for security, to prevent a third-party hijacking the filename. The only safe solution I can think of is to add a call to libssh2 that takes the knownhosts file as a buffer in memory. Then you can read it in yourself and just pass the buffer across the API call. We already do this libssh2_userauth_publickey(), for instance. Alex P.S. What I said earlier about still having some uses of FILE* in the API isn't true. There are none :) I was getting confused with libssh2's use of OpenSSL which used to use some of OpenSSL FILE* APIs. _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 17:13:45 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALGDfvM018316; Sun, 21 Nov 2010 17:13:43 +0100 Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALGDdZE018307 for ; Sun, 21 Nov 2010 17:13:40 +0100 Received: by qwf6 with SMTP id 6so737635qwf.41 for ; Sun, 21 Nov 2010 08:13:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=sAgjXHGB2RjDfemNxAiZjkAMO+m7XsM28Mo6+lj1Zds=; b=Uh7En40LL0MO6XNi5atNW+aD1iAVrCuWvMs7GDuvqGWKH0H6aYlYU1Pn5r2i9ZY9kR TdLyO4KJEVyAm0G9W1I//lGoYruexgMiJvDLnYajs8dkNBiXTtcQNY0Mcdf+AkVveHpe BGde/AorugdhGY3pkZ4ItnU09ztd54g6Cxi54= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=F6sJ8tV7OAsQXu0FijDAU1oaabpe3LOWOjkohH3epJofp3/JzWpp3zB9r8mNM2ajwH ICM5RfBetmZ9nDK8Tgb6skpuJ8KAtR2EEXLuEp67/hUhPZ63/TkgebnpsRXM7TILoonM SUB24Ur1OVOKA9quqJukG1NMowQTI01Ofsfbk= MIME-Version: 1.0 Received: by 10.229.85.196 with SMTP id p4mr4017128qcl.132.1290356012004; Sun, 21 Nov 2010 08:13:32 -0800 (PST) Received: by 10.229.80.6 with HTTP; Sun, 21 Nov 2010 08:13:31 -0800 (PST) In-Reply-To: References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 17:13:31 +0100 X-Google-Sender-Auth: qON3t3OyI-xSbyeAJwVzl4wmWHQ Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Tor Arntsen To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 17:13:45 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 17:13:40 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, Nov 21, 2010 at 15:17, Alexander Lamaison wrote: > This is a myth (one I've discussed at length at various times on this > list). =A0You're right that it is much more common on Windows than Unix > but this is luck rather than design. =A0It relies on the fact that on > Unix every .so is typically linked against the same version of libc so > only one instance is ever loaded. =A0So, you're lucky, on most Unix > systems your software won't crash ... until it does. > > Passing a FILE* between your code and the _C-runtime that created it_ > is safe. =A0Anything else is not. Er, no. It's not a myth, and certainly not related to luck. It would be a weird setup indeed if you manage to link a process so that it ends up with two different C runtime libraries. It's actually very difficult to do. For a normal Unix executable to be linked against two different libc.so.. I've yet to see it, and I've been working with all the major *nix systems. You would have to do tricks with static linking to achieve something like that (AIX has some weird way of handling multiple dynamic libc versions at the same time, but even there I can't actually think of how to achieve that inside the same process in any easy way.) Thus, passing FILE * between libs is perfectly safe, and it's in fact _by_ design and not the opposite.. and when it comes to file descriptors (ints as returned by open(2), not FILE *), you can even pass them between processes, as when in forking and passing on open file descriptors, or pass them via UNIX domain sockets. You'll find a great number of *nix libraries wich take a FILE * as function arguments, including X11, OpenSSL, graphviz, and many others. But if this is an issue on Windows, and we want libssh2 to be Windows compatible, then we'll have to stick to the least common denominator and make sure that it works there too. Ben Kibbey wrote: >I'm sorry if I'm asking the same question, but what about a file >descriptor returned from open(2)? The data structure rely on the OS >rather than libraries don't they? In *nix it's handled by the kernel and there aren't any issues. But I don't know how it's done on Windows. -Tor _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 17:49:06 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALGmpKK003780; Sun, 21 Nov 2010 17:49:03 +0100 Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALGmnSd003711 for ; Sun, 21 Nov 2010 17:48:50 +0100 Received: by qwf6 with SMTP id 6so760901qwf.41 for ; Sun, 21 Nov 2010 08:48:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=SJe894RaIbfQzjr95TA43/JYideoJC6PvCE3OygS/ig=; b=guJDQ98AnqG6nXD2doocPD+bVmIvF2MawAR3x8pyIfAbIcVSWCdBm7fMI6gpxgmpRD 55Wh2qMPMrs2N4gmP9g6R97uI6FKQrPs94Wwk5/Nc7TZio+9yybeR86DEBW8VN9Lgqki Q2Re7F9FfBn5TMhFOftsp1/CxPgDIbTH0gwrk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=hiVaLUTydKc79ltZfirb1Fm9XTrNqr4pV8/xqm8cl8q8N6B3Giv59BjhBKfE9nscnb atCV4SCoc9jlIikKB4THPwt4IRRO5T1z06q3h4xuMEtvvKnjGuwpH2E3EuavERADz3r6 55rN3h5Dq8vRsmrfFzBoolUEfUjeyN8296GRY= MIME-Version: 1.0 Received: by 10.229.229.200 with SMTP id jj8mr4055900qcb.295.1290358121979; Sun, 21 Nov 2010 08:48:41 -0800 (PST) Received: by 10.229.80.213 with HTTP; Sun, 21 Nov 2010 08:48:41 -0800 (PST) In-Reply-To: References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 16:48:41 +0000 X-Google-Sender-Auth: L9jyxwIt-MKyEnP1j8U-SJCkvFY Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Alexander Lamaison To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 17:49:06 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 17:48:50 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On 21 November 2010 16:13, Tor Arntsen wrote: > On Sun, Nov 21, 2010 at 15:17, Alexander Lamaison wro= te: > >> This is a myth (one I've discussed at length at various times on this >> list). =A0You're right that it is much more common on Windows than Unix >> but this is luck rather than design. =A0It relies on the fact that on >> Unix every .so is typically linked against the same version of libc so >> only one instance is ever loaded. =A0So, you're lucky, on most Unix >> systems your software won't crash ... until it does. >> >> Passing a FILE* between your code and the _C-runtime that created it_ >> is safe. =A0Anything else is not. > > Er, no. It's not a myth, and certainly not related to luck. It would > be a weird setup indeed if you manage to link a process so that it > ends up with two different C runtime libraries. It's actually very > difficult to do. For a normal Unix executable to be linked against two > different libc.so.. I've yet to see it, and I've been working with all > the major *nix systems. You would have to do tricks with static > linking to achieve something like that (AIX has some weird way of > handling multiple dynamic libc versions at the same time, but even > there I can't actually think of how to achieve that inside the same > process in any easy way.) Sure, it's hard to do on Linux, no denying that. If it weren't it would be forbidden to use FILE*s in API rather than simply not recommended. The point is that this isn't a Windows problem. It will affect any OS where you run a program that you have managed to link to two C-runtimes. > Thus, passing FILE * between libs is perfectly safe, and it's in fact > _by_ design and not the opposite.. and when it comes to file > descriptors (ints as returned by open(2), not FILE *), you can even > pass them between processes, as when in forking and passing on open > file descriptors, or pass them via UNIX domain sockets. You're probably right. My main objection to this should have been that file descriptors are POSIX rather than C and therefore not as portable. Just out of interest, what happens on Linux in this situation: libA.so links to libc.so.1 and libB.so links to libc.so.2 and your program links to libA.so and libB.so? Then you pass something from libA across a call to libB. Would calls normally be redirected to a single libc? Alex _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sun Nov 21 18:36:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALHaYmJ001831; Sun, 21 Nov 2010 18:36:43 +0100 Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oALHaWkW001806 for ; Sun, 21 Nov 2010 18:36:32 +0100 Received: by qyk35 with SMTP id 35so1313274qyk.20 for ; Sun, 21 Nov 2010 09:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=e0C5JOADN3RacTB2sg3kljSsONkNa6lK3X1zAv65w48=; b=S8+1py5xfi8tIj558mvqaMdn4QjTMEc/ipkinpZet6gQ3L/Kgs9s8f9Eg+Q42Wj8Ib nIEXoQti70YPpuT+FgI/ajSHW9HTN1A2zUh0/VfDPSC7tMi6VSA9jyBJusQlO59iVmf4 j/GWfgW1Wfzr8dWLHNI6XU2dsOs0dnAgK2+vI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=Q0GzeMfS6pRJvkCu0dZX/DtG30MSEM12ibNfWP3ttU/TEnPTi294ZKKAlx0l4dqiUt JdgfzwBJOuRqgxA2gHgttoU1SuGwnlMesf3grg0aFrs05ar6CVwoHGrXyVlpyNp09ZGE yH5hpCCqZIHRcgie7PutDrwZJSe3g1j8b0at0= MIME-Version: 1.0 Received: by 10.229.84.203 with SMTP id k11mr4123909qcl.56.1290360983294; Sun, 21 Nov 2010 09:36:23 -0800 (PST) Received: by 10.229.80.6 with HTTP; Sun, 21 Nov 2010 09:36:23 -0800 (PST) In-Reply-To: References: <201011211245.oALCj2BU013630@rs49.luxsci.com> <201011211335.oALDZ2xs019898@rs49.luxsci.com> Date: Sun, 21 Nov 2010 18:36:23 +0100 X-Google-Sender-Auth: nUk8shhtC5YsqBRyBR8D3I2r7SU Message-ID: Subject: Re: libssh2_knownhosts_writefile_fp() From: Tor Arntsen To: libssh2 development X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sun, 21 Nov 2010 18:36:43 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sun, 21 Nov 2010 18:36:32 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Sun, Nov 21, 2010 at 17:48, Alexander Lamaison wrote: > Just out of interest, what happens on Linux in this situation: libA.so > links to libc.so.1 and libB.so links to libc.so.2 and your program > links to libA.so and libB.so? =A0 Then you pass something from libA > across a call to libB. =A0Would calls normally be redirected to a single > libc? Well, first of all it is difficult to do :-) Mostly because you won't normally find a system with two different major versions of libc on a Linux system, and it's even harder on most Unix systems. So your apps and dynamic libraries would be linked against a libc.so.X, and although the original lib or application was built against libc.so.X.Y, at runtime Y will be whatever is installed at runtime, i.e. Y may be a later version. I did manage to find a system with two libc versions though. It's an old Linux system with linux kernel 2.0.38, and it contains a single remaining dynamic library which still links to libc.so.5, when everything else links to libc.so.6 (which every application and lib from the last decade will). The only reason this particular system has that mix is because it's the remains of my earliest 1992 Linux system which has been hand-upgraded over the years. In any case I made a test program which accessed that old library (which linked to libc.so.5) plus some normal file operations, and as the standard linking sequence automatically includes libc.so.6 I managed to end up with an application refering to both. An strace during the runtime dynamic linking sequence (i.e. startup) shows both libs being linked in: open("/lib/libc.so.6", O_RDONLY) =3D 4 open("/lib/libc.so.5", O_RDONLY) =3D 4 Now, what actually happens when doing file operations in this program I don't know. I don't have headers around for that old library (the one linked against libc.so.5) so I could only call some functions that I found with 'nm', I don't know what they do and I don't know if anyone of them would take e.g. a FILE* parameter, or a malloced area. (Note that in practice you wouldn't be able to end up with such an application as I just made because if you just link with the installed .so libraries you'll find that the .so version of the offending library points (via symlink) to a newer, libc.so.6-linked version. The old version may still be in use by an old application, and _that_ application will also be linked to libc.so.5. You really have to do such a trick manually if you want to get into that situation: You can have multiple major versions of a library installed, for compatibility with old applications, but you can have only a single set of header files and that set is the one which goes with the single .so file you can have installed. The .so will point to the latest .so.X version.) -Tor _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 23 10:09:49 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAN98gBj016720; Tue, 23 Nov 2010 10:09:44 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAN98etA016710 for ; Tue, 23 Nov 2010 10:08:40 +0100 Date: Tue, 23 Nov 2010 10:08:40 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Updated libssh2 benchmark results In-Reply-To: <000001cb86cd$7e7f22f0$7b7d68d0$@com> Message-ID: References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 23 Nov 2010 10:09:49 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 23 Nov 2010 10:08:40 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Wed, 17 Nov 2010, Mark Riordan wrote: > I ran a bunch more performance tests of the latest libssh2, and some other > SSH clients. Great! I'm on vacation right now so I won't dig into this until december, but I appreciate your data. I also wanted to mention that it might be worth trying different sizes and thresholds in the libssh2 code to see what (if any) differences they make. If anyone has a decent test setup, like perhaps just a ssh server not too close to me and is willing to offer me an account for further work on this, then please let me know. My own dev situation is a bit limited for this kind of task. -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Tue Nov 23 16:50:46 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oANFoOV2018301; Tue, 23 Nov 2010 16:50:42 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oANFoL5t018278 for ; Tue, 23 Nov 2010 16:50:22 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=rxjDK1mkXOqfBRPHH/H8l0X6ueCT2HZ1p/VXmdE8kmGaXHRJkbBJixf/0QUzuq+T2q GLRXjH6HDC8uKeK0odHTFgocnDQplQXfdkEiFX+x52jqpMimgXMEiyhfQX9as+iwPA9D Abh/+Z30NBtz1D1iu/qIjNiAaqJpkeasOHLCA=; bh=Jtf8l0OVwoZmSIqD3SS7i3TiJB0=; t=1290527413; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id e8380001469d07ab; Tue, 23 Nov 2010 10:50:13 -0500 From: "Mark Riordan" To: "'libssh2 development'" References: <000001cb86cd$7e7f22f0$7b7d68d0$@com> In-Reply-To: Subject: RE: Updated libssh2 benchmark results Date: Tue, 23 Nov 2010 09:50:13 -0600 Message-ID: <000001cb8b26$1df5b090$59e111b0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuK7m+jMrZ9jEsbQgya5GmBo6QaLQANqXZA Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 23 Nov 2010 16:50:46 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 23 Nov 2010 16:50:23 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se Daniel said: >Great! I'm on vacation right now so I won't dig into this until december, but >I appreciate your data. I also wanted to mention that it might be worth trying >different sizes and thresholds in the libssh2 code to see what (if any) >differences they make. > >If anyone has a decent test setup, like perhaps just a ssh server not too >close to me and is willing to offer me an account for further work on this, >then please let me know. My own dev situation is a bit limited for this kind >of task. Sounds good - I appreciate it. I will create an account for you on my personal SSH-accessible website at 1and1.com. I believe it's located in Kansas, USA. I'll forward the info to you under separate cover. Have a great vacation. Mark _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Wed Nov 24 23:20:12 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAOMJm5w011407; Wed, 24 Nov 2010 23:20:07 +0100 Received: from imail.ipswitch.com (imail.ipswitch.com [156.21.1.5]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAOMJjlD011389 for ; Wed, 24 Nov 2010 23:19:46 +0100 DKIM-Signature: q=dns/txt; v=1; c=relaxed/relaxed; s=ipswitch; d=ipswitch.com; a=rsa-sha1; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding; b=oOkLSN1Iey5rhs8/Gel3q5jcMuqEhLaXogRlKgh50c4aBBUXPp7fVXZnzO58IBhOvD Qui1R7e1v/Bhlc+217bcJhzsGsZIdP70Gz0lnDYu75gEU/zIWja39Yh4VZEoOvJYbMFF w5zFRhfrq+NxkrdpKs2eM06Xt26Y2p08ZiHF0=; bh=3j1h5t5q4h3wQxw7yRmlAP1Lggo=; t=1290637176; Received: from mrrnt [66.170.5.130] by ipswitch.com with ESMTP (SMTPD-11.03) id a2f00002340b1255; Wed, 24 Nov 2010 17:19:34 -0500 From: "Mark Riordan" To: "'libssh2 development'" Subject: How to append to remote file? Date: Wed, 24 Nov 2010 16:19:34 -0600 Message-ID: <001d01cb8c25$acd5ea50$0681bef0$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuMJayJ55dVmoleT16az1sPmb8u2w== Content-Language: en-us X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Wed, 24 Nov 2010 23:20:12 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Wed, 24 Nov 2010 23:19:47 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se How does one append to a remote file using libssh2? (Nevermind that appending to a file usually isn't a good idea.) I tried doing a libssh2_sftp_fstat_ex on the remote file to find its size, then seeking to the end of the file using libssh2_sftp_seek64. But libssh2_sftp_seek64 isn't a real seek. In this case, it causes subsequent calls to libssh2_sftp_write to write 0 bytes. Is there a real remote seek or append capability? I'm reluctant to open the file with LIBSSH2_FXF_APPEND, because of this comment in the code: "Danger will robinson... APPEND doesn't have any effect on OpenSSH servers" At any rate, I eventually tried the LIBSSH2_FXF_APPEND approach and got the same problem. I used a different SSH client to append a file to the same OpenSSH server and it did work. Sample code below. Thanks. Mark R remotemode = LIBSSH2_FXF_READ; if ("put" == settings.action || "append" == settings.action) remotemode = LIBSSH2_FXF_WRITE | LIBSSH2_FXF_CREAT; sftp_handle = libssh2_sftp_open(sftp_session, settings.remotefile.c_str(), remotemode, 0777); if (!sftp_handle) { fprintf(stderr, "Unable to open file with SFTP\n"); goto shutdown; } if(settings.debug > 0) fprintf(stderr, "libssh2_sftp_open() succeeded.\n"); if ("put" == settings.action || "append" == settings.action) { if("append" == settings.action) { LIBSSH2_SFTP_ATTRIBUTES attrs; if(libssh2_sftp_fstat_ex(sftp_handle, &attrs, 0) < 0) { printf("libssh2_sftp_fstat_ex failed\n"); } libssh2_sftp_seek64(sftp_handle, attrs.filesize); printf("Did a seek to position %ld\n", (long) attrs.filesize); } hand = open(settings.localfile.c_str(), O_BINARY | O_RDONLY, _S_IREAD); if (-1 == hand) { perror("opening local file"); exit(2); } char mem[32500]; long bytes_sent = 0; do { /* loop until we fail */ int nbytes = read(hand, mem, sizeof(mem)); if (nbytes > 0) { filesize += nbytes; int offset = 0, bytes_to_send = nbytes; do { rc = libssh2_sftp_write(sftp_handle, mem + offset, bytes_to_send); if (rc < 0) { printf("libssh2_sftp_write failed with code %d\n", rc); char *errmsg=NULL; int errmsg_len=0; rc = libssh2_session_last_error(session, &errmsg, &errmsg_len, 0); printf("libssh2_session_last_error() returned %s\n", errmsg); break; } else if(0==rc) { printf("libssh2_sftp_write returned 0, probably due to seek problems.\n"); rc = -1; break; } offset += rc; bytes_to_send -= rc; } while (bytes_to_send > 0); } else { break; } } while (rc >= 0); } close(hand); libssh2_sftp_close(sftp_handle); _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 26 10:28:44 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAQ9SJrd023396; Fri, 26 Nov 2010 10:28:40 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAQ9SIs2023389 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 26 Nov 2010 10:28:18 +0100 Received: (qmail 27131 invoked from network); 26 Nov 2010 09:28:16 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 26 Nov 2010 09:28:16 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Fri, 26 Nov 2010 09:28:16 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #199: Endless poll in libssh2_sftp_write X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/199 Message-ID: <043.d3f73542967b706ee368d12923800fce@libssh2.stuge.se> X-Trac-Ticket-ID: 199 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Fri, 26 Nov 2010 10:28:44 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Fri, 26 Nov 2010 10:28:18 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #199: Endless poll in libssh2_sftp_write ----------------------+----------------------------------------------------- Reporter: josusky | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.8 Component: SFTP | Version: 1.2.7 Keywords: | Blocks: Blocked By: | ----------------------+----------------------------------------------------- When connection is interrupted in the middle of libssh2_sftp_write (e.g. by ifdown) it hangs in syscall poll: {{{ #0 0x00007fb84a385203 in __poll (..., timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:87 #1 0x00007fb8481c7788 in _libssh2_wait_socket (...) at session.c:585 #2 0x00007fb8481c9c83 in libssh2_sftp_write (...) at sftp.c:1606 }}} Even if the interface goes up the application does not recover. I tested it with 1.2.7 as well as snapshot of 1.2.8. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 27 09:07:09 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAR86hTS019549; Sat, 27 Nov 2010 09:07:04 +0100 Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oAR86f6b019529 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT) for ; Sat, 27 Nov 2010 09:06:42 +0100 Received: by qyk8 with SMTP id 8so8378591qyk.20 for ; Sat, 27 Nov 2010 00:06:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=FieFfeh7Ez3XfslXKVRLeRTKATOM+C31RYHkYXaF9+I=; b=jGByB4nSfTSJZWGJjNovN674XrP2hm/C0sGwOPBqqPyGSao/tM89uHDo+KHwehaY4Y 8oYdB+riANIFEA4CfH5zlKSmF7kmwICP8tQ5TL1tZZI1Q/xldzzjsFITVyO2pQGsBIhR vZXds0iis6kBm4dNm+klgN+JQ+B17taPf71J0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=scKTK054+FhW0hZ3hIU3+QhejsPWhIqeCfp9bCDV7W4zfN7Kk/fmmkkTnNGsfRacMZ erCjw0bEzFITWt04amKbmS9bW5PVMU4OJVlj8j/an3QC1dbaOn2fL37Ml4tjxp6GW4cr Q/Grs/m3h098V520rXMtAF2cDNZ5k+hoSGBfU= MIME-Version: 1.0 Received: by 10.224.47.70 with SMTP id m6mr2744833qaf.280.1290845193382; Sat, 27 Nov 2010 00:06:33 -0800 (PST) Received: by 10.220.64.144 with HTTP; Sat, 27 Nov 2010 00:06:33 -0800 (PST) Date: Sat, 27 Nov 2010 16:06:33 +0800 Message-ID: Subject: [libssh2]libssh2_scp_recv always fails with "Timed out waiting for SCP response" From: =?UTF-8?B?5byg5Lqa6ZyP?= To: libssh2-devel@cool.haxx.se X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Sat, 27 Nov 2010 09:07:09 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Sat, 27 Nov 2010 09:06:43 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0641613366==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --===============0641613366== Content-Type: multipart/alternative; boundary=00151750e826362c74049604512e --00151750e826362c74049604512e Content-Type: text/plain; charset=UTF-8 hi guys I compile the example/scp.c and scp_nonblock.c. The result is that libssh2_scp_recv always fails with "Timed out waiting for SCP response". And command "scp" with the same username/password/filename is ok. Thanks for your help!!! --00151750e826362c74049604512e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable hi guys

I compile the example/scp.c and=C2=A0scp_nonblock.c.
The result is that libssh2_scp_recv = always fails with "Timed out waiting for SCP response".
And command "scp" wit= h the same username/password/filename is ok.
<= span class=3D"Apple-style-span" style=3D"-webkit-border-horizontal-spacing:= 2px; -webkit-border-vertical-spacing: 2px; font-size: medium;">
=
Thanks for your help!!!<= /font>

--00151750e826362c74049604512e-- --===============0641613366== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============0641613366==-- From libssh2-devel-bounces@cool.haxx.se Mon Nov 29 14:09:27 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATD95K7002350; Mon, 29 Nov 2010 14:09:23 +0100 Received: from giant.haxx.se (giant.haxx.se [80.67.6.50]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATD94hN002344 for ; Mon, 29 Nov 2010 14:09:04 +0100 Date: Mon, 29 Nov 2010 14:09:04 +0100 (CET) From: Daniel Stenberg X-X-Sender: dast@giant.haxx.se To: libssh2 development Subject: Re: Pipelining and recent sftp upload improvements In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-fromdanielhimself: yes MIME-Version: 1.0 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 29 Nov 2010 14:09:27 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 29 Nov 2010 14:09:04 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se On Wed, 17 Nov 2010, Will Cosgrove wrote: > The reason I'm posting is I recently added my own upload/download pipelining > API to libssh2 modeled after libssh's download pipeline API. It pushes a > bit more state management onto the user, but it seems to work fairly well > and doesn't require a large input buffer to be filled before hand. I was > wondering if there was any interest in my additions (which admittedly need > to be code-reviewed by someone more familiar to libssh2 than me) or if this > current method was going to be applied to downloading at some point and I > should just keep my changes to my projects. I might be interested. I have no idea how that pipeline API works so you'd have to explain that, and then I would also like to see some numbers or metrics that show it making a difference! -- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Mon Nov 29 20:03:21 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATJ2vvD007243; Mon, 29 Nov 2010 20:03:17 +0100 Received: from panic.com (posh.panic.com [38.103.165.4]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATJ2sKq007078 for ; Mon, 29 Nov 2010 20:02:54 +0100 Received: from [10.0.0.81] (unknown [38.103.165.36]) by panic.com (Postfix) with ESMTP id 8545830C602F for ; Mon, 29 Nov 2010 11:02:45 -0800 (PST) From: Will Cosgrove Mime-Version: 1.0 (Apple Message framework v1082) Subject: Re: Pipelining and recent sftp upload improvements Date: Mon, 29 Nov 2010 11:02:45 -0800 In-Reply-To: To: libssh2 development References: Message-Id: <6BB02424-4FE7-4A99-B409-087FA01996CE@panic.com> X-Mailer: Apple Mail (2.1082) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Mon, 29 Nov 2010 20:03:21 +0100 (CET) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Mon, 29 Nov 2010 20:02:55 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Precedence: list Reply-To: libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0730333595==" Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se --===============0730333595== Content-Type: multipart/alternative; boundary=Apple-Mail-1--782447109 --Apple-Mail-1--782447109 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Daniel (alt all), The pipelining API I created has two calls, one for the initial = sending/requesting data and one for the ack/response of data. It's = modeled loosely on the API in libssh and code found in openssh. The = API looks like this: LIBSSH2_API ssize_t=20 libssh2_sftp_write_async_begin(LIBSSH2_SFTP_HANDLE *handle, = libssh2_uint64_t offset, char = *buffer, size_t buffer_len, unsigned long *request_id); =09 LIBSSH2_API int=20 libssh2_sftp_write_async(LIBSSH2_SFTP_HANDLE *handle, unsigned long = request_id); LIBSSH2_API int=20 libssh2_sftp_read_async_begin(LIBSSH2_SFTP_HANDLE *handle, = libssh2_uint64_t offset, size_t buffer_maxlen, unsigned long = *out_request_id); LIBSSH2_API int=20 libssh2_sftp_read_async(LIBSSH2_SFTP_HANDLE *handle, char *buffer, = size_t buf_len, unsigned long request_id); For uploading, the write_async_begin call writes the buffer of a given = size. This is not guaranteed, so the ack call write_async returns the = actual size of the buffer handled. It is then up to the caller to = correctly re-send the remaining buffer at the correct offset. Writes = are paired using the request ID. For downloading, it's the same concept. read_async_begin requests bytes = of a given size at a given offset, read_async then actually returns that = data; this may not be the entire requested bytes, so the caller needs to = correctly re-request the remaining buffer at the correct offset. Reads = are paired using the request ID. The advantages of this is that you can call write_async_begin say, 10 = times in a row, then starting draining the acks using write_async, this = minimizes network latency (aka sitting on a select() call) and does a = better job maxing out available bandwidth. The same can be said about = downloading. The advantages of this method to the new write pipelining = on the 1.2.8 branch is that you don't have to pre-read a large buffer of = data into memory. The disadvantage is it's more leg-work for the = implementor because you have to track the offsets and drain replies = manually. Thinking out loud, it might be worth adding a convenance API = that takes a file path and does all this 'behind-the-scences' like the = openssh do_upload/do_download methods do. Now for the ever-so-important speed improvements. I'm testing against = an internal server (RAID'ed) on a gigabit ethernet. My before = benchmarks were about 12 MB/sec upload using 1.2.7 stable release. My = after is about 53 MB/sec upload. When shelling out to openssh's sftp, = I can get about 73 MB/sec. Ideally I'd like to get libssh up to ssh's = performance levels, but that's for another day. Cheers, Will On Nov 29, 2010, at 5:09 AM, Daniel Stenberg wrote: > On Wed, 17 Nov 2010, Will Cosgrove wrote: >=20 >> The reason I'm posting is I recently added my own upload/download = pipelining API to libssh2 modeled after libssh's download pipeline API. = It pushes a bit more state management onto the user, but it seems to = work fairly well and doesn't require a large input buffer to be filled = before hand. I was wondering if there was any interest in my additions = (which admittedly need to be code-reviewed by someone more familiar to = libssh2 than me) or if this current method was going to be applied to = downloading at some point and I should just keep my changes to my = projects. >=20 > I might be interested. I have no idea how that pipeline API works so = you'd have to explain that, and then I would also like to see some = numbers or metrics that show it making a difference! >=20 > --=20 >=20 > / daniel.haxx.se > _______________________________________________ > libssh2-devel = http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --Apple-Mail-1--782447109 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii ssize_t 
libssh2_sftp_write_async_begin(LIBSSH2_SFTP_HANDLE *handle, libssh2_uint64_t offset,
=   char = *buffer, size_t buffer_len, unsigned long *request_id);


int 
libssh2_sftp_write_async(LIBSSH2_SFTP_HANDLE *handle, unsigned long = request_id);

int 
libssh2_sftp_read_async_begin(LIBSSH2_SFTP_HANDLE *handle, libssh2_uint64_t offset, size_t buffer_maxlen, unsigned long = *out_request_id);

int 
libssh2_sftp_read_async(LIBSSH2_SFTP_HANDLE *handle, char *buffer, size_t buf_len, unsigned long = request_id);

For uploading, the = write_async_begin call writes the buffer of a given size.  This is = not guaranteed, so the ack call write_async returns the actual size of = the buffer handled.  It is then up to the caller to correctly = re-send the remaining buffer at the correct offset.  Writes are = paired using the request ID.

For downloading, = it's the same concept.  read_async_begin requests bytes of a given = size at a given offset, read_async then actually returns that data; this = may not be the entire requested bytes, so the caller needs to correctly = re-request the remaining buffer at the correct offset.  Reads are = paired using the request ID.

The advantages of = this is that you can call write_async_begin say, 10 times in a row, then = starting draining the acks using write_async, this minimizes network = latency (aka sitting on a select() call) and does a better job maxing = out available bandwidth.  The same can be said about downloading. =  The advantages of this method to the new write pipelining on the = 1.2.8 branch is that you don't have to pre-read a large buffer of data = into memory.  The disadvantage is it's more leg-work for the = implementor because you have to track the offsets and drain replies = manually.  Thinking out loud, it might be worth adding a convenance = API that takes a file path and does all this 'behind-the-scences' like = the openssh do_upload/do_download methods = do.

Now for the ever-so-important speed = improvements.  I'm testing against an internal server (RAID'ed) on = a gigabit ethernet.  My before benchmarks were about 12 MB/sec = upload using 1.2.7 stable release.  My after is about 53 MB/sec = upload.   When shelling out to openssh's sftp, I can get about 73 = MB/sec.  Ideally I'd like to get libssh up to ssh's performance = levels, but that's for another = day.

Cheers,
Will

On= Nov 29, 2010, at 5:09 AM, Daniel Stenberg wrote:

On = Wed, 17 Nov 2010, Will Cosgrove wrote:

The reason I'm posting is I recently added my own = upload/download pipelining API to libssh2 modeled after libssh's = download pipeline API.  It pushes a bit more state management onto = the user, but it seems to work fairly well and doesn't require a large = input buffer to be filled before hand.  I was wondering if there = was any interest in my additions (which admittedly need to be = code-reviewed by someone more familiar to libssh2 than me) or if this = current method was going to be applied to downloading at some point and = I should just keep my changes to my projects.

I = might be interested. I have no idea how that pipeline API works so you'd = have to explain that, and then I would also like to see some numbers or = metrics that show it making a difference!

--

/ daniel.haxx.se
_____________________= __________________________
libssh2-devel http:/= /cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel

= --Apple-Mail-1--782447109-- --===============0730333595== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel --===============0730333595==-- From libssh2-devel-bounces@cool.haxx.se Tue Nov 30 00:38:43 2010 Return-Path: Received: from giant.haxx.se (localhost.localdomain [127.0.0.1]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATNcI61029397; Tue, 30 Nov 2010 00:38:39 +0100 Received: from earth.stuge.se (earth.stuge.se [213.88.146.15]) by giant.haxx.se (8.14.3/8.14.3/Debian-9.1) with ESMTP id oATNcH5u029385 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 30 Nov 2010 00:38:17 +0100 Received: (qmail 22083 invoked from network); 29 Nov 2010 23:38:11 -0000 Received: from unknown (HELO earth.stuge.se) (127.0.0.1) by localhost with SMTP; 29 Nov 2010 23:38:11 -0000 MIME-Version: 1.0 From: "libssh2 Trac" X-Trac-Version: 0.12 Precedence: bulk Cc: libssh2-devel@cool.haxx.se Auto-Submitted: auto-generated X-Mailer: Trac 0.12, by Edgewall Software X-Trac-Project: libssh2 Date: Mon, 29 Nov 2010 23:38:11 -0000 X-URL: http://trac.libssh2.org/ Subject: [libssh2] #200: Compression not enabled correctly X-Trac-Ticket-URL: http://trac.libssh2.org/ticket/200 Message-ID: <045.9239c395ed7185ad408d27b835794318@libssh2.stuge.se> X-Trac-Ticket-ID: 200 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.3.5 (giant.haxx.se [127.0.0.1]); Tue, 30 Nov 2010 00:38:43 +0100 (CET) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.3.5 (giant.haxx.se [80.67.6.50]); Tue, 30 Nov 2010 00:38:17 +0100 (CET) X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.13 Reply-To: trac@libssh2.stuge.se, libssh2 development List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: libssh2-devel-bounces@cool.haxx.se Errors-To: libssh2-devel-bounces@cool.haxx.se #200: Compression not enabled correctly ------------------------+--------------------------------------------------- Reporter: riordanmr | Owner: Type: defect | Status: new Priority: normal | Milestone: 1.2.8 Component: protocol | Version: 1.2.7 Keywords: | Blocks: Blocked By: | ------------------------+--------------------------------------------------- In the current libssh2 in git (2010-11-29), compression does not work. I get a "decompression unitilized" error in comp_method_zlib_decomp. The "abstract" parameter is NULL. The problem seems to be that compression has not been initialized - similar to ticket 180. The error occurs during a call to _libssh2_packet_require in diffie_hellman_sha1 in kex.c. The initialization of compression doesn't occur until later in that same function. Here's a traceback: mwsftp.exe!comp_method_zlib_decomp(_LIBSSH2_SESSION * session=0x00ff5f98, unsigned char * * dest=0x019ed620, unsigned int * dest_len=0x019ed614, unsigned int payload_limit=40000, const unsigned char * src=0x010057c0, unsigned int src_len=818, void * * abstract=0x00ff603c) Line 234 C mwsftp.exe!fullpacket(_LIBSSH2_SESSION * session=0x00ff5f98, int encrypted=0) Line 209 + 0x3e bytes C mwsftp.exe!_libssh2_transport_read(_LIBSSH2_SESSION * session=0x00ff5f98) Line 545 + 0x10 bytes C mwsftp.exe!_libssh2_packet_require(_LIBSSH2_SESSION * session=0x00ff5f98, unsigned char packet_type='', unsigned char * * data=0x01002b38, unsigned int * data_len=0x01002b5c, int match_ofs=0, const unsigned char * match_buf=0x00000000, unsigned int match_len=0, packet_require_state_t * state=0x01002ba8) Line 1093 + 0x9 bytes C mwsftp.exe!diffie_hellman_sha1(_LIBSSH2_SESSION * session=0x00ff5f98, bignum_st * g=0x00feb5f8, bignum_st * p=0x00feb5a8, int group_order=256, unsigned char packet_type_init='', unsigned char packet_type_reply='', unsigned char * midhash=0x00000000, unsigned long midhash_len=0, kmdhgGPsha1kex_state_t * exchange_state=0x01002b30) Line 190 + 0x29 bytes C mwsftp.exe!kex_method_diffie_hellman_group14_sha1_key_exchange(_LIBSSH2_SESSION * session=0x00ff5f98, key_exchange_state_low_t * key_state=0x01002b18) Line 796 + 0x31 bytes C mwsftp.exe!_libssh2_kex_exchange(_LIBSSH2_SESSION * session=0x00ff5f98, int reexchange=0, key_exchange_state_t * key_state=0x01002b00) Line 1757 + 0x18 bytes C mwsftp.exe!session_startup(_LIBSSH2_SESSION * session=0x00ff5f98, unsigned int sock=3460) Line 672 + 0x14 bytes C mwsftp.exe!libssh2_session_handshake(_LIBSSH2_SESSION * session=0x00ff5f98, unsigned int sock=3460) Line 750 + 0xd bytes C mwsftp.exe!libssh2_session_startup(_LIBSSH2_SESSION * session=0x00ff5f98, int sock=3460) Line 769 + 0xd bytes C (Note: in order to enable standard zlib compression under Ubuntu, I had to add "Compression yes" to my sshd_config.) I'm attaching full tracing output. I tested against OpenSSH (on Ubuntu 10.04), Solaris, and WS_FTP SSH server. They all were unable to exchange keys with compression enabled, but they all worked with compression disabled. -- Ticket URL: libssh2 C library for writing portable SSH2 clients _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel