From libssh2-devel-bounces@cool.haxx.se Wed Nov 1 10:05:33 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vA194uci031902; Wed, 1 Nov 2017 10:05:24 +0100 Received: from m13-21.163.com (m13-21.163.com [220.181.13.21]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vA194rdK031885 for ; Wed, 1 Nov 2017 10:04:54 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=Date:From:Subject:MIME-Version:Message-ID; bh=cZT5T VPiZZKZ+FfwyJOtUGrqbQbjfuwqKh7x4L8XH2w=; b=Q4d84+1GyKBD05vMS4Pex BKbIMxWu7KiT8K/CHztPfnqJb9wUGle2ZrpuF+sZ8JYQbTi4Te6JHaphJElJCFa2 ZylKeLNSXf1tilHDVt0uTea5FS3jpICh3Z7FxCIBfdroQqJ30yiJy3XGZAegs4tk +ynpIbrC7jqJBCtCVXTgxs= Received: from kipade$163.com ( [180.168.80.138] ) by ajax-webmail-wmsvr21 (Coremail) ; Wed, 1 Nov 2017 17:04:35 +0800 (CST) X-Originating-IP: [180.168.80.138] Date: Wed, 1 Nov 2017 17:04:35 +0800 (CST) From: kipade To: libssh2-devel@cool.haxx.se Subject: How to execute a long-time command? X-Priority: 3 X-Mailer: Coremail Webmail Server Version SP_ntes V3.5 build 20160729(86883.8884) Copyright (c) 2002-2017 www.mailtech.cn 163com X-CM-CTRLDATA: li6YgWZvb3Rlcl9odG09MzQyOjU2 MIME-Version: 1.0 Message-ID: <43d2d814.b5ad.15f76d338d0.Coremail.kipade@163.com> X-Coremail-Locale: zh_CN X-CM-TRANSID: FcGowADX3ycjjvlZ0ezLAA--.28020W X-CM-SenderInfo: pnlstvrh6rljoofrz/1tbiTh+Dz1UC0eGfaAADsO X-Coremail-Antispam: 1U5529EdanIXcx71UUUUU7vcSsGvfC2KfnxnUU== X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Content-Type: multipart/mixed; boundary="===============0109749845==" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" --===============0109749845== Content-Type: multipart/alternative; boundary="----=_Part_171403_742756894.1509527075024" ------=_Part_171403_742756894.1509527075024 Content-Type: text/plain; charset=GBK Content-Transfer-Encoding: base64 SGVsbG8sCgpJIHdhbnQgdG8gZXhlY3V0ZSBhIGxhcmdlIHdvcmsgY29tbWFuZCBvbiB0aGUgcmVt b3RlIHZlcnNpb24gdXNpbmcKbGlic3NoMl9jaGFubmVsX2V4ZWMgaW50ZXJmYWNlLiBJIGNhbiBv bmx5IHJlYWQgdGhlIGxpbmVzIGJlZm9yZSB0aGUKcmVtb3RlIGpvYiBibG9ja2luZyAoaXQncyBh IGxvbmcgdGltZSB0byB3YWl0IGl0IGNvbXBsZXRlKS4KYW55IGhpbnRzIHdpbGwgYmUgdmVyeSBh cHByZWNpYXRlZC4= ------=_Part_171403_742756894.1509527075024 Content-Type: text/html; charset=GBK Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0ibGluZS1oZWlnaHQ6MS43O2NvbG9yOiMwMDAwMDA7Zm9udC1zaXplOjE0cHg7 Zm9udC1mYW1pbHk6QXJpYWwiPkhlbGxvLDxicj4KPGJyPgpJIHdhbnQgdG8gZXhlY3V0ZSBhIGxh cmdlIHdvcmsgY29tbWFuZCBvbiB0aGUgcmVtb3RlIHZlcnNpb24gdXNpbmcgPGJyPgpsaWJzc2gy X2NoYW5uZWxfZXhlYyBpbnRlcmZhY2UuIEkgY2FuIG9ubHkgcmVhZCB0aGUgbGluZXMgYmVmb3Jl IHRoZTxicj4KcmVtb3RlIGpvYiBibG9ja2luZyAoaXQncyBhIGxvbmcgdGltZSB0byB3YWl0IGl0 IGNvbXBsZXRlKS48YnI+CmFueSBoaW50cyB3aWxsIGJlIHZlcnkgYXBwcmVjaWF0ZWQuPC9kaXY+ PGJyPjxicj48c3BhbiB0aXRsZT0ibmV0ZWFzZWZvb3RlciI+PHA+Jm5ic3A7PC9wPjwvc3Bhbj4= ------=_Part_171403_742756894.1509527075024-- --===============0109749845== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwczovL2Nvb2wuaGF4eC5zZS9jZ2ktYmluL21haWxtYW4vbGlzdGluZm8vbGlic3No Mi1kZXZlbAo= --===============0109749845==-- From libssh2-devel-bounces@cool.haxx.se Mon Nov 13 16:15:29 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vADFEm1K014102; Mon, 13 Nov 2017 16:15:19 +0100 Received: from f1wall.ipetronik.com (f1wall.ipetronik.com [109.109.200.66]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vADFEmf0014071 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 13 Nov 2017 16:14:48 +0100 Received: from mx1.ipetronik.de (unknown [10.1.1.107]) by f1wall.ipetronik.com (Postfix) with ESMTPS for ; Mon, 13 Nov 2017 16:14:43 +0100 (CET) Received: from IPE_DOM-MTA by mx1.ipetronik.de with Novell_GroupWise; Mon, 13 Nov 2017 16:14:43 +0100 Message-Id: <5A09B6E10200000A0004E71C@mx1.ipetronik.de> X-Mailer: Novell GroupWise Internet Agent 14.2.2 Date: Mon, 13 Nov 2017 16:14:41 +0100 From: "Jerome Zimmermann" To: Subject: Re: libssh2_sftp_write blocks for about 3 minutes References: <59F330FC0200000A0004E327@mx1.ipetronik.de> <59F330FC0200000A0004E327@mx1.ipetronik.de> <25E09154-E711-44BC-A8EF-D65679049CD5@panic.com> In-Reply-To: <25E09154-E711-44BC-A8EF-D65679049CD5@panic.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=__Part7C4722F1.0__=" X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This is a MIME message. If you are reading this text, you may want to consider changing to a mail reader or gateway that understands how to properly handle MIME multipart messages. --=__Part7C4722F1.0__= Content-Type: multipart/alternative; boundary="=__Part7C4722F1.1__=" --=__Part7C4722F1.1__= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hello, @Will: Yes, the suggestion to change the timeout worked fine. But the problem with the high CPU load remains. I got new insights about the behaviour that the CPU load remains at 100% = in blocking mode. For the sake of clarity, I will first re-explain the case of the problem. The application (runs on a RTOS platform) establish a connection to a = SFTP-Server via a modem device (this implies a high latency connection) to send files = to the server. During the transfer I remove the connection cable to the modem antenna.=20 Then, the CPU loads remains on 100 % for over 3 minutes. The libssh2 remains in the libssh2_sftp_write function. More precise, the second do while loop of the BLOCK_ADJUST macro. Here, function libssh2_wait_socket always returns 0. Examining the return value of select() reveals that the socket is always ready for reading or writing even when the physical connection is no = longer available. Further, I verified the socket state with getsockopt() and connect(). But the socket is always in the "connect"-state.=20 Besides, the keep-alive mechanism of the libssh2 is deactivated. The socket is in the non-blocking mode and the libssh2 session in the = blocking mode. Is there a way to identify such a "broken connection" to break earlier the = loop and so avoid a CPU load of 100% ? Best regards Jerome Impressum/Imprint: https://www.ipetronik.com/impressum --=__Part7C4722F1.1__= Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Description: HTML =0A=0A=0A=0A
Hello,

@Will:
Yes, the suggestion to change the = timeout worked fine.
But the problem with the high CPU load remains.

I got new insights about the behaviour that the CPU = load remains at 100% in blocking mode.

For the = sake of clarity, I will first re-explain the case of the problem.

The application (runs on a RTOS platform) establish a = connection to a SFTP-Server
via a modem device (this implies a high = latency connection) to send files to the server.
During the transfer I = remove the connection cable to the modem antenna.
Then, the CPU loads = remains on 100 % for over 3 minutes.

The libssh2 = remains in the libssh2_sftp_write function.
More precise, the second do = while loop of the BLOCK_ADJUST macro.
Here, function libssh2_wait_socket= always returns 0.

Examining the return value of select() = reveals that the socket is always
ready for reading or writing even = when the physical connection is no longer available.
Further, I = verified the socket state with getsockopt() and connect().
But the = socket is always in the "connect"-state.

Besides,= the keep-alive mechanism of the libssh2 is deactivated.
The socket is = in the non-blocking mode and the libssh2 session in the blocking mode.

Is there a way to identify such a "broken connection" = to break earlier the loop and so avoid a CPU load of 100% ?

<= /div>
Best regards
Jerome


=0A Impressum/Imprint: https://www.ipetronik.com/impressum =0A =

=0A =0A
--=__Part7C4722F1.1__=-- --=__Part7C4722F1.0__= Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwczovL2Nvb2wuaGF4eC5zZS9jZ2ktYmluL21haWxtYW4vbGlzdGluZm8vbGlic3No Mi1kZXZlbAo= --=__Part7C4722F1.0__=-- From libssh2-devel-bounces@cool.haxx.se Mon Nov 20 09:36:32 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vAK8ZruT028707; Mon, 20 Nov 2017 09:36:22 +0100 Received: from f1wall.ipetronik.com (f1wall.ipetronik.com [109.109.200.66]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vAK8Zp4V028685 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 20 Nov 2017 09:35:52 +0100 Received: from mx1.ipetronik.de (unknown [10.1.1.107]) by f1wall.ipetronik.com (Postfix) with ESMTPS for ; Mon, 20 Nov 2017 09:35:46 +0100 (CET) Received: from IPE_DOM-MTA by mx1.ipetronik.de with Novell_GroupWise; Mon, 20 Nov 2017 09:35:46 +0100 Message-Id: <5A1293E10200000A0004E8FE@mx1.ipetronik.de> X-Mailer: Novell GroupWise Internet Agent 14.2.2 Date: Mon, 20 Nov 2017 09:35:45 +0100 From: "Jerome Zimmermann" To: "libssh2-devel@cool.haxx.se" Subject: Antw: Re: libssh2_sftp_write blocks for about 3 minutes References: <59F330FC0200000A0004E327@mx1.ipetronik.de> <59F330FC0200000A0004E327@mx1.ipetronik.de> <25E09154-E711-44BC-A8EF-D65679049CD5@panic.com> <5A09B6E10200000A0004E71C@mx1.ipetronik.de> In-Reply-To: <5A09B6E10200000A0004E71C@mx1.ipetronik.de> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=__Part427907F1.0__=" X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" This is a MIME message. If you are reading this text, you may want to consider changing to a mail reader or gateway that understands how to properly handle MIME multipart messages. --=__Part427907F1.0__= Content-Type: multipart/alternative; boundary="=__Part427907F1.1__=" --=__Part427907F1.1__= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Does nobody have an idea? =20 Impressum/Imprint: https://www.ipetronik.com/impressum --=__Part427907F1.1__= Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Description: HTML =0A=0A=0A=0A
Does = nobody have an idea?

 =0A =0A
=0A=0A

=0A Impressum/Imprint: https://www.ipetronik.com/impressum =0A =

=0A =0A
--=__Part427907F1.1__=-- --=__Part427907F1.0__= Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlic3NoMi1k ZXZlbCBodHRwczovL2Nvb2wuaGF4eC5zZS9jZ2ktYmluL21haWxtYW4vbGlzdGluZm8vbGlic3No Mi1kZXZlbAo= --=__Part427907F1.0__=-- From libssh2-devel-bounces@cool.haxx.se Tue Nov 21 14:26:59 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vALDQBIi030534; Tue, 21 Nov 2017 14:26:50 +0100 Received: from foo.stuge.se (foo.stuge.se [212.116.89.98]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vALDQAmE030257 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 21 Nov 2017 14:26:10 +0100 Received: (qmail 28094 invoked by uid 1000); 21 Nov 2017 13:17:49 -0000 Date: Tue, 21 Nov 2017 13:17:49 +0000 From: Peter Stuge To: libssh2 development Subject: Re: libssh2_sftp_write blocks for about 3 minutes Message-ID: <20171121131749.GX4167@stuge.se> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5A1293E10200000A0004E8FE@mx1.ipetronik.de> <5A09B6E10200000A0004E71C@mx1.ipetronik.de> X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id vALDQBIi030534 Jerome Zimmermann wrote: > the socket is always ready for reading or writing even when the > physical connection is no longer available. That's a good find, it's the key point. > Is there a way to identify such a "broken connection" to break > earlier the loop and so avoid a CPU load of 100% ? There probably is, but not in the application. This is a TCP stack tunable, so you have to study what your operating system allows you to configure. //Peter _______________________________________________ libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 24 14:11:45 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vAODBBch013904; Fri, 24 Nov 2017 14:11:36 +0100 Received: from f1wall.ipetronik.com (f1wall.ipetronik.com [109.109.200.66]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vAODB9Df013867 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 24 Nov 2017 14:11:09 +0100 Received: from mx1.ipetronik.de (unknown [10.1.1.107]) by f1wall.ipetronik.com (Postfix) with ESMTPS for ; Fri, 24 Nov 2017 14:11:04 +0100 (CET) Received: from IPE_DOM-MTA by mx1.ipetronik.de with Novell_GroupWise; Fri, 24 Nov 2017 14:11:04 +0100 Message-Id: <5A181A670200007E00022E2D@mx1.ipetronik.de> X-Mailer: Novell GroupWise Internet Agent 14.2.2 Date: Fri, 24 Nov 2017 14:11:03 +0100 From: "Jerome Zimmermann" To: Subject: Re: libssh2_sftp_write blocks for about 3 minutes References: <5A1293E10200000A0004E8FE@mx1.ipetronik.de> <5A09B6E10200000A0004E71C@mx1.ipetronik.de> <20171121131749.GX4167@stuge.se> In-Reply-To: <20171121131749.GX4167@stuge.se> Mime-Version: 1.0 Content-Disposition: inline X-MIME-Autoconverted: from quoted-printable to 8bit by giant.haxx.se id vAODB9Df013867 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id vAODBBch013904 Hello again, > Examining the return value of select() reveals that the socket is always > ready for reading or writing even when the physical connection is no longer available. > Further, I verified the socket state with getsockopt() and connect(). > But the socket is always in the "connect"-state. In the case when the physical cable is removed the application (client) does no longer get an acknowledged for the transmitted data packets (from the server). Subsequently, TCP Retransmission packets are sent. The used operating systems tries this for three minutes. When this time is elapsed the TCP socket connection is closed. So, the TCP socket is during this three minutes in the connection state, although there is no physical connection. I am not an TCP/IP expert, but is there in general a way to identify such a situation ? > The libssh2 remains in the libssh2_sftp_write function. > More precise, the second do while loop of the BLOCK_ADJUST macro. > Here, function libssh2_wait_socket always returns 0. I have the possibility to reduce this "blocking" time with function libssh2_session_set_timeout as with the retransmission timeout from the operating system. > Is there a way to identify such a "broken connection" to break > earlier the loop and so avoid a CPU load of 100% ? However, the problem with the high CPU load of 100% still remains. Is it possible to handle this issue in the libssh2 ? Best regards Jerome Impressum/Imprint: https://www.ipetronik.com/impressum >>> Peter Stuge 21.11.2017 14:17 >>> Jerome Zimmermann wrote: > the socket is always ready for reading or writing even when the > physical connection is no longer available. That's a good find, it's the key point. > Is there a way to identify such a "broken connection" to break > earlier the loop and so avoid a CPU load of 100% ? There probably is, but not in the application. This is a TCP stack tunable, so you have to study what your operating system allows you to configure. //Peter _______________________________________________ libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel _______________________________________________ libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Fri Nov 24 18:19:03 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vAOHIVto021798; Fri, 24 Nov 2017 18:18:54 +0100 Received: from mail.panic.com (mail.panic.com [38.103.165.3]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vAOHIR0X021658 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 24 Nov 2017 18:18:29 +0100 Received: from [10.0.1.7] (c-24-20-9-219.hsd1.or.comcast.net [24.20.9.219]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.panic.com (Postfix) with ESMTPSA id 8661360151 for ; Fri, 24 Nov 2017 09:18:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=panic.com; s=dkim; t=1511543901; bh=mhc3spDay7KKzR1LQzM1yEoFWDDZ8E7xcnrQlEfOtoc=; h=From:Date:Subject:References:In-Reply-To:To; b=vSpiaBqehHrdHr3jXekFV6fXCebfbgPVXv8U3RpXAQwj4+SGvLYFGLWuCE2GQDePO ESjmuOpFFbgBwUhY90J7JHy1fxFzGU4197CxahtiuqzrpSqoJeAMXR/ACJEE3qPimS p9QtMQ1heChvNVSItGW+XvpaqUZOPjNdgRkSid6Y= Mime-Version: 1.0 (1.0) Date: Fri, 24 Nov 2017 09:18:20 -0800 Subject: Re: libssh2_sftp_write blocks for about 3 minutes Message-Id: <3182C72D-7A7C-46EF-A7B6-429723B84671@panic.com> References: <5A1293E10200000A0004E8FE@mx1.ipetronik.de> <5A09B6E10200000A0004E71C@mx1.ipetronik.de> <20171121131749.GX4167@stuge.se> <5A181A670200007E00022E2D@mx1.ipetronik.de> In-Reply-To: <5A181A670200007E00022E2D@mx1.ipetronik.de> To: libssh2 development X-Mailer: iPhone Mail (15B202) X-MIME-Autoconverted: from quoted-printable to 8bit by giant.haxx.se id vAOHIR0X021658 X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Will Cosgrove via libssh2-devel Reply-To: libssh2 development Cc: Will Cosgrove Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id vAOHIVto021798 You should be getting a sigpipe signal (if you register for it) once the OS determines the socket has been broken. However, that will likely come after the timeout. There is no way I know of at the socket level to determine if the user physically unplugs a connection before a timeout. Some client OS' will notify you if this happens, but that would be above where libssh2 works. If I unplug my cable on macOS select() will wait using 0% CPU. So this behavior you're seeing likely requires tuning on your specific OS as Peter mentioned. Will > On Nov 24, 2017, at 5:11 AM, Jerome Zimmermann wrote: > > Hello again, > >> Examining the return value of select() reveals that the socket is always >> ready for reading or writing even when the physical connection is no longer available. >> Further, I verified the socket state with getsockopt() and connect(). >> But the socket is always in the "connect"-state. > > In the case when the physical cable is removed the application (client) does no > longer get an acknowledged for the transmitted data packets (from the server). > > Subsequently, TCP Retransmission packets are sent. > The used operating systems tries this for three minutes. > When this time is elapsed the TCP socket connection is closed. > So, the TCP socket is during this three minutes in the connection state, > although there is no physical connection. > > I am not an TCP/IP expert, but is there in general a way to identify such a situation ? > >> The libssh2 remains in the libssh2_sftp_write function. >> More precise, the second do while loop of the BLOCK_ADJUST macro. >> Here, function libssh2_wait_socket always returns 0. > > I have the possibility to reduce this "blocking" time with > function libssh2_session_set_timeout as with the retransmission timeout > from the operating system. > >> Is there a way to identify such a "broken connection" to break >> earlier the loop and so avoid a CPU load of 100% ? > > However, the problem with the high CPU load of 100% still remains. > > Is it possible to handle this issue in the libssh2 ? > > Best regards > Jerome > > > Impressum/Imprint: https://www.ipetronik.com/impressum > > > >>>> Peter Stuge 21.11.2017 14:17 >>> > Jerome Zimmermann wrote: >> the socket is always ready for reading or writing even when the >> physical connection is no longer available. > > That's a good find, it's the key point. > > >> Is there a way to identify such a "broken connection" to break >> earlier the loop and so avoid a CPU load of 100% ? > > There probably is, but not in the application. This is a TCP stack > tunable, so you have to study what your operating system allows you > to configure. > > > //Peter > _______________________________________________ > libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel > > > _______________________________________________ > libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel _______________________________________________ libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel From libssh2-devel-bounces@cool.haxx.se Sat Nov 25 02:27:14 2017 Return-Path: Received: from www.haxx.se (mail [127.0.0.1]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTP id vAP1Qc7F011433; Sat, 25 Nov 2017 02:27:07 +0100 Received: from foo.stuge.se (foo.stuge.se [212.116.89.98]) by giant.haxx.se (8.15.2/8.15.2/Debian-4) with ESMTPS id vAP1QZJZ011378 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 25 Nov 2017 02:26:36 +0100 Received: (qmail 15466 invoked by uid 1000); 25 Nov 2017 01:18:00 -0000 Date: Sat, 25 Nov 2017 01:18:00 +0000 From: Peter Stuge To: libssh2-devel@cool.haxx.se Subject: Re: libssh2_sftp_write blocks for about 3 minutes Message-ID: <20171125011800.GO4167@stuge.se> References: <5A1293E10200000A0004E8FE@mx1.ipetronik.de> <5A09B6E10200000A0004E71C@mx1.ipetronik.de> <20171121131749.GX4167@stuge.se> <5A181A670200007E00022E2D@mx1.ipetronik.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5A181A670200007E00022E2D@mx1.ipetronik.de> X-BeenThere: libssh2-devel@cool.haxx.se X-Mailman-Version: 2.1.22 Precedence: list List-Id: libssh2 development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: libssh2 development Content-Type: text/plain; charset="utf-8" Errors-To: libssh2-devel-bounces@cool.haxx.se Sender: "libssh2-devel" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by giant.haxx.se id vAP1Qc7F011433 Jerome Zimmermann wrote: > Subsequently, TCP Retransmission packets are sent. > The used operating systems tries this for three minutes. > When this time is elapsed the TCP socket connection is closed. > So, the TCP socket is during this three minutes in the connection state, > although there is no physical connection. > > I am not an TCP/IP expert, but is there in general a way to > identify such a situation ? "in general" doesn't fit so well in that sentence. Most OSes (even Windows! :) allows the TCP stack to be tuned, but there is no portable API to do so. You are on your own. It is worth considering here, that libssh2 does not create the socket, but you do, before you pass the socket to libssh2. This means that you have a good opportunity to tune the socket as desired, before you pass it to libssh2. But also keep in mind, that if you tune the TCP stack to be too aggressive, your connection will also be torn down by random intermittent network issues possibly far outside your control. //Peter _______________________________________________ libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel