Bugs item #2819559, was opened at 2009-07-10 14:06
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=703942&aid=2819559&group_id=125852
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SFTP
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: mera (mera-nn)
Assigned to: Daniel Stenberg (bagder)
Summary: high cpu usage during SFTP copying (nonblock mode)
Initial Comment:
Using test sftp copying binary , high cpu usage observed. Seems curl uses nonblock mode so sftp usage using curl library is high. Originally issue observed using curl library.
The logs below:
Using latest libssh2
[root_at_localhost simple]# pwd
/root/libssh2-1.1/example/simple
[root_at_localhost simple]#./sftp_write 192.168.35.75 user pass test.sftp /tmp/test.block
Fingerprint: 11 12 A1 54 1C 8D 65 40 DD 45 EC F9 12 25 A0 59
libssh2_sftp_init()!
libssh2_sftp_open()!
libssh2_sftp_open() is done, now send data!
all done
cpu usage is OK:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2611 root 15 0 4808 1544 1264 S 11.7 0.1 0:01.48 lt-sftp_write
[root_at_localhost simple]# ./sftp_write_nonblock 192.168.35.75 user pass test.sftp /tmp/test.nonblock
Fingerprint: 11 12 A1 54 1C 8D 65 40 DD 45 EC F9 12 25 A0 59
libssh2_sftp_init()!
libssh2_sftp_open()!
libssh2_sftp_open() is done, now send data!
all done
High cpu usage !!!
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2561 root 25 0 4804 1464 1244 R 99.3 0.1 0:09.71 lt-sftp_write_non_block
RedHat Enterprise Linux Server 5.1. But, I tested this scenario on 3 platforms. Same effect.
So, I think won't be a big problem to reproduce this issue on any Linux platform.
Please, confirm.
----------------------------------------------------------------------
>Comment By: Daniel Stenberg (bagder)
Date: 2009-09-08 19:35
Message:
Can you research further and figure out why this EAGAIN doesn't end up in
the loop in libssh2_sftp_read() waiting for the socket to become readable
again? The idea is that the recv() function when called from transport.c
should basically be pushed back all the way to the libssh2 entry-function
and there it should select() until the socket is again signaled to be ready
to get read from.
Also, it could be an idea to try this with the git version of libssh2 as
I've tried to clean up error code treatment and I might then have changed
behavior from what you get with 1.2.
----------------------------------------------------------------------
Comment By: Poul Serge (poulserge)
Date: 2009-09-08 12:57
Message:
"strace" output shows the following:
. . .
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4,
"\340\272\250\265\367C\7\337\340\03557\224\232\377gK\364\207\36\"\306W\v\t\37\3013\262?\5\206\256"...,
16384, MSG_NOSIGNAL) = 68
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
send(4,
"_\227\365\301^s\230Gqn\3469\374\334}\25]N\204\3!\226%D\32{\233$\367(\304\324\247"...,
68, MSG_NOSIGNAL) = 68
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4,
"\20k\243\224B\327>w\246\377\302\347\341\37\237-\264\202$\4\335\7D\272\22z\177+c\3031\0277"...,
16384, MSG_NOSIGNAL) = 68
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
read(3,
"Z\0\327\214\1\0\0\1\214\327\352b\1\0\0\1b\352l\3\f\3\22-\360\0\0\0\1\0\0\0018"...,
4096) = 4096
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
send(4,
"\217\7g@\16\226\0330\365\334,\256(J\233O\7\370l(\21_\356NI3\311\304\n@\254\36\362"...,
1092, MSG_NOSIGNAL) = 1092
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
recv(4, 0x974d890, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily
unavailable)
. . .
----------------------------------------------------------------------
Comment By: Daniel Stenberg (bagder)
Date: 2009-09-03 22:37
Message:
Have you tried to pinpoint what the problem is? Like running strace on the
process to see what it does?
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2009-09-03 10:51
Message:
I have tried latest version of libssh2. same effect . high cpu usage.
----------------------------------------------------------------------
Comment By: Daniel Stenberg (bagder)
Date: 2009-09-01 23:04
Message:
Thanks, can you try again with the most recent git code and see if you
still experience this problem? I've tried to go over the code and cleaned
up dubious code and it might just have fixed this. Not to mention that we
released 1.2 after the version this is reported against.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=703942&aid=2819559&group_id=125852
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2009-09-08