On Thu, 2010-08-26 at 23:45 +0200, Daniel Stenberg wrote:
> On Thu, 26 Aug 2010, jmk wrote:
>
> > I maintain a password auditing utility[1] and have been using libssh2 with
> > it for a while. Unfortunately, I seem to encounter thread safety issues from
> > time to time within the library. For example, if compiled with libgcrypt,
> > multi-threaded uses of the library consistently result in a segfault.
> > Everything appeared to behave as expected using libssh2 compiled with
> > openssl support up through version 1.2.6. Starting with 1.2.7,
> > multi-threaded uses result in segfaults (double frees) such as:
>
> Both libgcrypt and OpenSSL have mutex functions that are supposed to be used
> when handled multi-threaded. Do you use them?
I'm not directly calling any libgcrypt/openssl functions within this
code. I'm simply calling libssh2_session_init/libssh2_session_startup.
I'm assuming the openssl mutex use would need to take place within
libssh2, no?
> > I'm at a loss for where to start debugging this.
>
> If you're convinced this works with 1.2.6 and not 1.2.7, it could be an idea
> to bisect your way to the exact change that introduced this problem in the
> code. "git help bisect" !
I played with bisect, but unfortunately, I didn't get anywhere. The
segmentation faults are showing up now intermittently across multiple
versions of the library.
In an effort to simplify the problem (i.e. multiple concurrent SSH
connection attempts), I decided to just take the example ssh2.c code and
hack it to be multi-threaded. My thinking was that if I could start with
known working, extend it to multi-threaded connections, I could transfer
that to my application.
Here's my simple hack to kick off a few concurrent logons:
http://www.foofus.net/~jmk/tmp/ssh2.c.txt
As noted below, this resulted in intermittent segmentation faults. About
half of the time it'd finish fine, the other half I'd see the double
free message.
Should this hack have worked or am I completely off target here?
Thanks!
Joe
% ./ssh2
thread 0 created.
thread 1 created.
thread 2 created.
thread 3 created.
*** glibc detected
*** /home/jmk/tmp/libssh2/libssh2-git/example/.libs/ssh2: double free or
corruption (!prev): 0x0000000001621310 ***
======= Backtrace: =========
/lib/libc.so.6(+0x72966)[0x7f47b47fa966]
/lib/libc.so.6(+0x78433)[0x7f47b4800433]
/lib/libc.so.6(realloc+0xf0)[0x7f47b48008d0]
/usr/lib/libcrypto.so.0.9.8(CRYPTO_realloc+0x65)[0x7f47b4f7e4a5]
/usr/lib/libcrypto.so.0.9.8(lh_insert+0x12b)[0x7f47b4fdde2b]
/usr/lib/libcrypto.so.0.9.8(OBJ_NAME_add+0x6b)[0x7f47b4f80f5b]
/usr/lib/libcrypto.so.0.9.8(EVP_add_cipher+0x1b)[0x7f47b4fe501b]
/usr/lib/libcrypto.so.0.9.8(OpenSSL_add_all_ciphers
+0x10d)[0x7f47b4fe901d]
/usr/lib/libcrypto.so.0.9.8(OPENSSL_add_all_algorithms_noconf
+0xe)[0x7f47b4fe8efe]
/home/jmk/tmp/libssh2/libssh2-git/src/.libs/libssh2.so.1(libssh2_init
+0x39)[0x7f47b5511d09]
/home/jmk/tmp/libssh2/libssh2-git/example/.libs/ssh2[0x401113]
/lib/libpthread.so.0(+0x6894)[0x7f47b4ae7894]
/lib/libc.so.6(clone+0x6d)[0x7f47b485927d]
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2010-08-28