Ticket #3705 (closed defect: invalid)

Opened 2 years ago

Last modified 2 years ago

Reproducible segfault in Find File dialog

Reported by: zaytsev Owned by:
Priority: major Milestone: 4.8.19
Component: mc-search Version: 4.8.18
Keywords: Cc: egmont
Blocked By: Blocking:
Branch state: no branch Votes for changeset:

Description

I'm able to reliably crash the latest release while searching in Python sources for the following:

 ╔══════════════════════════ Find File ═══════════════════════════╗
 ║ Start at:                                                      ║
 ║ .                                                 [^] [ Tree ] ║
 ║ [ ] Enable ignore directories:                                 ║
 ║ .git:.hg:.svn:build                                        [^] ║
 ╟────────────────────────────────────────────────────────────────╢
 ║ File name:                      Content:                       ║
 ║ *                          [^]  python-config              [^] ║
 ║ [x] Find recursively            [x] Whole words                ║
 ║ [x] Using shell patterns        [ ] Regular expression         ║
 ║ [ ] Case sensitive              [x] Case sensitive             ║
 ║ [ ] All charsets                [ ] All charsets               ║
 ║ [ ] Skip hidden                 [ ] First hit                  ║
 ╟────────────────────────────────────────────────────────────────╢
 ║                      [< OK >] [ Cancel ]                       ║
 ╚════════════════════════════════════════════════════════════════╝

To reproduce:

  1. wget https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz
  2. tar xvf Python-2.7.12.tar.xz
  3. cd Python-2.7.12
  4. mc
  5. Search as specified, observe a segfault.

Backtraces:

Program terminated with signal 11, Segmentation fault.
#0  0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
(gdb) bt
#0  0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
#1  0x00000031724111b5 in ?? () from /lib/libpcre.so.3
#2  0x0000003172412051 in ?? () from /lib/libpcre.so.3
#3  0x000000317241ad64 in pcre_exec () from /lib/libpcre.so.3
#4  0x00000031728562c1 in g_match_info_next () from /lib/libglib-2.0.so.0
#5  0x00000031728564e3 in g_regex_match_full () from /lib/libglib-2.0.so.0
#6  0x0000000000480ae5 in mc_search__g_regex_match_full_safe (regex=0x1f59650, 
    string=0x1f7a600 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224", 
    string_len=36, start_position=0, match_options=G_REGEX_MATCH_NEWLINE_ANY, match_info=0x1f79a48, error=0x7fff8e4b0588)
    at ../../../lib/search/regex.c:270
#7  0x0000000000480c3f in mc_search__regex_found_cond_one (lc_mc_search=0x1f79a00, regex=0x1f59650, search_str=0x1f41640)
    at ../../../lib/search/regex.c:317
#8  0x0000000000480d4f in mc_search__regex_found_cond (lc_mc_search=0x1f79a00, search_str=0x1f41640)
    at ../../../lib/search/regex.c:366
#9  0x000000000048201e in mc_search__run_regex (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/regex.c:944
#10 0x000000000047ec01 in mc_search__run_normal (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/normal.c:104
#11 0x000000000044f685 in mc_search_run (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/search.c:295
#12 0x00000000004c2051 in search_content (h=0x1f31900, 
    directory=0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command", 
    filename=0x1f58cc3 "wininst-9.0-amd64.exe") at ../../../src/filemanager/find.c:1112
#13 0x00000000004c2a17 in do_search (h=0x1f31900) at ../../../src/filemanager/find.c:1386
#14 0x00000000004c2f93 in find_callback (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
    at ../../../src/filemanager/find.c:1531
#15 0x000000000041c66f in send_message (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
    at ../../../lib/widget/widget-common.h:209
#16 0x000000000041d253 in frontend_dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:528
#17 0x000000000041e7c6 in dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:1196
#18 0x00000000004c3697 in run_process () at ../../../src/filemanager/find.c:1687
#19 0x00000000004c37cf in do_find (start_dir=0x1f40220 "/home/zaytsev/src/python/Python-2.7.12", start_dir_len=38, 
    ignore_dirs=0x0, pattern=0x1f31750 "*", content=0x1f6d280 "python-config", dirname=0x7fff8e4b2000, filename=0x7fff8e4b2008)
    at ../../../src/filemanager/find.c:1728
#20 0x00000000004c3dee in find_file () at ../../../src/filemanager/find.c:1862
#21 0x00000000004b5b8c in find_cmd () at ../../../src/filemanager/cmd.c:936
#22 0x000000000043d793 in midnight_execute_cmd (sender=0x0, command=105) at ../../../src/filemanager/midnight.c:1209
#23 0x000000000043e019 in midnight_callback (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
    at ../../../src/filemanager/midnight.c:1544
#24 0x000000000041c66f in send_message (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
    at ../../../lib/widget/widget-common.h:209
#25 0x000000000041d141 in dlg_key_event (h=0x1f395d0, d_key=8255) at ../../../lib/widget/dialog.c:489
#26 0x000000000041e706 in dlg_process_event (h=0x1f395d0, key=8255, event=0x7fff8e4b21d0) at ../../../lib/widget/dialog.c:1165
#27 0x000000000041d2d9 in frontend_dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:541
#28 0x000000000041e7c6 in dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:1196
#29 0x000000000043d1a8 in create_panels_and_run_mc () at ../../../src/filemanager/midnight.c:952
#30 0x000000000043e826 in do_nc () at ../../../src/filemanager/midnight.c:1768
#31 0x000000000040f717 in main (argc=1, argv=0x7fff8e4b23d8) at ../../src/main.c:403
(gdb) bt full
#0  0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
No symbol table info available.
#1  0x00000031724111b5 in ?? () from /lib/libpcre.so.3
No symbol table info available.
#2  0x0000003172412051 in ?? () from /lib/libpcre.so.3
No symbol table info available.
#3  0x000000317241ad64 in pcre_exec () from /lib/libpcre.so.3
No symbol table info available.
#4  0x00000031728562c1 in g_match_info_next () from /lib/libglib-2.0.so.0
No symbol table info available.
#5  0x00000031728564e3 in g_regex_match_full () from /lib/libglib-2.0.so.0
No symbol table info available.
#6  0x0000000000480ae5 in mc_search__g_regex_match_full_safe (regex=0x1f59650, 
    string=0x1f7a600 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224", 
    string_len=36, start_position=0, match_options=G_REGEX_MATCH_NEWLINE_ANY, match_info=0x1f79a48, error=0x7fff8e4b0588)
    at ../../../lib/search/regex.c:270
        string_safe = 0x40 <Address 0x40 out of bounds>
        p = 0x3170f7fe40 ""
        end = 0x0
        ret = 0
#7  0x0000000000480c3f in mc_search__regex_found_cond_one (lc_mc_search=0x1f79a00, regex=0x1f59650, search_str=0x1f41640)
    at ../../../lib/search/regex.c:317
        mcerror = 0x0
#8  0x0000000000480d4f in mc_search__regex_found_cond (lc_mc_search=0x1f79a00, search_str=0x1f41640)
    at ../../../lib/search/regex.c:366
        mc_search_cond = 0x1f59520
        ret = COND__NOT_FOUND
        loop1 = 0
#9  0x000000000048201e in mc_search__run_regex (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/regex.c:944
        ret = MC_SEARCH_CB_NOTFOUND
        current_pos = 36
        virtual_pos = 36
        start_pos = 0
        end_pos = 0
#10 0x000000000047ec01 in mc_search__run_normal (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/normal.c:104
No locals.
#11 0x000000000044f685 in mc_search_run (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36, 
    found_len=0x7fff8e4b07f8) at ../../../lib/search/search.c:295
        ret = 0
#12 0x00000000004c2051 in search_content (h=0x1f31900, 
    directory=0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command", 
    filename=0x1f58cc3 "wininst-9.0-amd64.exe") at ../../../src/filemanager/find.c:1112
        ch = 0 '\000'
        line = 192
        pos = 2079
        found = 0
        found_len = 13
        found_start = 42750
        n_read = 4096
        off = 88058
        strbuf_size = 256
        result = "\000\000@\000\000\000\000\000\025\000\000\000\000\000\000\000P\225\365\001\000\000\000\000\260\224\365\001", '\000' <repeats 12 times>, " \025\364\001\000\000\000\000\200\266\367\001\000\000\000\000\060\360@\000\000\000\000\000\320#K\216\377\177\000\000P\225\365\001\000\000\000\000\000\031K\216\377\177\000\000\335\fH\000\000\000\000\000\350\030K\216\377\177\000\000Ì\365\001\000\000\000\000 \031K\216\377\177\000\000\240\207\367\001\000\000\000\000\260\224\365\001\000\000\000\000\220\232\367\001\000\000\000\000\320#K\216\377\177\000\000\000\000\000\000\000\000\000\000\240\207\367\001\000\000\000\000Ì\365\001\000\000\000\000@\031K\216\377\177\000\000O\rH\000\000\000\000\000\240\207\367\001\000\000\000\000\220\232\367\001\000\000\000\000\300\031K\216\377\177\000\000\060\372\367\001", '\000' <repeats 16 times>, "\004\000\000\000\300\031K\216\377\177\000\000E H\000\000\000\000\000p\031K\216\377\177\000\000\070\033K\216\377\177"...
        strbuf = 0x1f7d9e0 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224"
        i = 36
        s = {st_dev = 20, st_ino = 1411177, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0, st_rdev = 0, 
          st_size = 223744, st_blksize = 4096, st_blocks = 456, st_atim = {tv_sec = 1476990802, tv_nsec = 272561009}, st_mtim = {
            tv_sec = 1466891370, tv_nsec = 0}, st_ctim = {tv_sec = 1476990796, tv_nsec = 202056713}, __unused = {0, 0, 0}}
        buffer = "6\001\000\000\213\305+\351A;\307\017\204)\001\000\000H\213D$pL\215\214$\020\001\000\000L\215D$p\306\004\003eH\003\331H\215D$|H\215\224$\240\000\000\000H\213\313H\211D$ \350\234\367\377\377A;\307\017\204\302\t\000\000I\213\314A\377\305\350T\370\377\377\213\370\211D$<\203\370-u>H\213D$pL\215\214$\020\001\000\000L\215D$p@\210<\003H\215D$|H\377\303H\215\224$\240\000\000\000H\213\313H\211D$ \350J\367\377\377A;\307\017\204p\t\000\000\353\005\203\370+u$\213Ź\001\000\000\000+\351A;\307u\005A\213\357\353\021D\003\351I\213\314\350\350\367\377\377\213\370\211D$<@\017\266\317\353`\213Ź\001\000\000\000+\351A;\307t\\H\213D$p\003\361L\215\214$\020\001\000\000@\210<\003H\003\331H\215D$|L\215D$pH\215\224$\240\000\000\000H\213\313H\211D$ \350\315\366\377\377A;\307\017\204\363\b\000\000I\213\314A\377\305\350\205\367\377\377\213\370\211D$<\017\266\310\350\367m\377\377A;\307u\226\270\001\000\000\000D+\350D\211l$4A;\376t\017I\213"...
        file_fd = 101
        ret_val = 0
        vpath = 0x1f79310
        tv = {tv_sec = 1476991628, tv_usec = 667463}
        seconds = 0
        useconds = 28731
        status_updated = 0
#13 0x00000000004c2a17 in do_search (h=0x1f31900) at ../../../src/filemanager/find.c:1386
        search_ok = 1
        dp = 0x1f58cb0
        dirp = 0x1f78fe0
        directory = 0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command"
        tmp_stat = {st_dev = 20, st_ino = 1411177, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0, 
          st_rdev = 0, st_size = 223744, st_blksize = 4096, st_blocks = 456, st_atim = {tv_sec = 1476990802, 
            tv_nsec = 272561009}, st_mtim = {tv_sec = 1466891370, tv_nsec = 0}, st_ctim = {tv_sec = 1476990796, 
            tv_nsec = 202056713}, __unused = {0, 0, 0}}
        bytes_found = 21
        count = 21
#14 0x00000000004c2f93 in find_callback (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
    at ../../../src/filemanager/find.c:1531
        h = 0x1f31900
#15 0x000000000041c66f in send_message (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
    at ../../../lib/widget/widget-common.h:209
        ret = MSG_NOT_HANDLED
#16 0x000000000041d253 in frontend_dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:528
        d_key = 0
        wh = 0x1f31900
        event = {buttons = 1, x = -1, y = 32708864, type = 0}
#17 0x000000000041e7c6 in dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:1196
No locals.
#18 0x00000000004c3697 in run_process () at ../../../src/filemanager/find.c:1687
        ret = 0
#19 0x00000000004c37cf in do_find (start_dir=0x1f40220 "/home/zaytsev/src/python/Python-2.7.12", start_dir_len=38, 
    ignore_dirs=0x0, pattern=0x1f31750 "*", content=0x1f6d280 "python-config", dirname=0x7fff8e4b2000, filename=0x7fff8e4b2008)
    at ../../../src/filemanager/find.c:1728
        return_value = 0
        dir_tmp = 0x0
        file_tmp = 0x0
#20 0x00000000004c3dee in find_file () at ../../../src/filemanager/find.c:1862
        filename = 0x0
        dirname = 0x0
        v = 1
        start_dir = 0x1f40220 "/home/zaytsev/src/python/Python-2.7.12"
        pattern = 0x1f31750 "*"
        content = 0x1f6d280 "python-config"
        ignore_dirs = 0x0
        start_dir_len = 38
#21 0x00000000004b5b8c in find_cmd () at ../../../src/filemanager/cmd.c:936
No locals.
#22 0x000000000043d793 in midnight_execute_cmd (sender=0x0, command=105) at ../../../src/filemanager/midnight.c:1209
        res = MSG_HANDLED
#23 0x000000000043e019 in midnight_callback (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
    at ../../../src/filemanager/midnight.c:1544
        v = MSG_NOT_HANDLED
        command = 105
#24 0x000000000041c66f in send_message (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
    at ../../../lib/widget/widget-common.h:209
        ret = MSG_NOT_HANDLED
#25 0x000000000041d141 in dlg_key_event (h=0x1f395d0, d_key=8255) at ../../../lib/widget/dialog.c:489
        handled = MSG_NOT_HANDLED
#26 0x000000000041e706 in dlg_process_event (h=0x1f395d0, key=8255, event=0x7fff8e4b21d0) at ../../../lib/widget/dialog.c:1165
No locals.
#27 0x000000000041d2d9 in frontend_dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:541
        d_key = 8255
        wh = 0x1f395d0
        event = {buttons = 5128754, x = -1, y = 32740816, type = 0}
#28 0x000000000041e7c6 in dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:1196
No locals.
#29 0x000000000043d1a8 in create_panels_and_run_mc () at ../../../src/filemanager/midnight.c:952
No locals.
#30 0x000000000043e826 in do_nc () at ../../../src/filemanager/midnight.c:1768
        ret = 49
#31 0x000000000040f717 in main (argc=1, argv=0x7fff8e4b23d8) at ../../src/main.c:403
        mcerror = 0x0
        config_migrated = 0
        config_migrate_msg = 0x4dcfd5 "H\205\355t\034\061\333\353\002\220\220L\211\372L\211\366D\211\357A\377\024\334H\203\303\001H9\353r\352H\213\\$\bH\213l$\020L\213d$\030L\213l$ L\213t$(L\213|$0H\203\304\070Ð\220\220\220\220\220\220H\211\362H\211\376\277\001"
        exit_code = 1
(gdb) 

Change History

comment:1 Changed 2 years ago by zaytsev

The crash happens only if [x] Whole words is ticked.

comment:2 Changed 2 years ago by zaytsev

  • Cc egmont added

The problem is in the search engine and not directly related to the Find File dialog: if one opens Python-2.7.12/Lib/distutils/command/wininst-9.0.exe and searches in the viewer or editor for anything with [x] Whole words enabled, it will crash.

$ locale
LANG=en_US.UTF-8

comment:3 Changed 2 years ago by zaytsev

Sounds like something very much reminiscent of #3449, apparently mc_search__g_regex_match_full_safe isn't all that safe after all :-(

comment:4 Changed 2 years ago by egmont

It does not crash for me.

I'm on Ubuntu Yakkety (glib 2.50.0, libpcre3 8.39), tried mc 4.8.17 as shipped by the distro, as well as current git.

I executed mcview wininst-9.0.exe and searched for strings like "foo" (there is a match), "foobar", "asdf" and similar ones. "Whole words" enabled; tried normal as well as regex mode. Locale is also en_US.UTF-8.

Based on the stack trace I suspect a bug in your libpcre3. What version do you have of it?

Could you please add some debugging to your mc_search__g_regex_match_full_safe() so that it records in some binary-safe way the string/string_safe (and string_len) parameters that it passes to g_regex_match_full() so that we can manually examine whether it's valid UTF-8?

comment:5 follow-up: ↓ 6 Changed 2 years ago by zaytsev

Based on the stack trace I suspect a bug in your libpcre3. What version do you have of it?

Oh, it's many years out of date, my version is libpcre3 8.02 and glib 2.26.1. Okay, I can try to upgrade libpcre3 to the next release and see if it goes away. It could be also likewise a bug in glib though :-/ both are way too old. Sigh...

Could you please add some debugging to your mc_search__g_regex_match_full_safe() so that it records in some binary-safe way the string/string_safe (and string_len) parameters that it passes to g_regex_match_full()

Well, you have it in the full backtrace, right? Apparently it goes into the first branch, thinking it's valid. I will see if I can get time to investigate some more...

Thank you for your feedback! That it doesn't crash on a newer system is a very important piece of information.

comment:6 in reply to: ↑ 5 Changed 2 years ago by egmont

Replying to zaytsev:

Well, you have it in the full backtrace, right? Apparently it goes into the first branch, thinking it's valid.

That would be a problem, because the string is clearly invalid UTF-8. How can you tell from the backtrace that is goes to the first branch? I don't think you can tell it.

comment:7 follow-up: ↓ 8 Changed 2 years ago by zaytsev-work

My line of thinking was that in the backtrace string_safe = 0x40, which could mean that it's not initialised properly, as in when the first branch is taken, but, of course, you are right that one should build in additional logging instead of engaging in further speculation. In the mean time, I can confirm that it doesn't segfault on Debian 8 (current stable) :-/

comment:8 in reply to: ↑ 7 Changed 2 years ago by egmont

Replying to zaytsev-work:

My line of thinking was that in the backtrace string_safe = 0x40

I just noticed there was a "bt full" section as well after "bt" :)

How to read this format? Does this string_safe = 0x40 refer to the moment when g_regex_match_full() is called? Could gcc optimization tamper with these values?

Yeah let's do some additional poor man's (printf-style) debugging.

comment:9 Changed 2 years ago by andrew_b

So? Invalid?

comment:10 Changed 2 years ago by zaytsev

Just built pcre-8.39.tar.gz from source and installed in /opt/pcre:

../configure --with-search-engine=pcre --with-pcre=/opt/pcre

So, apparently, it's a bug in pcre and/or glib, which was fixed in later releases. The fact that it crashes in _pcre_xclass () was hinting at it, but the question of whether it's being passed valid data was an open one. Now, it's clear that the call is valid, and the problem is really somewhere inside pcre and/or glib.

Thanks everybody and sorry for the noise! I will build with newer pcre from now on...

comment:11 Changed 2 years ago by zaytsev

  • Status changed from new to closed
  • Resolution set to invalid
Note: See TracTickets for help on using tickets.