Ticket #3705 (closed defect: invalid)
Reproducible segfault in Find File dialog
| Reported by: | zaytsev | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | 4.8.19 |
| Component: | mc-search | Version: | 4.8.18 |
| Keywords: | Cc: | egmont | |
| Blocked By: | Blocking: | ||
| Branch state: | no branch | Votes for changeset: |
Description
I'm able to reliably crash the latest release while searching in Python sources for the following:
╔══════════════════════════ Find File ═══════════════════════════╗ ║ Start at: ║ ║ . [^] [ Tree ] ║ ║ [ ] Enable ignore directories: ║ ║ .git:.hg:.svn:build [^] ║ ╟────────────────────────────────────────────────────────────────╢ ║ File name: Content: ║ ║ * [^] python-config [^] ║ ║ [x] Find recursively [x] Whole words ║ ║ [x] Using shell patterns [ ] Regular expression ║ ║ [ ] Case sensitive [x] Case sensitive ║ ║ [ ] All charsets [ ] All charsets ║ ║ [ ] Skip hidden [ ] First hit ║ ╟────────────────────────────────────────────────────────────────╢ ║ [< OK >] [ Cancel ] ║ ╚════════════════════════════════════════════════════════════════╝
To reproduce:
- wget https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz
- tar xvf Python-2.7.12.tar.xz
- cd Python-2.7.12
- mc
- Search as specified, observe a segfault.
Backtraces:
Program terminated with signal 11, Segmentation fault.
#0 0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
(gdb) bt
#0 0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
#1 0x00000031724111b5 in ?? () from /lib/libpcre.so.3
#2 0x0000003172412051 in ?? () from /lib/libpcre.so.3
#3 0x000000317241ad64 in pcre_exec () from /lib/libpcre.so.3
#4 0x00000031728562c1 in g_match_info_next () from /lib/libglib-2.0.so.0
#5 0x00000031728564e3 in g_regex_match_full () from /lib/libglib-2.0.so.0
#6 0x0000000000480ae5 in mc_search__g_regex_match_full_safe (regex=0x1f59650,
string=0x1f7a600 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224",
string_len=36, start_position=0, match_options=G_REGEX_MATCH_NEWLINE_ANY, match_info=0x1f79a48, error=0x7fff8e4b0588)
at ../../../lib/search/regex.c:270
#7 0x0000000000480c3f in mc_search__regex_found_cond_one (lc_mc_search=0x1f79a00, regex=0x1f59650, search_str=0x1f41640)
at ../../../lib/search/regex.c:317
#8 0x0000000000480d4f in mc_search__regex_found_cond (lc_mc_search=0x1f79a00, search_str=0x1f41640)
at ../../../lib/search/regex.c:366
#9 0x000000000048201e in mc_search__run_regex (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/regex.c:944
#10 0x000000000047ec01 in mc_search__run_normal (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/normal.c:104
#11 0x000000000044f685 in mc_search_run (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/search.c:295
#12 0x00000000004c2051 in search_content (h=0x1f31900,
directory=0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command",
filename=0x1f58cc3 "wininst-9.0-amd64.exe") at ../../../src/filemanager/find.c:1112
#13 0x00000000004c2a17 in do_search (h=0x1f31900) at ../../../src/filemanager/find.c:1386
#14 0x00000000004c2f93 in find_callback (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
at ../../../src/filemanager/find.c:1531
#15 0x000000000041c66f in send_message (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
at ../../../lib/widget/widget-common.h:209
#16 0x000000000041d253 in frontend_dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:528
#17 0x000000000041e7c6 in dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:1196
#18 0x00000000004c3697 in run_process () at ../../../src/filemanager/find.c:1687
#19 0x00000000004c37cf in do_find (start_dir=0x1f40220 "/home/zaytsev/src/python/Python-2.7.12", start_dir_len=38,
ignore_dirs=0x0, pattern=0x1f31750 "*", content=0x1f6d280 "python-config", dirname=0x7fff8e4b2000, filename=0x7fff8e4b2008)
at ../../../src/filemanager/find.c:1728
#20 0x00000000004c3dee in find_file () at ../../../src/filemanager/find.c:1862
#21 0x00000000004b5b8c in find_cmd () at ../../../src/filemanager/cmd.c:936
#22 0x000000000043d793 in midnight_execute_cmd (sender=0x0, command=105) at ../../../src/filemanager/midnight.c:1209
#23 0x000000000043e019 in midnight_callback (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
at ../../../src/filemanager/midnight.c:1544
#24 0x000000000041c66f in send_message (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
at ../../../lib/widget/widget-common.h:209
#25 0x000000000041d141 in dlg_key_event (h=0x1f395d0, d_key=8255) at ../../../lib/widget/dialog.c:489
#26 0x000000000041e706 in dlg_process_event (h=0x1f395d0, key=8255, event=0x7fff8e4b21d0) at ../../../lib/widget/dialog.c:1165
#27 0x000000000041d2d9 in frontend_dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:541
#28 0x000000000041e7c6 in dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:1196
#29 0x000000000043d1a8 in create_panels_and_run_mc () at ../../../src/filemanager/midnight.c:952
#30 0x000000000043e826 in do_nc () at ../../../src/filemanager/midnight.c:1768
#31 0x000000000040f717 in main (argc=1, argv=0x7fff8e4b23d8) at ../../src/main.c:403
(gdb) bt full
#0 0x000000317241d907 in _pcre_xclass () from /lib/libpcre.so.3
No symbol table info available.
#1 0x00000031724111b5 in ?? () from /lib/libpcre.so.3
No symbol table info available.
#2 0x0000003172412051 in ?? () from /lib/libpcre.so.3
No symbol table info available.
#3 0x000000317241ad64 in pcre_exec () from /lib/libpcre.so.3
No symbol table info available.
#4 0x00000031728562c1 in g_match_info_next () from /lib/libglib-2.0.so.0
No symbol table info available.
#5 0x00000031728564e3 in g_regex_match_full () from /lib/libglib-2.0.so.0
No symbol table info available.
#6 0x0000000000480ae5 in mc_search__g_regex_match_full_safe (regex=0x1f59650,
string=0x1f7a600 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224",
string_len=36, start_position=0, match_options=G_REGEX_MATCH_NEWLINE_ANY, match_info=0x1f79a48, error=0x7fff8e4b0588)
at ../../../lib/search/regex.c:270
string_safe = 0x40 <Address 0x40 out of bounds>
p = 0x3170f7fe40 ""
end = 0x0
ret = 0
#7 0x0000000000480c3f in mc_search__regex_found_cond_one (lc_mc_search=0x1f79a00, regex=0x1f59650, search_str=0x1f41640)
at ../../../lib/search/regex.c:317
mcerror = 0x0
#8 0x0000000000480d4f in mc_search__regex_found_cond (lc_mc_search=0x1f79a00, search_str=0x1f41640)
at ../../../lib/search/regex.c:366
mc_search_cond = 0x1f59520
ret = COND__NOT_FOUND
loop1 = 0
#9 0x000000000048201e in mc_search__run_regex (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/regex.c:944
ret = MC_SEARCH_CB_NOTFOUND
current_pos = 36
virtual_pos = 36
start_pos = 0
end_pos = 0
#10 0x000000000047ec01 in mc_search__run_normal (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/normal.c:104
No locals.
#11 0x000000000044f685 in mc_search_run (lc_mc_search=0x1f79a00, user_data=0x1f7d9e0, start_search=0, end_search=36,
found_len=0x7fff8e4b07f8) at ../../../lib/search/search.c:295
ret = 0
#12 0x00000000004c2051 in search_content (h=0x1f31900,
directory=0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command",
filename=0x1f58cc3 "wininst-9.0-amd64.exe") at ../../../src/filemanager/find.c:1112
ch = 0 '\000'
line = 192
pos = 2079
found = 0
found_len = 13
found_start = 42750
n_read = 4096
off = 88058
strbuf_size = 256
result = "\000\000@\000\000\000\000\000\025\000\000\000\000\000\000\000P\225\365\001\000\000\000\000\260\224\365\001", '\000' <repeats 12 times>, " \025\364\001\000\000\000\000\200\266\367\001\000\000\000\000\060\360@\000\000\000\000\000\320#K\216\377\177\000\000P\225\365\001\000\000\000\000\000\031K\216\377\177\000\000\335\fH\000\000\000\000\000\350\030K\216\377\177\000\000Ì\365\001\000\000\000\000 \031K\216\377\177\000\000\240\207\367\001\000\000\000\000\260\224\365\001\000\000\000\000\220\232\367\001\000\000\000\000\320#K\216\377\177\000\000\000\000\000\000\000\000\000\000\240\207\367\001\000\000\000\000Ì\365\001\000\000\000\000@\031K\216\377\177\000\000O\rH\000\000\000\000\000\240\207\367\001\000\000\000\000\220\232\367\001\000\000\000\000\300\031K\216\377\177\000\000\060\372\367\001", '\000' <repeats 16 times>, "\004\000\000\000\300\031K\216\377\177\000\000E H\000\000\000\000\000p\031K\216\377\177\000\000\070\033K\216\377\177"...
strbuf = 0x1f7d9e0 "\213l$H\213\\$4A\203\375xt5A\203\375pt/@\017\266\317\350\025g\377\377\063\311;\301\017\204\224"
i = 36
s = {st_dev = 20, st_ino = 1411177, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0, st_rdev = 0,
st_size = 223744, st_blksize = 4096, st_blocks = 456, st_atim = {tv_sec = 1476990802, tv_nsec = 272561009}, st_mtim = {
tv_sec = 1466891370, tv_nsec = 0}, st_ctim = {tv_sec = 1476990796, tv_nsec = 202056713}, __unused = {0, 0, 0}}
buffer = "6\001\000\000\213\305+\351A;\307\017\204)\001\000\000H\213D$pL\215\214$\020\001\000\000L\215D$p\306\004\003eH\003\331H\215D$|H\215\224$\240\000\000\000H\213\313H\211D$ \350\234\367\377\377A;\307\017\204\302\t\000\000I\213\314A\377\305\350T\370\377\377\213\370\211D$<\203\370-u>H\213D$pL\215\214$\020\001\000\000L\215D$p@\210<\003H\215D$|H\377\303H\215\224$\240\000\000\000H\213\313H\211D$ \350J\367\377\377A;\307\017\204p\t\000\000\353\005\203\370+u$\213Ź\001\000\000\000+\351A;\307u\005A\213\357\353\021D\003\351I\213\314\350\350\367\377\377\213\370\211D$<@\017\266\317\353`\213Ź\001\000\000\000+\351A;\307t\\H\213D$p\003\361L\215\214$\020\001\000\000@\210<\003H\003\331H\215D$|L\215D$pH\215\224$\240\000\000\000H\213\313H\211D$ \350\315\366\377\377A;\307\017\204\363\b\000\000I\213\314A\377\305\350\205\367\377\377\213\370\211D$<\017\266\310\350\367m\377\377A;\307u\226\270\001\000\000\000D+\350D\211l$4A;\376t\017I\213"...
file_fd = 101
ret_val = 0
vpath = 0x1f79310
tv = {tv_sec = 1476991628, tv_usec = 667463}
seconds = 0
useconds = 28731
status_updated = 0
#13 0x00000000004c2a17 in do_search (h=0x1f31900) at ../../../src/filemanager/find.c:1386
search_ok = 1
dp = 0x1f58cb0
dirp = 0x1f78fe0
directory = 0x1f76fc0 "/home/zaytsev/src/python/Python-2.7.12/Lib/distutils/command"
tmp_stat = {st_dev = 20, st_ino = 1411177, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0,
st_rdev = 0, st_size = 223744, st_blksize = 4096, st_blocks = 456, st_atim = {tv_sec = 1476990802,
tv_nsec = 272561009}, st_mtim = {tv_sec = 1466891370, tv_nsec = 0}, st_ctim = {tv_sec = 1476990796,
tv_nsec = 202056713}, __unused = {0, 0, 0}}
bytes_found = 21
count = 21
#14 0x00000000004c2f93 in find_callback (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
at ../../../src/filemanager/find.c:1531
h = 0x1f31900
#15 0x000000000041c66f in send_message (w=0x1f31900, sender=0x0, msg=MSG_IDLE, parm=0, data=0x0)
at ../../../lib/widget/widget-common.h:209
ret = MSG_NOT_HANDLED
#16 0x000000000041d253 in frontend_dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:528
d_key = 0
wh = 0x1f31900
event = {buttons = 1, x = -1, y = 32708864, type = 0}
#17 0x000000000041e7c6 in dlg_run (h=0x1f31900) at ../../../lib/widget/dialog.c:1196
No locals.
#18 0x00000000004c3697 in run_process () at ../../../src/filemanager/find.c:1687
ret = 0
#19 0x00000000004c37cf in do_find (start_dir=0x1f40220 "/home/zaytsev/src/python/Python-2.7.12", start_dir_len=38,
ignore_dirs=0x0, pattern=0x1f31750 "*", content=0x1f6d280 "python-config", dirname=0x7fff8e4b2000, filename=0x7fff8e4b2008)
at ../../../src/filemanager/find.c:1728
return_value = 0
dir_tmp = 0x0
file_tmp = 0x0
#20 0x00000000004c3dee in find_file () at ../../../src/filemanager/find.c:1862
filename = 0x0
dirname = 0x0
v = 1
start_dir = 0x1f40220 "/home/zaytsev/src/python/Python-2.7.12"
pattern = 0x1f31750 "*"
content = 0x1f6d280 "python-config"
ignore_dirs = 0x0
start_dir_len = 38
#21 0x00000000004b5b8c in find_cmd () at ../../../src/filemanager/cmd.c:936
No locals.
#22 0x000000000043d793 in midnight_execute_cmd (sender=0x0, command=105) at ../../../src/filemanager/midnight.c:1209
res = MSG_HANDLED
#23 0x000000000043e019 in midnight_callback (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
at ../../../src/filemanager/midnight.c:1544
v = MSG_NOT_HANDLED
command = 105
#24 0x000000000041c66f in send_message (w=0x1f395d0, sender=0x0, msg=MSG_UNHANDLED_KEY, parm=8255, data=0x0)
at ../../../lib/widget/widget-common.h:209
ret = MSG_NOT_HANDLED
#25 0x000000000041d141 in dlg_key_event (h=0x1f395d0, d_key=8255) at ../../../lib/widget/dialog.c:489
handled = MSG_NOT_HANDLED
#26 0x000000000041e706 in dlg_process_event (h=0x1f395d0, key=8255, event=0x7fff8e4b21d0) at ../../../lib/widget/dialog.c:1165
No locals.
#27 0x000000000041d2d9 in frontend_dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:541
d_key = 8255
wh = 0x1f395d0
event = {buttons = 5128754, x = -1, y = 32740816, type = 0}
#28 0x000000000041e7c6 in dlg_run (h=0x1f395d0) at ../../../lib/widget/dialog.c:1196
No locals.
#29 0x000000000043d1a8 in create_panels_and_run_mc () at ../../../src/filemanager/midnight.c:952
No locals.
#30 0x000000000043e826 in do_nc () at ../../../src/filemanager/midnight.c:1768
ret = 49
#31 0x000000000040f717 in main (argc=1, argv=0x7fff8e4b23d8) at ../../src/main.c:403
mcerror = 0x0
config_migrated = 0
config_migrate_msg = 0x4dcfd5 "H\205\355t\034\061\333\353\002\220\220L\211\372L\211\366D\211\357A\377\024\334H\203\303\001H9\353r\352H\213\\$\bH\213l$\020L\213d$\030L\213l$ L\213t$(L\213|$0H\203\304\070Ð\220\220\220\220\220\220H\211\362H\211\376\277\001"
exit_code = 1
(gdb)
Change History
comment:2 Changed 7 years ago by zaytsev
- Cc egmont added
The problem is in the search engine and not directly related to the Find File dialog: if one opens Python-2.7.12/Lib/distutils/command/wininst-9.0.exe and searches in the viewer or editor for anything with [x] Whole words enabled, it will crash.
$ locale LANG=en_US.UTF-8
comment:3 Changed 7 years ago by zaytsev
Sounds like something very much reminiscent of #3449, apparently mc_search__g_regex_match_full_safe isn't all that safe after all :-(
comment:4 Changed 7 years ago by egmont
It does not crash for me.
I'm on Ubuntu Yakkety (glib 2.50.0, libpcre3 8.39), tried mc 4.8.17 as shipped by the distro, as well as current git.
I executed mcview wininst-9.0.exe and searched for strings like "foo" (there is a match), "foobar", "asdf" and similar ones. "Whole words" enabled; tried normal as well as regex mode. Locale is also en_US.UTF-8.
Based on the stack trace I suspect a bug in your libpcre3. What version do you have of it?
Could you please add some debugging to your mc_search__g_regex_match_full_safe() so that it records in some binary-safe way the string/string_safe (and string_len) parameters that it passes to g_regex_match_full() so that we can manually examine whether it's valid UTF-8?
comment:5 follow-up: ↓ 6 Changed 7 years ago by zaytsev
Based on the stack trace I suspect a bug in your libpcre3. What version do you have of it?
Oh, it's many years out of date, my version is libpcre3 8.02 and glib 2.26.1. Okay, I can try to upgrade libpcre3 to the next release and see if it goes away. It could be also likewise a bug in glib though :-/ both are way too old. Sigh...
Could you please add some debugging to your mc_search__g_regex_match_full_safe() so that it records in some binary-safe way the string/string_safe (and string_len) parameters that it passes to g_regex_match_full()
Well, you have it in the full backtrace, right? Apparently it goes into the first branch, thinking it's valid. I will see if I can get time to investigate some more...
Thank you for your feedback! That it doesn't crash on a newer system is a very important piece of information.
comment:6 in reply to: ↑ 5 Changed 7 years ago by egmont
Replying to zaytsev:
Well, you have it in the full backtrace, right? Apparently it goes into the first branch, thinking it's valid.
That would be a problem, because the string is clearly invalid UTF-8. How can you tell from the backtrace that is goes to the first branch? I don't think you can tell it.
comment:7 follow-up: ↓ 8 Changed 7 years ago by zaytsev-work
My line of thinking was that in the backtrace string_safe = 0x40, which could mean that it's not initialised properly, as in when the first branch is taken, but, of course, you are right that one should build in additional logging instead of engaging in further speculation. In the mean time, I can confirm that it doesn't segfault on Debian 8 (current stable) :-/
comment:8 in reply to: ↑ 7 Changed 7 years ago by egmont
Replying to zaytsev-work:
My line of thinking was that in the backtrace string_safe = 0x40
I just noticed there was a "bt full" section as well after "bt" :)
How to read this format? Does this string_safe = 0x40 refer to the moment when g_regex_match_full() is called? Could gcc optimization tamper with these values?
Yeah let's do some additional poor man's (printf-style) debugging.
comment:10 Changed 7 years ago by zaytsev
Just built pcre-8.39.tar.gz from source and installed in /opt/pcre:
../configure --with-search-engine=pcre --with-pcre=/opt/pcre
So, apparently, it's a bug in pcre and/or glib, which was fixed in later releases. The fact that it crashes in _pcre_xclass () was hinting at it, but the question of whether it's being passed valid data was an open one. Now, it's clear that the call is valid, and the problem is really somewhere inside pcre and/or glib.
Thanks everybody and sorry for the noise! I will build with newer pcre from now on...
comment:11 Changed 7 years ago by zaytsev
- Status changed from new to closed
- Resolution set to invalid
The crash happens only if [x] Whole words is ticked.