mcedit crash on safe (shadows)

[slyfox]

It's a forward of upstream ​https://bugs.gentoo.org/768285 ticket by Thomas Deutschmann where mc crashes on certain window sizes in shadow clipping code.

Backtrace is:

#0  g_log_structured_array (log_level=<optimized out>, fields=0x7fffe2dff550, n_fields=4)
    at ../glib-2.66.4/glib/gmessages.c:554
554     ../glib-2.66.4/glib/gmessages.c: No such file or directory.
(gdb) bt
#0  g_log_structured_array (log_level=<optimized out>, fields=0x7fffe2dff550, n_fields=4)
    at ../glib-2.66.4/glib/gmessages.c:554
#1  0x00007fb071105955 in g_log_default_handler (log_domain=log_domain@entry=0x7fb07115300e "GLib",
    log_level=log_level@entry=6,
    message=message@entry=0x56421febd830 "../glib-2.66.4/glib/gmem.c:112: failed to allocate 18446744073709551588 bytes", unused_data=unused_data@entry=0x0) at ../glib-2.66.4/glib/gmessages.c:3123
#2  0x00007fb071105bb1 in g_logv (log_domain=0x7fb07115300e "GLib", log_level=G_LOG_LEVEL_ERROR,
    format=<optimized out>, args=<optimized out>) at ../glib-2.66.4/glib/gmessages.c:1350
#3  0x00007fb071105e57 in g_log (log_domain=log_domain@entry=0x7fb07115300e "GLib",
    log_level=log_level@entry=G_LOG_LEVEL_ERROR, format=format@entry=0x7fb07115d790 "%s: failed to allocate %lu bytes")
    at ../glib-2.66.4/glib/gmessages.c:1415
#4  0x00007fb0711045a4 in g_malloc (n_bytes=18446744073709551588) at ../glib-2.66.4/glib/gmem.c:111
#5  0x000056421ef63ea1 in tty_colorize_area (y=12, x=122, rows=6, cols=-2, color=10) at tty-ncurses.c:565
#6  0x000056421ef64ec5 in tty_draw_box_shadow (y=11, x=-2, rows=7, cols=124, shadow_color=10) at tty.c:272
#7  0x000056421ef27b58 in frame_draw (f=0x56421febb550) at frame.c:80
#8  0x000056421ef27d65 in frame_callback (w=0x56421febb550, sender=0x0, msg=MSG_DRAW, parm=0, data=0x0) at frame.c:132
#9  0x000056421ef6c15d in widget_draw (w=0x56421febb550) at widget-common.c:526
#10 0x000056421ef298bd in group_draw (g=0x56421feb8200) at group.c:420
#11 0x000056421ef29d03 in group_default_callback (w=0x56421feb8200, sender=0x0, msg=MSG_DRAW, parm=0, data=0x0)
    at group.c:563
#12 0x000056421ef192c1 in dlg_default_callback (w=0x56421feb8200, sender=0x0, msg=MSG_DRAW, parm=0, data=0x0)
    at dialog.c:343
#13 0x000056421ef6cef5 in query_default_callback (w=0x56421feb8200, sender=0x0, msg=MSG_DRAW, parm=0, data=0x0)
    at wtools.c:112
#14 0x000056421ef6c15d in widget_draw (w=0x56421feb8200) at widget-common.c:526
#15 0x000056421ef6c733 in widget_default_set_state (w=0x56421feb8200, state=WST_FOCUSED, enable=1)
    at widget-common.c:779
#16 0x000056421ef29e0c in group_default_set_state (w=0x56421feb8200, state=WST_FOCUSED, enable=1) at group.c:610
#17 0x000056421ef18b4e in widget_set_state (w=0x56421feb8200, state=WST_FOCUSED, enable=1)
    at ../../lib/widget/widget-common.h:343
#18 0x000056421ef19948 in dlg_init (h=0x56421feb8200) at dialog.c:515
#19 0x000056421ef19a98 in dlg_run (h=0x56421feb8200) at dialog.c:574
#20 0x000056421ef6da61 in query_dialog (header=0x56421effa2ea "Save file",
    text=0x56421fddc940 "Confirm save file: \"/home/thomas/repositories/gentoo/dev-php/doctrine-instantiator/doctrine-instantiator-1.3.1.ebuild\"", flags=0, count=2) at wtools.c:350
#21 0x000056421efa163d in edit_save_confirm_cmd (edit=0x56421fe7edb0) at editcmd.c:2049
#22 0x000056421ef9b0b8 in edit_execute_cmd (edit=0x56421fe7edb0, command=53, char_for_insertion=-1) at edit.c:3826
#23 0x000056421ef99841 in edit_execute_key_command (edit=0x56421fe7edb0, command=53, char_for_insertion=-1)
--Type <RET> for more, q to quit, c to continue without paging--c
    at edit.c:3235
#24 0x000056421efab8dd in edit_callback (w=0x56421fe7edb0, sender=0x56421fe7ebc0, msg=MSG_ACTION, parm=53, data=0x0) at editwidget.c:971
#25 0x000056421ef14ceb in send_message (w=0x56421fe7edb0, sender=0x56421fe7ebc0, msg=MSG_ACTION, parm=53, data=0x0) at ../../lib/widget/widget-common.h:243
#26 0x000056421ef15053 in buttonbar_call (bb=0x56421fe7ebc0, i=1) at buttonbar.c:154
#27 0x000056421ef150c8 in buttonbar_callback (w=0x56421fe7ebc0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at buttonbar.c:171
#28 0x000056421ef28db4 in send_message (w=0x56421fe7ebc0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at ../../lib/widget/widget-common.h:243
#29 0x000056421ef29b44 in group_handle_hotkey (g=0x56421fe776a0, key=266) at group.c:497
#30 0x000056421ef29d34 in group_default_callback (w=0x56421fe776a0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at group.c:570
#31 0x000056421ef192c1 in dlg_default_callback (w=0x56421fe776a0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at dialog.c:343
#32 0x000056421efab546 in edit_dialog_callback (w=0x56421fe776a0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at editwidget.c:834
#33 0x000056421ef28db4 in send_message (w=0x56421fe776a0, sender=0x0, msg=MSG_HOTKEY, parm=266, data=0x0) at ../../lib/widget/widget-common.h:243
#34 0x000056421ef2991d in group_handle_key (g=0x56421fe776a0, key=266) at group.c:434
#35 0x000056421ef29d1e in group_default_callback (w=0x56421fe776a0, sender=0x0, msg=MSG_KEY, parm=266, data=0x0) at group.c:567
#36 0x000056421ef1900d in dlg_key_event (h=0x56421fe776a0, d_key=266) at dialog.c:251
#37 0x000056421ef199d1 in dlg_process_event (h=0x56421fe776a0, key=266, event=0x7fffe2e00340) at dialog.c:541
#38 0x000056421ef191ff in frontend_dlg_run (h=0x56421fe776a0) at dialog.c:320
#39 0x000056421ef19aa4 in dlg_run (h=0x56421fe776a0) at dialog.c:575
#40 0x000056421efac33d in edit_files (files=0x56421fe04d40 = {...}) at editwidget.c:1272
#41 0x000056421ef21970 in mc_maybe_editor_or_viewer () at filemanager.c:977
#42 0x000056421ef2316e in do_nc () at filemanager.c:1826
#43 0x000056421ef0b91f in main (argc=2, argv=0x7fffe2e005b8) at main.c:463

[slyfox]

mc-4.8.26-clip-768285.patch

[slyfox]

If I understand the trigger values correctly (from ​https://bugs.gentoo.org/768285#c3):

tty_colorize_area (y + 1, x + cols, rows - 1, 2, shadow_color); ('right' case)
y = 12;
x = 122;
rows = 6;
cols = 2;

tty_clip()

cols = -2; somehow, perhaps due to:

if (*x + *cols > COLS)

*cols = COLS - *x;

To get negative COLS should be 120.

Thus right hand side box clipping is not enough. Proposed patch:

lib/tty/tty-ncurses.c

@@ -135 +135 @@
         *rows += *y;
 
         if (*rows <= 0)
             return FALSE;
 
         *y = 0;
     }
 
     if (*x < 0)
     {
         *cols += *x;
 
         if (*cols <= 0)
             return FALSE;
 
         *x = 0;
     }
 
     if (*y + *rows > LINES)
         *rows = LINES - *y;
+    if (*rows <= 0)
+        return FALSE;
     if (*x + *cols > COLS)
         *cols = COLS - *x;
+    if (*cols <= 0)
+        return FALSE;
 
     return TRUE;
 }

[andrew_b]

Thanks for the patch!

Branch: 4192_shadows_out_of_screen.
changset:6394547dbffbad44ea50c64c282de4b610ca07bf

[andrew_b]

Merged to master: [a34525c5fa247f6f808e5034ccb7806117290f0d].