mc crashes on startup with subshell support as default in Freebsd

[support.od.ua]

At first I thought it was a problem in my version of the system and opened a ​ticket there.
After discussing crashdump, we think the problem is mc.

mc on the system can be run guaranteed with the -u option (--nosubshell).

​Here are the archives for the application mc (under FreeBSD) and crashdump mc.core.

I am ready to provide additional information upon request.

Backtrace full is:

(gdb) bt full
#0  0x00000000002c8aa6 in str_utf8_make_make_term_form (text=0x0, length=18446744073709551615) at strutilutf8.c:454
        result = {
          text = "\000 \000valid)\000  2021\000\061\000\000\273ь\000...\000\200\000\271ки\000\210...\000\201\000\200ования файлов", '\000' <repeats 3002 times>, width = 0, compose = 0}
        uni = 0
        left = 0
        actual = 0x34e410 <str_utf8_make_make_term_form.result> ""
#1  0x00000000002c7bfd in str_utf8_term_width2 (text=0x0, length=18446744073709551615) at strutilutf8.c:802
        result = 0x0
#2  0x00000000002c7c3c in str_utf8_term_width1 (text=0x0) at strutilutf8.c:811
No locals.
#3  0x00000000002c2359 in str_term_width1 (text=0x0) at strutil.c:654
No locals.
#4  0x000000000029c13f in setup_cmdline () at layout.c:976
        mw = 0x801a95c00
        prompt_width = 174
        y = 1
        tmp_prompt = 0x0
#5  0x000000000029b179 in setup_panels () at layout.c:889
        mw = 0x801a95c00
        start_y = 1
#6  0x000000000026320e in midnight_callback (w=0x801a95c00, sender=0x0, msg=MSG_INIT, parm=0, data=0x0) at filemanager.c:1498
        command = 0
#7  0x00000000002516da in send_message (w=0x801a95c00, sender=0x0, msg=MSG_INIT, parm=0, data=0x0) at ../../lib/widget/widget-common.h:243
        ret = MSG_NOT_HANDLED
#8  0x000000000025155b in dlg_init (h=0x801a95c00) at dialog.c:503
        g = 0x801a95c00
        wh = 0x801a95c00
#9  0x0000000000251d95 in dlg_run (h=0x801a95c00) at dialog.c:574
No locals.
#10 0x00000000002630dd in do_nc () at filemanager.c:1837
        ret = 8
#11 0x000000000023dce0 in main (argc=1, argv=0x7fffffffda58) at main.c:463
        mcerror = 0x0
        config_migrated = 0
        config_migrate_msg = 0x0
        exit_code = 1

# env LC_MESSAGES=C mc -V
GNU Midnight Commander 4.8.26
Built with GLib 2.66.7
Built with S-Lang 2.3.2 with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems:
 cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish
Data types:
 char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

# env LC_MESSAGES=C mc -F
Home directory: /root
Profile root directory: /root

[System data]
    Config directory: /usr/local/etc/mc/
    Data directory:   /usr/local/share/mc/
    File extension handlers: /usr/local/libexec/mc/ext.d/
    VFS plugins and scripts: /usr/local/libexec/mc/
        extfs.d:        /usr/local/libexec/mc/extfs.d/
        fish:           /usr/local/libexec/mc/fish/

[User data]
    Config directory: /root/.config/mc/
    Data directory:   /root/.local/share/mc/
        skins:          /root/.local/share/mc/skins/
        extfs.d:        /root/.local/share/mc/extfs.d/
        fish:           /root/.local/share/mc/fish/
        mcedit macros:  /root/.local/share/mc/mc.macros
        mcedit external macros: /root/.local/share/mc/mcedit/macros.d/macro.*
    Cache directory:  /root/.cache/mc/

# env LC_MESSAGES=C mc --configure-options
 '--with-internal-edit' '--enable-charset' '--enable-nls' '--enable-vfs-sftp' '--with-screen=slang' '--with-slang-includes=/usr/local/include' '--disable-vfs-smb' '--without-smb-configdir' '--without-smb-codepagedir' '--with-subshell' '--disable-x' '--prefix=/usr/local' '--localstatedir=/var' '--mandir=/usr/local/man' '--disable-silent-rules' '--infodir=/usr/local/share/info/' '--build=amd64-portbld-freebsd12.2' 'build_alias=amd64-portbld-freebsd12.2' 'CC=cc' 'CFLAGS=-pipe  -DLIBICONV_PLUG -g -fstack-protector-strong -fno-strict-aliasing ' 'LDFLAGS= -L/usr/local/lib -fstack-protector-strong ' 'LIBS=' 'CPPFLAGS=-I/usr/local/include -DLIBICONV_PLUG' 'CPP=cpp' 'PKG_CONFIG=pkgconf'

# uname -a
FreeBSD vb-12.2.0.domain.com 12.2-STABLE FreeBSD 12.2-STABLE #0 r369260M: Sat Feb 13 11:28:24 EET 2021     root@vb-12.2.0.domain.com:/usr/obj/usr/src/amd64.amd64/sys/vb-12.2.0.domain.com.1  amd64

[support.od.ua]

I suggest such a patch to fix the problem

[andrew_b]

There was a fix of subshell related bug (#4182) after .26 release. It might fix this bug also. Could you please try the recent master?

[andrew_b]

If is still present in master, I would like to understand why subshell_prompt is NULL.
I can't reproduce this bug on Linux under screen/term and csh.
You patch could help but it doesn't fix the problem. I think, the root of problem is in subshell initialization.

[support.od.ua]

Replying to andrew_b:

There was a fix of subshell related bug (#4182) after .26 release. It might fix this bug also. Could you please try the recent master?

Okay.
I'll take the code from the master branch today or tomorrow.
I will try to reproduce the problem.
And only then I will apply my patch and test it.

P.S. FreeBSD 12.2 rewrote vt (vt - virtual terminal console driver)

[support.od.ua]

A similar situation.

# env LC_CTYPE=C LC_ALL=C date
Sat Feb 27 21:56:39 EET 2021
# env LC_CTYPE=C LC_ALL=C ls -l /usr/local/bin/mc
-r-xr-xr-x  1 root  wheel  3162616 Feb 25 18:46 /usr/local/bin/mc

Backtrace full is:

(gdb) bt full
#0  0x00000000002c85e6 in str_utf8_make_make_term_form (text=0x0, length=18446744073709551615) at strutilutf8.c:454
        result = {
          text = "\000 \000valid)\000  2021\000\061\000\000\273ь\000...\000\200\000\271ки\000\210...\000\201\000\200ования файлов", '\000' <repeats 3002 times>, width = 0, compose = 0}
        uni = 0
        left = 0
        actual = 0x34df10 <str_utf8_make_make_term_form.result> ""
#1  0x00000000002c773d in str_utf8_term_width2 (text=0x0, length=18446744073709551615) at strutilutf8.c:802
        result = 0x0
#2  0x00000000002c777c in str_utf8_term_width1 (text=0x0) at strutilutf8.c:811
No locals.
#3  0x00000000002c1e99 in str_term_width1 (text=0x0) at strutil.c:654
No locals.
#4  0x000000000029c224 in setup_cmdline () at layout.c:978
        mw = 0x801a94c00
        prompt_width = 174
        y = 1
        tmp_prompt = 0x0
#5  0x000000000029b249 in setup_panels () at layout.c:889
        mw = 0x801a94c00
        start_y = 1
#6  0x00000000002631ae in midnight_callback (w=0x801a94c00, sender=0x0, msg=MSG_INIT, parm=0, data=0x0) at filemanager.c:1498
        command = 0
#7  0x000000000025167a in send_message (w=0x801a94c00, sender=0x0, msg=MSG_INIT, parm=0, data=0x0) at ../../lib/widget/widget-common.h:243
        ret = MSG_NOT_HANDLED
#8  0x00000000002514fb in dlg_init (h=0x801a94c00) at dialog.c:503
        g = 0x801a94c00
        wh = 0x801a94c00
#9  0x0000000000251d35 in dlg_run (h=0x801a94c00) at dialog.c:574
No locals.
#10 0x000000000026307d in do_nc () at filemanager.c:1837
        ret = 8
#11 0x000000000023dc64 in main (argc=1, argv=0x7fffffffda58) at main.c:454
        mcerror = 0x0
        exit_code = 1

# env LC_MESSAGES=C mc -V
GNU Midnight Commander 4.8.26-27-gb6de6e8f2
Built with GLib 2.66.7
Built with S-Lang 2.3.2 with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems:
 cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish
Data types:
 char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

# env LC_MESSAGES=C mc -F
Home directory: /root
Profile root directory: /root

[System data]
    Config directory: /usr/local/etc/mc/
    Data directory:   /usr/local/share/mc/
    File extension handlers: /usr/local/libexec/mc/ext.d/
    VFS plugins and scripts: /usr/local/libexec/mc/
        extfs.d:        /usr/local/libexec/mc/extfs.d/
        fish:           /usr/local/libexec/mc/fish/

[User data]
    Config directory: /root/.config/mc/
    Data directory:   /root/.local/share/mc/
        skins:          /root/.local/share/mc/skins/
        extfs.d:        /root/.local/share/mc/extfs.d/
        fish:           /root/.local/share/mc/fish/
        mcedit macros:  /root/.local/share/mc/mc.macros
        mcedit external macros: /root/.local/share/mc/mcedit/macros.d/macro.*
    Cache directory:  /root/.cache/mc/

# env LC_MESSAGES=C mc --configure-options
 '--with-internal-edit' '--enable-charset' '--enable-nls' '--enable-vfs-sftp' '--with-screen=slang' '--with-slang-includes=/usr/local/include' '--disable-vfs-smb' '--without-smb-configdir' '--without-smb-codepagedir' '--with-subshell' '--disable-x' '--prefix=/usr/local' '--localstatedir=/var' '--mandir=/usr/local/man' '--disable-silent-rules' '--infodir=/usr/local/share/info/' '--build=amd64-portbld-freebsd12.2' 'build_alias=amd64-portbld-freebsd12.2' 'CC=cc' 'CFLAGS=-pipe  -DLIBICONV_PLUG -g -fstack-protector-strong -fno-strict-aliasing ' 'LDFLAGS= -L/usr/local/lib -fstack-protector-strong ' 'LIBS=' 'CPPFLAGS=-I/usr/local/include -DLIBICONV_PLUG' 'CPP=cpp' 'PKG_CONFIG=pkgconf'

[support.od.ua]

Upd.
Do not pay attention to the date of the binary.
When installing a package from ports, an archive date is set on the application.
And I created the archive before the time was updated by ntpd.

[andrew_b]

I installed FreeBSD 12.2 under VirualBox? and built the resent master.

In the read_subshell_prompt() (src/subshell/common.c)

   1545     while (subshell_alive
   1546            && (rc = select (mc_global.tty.subshell_pty + 1, &tmp, NULL, NULL, &timeleft)) != 0)

select() returns 0. Therefore subshell_prompt is NULL. NULL dereference is the cause of MC crash.

Playing with timeout gets different results: { 0, 500 } and less gets stable crash, { 0, 700 } gets random crash/success, { 0, 1000 } gets stable success. In any case, playing with timeout is not our way.

Please test the attached 4213_cmdline.patch. In my case it fixes this bug.

[andrew_b]

4213_cmdline.2.patch​ is correct fix.

[andrew_b]

@support.od.ua, please let me know if this patch works or not.

[support.od.ua]

Replying to andrew_b:

@support.od.ua, please let me know if this patch works or not.

I will wait for a more correct solution to the bug.

[zyxmon]

I have added these patches to Entware build and mc started OK without segfault. One of libreelec users reports that these patched version freezes when changing directories, but it works OK with -u flag.

[support.od.ua]

Replying to zyxmon:

I have added these patches to Entware build and mc started OK without segfault. One of libreelec users reports that these patched version freezes when changing directories, but it works OK with -u flag.

The -u flag disables reproduction of the subshell problem.

[andrew_b]

Ticket #4236 has been marked as a duplicate of this ticket.

[andrew_b]

Branch: 4213_freebsd_crash_workaround
changeset:7e9e75307ecc84204f94d5275aa54fdf45e89c81

This isn't a real fix but just a workaround.

[ossi]

it would be better if the commit message clearly stated that the select times out, and said what exactly it was waiting for. it's not really necessary to quote the code when its location and function is described.

we currently have at least two other bugs related to slow shell startup.
and to make it even more fun, i had a random several-second hang related to command buffer synchronization on my vanilla debian unstable. this strongly suggests a race condition.
i can't help but think that at least one of the issues is probably somehow related.

[andrew_b]

Mergrd to master: [77545fbb669492898bec842153a71511d47de68c].

[andrew_b]

The ticket isn't closed because the real fix is required.

[andrew_b]

changeset:443a8106f1af55be2c995af10c4da80da430c2b1 makes the command line inaccessible if mc runs with disabled subshell (mc -u).
changeset:0e6721c32b3a34ffe0ba7d1456b5400ff8b8aeec fixes the command line.

[zaytsev]

BSD people have fixed the unneeded nox port, but not sure what's up with the subshell. Nobody is complaining.