Subshell returns to panels view on key press (any key).

[pgpb]

1 How to reproduce
══════════════════

1. `mc -b': launch MC
2. `ctrl-o': hide panels (enter subshell)
3. `ls': list directory contents

Expected:

To see the contents of the current directory

Actual:

MC returns to the panel view when pressing any key, i.e., it's not
possible to even finish typing the command `ls'.

2 Homebrew formula info
═══════════════════════

I installed `mc' using Homebrew, however the bug is present also when building

MC from sources using the versions given in this ticket:

$ brew info mc midnight-commander: stable 4.8.27
(bottled), HEAD Terminal-based visual file manager
<​https://www.midnight-commander.org/>

From:
<​https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/midnight-commander.rb>

==> Dependencies Build: pkg-config ✔ Required: glib ✔,
libssh2 ✔, openssl@1.1 ✔, s-lang ✔

Which downloads v4.8.27 via: the following URL:
<​https://www.midnight-commander.org/downloads/mc-4.8.27.tar.xz>

3 `mc' info
═══════════

┌────
│ LC_MESSAGES=C mc -V
└────

┌────
│ GNU Midnight Commander 4.8.27
│ Built with GLib 2.70.0
│ Built with S-Lang 2.3.2 with terminfo database
│ With builtin Editor
│ With subshell support as default
│ With support for background operations
│ With mouse support on xterm
│ With internationalization support
│ With multiple codepages support
│ Virtual File Systems:
│ cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish
│ Data types:
│ char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;
└────

┌────
│ LC_MESSAGES=C mc -F
└────

┌────
│ Home directory: /Users/user1
│ Profile root directory: /Users/user1
│
│ [System data]
│ Config directory: /usr/local/Cellar/midnight-commander/4.8.27/etc/mc/
│ Data directory: /usr/local/Cellar/midnight-commander/4.8.27/share/mc/
│ File extension handlers: /usr/local/Cellar/midnight-commander/4.8.27/libexec/mc/ext.d/
│ VFS plugins and scripts: /usr/local/Cellar/midnight-commander/4.8.27/libexec/mc/
│ extfs.d: /usr/local/Cellar/midnight-commander/4.8.27/libexec/mc/extfs.d/
│ fish: /usr/local/Cellar/midnight-commander/4.8.27/libexec/mc/fish/
│
│ [User data]
│ Config directory: /Users/user1/.config/mc/
│ Data directory: /Users/user1/.local/share/mc/
│ skins: /Users/user1/.local/share/mc/skins/
│ extfs.d: /Users/user1/.local/share/mc/extfs.d/
│ fish: /Users/user1/.local/share/mc/fish/
│ mcedit macros: /Users/user1/.local/share/mc/mc.macros
│ mcedit external macros: /Users/user1/.local/share/mc/mcedit/macros.d/macro.*
│ Cache directory: /Users/user1/.cache/mc/
└────

4 `git bisect' results
══════════════════════

Using `git bisect' I found the following:

• good version: 4.8.1.7
• first bad commit found: 109af564bce7e86a0421680ede308be847339861

5 Currently using /good/ version: `4.8.1-stable'
════════════════════════════════════════════════

I'm currently using MC built from sources using branch:
`4.8.1-stable', which does not have the bug described in this ticket.

I will also report this issue to Homebrew, however since the issue
appears when building MC from sources I believe it also deserves a
ticket here.

[pgpb]

Github issue tracker for the Homebrew formula: ​https://github.com/Homebrew/homebrew-core/issues/97718

[ossi]

it makes no sense that the commit you identified would cause the symptoms you describe. your re-builds are probably not clean enough (automake-generated makefiles have imperfect dependencies). make sure to always start from scratch (and use ccache to save some energy).

what you're describing sounds like mc didn't start a subshell at all, maybe because it misidentified a dumb terminal. have a look at the output of ps fx.

[zaytsev]

Does it really have to do with -b or it happens always? Sounds like another case, where some weird zsh plugins prevent subshell startup or something. Try to change the shell to bash to make sure that it's a zsh problem.

There is a patch to allegedly solve the zsh problem here:

https://midnight-commander.org/ticket/3121

Unfortunately, no zsh users tried it and the author can't explain it.

[pgpb]

Replying to ossi:

it makes no sense that the commit you identified would cause the symptoms you describe. your re-builds are probably not clean enough (automake-generated makefiles have imperfect dependencies). make sure to always start from scratch (and use ccache to save some energy).

Sorry I'm no C expert and also not familiar with the code base, however, I also thought that the /bad/ commit seems completely unrelated to the bug.

During the =git bisect= session, I'm using this command to rebuild, which I thought would be enough to ensure each build is a clean one:

git clean -xdf && autoreconf -i && ./autogen.sh && ./configure && make

what you're describing sounds like mc didn't start a subshell at all, maybe because it misidentified a dumb terminal. have a look at the output of ps fx.

The output of {{{ps fx}} is rather long but there are many instances of bash, like this:

  ps -fx | grep bash
  501 60523 28859   0 Mo06pm  ttys000    0:00.76 -bash
  501 28860 28859   0 So09pm  ttys002    0:00.56 -bash
  501 95240     1   0 Sa11am  ttys003    0:00.33 bash -rcfile .bashrc
  501 76817 28859   0 Mi09pm  ttys004    0:00.38 -bash
  501 99175     1   0 Sa11am  ttys005    0:00.33 bash -rcfile .bashrc
  ... many similar lines
  501 11245     1   0 Di08pm  ttys040    0:00.34 bash -rcfile .bashrc
  501 13813 28859   0 Di08pm  ttys041    0:13.64 -bash
  501 39846 39845   0 10:23am ttys049    0:00.43 bash -rcfile .bashrc
  501 40651 28859   0 10:24am ttys050    0:00.37 -bash
  501 41643 40651   0 10:26am ttys050    0:00.01 grep bash

[zaytsev]

This is weird, it looks like you are using bash instead of zsh on macOS? Have you tried it on a clean user, the next step is to understand whether it works "normally" or it has to do with your configuration.

[pgpb]

Replying to zaytsev:

Does it really have to do with -b or it happens always?

I'm working with bash. The -b is there only because the default MC theme is a bit hard on my eyes.

[zaytsev]

So does it happen on a clean user, or it has to do something with your bash configuration?

[pgpb]

Replying to zaytsev:

This is weird, it looks like you are using bash instead of zsh on macOS?

Yes, I use bash as my default shell. I never switched to zsh.

Have you tried it on a clean user, the next step is to understand whether it works "normally" or it has to do with your configuration.

Could you elaborate, what do you mean a clean user? Something like create a new account and install from there?

Btw, when I build MC from sources using the git tag 4.8.1.7 the problem goes away. I was hoping that would mean it's not something to do with my configuration.

Here's a short version of how I tested during the =git bisect= session:

1. git checkout master - clean checkout
2. git bisect start - start a bisect session
3. mark good and bad versions:

a) git bisect bad - mark master as bad
b) git checkout 4.8.1.7 and git bisect good - mark tag 4.8.1.7 as a good version

4. git clean -xdf && autoreconf -i && ./autogen.sh && ./configure && make - build from sources, *do not install*.
5. ./src/mc -V - verify version
6. ./src/mc -b - run built binary -- not yet installed -- forcing black and white display
7. test if the subshell (ctrl-o) is working by running ls
8. when subshell does not work, mark commit as bad with git bisect bad, otherwise mark commit as good with git bisect good
9. git then selects a new version to test
10. go to step 3. clean build, repeat until the bisect session is over

[andrew_b]

Replying to pgpb:

Btw, when I build MC from sources using the git tag 4.8.1.7 the problem goes away.

4.8.x is more than 9 years old. Forget it. There are a tones of changes in the mc sources since that time.

[pgpb]

Yeah, I feel you. I now have a workaround (a self-published ​Homebrew tap) so I don't really mind if you close this ticket.

[zaytsev]

Could you elaborate, what do you mean a clean user? Something like create a new account and install from there?

Yes, this is the easiest and quickest way to test, whether it has something to do with your local configuration or is generally broken on your system, so the breakage does not depend on your local settings. For example it might have something to do with your shell prompt or bash initialisation scripts.

[pgpb]

So, I just checked using a clean `bash' session, i.e., no customizations
and I am still able to reproduce the same bug. Here are the detailed
steps:

1. delete all installations of `mc'

2. start a new `bash' session explicitly not loading my /profile/ or /rc/ files:

┌────
│ bash --noprofile --norc
└────

3. Install Formula from `homebrew-core'

┌────
│ brew install mc
│
│ # verify version
│ LC_MESSAGES=C mc -V
│ GNU Midnight Commander 4.8.27
│ Built with GLib 2.70.0
│ Built with S-Lang 2.3.2 with terminfo database
│ With builtin Editor
│ With subshell support as default
│ With support for background operations
│ With mouse support on xterm
│ With internationalization support
│ With multiple codepages support
│ Virtual File Systems:
│ cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish
│ Data types:
│ char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;
└────

4. Test if the bug still exists by following the original steps in this ticket. The bug is still reproducible wether I invoke mc or `mc -b`.

[ossi]

that ps output isn't what i asked for, as you didn't follow the instructions adequately - i really did mean fx without a leading dash. specifically, the f means "forest", aka tree; the x is actually unnecessary. and don't grep, unless you add -C1.

anyway, the bash instances invoked with -rcfile .bashrc do indeed look like mc subshells. check that all mc instances have a subshell, and that they behave the same way.

your build cleaning procedure looks as thorough as it gets.
are you using ccache (or something like that)? if that was buggy in some way, this might explain things.

one hypothesis would be that bisect is getting confused by merges, but i don't see anything particularly complex in the vicinity of the named commit.

maybe you messed up with 'good' and 'bad'?

did you do all tests as a regular user? is there a difference when you launch mc as root?

try comparing the strace (and ltrace) output of the startup of a good and a bad build.

[zaytsev]

Your test without creating a fresh user doesn't make any sense. There is often a reason why we ask to do certain things in a certain way. While launching the subshell, mc will pick up any customisations even if you have launched it from a shell, which itself was clean.

[pgpb]

Replying to zaytsev:

Your test without creating a fresh user doesn't make any sense. There is often a reason why we ask to do certain things in a certain way. While launching the subshell, mc will pick up any customisations even if you have launched it from a shell, which itself was clean.

Ok, I just had time to test this. I misunderstood how the subshell works. I have created new user with a pristine Bash config. The issue is gone.

1. Created a new user mc-tester using the Users & Groups panel in System Preferences
2. su -l mc-tester

I guess I have to spend a bit more time to find what part of my Bash config is causing trouble.

[zaytsev]

Alright, thanks for the confirmation. The problem could be related to your custom prompt, or PRECMD hooks, or pretty much anything, really. Unfortunately, we can only guarantee that the subshell works with the pristine shell.

If you have your own very special customisations, then you have to make sure that they are compatible with our way of hooking into your shell, because there is unfortunately no stable and reliable API that the shells provide to drive them as embedded subshells.

You can check in mc source how we hook into different shells and make sure the prompt is synchronised with mc...

[andrew_b]

Replying to zaytsev:

there is unfortunately no stable and reliable API that the shells provide to drive them as embedded subshells.

I'd say that there is no any API at all. All good or bad working interactions with shells are hacks.