Subshell/Command line prompt is empty/missing

[z0rc]

See attached screenshot. The issue persist with TERM=(xterm|screen).*. Tested with konsole and xterm both on current Debian Sid, Ubuntu 12.04 and Ubuntu 13.10. Tested with bash and zsh. The issue doesn't exists with 'linux' terminal (without xorg, plain console at Ctrl+Alt+F1)

mc -V
GNU Midnight Commander 4.8.11
Built with GLib 2.36.4
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ext2undelfs, ftpfs, sftpfs, fish
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

I bisected issue to:

e35f044ccdd41922f925c99e6d50930ea8c7c47e is the first bad commit
commit e35f044ccdd41922f925c99e6d50930ea8c7c47e
Author: Andrew Borodin <aborodin@vmail.ru>
Date:   Tue Jan 1 19:53:11 2013 +0400

    (subshell_prompt): changed to GString.
    
    (read_subshell_prompt): refactoring to ret rid of low-level memory reallocation.
    
    Signed-off-by: Andrew Borodin <aborodin@vmail.ru>

[andrew_b]

Sorry, but I'm unable reproduce this bug.

[andrew_b]

Ok, let step by step.

Is it happened immediately after start or after any actions?
What is your $PS1?
Does your $PS1 depend on your $TERM value or any other condition?

[z0rc]

OK, I've made additional tests. I was wrong about bash, it happens only with zsh. $PS1 is irrelevant as see this behavior with empty zshrc. Though it happens, but not so often. Usually this happens with mc start and the prompt may appear on dir change ...or may not. From my perspective it's more like race condition somewhere and the heavier my zshrc, the often it happens.

Just in case you can check my zshrc at ​https://github.com/z0rc/dotfiles/blob/master/zshrc.

I'll try to dig into gdb later this week.

[egmont]

I can't reproduce, just looking at the code, but line 1022: g_string_set_size (subshell_prompt, 0) is fishy, it's inside the while(select(...)) loop that takes care of assembling multiple short reads, but I guess it should be before that loop.

[andrew_b]

Replying to egmont:

I can't reproduce, just looking at the code, but line 1022: g_string_set_size (subshell_prompt, 0) is fishy, it's inside the while(select(...)) loop that takes care of assembling multiple short reads, but I guess it should be before that loop.

This was fixed in #3001.

[z0rc]

I'm continuing my investigation. Current situation:

Breakpoint 3, setup_cmdline () at layout.c:816
816     {
(gdb) bt full
#0  setup_cmdline () at layout.c:816
        prompt_len = <optimized out>
        y = <optimized out>
        tmp_prompt = <optimized out>
#1  0x0000000000436752 in do_load_prompt () at layout.c:1328
        ret = <optimized out>
#2  0x0000000000436779 in load_prompt (fd=<optimized out>, unused=<optimized out>) at layout.c:1350
No locals.
#3  0x00000000004334b0 in check_selects (select_set=select_set@entry=0x7fffffffcf40) at key.c:592
        p = 0x7ff1a0
#4  0x0000000000434a78 in check_selects (select_set=0x7fffffffcf40) at key.c:559
No locals.
#5  tty_get_event (event=event@entry=0x7fffffffd000, redo_event=0, block=block@entry=1) at key.c:2069
        nfd = <optimized out>
        select_set = {fds_bits = {0 <repeats 16 times>}}
        c = <optimized out>
        flag = 1
        time_out = {tv_sec = 8375264, tv_usec = 0}
        time_addr = <optimized out>
        dirty = 1
#6  0x000000000041a417 in frontend_dlg_run (h=0x7fb640) at dialog.c:567
        d_key = <optimized out>
        event = {buttons = 0, x = -1, y = 13, type = (GPM_UP | GPM_DOUBLE)}
#7  dlg_run (h=0x7fb640) at dialog.c:1256
No locals.
#8  0x000000000043bdf5 in create_panels_and_run_mc () at midnight.c:959
No locals.
#9  do_nc () at midnight.c:1774
        ret = <optimized out>
        midnight_colors = {1, 1, 1, 1, 1}
#10 0x000000000040a065 in main (argc=1, argv=0x7fffffffd288) at main.c:400
        error = 0x0
        config_migrated = 0
        config_migrate_msg = 0x7ffff7ffe5c0 " \345\377\367\377\177"
        exit_code = 1
(gdb) print subshell_prompt->str
$23 = (gchar *) 0x823c60 "\033[0m\033[27m\033[24m\033[J[mc][\033[01;33mkoumakan\033[00m][\033[01;32m~/rebuild\033[00m]% \033[K"
(gdb) cont
Continuing.

Breakpoint 3, setup_cmdline () at layout.c:816
816     {
(gdb) bt full
#0  setup_cmdline () at layout.c:816
        prompt_len = <optimized out>
        y = <optimized out>
        tmp_prompt = <optimized out>
#1  0x0000000000436752 in do_load_prompt () at layout.c:1328
        ret = <optimized out>
#2  0x0000000000436779 in load_prompt (fd=<optimized out>, unused=<optimized out>) at layout.c:1350
No locals.
#3  0x00000000004334b0 in check_selects (select_set=select_set@entry=0x7fffffffcf40) at key.c:592
        p = 0x7ff1a0
#4  0x0000000000434a78 in check_selects (select_set=0x7fffffffcf40) at key.c:559
No locals.
#5  tty_get_event (event=event@entry=0x7fffffffd000, redo_event=0, block=block@entry=1) at key.c:2069
        nfd = <optimized out>
        select_set = {fds_bits = {0 <repeats 16 times>}}
        c = <optimized out>
        flag = 1
        time_out = {tv_sec = 8375264, tv_usec = 0}
        time_addr = <optimized out>
        dirty = 1
#6  0x000000000041a417 in frontend_dlg_run (h=0x7fb640) at dialog.c:567
        d_key = <optimized out>
        event = {buttons = 0, x = -1, y = 13, type = (GPM_UP | GPM_DOUBLE)}
#7  dlg_run (h=0x7fb640) at dialog.c:1256
No locals.
#8  0x000000000043bdf5 in create_panels_and_run_mc () at midnight.c:959
No locals.
#9  do_nc () at midnight.c:1774
        ret = <optimized out>
        midnight_colors = {1, 1, 1, 1, 1}
#10 0x000000000040a065 in main (argc=1, argv=0x7fffffffd288) at main.c:400
        error = 0x0
        config_migrated = 0
        config_migrate_msg = 0x7ffff7ffe5c0 " \345\377\367\377\177"
        exit_code = 1
(gdb) print subshell_prompt->str
$24 = (gchar *) 0x802ce0 "\033[?1h\033="

This is strange as at directory change we enter setup_cmdline two times, first enter with correct prompt, second with bogus. At second break I can see the valid prompt in mc, which later gets changed to nothing. I'll continue to dig this up. If you have any hints, please share.

[andrew_b]

I think you should check subshell_prompt in read_subshell_prompt().

[z0rc]

Please close this as invalid. It appears my problem after all. Though it wasn't obvious to spot. Basically zsh has two prompts: left and right, mc was interpreting both of them. I had an option to set RPROMPT to be "" (empty string) if zsh is running under mc, it was working just fine. Right now I have to completely undefine RPROMPT, so mc won't interpret it.

[z0rc]

I spoke to soon and reopening this ticket. Sorry.

RPROMPT has nothing to do with this as it won't affect the situation, as I though initially. "\033[?1h\033=" is present always, event when RPROMPT isn't set. This looks like mark of prompt end, as they present event with empty PROMPT.

I still think this is some kind of race condition. I can catch empty prompt comes after valid when setting up just watch on subshell_prompt variable. But when I break on read_subshell_prompt the empty shell won't appear and issue won't show up. Also I cannot catch issue under strace, then mc behaves just fine.

[andrew_b]

(In #3125) Merged to master: [75073307b9f8d500e4b04318f7a78e5b5bf2b884].

[xor512]

/etc/skel/.zshrc from manjaro-zsh-config package

[xor512]

no prompt (happens after typing ls -> enter several times)

[xor512]

.zshrc from manjaro forum which probably do not depend on manjaro-zsh-config package and can be probably used on plain Arch

[xor512]

This still happens with config from manjaro-zsh-config package in 5.4.15-2-MANJARO. This is the standard config from manjaro-zsh-config and can be found in /etc/skel/.zshrc after installing manjaro-zsh-config package.

I've created a topic on a forum: ​https://forum.manjaro.org/t/prompt-disappears-in-zsh-mc-after-typing-comands-sometimes/122552 but I think this may be also an issue with mc vs zsh, which is to be fixed in mc.

To reproduce the issue one should use the config attached https://midnight-commander.org/attachment/ticket/3121/.zshrc (I'm not sure it will work without the full manjaro-zsh-package though and that manjaro-zsh-package can be installed on plain Arch, however the issue is also reproducible with the following config: ​https://forum.manjaro.org/t/unstable-manjaro-zsh-config/17899/105, it is attached as https://midnight-commander.org/attachment/ticket/3121/.zshrc_arch) and then type ls -> enter several times. This does not always happen but often enough (1 times out of 10 at least). It does not happen without .zshrc file at all.

I'm using xfce4-terminal, but this also happens in xterm (see https://midnight-commander.org/attachment/ticket/3121/prompt_sometimes_disappears.png).

[kvaster]

I'm using zsh with mc and I have disabled rprompt for mc. But I still have empty prompt sometimes. It seems that zsh adds empty line from time to time. There is already check in mc in set_prompt_string function, but zsh adds empty line with escape sequences. I've created simple UGLY patch and now prompt works as expected with zsh and git goodies:

diff --git a/src/subshell/common.c b/src/subshell/common.c
index 65f5718..5b5162f 100644
--- a/src/subshell/common.c
+++ b/src/subshell/common.c
@@ -724,10 +724,30 @@ parse_subshell_prompt_string (const char *buffer, int bytes)
 static void
 set_prompt_string (void)
 {
+    int m, i;
+
     if (mc_global.mc_run_mode != MC_RUN_FULL)
         return;

-    if (subshell_prompt_temp_buffer->len != 0)
+    m = 0;
+    for (i = 0; i < subshell_prompt_temp_buffer->len; i++) {
+        char c = subshell_prompt_temp_buffer->str[i];
+        if (c == 27) {
+            m = 2;
+        } else if (m == 0) {
+            m = 1;
+            break;
+        } else if (m == 2) {
+            m = c == '[' ? 3 : 0;
+        } else if (m == 3) {
+            m = c == '?' ? 4 : 0;
+        } else if (m == 4) {
+            if (c < '0' || c > '9')
+                m = 0;
+        }
+    }
+
+    if (m == 1)
         g_string_assign (subshell_prompt, subshell_prompt_temp_buffer->str);

     setup_cmdline ();

[zaytsev]

Could this be the solution for the zsh problem in #4198 ?

[andrew_b]

Replying to kvaster:

I've created simple UGLY patch and now prompt works as expected with zsh and git goodies:

It would be great if you provide more or less detailed description about your code.

[andrew_b]

It seems to me that patch in comment:17 does the same as function strip_ctrl_codes().

[andrew_b]

Related to #4258.

[newsboost]

Hi all. I'll just add that I can confirm I've also just experienced this issue and then add a simpler workaround that doesn't require any compiling of source code (alternative and easier work-around than https://midnight-commander.org/ticket/3121#comment:17):

First my system:

Fully up-to-date Arch Linux running with Midnight Commander 4.8.28 and I also use zsh. Interestingly enough I actually have 2 laptops with exactly the same configuration (~/.zshrc and a sourced config-file common to zsh and bash). But this CTRL+O subshell issue only happened on one of the pc's (don't know why, it's very strange because both runs uptodate Arch Linux, so the difference should mostly be hardware and maybe minor differences in installed packages although most is the same).

"Work-around solution":

I ended up with first disabling the last command in my startup zsh-config: "neofetch" - later I changed it to this:

[[ $(realpath /proc/$PPID/exe) == */bin/mc ]] || neofetch

which effectively prevents "neofetch" from running when the CTRL+O starts a subshell, which was the culprit on my system.

Concluding remarks:
I did not try to compile anything but hope this bug will be fixed and until that, I think this is a really good and easy work-around for me, which I hope also can help other who experiences this issue. You can read more details here, because the work-around wasn't actually my idea but I posted the issue in the Arch Linux forum (​https://bbs.archlinux.org/viewtopic.php?pid=2053275#p2053275) and now I hope sharing this info will help fix the bug upstream and properly so the workaround isn't needed in the future. Please let me know if this helps other with the same issue so we maybe can find the culprit, if the issue remains without neofetch on some systems, which I suspect and I'll be happy to help, if there are questions.

UPDATE: The above workaround does not always work - this works (on my system):
hmm, I just downloaded mc-4.8.28.tar.xz (am new user here, couldn't see how to get the git-repo, but this seems ok) and compiled it and tried to reproduce this and see if I could see/learn more. But it starts up with "Error. Unable to load 'default' skin. Default skin has been loaded" (which I think is contradictory?). After pressing ESC+enter a few times it opens up in black/white - I then hit CTRL+O, but then the issue persists and the subshell-workaround above doesn't help (it does however write: "zsh:1: permission denied: ..", which is more helpful than the mc from my package manager). However, what *DOES* work is to:

mv -i .zshrc .zshrc__

which is also how I found the above-mentioned work-around (by "bisecting", i.e. commenting things out until I could see what did the difference).

UPDATE 2: The comment no.17-workaround does not work on my system

I just tried applying the patch described at https://midnight-commander.org/ticket/3121#comment:17 and it made a difference: Now I got colors instead of black/white and furthermore I don't get any annoying "Error. Unable to load 'default' skin"-startup error. So my expectations were high, but unfortunately this patch does not fix my CTRL+O (subshell) issue... hmm. I understand this issue is difficult to track down as many people cannot reproduce it. If you have any ideas for what I could try, to get a better understanding, please write, I would be happy to help as I'm a long-time user of "mc" and with it should be bugfree...

UPDATE 3: Preliminary learnings with GDB

I've currently applied the comment no17-patch and just learned from searching a bit that init_subshell()-should be a breakpoint-place but also execute.c:491 and 501. I attached to the PID of the compiled mc (incl patch-17) and hit CTRL+O and now I can see what happens:

• In line 491 of execute.c, it skips the if-block because mc_global.tty.use_subshell=0.
• Next it goes to line 501 of execute.c, where the variable "outputs_starts_shell" is 0, so it ends up in the "else"-case where it just runs "get_key_code(0)", instead of running the fputs and my_system-stuff (lines 503-506).
• The minute I step on line 509 gdb waits for me to press a key.
• As soon as I press any key, we're at line 512 (this is exactly the issue I'm seeing and others have reported, CTRL+O subshell doesn't work, as soon as we press a key "mc" goes back to it's dual-split-view).
• At this time things are just wrong so debugging further from here will not help me.

I would need to understand better why the situation above has happened, this will not be today, I'm a bit tired now. Anyway, if I should test anything or if you have ideas, please write (I hope I'll receive an email notification?) and I'll try it out so this weird zsh-issue can be fixed, I hope this helps...

[tonal]

Do not work SubShell? for me. :(
My Os - KDE Neon.
After upgrade base ubuntu from 20.04 to 22.04 mc subshell do not working.

My environment:

$ mc -V
GNU Midnight Commander, версия 4.8.28
Скомпилирован с библиотекой GLib версии 2.72.1
С библиотекой S-Lang 2.3.2 и с базой данных terminfo
Скомпилирован с библиотекой libssh2 версии 1.10.0
Со встроенным редактором и поддержкой Aspell
C поддержкой внутренней командной оболочки
С поддержкой фоновых операций
С поддержкой мыши в xterm и консоли Linux
С поддержкой событий X11
С поддержкой интернационализации
С поддержкой многих кодировок
With ext2fs attributes support
Виртуальная файловая система:
 cpiofs, tarfs, sfs, extfs, ext2undelfs, ftpfs, sftpfs, fish
Тип данных:
 char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

$ neofetch
OS: KDE neon 5.26 x86_64 
Host: TravelMate P259-MG V1.51 
Kernel: 5.15.0-1021-oracle 
Packages: 7542 (dpkg), 44 (nix-user), 10 (flatpak), 20 (snap) 
Shell: bash 5.1.16 
Resolution: 1366x768 
DE: Plasma 5.26.2 
WM: KWin 
Theme: Breeze Light [Plasma], Breeze [GTK3] 
Terminal: konsole 
CPU: Intel i5-6200U (4) @ 2.800GHz 
GPU: Intel Skylake GT2 [HD Graphics 520] 
GPU: NVIDIA GeForce 940MX 
Memory: 8579MiB / 31975MiB 

[ce72]

Same problem exists in cygwin on official Windows Terminal Console using the official cygwin package (mc 4.8.27 with subshell support):

• Ctrl-o shows the screen without prompt. When I enter text it jumps back to the mc screen after the first char.
• The aliases from .bashrc (and .zshrc either) are missing in the mc input line (tested both bash and zsh)
• each invocation of mc leaves a zombie proecess zsh.exe (or bash.exe) in task manager

When I start from there with mintty mc then everything is working fine (ctrl-o opens a screen, I can edit there and the rc file has been sourced successfully.

Apparently mc has issues with some terminal clients?

[tonal]

Debugging revealed that when the subshell is initialized, the feed_subshell function returns FALSE.
This happens when initialization takes longer than 1 second.

To fix this, you can add an additional parameter to the feed_subshell function indicating that it is initializing and needs to wait longer.
Or to allow you to specify the timeout in the settings.

[antonio_so]

tonal is correct.
The culprit is here: ​https://github.com/MidnightCommander/mc/blob/3250536c157b77e26794f436976fc9d2df514d28/src/subshell/common.c#L752

If your profile loads slower than 1 second - you have no subshell.
I've changed wtime.tv_sec to 10 seconds, built mc locally and I have no problems since.

[andrew_b]

Branch: 3121_empty_subshell_prompt
changeset:1ac5c22517ed569817ce05ff6e29c7fba3da200a

[andrew_b]

Merged to master: [ebb011a0472ba4112d58bdf960cdbd6fc21cf878].

[wentasah]

The problem still exists in 4.8.29. I think I know why and have a patch with a reliable fix. From my commit message:

When using zsh with ​https://starship.rs prompt generator, MC sometimes fails to show the subshell prompt. This is not deterministic. Sometimes the prompt is shown and sometimes it isn't.

The reason is that the shell prints the prompt in multiple chunks. The first chunk contains the "real" prompt and the second is an escape sequence for enabling bracketed paste mode. If both chunks are read by MC in a single invocation of read_subshell_prompt(), the prompt is shown correctly. If, however, read_subshell_prompt() reads each chunk in separate invocations (because the second chunk is not ready during the first invocation), the prompt is not shown. More precisely, only the bracketed paste mode escape sequence is shown as a prompt in MC.

This can be demonstrated with the following commands:

    export SHELL=$(which zsh)
    export ZDOTDIR=/tmp/zshdotdir
    export STARSHIP_CONFIG=/tmp/starship-test.toml
    mkdir -p "$ZDOTDIR"
    echo 'eval "$(starship init zsh)"' > "$ZDOTDIR/.zshrc"
    echo 'format = "XXXX: $directory$character"' > "$STARSHIP_CONFIG"
    mc

In my case, the prompt is usualy shown after mc start and it disappears after changing a directory in mc. In that case, the prompt is read() in the following two chunks:

• 63 bytes: \xd\x1b[0m\x1b[27m\x1b[24m\x1b[J\xd\xaXXXX: \x1b[1;36mmc/.git\x1b[0m \x1b[1;32m\xe2\x9d\xaf\x1b[0m \x1b[K
• 8 bytes: \x1b[?2004h

To fix the problem, we remove clearing of the prompt string in read_subshell_prompt(). It is sufficient that the prompt is cleared when receiving '\n' and in feed_subshell().

I'll attach the patch here. Let me know whether this is sufficient of a Github PR is more appropriate these days.

[andrew_b]

Thanks!

Branch:3121_empty_subshell_prompt
changeset:02916466c318a996253c619971640b4b35fa2b31

[andrew_b]

Merged to master: [1d845d0dc53bc41a63ccabc7e1fc949e33963ded].