Ticket #3121 (closed defect: fixed)

Opened 9 years ago

Last modified 2 months ago

Subshell/Command line prompt is empty/missing

Reported by: z0rc Owned by: andrew_b
Priority: major Milestone: 4.8.29
Component: mc-core Version: 4.8.28
Keywords: subshell Cc: egmont@…, congest, aurrak, reagle, tonal.promsoft@…
Blocked By: Blocking:
Branch state: merged Votes for changeset: committed-master

Description (last modified by andrew_b) (diff)

See attached screenshot. The issue persist with TERM=(xterm|screen).*. Tested with konsole and xterm both on current Debian Sid, Ubuntu 12.04 and Ubuntu 13.10. Tested with bash and zsh. The issue doesn't exists with 'linux' terminal (without xorg, plain console at Ctrl+Alt+F1)

mc -V
GNU Midnight Commander 4.8.11
Built with GLib 2.36.4
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ext2undelfs, ftpfs, sftpfs, fish
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

I bisected issue to:

e35f044ccdd41922f925c99e6d50930ea8c7c47e is the first bad commit
commit e35f044ccdd41922f925c99e6d50930ea8c7c47e
Author: Andrew Borodin <aborodin@vmail.ru>
Date:   Tue Jan 1 19:53:11 2013 +0400

    (subshell_prompt): changed to GString.
    
    (read_subshell_prompt): refactoring to ret rid of low-level memory reallocation.
    
    Signed-off-by: Andrew Borodin <aborodin@vmail.ru>

Attachments

41.png (128.6 KB) - added by z0rc 9 years ago.
.zshrc (9.5 KB) - added by xor512 3 years ago.
/etc/skel/.zshrc from manjaro-zsh-config package
prompt_sometimes_disappears.png (18.2 KB) - added by xor512 3 years ago.
no prompt (happens after typing ls -> enter several times)
.zshrc_arch (11.0 KB) - added by xor512 3 years ago.
.zshrc from manjaro forum which probably do not depend on manjaro-zsh-config package and can be probably used on plain Arch

Change History

Changed 9 years ago by z0rc

comment:1 Changed 9 years ago by andrew_b

Sorry, but I'm unable reproduce this bug.

comment:2 Changed 9 years ago by andrew_b

  • Cc aborodin@… removed
  • Version changed from master to 4.8.11
  • Component changed from mc-tty to mc-core

comment:3 Changed 9 years ago by andrew_b

Ok, let step by step.

Is it happened immediately after start or after any actions?
What is your $PS1?
Does your $PS1 depend on your $TERM value or any other condition?

comment:4 Changed 9 years ago by z0rc

OK, I've made additional tests. I was wrong about bash, it happens only with zsh. $PS1 is irrelevant as see this behavior with empty zshrc. Though it happens, but not so often. Usually this happens with mc start and the prompt may appear on dir change ...or may not. From my perspective it's more like race condition somewhere and the heavier my zshrc, the often it happens.

Just in case you can check my zshrc at https://github.com/z0rc/dotfiles/blob/master/zshrc.

I'll try to dig into gdb later this week.

comment:5 follow-up: ↓ 7 Changed 9 years ago by egmont

I can't reproduce, just looking at the code, but line 1022: g_string_set_size (subshell_prompt, 0) is fishy, it's inside the while(select(...)) loop that takes care of assembling multiple short reads, but I guess it should be before that loop.

comment:6 Changed 9 years ago by egmont

  • Cc egmont@… added

comment:7 in reply to: ↑ 5 Changed 9 years ago by andrew_b

Replying to egmont:

I can't reproduce, just looking at the code, but line 1022: g_string_set_size (subshell_prompt, 0) is fishy, it's inside the while(select(...)) loop that takes care of assembling multiple short reads, but I guess it should be before that loop.

This was fixed in #3001.

comment:8 follow-up: ↓ 9 Changed 9 years ago by z0rc

I'm continuing my investigation. Current situation:

Breakpoint 3, setup_cmdline () at layout.c:816
816     {
(gdb) bt full
#0  setup_cmdline () at layout.c:816
        prompt_len = <optimized out>
        y = <optimized out>
        tmp_prompt = <optimized out>
#1  0x0000000000436752 in do_load_prompt () at layout.c:1328
        ret = <optimized out>
#2  0x0000000000436779 in load_prompt (fd=<optimized out>, unused=<optimized out>) at layout.c:1350
No locals.
#3  0x00000000004334b0 in check_selects (select_set=select_set@entry=0x7fffffffcf40) at key.c:592
        p = 0x7ff1a0
#4  0x0000000000434a78 in check_selects (select_set=0x7fffffffcf40) at key.c:559
No locals.
#5  tty_get_event (event=event@entry=0x7fffffffd000, redo_event=0, block=block@entry=1) at key.c:2069
        nfd = <optimized out>
        select_set = {fds_bits = {0 <repeats 16 times>}}
        c = <optimized out>
        flag = 1
        time_out = {tv_sec = 8375264, tv_usec = 0}
        time_addr = <optimized out>
        dirty = 1
#6  0x000000000041a417 in frontend_dlg_run (h=0x7fb640) at dialog.c:567
        d_key = <optimized out>
        event = {buttons = 0, x = -1, y = 13, type = (GPM_UP | GPM_DOUBLE)}
#7  dlg_run (h=0x7fb640) at dialog.c:1256
No locals.
#8  0x000000000043bdf5 in create_panels_and_run_mc () at midnight.c:959
No locals.
#9  do_nc () at midnight.c:1774
        ret = <optimized out>
        midnight_colors = {1, 1, 1, 1, 1}
#10 0x000000000040a065 in main (argc=1, argv=0x7fffffffd288) at main.c:400
        error = 0x0
        config_migrated = 0
        config_migrate_msg = 0x7ffff7ffe5c0 " \345\377\367\377\177"
        exit_code = 1
(gdb) print subshell_prompt->str
$23 = (gchar *) 0x823c60 "\033[0m\033[27m\033[24m\033[J[mc][\033[01;33mkoumakan\033[00m][\033[01;32m~/rebuild\033[00m]% \033[K"
(gdb) cont
Continuing.

Breakpoint 3, setup_cmdline () at layout.c:816
816     {
(gdb) bt full
#0  setup_cmdline () at layout.c:816
        prompt_len = <optimized out>
        y = <optimized out>
        tmp_prompt = <optimized out>
#1  0x0000000000436752 in do_load_prompt () at layout.c:1328
        ret = <optimized out>
#2  0x0000000000436779 in load_prompt (fd=<optimized out>, unused=<optimized out>) at layout.c:1350
No locals.
#3  0x00000000004334b0 in check_selects (select_set=select_set@entry=0x7fffffffcf40) at key.c:592
        p = 0x7ff1a0
#4  0x0000000000434a78 in check_selects (select_set=0x7fffffffcf40) at key.c:559
No locals.
#5  tty_get_event (event=event@entry=0x7fffffffd000, redo_event=0, block=block@entry=1) at key.c:2069
        nfd = <optimized out>
        select_set = {fds_bits = {0 <repeats 16 times>}}
        c = <optimized out>
        flag = 1
        time_out = {tv_sec = 8375264, tv_usec = 0}
        time_addr = <optimized out>
        dirty = 1
#6  0x000000000041a417 in frontend_dlg_run (h=0x7fb640) at dialog.c:567
        d_key = <optimized out>
        event = {buttons = 0, x = -1, y = 13, type = (GPM_UP | GPM_DOUBLE)}
#7  dlg_run (h=0x7fb640) at dialog.c:1256
No locals.
#8  0x000000000043bdf5 in create_panels_and_run_mc () at midnight.c:959
No locals.
#9  do_nc () at midnight.c:1774
        ret = <optimized out>
        midnight_colors = {1, 1, 1, 1, 1}
#10 0x000000000040a065 in main (argc=1, argv=0x7fffffffd288) at main.c:400
        error = 0x0
        config_migrated = 0
        config_migrate_msg = 0x7ffff7ffe5c0 " \345\377\367\377\177"
        exit_code = 1
(gdb) print subshell_prompt->str
$24 = (gchar *) 0x802ce0 "\033[?1h\033="

This is strange as at directory change we enter setup_cmdline two times, first enter with correct prompt, second with bogus. At second break I can see the valid prompt in mc, which later gets changed to nothing. I'll continue to dig this up. If you have any hints, please share.

comment:9 in reply to: ↑ 8 Changed 9 years ago by andrew_b

I think you should check subshell_prompt in read_subshell_prompt().

comment:10 Changed 9 years ago by z0rc

Please close this as invalid. It appears my problem after all. Though it wasn't obvious to spot. Basically zsh has two prompts: left and right, mc was interpreting both of them. I had an option to set RPROMPT to be "" (empty string) if zsh is running under mc, it was working just fine. Right now I have to completely undefine RPROMPT, so mc won't interpret it.

comment:11 Changed 9 years ago by andrew_b

  • Status changed from new to closed
  • Resolution set to invalid
  • Milestone Future Releases deleted

comment:12 Changed 9 years ago by z0rc

  • Status changed from closed to reopened
  • Resolution invalid deleted
  • Milestone set to Future Releases

I spoke to soon and reopening this ticket. Sorry.

RPROMPT has nothing to do with this as it won't affect the situation, as I though initially. "\033[?1h\033=" is present always, event when RPROMPT isn't set. This looks like mark of prompt end, as they present event with empty PROMPT.

I still think this is some kind of race condition. I can catch empty prompt comes after valid when setting up just watch on subshell_prompt variable. But when I break on read_subshell_prompt the empty shell won't appear and issue won't show up. Also I cannot catch issue under strace, then mc behaves just fine.

comment:13 Changed 9 years ago by andrew_b

  • Blocked By 3125 added

comment:14 Changed 9 years ago by andrew_b

  • Blocked By 3125 removed

comment:15 Changed 9 years ago by andrew_b

  • Status changed from reopened to closed
  • Resolution set to fixed
  • Milestone changed from Future Releases to 4.8.12

Changed 3 years ago by xor512

/etc/skel/.zshrc from manjaro-zsh-config package

Changed 3 years ago by xor512

no prompt (happens after typing ls -> enter several times)

Changed 3 years ago by xor512

.zshrc from manjaro forum which probably do not depend on manjaro-zsh-config package and can be probably used on plain Arch

comment:16 Changed 3 years ago by xor512

  • Status changed from closed to reopened
  • Resolution fixed deleted

This still happens with config from manjaro-zsh-config package in 5.4.15-2-MANJARO. The prompt is approx. 9 out of 10 times there but once in a while after typing a command (like ls) it is gone. To bring it back one has to type some command again (one or more times).

This is the standard config from manjaro-zsh-config and can be found in /etc/skel/.zshrc after installing manjaro-zsh-config package.

I've created a topic on a forum: https://forum.manjaro.org/t/prompt-disappears-in-zsh-mc-after-typing-comands-sometimes/122552 but I think this may be also an issue with mc vs zsh, which is to be fixed in mc.

To reproduce the issue one should use the config attached https://midnight-commander.org/attachment/ticket/3121/.zshrc (I'm not sure it will work without the full manjaro-zsh-config package though and that this package can be installed on plain Arch easily, however the issue is also reproducible with the another config: https://forum.manjaro.org/t/unstable-manjaro-zsh-config/17899/105 (which seems to be not "Manjaro-specific" and is attached as https://midnight-commander.org/attachment/ticket/3121/.zshrc_arch) and then type "ls -> enter > ls -> enter..." until prompt is gone. This does not always happen but often enough (1 times out of 10 approx). By "does not always happen" I mean that it always happens if you will type "ls -> enter" enough for it to happen. But how many times one has to do it is random. I confirm that it seems like a race condition. It does not happen without .zshrc file at all.

I'm using xfce4-terminal, but this also happens in xterm (see https://midnight-commander.org/attachment/ticket/3121/prompt_sometimes_disappears.png).

I have a workaround for this based on the comment: https://midnight-commander.org/ticket/3121?cnum_edit=16#comment:10 in .zshrc:

# mc f... up prompt (it disappears after running commands) sometimes for some reason
if ps $PPID | grep mc; then
    # this removes git_prompt_string cool stuff but I have no other solution for now
    RPROMPT=""
fi

but this removes some goodies one make for git in RPROMPT.

Last edited 3 years ago by xor512 (previous) (diff)

comment:17 follow-up: ↓ 19 Changed 12 months ago by kvaster

I'm using zsh with mc and I have disabled rprompt for mc. But I still have empty prompt sometimes. It seems that zsh adds empty line from time to time. There is already check in mc in set_prompt_string function, but zsh adds empty line with escape sequences. I've created simple UGLY patch and now prompt works as expected with zsh and git goodies:

diff --git a/src/subshell/common.c b/src/subshell/common.c
index 65f5718..5b5162f 100644
--- a/src/subshell/common.c
+++ b/src/subshell/common.c
@@ -724,10 +724,30 @@ parse_subshell_prompt_string (const char *buffer, int bytes)
 static void
 set_prompt_string (void)
 {
+    int m, i;
+
     if (mc_global.mc_run_mode != MC_RUN_FULL)
         return;

-    if (subshell_prompt_temp_buffer->len != 0)
+    m = 0;
+    for (i = 0; i < subshell_prompt_temp_buffer->len; i++) {
+        char c = subshell_prompt_temp_buffer->str[i];
+        if (c == 27) {
+            m = 2;
+        } else if (m == 0) {
+            m = 1;
+            break;
+        } else if (m == 2) {
+            m = c == '[' ? 3 : 0;
+        } else if (m == 3) {
+            m = c == '?' ? 4 : 0;
+        } else if (m == 4) {
+            if (c < '0' || c > '9')
+                m = 0;
+        }
+    }
+
+    if (m == 1)
         g_string_assign (subshell_prompt, subshell_prompt_temp_buffer->str);

     setup_cmdline ();

comment:18 Changed 12 months ago by zaytsev

  • Cc congest, aurrak, reagle added

Could this be the solution for the zsh problem in #4198 ?

comment:19 in reply to: ↑ 17 Changed 12 months ago by andrew_b

Replying to kvaster:

I've created simple UGLY patch and now prompt works as expected with zsh and git goodies:

It would be great if you provide more or less detailed description about your code.

comment:20 Changed 10 months ago by andrew_b

It seems to me that patch in comment:17 does the same as function strip_ctrl_codes().

comment:21 Changed 10 months ago by andrew_b

  • Description modified (diff)
  • Milestone changed from 4.8.12 to Future Releases

Related to #4258.

comment:22 Changed 5 months ago by newsboost

Hi all. I'll just add that I can confirm I've also just experienced this issue and then add a simpler workaround that doesn't require any compiling of source code (alternative and easier work-around than https://midnight-commander.org/ticket/3121#comment:17):

First my system:

Fully up-to-date Arch Linux running with Midnight Commander 4.8.28 and I also use zsh. Interestingly enough I actually have 2 laptops with exactly the same configuration (~/.zshrc and a sourced config-file common to zsh and bash). But this CTRL+O subshell issue only happened on one of the pc's (don't know why, it's very strange because both runs uptodate Arch Linux, so the difference should mostly be hardware and maybe minor differences in installed packages although most is the same).

"Work-around solution":

I ended up with first disabling the last command in my startup zsh-config: "neofetch" - later I changed it to this:

[[ $(realpath /proc/$PPID/exe) == */bin/mc ]] || neofetch

which effectively prevents "neofetch" from running when the CTRL+O starts a subshell, which was the culprit on my system.

Concluding remarks:
I did not try to compile anything but hope this bug will be fixed and until that, I think this is a really good and easy work-around for me, which I hope also can help other who experiences this issue. You can read more details here, because the work-around wasn't actually my idea but I posted the issue in the Arch Linux forum (https://bbs.archlinux.org/viewtopic.php?pid=2053275#p2053275) and now I hope sharing this info will help fix the bug upstream and properly so the workaround isn't needed in the future. Please let me know if this helps other with the same issue so we maybe can find the culprit, if the issue remains without neofetch on some systems, which I suspect and I'll be happy to help, if there are questions.

UPDATE: The above workaround does not always work - this works (on my system):
hmm, I just downloaded mc-4.8.28.tar.xz (am new user here, couldn't see how to get the git-repo, but this seems ok) and compiled it and tried to reproduce this and see if I could see/learn more. But it starts up with "Error. Unable to load 'default' skin. Default skin has been loaded" (which I think is contradictory?). After pressing ESC+enter a few times it opens up in black/white - I then hit CTRL+O, but then the issue persists and the subshell-workaround above doesn't help (it does however write: "zsh:1: permission denied: ..", which is more helpful than the mc from my package manager). However, what *DOES* work is to:

mv -i .zshrc .zshrc__

which is also how I found the above-mentioned work-around (by "bisecting", i.e. commenting things out until I could see what did the difference).

UPDATE 2: The comment no.17-workaround does not work on my system

I just tried applying the patch described at https://midnight-commander.org/ticket/3121#comment:17 and it made a difference: Now I got colors instead of black/white and furthermore I don't get any annoying "Error. Unable to load 'default' skin"-startup error. So my expectations were high, but unfortunately this patch does not fix my CTRL+O (subshell) issue... hmm. I understand this issue is difficult to track down as many people cannot reproduce it. If you have any ideas for what I could try, to get a better understanding, please write, I would be happy to help as I'm a long-time user of "mc" and with it should be bugfree...

UPDATE 3: Preliminary learnings with GDB

I've currently applied the comment no17-patch and just learned from searching a bit that init_subshell()-should be a breakpoint-place but also execute.c:491 and 501. I attached to the PID of the compiled mc (incl patch-17) and hit CTRL+O and now I can see what happens:

  • In line 491 of execute.c, it skips the if-block because mc_global.tty.use_subshell=0.
  • Next it goes to line 501 of execute.c, where the variable "outputs_starts_shell" is 0, so it ends up in the "else"-case where it just runs "get_key_code(0)", instead of running the fputs and my_system-stuff (lines 503-506).
  • The minute I step on line 509 gdb waits for me to press a key.
  • As soon as I press any key, we're at line 512 (this is exactly the issue I'm seeing and others have reported, CTRL+O subshell doesn't work, as soon as we press a key "mc" goes back to it's dual-split-view).
  • At this time things are just wrong so debugging further from here will not help me.

I would need to understand better why the situation above has happened, this will not be today, I'm a bit tired now. Anyway, if I should test anything or if you have ideas, please write (I hope I'll receive an email notification?) and I'll try it out so this weird zsh-issue can be fixed, I hope this helps...

Last edited 5 months ago by newsboost (previous) (diff)

comment:23 Changed 3 months ago by tonal

  • Cc tonal.promsoft@… added

comment:24 Changed 3 months ago by tonal

Do not work SubShell? for me. :(
My Os - KDE Neon.
After upgrade base ubuntu from 20.04 to 22.04 mc subshell do not working.

My environment:

$ mc -V
GNU Midnight Commander, версия 4.8.28
Скомпилирован с библиотекой GLib версии 2.72.1
С библиотекой S-Lang 2.3.2 и с базой данных terminfo
Скомпилирован с библиотекой libssh2 версии 1.10.0
Со встроенным редактором и поддержкой Aspell
C поддержкой внутренней командной оболочки
С поддержкой фоновых операций
С поддержкой мыши в xterm и консоли Linux
С поддержкой событий X11
С поддержкой интернационализации
С поддержкой многих кодировок
With ext2fs attributes support
Виртуальная файловая система:
 cpiofs, tarfs, sfs, extfs, ext2undelfs, ftpfs, sftpfs, fish
Тип данных:
 char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;
$ neofetch
OS: KDE neon 5.26 x86_64 
Host: TravelMate P259-MG V1.51 
Kernel: 5.15.0-1021-oracle 
Packages: 7542 (dpkg), 44 (nix-user), 10 (flatpak), 20 (snap) 
Shell: bash 5.1.16 
Resolution: 1366x768 
DE: Plasma 5.26.2 
WM: KWin 
Theme: Breeze Light [Plasma], Breeze [GTK3] 
Terminal: konsole 
CPU: Intel i5-6200U (4) @ 2.800GHz 
GPU: Intel Skylake GT2 [HD Graphics 520] 
GPU: NVIDIA GeForce 940MX 
Memory: 8579MiB / 31975MiB 
Last edited 2 months ago by andrew_b (previous) (diff)

comment:25 Changed 3 months ago by ce72

Same problem exists in cygwin on official Windows Terminal Console using the official cygwin package (mc 4.8.27 with subshell support):

  • Ctrl-o shows the screen without prompt. When I enter text it jumps back to the mc screen after the first char.
  • The aliases from .bashrc (and .zshrc either) are missing in the mc input line (tested both bash and zsh)
  • each invocation of mc leaves a zombie proecess zsh.exe (or bash.exe) in task manager

When I start from there with mintty mc then everything is working fine (ctrl-o opens a screen, I can edit there and the rc file has been sourced successfully.

Apparently mc has issues with some terminal clients?

comment:26 Changed 3 months ago by tonal

Debugging revealed that when the subshell is initialized, the feed_subshell function returns FALSE.
This happens when initialization takes longer than 1 second.

To fix this, you can add an additional parameter to the feed_subshell function indicating that it is initializing and needs to wait longer.
Or to allow you to specify the timeout in the settings.

comment:27 Changed 2 months ago by antonio_so

tonal is correct.
The culprit is here: https://github.com/MidnightCommander/mc/blob/3250536c157b77e26794f436976fc9d2df514d28/src/subshell/common.c#L752

If your profile loads slower than 1 second - you have no subshell.
I've changed wtime.tv_sec to 10 seconds, built mc locally and I have no problems since.

comment:28 Changed 2 months ago by antonio_so

  • Version changed from 4.8.11 to 4.8.28

comment:29 Changed 2 months ago by andrew_b

  • Owner set to andrew_b
  • Status changed from reopened to accepted
  • Branch state changed from no branch to on review
  • Milestone changed from Future Releases to 4.8.29

Branch: 3121_empty_subshell_prompt
changeset:1ac5c22517ed569817ce05ff6e29c7fba3da200a

comment:30 Changed 2 months ago by andrew_b

  • Votes for changeset set to andrew_b
  • Branch state changed from on review to approved

comment:31 Changed 2 months ago by andrew_b

  • Status changed from accepted to testing
  • Votes for changeset changed from andrew_b to committed-master
  • Resolution set to fixed
  • Branch state changed from approved to merged

comment:32 Changed 2 months ago by andrew_b

  • Status changed from testing to closed
Note: See TracTickets for help on using tickets.