Console saving does not work in FreeBSD

[hempalex]

After update to 4.8.13 from freebsd ports (misc/mc) console saving stopped to work.

I'm tried FreeBSD 8.4, 9.1, 10.0 on about 10 different servers.

Steps to reproduce

1. ssh to server
2. run mc
3. in mc run, for example "ls", giving a list of files in current dir
4. Press Ctrl-o, panels have been hidden, but screen is empty - but i'm expecting an output from "ls"

mc -v output:

GNU Midnight Commander 4.8.13
Built with GLib 2.36.3
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish, smbfs
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

Strange enough, but rolling back to 4.8.12 does not solved problem!
Additionally I tried to remove a .cache/.config/.local directories, but no luck...

[hempalex]

$ LC_MESSAGES=C mc -F
Root directory: /home/alexm

[System data]

Config directory: /usr/local/etc/mc/
Data directory: /usr/local/share/mc/
File extension handlers: /usr/local/libexec/mc/ext.d/
VFS plugins and scripts: /usr/local/libexec/mc/

extfs.d: /usr/local/libexec/mc/extfs.d/
fish: /usr/local/libexec/mc/fish/

[User data]

Config directory: /home/alexm/.config/mc/
Data directory: /home/alexm/.local/share/mc/

skins: /home/alexm/.local/share/mc/skins/
extfs.d: /home/alexm/.local/share/mc/extfs.d/
fish: /home/alexm/.local/share/mc/fish/
mcedit macros: /home/alexm/.local/share/mc/mc.macros
mcedit external macros: /home/alexm/.local/share/mc/mcedit/macros.d/macro.*

Cache directory: /home/alexm/.cache/mc/

$ mc --configure-options

'--with-internal-edit' '--enable-charset' '--enable-nls' '--enable-vfs-smb' '--with-smb-configdir=/usr/local/etc' '--with-smb-codepagedir=/usr/local/etc/codepages' '--with-subshell' '--disable-x' '--with-screen=slang' '--with-slang-includes=/usr/local/include' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=amd64-portbld-freebsd9.3' 'build_alias=amd64-portbld-freebsd9.3' 'CC=cc' 'CFLAGS=-O2 -pipe -fno-strict-aliasing' 'LDFLAGS= -L/usr/local/lib' 'LIBS=' 'CPPFLAGS=-I/usr/local/include' 'CPP=cpp' 'PKG_CONFIG=pkgconf'

[maxout]

same issue(freebsd 10, mc 4.8.13), very annoying

[egmont]

Strange enough, but rolling back to 4.8.12 does not solved problem!

Are you sure this ever worked?

According to my best knowledge, although I might be wrong:

• mc ships a cons.saver binary. Its intent is to save the contents underneath if running on the Linux console (tty1-tty6). This is a workaround for the Linux console not having an "alternate screen" (the one which doesn't have a scrollbar, and apps like mc, less, vim etc. switch to, and the original contents are automatically restored when quitting this mode). I don't know if cons.saver supports FreeBSD, and it definitely can't work over ssh or any other channel since it requires access to the local /dev/vcs* files.

• When running inside a graphical terminal (e.g. xterm), it's the terminal that supports alternate screen. MC switches to this mode when it starts up (you should tell e.g. by the scrollbar no longer working), and it's also the terminal emulator itself that restores the contents when quitting mc or when pressing Ctrl-O (mc only asks the terminal emulator to leave the alternate screen mode).

I suspect either a broken terminal emulator, or a misunderstanding of what's supposed to work and wishing for something that never worked.

Could you please provide exact details about the OS and terminal in front of which you're sitting and ssh'ing to FreeBSD?

[maxout]

As for me, it worked all years before on all oses i've ever seen. Including any versions of freebsd. And now it stops working after update of mc to 4.8.13.
Now I have two identical servers with Freebsd 10 on board, one with updated mc and one with old mc (4.8.11). Old mc saves console output, while new mc does not.
This behavior still the same when i sshing from Debian with xterm, and from windows with putty.

[egmont]

Hmmm, this might indeed be a problem with mc.

It would be nice to see the result 'typescript' files of a run of mc inside 'script', both with working and nonworking mc. The files contain tons of hard-to-understand stuff, so it'd help if otherwise everything (e.g. directories) would be the exact same. It would be nice to do the minimal possible stuff inside mc. That is:

Run 'script', run 'mc', press Ctrl-O, press Ctrl-O again, quit mc, quit script (e.g. Ctrl-D), save the 'typescript' file.

Beware that the file might contain sensible information (list of filenames), preferably do it with empty directories on both panels.

Comparing the two typescript files (with mc-4.8.11 vs. mc-4.8.13) might reveal what's the problem.

In the mean time, if you could do a 'git bisect' to catch the malicious commit, that'd be even better :)

(Unfortunately I don't have FreeBSD, that's why I'm asking for this.)

[maxout]

old mc

[maxout]

new mc

[maxout]

Sorry, but I can't find the differences between typescript files attached

The sequence was the same on both machines:

script
mc

ctrl-o
enter (to get clean prompt)

ls

enter (here were receive ls output)
ctrl-o (mc panels are back)
ctrl-o (back to command line, now can see ls output for old mc, and completely blank screeen for new mc)
ctrl-o (mc panels are back)
F10
enter
ctrl-d

[maxout]

old mc with timestamping

[maxout]

new mc with timestamping

[maxout]

I've recorded all the same again, now with timestamping (script -r)
New files can be played with "script -p <filename>", with clearly visible buggy behavior of new mc.

[egmont]

(FreeBSD script's timing seems to be incompatible with Linux's, I don't have an option to play it back with timing :( I can do a slowcat on the raw version.)

The buggy version never prints [?47h (nor [?1047h or [?1049h), meaning it does not switch to the alternate screen.

You could also verify this by first producing more than a screenful of update (e.g. ls -l /etc) and then starting mc. With the correct mc you shouldn't be able to use the terminal's scrollbar to scroll back (when mc's panels are on). With the buggy one you'll be able to scroll back.

Looks like subshell support is not being used. (I should have thought of it at the beginning.) What does "mc --version" say about subshell?

Are you using FreeBSD packages of mc, or compiling yourself? If the former, it _could_ be a bug in FreeBSD's way of packaging it, maybe they changed something. If the latter, is 4.8.12 buggy for you too, as for the original reporter? Could you maybe please do a git bisect between .11 and .12?

What shell are you using? (There were zsh changes in .12).

[maxout]

root@dedic53 [~/aa]# mc --version
GNU Midnight Commander 4.8.13
Built with GLib 2.36.3
Using the S-Lang library with terminfo database
With builtin Editor
With subshell support as default
With support for background operations
With mouse support on xterm
With support for X11 events
With internationalization support
With multiple codepages support
Virtual File Systems: cpiofs, tarfs, sfs, extfs, ftpfs, sftpfs, fish, smbfs
Data types: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

[maxout]

I've tried mc from ports first (compiling myself). When noticed the bug, reinstalled it from packages - nothing changed.

[maxout]

I've tried zsh and bash - all the same

[maxout]

"You could also verify this by first producing more than a screenful of update (e.g. ls -l /etc) and then starting mc. With the correct mc you shouldn't be able to use the terminal's scrollbar to scroll back (when mc's panels are on). With the buggy one you'll be able to scroll back."

Confirmed, things happened exactly as you describe it. Terminal's scrollbar stays usable for buggy mc, and disappears for old one.

[maxout]

Let's play with portdowngrade utility =)
I've just now tried to install 4.8.11 (port revision r340725 just before .12) - it works.
Then installed first revision of 4.8.12 (port revision r349958) - works too.
Then latest port revision of 4.8.12 (r357939) - works again.
First port revision of 4.8.13 (r369133) - doesn't work.

[maxout]

At last I've found the reason.
There is a freebsd port specific patch, named patch-libttywin.c
I've noticed it was changed since 4.8.13
Old version:

--- ./lib/tty/win.c.orig<------>2013-09-02 19:13:32.000000000 +0200
+++ ./lib/tty/win.c<--->2013-12-07 10:33:20.000000000 +0100
@@ -96,7 +96,7 @@
 void
 do_enter_ca_mode (void)
 {
-    if (mc_global.tty.xterm_flag && smcup != NULL)
+    if (mc_global.tty.xterm_flag)
     {
         fprintf (stdout, /* ESC_STR ")0" */ ESC_STR "7" ESC_STR "[?47h");
         fflush (stdout);
@@ -108,7 +108,7 @@
 void
 do_exit_ca_mode (void)
 {
-    if (mc_global.tty.xterm_flag && rmcup != NULL)
+    if (mc_global.tty.xterm_flag)
     {
         fprintf (stdout, ESC_STR "[?47l" ESC_STR "8" ESC_STR "[m");
         fflush (stdout);

New version:

--- lib/tty/win.c.orig<>2014-09-02 09:23:58 UTC
+++ lib/tty/win.c
@@ -107,7 +107,7 @@
 void
 do_exit_ca_mode (void)
 {
-    if (mc_global.tty.xterm_flag && rmcup != NULL)
+    if (mc_global.tty.xterm_flag)
     {
         fprintf (stdout, ESC_STR "[?47l" ESC_STR "8" ESC_STR "[m");
         fflush (stdout);

I've replaced this new patch with old one (from 4.8.12) and build 4.8.13.
MC works.

Found the solution, but have no idea what to do with this information :(
Should I create bug report to freebsd port maintainer?

[egmont]

[This comment might not be relevant seeing your most recent comment, but posting it anyway, might be useful in the future]

Replying to maxout:

Then latest port revision of 4.8.12 (r357939) - works again.

Strange, this differs from the original reporter's observation.

Could you please locate the faulty change:

git clone ​git://github.com/MidnightCommander/mc.git
cd mc
git bisect start
git bisect good 4.8.11
git bisect bad 4.8.13
./autogen.sh
./configure

and then repeat this in a loop:

make
./src/mc
test to see if it works:
o if Ctrl-O works as expected then execute: git bisect good
o if doesn't work then execute: git bisect bad
o (if fails to compile or you're unable to test, execute: git bisect skip)

each step of the loop will take a few minutes, and you'll have around 7 steps. At the end it'll tell you the faulty mc commit.

[egmont]

(Note: the "-" sign (removal of line) was converted to a bullet in your patch.)

FreeBSD's change of the patch definitely looks broken, they've modified a condition in a certain way when leaving the alternate screen, but dropped the same modification of the condition when entering it. It's their fault, please notify them.

On the other hand, we should understand why such patch is needed at all, and modify mainstream mc so that freebsd doesn't need a patch.

[maxout]

Just for information, here is a bugreport:
​https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194627

[andrew_b]

Closed as duplicate of #2633.

[andrew_b]

Ticket #3368 has been marked as a duplicate of this ticket.