mc compiled with ncurses support does not handle mouse

[kloczek]

Fedora rawhide, ncurses-6.1-8.20180923

When configures --with-screen=ncurses does not handle scroll, and clicks mouse events emitting in mc command line terminal sequences

[kloczek]

Another issue: alt-n and alt-p (next, prev command line from history) does not refresh command line. Cursor is moved to the end of the command but whole command is just blank string.

[kloczek]

Just checked and bug still is present in 4.8.22.

[kloczek]

Just checked and bug still is present in 4.8.23 :(

[egmont]

I've just tried and it works for me as expected (Ubuntu 19.04, mc-4.8.23 compiled from source against ncursesw).

Could you please give more details: Do you take mc from Rawhide or compile it yourself? What terminal emulators? Does it work in other terminals? What does mc --help say? What does ldd /path/to/mc | grep ncurses report?

Another issue

One issue per report, please. Feel free to open a new one for alt-n/alt-p. (By the way, they also work for me.)

[andrew_b]

Perhaps this is depending on ABI version (5 or 6).

[kloczek]

So just FTR .. I'm using 6.

[andrew_b]

In ncurses-5:

$ infocmp xterm | grep kmous
	kind=\E[1;2B, kmous=\E[M, knp=\E[6~, kpp=\E[5~,

In ncurses-6:

$ infocmp xterm | grep kmous
	kind=\E[1;2B, kmous=\E[<, knp=\E[6~, kpp=\E[5~,

Why?

As a result, in case of ncurses-6 we have

(gdb) p xmouse_seq
$7 = 0x75559a "\033[<"
(gdb) p xmouse_extended_seq
$8 = 0x4c487f "\033[<"

[egmont]

Not sure how to check the ABI version. I have ncurses 6.1+20181013 (Ubuntu 19.04) and infocmp reports kmous=\E[M.

Not sure how terminfo description is relevant per se, probably not, since as far as I recall mc uses its own mouse parsing (I might remember incorrectly or miss something). It could be however that ncurses ABI 6 defaults to extended SGR 1006 mouse mode, and this somehow conflicts with mc also explicitly enabling this mode. No idea though how switching to this mode twice could be harmful, further debugging is needed.

[andrew_b]

Replying to egmont:

Not sure how to check the ABI version.

As I can see in ​deb rules, ABI version is 6.

[kloczek]

Just for FTR:

$ infocmp | grep kmous
	kind=\E[1;2B, kmous=\E[<, knp=\E[6~, kpp=\E[5~,
$ echo $TERM
xterm-256color

[macias]

I am not sure if my case is related, similar, same, but in any case, I have openSUSE 13.2 Harlequin and openSUSE Tumbleweed. The first one has ncurses 5 installed, the latter version 6. In the first system I can run KDE3 Konsole, or xterm, no gpm and when I click mouse within MC, the mouse is normally registered.

But in Tumbleweed, even if I set $TERM to xterm, or run gpm, mouse clicks in Konsole are translated to random key typing. I.e. let's say I click 3 times LMB, then I get for example "zx9" in commnad line in MC. Oddly, when I run xterm for real, and then run mc, the mouse clicks are property translated.

If I should make some tests, please let me know (basically this is the last issue that prevents me to migrate to Tumbleweed, so I would be happy to help).

[egmont]

I can reproduce something similar to the latest comment, and I understand what is going on for the issue I can reproduce (which is probably the same as the original report).

I don't know how clicking could result in something like "zx9" appearing in the command line, it looks like the terminal doesn't emit extended mouse events, even though it supports them and mc asks for them. What I get is something like "9;16M9;16m" (numbers, semicolons, uppercase M and lowercase m letters).

For me the bug happens if and only if these are both met:

• mc is compiled against ncurses(w);
• for the given TERM env var, "infocmp -a1 | grep kmous=" reports "\E[<" (rather than "\E[M").

Note that the concrete choice of the terminal emulator doesn't matter, it's the same in xterm, konsole, gnome-terminal etc. [Well, the concrete choice of the terminal matters as much as it influences the value of TERM, since infocmp's output depends on this.]

Could you guys please test with mc compiled against ncurses(w), multiple TERM values (e.g. "xterm", "xterm-256color", "konsole", "konsole-256color"), multiple terminal emulators, and confirm these (or report how it works for you if it's different):

• If and only if clicks work correctly with a certain TERM, then with that same TERM the output of "infocmp -a1 | grep kmous=" is "\E[M";
• If and only if clicks don't work but generate garbage with a certain TERM, then with that same TERM the output of "infocmp -a1 | grep kmous=" is "\E[<", and also the output of "infocmp -a1 | grep xm=" is a somewhat longer string beginning with "\E[<";
• As long as TERM is set to the same value, the choice of terminal emulator (e.g. xterm, konsole, gnome-terminal etc.) is irrelevant (but leave urxvt out of this game, that one doesn't support SGR 1006 mode).

---

lib/tty/tty.c tty_init_xterm_support() assumes that the value of the "kmous" property describes the beginning of the old-fashioned mouse sequence, and that the new-fashioned (SGR 1006 extension) isn't described in terminfo. The former (kinda expected to be "\e[M") is stored in xmouse_seq, the latter one is hardcoded to be "\e[<" and is stored in xmouse_extended_seq.

lib/tty/key.c at the end of tty_get_event() checks the incoming mouse event against two or three constants: MCKEY_MOUSE, KEY_MOUSE and MCKEY_EXTENDED_MOUSE. Only in the third case is the event parsed according to the extended mode, as per the parameter passed to xmouse_get_event() just below. The two values beginning with MC are mc's own stuff, assigned by mc's parser. The middle value KEY_MOUSE is specific to ncurses, and is left out for slang builds, this is the key branching point why ncurses and slang behave differently. Let's stick to ncurses.

In my buggy case, the value is this ncurses-specific KEY_MOUSE, and it's followed by the extended protocol. However, xmouse_get_event() is called with a FALSE parameter, so it tries to parse it as the old-fashioned protocol.

In my correct case, the value is one of the MC ones, so ncurses doesn't catch the mouse event. In the mean time, for these terminals I don't have an "xm" terminfo field which would describe how to parse mouse events. I guess these two are correlated, but this theory is yet to be verified.

So what happens is a badly split responsibility with insufficient communication: ncurses catches that it's a(n extended) mouse event, and then mc tries to parse that as an (old-fashioned) event.

I don't know if it's possible to ask ncurses not to recognize mouse events and never to emit KEY_MOUSE. If it's possible, it could be a clean solution, the entire responsibility of parsing the event would be in mc (with the help of terminfo, but not ncurses).

Another solution is to make it entirely ncurses's responsibility: if KEY_MOUSE is detected then continue with ncurses's getmouse(). This would perhaps result in the ncurses and slang branches diverging a bit further, but probably nothing serious.

Yet another, although quite hacky solution could be to try to detect which protocol the event is in, e.g. by string-comparing "kmous" or the first three bytes of "xm" against well-known hardcoded values, or even study the "xm" format string, e.g. see if it has semicolons, "%d" literals for decimals etc., or even attempt to parse the subsequent data according to the format string of "xm" (although that only specifies the syntax, not the semantics, and sounds a pretty unpleasant task to do anyway).

---

There is another danger, also affecting slang. I haven't dug in the code to check what happens exactly.

If terminfo's kmous is "\E[<", lib/tty/tty.c tty_init_xterm_support() assigns this string to both the xmouse_seq and xmouse_extended_seq variables.

When it comes to parsing, the incoming data is matched against two identical patterns. Which one will resolve first? Does it depend on random? I don't know, I haven't checked. But at least it's pretty error-prone. Even if it's deterministic now, an irrelevant future change can easily break it.

So I'd say it's a matter of luck whether the mouse event gets resolved as MCKEY_MOUSE or MCKEY_EXTENDED_MOUSE (two different internal identifiers), and thus whether xmouse_get_event() will parse the rest according to the old or the new protocol.

I'd say tty_init_xterm_support() should be made more robust. Maybe if xmouse_seq (as looked up from terminfo's kmous) happens to be the same as the hardcoded xmouse_extended_seq then unset the former one, to guarantee we'll resolve the one we handle as extended.

[reavertm]

I found the following terminals to work with app-misc/mc-4.8.23-r2:0 compiled with ncurses sys-libs/ncurses-6.1_p20190609:0/6 (ABI=6) :

xterm-x10mouse
xterm+x10mouse
xterm-x11mouse
xterm+x11mouse
xterm-x11hilite
xterm+x11hilite
xterm-xfree86
xterm-new
xterm-basic
xterm-bold
xterm-color

$ infocmp -a1 | grep kmous
        kmous=\E[M,

I bet there are others.
Interestingly xterm, any konsole*, xterm-16color, xterm-88color, xterm-256olor - don't work with this ncurses.

$ infocmp -a1 | grep kmous
        kmous=\E[<,
$ infocmp -a1 | grep xm=
        xm=\E[<%p1%d;%p2%d;%p3%d;%?%p4%tM%em%;,

[egmont]

A few more observations:

ncurses-6.2 has just been released.

I'm looking at misc/terminfo.src, both in 6.1 and 6.2:

The definition of "xterm" and "xterm-256color" both uses "xterm-new", which in turn uses "xterm+sm+1006", which in turn defines "kmous" and "xm" to the "\E[<"-style values.

Yet, on my Ubuntu 19.10 with ncurses_6.1+20190803, I get "kmous=\E[M" and no "xm". I can't see a relevant Debian or Ubuntu patch either.

Which means that I'm definitely missing something here, and I don't see the reason yet why Ubuntu doesn't behave like Fedora (in the original report) or whichever other distros you guys use and have tested. We could further investigate, but more importantly, we should expect that Debian and Ubuntu will also soon catch up with these upstream changes.

The value of "xm" isn't the same in 6.1 and 6.2, there's a new leading "%i" introduced, and the three numerical parameters have been shuffled around (6.1: "p1 p2 p3", 6.2: "p3 p1 p2") which I won't further dig into, but probably is another reason for not trying to parse these values.

[egmont]

Fix

[egmont]

Please test the attached patch.

It's a quick ugly hotfix, no cleanup refactoring.

[andrew_b]

Yes, it works.

[macias]

Thank you very much for all your work and replies. For now I have only partial results. I compiled rpm with provided patch, it does not help, but it is only because (it looks) openSUSE spec for mc defaults to slang.

I tried to switch to ncurses, but at configure phase I have this error:

configure: error: Cannot find a library providing stdscr

I looked for it, and found only remark that maybe ncurses is split into two libraries "ncurses" and "tinfo", however I cannot find the latter in the repo (I am not sure if the idea of the split is correct in the first place).

So my first post about problems is related to mc+slang, not mc+ncurses. And as I am writing this I tested clicks again. It looks like one character takes two clicks, and the character shown depends on the line the mouse cursor is in.

For example:
row 0 (i.e. the top one) -- !
row 1 -- "
row 2 -- #
row 3 -- $

Hmm, ok, I clicked now persistently row after row, and it does not look random, it is just ASCII-7.

I would be happy to test mc with ncurses, but for now I don't know how to overcome this error with "stdscr".

[egmont]

So, both the symptoms (the characters that appear) and the technical details (slang vs. ncurses) differ for you from the one reported here.

It's getting a bit hard to follow what happens, so could you please open a new bug for your case?

Let's leave this one for the ncurses + kmous=\e[< story.

[macias]

Of course, here is the "fork" for slang -- #4063

[andrew_b]

Branch: 3954_ncurses_mouse
changeset:b92c7f86b4be3b200e0f2de713a4c40b28599e61

[andrew_b]

Merged to master: [d9da121b33df6ab081dfee8d259695678951367f].

[reavertm]

I tried this patch in Gentoo with ncurses-6.2 on top of mc-4.8.24 and I confirm that it fixes the issue. Thanks!

[anton_kg]

Hi guys,

I'm facing the same bug with 4.8.26 and konsole-20.12.3
The patch seems there.
What I'm missing? Is any specific option to compile?
~ $ infocmp -a1 | grep kmous

kmous=\E[<,

~ $ infocmp -a1 | grep xm=

xm=\E[<%i%p3%d;%p1%d;%p2%d;%?%p4%tM%em%;,