copying wrapped line from mcview adds EOL when pasting

[nerijus]

For example, the line is:
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0

When wrap mode is on (F2), the line is displayed in mc viewer as:
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/
24.6.0

I select this wrapped line with a mouse and then paste it. It pastes with a new line between "Thunderbird/" and "24.6.0". Is it possible to paste without the new line character, as a one whole line? AFAIR it pasted w/o EOL in some earlier versions. Tried with a few different terminal apps (gnome-terminal, xterm, PAC Manager).

[egmont]

This is a technically interesting issue :)

I guess it's clear that if your file happens to have an explicit newline after "Thunderbird/" then you'd expect that newline to be copy-pasted, even though the file (at terminal width of 80) is displayed identically.

First, let's see what terminal emulators do. Let's stick to the simplest case: a stream of text with embedded newlines being output. I've quickly checked that xterm, gnome-terminal/vte, konsole, urxvt, suckless/st, putty/pterm all properly remember whether there was an explicit newline printed at the end of the line, or whether the text simply just overflowed to the next one. They use this information for copy-pasting, as expected. I found terminology to be an exception: ​https://phab.enlightenment.org/T1387

Now what I did not check is: what happens if you alter the screen contents by all weird series of moving around the cursor crazily, printing characters here-and-there, and performing even more hairy things (e.g. deleting a line, scrolling a region, clearing to the end of the line, let alone the issue of wrapped CJKs at the right margin). I guess all terminals behave differently, but hopefully we could locate a reasonable subset where the expected behavior is clearly defined, and fix terminal bugs (if any).

The next step would be to add support to slang and ncurses - I'm not familiar with any of them, but a quick glimpse suggests to me that neither of them have such support [I might be wrong, though]. They are about controlling the contents of the visible character cells, and not about controlling some additional invisible data. (There's a very similar issue with trailing whitespaces - the crazy ways of slang/ncurses optimizing for screen redrawal might cause extra or missing trailing whitespaces when copy-pasting. A third similar issue is TABs in terminals that try to remember them (e.g. gnome-terminal), using ncurses/slang pretty much means they'll be converted to spaces.) It would need methods to set whether a line is terminated by explicit newline or by overflow; and to set how many whitespaces there are at the end of the line. They'd need to perform screen updates in ways so that the terminal emulator's concept of these values is whatever's desired.

Sounds doable but probably too much work for addressing such a little issue. My experience about such cross-components too-much-work too-little-benefit issues is that it's probably not going to happen.

And then there's one more important problem: what about copy-pasting when wrap mode (F2) is off? I guess the ideal behavior would be to still copy all the logical text (including words that are not shown on the screen right now), just as it's done with any graphical editors, right? Now it's clear that the terminal cannot copy-paste data it is not even aware of.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

So probably the solution is to forget about the terminal's copy-paste feature (it's just never going to behave as expected), and let mcview handle the mouse highlight (just as mcedit does).

I'm not sure what mcedit does with its selection, whether it remains mc's private business or propagages it to the copy-paste buffer. I can see two ways the latter can happen, one is using mc's direct X connection (it could modify the X primary/clipboard), the other one is OSC 52 escape sequence (which is supported by xterm and probably noone else).

We {c,sh}ould think about how mc could be improved along these lines.

[nerijus]

I guess it's clear that if your file happens to have an explicit newline after "Thunderbird/" then you'd expect that newline to be copy-pasted, even though the file (at terminal width of 80) is displayed identically.

It would be nice to have, but not critical - if I paste the long line to a console and it pastes without EOL, it is not executed. Now the real world example which happened to me. The long line was something like that:

....some text....; rm -rf /u
sr/share....

The line got executed and tried to rm -rf /u - I was lucky that wrapping did not occur after "rm -rf /".

I'd still suggest changing behavior to what was earlier and not pasting EOL unless the line ends with spaces.

[egmont]

Replying to nerijus:

I'd still suggest changing behavior to what was earlier and not pasting EOL unless the line ends with spaces.

If this was indeed the behavior earlier, I'm quite certain it was a change in your preferred terminal emulator (or maybe, but I don't think so, slang or ncurses). Or, it was (and maybe still is) "random" (i.e depending on what's exactly on the screen and how that content is updated as it changes over time).

I'm quite sure that mc has no way to influence whether there's an explicit newline at the end of the line or not.

[egmont]

Related: ​https://bugzilla.gnome.org/show_bug.cgi?id=697571

[nerijus]

You are right - it was my older terminal (multi-gnome-terminal). I'll ask my now used PAC terminal to fix this.
It would still be nice to fix/workaround this in mc, if possible.

[egmont]

multi-g-t has been abandoned for ~10 years, I'm surprised there are still people who used it recently :)

There's two steps to such a copy-paste.

The first is highlighing, and copying to the primary/selection buffers. Here the correct behavior is to copy that newline (under certain circumstances which seem to hold when using ncurses/slang), there's nothing you could do. Changing it not to copy that newline would lead to undesired misbehavior with many use cases. The only thing that could be done is to extend ncurses/slang or rewrite mc so its paints its canvas in different ways that lead to not copying those newlines - none of which is likely to happen.

The second is when pasting, so that newlines are not pasted automatically. This is the right place to catch the problem. Note that PAC Manager uses VTE (the same terminal component as gnome-terminal uses), albeit a pretty old and unmaintained version thereof. So if you start this way, you'll end up at the gnome-terminal bug I linked in comment 4.

[nerijus]

With bash-4.4/readline-7.0 if I put "set enable-bracketed-paste on" in ~/.inputrc, pasting a string with EOLs does not execute it. But with mc there is one problem. If I launch mc, press Ctrl-O and paste, it gets executed. But if I paste again, it does not. Or if I launch mc, press Ctrl-O, press Enter and then paste - it does not paste EOLs too.

[egmont]

Could you please open a new issue for that?

The current one is about the viewer, and selecting text there. It's the source of the selection ("copy"), and an additional newline is added where not expected.

Your issue is in mc's command line, and it's the target ("paste").

Technically these two have nothing to do with each other.

[nerijus]

It's about copying from mcviewer, but pasting was always in mc, in my case always in the shell, after pressing Ctrl-O. I changed component of this ticket.

[egmont]

Dear nerijus,

This bugreport here is about copying from mcview. (It is irrelevant where you paste that data later.)

Your new finding is about pasting to the command line. (It is irrelevant where the pasted data comes from.)

What's the common in these two bugs? Again: nothing.

I'm asking you again to file a new bugreport, and more importantly, I'm kindly asking you not to mess with this bugreport's metadata and not to try to hijack it for something else. Thank you!

[nerijus]

Why do you think so? I reported it, and I called it "pasting wrapped line adds EOL". I reported it about pasting in the first place.
New ticket #3874.

[egmont]

Replying to nerijus:

I reported it

I missed this, sorry.

, and I called it "pasting wrapped line adds EOL".

I thought it was a fairily obvious sloppy wording. You copy from mcview, and you didn't even mention where you paste to (well, definitely not to mcview of course). There's absolutely no way the contents of the clipboard could be correct, and get corrupted whenever the contents are pasted anywhere, but if and only if it was copied from mcview at the first place. This is nonsense. I think it's damn obvious that it must be the copying to the selection/clipboard that goes wrong in the originally reported bug.