Tilde + username completion in copy/move dialogs does not work

[ZlatkO]

Completion does not work for tilde + username expansion (i.e., to expand to a certain user's $HOME) in copy/move dialogs. If I enter a tilde character in the destination field followed by one or more characters (ie. ~apa) and then hit <Alt+TAB>, nothing happens. If I complete the username myself, followed by a slash (ie. ~apache/), then the rest of the <Alt+TAB> completion works as expected.

So it seems that MC actually does understand the ~username/ concept as such in the copy/move dialogs (i.e., files do end up in the correct destination), but it can't complete the ~userna<Alt+TAB> part by itself. I'm pretty sure that this did already work in some earlier version, but I'm not 100% sure. :-)

Fun fact: in MC's command line widget, this works just fine, i.e., entering ~apa<Alt+TAB> gets properly completed to ~apache/. So the command line widget's completion code path is probably a different one than the copy/move dialog's one.

[andrew_b]

This is totally unclear for me. What is ~apache? What is username?

Please provide a step-by-step description how to reproduce this bug.

[zaytsev]

What do you mean by the command line widget? Do you mean Quick CD or the shell dialog? If it's the latter, then yes, of course, expansion works, because your shell does this, and it has nothing to do with mc. In as far as copy & move dialogs are concerned, there is special code to interpret ~, but apparently it's not tied to the simple completion engine for the widget...

@Andrew he probably means that if you press Esc-Tab in the copy dialog path entry box, then you get completion, but if you enter ~ and press Esc-Tab, then there is no completion, but he expects expansion in /home/username.

[ossi]

adjusted misleading wording - the expansion itself works just fine.
also, replaced plain <TAB>, as this obviously can't work (at least in a default config).

[andrew_b]

Replying to ZlatkO:

So it seems that MC actually does understand the ~username/ concept as such in the copy/move dialogs (i.e., files do end up in the correct destination), but it can't complete the ~userna<Alt+TAB> part by itself.

The input line for target file/directory supports completion for file names only. This benavior was introduced in [f866709186627d57acf585fc9246c81c994594bc].

I'm pretty sure that this did already work in some earlier version, but I'm not 100% sure. :-)

This early version is 4.8.7 released at the end of 2012.

Fun fact: in MC's command line widget, this works just fine, i.e., entering ~apa<Alt+TAB> gets properly completed to ~apache/. So the command line widget's completion code path is probably a different one than the copy/move dialog's one.

Correct. Input line supports all available completions.

[ZlatkO]

@andrew_b I take it you now know what kind of completion I mean? If not, please let me know and I'll explain in more detail.

@zaytsev By "command line widget" I mean the part of MC that you can switch on & off with F9 => Options => Layout => Other options => Command prompt.

@ossi Yes, of course I meant <ALT>+<TAB>, <ESC>+<TAB> or whatever else you can re-bind it to. I just left it out for brevity. Sorry if this did cause confusion.

[ZlatkO]

Replying to andrew_b:

The input line for target file/directory supports completion for file names only. This benavior was introduced in [f866709186627d57acf585fc9246c81c994594bc].

So does this mean that it is intentional that ~userna<ALT>+<TAB> completion does not work in copy/move dialogs? If yes, is there a reason for this, or is it just a feature that nobody ever requested? One could argue that a user's home directory is actually a part of that filename - and, as I said, once I fill in the username manually, MC takes care of the rest of the filename completion just fine.

[andrew_b]

Replying to ZlatkO:

Replying to andrew_b:

The input line for target file/directory supports completion for file names only. This benavior was introduced in [f866709186627d57acf585fc9246c81c994594bc].

So does this mean that it is intentional that ~userna<ALT>+<TAB> completion does not work in copy/move dialogs?

Yes.

If yes, is there a reason for this, or is it just a feature that nobody ever requested?

This feature is about 9 years old.

One could argue that a user's home directory is actually a part of that filename - and, as I said, once I fill in the username manually, MC takes care of the rest of the filename completion just fine.

I don't understand you or you don't understand how username completion works.
Username completion gets username from the password database (see getpwent(1)). No any filenames are used.

[ZlatkO]

Replying to andrew_b:

Replying to ZlatkO:

So does this mean that it is intentional that ~userna<ALT>+<TAB> completion does not work in copy/move dialogs?

Yes.

Okay, if that's the case, then there's not much I can do about it, I guess. :-)

I don't understand you or you don't understand how username completion works.
Username completion gets username from the password database (see getpwent(3)). No any filenames are used.

Yes, I know. My point was that once you do lookup a user's $HOME with getpwent(), you can use the result for completing the actual (rest of the) filename. But that's technical nitpicking, I admit. ;-)

Feel free to close the ticket, and sorry for the noise & confusion.

[andrew_b]

Replying to ZlatkO:

Yes, I know. My point was that once you do lookup a user's $HOME with getpwent(),

No. We lookup a user's name, not a user's home directory. The user's name and $HOME are the same in most cases, but this is not a mandatory requirement.

Feel free to close the ticket, and sorry for the noise & confusion.

Ok.

[ossi]

closing this doesn't make any sense to me. there is no plausible reason why tilde expansion should be supported in all (mentioned) contexts, but be completed only in some. the fact that a user name is being completed is of secondary concern - it's still path completion, and it should be always consistent with what the context actually supports (which in turn should be consistent among all contexts; everything else would be a UX nightmare).

mc doesn't even support actual username completion (that is, completing plain user names, like the argument to passwd) in the command prompt, and it cannot reasonably do so without a completion config system like the shells have. so the whole argument seems misapplied (and the dialog flag is a misnomer).

[zaytsev]

No. We lookup a user's name, not a user's home directory.

So why wouldn't we look up the home directory as well, it's there as pw_dir, right? I think that tilde expansion in path completion would be a nice feature, so I'd keep the ticket open...

[ZlatkO]

Replying to andrew_b:

Replying to ZlatkO:

Yes, I know. My point was that once you do lookup a user's $HOME with getpwent(),

No. We lookup a user's name, not a user's home directory.

Okay, since both @ossi and @zaytsev picked this up one more time, and I fully agree with their reasoning, I'll also give it another go: getpwent() returns a pointer to a struct passwd{}, so once you do the call, you already have both: the username and his home directory (plus the rest of the struct fields). Differentiating between "we lookup a username" and "we lookup a user's home directory" doesn't make any sense. You do one syscall, and get both data points.

The user's name and $HOME are the same in most cases, but this is not a mandatory requirement.

Yes, I know, and did I not state anything to that effect, at least not intentional. I know that there is no strict $username => /home/$username mapping, if that's what you meant (if you meant something else, then sorry, I did not get it). In my own example, ~apache (ie. the completion of the home dir of the "apache" user) would complete to "/srv/httpd", which is fine.

Again: in MC's command prompt, if I enter "cd ~g<ESC><TAB>", MC presents a small popup where I can choose between ~games/, ~gdm/ and ~gnome/ (see attached screenshot). If I try the same in the destination field in a copy/move dialog box, this does not work (see attached screenshot). I think this is what @ossi meant by "UX nightmare": same app (MC), same context (path completion [*]), same user input ("~g<ESC><TAB>"), different result. ;-)

[*] Yes, path completion: I want to cd to a different path, and I want to copy something to a certain path, and I want MC to autocomplete that path for me. As a user, I do not care for whatever kind of lookups MC has to do internally to do that. In fact, it already does it for me perfectly fine in the command prompt, so why does it refuse to do the same thing for me in the copy/move dialog box?

[ZlatkO]

path/username completion works fine in the command prompt

[ZlatkO]

path/username completion does not work in the copy/move dialog box

[ZlatkO]

Ooooh, I've only seen this right now:

@Andrew he probably means that if you press Esc-Tab in the copy dialog path entry box, then you get completion, but if you enter ~ and press Esc-Tab, then there is no completion, but he expects expansion in /home/username.

Sorry, my bad! I usually find it rather annoying when people do not read comments completely, and now I made the same mistake. Again, sorry! So that explains the "... not a mandatory requirement" comment that I couldn't quite make sense of, I guess.

@zaytsev Almost, but not quite. :-) The beginning of your interpretation ("he probably means that if you press Esc-Tab in the copy dialog path entry box, then you get completion, but if you enter ~ and press Esc-Tab, then there is no completion") is correct, but the last part of it, the conclusion ("but he expects expansion in /home/username."), is not.

Just if that's not clear meanwhile from my other comments: I do not expect MC to replace the ~whatever part with the actual result of the lookup. But I do expect MC's <ESC><TAB> behaviour with regard to tilde/username completion to be the same in the command prompt and in the copy/move dialog. And I would clearly prefer the command prompt behaviour in the copy/move dialog, and not the other way round. ;-)

[andrew_b]

Replying to zaytsev:

No. We lookup a user's name, not a user's home directory.

So why wouldn't we look up the home directory as well, it's there as pw_dir, right? I think that tilde expansion in path completion would be a nice feature, so I'd keep the ticket open...

tilde expansion in path completion is working. If you type ~/ and press ESC-TAB you can see all directories and files in your $HOME.

[ZlatkO]

Replying to andrew_b:

tilde expansion in path completion is working. If you type ~/ and press ESC-TAB you can see all directories and files in your $HOME.

That's right. Given that "plain tilde expansion" is (for all intents and purposes) just a special case of "username tilde expansion" (in the sense that if you don't give it (part of) a username but only the tilde itself, then it takes $USER as username by default [*]), this behaviour is consistent with the fact that if I do ~otheruser/<ESC><TAB> (note the trailing slash) in the copy/move dialog, then MC does the rest of the completion just fine, as I stated in my original report. Also if I do ~/<ESC><TAB> (again, with a trailing slash), further completion is fine.

Note that this is different to the command prompt behaviour, where if I do ~<ESC><TAB> (without the trailing slash), then I do get a selection popup containing all different user home directores available on my system instead, see screenshot. And that's exactly the part that does not work in the copy/move dialog box.

[*] On my local workstation, for example, ~/ is equivalent to ~zlatko/. If I do "su - gdm", then ~/ is equivalent to ~gdm/, and so on.

[ZlatkO]

complete list of available user home dirs on command prompt

[andrew_b]

Again: these are not home dirs. These are user names.

[ossi]

Replying to andrew_b:

Again: these are not home dirs. These are user names.

and the argument you're making is ...?

[ZlatkO]

Replying to andrew_b:

Again: these are not home dirs. These are user names.

The third screenshot I attached clearly does show directories. "~zlatko/" is my home dir. My username is "zlatko", my home dir is "~zlatko/" (which is "/home/zlatko"). GDM's username is "gdm", GDM's home dir is "~gdm/" (which is "/usr/local/var/gdm"). DBUS' username is "messagebus", DBUS' home dir is "~messagebus/" (which is "/usr/local/var/run/dbus").

Yes, every single one of these directories is composed by "~" + username + "/". But that does not make them "non-directories" all of a sudden, they are still directories. You can cd into them, ls their contents, and do all the other good stuff that directories were created for.

In terms of semantics, I can continue the completion in MC at this point to select, say, an executable located in one of the users' home dir and hit enter, so the executable gets executed. I do not see how I could possibly do this if MC would actually see this list as a list of usernames. Run an executable located within a username?! ;-)

Bash doesn't even need the trailing slash:

[zlatko@disclosure:~]$ file zlatko
zlatko: cannot open `zlatko' (No such file or directory)
[zlatko@disclosure:~]$ file ~zlatko
/home/zlatko: directory
[zlatko@disclosure:~]$ file ~messagebus/
/usr/local/var/run/dbus/: directory
[zlatko@disclosure:~]$ 

I'm really sorry if I come across as completely stubborn here, but I just don't get why you insinst on MC's selection box (my third screenshot) containing usernames and not directories. I somehow have the slight suspicion that you actually mean something totally different, but I just can't seem to wrap my head around where exactly our trains of thought diverge. Please bear with me.

[andrew_b]

Replying to ZlatkO:

I just don't get why you insinst on MC's selection box (my third screenshot) containing usernames and not directories.

Because I see the code: pw_name not pw_dir is used. Probably this is a mistake, but it is more than 20 years old.

[ossi]

Probably this is a mistake

no, it's not. it completes user names as part of filename completion, as the filename will be subjected to tilde expansion by the consumer. this really isn't all that hard ...

[ZlatkO]

Aaaaaah, now we're getting somewhere! So, by looking at the code, it seems that we are both correct. It is in fact a list of usernames, but prepended with a tilde "~" character, and appended with a path separator - which, by summoning arcane Unix magic & shell conventions, turns it into a list of perfectly usable directories.

While you were only looking at the code, completely ignoring the UI, I did the exact opposite, only looking at what the UI showed me, completely ignoring the underlying code. I'm glad we could figure that out, I already started to question my sanity (well, not really :-D).

Also, I don't think that it's a mistake to use pw_name here, it's perfectly fine. I think filling the popup box with "~" + pw_name + PATH_SEP_STR is cleaner and easier to read than using the actual resulting value of pw_dir. It would also be confusing to start typing, say, "~apa<ESC><TAB>", and suddenly have "/srv/httpd" or "/var/www/html" as an auto-completion. Plus you couldn't really <BACKSPACE> properly to change your typing from "~apache/" to, say, "~adm/" or "~avahi/".

What I could imagine, though, would be to simply display the actual value of pw_dir for each of the possible completions, but make it non-editable. Taking my first screenshot as an example, instead of having the popup display just ...

~games/
~gdm/
~gnome/

... it could display ...

~games/ [/usr/games]
~gdm/   [/usr/local/var/gdm]
~gnome/ [/home/gnome]

... or use round brackets () instead of square [] ones, or whatever. The required string values would already be availabe in the corresponding pw_dir fields while building the list from the pw_name fields.

Soooooo ... with that huge confusion finally out of the way, what do you say to my original request? :-)