ftp failures - leading white space in file name

[guido]

In 4.8.0-pre1, ftp downloads fail with the error message "Cannot read ftp...Permission denied (13)", IF the panel listing shows the file name indented with white space. The dir is correctly displayed.

This is most noticeable on sites that have CD images (*.iso) ready to burn, although other dirs cause problems, too, when indented with spaces.

As an example check out ​ftp://ftp.kernel.org/pub/dist/knoppix.

Properly left justified names can be retrieved just fine.

This never occurred with my last version 4.7.5.2.

[slavazanko]

This happens after #81.

Main think of #81 is: remember position of first '..' (two dots directory) in 'ls'command output, then apply this position to names of other files in directory. For example:

-rwxr-xr-x    1 0        0            4096 Mar 02 16:30 ..
-rw-r--r--    1 0        0       188743680 Mar 02 16:30 file1.ext
-rw-r--r--    1 0        0             189 Mar 03 10:52 file2.ext
-rw-r--r--    1 0        0              63 Mar 09 10:35   file3.ext
-rw-r--r--    1 0        0              71 Mar 09 10:35 file4.ext

in ths case before #81 we got 'file3.ext' (because string was split by spaces), after #81 filename is ' file3.ext' (with two leading spaces) and it's correct.

But if output didn't formatted correctly and columns positions not equal between lines of 'ls' output we'll have troubles with file names. For expample:

-rwxr-xr-x    1 0        0            4096 Mar 02 16:30 ..
-rw-r--r--    1 0        0         188743680 Mar 02 16:30 file1.ext
-rw-r--r--    1 0        0             189 Mar 03 10:52 file2.ext
-rw-r--r--    1 0        0              63 Mar 09 10:35   file3.ext
-rw-r--r--    1 0        0            71 Mar 09 10:35 file4.ext

In this case 'file1.ext' will be '0 file1.ext', ' file3.ext' will be right file name and 'file4.ext' will be 'le4.ext' (see position of two dots). For example, see ftp.oregonstate.edu/pub/opensuse/distribution/11.4/iso. Open this url in latest mc(with appled #81 patches) and in any other ftp-client such as lftp:

lftp -e 'open ftp.oregonstate.edu; cd /pub/opensuse/distribution/11.4/iso; ls; quit'

I don't know how resolve this issue, may be, need to calculate number of spaces between 'time' field and 'two dots' directory name, then calculating needed number of spaces in another filenames in 'ls' command output... Is this right way?

[guido]

The bash command 'ls -al' produces a variable length string for each item in the
directory. It has 9 white-space delimited fields. That string should be properly
parsed with string manipulation routines.

We should NOT do column alignment arithmetic on the last field to align all with
'.' & '..' entries.

Attached is a bash example script, lsl.sh. Obviously, the same needs to be done
in c for mc.

Who is the maintainer of the relevant mc modules? He or she should have a look
at the original code. Remember, this used to work in earlier mc releases.



lsl.sh script

#!/bin/bash

LS=(
'drwxrwxrwt 5 root root 4096 Aug 18 09:19 .'
'drwxr-xr-x 17 root root 4096 Aug 17 10:49 ..'
'-rw-r--r-- 1 536 536 4001599488 Aug 01 15:52

KNOPPIX_V6.7.0DVD-2011-08-01-EN.iso'

'-rw-r--r-- 1 536 536 70 Aug 01 20:06

KNOPPIX_V6.7.0DVD-2011-08-01-EN.iso.md5'

'-rw-r--r-- 1 536 536 385468 Aug 02 19:38 packages-dvd.txt'
)

echo "Output from some ls -l; fields are NOT aligned"
for (( L=0 ; L<6 ; L++ ))

do

LINE=${LS[L]}
echo $LINE

done

echo "Line parsed into an array; element 8 is file name"
for (( L=0 ; L<6 ; L++ ))

do

LINE=( ${LS[L]} )
LINEARRAY=( $LINE )
FILENAME=${LINE[8]} #element numbers start at 0
echo $FILENAME

done

echo "Note that there is now NO white space."

[asy]

what do you think about regexp ?

echo "-rw-r--r--    1 0      0      63 Mar 09 10:35   file3.ext" | sed -e "s/^[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} //"

[guido]

Try this bash script, lsl2.sh

#!/bin/bash

printf "\nsed\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
sed -e "s/^{[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} "
printf "2 leading spaces?\n"}

printf "\ncut + rev\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
rev | cut -f 1 -d ' ' - | rev

printf "\ngawk\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
gawk '{ print $9 }'
echo

#But, MC is written in 'c', so we should be using its native libs,
#not, the Linux basic command library.

[ossi]

Replying to asy:

sed -e "s/^[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} //"

the attribute field can contain a few more different characters.
i'm not sure this is fully portable, anyway; when i did such stuff a decade ago with ultrix boxes from the pre-stone-age, i had to special-case it all the time due to different columns/counts. and in the worst case, excessively long fields can run into others (the gnu tools prefer to blow the alignment, which is way saner). but then, who cares.

Replying to guido:

But, MC is written in 'c', so we should be using its native libs,
not, the Linux basic command library.

which is a good thing, as there is no need to actually mess with regexps. ;)

[asy]

Replying to guido:

printf "2 leading spaces?\n"

Yes. Please see #81. There describes files with leading spaces

Replying to ossi:

the attribute field can contain a few more different characters.

It is not a problem :-)

i'm not sure this is fully portable

This regular expression can be replaced by a simple string processing. It is not very complicated. But I thought about GCC Regex Library

[andrew_b]

We can try the ls -la parser from gnome-vfs: ​http://git.gnome.org/browse/gnome-vfs/tree/libgnomevfs/gnome-vfs-parse-ls.c.

[guido]

That looks like a good candidate. A cursory glance at the code seems to indicate that the mc parser originally came from the gnome module.

[ossi]

Replying to guido:

A cursory glance at the code seems to indicate that the mc parser originally came from the gnome module.

i suggest you have a glance at the history of gnome. :D

[guido]

I restate my claim as a question. Who originally wrote the vfs_parse_ls routines?

[slavazanko]

Who originally wrote the vfs_parse_ls routines?

I think, you may directly ask Miguel de Icaza :)

So, created branch 2594_ftp_leading_spaces, initial changeset:c9db2a784c40e697a27253b690597de3c7a4fe8a

Review, please. I hope, this branch will be really helpfull.

Btw, we have fixed trouble with anonymous access to our repo:

git clone git://www.midnight-commander.org/git/mc.git

git-daemon was listen just on ipv6-only address (after upgrading git on server). Sorry for inconvenience.

[slavazanko]

Merge changeset:d75cb6efb667c3b74456cc9ea3d68204d654a664

[slavazanko]

Cherry-picked to stable:

git log --pretty=oneline 0ebdee3bc^..1fb763a63