Ticket #2594 (closed defect: fixed)

Opened 8 years ago

Last modified 8 years ago

ftp failures - leading white space in file name

Reported by: guido Owned by: slavazanko
Priority: critical Milestone: 4.8.0-pre2
Component: mc-vfs Version: 4.8.0-pre1
Keywords: VFS ftp leading spaces Cc:
Blocked By: Blocking:
Branch state: merged Votes for changeset: commited-master commited-stable

Description

In 4.8.0-pre1, ftp downloads fail with the error message "Cannot read ftp...Permission denied (13)", IF the panel listing shows the file name indented with white space. The dir is correctly displayed.

This is most noticeable on sites that have CD images (*.iso) ready to burn, although other dirs cause problems, too, when indented with spaces.

As an example check out ftp://ftp.kernel.org/pub/dist/knoppix.

Properly left justified names can be retrieved just fine.

This never occurred with my last version 4.7.5.2.

Attachments

lsl.sh (715 bytes) - added by guido 8 years ago.

Change History

comment:1 Changed 8 years ago by slavazanko

  • Owner set to slavazanko
  • Status changed from new to accepted

comment:2 Changed 8 years ago by slavazanko

This happens after #81.

Main think of #81 is: remember position of first '..' (two dots directory) in 'ls'command output, then apply this position to names of other files in directory. For example:

-rwxr-xr-x    1 0        0            4096 Mar 02 16:30 ..
-rw-r--r--    1 0        0       188743680 Mar 02 16:30 file1.ext
-rw-r--r--    1 0        0             189 Mar 03 10:52 file2.ext
-rw-r--r--    1 0        0              63 Mar 09 10:35   file3.ext
-rw-r--r--    1 0        0              71 Mar 09 10:35 file4.ext

in ths case before #81 we got 'file3.ext' (because string was split by spaces), after #81 filename is ' file3.ext' (with two leading spaces) and it's correct.

But if output didn't formatted correctly and columns positions not equal between lines of 'ls' output we'll have troubles with file names. For expample:

-rwxr-xr-x    1 0        0            4096 Mar 02 16:30 ..
-rw-r--r--    1 0        0         188743680 Mar 02 16:30 file1.ext
-rw-r--r--    1 0        0             189 Mar 03 10:52 file2.ext
-rw-r--r--    1 0        0              63 Mar 09 10:35   file3.ext
-rw-r--r--    1 0        0            71 Mar 09 10:35 file4.ext

In this case 'file1.ext' will be '0 file1.ext', ' file3.ext' will be right file name and 'file4.ext' will be 'le4.ext' (see position of two dots). For example, see ftp.oregonstate.edu/pub/opensuse/distribution/11.4/iso. Open this url in latest mc(wuth appled #81 patches) and in any other ftp-client such as lftp:

lftp -e 'open ftp.oregonstate.edu; cd /pub/opensuse/distribution/11.4/iso; ls; quit'

I don't know how resolve this issue, may be, need to calculate number of spaces between 'time' field and 'two dots' directory name, then calculating needed number of spaces in another filenames in 'ls' command output... Is this right way?

Version 3, edited 8 years ago by slavazanko (previous) (next) (diff)

Changed 8 years ago by guido

comment:3 Changed 8 years ago by guido

The bash command 'ls -al' produces a variable length string for each item in the
directory. It has 9 white-space delimited fields. That string should be properly
parsed with string manipulation routines.

We should NOT do column alignment arithmetic on the last field to align all with
'.' & '..' entries.

Attached is a bash example script, lsl.sh. Obviously, the same needs to be done
in c for mc.

Who is the maintainer of the relevant mc modules? He or she should have a look
at the original code. Remember, this used to work in earlier mc releases.



lsl.sh script

#!/bin/bash

LS=(
'drwxrwxrwt 5 root root 4096 Aug 18 09:19 .'
'drwxr-xr-x 17 root root 4096 Aug 17 10:49 ..'
'-rw-r--r-- 1 536 536 4001599488 Aug 01 15:52

KNOPPIX_V6.7.0DVD-2011-08-01-EN.iso'

'-rw-r--r-- 1 536 536 70 Aug 01 20:06

KNOPPIX_V6.7.0DVD-2011-08-01-EN.iso.md5'

'-rw-r--r-- 1 536 536 385468 Aug 02 19:38 packages-dvd.txt'
)

echo "Output from some ls -l; fields are NOT aligned"
for (( L=0 ; L<6 ; L++ ))

do

LINE=${LS[L]}
echo $LINE

done


echo "Line parsed into an array; element 8 is file name"
for (( L=0 ; L<6 ; L++ ))

do

LINE=( ${LS[L]} )
LINEARRAY=( $LINE )
FILENAME=${LINE[8]} #element numbers start at 0
echo $FILENAME

done

echo "Note that there is now NO white space."

comment:4 Changed 8 years ago by asy

what do you think about regexp ?

echo "-rw-r--r--    1 0      0      63 Mar 09 10:35   file3.ext" | sed -e "s/^[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} //"

comment:5 follow-ups: ↓ 6 ↓ 7 Changed 8 years ago by guido

Try this bash script, lsl2.sh

#!/bin/bash

printf "\nsed\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
sed -e "s/[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} "
printf "2 leading spaces?\n"

printf "\ncut + rev\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
rev | cut -f 1 -d ' ' - | rev

printf "\ngawk\n"
echo "-rw-r--r-- 1 0 0 63 Mar 09 10:35 file3.ext" | \
gawk '{ print $9 }'
echo

#But, MC is written in 'c', so we should be using its native libs,
#not, the Linux basic command library.

comment:6 in reply to: ↑ 5 Changed 8 years ago by ossi

Replying to asy:

sed -e "s/^[drwx-]*\( *[A-Za-z0-9:]*\)\{7\} //"

the attribute field can contain a few more different characters.
i'm not sure this is fully portable, anyway; when i did such stuff a decade ago with ultrix boxes from the pre-stone-age, i had to special-case it all the time due to different columns/counts. and in the worst case, excessively long fields can run into others (the gnu tools prefer to blow the alignment, which is way saner). but then, who cares.

Replying to guido:

But, MC is written in 'c', so we should be using its native libs,
not, the Linux basic command library.

which is a good thing, as there is no need to actually mess with regexps. ;)

comment:7 in reply to: ↑ 5 Changed 8 years ago by asy

Replying to guido:

printf "2 leading spaces?\n"

Yes. Please see #81. There describes files with leading spaces

Replying to ossi:

the attribute field can contain a few more different characters.

It is not a problem :-)

i'm not sure this is fully portable

This regular expression can be replaced by a simple string processing. It is not very complicated. But I thought about GCC Regex Library

comment:8 Changed 8 years ago by andrew_b

comment:9 follow-up: ↓ 10 Changed 8 years ago by guido

That looks like a good candidate. A cursory glance at the code seems to indicate that the mc parser originally came from the gnome module.

comment:10 in reply to: ↑ 9 Changed 8 years ago by ossi

Replying to guido:

A cursory glance at the code seems to indicate that the mc parser originally came from the gnome module.

i suggest you have a glance at the history of gnome. :D

comment:11 Changed 8 years ago by guido

I restate my claim as a question. Who originally wrote the vfs_parse_ls routines?

comment:12 Changed 8 years ago by slavazanko

  • Keywords VFS ftp leading spaces stable-candidate added
  • Branch state changed from no branch to on review
  • Milestone changed from Future Releases to 4.8.0-pre2

Who originally wrote the vfs_parse_ls routines?

I think, you may directly ask Miguel de Icaza :)

So, created branch 2594_ftp_leading_spaces, initial changeset:c9db2a784c40e697a27253b690597de3c7a4fe8a

Review, please. I hope, this branch will be really helpfull.

Btw, we have fixed trouble with anonymous access to our repo:

git clone git://www.midnight-commander.org/git/mc.git

git-daemon was listen just on ipv6-only address (after upgrading git on server). Sorry for inconvenience.

comment:13 Changed 8 years ago by andrew_b

  • Votes for changeset set to andrew_b

comment:14 Changed 8 years ago by angel_il

  • Votes for changeset changed from andrew_b to andrew_b angel_il
  • Branch state changed from on review to approved

comment:15 Changed 8 years ago by slavazanko

  • Status changed from accepted to testing
  • Votes for changeset changed from andrew_b angel_il to commited-master
  • Resolution set to fixed
  • Branch state changed from approved to merged

comment:16 Changed 8 years ago by slavazanko

  • Status changed from testing to closed
  • Keywords stable-candidate removed
  • Votes for changeset changed from commited-master to commited-master commited-stable

Cherry-picked to stable:

git log --pretty=oneline 0ebdee3bc^..1fb763a63
Note: See TracTickets for help on using tickets.