case insensitive sort puts entries in panel in unexpected order

[yury_t]

If "Case sensitive" is unchecked on the "Sort options", then entries get sorted not only disregarding the case, but also discarding the leading non-alpha characters.

Reproducing:

1. In some dir, make dir +++b
2. In the same dir, make dir a

3.1. With "case sensitive" ON (and "reverse" OFF) these entries are output in such order: +++b, then a. That is a correct order.
3.2. With "case sensitive" OFF these entries are output in such order: a, then +++b. So, the procedure seems to ignore the leading non-alphas, which is INCORRECT.

[andrew_b]

What is your locale?

[yury_t]

The locale is be_BY.UTF-8. However, I don't think the locale is to blame.

[andrew_b]

I see the same in ru_RU.KOI8-R.

Sort procedure doesn't ignore anything. In general, MC uses strcmp(3) for case sensitive sort (filenames are converted to lower case before comparision).

In case insensitive sort, strcoll(3) is used, and filenames are used as is without any modifications. Therefore sort result is depends on strcoll() return value.

[andrew_b]

Replying to andrew_b:

In general, MC uses strcmp(3) for case sensitive sort (filenames are converted to lower case before comparision).

In case insensitive sort, strcoll(3) is used, and filenames are used as is without any modifications. Therefore sort result is depends on strcoll() return value.

Sorry, I was wrong.

In case sensitive sort, the strcmp(3) is used, and native filenames are used.

In case insensitive sort, the strcoll(3) is used, and filenames are converted to lower case before comparision.

[yury_t]

So, what's the final word on this? Will MC sort the entries correctly with case insensivity on?

[yury_t]

I suppose this incorrect working of strcoll(3) may be attributed to the collation tables in UTF-8 locales in newer distributions of glibc (e.g., as packaged with the slackware 13.0 but not with the Slackware 12.2).

[yury_t]

Ultimately, yes, this is indeed caused by the incorrect collation tables in the UTF-8 locales. Using the MC with the LC_COLLATE=C produces correct sort order, at least for the ASCII names.

Could the case insensitive sort be made to use the strcmp with tolower or something?

[wayfarer]

In version 2.8 of GLib new function "g_utf8_collate_key_for_filename" was introduced, exactly for this purpose - to have correct sorting of filenames:

​http://library.gnome.org/devel/glib/unstable/glib-Unicode-Manipulation.html#g-utf8-collate-key-for-filename

Unfortunately, it still doesn't consider all special chars, but at least dots are handled properly - and it's a big part of a problem. So maybe it is worth using?

It's really the collation table, which should be blamed: as far as I understand, in all locales special symbols are just ignored during collation. In C locale it works right, cause there collation order is just the same as order of standard ASCII codes.

OTOH, it's probably a right solution for general-purpose sorting to ignore special chars. But sorting file names has its specialties. So IMHO using above mentioned function and asking GLib guys to extend it to deal with other chars would be the better solution than trying to fix collation tables.

[andrew_b]

Replying to wayfarer:

In version 2.8 of GLib new function "g_utf8_collate_key_for_filename" was introduced, exactly for this purpose - to have correct sorting of filenames:
​http://library.gnome.org/devel/glib/unstable/glib-Unicode-Manipulation.html#g-utf8-collate-key-for-filename

Unfortunately, it still doesn't consider all special chars, but at least dots are handled properly - and it's a big part of a problem. So maybe it is worth using?

Unfortunately, g_utf8_collate_key_for_filename() doesn't provide the correct sorting order. Look at the #1536. That's why we don't use this function.

[wayfarer]

Replying to andrew_b:

Unfortunately, g_utf8_collate_key_for_filename() doesn't provide the correct sorting order. Look > at the #1536. That's why we don't use this function.

Hm, but they did that trick with numbers intentionally. I think, different opinions about what "correct sorting order" is, are possible here.

But the root of all these problems, I suppose, is that "case sensitive" and "case insensitive" sorting in MC are inherently inconsistent, since different methods are used. Is it possible to use the same method, whether string comparison or collation, in both "sensitive" and "insensitive" modes? In ideal case, the exact behavior could be selected using MC settings/config params.

Cause solution provided in #1894 seems just like a workaround for one specific case.

[flocsy]

13 years later...

I use mc 4.8.28 on MacOS (13.1 Ventura) installed from homebrew, and still the case-insensitive sort is "random", even when talking about ASCII and even English letters like "a" and "c"

For me even setting LC_COLLATE=C didn't help

[mjosifek]

I use brew package (4.8.30) on macOS 13.5.2 and I can confirm that sort order is "random" if case-insensitive is enabled. Eg. filenames starting with c or C are always first. LANG=en_US.UTF-8

[vfxsup]

I use version 4.8.31 on MacOS 14.4.1 and I'm experiencing the same problem as the previous commenter. "Creative Cloud Files" comes before "Applications" (followed by "Desktop") when case-sensitive sort is disabled.

This is a pretty major defect in my opinion.

[zaytsev]

So Andrew maybe we could indeed lowercase stuff or something like that or what do you think?