Find File "Whole words" search bug

[x906]

when searching in files for non english word with "Whole words" set "on" - then nothig will be found
try search word: "время" and also "time" in attached file
mc ver: 4.7.4-90-g1e265ea

[gotar]

Works fine with polish diacritics and ISO-8859-2 locale (LC_CTYPE to be exact). Are you using UTF-8? Try with KOI8-R.

[gotar]

However it doesn't work for me in mcedit and mcview.

[x905]

Replying to gotar:

Are you using UTF-8? Try with KOI8-R.

yes, UTF-8
KOI8-R is not acceptable for me

[andrew_b]

It seems, we have a global bug in search engine. For me, search of whole non-ASCII words (cyrillic, for example) doesn't work ar all: neither in files neither in editor nor in viewer.

[andrew_b]

Replying to andrew_b:

doesn't work ar all

doesn't work at all

[gotar]

x905 - I suspected it may be UTF-8 related, but apparently it's not
andrew_b - what is weird that it works for me (latin2 characters) in Find file (and only there)

[andrew_b]

Replying to gotar:

andrew_b - what is weird that it works for me (latin2 characters) in Find file (and only there)

It doesn't work with russian cyrillic (as KOI8-R as UTF-8) words (for example, "время" in ticket text). Moreover, search using "\bвремя\b" regular expression ("Regular expression" is on, "Whole words" is off) also doesn't find anything.

[slavazanko]

As i found, non-latin chars isn't in word: ​http://www.regular-expressions.info/wordboundaries.html

"In all flavors, the characters [a-zA-Z0-9_] are word characters"

I don't know how fix this trouble, sorry :(

gotar: is 'Search whole words' works with 'ą,ć,ę,ł,ń,ó,ś,ź,ż' letters (and with uppercase analogs)?

[x905]

Replying to slavazanko:

I don't know how fix this trouble, sorry :(

maybe look at source of grep?
this works: grep -iw "время" ./f1

[andrew_b]

Replying to x905:

maybe look at source of grep?

Yes :). We found a soluton in grep:

      static char const word_beg[] = "(^|[^[:alnum:]_])(";
      static char const word_end[] = ")([^[:alnum:]_]|$)";

This works for me in KOI8-R.

[slavazanko]

Created branch 2396_find_whole_words (parent: master)

changeset:c859e906d0bd6e91b1e52a8002c98de043dcb817

All other changeset is typo fixes and code refactoring:

• 466b34b8fb1ebf30e19dc30ccd75c85623f85aef: Code cleanup for avoid compiler warnings
• 4af1277e4fc7e093de1b72ac2d5fa02bd4c30413: Fixed bit operations in mc_search_regexprocess_append_str()
• 2793a312492079fe99ad56bfeb81802886a1a5d6: Removed mc_search_cond_t->len (used mc_search_cond_t->str->len instead).
• 100df42d9578af79ce4b80d5a6572d9a81fbb683: Avoid extra-allocation of string while prepare to regexp-search.

Review, please.

x905: thanks for tip :)

[x905]

not work: in new attached file (f2) mc finds all first 1-6 lines
but grep also fail on line 6 :(

mc 4.7.4-103-g4e2ffca

also found another bug in this version: then press F1, then error window appears: "Cannot open file /usr/local/share/mc/help/mc.hlp"

[slavazanko]

See new start changeset:f46302b2651bee6246f9f3349cdbbb67144fb284
Bug should be fixed :)

Review branch again, please.

[x905]

better, but not complete - line "6. невремя" still in search results
(4.7.4-102-g02acc44)

[andrew_b]

Replying to x905:

also found another bug in this version: then press F1, then error window appears: "Cannot open file /usr/local/share/mc/help/mc.hlp"

Did you run new mc binary with old mc environment, i.e. without installation? If yes, this is not a bug. Some files changed their locations (#1424):

Install help files into /usr/share/mc/help instead of /usr/share/mc.
Install hint files into /usr/share/mc/hints instead of /usr/share/mc.

[x905]

yes, with help is my fault - i do sudo make install, but has another instances of mc running

[gotar]

Replying to slavazanko:

As i found, non-latin chars isn't in word: ​http://www.regular-expressions.info/wordboundaries.html

"In all flavors, the characters [a-zA-Z0-9_] are word characters"

gotar: is 'Search whole words' works with 'ą,ć,ę,ł,ń,ó,ś,ź,ż' letters (and with uppercase analogs)?

Yes and no:
yes - it does find my string (in 'find file', not mcedit or mcview)
no - it treats every letter as separate word (i.e. despite of 'Search whole words' any substring of example 'ąćęśłżźńó' is being found, which means that all the characters are treated as word boundaries).

[slavazanko]

Okay, check branch now. I have changed regexp for emulating '\b' behaviour (changeset:d5aa913edffc824075c72bcdd6411657df91f347). Hope this helps...

Review again, please.

[andrew_b]

Fine. This is works for me.

[x905]

works

[andrew_b]

Don't forget update po files after merge to master.

[slavazanko]

Merged to master: b60f00df0d8d1d52840ad81ed6529672957d555c
Updated *.po files: 5bf5dd170e00a4dc8e3fbf862dfbc23c1774a79e

[shirsch]

Running 4.7.0.9 on Ubuntu Lucid x86_64. String search is "intermittant". It works on some files and not others. When not working, it can find nothing.

[slavazanko]

Running 4.7.0.9 on Ubuntu Lucid x86_64. String search is "intermittant". It works on some files and not others. When not working, it can find nothing.

Stop reopen! :)

Bug was fixed in our 'master' branch (in repository) and fix will included in near future release (4.7.0.10 in your case). Just await for new version. ;)

[slavazanko]

Cherry-picked in stable branch:

• a4673b9256e379527f89c198de9157178af8bf15
• 0c6fedf2582a68678eae9604ba24f42c4347fdf4
• 1eabbc1e0214779dec2b0f046a5a078bb3963c4d

[gotar]

Replying to slavazanko:

Stop reopen! :)

So shouldn't you changed resolution to fixed again? ;)

[andrew_b]

Replying to gotar:

So shouldn't you changed resolution to fixed again? ;)

Done.

[egmont]

Commenting on this 5 year old ticket:

The fix caused a regression, in fact it further broke a feature that was already broken a bit: Regex search with Whole words enabled. Before this patch it just ignored Whole words, now it doesn't even highlight the match.

The regex could be made simpler by using lookahead/lookbehind. If using that, coincidentally, the further regression doesn't occur.

At this point, for me totally reverting the patch back to using \b doesn't break anything: the originally reported bug is not reproducible, it works as expected. Could be a fix in glib in the last 5 years, or anything else.

Details in #3524.

Please let me know if you know how to reproduce the bug with the change being reverted.