Search in internal viewer is very slow

[bilbo]

I am searching for simple word in internal viewer in 24 Mb text file. Internal mc viewer needs 22 seconds when positioned at start of the file to reach end of the file and tell me the word is not found.

I am using normal, case sensitive search (no regexps or alike)

grep needs only 31 milliseconds for the same.

Also, if I use Find file function in MC to search for the string (typing the filename and the search string in appropriate boxes), it takes only about one second to search the file - so this type of search is fast.

I am using fresh GIT version of MC

[iNode]

What the file name which you try to view?

Isn't it has some .N (where N is 0..9) extension
or any other that mcview can try to preformat?

Search in Raw mode (F8 in mcview) also slow?

[bilbo]

The name of the file is "Inbox" (the file is mailbox in unix mailbox format). I am not aware of mcview doing any preformatting (it is not manpage or anything with same extension as manpage).

When I switch from "Parse" to "Raw" and back, the performance is the same. Also, the search can't be interrupted by Esc, Ctrl-C or any other means.

[bilbo]

I tried running callgrind and I found out that for each byte of file, there is one call to mc_search_get_char, view_search_cmd_callback, view_read_continue, view_get_char and str_translate_char. Also, for each byte there are two (two calls to convert charset in one byte?) calls to g_iconv - these account for 65% of eaten CPU time - perhaps characters should get iconv'ed using some buffer? Maybe buffer could be used for other calls too (some "view_get_chars_into_buffer" instead of view_get_char?), so the overhead of calling a function would be done only once per reasonably sized buffer (few Kb?) and nor per each byte.

[ossi]

the approach to convert the data from the file buffer is not particularly efficient; doing it char by char is insane. it would be much more efficient to recode the search strings into a representation which might appear in the actual file and then do a simple byte-based search directly on the file buffer.

[angel_il]

it's trouble not search engine, but viewer.

[bilbo]

Simple string search is quite important finction in a text viewer.

I have also noticed, that when I merely press End key to scroll to end of the 24 Mb file, it needs about 11 seconds (half the time of searching) to get there (time proportinally increases with file size, so looking at end of 1 GB XML dump needs several minutes). This is also too slow. Older mcview (4.6.2) is fast (searching and scrolling in the file are both done in less than a second), so I think this is regression since that version.

[slavazanko]

This was fixed into #1557.

[marrtins]

I'm using mc-4.7.0-pre4 and it seems that this is *not* fixed yet :( Hitting END button and scrolling to the end of large file takes huge amount of time.

[richlv]

not sure whether it's related, but mcedit is also very slow in latest releases.
whencompared with older versions, it notably slows down when scrolling in the file, and going to the end of a file (even one that's a few kb large) happens after some delay.
does that have the same root cause or should a new report be filed ?
these slowdowns are especially easy to notice for longtime mc users who are accustomised to f3/down combination on several hundred meg logfiles :)

[slavazanko]

not sure whether it's related, but mcedit is also very slow in latest releases.

Yep, this happens after adding UTF-8 support into upstream.

does that have the same root cause or should a new report be filed ?

Nope. This ticket about slow search in viewer. Create new report, please.

defect to enhancement

Dislike.

[richlv]

Replying to slavazanko:
...

does that have the same root cause or should a new report be filed ?

Nope. This ticket about slow search in viewer. Create new report, please.

thanks. bug against mcedit reported as #1843
...

[slavazanko]

I am searching for simple word in internal viewer in 24 Mb text file.
Internal mc viewer needs 22 seconds when positioned at start of the
file to reach end of the file and tell me the word is not found.

Well... as fact, this is trouble described in #1585. You may check it:

1) echo "FIND_ME" >>big_file.txt
2) open big_file.txt in viewer
3) press 'End' key. Viewer after some pause will positioning at end of file.
4) press 'Home' key. Viewer will return at start of file.
5) Press F7 and enter search string "FIND_ME"

Search will much faster in this test case. Slowdown in mcview_ccache_* functions... IMHO :)

Illiya: I reassign ticket to you. After fix #1585 need to recheck test cases again.

[slavazanko]

Well, search now fast :)

[angel_il]

fixed in #1585