Ticket #1417 (closed defect: wontfix)

Opened 10 years ago

Last modified 10 years ago

Search in internal viewer is very slow

Reported by: bilbo Owned by: angel_il
Priority: critical Milestone: 4.7
Component: mcview Version: 4.7.0-pre4
Keywords: Cc: richlv@…
Blocked By: Blocking:
Branch state: Votes for changeset:

Description

I am searching for simple word in internal viewer in 24 Mb text file. Internal mc viewer needs 22 seconds when positioned at start of the file to reach end of the file and tell me the word is not found.

I am using normal, case sensitive search (no regexps or alike)

grep needs only 31 milliseconds for the same.

Also, if I use Find file function in MC to search for the string (typing the filename and the search string in appropriate boxes), it takes only about one second to search the file - so this type of search is fast.

I am using fresh GIT version of MC

Change History

comment:1 Changed 10 years ago by iNode

What the file name which you try to view?

Isn't it has some .N (where N is 0..9) extension
or any other that mcview can try to preformat?

Search in Raw mode (F8 in mcview) also slow?

comment:2 Changed 10 years ago by bilbo

The name of the file is "Inbox" (the file is mailbox in unix mailbox format). I am not aware of mcview doing any preformatting (it is not manpage or anything with same extension as manpage).

When I switch from "Parse" to "Raw" and back, the performance is the same. Also, the search can't be interrupted by Esc, Ctrl-C or any other means.

comment:3 Changed 10 years ago by bilbo

I tried running callgrind and I found out that for each byte of file, there is one call to mc_search_get_char, view_search_cmd_callback, view_read_continue, view_get_char and str_translate_char. Also, for each byte there are two (two calls to convert charset in one byte?) calls to g_iconv - these account for 65% of eaten CPU time - perhaps characters should get iconv'ed using some buffer? Maybe buffer could be used for other calls too (some "view_get_chars_into_buffer" instead of view_get_char?), so the overhead of calling a function would be done only once per reasonably sized buffer (few Kb?) and nor per each byte.

comment:4 Changed 10 years ago by ossi

the approach to convert the data from the file buffer is not particularly efficient; doing it char by char is insane. it would be much more efficient to recode the search strings into a representation which might appear in the actual file and then do a simple byte-based search directly on the file buffer.

comment:5 Changed 10 years ago by angel_il

it's trouble not search engine, but viewer.

comment:6 Changed 10 years ago by bilbo

Simple string search is quite important finction in a text viewer.

I have also noticed, that when I merely press End key to scroll to end of the 24 Mb file, it needs about 11 seconds (half the time of searching) to get there (time proportinally increases with file size, so looking at end of 1 GB XML dump needs several minutes). This is also too slow. Older mcview (4.6.2) is fast (searching and scrolling in the file are both done in less than a second), so I think this is regression since that version.

comment:7 Changed 10 years ago by slavazanko

  • Status changed from new to closed
  • Resolution set to wontfix

This was fixed into #1557.

comment:8 Changed 10 years ago by slavazanko

  • Milestone changed from 4.7 to 4.7.0-pre2

comment:9 Changed 10 years ago by marrtins

  • Priority changed from major to critical
  • Status changed from closed to reopened
  • Version changed from master to 4.7.0-pre4
  • Resolution wontfix deleted

I'm using mc-4.7.0-pre4 and it seems that this is *not* fixed yet :( Hitting END button and scrolling to the end of large file takes huge amount of time.

comment:10 Changed 10 years ago by richlv

  • Cc richlv@… added

not sure whether it's related, but mcedit is also very slow in latest releases.
whencompared with older versions, it notably slows down when scrolling in the file, and going to the end of a file (even one that's a few kb large) happens after some delay.
does that have the same root cause or should a new report be filed ?
these slowdowns are especially easy to notice for longtime mc users who are accustomised to f3/down combination on several hundred meg logfiles :)

comment:11 Changed 10 years ago by andrew_b

  • Milestone changed from 4.7.0-pre2 to 4.7

comment:12 Changed 10 years ago by slavazanko

  • Owner set to slavazanko
  • Status changed from reopened to accepted

comment:13 Changed 10 years ago by angel_il

  • Type changed from defect to enhancement

comment:14 follow-up: ↓ 15 Changed 10 years ago by slavazanko

  • Type changed from enhancement to defect

not sure whether it's related, but mcedit is also very slow in latest releases.

Yep, this happens after adding UTF-8 support into upstream.

does that have the same root cause or should a new report be filed ?

Nope. This ticket about slow search in viewer. Create new report, please.

defect to enhancement

Dislike.

comment:15 in reply to: ↑ 14 Changed 10 years ago by richlv

Replying to slavazanko:
...

does that have the same root cause or should a new report be filed ?

Nope. This ticket about slow search in viewer. Create new report, please.

thanks. bug against mcedit reported as #1843
...

comment:16 Changed 10 years ago by slavazanko

  • Status changed from accepted to assigned
  • Owner changed from slavazanko to angel_il
  • Blocked By 1585 added

I am searching for simple word in internal viewer in 24 Mb text file.
Internal mc viewer needs 22 seconds when positioned at start of the
file to reach end of the file and tell me the word is not found.

Well... as fact, this is trouble described in #1585. You may check it:

1) echo "FIND_ME" >>big_file.txt
2) open big_file.txt in viewer
3) press 'End' key. Viewer after some pause will positioning at end of file.
4) press 'Home' key. Viewer will return at start of file.
5) Press F7 and enter search string "FIND_ME"

Search will much faster in this test case. Slowdown in mcview_ccache_* functions... IMHO :)

Illiya: I reassign ticket to you. After fix #1585 need to recheck test cases again.

comment:17 Changed 10 years ago by slavazanko

  • Blocked By 1585 removed

comment:18 Changed 10 years ago by slavazanko

  • Status changed from assigned to testing
  • Resolution set to wontfix

Well, search now fast :)

comment:19 Changed 10 years ago by slavazanko

  • Status changed from testing to closed

comment:20 Changed 10 years ago by angel_il

fixed in #1585

Note: See TracTickets for help on using tickets.