Ticket #1585 (closed defect: fixed)

Opened 10 years ago

Last modified 8 years ago

MC viewer is terribly slow in inefficient when jumping to the end of big files

Reported by: birdie Owned by: angel_il
Priority: critical Milestone: 4.7.1
Component: mcview Version: master
Keywords: Cc: tux@…, gotar, birdie,
Blocked By: Blocking:
Branch state: Votes for changeset: committed-master

Description

Try to view a big enough (>10MB) file in Midnight Commander internal viewer and hit "End" button - the bigger is file, the longer it will take to scroll to the end of the file.

Jumping to the end of 50MB text file takes 5 (five!) seconds on my 2600MHz AMD 64 CPU. It is unacceptable.

It doesn't matter if you have "Wrap" or "UnWrap?" setting.

An example file is attached.

Attachments

vnetlib.7z (30.8 KB) - added by birdie 10 years ago.
Sample file (use 7z to unpack it)
midnight_commander-3.2.1.png (12.5 KB) - added by birdie 9 years ago.
testcase.bin (4.9 KB) - added by himikof 9 years ago.
Strange behaviour testcase
bol_eol.diff (6.1 KB) - added by himikof 9 years ago.
Patch for mcview_bol/mcview_eol
0001-Fixed-left-right-movement-in-hex-mode.patch (2.5 KB) - added by himikof 9 years ago.
Fixes a few issues in left/right movement in hex viewer

Change History

Changed 10 years ago by birdie

Sample file (use 7z to unpack it)

comment:1 Changed 10 years ago by ossi

now try this with something *big* (say, /proc/kcore) and see your system choke.

comment:2 Changed 10 years ago by birdie

Are you scoffing at me or what?

MC 4.5.x and Far File Manager scroll to the end of any files instantly.

comment:3 follow-up: ↓ 4 Changed 10 years ago by mike.dld

That's because MC scans all the file from top to bottom to display current line number in the status bar. The question is, do we really need this in viewer (FAR, for example, displays just a position percentage).

comment:4 in reply to: ↑ 3 Changed 10 years ago by birdie

Replying to mike.dld:

That's because MC scans all the file from top to bottom to display current line number in the status bar. The question is, do we really need this in viewer (FAR, for example, displays just a position percentage).

What really matters is an ability to view files of any size.

comment:5 Changed 10 years ago by bilbo

  • Cc tux@… added

comment:6 Changed 9 years ago by gotar

  • Cc gotar@… added

+1
Line number is irrevelant if I can't even view it.

comment:7 follow-up: ↓ 8 Changed 9 years ago by marrtins

Hi to all mc developers! I work on quite large log-files on daily basis and this is getting *really* annoying. Pleasure to work on old servers with old mc. Please make this *feature* go away or at least configurable. Thank you very much!

comment:8 in reply to: ↑ 7 Changed 9 years ago by birdie

Replying to marrtins:

Hi to all mc developers! I work on quite large log-files on daily basis and this is getting *really* annoying. Pleasure to work on old servers with old mc. Please make this *feature* go away or at least configurable. Thank you very much!

Use older MC releases, until this bug is fixed.

comment:9 Changed 9 years ago by angel_il

  • Type changed from defect to enhancement

comment:10 Changed 9 years ago by gotar

  • Type changed from enhancement to defect

This is seriuos defect, please don't colorize stats by changing it to enhancement.

comment:11 Changed 9 years ago by angel_il

please publish results testing 4.6.х VS 4.7.0 (current master)
test case:
view file ~3Gb (open in viewer and go to EOF)

comment:12 Changed 9 years ago by birdie

200MB text file:

MC 4.7.0-pre4: 40 seconds
MC 4.6.2-fc11: 11 seconds

However both results are unacceptable.

comment:13 Changed 9 years ago by birdie

3GB file is 15 times bigger than 200MB, so jumping to the end of file will take 10 (ten) minutes in MC 4.7.

comment:14 Changed 9 years ago by angel_il

  • Owner set to angel_il
  • Status changed from new to accepted

ok, you right, i'm not... i try make viewer faster...

Changed 9 years ago by birdie

comment:15 follow-up: ↓ 16 Changed 9 years ago by birdie

Out of curiosity I've just compiled MC 3.2.1 (the oldest version I could found on the net).

It jumps to the end of any file instantly. :)

comment:16 in reply to: ↑ 15 Changed 9 years ago by angel_il

Replying to birdie:

Out of curiosity I've just compiled MC 3.2.1 (the oldest version I could found on the net).

It jumps to the end of any file instantly. :)

That's because MC scans all the file from top to bottom... (c)

comment:17 Changed 9 years ago by slavazanko

  • Blocking 1417 added

(In #1417) > I am searching for simple word in internal viewer in 24 Mb text file.

Internal mc viewer needs 22 seconds when positioned at start of the
file to reach end of the file and tell me the word is not found.

Well... as fact, this is trouble described in #1585. You may check it:

1) echo "FIND_ME" >>big_file.txt
2) open big_file.txt in viewer
3) press 'End' key. Viewer after some pause will positioning at end of file.
4) press 'Home' key. Viewer will return at start of file.
5) Press F7 and enter search string "FIND_ME"

Search will much faster in this test case. Slowdown in mcview_ccache_* functions... IMHO :)

Illiya: I reassign ticket to you. After fix #1585 need to recheck test cases again.

comment:18 Changed 9 years ago by angel_il

OK... birdie, build 608e7b497032b4d0be155f56dcb19878872e3513 and try it...
before 608e7b497032b4d0be155f56dcb19878872e3513 mcview jumps to the EOF instantly.

git reset --hard 608e7b497032b4d0be155f56dcb19878872e3513
./autogen.sh
./configure
make

comment:19 Changed 9 years ago by ossi

the correct command is

git checkout 608e7b497032b4d0be155f56dcb19878872e3513

with reset --hard you will mess up the state of your local master branch. which is no biggie when you know how to recover ...

comment:20 Changed 9 years ago by angel_il

branch: 1585_slow_viewer

first step to make viewer faster

comment:21 Changed 9 years ago by angel_il

comment:22 Changed 9 years ago by angel_il

  • severity changed from no branch to on review

comment:23 Changed 9 years ago by angel_il

  • Cc gotar, birdie added; gotar@… removed
  • Version changed from 4.7.0-pre2 to master
  • Milestone changed from 4.7 to 4.7.1

Changed 9 years ago by himikof

Strange behaviour testcase

comment:24 Changed 9 years ago by slavazanko

  • severity changed from on review to on rework

comment:25 Changed 9 years ago by himikof

I have made a patch that fixes previously reported test case and removes unneeded eol_width parameter. Moving up-down seems to be working.

Changed 9 years ago by himikof

Patch for mcview_bol/mcview_eol

Changed 9 years ago by himikof

Fixes a few issues in left/right movement in hex viewer

comment:26 Changed 9 years ago by angel_il

comment:27 Changed 9 years ago by angel_il

  • severity changed from on rework to on review

comment:28 Changed 9 years ago by slavazanko

  • Votes for changeset set to slavazanko

comment:29 Changed 9 years ago by andrew_b

  • Votes for changeset changed from slavazanko to slavazanko andrew_b
  • severity changed from on review to approved

comment:30 Changed 9 years ago by angel_il

  • Votes for changeset changed from slavazanko andrew_b to committed-master
  • severity changed from approved to merged

comment:32 Changed 9 years ago by birdie

I haven't tested the patch yet, but I still say thank you very much!

comment:33 Changed 9 years ago by angel_il

  • Status changed from accepted to testing
  • Resolution set to fixed

comment:34 Changed 9 years ago by slavazanko

  • Keywords stable-candidate added

comment:35 Changed 9 years ago by slavazanko

  • Blocking 1417 removed

comment:36 Changed 9 years ago by angel_il

  • Status changed from testing to closed

comment:37 Changed 8 years ago by andrew_b

  • Keywords stable-candidate removed
Note: See TracTickets for help on using tickets.