Ticket #1585 (closed defect: fixed)

Opened 15 years ago

Last modified 3 years ago

MC viewer is terribly slow in inefficient when jumping to the end of big files

Reported by: birdie Owned by: angel_il
Priority: critical Milestone: 4.7.1
Component: mcview Version: master
Keywords: Cc: tux@…, gotar, birdie
Blocked By: Blocking:
Branch state: merged Votes for changeset: committed-master

Description

Try to view a big enough (>10MB) file in Midnight Commander internal viewer and hit "End" button - the bigger is file, the longer it will take to scroll to the end of the file.

Jumping to the end of 50MB text file takes 5 (five!) seconds on my 2600MHz AMD 64 CPU. It is unacceptable.

It doesn't matter if you have "Wrap" or "UnWrap?" setting.

An example file is attached.

Attachments

vnetlib.7z (30.8 KB) - added by birdie 15 years ago.
Sample file (use 7z to unpack it)
midnight_commander-3.2.1.png (12.5 KB) - added by birdie 14 years ago.
testcase.bin (4.9 KB) - added by himikof 14 years ago.
Strange behaviour testcase
bol_eol.diff (6.1 KB) - added by himikof 14 years ago.
Patch for mcview_bol/mcview_eol
0001-Fixed-left-right-movement-in-hex-mode.patch (2.5 KB) - added by himikof 14 years ago.
Fixes a few issues in left/right movement in hex viewer

Change History

Changed 15 years ago by birdie

Sample file (use 7z to unpack it)

comment:1 Changed 15 years ago by ossi

now try this with something *big* (say, /proc/kcore) and see your system choke.

comment:2 Changed 15 years ago by birdie

Are you scoffing at me or what?

MC 4.5.x and Far File Manager scroll to the end of any files instantly.

comment:3 follow-up: ↓ 4 Changed 15 years ago by mike.dld

That's because MC scans all the file from top to bottom to display current line number in the status bar. The question is, do we really need this in viewer (FAR, for example, displays just a position percentage).

comment:4 in reply to: ↑ 3 Changed 15 years ago by birdie

Replying to mike.dld:

That's because MC scans all the file from top to bottom to display current line number in the status bar. The question is, do we really need this in viewer (FAR, for example, displays just a position percentage).

What really matters is an ability to view files of any size.

comment:5 Changed 15 years ago by bilbo

  • Cc tux@… added

comment:6 Changed 14 years ago by gotar

  • Cc gotar@… added

+1
Line number is irrevelant if I can't even view it.

comment:7 follow-up: ↓ 8 Changed 14 years ago by marrtins

Hi to all mc developers! I work on quite large log-files on daily basis and this is getting *really* annoying. Pleasure to work on old servers with old mc. Please make this *feature* go away or at least configurable. Thank you very much!

comment:8 in reply to: ↑ 7 Changed 14 years ago by birdie

Replying to marrtins:

Hi to all mc developers! I work on quite large log-files on daily basis and this is getting *really* annoying. Pleasure to work on old servers with old mc. Please make this *feature* go away or at least configurable. Thank you very much!

Use older MC releases, until this bug is fixed.

comment:9 Changed 14 years ago by angel_il

  • Type changed from defect to enhancement

comment:10 Changed 14 years ago by gotar

  • Type changed from enhancement to defect

This is seriuos defect, please don't colorize stats by changing it to enhancement.

comment:11 Changed 14 years ago by angel_il

please publish results testing 4.6.х VS 4.7.0 (current master)
test case:
view file ~3Gb (open in viewer and go to EOF)

comment:12 Changed 14 years ago by birdie

200MB text file:

MC 4.7.0-pre4: 40 seconds
MC 4.6.2-fc11: 11 seconds

However both results are unacceptable.

comment:13 Changed 14 years ago by birdie

3GB file is 15 times bigger than 200MB, so jumping to the end of file will take 10 (ten) minutes in MC 4.7.

comment:14 Changed 14 years ago by angel_il

  • Status changed from new to accepted
  • Owner set to angel_il

ok, you right, i'm not... i try make viewer faster...

Changed 14 years ago by birdie

comment:15 follow-up: ↓ 16 Changed 14 years ago by birdie

Out of curiosity I've just compiled MC 3.2.1 (the oldest version I could found on the net).

It jumps to the end of any file instantly. :)

comment:16 in reply to: ↑ 15 Changed 14 years ago by angel_il

Replying to birdie:

Out of curiosity I've just compiled MC 3.2.1 (the oldest version I could found on the net).

It jumps to the end of any file instantly. :)

That's because MC scans all the file from top to bottom... (c)

comment:17 Changed 14 years ago by slavazanko

  • Blocking 1417 added

(In #1417) > I am searching for simple word in internal viewer in 24 Mb text file.

Internal mc viewer needs 22 seconds when positioned at start of the
file to reach end of the file and tell me the word is not found.

Well... as fact, this is trouble described in #1585. You may check it:

1) echo "FIND_ME" >>big_file.txt
2) open big_file.txt in viewer
3) press 'End' key. Viewer after some pause will positioning at end of file.
4) press 'Home' key. Viewer will return at start of file.
5) Press F7 and enter search string "FIND_ME"

Search will much faster in this test case. Slowdown in mcview_ccache_* functions... IMHO :)

Illiya: I reassign ticket to you. After fix #1585 need to recheck test cases again.

comment:18 Changed 14 years ago by angel_il

OK... birdie, build 608e7b497032b4d0be155f56dcb19878872e3513 and try it...
before 608e7b497032b4d0be155f56dcb19878872e3513 mcview jumps to the EOF instantly.

git reset --hard 608e7b497032b4d0be155f56dcb19878872e3513
./autogen.sh
./configure
make

comment:19 Changed 14 years ago by ossi

the correct command is

git checkout 608e7b497032b4d0be155f56dcb19878872e3513

with reset --hard you will mess up the state of your local master branch. which is no biggie when you know how to recover ...

comment:20 Changed 14 years ago by angel_il

branch: 1585_slow_viewer

first step to make viewer faster

comment:21 Changed 14 years ago by angel_il

comment:22 Changed 14 years ago by angel_il

  • severity changed from no branch to on review

comment:23 Changed 14 years ago by angel_il

  • Cc gotar, birdie added; gotar@… removed
  • Version changed from 4.7.0-pre2 to master
  • Milestone changed from 4.7 to 4.7.1

Changed 14 years ago by himikof

Strange behaviour testcase

comment:24 Changed 14 years ago by slavazanko

  • severity changed from on review to on rework

comment:25 Changed 14 years ago by himikof

I have made a patch that fixes previously reported test case and removes unneeded eol_width parameter. Moving up-down seems to be working.

Changed 14 years ago by himikof

Patch for mcview_bol/mcview_eol

Changed 14 years ago by himikof

Fixes a few issues in left/right movement in hex viewer

comment:26 Changed 14 years ago by angel_il

comment:27 Changed 14 years ago by angel_il

  • severity changed from on rework to on review

comment:28 Changed 14 years ago by slavazanko

  • Votes for changeset set to slavazanko

comment:29 Changed 14 years ago by andrew_b

  • Votes for changeset changed from slavazanko to slavazanko andrew_b
  • severity changed from on review to approved

comment:30 Changed 14 years ago by angel_il

  • Votes for changeset changed from slavazanko andrew_b to committed-master
  • severity changed from approved to merged

comment:31 Changed 14 years ago by angel_il

Last edited 3 years ago by andrew_b (previous) (diff)

comment:32 Changed 14 years ago by birdie

I haven't tested the patch yet, but I still say thank you very much!

comment:33 Changed 14 years ago by angel_il

  • Status changed from accepted to testing
  • Resolution set to fixed

comment:34 Changed 14 years ago by slavazanko

  • Keywords stable-candidate added

comment:35 Changed 14 years ago by slavazanko

  • Blocking 1417 removed

comment:36 Changed 14 years ago by angel_il

  • Status changed from testing to closed

comment:37 Changed 13 years ago by andrew_b

  • Keywords stable-candidate removed

comment:38 Changed 3 years ago by andrew_b

  • Cc changed from tux@centrum.cz, gotar, birdie, to tux@centrum.cz, gotar, birdie
  • Branch state set to merged
Note: See TracTickets for help on using tickets.