Ticket #2338 (closed defect: fixed)

Opened 9 years ago

Last modified 8 years ago

Use off_t for free space calculation (free space on filesystems >2TB not properly displayed)

Reported by: swegener Owned by: andrew_b
Priority: major Milestone: 4.8.1
Component: mc-core Version: 4.8.0
Keywords: Cc: mc.trac@…
Blocked By: Blocking:
Branch state: merged Votes for changeset: committed-master

Description

The free space calculation currently uses ints, which are just 32bit on plain x86 systems and result in free space on filesystems >2TB not being properly displayed. See attached patch.

Attachments

off_t-for-size-calculation.patch (855 bytes) - added by swegener 9 years ago.
This is needed to show correct free space information for todays large filesystems on 32bit systems.
config.h (23.5 KB) - added by mak 8 years ago.
config.h while builing with brew
config.2.h (24.1 KB) - added by mak 8 years ago.

Change History

Changed 9 years ago by swegener

This is needed to show correct free space information for todays large filesystems on 32bit systems.

comment:1 follow-up: ↓ 2 Changed 9 years ago by andrew_b

Thanks!

Do we really have use signed off_t instead of unsigned uintmax_t?

Generally, we have 3 different types for FS description: my_statfs (mountlist.h:26), fs_usage (fsusage.h:36) and again fs_usage (mountlist.c:139). I think all that structures should be merged into single one.

comment:2 in reply to: ↑ 1 Changed 9 years ago by swegener

Replying to andrew_b:

Do we really have use signed off_t instead of unsigned uintmax_t?

I've taken off_t, because when showing these values in src/info.c, we pass them through size_trunc_len(), which takes an off_t. In the end any integer that is 64bit on 32bit systems is fine, but we should be consistent.

comment:3 Changed 8 years ago by mak

  • Cc mc.trac@… added
  • Version changed from 4.7.3 to 4.7.5.3
  • Branch state set to no branch

since i run into this bug daily, i was wondering if there has been any progress on the issue?

in the current version(4.7.5.3) i see this, when browsing my 6TB RAID 5:

mc: 793G/1493G (53%)
os: 700G/5.5T 13%

Last edited 8 years ago by mak (previous) (diff)

comment:4 Changed 8 years ago by mak

  • Summary changed from Use off_t for free space calculation to Use off_t for free space calculation (free space on filesystems >2TB not properly displayed)
  • Milestone changed from 4.7 to 4.8

comment:5 Changed 8 years ago by andrew_b

  • Owner set to andrew_b
  • Status changed from new to accepted
  • Branch state changed from no branch to on review

Branch: 2338_free_space_calculation (parent: master).
changeset:8846ecde8be2c656bab23c24cbbf822b7a352b3e
Please review.

Last edited 8 years ago by andrew_b (previous) (diff)

comment:6 Changed 8 years ago by slavazanko

  • Votes for changeset set to slavazanko

comment:7 follow-up: ↓ 8 Changed 8 years ago by mak

because i'm unable to build the git-tree(some stuff missing on osx apparently, as autogen.sh fails), i applied the patch from git to 4.7.5.3. patch worked, however now i'm seeing:

mc: 2024G/1493G (135%) on my 5.5tb disc.
os: 1517G/4072G 28%

comment:8 in reply to: ↑ 7 Changed 8 years ago by andrew_b

Replying to mak:

mc: 2024G/1493G (135%) on my 5.5tb disc.
os: 1517G/4072G 28%

Would you show exact that values in bytes?

comment:9 Changed 8 years ago by slavazanko

  • Votes for changeset slavazanko deleted
  • Branch state changed from on review to on rework

comment:10 Changed 8 years ago by mak

  • mc

Free space: 1982G/1493G (132%)

  • disk utility

Capacity : 6 TB (6,000,699,228,160 Bytes)
Free Space : 4.33 TB (4,327,139,147,776 Bytes)
Used : 1.67 TB (1,673,560,080,384 Bytes)

  • df (1k blocks)

5860057840 1634336016 4225721824 28%

…hope this helps.

comment:11 Changed 8 years ago by andrew_b

Thanks! Would you attach your config.h here?

Changed 8 years ago by mak

config.h while builing with brew

comment:12 Changed 8 years ago by andrew_b

Hmm, the _FILE_OFFSET_BITS macro is not defined in your config.h. It should be

#define _FILE_OFFSET_BITS 64

which is provided by AC_SYS_LARGEFILE macro. If you run configure without --disable-largefile option, _FILE_OFFSET_BITS should be defined.

BTW, what system do you use?

comment:13 Changed 8 years ago by mak

hmm. looks like another perk of running os x(10.6.8 and 10.7.1).

getting this while building with --enable-largefile

checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no

comment:14 Changed 8 years ago by zap

I also hit the same problem on a x86_64 FreeBSD system, mc 4.7.5.4.

df output:

Filesystem 1K-blocks        Used      Avail Capacity  Mounted on
/dev/da0   7802994930 1643615732 6159379198    21%    /

Midnight shows: 1778G/16Z

Applied the above git patch and recompiled.

Now Midnight shows: 5874G/7442G

comment:15 Changed 8 years ago by andrew_b

Please test 2338_free_space_calculation branch. If you cannot get this branch from our repo, you can get the following snapshot: http://repo.or.cz/w/midnight-commander.git/snapshot/947be78a1b4070a35461b23ffc7c8bad32740afb.tar.gz

comment:16 follow-up: ↓ 17 Changed 8 years ago by mak

sadly i'm not able to build the snapshot.
this, however has nothing to do with the code, it's simply impossible to run autogen.sh successfully. lets call it another limitation of osx.

Last edited 8 years ago by mak (previous) (diff)

comment:17 in reply to: ↑ 16 Changed 8 years ago by andrew_b

Replying to mak:

it's simply impossible to run autogen.sh successfully.

I've created tarball for you.

comment:18 Changed 8 years ago by mak

thx for the tarball.

when building i get this error:

make[5]: * No rule to make target changesetfs', needed by all-am'. Stop.
make[4]:
* [all-recursive] Error 1
make[3]: * [all-recursive] Error 1
make[2]:
* [all-recursive] Error 1
make[1]: * [all-recursive] Error 1
make:
* [all] Error 2

i also attached the config.h file.

Changed 8 years ago by mak

comment:19 Changed 8 years ago by andrew_b

Ah, sorry. This bug was fixed in master after created branch. I think, build with --disable-vfs-extfs option as a quick workaround can help you.
I've created another tarball. Please test it.

comment:20 follow-up: ↓ 21 Changed 8 years ago by mak

build worked, but after starting it crashed instantly:

mc(61865,0x7fff70040cc0) malloc: * error for object 0x10201d048: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap

comment:21 in reply to: ↑ 20 Changed 8 years ago by andrew_b

Replying to mak:

build worked, but after starting it crashed instantly:

mc(61865,0x7fff70040cc0) malloc: * error for object 0x10201d048: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap

Would you get a backtrace using following instructions?

comment:22 follow-up: ↓ 23 Changed 8 years ago by mak

del

Last edited 8 years ago by mak (previous) (diff)

comment:23 in reply to: ↑ 22 Changed 8 years ago by andrew_b

There is no debug info here. If you use gcc, run

make CFLAGS='-g -ggdb3'

comment:24 follow-up: ↓ 25 Changed 8 years ago by mak

sorry 'bout that:

bt

#0  0x00007fff8044b0b6 in __kill ()
#1  0x00007fff804eb9f6 in abort ()
#2  0x00007fff80403195 in free ()
#3  0x0000000100093cf7 in free_mount_entry (me=0x101301490) at mountlist.c:295
#4  0x00000001000940c7 in free_my_statfs () at mountlist.c:1387
#5  0x00000001000940f4 in init_my_statfs () at mountlist.c:1400
#6  0x0000000100086694 in show_free_space (panel=0x10180f200) at panel.c:1032
#7  0x0000000100086ce1 in show_dir (panel=0x10180f200) at panel.c:1123
#8  0x00000001000872d9 in paint_frame (panel=0x10180f200) at panel.c:1306
#9  0x000000010008aa7f in panel_callback (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at panel.c:3003
#10 0x0000000100046e39 in send_message (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at widget-common.h:132
#11 0x0000000100046e07 in dlg_broadcast_msg_to (h=0x101301190, msg=WIDGET_DRAW, reverse=1, flags=0) at dialog.c:145
#12 0x0000000100048316 in dlg_broadcast_msg (h=0x101301190, msg=WIDGET_DRAW, reverse=1) at dialog.c:901
#13 0x0000000100048717 in dlg_redraw (h=0x101301190) at dialog.c:1060
#14 0x00000001000487f1 in init_dlg (h=0x101301190) at dialog.c:1097
#15 0x0000000100048942 in run_dlg (h=0x101301190) at dialog.c:1154
#16 0x00000001000806c6 in create_panels_and_run_mc () at midnight.c:897
#17 0x0000000100081f0a in do_nc () at midnight.c:1649
#18 0x0000000100007aa1 in main (argc=1, argv=0x7fff5fbfeb38) at main.c:510

bt full

#0  0x00007fff8044b0b6 in __kill ()
No symbol table info available.
#1  0x00007fff804eb9f6 in abort ()
No symbol table info available.
#2  0x00007fff80403195 in free ()
No symbol table info available.
#3  0x0000000100093cf7 in free_mount_entry (me=0x101301490) at mountlist.c:295
No locals.
#4  0x00000001000940c7 in free_my_statfs () at mountlist.c:1387
	next = (struct mount_entry *) 0x1013014c0
#5  0x00000001000940f4 in init_my_statfs () at mountlist.c:1400
No locals.
#6  0x0000000100086694 in show_free_space (panel=0x10180f200) at panel.c:1032
	rpath = "??_?\000\000\000?30\001\001\000\000\000\000TR\000\001\000\000\000\000\020R\000\001\000\000\000\b", '\0' <repeats 15 times>, "`?_?\000\000 \r@??\000\000??_?\000\000??\003\000\001\000\000\000????\000\000\000\000?\000\000\000\000\000\000\000\001\000\000\000\b\000\000\000\001\000\000\000\001\000\000\000\001\000\000\000\000\000\000\000??\003\000 \000\001\000P?_?\000\000m?\003\000\001\000\000\000\001\000\000\000?\000\000\000?", '\0' <repeats 16 times>, "\020R\000\001\000\000\000?E\003p?\000\000\000\000\000\000\000\000\000\000??_?\000\000?\t@??\000\000??"...
	vpath = (vfs_path_t *) 0x101303090
	myfs_stats = {
  type = -1, 
  typename = 0x101819d38 "hfs", 
  mpoint = 0x1013013b0 "/Volumes/data", 
  device = 0x1013013a0 "/dev/disk1s2", 
  avail = 98268964, 
  total = 732238672, 
  nfree = 24567241, 
  nodes = 183059666
}
	old_cwd = 0x101301620 "/Users/mak/_get"
#7  0x0000000100086ce1 in show_dir (panel=0x10180f200) at panel.c:1123
	tmp = (gchar *) 0x101303340 "\n"
#8  0x00000001000872d9 in paint_frame (panel=0x10180f200) at panel.c:1306
	side = 0
	width = 0
	format_txt = (GString *) 0x1a00621b00
#9  0x000000010008aa7f in panel_callback (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at panel.c:3003
	panel = (WPanel *) 0x10180f200
	bb = (WButtonBar *) 0x101301190
#10 0x0000000100046e39 in send_message (w=0x10180f200, msg=WIDGET_DRAW, parm=0) at widget-common.h:132
No locals.
#11 0x0000000100046e07 in dlg_broadcast_msg_to (h=0x101301190, msg=WIDGET_DRAW, reverse=1, flags=0) at dialog.c:145
	w = (Widget *) 0x10180f200
	p = (GList *) 0x10061f2e0
	first = (GList *) 0x1006212a0
#12 0x0000000100048316 in dlg_broadcast_msg (h=0x101301190, msg=WIDGET_DRAW, reverse=1) at dialog.c:901
No locals.
#13 0x0000000100048717 in dlg_redraw (h=0x101301190) at dialog.c:1060
No locals.
#14 0x00000001000487f1 in init_dlg (h=0x101301190) at dialog.c:1097
No locals.
#15 0x0000000100048942 in run_dlg (h=0x101301190) at dialog.c:1154
No locals.
#16 0x00000001000806c6 in create_panels_and_run_mc () at midnight.c:897
No locals.
#17 0x0000000100081f0a in do_nc () at midnight.c:1649
	midnight_colors = {9, 9, 9, 9, 9}
#18 0x0000000100007aa1 in main (argc=1, argv=0x7fff5fbfeb38) at main.c:510
	error = (GError *) 0x0
	isInitialized = 1
Last edited 8 years ago by mak (previous) (diff)

comment:25 in reply to: ↑ 24 Changed 8 years ago by andrew_b

Thanks! Please test this tarball.

comment:26 Changed 8 years ago by mak

thx again.

build worked and i'm seeing the correct values in the bottom bar.

mc:
521G/5589G (9%)

os:
521G/5589G 91%

comment:27 Changed 8 years ago by andrew_b

  • Version changed from 4.7.5.3 to 4.8.0
  • Branch state changed from on rework to on review
  • Milestone changed from 4.8 to 4.8.1

comment:28 Changed 8 years ago by slavazanko

  • Votes for changeset set to slavazanko

comment:29 Changed 8 years ago by angel_il

  • Votes for changeset changed from slavazanko to slavazanko angel_il
  • Branch state changed from on review to approved

comment:30 Changed 8 years ago by andrew_b

  • Status changed from accepted to testing
  • Votes for changeset changed from slavazanko angel_il to committed-master
  • Resolution set to fixed
  • Branch state changed from approved to merged

Merged to master.
changeset:d7ed6615ef980daf84d715d3ea32137f9c9007fd

git log --pretty=oneline bbbb918..d7ed661

comment:31 Changed 8 years ago by andrew_b

  • Status changed from testing to closed
Note: See TracTickets for help on using tickets.