Ticket #3730 (new task) — at Version 5

Opened 8 years ago

Last modified 8 years ago

Introduce a tester for extfs helpers

Reported by: mooffie Owned by:
Priority: major Milestone: 4.8.19
Component: mc-vfs Version: master
Keywords: Cc:
Blocked By: #3696 Blocking: #3729
Branch state: merged Votes for changeset:

Description (last modified by mooffie) (diff)

Here's a tester intended to test the "list" command of extfs helpers.

(Others would use the phrase "test framework" instead of "tester". I'm trying to sound friendly.)

I provide with it tests for uzip, urar, uzoo, and lslR. That's to demonstrate the power of the tester. If this tester gets merged into MC I'll provide tests for more helpers (anybody could, though; it's very easy).

Why

The dire problem of us not currently having tests for covering extfs has been mentioned several times here.

How it works

The tester works by feeding the helpers example input, parsing their output and comparing it to the known correct output.

To understand this better, I uploaded the tester here for easier browsing; specifically:

  • See the README (html format).
  • See the data directory. The purpose of the '.input' and '.output' files should be self evident even if you don't read the docs.
  • As you'll shortly see, each helper needs a very minor modification to make it possible to feed it "fake" input.

How it looks like

(a) Either do "make check"; or:
(b) Run the tester directly, with "run", to see colorful output:

(b1) ...when everything works alright:

http://i.imgur.com/CivkhnF.png

(b2) ...when some helper has a bug:

http://i.imgur.com/kjmVK3G.png

Change History

comment:1 Changed 8 years ago by mooffie

Some random notes:

  1. Yeah, I know the masses are clamoring for FISH tests. At the moment I'm not too familiar with the FISH scripts and I want to leave that out of this ticket.
  1. A few words about "integration tests" vs "unit tests": There's another method to do testing, if MC had scripting support. That method lets us discover many bugs in the VFS, even bugs we didn't intend to cover. But, as Yury explains there, those are "integration tests", and as such they have a drawback: they don't give us the direct control "unit tests" give us on the input each component is fed. I.e., in our case we need to feed a helper various inputs, not just the input our system provides, and unit tests are better for this.

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

Changed 8 years ago by mooffie

comment:2 Changed 8 years ago by mooffie

  • Blocked By 3696 added

(Since this ticket includes a test for uzoo, it's dependent on #3696.)

comment:3 Changed 8 years ago by mooffie

  • Blocking 3729 added

(In #3729) It will be very easy to fix these once we have #3730 in.

comment:4 Changed 8 years ago by mooffie

  • Description modified (diff)

comment:5 Changed 8 years ago by mooffie

  • Description modified (diff)

(My web host is down so I copied the files to GitHub Pages and updated the links. The downside is that GitHub doesn't let me set "text/plain" MIME type on all files so you won't be able to browse them online. But it's not a big deal: just look in the patches. I wanted a web hosting just because of README.html.)

Note: See TracTickets for help on using tickets.