Ticket #3730: 3730-0003-extfs-documentation-for-the-tester.patch

File 3730-0003-extfs-documentation-for-the-tester.patch, 7.6 KB (added by mooffie, 3 years ago)
  • tests/src/extfs-helpers-listcmd/Makefile.am

    From 0594f838939c899f764076cee7f1c03d1941e458 Mon Sep 17 00:00:00 2001
    From: Mooffie <mooffie@gmail.com>
    Date: Sun, 20 Nov 2016 23:20:16 +0200
    Subject: [PATCH 03/11] extfs: documentation for the tester.
    
    ---
     tests/src/extfs-helpers-listcmd/Makefile.am    |  13 ++
     tests/src/extfs-helpers-listcmd/README         | 171 +++++++++++++++++++++++++
     tests/src/extfs-helpers-listcmd/README.css.inc |  19 +++
     3 files changed, 203 insertions(+)
     create mode 100644 tests/src/extfs-helpers-listcmd/README
     create mode 100644 tests/src/extfs-helpers-listcmd/README.css.inc
    
    diff --git a/tests/src/extfs-helpers-listcmd/Makefile.am b/tests/src/extfs-helpers-listcmd/Makefile.am
    index bd4dbe1..8c6105b 100644
    a b run: 
    4646        chmod +x $@ 
    4747# (We can alternatively create run from a run.in template 
    4848# with 'AC_CONFIG_FILES[run, chmod +x run]'.) 
     49 
     50# 
     51# Documentation 
     52# 
     53 
     54doc: README.html 
     55 
     56# (Thanks to VPATH we don't need to write "$(srcdir)/README". doc/hlp/Makefile.am needlessly does this.) 
     57README.html: README 
     58        pandoc --include-in-header=$(srcdir)/README.css.inc -N --old-dashes --toc --standalone -o $@ $< 
     59 
     60EXTRA_DIST += README.css.inc 
     61CLEANFILES += README.html 
  • new file tests/src/extfs-helpers-listcmd/README

    diff --git a/tests/src/extfs-helpers-listcmd/README b/tests/src/extfs-helpers-listcmd/README
    new file mode 100644
    index 0000000..d601f7d
    - +  
     1--- 
     2title: A tester for extfs helpers 
     3... 
     4 
     5Guide 
     6===== 
     7 
     8Introduction 
     9------------ 
     10 
     11The extfs filesystem is composed of various helpers (uzip, urar, uarc, 
     12...). One command every helper must answer to is "list", to list the 
     13files on its filesystem. 
     14 
     15The purpose of this tester is to test this "list" facet of every helper 
     16to ensure that it indeed works, at present, and that we won't 
     17inadvertently break it, in the future, as we modify its code or MC's 
     18code. 
     19 
     20Key concept: Inputs 
     21------------------- 
     22 
     23Most helpers work by parsing the output of some 3'rd party software. 
     24Which for them becomes the *input*. Helpers sometimes support **several 
     25variations** of such input. E.g., the uzip helper supports three 
     26variations. 
     27 
     28The tester keeps a repository, in the data folder, of the various inputs 
     29each helper proclaims to support. Each input is stored in a file with an 
     30`.input` suffix. 
     31 
     32Key concept: Outputs 
     33-------------------- 
     34 
     35Along with each input file, the data folder also holds the output the 
     36helper is expected to produce given the corresponding input. Each output 
     37is stored in a file with an `.output` suffix. 
     38 
     39We call this output "the expected output". 
     40 
     41Incidentally, an `.output` file stores not the _raw_ output of the helper 
     42but its output _after parsing_. In other words, what's stored is the 
     43unambiguous _meaning_ of the helper's output. This means that as long as 
     44the helper's code isn't modified in a way that changes the meaning of its 
     45output, the `.output` file remains up-to-date. 
     46 
     47How the tester works 
     48-------------------- 
     49 
     50The tester feeds each helper its prepared inputs and reads back the 
     51helper's "list" answer -- the helper's **output**. This output is a list 
     52of files in a format similar to `ls -l`, which MC is able to parse. The 
     53tester checks that this output parses without errors (errors are, for 
     54example, dates in unsupported format). It then compares this parsed 
     55output (which we call "the actual output") with a previously saved copy 
     56of this output which is known to be correct (and which we call "the 
     57expected output", mentioned in the previous section). This previously 
     58stored output too is in the data folder, in files with `.output` suffix. 
     59 
     60If there's any discrepancy between the *actual output* and the 
     61*expected output*, the test fails. 
     62 
     63Running the tester 
     64------------------ 
     65 
     66You can run the tester with `make check`. 
     67 
     68But you'll find it more appealing to run the tester with the `run` 
     69script. You'll get a colorful description of what's going on. 
     70 
     71(`run` is created by running `make check` for the 1st time, in the build 
     72tree.) 
     73 
     74Reference 
     75========= 
     76 
     77The data folder 
     78--------------- 
     79 
     80There are several types of files in the data folder: 
     81 
     82### Input file ### 
     83 
     84An input file is named: 
     85 
     86> `<helper-name>[.optional-embedded-description].input` 
     87 
     88You create such files simply by redirecting the 3'rd party software's 
     89output to a file. 
     90 
     91### Output file ### 
     92 
     93This file is named the same as the corresponding input file but with an 
     94`.output` suffix. 
     95 
     96The easiest way to create these files is by invoking the `run` script 
     97with the `--create-output` option. 
     98 
     99### Environment file ### 
     100 
     101Optional. This file defines environment variables the helper may use to 
     102determine the variant of the input. This file is named the same as the 
     103corresponding input file but with an `.env_var` suffix. 
     104 
     105### Arguments file ### 
     106 
     107Optional. This file defines extra command-line options to pass to the 
     108[parser](#mc_parse_ls_l). This file is named the same as the 
     109corresponding input file but with an `.args` suffix. 
     110 
     111The contents of an output file must be the same no matter on what 
     112computer and at what time we generate it. Therefore we need to tell the 
     113parser to drop any non-fixed elements in that file. E.g., if the dates 
     114used are relative (as is the case for the default `ls` dates), we need to 
     115drop them with `--drop-mtime`. Similarly, if a helper returns user and 
     116group _names_ that are different than the running user's, they must be 
     117dropped with `--drop-ids`. 
     118 
     119### Other files ### 
     120 
     121Any other file is ignored by the tester. 
     122 
     123mc_parse_ls_l 
     124------------- 
     125 
     126This program (built with `make check`) is at the heart of the tester 
     127mechanism. It parses a list of files, in a format similar to `ls -l`, 
     128just as MC would. This program is used to parse (and thereby verify) the 
     129output of the helpers. _You don't need to invoke it yourself;_ but, for 
     130educational purpose, here are a few examples: 
     131 
     132    $ LC_ALL=C ls -l | ./mc_parse_ls_l 
     133 
     134    $ LC_ALL=C ls -l | ./mc_parse_ls_l --symbolic-ids 
     135 
     136    $ LC_ALL=C ls -l | ./mc_parse_ls_l --output-format yaml 
     137 
     138test_all 
     139-------- 
     140 
     141This is the tester itself. You invoke it with `make check`, or with the 
     142`run` script. Invoking it directly is a bit involving because you need to 
     143provide it with 2 or 3 directory paths. `run` does this work for you. 
     144 
     145MC_TEST_EXTFS_LIST_CMD 
     146---------------------- 
     147 
     148When a helper runs under the tester, the environment variable 
     149`MC_TEST_EXTFS_LIST_CMD` holds the command that's to provide input. The 
     150helper's source code must be modified to use this command instead of the 
     151command it usually uses. This is the device which lets us plug our own 
     152input into the helper and *without which a helper can't be tested!* 
     153 
     154Let's have a little example. The uzoo helper originally has: 
     155 
     156    ZOO=zoo 
     157    ... 
     158    mczoofs_list () { 
     159      $ZOO lq "$ARCHIVE" | mawk '......' 
     160    } 
     161    ... 
     162 
     163To make this helper testable, we need to change the first line to: 
     164 
     165    ZOO=${MC_TEST_EXTFS_LIST_CMD:-zoo} 
     166 
     167(or equivalent.) 
     168 
     169The command in `MC_TEST_EXTFS_LIST_CMD` is a black-box for the helper, 
     170and it intentionally ignores any arguments passed to it (so that `lq 
     171"$ARCHIVE"`, above, won't cause problems). 
  • new file tests/src/extfs-helpers-listcmd/README.css.inc

    diff --git a/tests/src/extfs-helpers-listcmd/README.css.inc b/tests/src/extfs-helpers-listcmd/README.css.inc
    new file mode 100644
    index 0000000..a99a894
    - +  
     1<style type="text/css"> 
     2 
     3body { 
     4  padding: 1em 2.5em; 
     5  line-height: 140%; 
     6} 
     7 
     8code, pre { 
     9  background-color: #FFB; 
     10  padding: 2px 3px; 
     11} 
     12 
     13h1, h2 { text-indent: -0.5em; } 
     14h1, h2, h3 { 
     15  color: #005A9C; 
     16  font-family: sans-serif; 
     17} 
     18 
     19</style>