CVRANAL reads the coverage (.cvr) files written by the test version of a program and writes a report showing which code blocks have never been executed in the current series of tests, and which have been executed most frequently (execution hot-spots). Based on this report, it should be possible to devise additional tests to reduce the amount of untested code. The analysis of execution hot-spots may also suggest ways to make the program run faster.
CVRANAL is also able to merge the execution frequency data with the original source code (or a copy thereof). The number of times each block has been executed is inserted as an end-of-line ('!') comment, starting at a specified column (e.g. 133). The comment is suppressed on lines which extend beyond the specified column.
Coverage Analysis is an important tool for anyone involved in software quality assurance. Clearly a program cannot be said to have been properly tested if there are parts of the source code which have never been executed, and programmers are often surprised to find that even quite extensive testing can leave large amounts of untested code. Any thorough QA procedure should use coverage analysis in conjunction with both static and dynamic analysis, since each approach will find problems that the others cannot detect.
To start a run, simply type cvranal followed by the name of the coverage file(s) you wish to process. You can use wild-cards to specify more than one file. For example:
processes the contents of all .cvr files in the current directory.
Under Windows, .cvr is assumed if no extension is specified (b* is treated as b*.cvr). The default file name is *.cvr. Thus, if you just type cvranal, all .cvr files in the current directory are processed.
On Linux and Mac, the extension must be specified, and there is no default file name.
The operation of CVRANAL may be modified by command line switches. Currently the following switches are available:
specifies the configuration file name (see section 4.3). If you don't specify a file name in this way, CVRANAL assumes that the configuration file is in a file called cvranal.fig. In either case, the search rules defined in section 1.4 are followed. If no configuration file can be found, CVRANAL uses internal defaults for all configuration items.
If a '?' is appended (e.g. FIG=CVRANAL.FIG? or FIG=?), CVRANAL lists the contents of the active configuration file to the screen, with a pause after each screen full.
specifies the value of item nnn in the CVRANAL configuration file (see section 4.4). Items specified in this way over-ride those read from the configuration file.
specifies the name of the file to which the main CVRANAL output report is sent. The default file name is cvranal.out. An abbreviated version of this report is written to the screen as CVRANAL runs.
cvranal a*.cvr to=cvrfred.out fig=cvrfred.fig 513=0
This command causes CVRANAL to analyze all files in the current directory whose name fits the template a*.cvr. Configuration options are read from cvrfred.fig, but with item 513 reset to 0. The output report is sent to cvrfred.out.
The CVRANAL configuration file has the same general format as the SPAG and GXCHK configuration files (see Section 2.5, 3.3). The item numbers for CVRANAL are in the range 500-600, and all except one are integers. There is no overlap between CVRANAL, GXCHK and SPAG item numbers.
An example configuration file is shown below. The meaning of each data item is explained in detail in the next section. The values shown in this example are, in fact, the defaults which are used by CVRANAL if no configuration file is found.
=========================== CVRANAL controls ============================== 501=0 0 - normal run 1 - set usage counts to zero 502=0 0 - Unicode box drawing chars 1 - use only ASCII characters 503=200 number of lines/page in CVRANAL.OUT (if >=200 no pages within reports) 504=78 width of output reports 510=2 0=No check for unexercised code 1=Report unexercised code to CVRANAL.OUT 2=line number summary to CVRANAL.OUT 511=200 Number of hotspots to CVRANAL.OUT 512=7 Number of hotspots to screen 513=73 0=don't modify source code >0 insert usage count at column n 550= List (separated by ;) of directories containing source code
Type of Run
Specifies whether this is an analysis or re-initialization run. Possible values are:
Box Drawing Characters
Specifies which box drawing characters should be used in the output report. Possible values are:
Lines per Page
Specifies the required number of lines per page for the CVRANAL output file. If this item is set to 200 or more, there is no limit to the number of lines per page, though a new page will be output at the head of each report.
Page Width in Characters
Specifies the target width in characters of the CVRANAL output file. Must be between 65 and 160.
Unexercised Code Report
Specifies the destination(s) for the report on code which has not been executed in the current series of tests. Possible values are:
Execution Hot-spot Report (file)
The execution hot-spot report lists hot-spots in order of execution count, starting with the most used block. This item specifies how many should be listed in the main output file (normally cvranal.out). The maximum permitted value is 800.
Execution Hot-spot Report (screen)
This item specifies how many execution hot-spots should be listed to the screen. A small value is generally preferred. The maximum permitted value is 800.
Source Code Annotation
Specifies whether CVRANAL is to annotate the original source code (or a copy) with comments showing the execution count of each code block. Possible values are:
The annotation takes the form of a '!' character followed by an integer which is the execution count. For large execution counts, a 'K' or 'M' may be used to represent thousands or millions of executions. The '!' character is used to make Fortran 95 compilers see the count as a comment. For Fortran 77 compilers which do not recognize '!', it is important that the value of this item be set to 73 or more, so that the count appears as a sequence number.
CVRANAL will not annotate a source code line if the text of the line appears to go beyond the column specified (except that it will overwrite previous annotations).
Source Code Directories
Specifies the name of the directory containing the original source code (or the copy you wish to annotate). You may also specify a list of directories, separated by semicolons, which are to be searched for source code files. Note that the original source code is required here - NOT the instrumented version created by SPAG.
This item is not required if an absolute path to the source files was specified when SPAG created the test version of the program.
CVRANAL may be used as often as required, at any stage during the testing process. However if the source code is changed in any way, it is necessary to use SPAG to re-instrument the changed code before compiling and linking the revised test program. Once this is done, we recommend that you re-initialize all the coverage data (by running CVRANAL with item 501 set to 1), even for source files which have not been changed. If this is not done, the significance of the execution hot-spot data is questionable, as different parts of the code will be using different base points.
In the probe routines as supplied, execution counts are stored in a standard Fortran INTEGER array, and there is no guard against overflow. On most machines, this means that if a block of code is executed more than about 2147 million times, the execution count becomes very large and negative. In most cases, this is an acceptable compromize, as it is rare for execution counts to be so large. However, a version of the probe routines in which execution counts are double precision floating point numbers may easily be constructed by changing the type of the variable TCOunt from INTEGER to DOUBLE PRECISION in the probes source code file (probes.f90 in the plusFORT installation directory). This modified version may be slower, and uses more memory, but has an effectively unlimited capacity.
If CVRANAL reports that all code blocks are untested, even in routines that you know must have been executed, then it is very likely that the test program did not finish normally. In this case you may also find that the timing reports have not been written to probes.log. The .cvr files are updated by the call to the routine (normally called PR$EXI) which SPAG inserts before every STOP statement, and before the END statement of the main PROGRAM. If the test program halts in some other way, for example because of a run-time error, or because the user interrupted execution, then the .cvr files will not be updated to reflect the usage in the current run. If in doubt, check the last modification time of probes.log and the the .cvr files
This can also happen if the program is halted by a call to routine which has not been instrimented by SPAG. For example, many compilers provide a run-time routine (e.g EXIT) to halt execution. In such cases, it is necessary to hand-edit the instrumented code to insert a call to PR$EXI before the call to EXIT. Thus:-
CALL BL$ENT(2158) CALL EXIT(16)
would be changed to:-
CALL BL$ENT(2158) CALL PR$EXI CALL EXIT(16)
The probe routines contain code which checks for error conditions, and may halt execution with one of the following messages, for example, if a very large program causes overflow of an internal array:-
Profiler - Slot Overflow Profiler - File list too long Profiler - Total line limit exceeded Profiler - File buffer overrun Profiler - Unable to open coverage data file Profiler - Invalid traceback in SB$EXI
The cure in the first four cases is to increase the dimensions (MAXSUB, MAXLIN and BUFSIZ) specified in probes.f90 and rebuild the instrumented program. Please note the comments in probes.f90 regarding possible values of MAXSUB. The fifth may indicate that a .cvr file has been moved or deleted, or has been locked by another program. The last message indicates a logical error, and should not happen unless the source files were incorrectly hand-edited.