textfiles/virus/eval.vir

                    Anti-Viral Product Evaluation
                          May 5, 1989

     This evaluation paper has been written by Jim Goodwin, Lynn
Marsh and Tim Sankary.  It is copyrighted, 1989, and is intended
for circulation among fellow members of the virus research
community who use IBM PCs or compatibles.  We do not consider it
complete, since we did not evaluate every available product, and
it is not intended as a public guide to selecting antiviral
programs.  We hope, however, that it will prove useful to other
members of the community who work with live viruses and need
ongoing protection for their systems.  This document may be
freely copied and distributed providing the disclaimer and
copyright are kept intact, and no changes, additions or deletions
are made to the text.
     We would like to acknowledge the ample research data
provided by Jim Bates and Rusty Davis in England, Ivan Grebert of
Acal Corporation in Paris, Colin Haynes of the International
Computer Virus Institute, and the many volunteer researchers from
the Silicon Valley area that contributed so much to our efforts. 
We would also like to acknowledge the HomeBase users group for
providing their detailed log of infection occurrences and other
epidemiological data.

  
The Need for a Reasonable Evaluation:
     In the April issue of PC Magazine you will find a review of
11 antiviral products.  The review, while well intentioned,
tested products against only two viruses (plus one simulated
virus that was developed by the magazine).  None of the viruses
were boot sector infectors (viruses which attach to the boot
sector) and none were among the most common viruses.  Since the
vast majority of virus infections are boot sector infections, and
since most viruses are much more difficult to detect than the two
chosen, the results of the review were next to meaningless.  The
PC Magazine review was similar to many others published in the
past year.  It was performed without adequate access to the
viruses actually causing problems in the user community.  
     A second problem with these reviews, is that many of the
reviewers have had limited experience with the broad range of
infections that have occurred within the past 18 months.  They
base evaluations on assumptions that do not hold for the real
world.  This is not necessarily the fault of the reviewers. 
Viruses are a new phenomenon and few people have dedicated their
time and resources to a long term study.  A reviewer who has had
experience with only one or two viruses might naturally draw
incorrect conclusions about "generic" virus issues.
     For example, a number of viruses infect programs using
common DOS calls (interrupt 21 or other interrupt call).  This
type of infection can be easily detected and prevented.  An
entire class of products, called Filters, has grown up around the
assumption that virus infections can be prevented by redirecting
certain interrupts and intercepting the infection replication
process. It works for a few viruses.  The vast majority of
infections, though, are caused by viruses that use non-standard
I/O, and these infections cannot be prevented through interrupt
re-vectoring techniques.  Thus, filter type products - included
among them are C-4 and Flu-Shot+ - are virtually useless against
most viruses.  Yet many reviewers, and some product developers,
still believe that viruses can be stopped through re-directing
system interrupts.
     
The criteria:
     A lot of time and effort has gone into the various checksum,
encryption, logging and chaining algorithms proposed as safe
techniques for detecting viruses.  And much discussion and
argumentation has gone one regarding the various merits of high
security algorithms.  Yet, every generic application infector
that we have seen to date could have been detected by merely
checking to see if the SIZE of the file had changed.  Developing
such a virus detector requires less than an hour of programming
time and is as effective as available products costing hundreds
of dollars.   We're not suggesting that size checking should be
the criteria for detecting viruses (we know better), we are
merely pointing out the vast gulf between theory and current
reality.  We understand that viruses of today may not reflect the
situation two years from now, and we also understand that current
boot sector viruses and certain operating system viruses pose a
special case to our size example, but the first step in solving
any problem must be a solid understanding of the current state of
the problem.  And the current problem is in a different world
from the theoretical solutions proposed for it.
     An astute reader might ask at this point why we would be
concerned if the proposed solutions to viruses were overkill. 
Isn't it better, you might think, to include as much protection
as is available, to get as close to 100% security as possible? 
We think not.  Beta testing of virus products in many
corporations and our own experience with these products over the
past year has shown that, beyond a certain point of
reasonableness, increased security functions begin to hinder the
computing process.  Either increases in required run time, or
user constraints or annoying additions to the system make the
products so cumbersome to use that the user ultimately discards
them.  Alternately, false alarms and questionable product
conditions desensitize the user, and thus real virus alarms, when
they occur, are disregarded.
     Again, we are not saying that sound security principles
should not be included in a given product.  We are only
suggesting that the search for the 100% solution must have its
limits.  The theoretical discussions about batch file viruses,
viruses that can imbed themselves within a program without
changing initial branch addresses, and viruses that can infect
without making any modifications to a program are interesting and
entertaining.  But if you are selecting a product based on the
ability to detect such viruses, then you will be disappointed.
     In general then, our criteria for evaluating antiviral
programs are:

     1.   The program's effectiveness against existing viruses. 
          There are anywhere from two dozen to over 50 different
          PC viruses (depending on how you classify them) that
          can infect your system today.  If the product cannot
          detect these viruses, then it certainly cannot detect
          tomorrow's viruses.  We rated this criteria the
          highest.

     2.   The techniques used by the program to anticipate new
          viruses.  We have to admit to some subjectivity here. 
          No-one really knows what virus may pop up tomorrow, but
          reasonable people can make reasonable guesses (Tim
          Sankary is the only member of this review team who
          admits to being unreasonable).  We do expect to see
          viruses in the next few years that can imbed themselves
          inside a generic COM or EXE program without changing
          its size.  We anticipate system infectors and other
          program-specific viruses that can imbed themselves AND
          not change initial branch instructions.  (We feel these
          viruses, however, will be limited to common programs
          such as IBMBIO, IBMSYS, COMMAND.COM etc.).  We
          anticipate viruses that will encrypt themselves in such
          a way that every infection will be different (1704
          nearly achieves that now).  We anticipate boot sector
          viruses that will not need to save and execute the
          original boot sector.  We also expect viruses that will
          entirely replace system modules, such as the command
          interpreter.

     3.   The usability of the software.  This is the most
          subjective criteria and we accordingly weighted it the
          least.  We decided, however, that if we felt like
          screaming, smashing the monitor or savagely beating the
          family pets while trying to install or use the program,
          then we would subtract points for lack of user
          friendliness.

The Viruses:
     Jim Goodwin insisted that there were 61 PC viruses and that
we should test them all.  He includes in this list three versions
of the Pakistani Brain that differ only in the imbedded text and
volume label copyright display, and four identical versions of
the 1704 that differ only in their activation dates.   Lynn
Marsh, who has a new beau, and, we suspected, would like to
spend time with him, suggested that there were only 14 base PC
viruses.  Any modifications to these viruses, she insisted, were
inconsequential and should be ignored.  A compromise was reached
along the following lines:

          Any modification to a base virus that materially
          altered its ability to be detected would be considered
          a different virus for our testing purpose.  

     Frankly, the definition didn't help us much because we
continued to squabble, but it eventually worked itself out.  It
became clear that certain modifications to base viruses did
indeed materially affect our test results.  As an example, one
modification to the Israeli virus, called the New Jerusalem,
performs a format of the hard disk when it activates, and it
additionally does not have the EXE infector bug that the
original Israeli had.  When this virus activated, one antiviral
products that was able to detect the original Israeli file-delete
activation and prevent it, was unable to detect the modified
virus's format attempt.  There were numerous other such
examples.  Even machine or configuration type changes (such as
the numerous 1704 modifications) had an effect on testing under
certain circumstances.  We finally narrowed the field down to 27
distinct viruses, 11 of which were boot sector infectors.
     We realize that our test base is skewed if you compare it
to infection reporting statistics (where over 80% of infections
are boot sector infections), but we feel the sampling will become
more valid over time, since the boot infector ratio appears to be
slowly declining.  
     
The Testing:
     All testing was performed on systems with fixed disks. 
Where applicable, the infection was introduced onto the hard
disk.  The only exceptions to this were five boot sector viruses
which would not replicate onto a fixed disk.  When testing
against these floppy-only viruses, a 5 and 1/4 inch, 360KB
diskette was used.  The test systems each contained over 300
executable programs, approximately 2/3 EXE programs and 1/3 COM
programs, arranged in multiple levels of directories.  Programs
with overlay structures were also included.  DOS 2.0 and 3.3 were
both used, and testing was performed with and without the memory
resident program and shell routine - Carousel and Norton
Commander.  Monochrome and VGA graphics adaptors were also
included.
     All product detection tests were made while boot sector
viruses were already in memory and in control.  This was a
critical point for us.  For example, the Pakistani Brain is a
trivial virus to detect if you insert an infected floppy into an
uninfected system and run a detection program against it.  If you
boot from an infected diskette, however, the detection process
becomes much more difficult (since the virus traps all attempts
to read the boot sector).  We found only one generic product that
was able to detect the Brain while it was active.
     When testing against generic COM and EXE infectors, we used
two approaches.  First, we loaded the protection software onto a
clean machine and then infected it.  Second, we infected a
machine with the virus, then installed the protection software,
and then allowed the virus to continue the infection process.
     Throughout the review process, we considered a product to be
ineffective against a given virus if any of the following
occurred:

     -    The program was unable to detect the presence of
          infection activity during its normal check cycle.
     -    The system hung when the virus was introduced, or
          during the check cycle, and no warning indication was
          given by the program prior to the hang-up.  (This
          assumed, of course, that the virus ran normally without
          the prevention product being present)
     -    A loss of data occurred during the checking process.

     A product was considered to be effective against a given
virus if all of the following occurred:

     -    The product identified the presence of infection
          activity.
     -    The product was able to identify each and every
          infected component of the system, name each infected
          program, and specify the program's directory path.

     Usability ratings were loosely handled as follows:

     1.   Global detection products that required more than two
          seconds per program for a system scan (ten minutes on
          our test system) scored high on our aggravation scale.
     2.   Programs that required us to use new system command
          structures or required us to modify the way in which we
          normally interface with the operating system or our
          application programs were placed in the questionable
          category.
     3.   Programs that required constant attention to the user's
          manual in order to be useful were frowned on. 
          (Allowances were made for Tim Sankary's slow thought
          processes).
     4.   Programs that caused false alarms were given an
          annoyance ratio proportional to the number of false
          alarms.
     5.   Programs that installed in ten minutes and remained
          invisible thereafter were well liked and much
          appreciated.

     Please don't mistake our lighthearted attitude to the user
friendly category.  It's just that we could not come up with a
really objective measure here.  No matter how hard we tried, it
usually ended up being a matter of personal opinion.  Keep in
mind that we weighted the whole user interface area low in
importance.

The Products:
     We were able to identify over twenty PC products being
distributed through vendor channels and through public
domain/shareware channels.  We chose five to review that we felt
were the most commonly available and most widely used.



C-4
From McAfee Associates, 4423 Cheeney St, Santa Clara, CA 95054
408 988 3832
 
*** NOT RECOMMENDED ***

     C-4 is a classic virus filter product which is simple to
install, easy to use and creates few false alarms.  It is a
memory resident program that requires about 12K of memory (not
much) and seems to run efficiently, consuming few system
resources.  The instruction manual is brief, concise and to the
point.  It comes with an automatic install utility, and the
installation takes about 30 seconds.  From there on it's
automatic.  The checking function can be easily turned on and off
through a keyboard toggle, and a simple mechanism for excluding
"safe" programs is included.   A pop-up window appears whenever a
violation is reported, and the name of the violating program, and
its target, are displayed.  Programs that violate C-4's filter
criteria can be frozen and prevented from continuing the suspect
activity.  All in all we found this product to be well designed,
solid, easy to use and fairly unobtrusive.  A solid piece of
software engineering.
     So what's the problem?  Well, it doesn't work.  Like all
filter products, it is limited to viruses that conform to
standard operating system conventions.  These conventions include
using interrupts rather than branching directly into the BIOS,
keeping the original boot sector intact, not modifying the
command interpreter, etc.  As we all know, not all viruses play
by these rules.  
     The net result of our testing showed that C-4 was unable to
prevent or detect any of the boot sector viruses.  Additionally,
if the system was infected before loading c-4, it was unable to
detect future infections from any memory resident.
     We cannot recommend this program.



Flu-shot+  (Shareware)
from Software Concepts Design, 594 Third Avenue, NY, NY 10016
212 889 6438   

*** NOT RECOMMENDED ***

     FluShot+ is a mixture of filter program and detection
program.  Like C-4, it attempts to trap system interrupts and
catch viruses in the act of replication.  Like C-4, it is equally
unsuccessful.  The infection detection aspects of the program add
little to its ability to protect against infection, but they do
contribute substantially to the overall cumbersome and
frustrating user interface. 
     The complicated documentation and installation required by
FluShot+, however, was not our overriding concern.  The program
simply did not work.  No boot sector virus was stopped or
detected by FluShot+, and the false alarm rate was high enough to
motivate many system users to ignore a real virus infection,
whenever one could be detected.
     If we add to this the numerous quirks of the program, such
as problems running with graphics software and conflicts with
certain memory resident programs, we find little positive value
in it.
     We cannot recommend this program.
 

Sentry  (Shareware)
From McAfee Associates, 4423 Cheeney St, Santa Clara, CA 95054
408 988 3832

*** HIGHLY RECOMMENDED ***

     Every so often an easier, simpler approach really does work,
and Sentry appears to be a one-in-a-million jewel of simplicity
and effectiveness.  The most invisible product that we tested,
Sentry can be installed by anyone able to type the word
"install", and thereafter nothing more is seen or heard of it
until a virus hits the system.  When it does, it's certain to get
caught.  Sentry was the only product able to catch every one of
our test viruses.
     It does have some small faults however.  First, it
increases the system boot-up time by about 10 seconds for every
100 programs in your system.  For the average user this will not
be a problem (the average person uses less than 50 programs, we
are told).  For some folks however, this may become burdensome. 
If you are one of those rare people who use (or at least have)
2,000 programs or more, you can expect to wait over 5 minutes
extra every time you boot your system.  
     A second fault is that people who do a lot of programming or
software development will constantly be changing executable files
on the disk.  Sentry will prod you about these changes every time
you boot.  The only way to shut it up is to re-install it so that
it can take a new snapshot of the current system state.  We all
found this annoying (although, to be fair, every product that we
have seen has this same annoyance).  One way around it is to do
all compiles, links, etc. in a given subdirectory and instruct
Sentry to ignore all the happenings in that subdirectory.  This
works quite well.  If you do not frequently compile, or daily
update your software to new versions, however, then Sentry should
remain innocuous.
     A final caution about Sentry.  It does not work properly in
the DOS 4.0 environment and should not be used in this
environment.  We understand that a new version that will correct
this problem is currently under development.
     Sentry works by creating a snapshot file of all critical
system elements and comparing that snapshot file to the current
state of the system at boot time.  If you power down or re-boot
your system at least once a week, then Sentry will flag any
infection long before the infection will activate and cause
damage.  If you are running in a networked environment, or in any
other environment where the machine is seldom turned off or re-
booted, then Sentry can be manually invoked by typing the command
- SENTRY.  
     Sentry uses a unique approach to detecting a virus.  It
does not checksum the entire program, but only those areas of the
program would would have to change when any virus attaches to the
program.  This allows it to execute very rapidly, and thus makes
periodic scans of the entire system feasible.  This separates
Sentry from all other products.  The second separator, of course,
is that it is effective against all of the viruses that currently
exist.  We believe that this effectiveness will continue for new
viruses.  
     
Virus-Pro
From International Security Technologies, 515 Madison Avenue, NY,
NY 10022   212 288 3101

** RECOMMENDED **

     Virus-Pro is a product designed for large corporations, and
we include it here for those researchers studying epidemiological
data using multiple computers as a study base.
     Virus-Pro is much more than a virus detector.  Virus-Pro
includes sophisticated audit trails and history information that
can be used track the origin of an infection within an
organization, and to monitor the use and movement of programs
from PC to PC.  It does require a fair amount of run time for the
checking process, and a dedicated Virus-Pro systems administrator
or co-ordinator is needed, but it is an excellent system level
product.
     The basic function of Virus-pro is to monitor the status of
the executable programs on the logical drives and to report on
changes and exceptions.  Virus-Pro stores five parameters about
each executable or hidden file in a scan file.  These parameters
are:
     (1)  The name, extension and path
     (2)  The size in bytes
     (3)  The date-time stamp
     (4)  The attributes (hidden, system, and read-only).
     (5)  A checksum of the program

     In addition, the program stores information about the
logical drive's boot track.  Virus-Pro then compares the scan
file with both a prior scan file from the same logical drive and
a baseline file which has been created using scans of individual
software distribution diskettes.  Differences in or matches to
one or more of these five parameters are used to determine the
presence of infection.  
     Administrative software makes it easy for an organization's
Virus-Pro co-ordinator to prepare diskettes for site co-
ordinators.  Each site co-ordinator has similar facilities to
make Virus-Pro diskettes for his or her PC "owners".  PC owner
diskettes include a disk scanning and analysis program.  Site co-
ordinators use a program called MAKEBASE to place data extracted
from vendor diskettes into baseline files which a baseline
analysis program compares with the disk scan outputs.  The
analysis can spot viruses, pirated software, wrong program
versions and a host of other inconsistencies of interest to a co-
ordinator.  Two system-wide administrative programs maintain
master files of site co-ordinators and PC owners, print complete
name/address/phone number lists of co-ordinators and owners,
prepare diskettes, and provide other administrative functions.
     Virus-Pro is the most comprehensive system level antivirus
product that we have seen or heard of.  It does however require
more maintenance than stand-alone utility antiviral products, and
it did fail to catch four of the boot sector viruses (but caught
all others).  In spite of this, We feel that it provides a fair
level of protection, and excellent audit trail capabilities for
tracking virus spread.
     A note of caution:  This is not a product for the individual
user of a stand-alone system.  It is specifically designed for
the corporate environment.


Disk Defender
From Director Technologies, 906 University Place, Evanston, IL 
60201   312 491 2334

** RECOMMENDED **
     
     Disk defender is an add-on board for IBM PCs and
compatibles.  The product write protects the hard disk from
erasure or modification to programs or data files that do not
require frequent changes.  It can therefor protect against
viruses trying to attach to system or application programs, or
even to the boot sector.  It blocks their attempts and provides a
visual indication that disk writes are being attempted to a write
protected area.
     A switch attached to the board write protects the entire
disk, just a portion, or none of the disk.  The switch can be
set, then removed and stored in a secure place.  In addition, the
board allows a portion of the hard disk to be write protected,
while allowing normal writes to other areas.
     Disk defender allows the hard disk to be divided into two
active DOS partitions and allows the user to designate an area or
zone as read only or as read/write.  Indicator lights on the
switch box illuminate when an attempt is made to write to a
protected partition.
     The disk defender is one of the most effective antiviral
products available for protecting the hard disk..  Clearly, if a
virus cannot physically access its host program, then it cannot
infect the system.  It does not, however, protect against floppy
viruses.  There is no software utility included with the package
to prevent or detect floppy boot sector infectors, for example. 
Thus the 5 floppy based boot viruses lived and prospered quite
happily in the system with Disk Defender installed.    There are
some other drawbacks as well.  Installation is non trivial and
requires a backup of all data and a re-format of the hard disk. 
Then all data and programs must be restored.  Disk defender also
requires that files be re-organized, and some application
programs will have to be reconfigured if they use the C drive for
temporary storage.  Thus, a degree of flexibility is lost which
may be unacceptable to some people.
     In spite of its limits, however, Disk Defender is a highly
reliable and secure product for protecting your hard disk.




Jim Goodwin, Lynn Marsh and Tim Sankary

From the HomeBase Virus Research Group
408 988 4004