textfiles/virus/verify_r.txt

  
  
  
  
  
  
   Virus Verification and Removal -- Tools and Techniques 
  
                       David M. Chess 
            High Integrity Computing Laboratory 
            IBM Thomas J. Watson Research Center 
                    Yorktown Heights, NY 
  
                       Nov. 18, 1991 
  
  
  
HISTORY 
  
  
This is an updated version of a paper that originally 
appeared in the November 1991 issue of Virus Bulletin. 
Since this sort of technology is continually evolving, it 
seemed reasonable to make an update available on the net; in 
particular, the virus-removal language has been considerably 
enhanced since the paper was originally written.  Comments 
are welcome, on VIRUS-L (comp.virus), or directly to the 
author (chess at watson.ibm.com). 
  
  
  
  
INTRODUCTION 
  
  
The first line of defense against computer viruses consists 
of programs that detect that something is probably wrong. 
These include modification detectors, integrity shells, 
known-virus scanners, access-control programs, and similar 
things.  Their main function is to alert the user of a 
machine that a virus, some virus, is probably present.  The 
important thing is the alert; since something is likely to 
be wrong, the user should stop what he is doing, and take 
action to correct the problem.  It doesn't matter much at 
this stage what the alert says; a first-line anti-virus 
system that always said simply "Something virus-like may be 
going on!" would be sufficient for most environments, if it 
was usually right. 
  
Once the alert has been given, and the infected system taken 
out of immediate contact with other systems, other kinds of 
software become important.  Before we can decide how to 
clean up an infected system, and even where else to look for 
infection, we need to know exactly what the infection 
consists of.  Once that has been determined, we can take 
steps to restore the infected parts of the system to an 
uninfected state, and to recover from any other damage the 
virus may have caused.  This paper is a description of one 
part of the second-line toolbox, the virus verifier and 
remover. 
  
  
  
VIRUS VERIFIERS 
  
  
A virus verifier is a program that, given a file or disk 
that is probably infected with a given virus, determines 
with a high degree of certainty whether the virus is a known 
strain, or a new variant.  This is, of course, important to 
know: if the virus is different from any known strain, it 
will have to be analyzed for new effects before we can be 
confident that we know just what to do to clean up after it. 
On the other hand, if the virus is identical to a known 
strain, we already know what to do.  It is particularly 
important to perform verification in a program that attempts 
to automatically remove the virus infection from an object, 
restoring it to its original uninfected form. 
  
Abstractly, a verifier is a program that, given another 
program as input, determines whether or not the given 
program is part of the set of possible "offspring" of a 
particular virus.  For many classes of viruses, including 
all the viruses actually widespread at the moment, this is 
easy to do.  Almost all known viruses consist almost 
entirely of code that does not change from infection to 
infection, except perhaps for a simple XOR-type garbling, 
and data areas that are either constant, or change in simple 
ways (or that can be ignored entirely for the purposes of 
verification).  Given a suspect file F and a known virus V, 
it is therefore always relatively simple to answer the 
question "is F a file that could have been produced by 
infection with virus V?".  It is an open question of some 
theoretical interest whether or not some future virus might 
make this harder to do!  Reliably determining whether a file 
is infected with any virus at all is of course known to be 
impossible, but we have no similar result about determining 
the presence of a specific virus. 
  
There are various concrete decisions and tradeoffs involved 
in writing a virus verifier; this section will list a few of 
them, and the next sections will describe the 
verifier/remover currently being developed and used at the 
High Integrity Computing Lab at IBM's Watson Research 
Center. 
  
A verifier may be an independent tool, or it may be 
integrated into a virus detector.  An integrated 
detector/verifier can be quicker and more convenient, since 
there's no need for a user to find and run a verifier once 
the detector goes off.  On the other hand, since most copies 
of any virus detector will never in fact detect a virus 
(most of the world's computers are not infected, after all), 
integrating a verifier along with the detector is in some 
sense inefficient, in that it adds significant code to the 
detector that may never be used.  Given how much more 
expensive human time is than CPU time and disk space these 
days, integrated tools are likely to be more cost-effective 
in the long run.  On the other hand, detection and 
verification will always be two different activities, 
because it is very desirable for a detector to detect small 
variants of known viruses as viruses, whereas a verifier 
must be able to identify any variation as a variation. 
Detection algorithms are typically run very often, and must 
be fast.  Verification algorithms, on the other hand, are 
run rarely (only when a virus is detected), and speed is 
typically not a major issue. 
  
To determine whether or not a given object is infected with 
a known strain of a virus, a verifier must know what the 
known strain looks like.  This may be done either with an 
actual copy of the code of the known strain of the virus, or 
by using a CRC or similar modification-detection algorithm. 
It's not generally desirable to include the entire code of a 
virus with widely-distributed tools, for obvious reasons! 
On the other hand, even a good difficult-to-invert digital 
signature algorithm is not as reliable as a byte-for-byte 
comparison, and it is vulnerable to a virus author 
intentionally creating a variant that looks to the verifier 
like a known strain.  (This can be made arbitrarily hard 
through the use of cryptographic checksums and related 
technologies, at some increase in runtime and complexity.) 
  
Lastly, a verifier may use either special-purpose code, with 
one or more routines being written in some compiled language 
for each new strain discovered, or it may be written as an 
interpreter for a high-level virus-description language.  A 
high-level language is generally simpler to program in 
reliably; on the other hand, this is only true because it is 
less expressive, which implies that there will be cases 
(viruses that are exotically self-garbling, for instance) in 
which it will be necessary to drop into the lower-level 
programming language again. 
  
  
  
VERV - A PROTOTYPE VIRUS VERIFIER AND REMOVER 
  
  
At HICL, we are currently using and developing a virus 
verifier and remover called "VERV" for PC-DOS viruses.  The 
current version can verify over 40 different viruses and 
variants, which accounts for nearly all of the actual 
infections that we see in day-to-day operation.  It has 
recently been enhanced to attempt to remove about a dozen of 
the most common file-infecting viruses (we have other tools, 
which will eventually be integrated, for removing 
boot-sector-infecting viruses).  As well as being used in 
the lab, and as a research prototype, VERV is used by IBM's 
internal Computer Emergency Response Teams (CERTs), as part 
of routine incident handling. 
  
It is an independent tool at the moment; in the long run, we 
expect to integrate it with our other anti-virus programs. 
It can use either a CRC algorithm or a byte-for-byte 
comparison to verify the identity of a virus.  In the 
laboratory, we use the byte-for-byte compare to test new 
samples against old ones.  In the field, our users use the 
CRC algorithm to verify the virus in infected objects before 
applying cleanup measures. 
  
VERV includes an interpreter for a small virus-description 
language.  Virus-description languages, for this and other 
purposes, have been around for some time; Christoph Fischer 
at the University of Karlsruhe, Morton Swimmer in Hamburg, 
Alan Solomon in the UK, and no doubt many others in the 
field, have worked on similar things (personal 
correspondence; one motivation for this paper is to 
encourage others, who have perhaps done it better, to 
publish their work).  VERV's language is very simple, and 
provides for lower-level hooks (instructions to call 
special-purpose C routines) when a virus requires actions 
that cannot be described in the high-level language.  We 
will describe the language in some detail, not because it is 
particularly interesting as a language, or because we think 
we have it all correct and optimal, but rather so that other 
people working on the same sorts of things can benefit from 
both our ideas and our mistakes.  We hope this will help 
inspire continued discussion and exchange. 
  
  
  
  
VERV'S VIRUS-DESCRIPTION LANGUAGE 
  
  
  
The file from which VERV reads virus descriptions consists 
of a number of virus-description blocks.  Each block has the 
following structure: 
  
          One or more VIRUS records 
          A NAME record 
          One or more LOAD records 
          Zero or more DEGARBLE and related records 
          Zero or more ZERO records 
          One or more check records 
          Zero or more REPAIR blocks 
  
For instance, the block for the Slow-1721 virus currently 
looks like this: 
  
  VIRUS slow slow-1721 
  NAME the Slow-1721 virus 
  LOAD P-COM 0 6B4 
  LOAD S-EXE 0 6B4 
  DEXOR1 001E 06AD 0012 0000      ; Degarble the code 
  DEXOR1 00EB 0159 0061 0001      ;   and the data area 
  ZERO 0012 1                     ; Zero the one-byte code-garble key 
  ZERO 0061 1                     ;   and the data-garble key 
  CODE  0000 00EA 38d5dc08        ; Code up to first data area 
  CONST 0144 014E 0ff22ad9        ; COMMAND.COM 
  CODE  015A 063C 74e00962        ; Code between data areas 
  CODE  0657 06AD ad3b0b41        ; After the second data area 
  
The VIRUS records simply give a list of one-word aliases for 
the virus, that are used on the command line to tell VERV 
which virus to look for.  These aliases are not the full 
primary name of the virus (that is given on the NAME 
record); they are just short abbreviations that the user can 
use on the command line. 
  
A very useful extension here would be for VERV to support 
virus families, so that a single command would cause testing 
for all members of the Jerusalem family, or the Flip family, 
and so on.  When integrated into the virus detector, of 
course, the detector will directly inform the verifier which 
virus or viruses to test for. 
  
The LOAD records describe where in an infected object of a 
given type the virus can be found.  The tokens on a LOAD 
record are an object type, followed by either an offset and 
a length, or the word SPECIAL and a number.  The offset 
tells VERV where, relative to the effective entry-point of 
that sort of object, to start loading; the length tells how 
many bytes to load.  For viruses that are not always at a 
fixed offset from the initial entrypoint, the SPECIAL 
keyword causes VERV to invoke an internal routine, coded in 
C, to perform the loading. 
  
The Slow virus is an EXE-infector, and a prepending COM 
infector; the LOAD records in this example tell VERV to load 
the first 06B4 bytes of a COM-format file, and the first 
06B4 bytes after the entry point of an EXE-format file. 
(EXE-format files are those that begin with the letters 
"MZ"; DOS loads these differently from COM-format files, 
which begin with any other bytes.)  Other object types 
supported include: 
  
o   E9-COM, for viruses that infect COM files by changing 
    the first three bytes to a long jump to the virus (E9 is 
    the hex code for a long jump), 
o   E8-COM, for viruses that infect COM files by changing 
    the first three bytes to a long CALL to the virus (E8 is 
    a long call), 
o   MBR, for viruses that infect hard disk master boot 
    records and diskette boot records, and fit in a single 
    sector, 
o   DISKETTE, for other sorts of diskette infectors (those 
    that do not fit in a single sector), 
o   HARDDISK, for other sorts of hard disk infectors (those 
    that infect system boot records, and/or occupy more than 
    one sector). 
  
A description block will have as many LOAD records as there 
are types of object that the virus can infect. 
  
The DEXOR1 records tell VERV to perform a certain common 
type of degarbling: a one-byte XOR with data to be found at 
a fixed offset in the virus.  The details are not terribly 
important here.  A more general record, consisting of just 
the word DEGARBLE followed by a number, causes VERV to 
invoke an internal C-language routine to perform degarbling. 
  
Once the loading and degarbling have been done, VERV has a 
complete "virus image" in its internal buffer.  A 
command-line switch (described later) can instruct VERV to 
save the contents of this buffer to a file, for later 
examination. 
  
The ZERO records describe variable areas within the virus, 
that should be set to zero before checks are done.  This is 
really just a convenience, to reduce the number of 
check-type records needed. 
  
There are three basic types of check records, describing 
different tests to be done on the degarbled and zero'd virus 
image now in the buffer: 
  
o   CODE records describe areas of virus code.  The numbers 
    given are the start and end offsets of the area, and the 
    expected CRC value of the data there.  VERV uses a 
    31-bit CRC, with a custom polynomial.  This is not 
    strongly resistant to intentional reverse engineering; a 
    more difficult-to-invert algorithm may be desirable 
    later on.  If any CODE areas are found to be different 
    than expected, VERV will report that this is not the 
    usual strain of the virus. 
o   CONST records describe constant areas that should not 
    change, and whose values effect the actual running of 
    the virus.  CONST areas are currently treated exactly 
    like CODE areas. 
o   TEXT records describe areas of the virus that are not 
    expected to change, but do not significantly effect the 
    operation of the virus.  If a sample differs from the 
    given description only in one or more TEXT areas, VERV 
    will report a "text variant" of the virus.  This is 
    useful for message areas within a virus that are not 
    actually used, or that are simply displayed to the user. 
    These areas can be interesting in tracking how the virus 
    is spreading, by correlating incidents that involve the 
    same "text variant", but they do not effect cleanup or 
    prevention. 
  
Normally, VERV performs its CRC calculation on each area 
within the virus, and compares the results to the expected 
values.  A command-line switch (described in more detail 
below) can be used to tell VERV to read a standard copy of 
the virus from another file instead, and do byte-by-byte 
comparison between the two.  This is more reliable, but of 
course it requires having a sample of the usual strain of 
the virus present to verify against. 
  
Another example, illustrating the use of special C routines, 
is the block for the 1701 virus: 
  
  VIRUS 1701 
  NAME the 1701 virus 
  LOAD E9-COM -1 06A5 
  DEGARBLE 1 
  CODE  0001 0026 19989c7e        ; Degarble, MOV, jmp-in 
  CODE  0076 06A4 c03a91c5        ; Main code 
  
Here, the "DEGARBLE 1" record causes VERV to invoke an 
internal routine to degarble the data in the buffer, using 
the 1701's own algorithm.  It would be possible to enhance 
the virus-description language enough that the 1701's 
degarbling algorithm could be expressed in it directly. 
This would complicate the language considerably, though, and 
would somewhat lessen the advantage that a special 
high-level language has over native C code; so far, we have 
decided against such enhancements. 
  
  
  
REPAIR 
  
  
For many viruses and many infected objects, it's possible to 
restore the object to what it looked like before it was 
infected, or at least to a state in which it will function 
in just the same way.  Unfortunately, this isn't always 
possible; the classic example is the 1813 (Jerusalem) virus 
infecting an EXE-format file.  While it's usually possible 
to undo the infection, sometimes the resulting file is 
missing data that was in the uninfected original, and it's 
not always possible to tell that this has happened.  The 
best an 1813-remover can do on the EXE file, therefore, is 
something that is quite likely to work, but might not.  In 
most cases, though, sufficiently-reliable repair is 
possible, and particularly in large infections of 
non-critical machines, repair is sometimes a cost-effective 
option. 
  
A description of a virus in VERV's language includes one 
repair block for every type of object that the virus may 
infect.  Each repair block consists of a header record 
"REPAIR <object type>", followed by one or more 
repair-operation records.  Currently defined repair 
operations include: 
  
o   an FCOPY_TO record, that copies bytes from the start of 
    the infected file up to a given number of bytes from the 
    virus entry point (this is used to remove appending 
    viruses), 
o   an FCOPY_FROM record that copies bytes from the infected 
    file, starting a given number of bytes from the virus 
    entry point, and ending a given number of bytes before 
    the end of the file (this is used to remove prepending 
    viruses), 
o   a BWRITE record, that copies so many bytes from a given 
    offset in VERV's internal buffer (which initially holds 
    an image of the virus) to a given offset in the file 
    being repaired (this is used, for instance, to repair 
    the first few bytes of an infected COM file, or the 
    header of an infected EXE file), 
o   a BREAD record, that loads a given number of bytes from 
    a given offset (relative to the start of the infected 
    file) into VERV's buffer, 
o   an EXE_LENGTH_BUG record, that tells VERV that this 
    particular virus has the common bug that it assumes that 
    the image length in the header of an EXE file is the 
    same as the file's length, and therefore damages (by 
    overlaying some data) any EXE file that contains data 
    after the EXE image, 
o   a 64K_COM_BUG record, which tells VERV that this virus 
    has the common bug that it assumes that any file it 
    thinks of as a COM file must be less than 64K bytes 
    long, 
o   an EXE_LENGTH_ADJUST record, that treats two words 
    within VERV's buffer as the "page count" and "last page 
    length" fields from a DOS EXE-file header, and subtracts 
    a given constant value, adjusting them accordingly, 
o   an R_SPECIAL record, to cause VERV to invoke an internal 
    C routine to perform some function not directly 
    implemented in the language. 
  
For instance, the repair block for the usual 1813 or 
Jerusalem virus currently looks like this: 
  
  REPAIR S-EXE 
    EXE_LENGTH_BUG 
    FCOPY_TO -0C5 
    EXE_LENGTH_ADJUST 0053 0051 0710 
    BWRITE 0043 0010 2       ; Fix SP 
    BWRITE 0045 000E 2       ; Fix SS 
    BWRITE 0047 0014 2       ; Fix IP 
    BWRITE 0049 0016 2       ; Fix CS 
    BWRITE 0051 0002 4       ; Fix image length 
  * Fixing COM files 
  REPAIR P-COM 
    64K_COM_BUG 
    FCOPY_FROM 0710 -5 
  
The two BUG records cause VERV to print warnings to the user 
that some files may not function correctly, and to refuse to 
repair (later versions may offer to erase) any files that 
are obviously not correctly repairable.  The FCOPY records 
pick out just the part of the file that does not contain the 
virus, and the EXE_LENGTH_ADJUST and BWRITE records restore 
and replace approximately the original EXE file header.  EXE 
files that are successfully repaired will differ from the 
original file only in having been rounded up to a multiple 
of sixteen bytes (and the corresponding change in the EXE 
file header). 
  
After repair is completed, VERV restarts processing on the 
repaired file, to ensure that there is not another instance 
of the virus present.  If the virus is present in the file 
multiple times, all will be removed.  Once VERV is 
integrated with a virus scanner, the repaired file will be 
automatically re-scanned for all viruses, and any found will 
be re-verified and removed. 
  
Repair processing is only performed if the user has 
requested it on the command line, and if VERV finds that the 
virus is indeed exactly the known strain of the virus.  In 
small infections, or in situations where correct operation 
of the objects involved is particularly crucial, we continue 
to recommend that infected objects be destroyed (files 
erased, diskettes formatted, and so on), and replaced from 
uninfected sources. 
  
  
  
  
VERV OPTIONS 
  
  
The functions of VERV's command-line switches include: 
  
o   Reading the virus to be tested from an image file, 
    instead of from a normally-infected object; this can be 
    useful, for instance, in testing a boot-sector infector 
    that has been received as a binary dump of the boot 
    sector rather than on diskette. 
o   Overriding the default virus-description file (contained 
    within VERV.EXE itself), allowing easy testing of new or 
    experimental descriptions. 
o   Producing detailed progress messages and data displays 
    during processing, to help pinpoint differences found or 
    errors encountered. 
o   Specifying that, rather than using a CRC, VERV should 
    compare the relevant parts of the object to be tested 
    with a standard sample of the virus stored in an image 
    file, or a standard infected specimen. 
o   Producing a dump of the virus image, after all 
    degarbling, but before any zeroing has been done.  This 
    image can then be used for storage, analysis, or 
    transmission, or for later use as input to VERV for 
    byte-by-byte comparisons. 
  
  
  
  
STATUS AND FUTURE GOALS 
  
  
VERV is currently in use by a small number of people within 
IBM who deal with virus infections.  Its availability has 
greatly reduced the time spent by technical people in doing 
semi-manual verification, and has therefore sped up the 
response time to virus incidents.  Adding a typical 
newly-analyzed virus to VERV is generally quite simple, 
involving a few lines in the VERV language, and sometimes a 
small piece of C code to handle a new garbling algorithm. 
  
The virus-removal language has just recently been 
implemented, and is not yet in wide use. 
  
Our near-term plans for VERV include support for families of 
viruses, and the ability to verify a virus in a number of 
objects at once.  This will ease integration with our virus 
detectors; when a detector detects a signature that 
corresponds to a virus, or a family of viruses, in a number 
of files, it will be able to verify the identity of the 
virus with a single call to VERV. 
  
If transmission bandwidth, CPU cycles, and disk space were 
free, and programming was easy, every workstation would be 
protected by a seamless "immune system".  Objects infected 
with existing viruses would be detected automatically, the 
identity of the virus verified and reported to a central 
location, and the object destroyed or repaired, with minimal 
user intervention.  New viruses would be detected 
automatically with some high degree of confidence, 
first-pass signature patterns would be extracted 
automatically where possible and communicated to a central 
clearinghouse, along with a sample of the suspicious object. 
Viruses would very rarely, if at all, spread widely. 
  
One of our main focuses at HICL is studying what part of 
that ideal scenario is feasible, in both current and future 
systems.  The prototype VERV is a small part of our 
experimentation with parts of that system that are also 
immediately useful to users in the near term.  We would 
welcome similar descriptions by others in the field, of work 
that they are doing in similar directions.