textfiles/programming/epstein.txt

       Neural Nets Improve Hospital Treatment Quality and Reduce Expenses

                                  June 9, 1990
                              by Jeannette Lawrence


    A new hospital information and patient prediction system has improved
    the quality of care, reduced the death rates and saved millions of
    dollars in resources at Anderson Memorial Hospital in South Carolina.
    The CRTS/QURI system uses neural networks trained with BrainMaker (from
    California Scientific Software) to predict the severity of illness and
    use of hospital resources.  Developed by Steven Epstein, Director of
    Systems Development and Data Research, the CRTS/QURI system's goal is to
    provide educational information and feedback to physicians and others to
    improve resource efficiency and patient care quality.

    The first study showed that the program was directly responsible for
    saving half a million dollars in the first 15 months even though the
    program only included half of the physicians and three diagnoses.  Since
    then, the number of diagnoses and physicians included in the program
    have increased.  The quality of care has improved such that there are
    fewer deaths, fewer complications, and a lowered readmission rate.
    Expenses have been reduced by fewer unnecessary tests and procedures,
    lowered length of stays, and procedural changes.

    Over the past five months the hospital's accounting department has
    reported a savings in the millions of dollars, although it is difficult
    to say exactly how much of this is directly due to CRTS/QURI.  According
    to Epstein, "The hospital may now be experiencing a "spillover effect"
    of new physician knowledge and behavior into areas not initially
    targeted by the CRTS/QURI system." The reported success has motivated
    several other hospitals to join in the program and has provided the
    impetus to begin a quality program with the state of South Carolina.

    At the core of this new system lies several back-propagation neural
    networks which were designed using the BrainMaker program on a PC.
    Individually trained neural networks learn how to classify and predict
    the severity of illness for particular diagnoses so that quality and
    cost issues can be addressed fairly.  After attempts to use regression
    analysis to predict severity levels for several diagnoses failed,
    Epstein turned to the BrainMaker program for a new approach and taught
    his neural networks to classify and predict severity with 95% accuracy.
    The neural networks are also used to predict the mode of discharge -
    routine through death - for particular diagnoses.

    Without these neural networks, the system could not automatically
    compare cases with the same diagnoses and like severity levels.  Most
    commercially available severity programs provide mortality as the only
    measure of quality.  Physicians typically reject systems which only
    compare costs or mortality rates as being too crude or unfair.

    In order to provide more meaningful feedback and valid comparisons
    within homogeneous groups of patients, the CRTS Acuity Indexing Scale
    was developed.  Patients are rated for the severity of illness on a
    scale from -3 to +3.  Training information is based upon the length of
    stay in the hospital which has a direct relationship to the severity of
    the illness (acuity).  When making predictions, a -3 patient is expected
    to require the least hospitalization and resources, a 0 patient average,
    and a +3 patient the most.  Cases which are run through the trained
    network can then be indexed in a database by their severity level and
    retrieved by such for comparisons.



                             Neural Network Designs

    The neural network was trained to make the severity prediction using
    variables of seven major types: diagnosis, complications/comorbity, body
    systems involved (e.g., cardiac and respiratory), procedure codes and
    their relationships (surgical or nonsurgical), general health indicators
    (smoking, obesity, anemia, etc.), patient demographics (race, age, sex,
    etc.), and admission category.

    Using a neural network to learn the effects that these variables have on
    the severity of illness takes the CRTS/QURI a step beyond the other
    programs available for indexing severity of illness.  With a neural
    network, it is possible to discover relationships that were previously
    unrecognized.  There is no need to define how the variables are related
    to the output, the neural network learns it on its own.  BrainMaker
    provides the ability to view how any one of these variables effects the
    output of the network.  Using BrainMaker's neuron graphics any input can
    be varied over some range and the effect that this has can be plotted.

    The neural network learns (just as profession literature states) that
    elderly patients with the same diagnosis require more care than younger
    patients and that patients with family support tend to require shorter
    hospital stays.  Unlike traditional programs, there is no need to
    translate premises like these into program statements or formulas.  The
    neural network learns these and other associations by viewing case after
    case until it picks up the patterns using the back-propagation
    algorithm.

    Three years of patient data was chosen for training.  There were
    approximately 80,000 patients to choose from and 473 primary diagnoses.
    The cases were pre-selected with multiple regression techniques to
    eliminate outliers so that no "bizarre" cases were used for training the
    neural network.  For a given diagnosis, about 400 to 1000 cases were
    used to train on.

    Data was collected from automated medical records, the QURI program
    (explained in the next section), and Federally required files for
    Medicare patients.  These were downloaded and read into the CRTS
    database which is in an rBase format on a PC.  The selected training
    data was output from the rBase file as a dBase file because there is a
    direct link to BrainMaker this way.  The dBase file was read by
    BrainMaker's preprocessor called NetMaker which automatically translates
    the data into neural network files for BrainMaker.  These files define
    the network architecture, the screen display, and the training and
    testing data in BrainMaker format.  After these automatic conversions
    were done, the neural network was trained in BrainMaker.

    During the initial neural network design phase, decisions had to be made
    as to which variables were important so that unnecessary data collection
    could be avoided.  In order to help make this decision, Epstein used
    BrainMaker's Hinton diagrams to see which inputs affected the trained
    network the most.  These diagrams present a picture of connection
    strengths (weights) at the hidden and output layers.  A neuron which
    connects to all the neurons in the next layer with the same strength
    transmits no useful information.

    Two neural networks for each diagnosis were trained - one to predict the
    use of resources and the other to predict the type of discharge.  For a
    single diagnosis network, there are 26 input variables and one output
    variable.  BrainMaker trained in about 4 hours using a NorthGate 386
    running at 33 MHz.  Epstein began training with a .1 tolerance which
    means every prediction for every case must be 90% accurate.  Then he
    lowered the tolerance to .085 and eventually stopped with the network
    trained at .05 training tolerance (95% accurate).  Then a test set of
    cases was run through.  The BrainMaker program has built-in testing
    capability.  In addition, Epstein and Jones verified the results of the
    network before using the network to categorize patients and make
    predictions.


                              The CRTS/QURI System

    Once the cases are indexed according to severity level, the CRTS
    (Computerized Resource Tracking System) can be used to provide
    educational feedback to physicians concerning quality care issues and
    the use of hospital resources.  CRTS combines computer software which
    produces automated graphs, reports and summaries along with a
    comprehensive educational format.  An artificial intelligence program,
    VP Expert, is used to generate individual physician's reports.

    The CRTS program is more affordable than other indexing systems in that
    no additional chart abstracting is necessary.  The majority of
    information is retrieved from the hospital's "mainframe" computer.
    Information sources include the medical records abstract, UB82 files
    (medicare), and QURI data.

    The "QURI" portion of the CRTS/QURI system is an integrated data
    collection and reporting program.  Epstein developed this stand-alone
    menu-driven, user-friendly software program which provides computer
    integration for departments which provide additional data for quality
    analysis.

    QURI stands for Quality assurance, Utilization review, Risk management
    and Infection control.  Quality Assurance monitors various criteria for
    specific procedures and diseases.  Utilization Review monitors resource
    related items such as admission justification and intensity of services.
    Risk Management determines things that put a patient or hospital at risk
    (such as a fall out of bed), and Infection Control monitors infections
    acquired during the hospital stay.

    CRTS is used for evaluation at various levels.  At the hospital level,
    comparisons are made to data published by the profession to identify
    areas of major concern for the hospital.  Individual diagnoses that show
    problems of quality and/or finance are identified.  At the physician
    specialty group level, treatment for various severity levels of a
    diagnosis are discussed.  At the individual physician level, one page of
    written summary and five pages of comparative graphs are provided.  This
    confidential report shows how a physician's patients compared to the
    hospital wide average for a particular diagnosis.  Comparisons are
    provided for areas such as the types of procedures used, ancillary
    services used, patient demographics, infection rate, complications,
    readmissions, and predicted (versus actual) illness severity level,
    charges, and mortality rate.  A summary of problem areas and a list of
    suggestions are also included in the report.

    CRTS is implemented in two primary modules 1) collecting and analyzing
    the data with neural networks, and 2) conducting the physicians'
    education program.  Epstein, who originally designed the system for his
    dissertation, is in charge of the data-related portions.  Dr. Fred
    Jones, Executive Vice President of Medical Affairs at Anderson Memorial,
    sets up physician education (feedback) programs, trains them in the use
    of system, and verifies trained neural networks results.


                               Physician Response

    Getting the physicians to accept the new system was no easy task.
    "Initially some of the physicians must have thought the program was a
    Communist Plot.  One even stood up in an early meeting to declare the
    whole thing 'crap'," says Epstein.  He explains that Jones has done a
    tremendous job in getting the physicians involved.  The physicians are
    given a say as to what information is important, what should be in the
    reports, and how to use the data.  Epstein continues, "The problem we're
    having now is that we're OVERWHELMED with requests for more diagnosis
    studies and more special reports from the physicians."


                       What Can Be Learned from CRTS/QURI

    The QURI system includes the following components: infection
    surveillance, drug use evaluation, blood usage review, surgical case
    review, monitoring and evaluation, physician demographics, case
    review/peer review, incident/occurrence monitoring, claims management
    and utilization review.  This information, as well as information from
    patient's charts is used to provide reports, graphs and charts to
    individual physicians or specialty groups.

    Hospital-wide comparisons can be trended over time and be compared to
    statewide averages or averages reported in the literature when
    available.  Comparisons between the 473 diagnoses can be evaluated and
    compared to literature or statewide averages when available.  Using the
    severity groups, computer generated graphs and charts can be presented
    to physician specialty groups and/or individual physicians displaying
    how they compare to their average hospitalwide peer or to their average
    within a specialty group for many resource and quality issues (see
    Comparative Information sidebar).  Cases with similar severity levels
    can be found and treatments compared.  Trends are also revealed in these
    same graphs depicting how a physician compares throughout severity
    levels.  A physician or group may practice more efficiently and with
    improved quality when treating higher severity level patients.  More
    indepth studies can be provided, such as how a physician's expired
    patient with a certain severity level was treated, what procedures and
    services were used, etc., versus other similarly ill patients.
    During a report review a question may come up such as why did Risk
    Management note that there was a problem?  One possible scenario might
    be that the patient fell out of bed and a closer look at the report
    would show if Quality assurance reported that too much sedation was
    given.  One report from the CRTS brings it all together.


                        How It Began at Anderson Memorial

    The first diagnosis chosen for the program was DRG89 - pleurisy/
    pneumonia.  Compared to the national norm, the hospital mortality rate
    was higher.  In addition, the hospital was losing money from cost
    overruns since insurance companies and medicare coverage have limits on
    what they will pay.  It was learned that pneumonia of unknown etiology
    had the highest mortality.  The reports showed that patients of family
    practitioners had longer lengths of stay.

    A typical scenario unfolded.  Physicians would start their patients on
    antibiotics, wait for lab results (which were inconclusive more than 25%
    of the time) then ask for an internal medicine consult on the third day.
    Then treatment started all over.  These patients stayed an average of 4
    days longer and the hospital averaged a $2764 loss.  Since this
    discovery, Family Practitioners now get an earlier consult.  Further
    research found that the sputum collection procedure was a problem.  The
    hospital now uses respiratory therapy to collect specimens.  Samples are
    quickly tested and thrown out if they are spit instead of sputum from
    the lungs.  The inadequate sputum rate decreased from 26% to 5%.  The
    improvements caused a drop of $1091 in treating an average case of
    pneumonia during the pilot program even though only half of the
    physicians were involved in the program.

    Other improvements for DRG89 followed.  The length of stay and charges
    continued to fall through 1989.  Complications and mortality rates also
    decreased.  The use of ancillary services and ICU decreased.  When
    another diagnosis was added to the program, cerebovascular disease, the
    hospital experienced a decrease in total charge of $1043 per patient,
    even though CAT scans increased.  Average length of stay also decreased.
    Anderson Memorial is now using nine diagnoses in its program.

                                What's Downstream

    In response to the national concern for the quality of health care, the
    South Carolina Hospital Association voted to approve a statewide Quality
    Indicators Project with the state.  The tracking method was developed by
    the Systems Development and Data Research department at Anderson
    Memorial.  Epstein is currently working with the state of South Carolina
    to develop a software package which will look at quality issues for
    statewide comparisons.

    Seen as only the beginning, Epstein hopes to use the results of this
    statewide quality project as a basis for designing another neural
    network for use in the hospital.  This neural network would screen all
    patient charts and other data to provide a quality index (based upon the
    statewide issues such as mortality, infections, etc.).  He proposes that
    the results could be used to decide which cases should be reviewed,
    instead of randomly picking charts to look at.  "There are too many
    patients and too much data for someone to look at every single one and
    decide which ones have quality issues that need looking into with a peer
    review," Epstein says.  This new neural network would make educating
    physicians a much more efficient process as well as improve the quality
    of care even further.

    Currently, Anderson Memorial is working with three other hospitals as
    they start their own program.  A recent presentation of the CRTS/QURI
    system at an American College of Physician Executives conference
    produced an overwhelming response.  The hospital has been beseiged with
    requests for more information from other interested hospitals.

    In the very near future at a meeting with seven hospitals from a
    hospital network, Epstein will propose a hospital-group program.  In
    this program, there would be one neural network trained for each
    hospital, plus one for all the hospitals using one diagnosis initially.
    An overall advisory group would be formed.  Comparisons could be made
    between hospitals, between physicians and the seven-hospital average, as
    well as all the currently implemented comparisons.  If approved, he
    plans to start the program in a few months.  After a six-month pilot
    program, results will be checked and verified.  And this is only the
    beginning.

*****************************************************************************


    The CRTS program can provide comparative information for the following:

        Resource Indicators                     Quality Indicators
        -------------------                     ------------------
        total patient charge                    inpatient mortality
        length of stay                          unplanned readmissions
        ancillary charge                        hospital-acquired infections
        ancillary ratio                         surgical-wound infections
        ICU days                                neonatal mortality
        number of consultants                   perioperative mortality
        cardiopulmonary charges                 Cesaerean sections
        physical therapy charges                unplanned returns to special
        laboratory charges                              care unit
        radiology charges                       unplanned returns to O.R.
        catscan charges
        pharmacy charges
        IV solutions required

*****************************************************************************

    Note: The bulk of this material is expected to be printed in PCAI
    approximately November, 1990.