This document requires Netscape 3.x or compatible Web Browser.

---

Biostatistics for the Clinician

Biostatistics for the Clinician

University of Texas-Houston
Health Science Center

Lesson 3.2

Decision Analysis

Lesson 3: Clinical Decision Making in a Multivariable Environment 3.2 - 1

---

Biostatistics for the Clinician

3.2 Decision Analysis

3.2.1 Decision Trees

Now for a brief look at decision analysis, an increasingly important part of medicine. In fact, a few years ago it spawned an entirely new journal that's good to be aware of called Medical Decision Making. You might wonder what kinds of articles are in this sort of journal. How about, "The Roles of Experience and Domain of Expertise in Using Numerical and Verbal Probability Terms in Medical Decisions". The journal features scientists that are looking at using artificial intelligence and other kinds of input about patient's decision making, patient's deciding upon what is a good outcome for them, what is a bad outcome, and how physicians make choices about whether or not to use a particular treatment. It has scientific analyses of decision making applied in the medical area. Decision analysis is a very important area, and is becoming increasingly important with the advent of managed care.

In developing a decision analysis you break down the medical analysis into a series of events (see the Decision Analysis figure) below. Some of those events are chance events. That is, you perform a treatment and it may or may not work. Or, it works with some probability. That's a chance node. There are certain nodes for events where you make a decision. Those are choice or decision nodes and you still have the chance nodes. Particularly in areas of cancer treatment, where you might have a very complicated protocol where these decision trees can be very complex, there are many chance nodes, and there are many decision nodes. Decision nodes might involve questions like, "Shall I use this particular combination?", "Shall I use it 3 days?" "Shall I use it 5 days?" and so on?

Decision Analysis

The decision trees can be very complex. But, they illuminate and clarify the decision process that you as physicians and your colleagues go through. In other words, decision trees make very clear, the series of processes that you're going to need to go through to move a patient from diagnosis to cure.

Look at the Decision Analysis figure again now. You can see that decision trees have nodes and branches. First, on the left there's a choice node. So you choose to go with Strategy 1 or Strategy 2. If you choose Strategy 1 there are a couple of events (Event 1 and Event 2) that can happen with certain probabilities. These two different possible events define a probabilistic or "Chance Node". So Event 1 determines Outcome 1 with a certain probability. Likewise, Event 2 determines Outcome 2 with a certain probability. If you choose Stategy 1 you have a certain chance of Event 1 or Event 2 happening and each leads to a different outcome. So what you do, essentially, is to diagram the decisions that you're going to make, the Choice Nodes, the Chance Nodes, the events and their outcomes, as you see in the figure.

Decision Analysis
Practice Exercise 1:	Decision analysis makes use of: No Response Decision nodes Chance nodes Decision trees All except "No Response" above

Lesson 3: Clinical Decision Making in a Multivariable Environment 3.2 - 2

---

Biostatistics for the Clinician

3.2.5 Strategies for attaching values to branches

The next thing that you do is, you figure out a strategy for valuing these outcomes and that's usually called a "utility". It could be a cost. It could be the days of quality life left for a patient. For example this patient might be a bed ridden patient. So you have to figure out what the quality of life over the lifetime of that patient is in order to figure out the value for that. So these outcomes are kind of specific outcomes and you have to attach values to them or utilities. Much of what this medical decision making is about is "How do you attach values?" Who decides the values? Does the patient decide them? Does some specialist decide them? Is there a cluster of decision makers with general physicians, some specialists and patients that make these decisions and so on?

Once you outline the process like this, you go through a mathematical process called folding back. You multiply these things out and it tells you the best strategies. So it's a way, if you can describe the process, and you can find in the literature what the probabilities of the outcomes are, and you can figure out a way to quantify the quality or utility of these outcomes, to determine which strategy to use.

Human beings are very poor at making decisions like this that involve a lot of decisions down the line. We're wonderful at gathering information in parallel. For example, recognizing a person's face is a tremendously difficult problem of parallel information processing. We do it instantly. But you try and tell me which is the best strategy given a tree which has 10 nodes down here -- we're terrible at it. So decision analysis helps us in our decision processes where they are the weakest. It is suggested that you look at Mike Hagen's article. He goes through this process to the point of describing whether a patient should have cabbage or should have medical treatment for stenosis.

Decision Analysis
Practice Exercise 2:	Decision analysis helps physicians clarify which is the best: No Response Decision node Chance node Decision tree Decision strategy

Lesson 3: Clinical Decision Making in a Multivariable Environment 3.2 - 3

---

Biostatistics for the Clinician

Now the other main point that you should know about in medical decision making is the following. Once you've done the decision analysis, as previously discussed, you can go back and do a sensitivity analysis. Sensitivity analysis tells you how dependent your strategy selection is upon the probabilistic outcomes. Say the literature does not make clear whether the probability of death in this procedure is .02 or .1. Say the literature is vague about that. The appropriate numbers can be put into a tree like that in the figure below to help determine the strategies.

Decision Tree

What you want to be able to do is to come back in the decision tree and say, "What if it is .1?", or, "What if it is .2?", how will that change the strategy? More specifically, you can determine how sensitive the strategy is to these numbers from the literature. With that information you have a better way of evaluating the quality of a decision tree. Sensitivity analysis then shows how sensitive the strategy is to the probabilities you plug in (see the figure below). That's a critical element in medical decision making, because you cannot always find the best numbers.

Two-Way Sensitivity Analysis Surgical Mortality Versus Medical Mortality
Med. Mort. Prob.
	Surgical Mortality Probability

Decision Analysis
Practice Exercise 3:	Sensitivity analysis helps physicians determine how much the medical outcomes in the decision tree depend on the specific: No Response Probabilities Decisions Strategies Chance nodes

Lesson 3: Clinical Decision Making in a Multivariable Environment 3.2 - 4

---

Biostatistics for the Clinician

Spend some time and look at this little guy below here. Make sure you understand your errors, alpha error and beta error. Remember alpha, Type I error, is the risk of a false positive. Beta, or Type II error is the risk of a false negative. Power is 1-beta, the probability that the experiment will be able to detect an effect that is really there.

Types of Errors
	Reality
Study's Conclusion	True Effect	No Effect
True Effect (reject null hyp.)	Truth	Type I or Alpha Error
No Effect (don't reject null hyp.)	Type II or Beta Error	Truth

Play with some p values for beta in your head, for betas and alphas for, say, a treatment that doesn't hurt a person a lot, for a treatment that's very dangerous, for a disease that's devastating, for a disease that doesn't cause much harm. Then decide what kinds of experiments you would design and what kinds of errors you would have with regard to those.

3.2.2 C.R.A.P. Detectors Review

Now, lets review a few key pitfalls that you want to make sure you avoid as potentially both a consumer and a producer of medical research. First, beware of the large sample, hundreds of thousands of people, because the effects will always be statistically significant but they well may not be clinically important to you.

C.R.A.P. Detectors Review
C.R.A.P. Detector #3.1	Beware the large sample. Effects can be statistically significant and clinically inconsequential.

Beware the sweeping generalization. Results of any study only apply to populations studied in the study's samples. In fact, you can toss out a lot of the literature, because it doesn't apply to your patient population for one reason or another. Ignore reading it because it doesn't apply to where you're practicing medicine with your patient clientel.

C.R.A.P. Detectors Review
C.R.A.P. Detector #3.2	Beware the sweeping generalization. Results of any study apply only to populations similar to the study sample.

Beware the small sample. It's hard to find small differences and no differences mean nothing. It may be that you're passing up a great treatment and the only reason is that your samples are too small to find that difference.

C.R.A.P. Detectors Review
C.R.A.P. Detector #3.3	Beware the small sample. It's hard to find significant differences, and no difference means nothing.

So if you find non-significant results, be very careful to examine the power of that experiment to detect the effect size you're looking for there.

Now what about multiple comparisons? They are a statistical no-no. Multiple comparisons occur any time you conduct multiple statistical tests. Each additional test inflates the error rate for the whole experiment. As an obvious example, let's say you conducted 20 tests at the .05 alpha level. It should be apparent that because you have 20 tests, each has 1 chance out of 20 of being significant simply on a random basis. So, it is to be expected that at least one of the 20 tests will be significant as a Type I error. Of course there's no way to know which significicant tests are Type I errors and this calls into question the entire study. Consequently, there are special ways to handle these kinds of data.

The main point is, you want to keep your experiment-wise error rate to your preset alpha level, typically .05 or less. Say you do a Chem 7 or an SMA 20 or something like that. You're likely to get a number of values that are Type I errors anytime you perform a whole bunch of tests on a statistical sample. This practice violates one of the fundamental rules of statistics. Anytime you do a lot of statistical tests you're likely to have some or many those values tests result in Type I errors, in other words there are those flukes you get from time to time where when you sample from a distribution you get a group whose mean is way out in the tails, purely by chance. This problem is called alpha level inflation and it is to be avoided. You can avoid it with special techniques that statisticians know about, that are designed to deal with these situations, manage the error rates, and hold the alpha level to prespecified values. You're dealing with these issues in the clinic all the time. Test after test is done. You expect to have some Type I errors. So, you don't say this patient needs to go to the hospital because they've got a calcium that's out of range. You check it a second time. Check it again to be sure there's no error.

C.R.A.P. Detectors Review
C.R.A.P. Detector #3.4	Multiple comparisons are a statistical no-no! There are special ways to handle these kinds of data.

You now have a better idea of some of the main concepts in statistics that are useful for physicians.

Correlation and Regression Analysis
Practice Exercise 4:	Which of the following concerns is important when conducting or critically evaluating medical research studies? No Response Large samples Small samples Sweeping generalizations Multiple comparisons All except "No Response"

---

Final Instructions

Press Button below for your score. After completing Lesson 3.2, including all practice exercises, press the "Submit... " button below for Lesson 3.2 research participation credit. After you press "Submit..." it is possible Netscape may tell you it is unable to connect because of unusually high system demands. If you receive no error message upon submission you're OK. But, if Netscape gives you an error message after you press the "Submit..." button, wait a moment and resubmit or consult the attendant. Finally, press the "End of Instruction..." button below to correctly end Lesson 3.2. You have now finished the Biostatistics for the Clinician instruction. Biostatistics for the Clinician Be sure to take the posttest during April 28 - May 4.

Lesson 1: Summary Measures of Data 3.2 - 5

---