Evaluating Judges

Harris Hartz - Tenth Circuit Court of Appeals

Posted in , , , ,

Improving the quality of the judiciary is a noble cause. I welcome the participation of social scientists in the endeavor. But it is an open question whether social science can meaningfully contribute. Left to their own devices, social scientists are likely to produce work of dubious value. Perhaps judges can steer them to ask the right questions. Even then, however, the enterprise of improving judicial quality may not lend itself to the scientific method. This Essay addresses these issues in one area of research—the attempt to rate and rank judges.

Researchers trying to rate and rank judges have high aspirations. Some suggest that their measures of quality can be used to select judges for advancement, to determine what backgrounds produce the best judges, and to identify models to whom fellow judges can defer. This response to their efforts has three parts. Part I critiques some of the work to date. (The critique is brief because it was the principal subject of a prior Duke symposium.1) It argues that ranking—and even rating—judges is unlikely to produce the promised benefits; that the measures thus far used by social scientists miss the mark; and that those measures, to the extent that they influence judges, could encourage bad practices. Part II, the heart of this Essay, lists desirable qualities in appellate judges. Perhaps social scientists can find methods to measure these qualities objectively and accurately. Part III concludes, however, that even if social scientists cannot find such methods, efforts to identify the qualities of a good judge can be beneficial if they spur intelligent, respectful dialogue to encourage conscientious judges to improve their work.

Rating and Ranking Judges

Based on my observations and experience, some of the hopes expressed for rating and ranking judges are quixotic. For example, it would be remarkable if the electorate or an appointing authority made decisions any differently if candidates for judicial advancement were ranked by social scientists. Once a candidate passes the “qualified” hurdle, decisions are more likely to be made based on political and interpersonal skills. Likewise, studies of those judges who are ranked highly are unlikely to reveal what backgrounds produce the best judges; my personal experience is that excellent judges come from all sorts of backgrounds. And although law clerks may tend to refer their judges to opinions by highly ranked authors, the judges themselves are unlikely to pay obeisance. Most judges are independent sorts, with sufficient egos not to be dazzled by prestige. They do not adopt an idea just because it comes from a renowned person; they need to be persuaded on the merits.

In addition, I question whether the task of rating judges can be done well, and I worry that defective rating methods can have adverse consequences. Recent research has attempted to develop objective measures of judicial quality. As some participants in this Symposium have observed, however, the research appears to be driven by the availability of data. That is, social scientists have limited themselves to data that can be collected from available sources and then convinced themselves that certain subsets of that data reflect judicial quality, enabling them to rank judges accordingly. But although such measures can be precise, they may not measure what is useful and may encourage questionable conduct, as has happened in response to ratings of educational institutions. For example, I doubt that a judge’s independence can be captured by counting the number of dissents or disagreements with colleagues (either all colleagues or only those of the same political party). A judge who disagrees may simply be close-minded and unable to persuade colleagues. Worse, if a judge’s reputation turns on the frequency of disagreement, those seeking to improve their stature may try to find grounds on which to disagree, rather than making an effort to find common ground. Or if one measures a judge’s contribution to the law by how often the judge is cited, judges (who are well aware that the best way to get cited is to be the first to opine on a subject) may be tempted to produce more dicta and address issues not presented by the parties.

But replacing objective measures by subjective ones is not the answer. Reputation is a measure that feeds on itself, at times with very little of substance to support it. The compliment “highly underrated” can be as useful in describing judges as football players. Evelyn Waugh once observed that one can acquire a great reputation by being “dogmatic, plausible, and vain.”2 Judicial reputations are hardly immune from questionable influences.3

Judicial Quality

As the organizers of this Symposium have suggested, rather than starting with measures for which there are data and then deciding what those measures say about quality, perhaps one should reverse the process, deciding what constitutes judicial quality and then exploring how to measure those constituents. This Section responds to that suggestion with a list of what I believe to be desirable qualities in an appellate judge. Others may use rather different measures. Academics may be most concerned with whether the judge’s opinions are good teaching tools and raise interesting ideas. Attorneys may be most concerned with whether their clients win. Politicians may like judges whose opinions reach results that achieve wide popularity.

My perspective, of course, is that of an appellate judge. I will try to answer the question: “What do you look for in a fellow judge?” I am sure that to some extent my views are idiosyncratic. But I believe that each of my criteria would receive the support of a large fraction, perhaps a majority, of appellate judges.

Before I set forth my criteria, I would point out one possible criterion that is absent: to be a good appellate judge, a colleague does not have to agree with me. Any appellate judge with only a modest amount of experience would recognize that if “agrees with me” is a necessary criterion for a good judge, then only one judge would meet the standard. The simple truth is that no judge’s colleagues are as insightful and wise as the judge himself or herself. Criteria should be attainable in the real world.

I will group my criteria under four headings: Treatment of Colleagues, Treatment of Litigants, Treatment of the Law, and Treatment of the Institution. My order of discussion is not meant to suggest order of importance.

A.     Treatment of Colleagues

A good judge treats colleagues fairly. If it is true that ninety percent of life is just showing up, then treatment of colleagues belongs in the other ten percent. Just showing up won’t cut it. Colleagues should do their share of both the glamorous and the nitty-gritty work of the court. The part of a judge’s work to which the public pays most attention is the production of published opinions. But a judge who focuses only on producing published opinions is not a good colleague. To be sure, most judges would like to devote a greater percentage of their time to working on published opinions that raise challenging issues (although taking a break to resolve a routine case can often bring a welcome sense of accomplishing something). But writing a published opinion in a case that does not warrant it (because it says nothing new) wastes paper and the time of those who feel a need to read it. More importantly, if a judge shirks all duties except the preparation of published opinions, the other, less-interesting duties are shared disproportionately by the judge’s colleagues. These duties include participating in calendars of cases that are likely to result in unpublished opinions, disposing of motions, and serving on one of the many committees that handle court rules and administrative matters. Judges have different interests and may prefer some of these tasks to others. Which they select is immaterial; the important thing is to do one’s share.

B.     Treatment of Litigants

The first duty of appellate judges is to decide the cases before them. The process of decisionmaking must be fair, and appear to be fair, to the litigants. The fairness of treatment on appeal, unlike the fairness of treatment at trial, cannot be measured on the basis of face-to-face encounters. Appellate judges rarely are seen by the parties to a dispute; and most of the time they are not seen even by the parties’ attorneys (and then only for a few minutes, in the highly artificial setting of oral argument). Thus, fairness must be assessed by reading opinions. An opinion reflects fairness to the litigants when it has the following features:

1.   Takes Care with the Facts

The factual context of a case should be stated fairly (including “unfavorable” facts) and accurately. If a fact is worth incorporating in an opinion, it is worth setting forth correctly. Not all facts in an opinion are critical to the resolution of the dispute; but it is important to check the accuracy of even background facts, because an opinion that treats facts cavalierly will suggest that the court has been inattentive to the case. Perhaps the easiest way for a losing attorney to convince a client that the court did not give fair consideration to the client’s contentions is to point out that the court omitted important facts or did not get the facts straight.

2.   Sticks to the Record

An appellate court should decide the case on the record produced in the lower court. When the court goes outside the record, it relies on matters that the parties had no opportunity to contest and that the lower court had no opportunity to take into account. To go outside the record to resolve a dispute will thus offend the sense of justice of the party injured by the practice, and will also likely frustrate the trial court that is being reversed on grounds not presented to it.

3.   Addresses the Parties’ Contentions

A party will not feel that the court has acted fairly if the court does not address the party’s contentions on appeal. This is not to say that every contention must be resolved on the merits. Often a contention is mooted by the resolution of another issue. The court also may refuse to consider a contention because it was not preserved below or not properly presented on appeal. In addition, unfortunately, some briefs are so poorly prepared that the court must characterize the party’s contentions differently than the party did, and thus the issues addressed in the opinion may not match the party’s listed contentions. A good judge, however, will take care that the opinion recognizes all properly presented arguments. I can understand why a court may occasionally write something like, “We have reviewed appellant’s other arguments and none has merit”; but I am not fond of the practice. At the least, I would want to state that the appellate court substantially agrees with the lower court’s analysis of the issue.

4.   Is Evenhanded

A judge should be evenhanded, applying the same rule of law in a similar fashion to all parties. A judge may be either strict or lenient about what it takes to preserve an issue; but the judge should not be lenient when considering preservation by a personal-injury plaintiff and strict when considering preservation by a personal-injury defendant. Similarly, a judge may have lenient or restrictive views on when parties are entitled to standing; but the judge should not be lenient for those exploiting mineral resources and restrictive for those opposed to such exploitation. Of course, there are exceptions to the general rule. Pro se litigants should be granted some leeway, so long as the court does not become their counsel. And the government can be expected to toe the line more closely than others. But lack of evenhandedness can be the most telling indication of appellate bias against an individual litigant or category of litigants.

C.     Treatment of Law

Although an appellate court’s first duty is to decide the case before it, the legal doctrine set forth in the court’s opinion is almost always the most important consequence of the court’s decision to society as a whole. It matters little to the general reader of the opinion whether the court erred in its decision because it got the facts wrong, went outside the record, or failed to consider an argument by the losing party. A distinct set of considerations governs whether a judge does a good job in setting forth legal doctrine. A judge’s opinions should satisfy the following criteria:

1.   Describe the Case Law Honestly

Even when precedent does not determine the result in the case, prior case law is likely to impose significant constraints or be persuasive. The reader of an opinion should be able to assume that the author accurately describes the facts and holdings in cases cited in the opinion. Of course, progress (or at least development) in the law often occurs when precedents are re-examined and recharacterized, and a “better” explanation is then provided for the results in those cases.4 But there is no reason for a court today to be deceptive about what it is doing in that regard.

2.   Help Develop the Law

A good judge contributes to the development of the law. I do not mean that the good judge always “gets it right.” Early in my career I agonized about whether I was stating the law absolutely correctly. Then I realized that I simply do not have the experience and intelligence to come up with all the relevant considerations, much less wisely evaluate them, in deciding how to frame a rule of law. All I could expect from myself was to work as diligently and intelligently as I could on the matter. I can hardly expect more from others. But even if the judge does not get it right, the judge can make a contribution by sharing the product of his or her diligence and intelligence. Perhaps the most important component of this contribution is clarity. Rather than just pronouncing the result, the opinion should clearly explain how the court arrived at its conclusion. If the judge believes that there are three important considerations supporting the conclusion, the opinion should recite those considerations and explain how they interact to compel the result. Others may later show that there are really four important considerations, but the judge who came up with the first three has made the analysis easier for those who follow. I value judges who advance the law by sharing the product of their diligent, intelligent efforts, even when I disagree with their conclusions. One of the joys of appellate judging is the interchange of ideas that leads to a better opinion than any single judge on the panel could produce; this interchange can be among judges who agree on a result, but the best work often comes when there is a dissenting voice.

3.   Be Consistent

A judge should be consistent in the process of arriving at doctrine. For example, does the judge have a consistent practice in deciding whether to overturn a precedent or to apply stare decisis? A judge could consistently decide to overturn precedent whenever the judge believes the precedent to have been poorly reasoned; but then the judge is not entitled to rely on stare decisis doctrine to criticize fellow judges for overturning a precedent that they believe to have been poorly reasoned. If a judge bases a constitutional doctrine not on the specific language of the Constitution but on the structure created by the document, the judge cannot criticize others for using the same methodology to reach a result opposed by the judge. Likewise, judges should be consistent in applying canons of statutory construction and in using the results of research in the hard and soft sciences.

D.     Treatment of the Institution

The authority of courts in this country is founded on the reputation of the judiciary. An appellate judge has a duty to contribute positively to that reputation. The judge’s work may well advance the reputation of the judge, but it should not do so at the expense of the courts themselves. A few thoughts on what makes judges good in this respect:

1.   Persuasiveness

Courts may have the power to rule however they wish. But their legitimacy is based on the persuasiveness of their opinions. Accordingly, I value colleagues who write coherently, logically, and convincingly.

2.   Respect for the Courts and the Reader

Persuasiveness of opinions may be the most important factor in establishing the legitimacy of judicial decisions, but it is not the only one. The style and tone of an opinion can also advance or detract from the prestige of the deciding court and of the judicial system as a whole. Opinions that carp at or demean other judges or their work can only cause the public to adopt a similar attitude. And attempts at eloquence that degenerate into bloated prose can provide ammunition for the view that the courts have lost touch with the community.

3.   Modesty

The purpose of writing opinions is not to create a reputation for the author. Grand pronouncements, declarations of “new” legal principles, and treatise-like discussions that go beyond the needs of the case may well establish the author’s brilliance. But they are at least as likely to be examples of “writing more than one knows”5 and thus provide numerous opportunities for later judges to correct errors and cabin dicta. Such writing can also create an aura of judicial willfulness. Judicial craftsmanship often consists in explaining how the law expressed in an opinion follows naturally from prior case law. The more an author signals that the law expressed in the opinion is essentially the creation of the author’s brilliance, the less the reader will be convinced that the court’s work represents the rule of law rather than the rule of persons who happen to be judges.

Measuring Quality

Now that I have listed what I believe to be attributes of a good appellate judge, one might ask what good the list does. Perhaps some of the listed attributes can be measured objectively. After all, experts have developed ways of determining how much time it takes trial judges to perform various tasks; and the time may come when someone believes it possible to measure whether an appellate judge is doing a fair share of the court’s work. Also, measures that are not totally objective may be developed to evaluate whether a judge treats litigants evenhandedly or develops legal doctrine in a consistent manner. I have my doubts. Yet even without such measures, a list of desirable qualities can serve a useful purpose. The most productive engine for improving judicial performance is the conscientiousness of those who wear robes. If judges and the consumers of their work can exchange views about desirable qualities for appellate judges, then judges who care about their craft (who comprise the great majority of the profession) can consider those views and conduct periodic self-evaluations. I frequently see my colleagues borrow good practices from one another, often without any discussion between them. I am confident that as judges engage in conversation and introspection regarding quality, we will continue to improve the way we do our jobs.


Copyright © 2010 Duke University Law Journal.

Harris Hartz is a Judge on the U.S. Court of Appeals for the Tenth Circuit.

This material is based upon work supported by the National Science Foundation under Grant No. SES-0946437.

  1. Symposium, Measuring Judges and Justice, 58 DUKE L.J. 1173 (2009).
  3. A study of cardiac surgeons is quite revealing. The study examined the success rate of the surgeons, correcting for the patient’s risk. It compared the success rates to potential predictors of success, such as appearance in a Best Doctor’s list, prestige of medical school and residency program, age, years of experience, and number of times a surgeon had performed the surgery during the prior three years. The only factor that had any value as a predictor (it was a good predictor) was the number of times that the doctor had performed the surgery during the prior three years. Prestige of residency programs correlated well with whether one was listed as a Best Doctor, but did not correlate with surgery success. See Arthur J. Hartz, Jose S. Pulido & Evelyn M. Kuhn, Are the Best Coronary Artery Bypass Surgeons Identified by Physician Surveys?, 87 AM. J. PUB. HEALTH 1645, 1645–48 (1997).
  4. See, e.g., MacPherson v. Buick Motor Co., 111 N.E.1050, 1053 (N.Y. 1916) (Cardozo, J.) (abandoning the privity doctrine for product defect claims).
  5. The quoted phrase is stolen from the late, beloved Dean Lee Teitelbaum. When asked why he appeared unhappy despite a period of highly productive scholarship, he responded, “I’m afraid I’ve written more than I know.”

Post a Comment (all fields are required)

You must be logged in to post a comment.