Introduction

The Symposium organizers urged me to present this Article from two perspectives: one as the former Chair of the Committee on Legal Education and Admission to the Bar of the Association of the Bar of the City of New York (“ABCNY”) and one as a clinical law professor concerned with improving the methods by which we teach law students and assess their level of competence. While there clearly has been considerable overlap in my meeting these two sets of responsibilities, my work on each has proceeded independently for the most part, until now.

In my capacity as Chair, I recently oversaw our committee’s publication of the Joint Committee Report: Public Service Alternative Bar Examination (“Joint Proposal”).[1] In lieu of the existing bar exam,[2] this controversial Joint Proposal[3] sets forth multiple methods of assessing applicants for admission to the bar.[4] The core of the Joint Proposal involves the applicants’ provision of public services in the New York State court system. Both on-site supervisors and outside academic-type graders would evaluate the services for bar admission purposes. The Joint Proposal also includes additional methods of assessment such as the use of simulated lawyering encounters (for example, mock interviews of clients or witnesses or counseling sessions with clients). Outside clinical academics would evaluate these simulations.

In my law teacher role, I have been developing the concept of standardized clients as a more elaborate and more specific technique for assessing law student performance through simulation.[5] I based this method of evaluating lawyering performance on a medical education model—standardized patients—that uses lay persons to role-play patients and then to provide written evaluations of the students’ interaction with the patient. The written feedback is on structured forms drafted by medical professors. Adapted for law schools, the actors portray clients and witnesses, and the law professors prepare the evaluation checklists.[6] My primary goal for developing this model for law student use has been curricular improvement and not licensure revisions. Nevertheless, there is a logical and practical nexus between education and licensing. This was dramatically demonstrated when the medical profession recently incorporated standardized patient evaluations into the medical licensing exam.[7] The use of this simulation technique both in the medical school curriculum and now in the medical licensing exam has encouraged me to connect my work on these legal profession projects for this Symposium.[8]

My bar association committee work began as a follow-up to a report that exhaustively and critically looked at the bar admission process in New York.[9] The goal of the current committee’s work was to move beyond a critique of what exists and even beyond generalized recommendations for improvement. The result is a proposed alternative screening process that would be implemented in a small pilot project. It would establish a fair and valid licensing procedure to assess a bar applicant’s qualifications and better reflect the array of skills and knowledge needed to be a competent lawyer. Whether it ultimately would supplement or replace the existing bar exam was among the many questions purposefully left unanswered in the Joint Proposal for a Public Service Alternative Bar Examination (“PSABE”). The Joint Proposal calls for very careful scrutinizing of the results of the pilot. That analysis would serve to guide decisions from that point.

While the comparison with the medical profession may be logical, there is no direct empirical evidence to support the law profession side of the equation. As this Article describes below, medical educators have been working with standardized patients for more than three decades and have accumulated an extraordinary amount of data that validates their use of this method.[10] Much work remains before legal educators can realistically talk about using a similar technique as a component of law school grades, let alone as part of a high stakes test such as the bar exam. Thus, while not an immediate option, it is possible that at some future date the use of standardized clients might enhance the efficacy of a PSABE. None of this appropriate skepticism, however, should preclude serious consideration of the Joint Proposal for a PSABE, irrespective of the possible future use of standardized clients. Nor should this caution deter legal educators from considering the standardized client as a future component of the public service bar exam or of any other innovative bar admission proposal.[11] Some people may feel that it is too much of a stretch to initiate discussion about changing bar exams simply on the basis of the experience of medical educators. I remain encouraged, however, by John Sexton’s call for us to “[think] outside the box” regarding the training of lawyers.[12]

Part I describes the Joint Proposal for a PSABE, its genesis, its political development, and key unanswered questions. Part II discusses in some detail my recent work in further developing the standardized client in a law school curricular context. Part III discusses the similarities between some of the tasks of doctors and of lawyers. Then, building on the experience of medical educators, this Article sets forth proposed empirical analyses for the use of standardized clients. The legal profession might do well, once again, to follow the lead of the medical profession in better serving the public’s continuing need for competent and caring professionals. Stating it differently: Is it reasonable to think ahead to a time when the standardized client will be an integral part of the bar exam?

I.   The Public Service Alternative Bar Exam

A.   The Genesis of the PSABE Proposal

The Joint Proposal[13] is the product of an intensive joint effort of two independent bar association committees from two similarly independent and highly respected bar associations. From the start, these collaborative energies were quite purposeful. The Chair of the New York State Bar Association (“NYSBA”) Committee, Anthony Davis, and I had worked together on a comprehensive critique of the New York bar admission process a decade earlier. A third central figure was Dean Kristin Booth Glen, whose views on the deleterious effects of the bar exam and the need for drastic changes were reasonably well known in New York.[14] Recognizing that a radical departure regarding bar admission was appropriate and necessary, we concluded that a potential combined effort of these two committees (and, indirectly, the two bar associations) would be the most effective way to engender serious consideration by those in a position to actually make changes. While there has been very little official response to the Joint Proposal,[15] there has been a considerable amount of public reaction and continued interest in the ideas reflected in the PSABE Joint Proposal.[16] Indeed, this Symposium is just such an expression of that interest.

However laudatory or appropriate that collaborative political goal, it was difficult to implement. The two relatively large committees consisted of strong, experienced, and independent‑minded members representing a diverse group of practitioners, judges, and academics, as well as several very recent law graduates.[17] While there was an early consensus in favor of trying to formulate a new and distinct approach to bar admission requirements, there was clearly no consensus as to what that approach might be. The committees recognized that the profession and the public wanted (or needed) some kind of screening or credentialing process.[18] The committees also widely supported the notion that anything that it might propose would have to be attainable; it would have to pass the “laugh test” for officials to take it seriously. This latter point, for example, was the basis of making the Joint Proposal a very small pilot program (so that the bar could effectuate the program without any immediate revolutionary change in the legal profession).[19] We circulated an earlier draft of Dean Glen’s public service proposal,[20] and it became the vehicle for much of our discussion. Ultimately, it was the basis upon which we shaped the numerous compromises that the final Joint Proposal reflects.

B.   The Key Provisions of the PSABE

The underlying premise of the PSABE is that the current bar admission requirements do not adequately address the full array of skills necessary for competent lawyering[21] and that the almost exclusive reliance on the current bar exam is an expensive and wasteful rite of passage, which also presents an insuperable barrier to admission for a significant number of candidates. The bar also fails to acknowledge the different categories of intelligence required to be a competent lawyer that a written exam cannot test.[22] At the same time, the committees accepted the reality that neither a more appropriate bar exam nor an increase in clinical education would fully prepare a successful applicant to practice law.[23] The goal was to reform the bar exam to reflect lawyering competencies better, which might, in turn, have a positive influence on shaping law school curriculums. The more modest aim of the PSABE was to reduce the gap between what the bar tests and what it requires for minimally competent law practice.[24]

The first pilot group would consist of 50 randomly selected applicants (chosen by application and lottery) for admission to the bar.[25] The pilot program would require applicants to provide three months of public service in the New York state court system.[26] That service could include drafting opinions for judges, conferencing cases (to set discovery schedules or to conduct settlement discussions), assisting unrepresented litigants, or mediating cases. On-site supervisors and outside evaluators would evaluate the work of applicants.[27] The pilot would also require applicants to complete satisfactorily a performance-type written exam,[28] the Multistate Professional Responsibility Exam (“MPRE”), and one or more videotaped simulation exercises. To be eligible to participate in the PSABE, an applicant would have to satisfy a relatively easy law school course requirement—a practical skills course and a New York State Civil Procedure course. After a short orientation period, the participating applicants would begin their placement in the courts. Toward the end of the three‑month placement period, the applicants would complete the written performance test and the videotaped simulation exercises. The applicants also would have to perform, pro bono, 150 hours of similar services in the New York courts during the first 18 months after admission.

Aside from the costs and administrative aspects of this alternative bar exam,[29] ensuring fairness concerned the two committees (like all boards of law examiners). Any bar admission screening process must be a valid and reliable method of testing applicants for that knowledge and those skills that are relevant to being a minimally competent lawyer. Validity requires that the method of testing measure “what it purports to measure.”[30] Direct observation, by qualified evaluators, of the performance of the task being assessed easily satisfies that requirement. The evaluation also needs to be reliable; it must be “[consistent] from one . . . test‑taker to another.”[31] In light of the multiple reviewers that the PSABE anticipates using, satisfying reliability standards will be a challenge.[32]

C.   The Key Unanswered Questions

    There are really two key unanswered questions. One is who will finance the pilot (let alone a larger application of the PSABE). For now, that remains unanswered.[33] The second is the fairness question. How can the bar administer the PSABE in a fair and valid manner? For example, one of the methods of assessment in the PSABE calls on outside evaluators to grade an applicant’s performance either by participating in a simulated interviewing or counseling exercise or by observing an actual interview by an applicant (for example, interviewing a pro se litigant). Putting aside the expense issue, the expectation is that clinical law professors would perform these evaluations. Using numerous professors might lead to variations in grading criteria.[34] Evaluating actual interviews of real litigants would likely result in substantial variation in the circumstances encountered by the applicant. It could be unfair to evaluate one applicant interviewing a mentally-disturbed litigant while a second clinical professor evaluates another applicant who is assisting an intelligent, calm litigant.[35] This parallels earlier versions of the medical licensing exam. Doctors would evaluate interns seeing real patients regardless of the nature of the patients’ illnesses. The variation, inconsistencies, and lack of uniformity led to the conclusion that it was not a fair method. That conclusion, in turn, has led, in part, to the use of standardized patients, which is a much more reliable, consistent, and fair method. In law, these aspects of potential unfairness, as noted above, lead back to the standardized client as a possible solution. If further research establishes that the standardized client method is valid (like the standardized patient in medicine), this method of evaluation could replace the law professor assessments. It would be much less expensive and probably more consistent, making it more reliable and fair. Indeed, that proposition is the fundamental thesis of this Article.

Another means of ensuring fairness that the committees addressed in the Joint Proposal is the use of multiple methods of evaluation. In addition to an assessment of a simulated lawyering activity, the Joint Proposal calls for the successful completion of an extensive written performance test and the MPRE.[36] It also requires a passing score from an evaluator who observes actual lawyering work (for example, conferencing or mediating cases). By using more than one or two methods, the proposal limits the impact of any single method. Each different score can act as a check on over-reliance on any single indicia of competence.

The PSABE Joint Proposal is a work in progress. The goal was to build on the extensive progress over the past 30 years in developing tools to assess lawyering performance in clinical education. Working with small numbers in a pilot, the objective is to closely examine how well this alternative works. The possible inclusion of standardized clients in a pilot simply adds to the potential value of careful empirical analysis of the results of a pilot. Simultaneous empirical studies of the use of standardized clients, as described below, could be reinforcing. The additional public service benefit is simply another potential plus from using the PSABE. Both the PSABE and the use of standardized clients reflect efforts to consider objectively new ways to make the bar admission screening process more reflective of what practicing lawyers actually do.

II.  The Continued Development of the Standardized Client at New York Law School

A.  Context for Refining the Standardized Client

    From the first day of law school, legal educators communicate to students in many different ways about what is important and what is not so important in the legal education that is to follow, and in turn, what is important to being a good lawyer. In almost every law school in the country, the first-year curriculum is essentially the same. It typically consists of eight to ten required courses, including a writing and research course and perhaps a course like the Lawyering course discussed later in this Article. The other courses are typically Contracts, Torts, Civil Procedure, Property, Criminal Law, and perhaps Ethics or Constitutional Law. The vast majority of teachers of those courses use standard casebooks and some variation of the Socratic method. All teach doctrinal reasoning, legal reasoning, and legal analysis. While some first-year teachers use the problem method and a few even use simulation in one way or another, both are exceptions to the norm. None of this is news to anyone who has been in legal academia for any period of time.

Given these norms, students understandably view any efforts to teach basic interpersonal lawyering skills as diversions from their main task of learning doctrine and legal reasoning and analysis.[37] That is what is important to success in the first year and to being a good lawyer; this is the message that we communicate to them. This is a distorted view of what competent lawyering entails. Nevertheless, for many decades, that has been and still is the reality of the first year of law school.[38] The standardized client method presents an opportunity to make a substantial dent in the armor surrounding this long tradition of what happens in the first year of law school.[39] This Article does not mean to overstate the potential significance of this technique, but it urges a closer look at the possibilities.

New York Law School has been working with the standardized client concept for more than five years. As already noted, it builds on medical education’s standardized patient that uses trained lay persons: first to portray patients who are then interviewed and examined by medical students and second to provide a written evaluation of the medical student’s performance. Our adaptation of this teaching method in law school is the standardized client, whose function is to portray a client or a witness and to provide each law student with a written critique on an evaluation form that the law professors’ school drafts.[40]

Legal education has used simulation for some time in many different contexts.[41] It has not used a structured, written, lay-person[42] evaluation of a law student’s performance.[43] Only the law professors grade the students. Unlike all other forms of graduate education, legal educators have generally not employed readers (either lawyers or law students) to grade exams or even quizzes.[44] Thus, the notion of a non-lawyer participating in the process that determines the successful completion of law school would be an unusual, if not revolutionary, change in the American legal education. The typical person who has portrayed the standardized client and provided the written evaluation at New York Law School has not been a lawyer.

Medical educators addressed this issue by conducting repeated comprehensive empirical analyses of the use of the standardized patient over many years to validate this method of evaluation.[45] As a result of these studies, medical educators have been using graded standardized patient exercises as part of the required medical school curriculum: first on a modest low-stakes level and, more recently, as a requirement for medical school graduation.[46] As noted above, the medical profession has recently legitimized this method of gauging the capacity and competence of future physicians on a high-stakes level. Beginning in 2004, the standardized patient will be an integral part of the medical licensing exam.[47]

Legal educators, on the other hand, have yet to do these kind of empirical evaluations of the use of the standardized client.[48] Indeed, legal educators have not done any studies. The completion of these types of studies, therefore, is a major goal of our continuing work in this area. It is by no means a foregone conclusion that the legal profession will reach the same results as the medical profession regarding the validity of this method. There are numerous questions regarding the efficacy of this means of evaluating law students that remain to be answered. One major issue, for example, is how much training the standardized clients require and, in turn, how to monitor their performance.

B.  The Lawyering Course[49]

The central vehicle for New York Law School’s current use of the standardized client is the first-year required course—Lawyering. The goal of this two-credit course is to introduce students to fact analysis, interviewing, and counseling (basic clinical practice skills), while reinforcing basic legal reasoning and analysis skills. We also seek to use this contextualized approach to legal and factual analysis to teach humane and ethical lawyering. Thus, in addition to the use of the standardized client as a new and unusual method of providing individualized feedback to students, the Lawyering course represents an alteration of the nature of the traditional first-year law school curriculum, a daunting challenge in itself. We have sought to integrate the innovative methods of the Lawyering course with the traditional goal of teaching students how to analyze, to reason, and to articulate a synthesis of the law and facts. Putting it slightly differently, we have concluded that we can accommodate these changes more easily by not disturbing those traditional first-year objectives in any significant way. If the program can enhance those objectives, it would be a welcomed side effect. Indeed, we have sought to communicate that legal reasoning and analysis is a critical and necessary component of competent lawyering but that it is not sufficient to achieve competency.

Our development of the Lawyering course began about ten years ago and has proceeded with those guidelines in mind. This section briefly describes the genesis of the Lawyering course as well as its current expanded version, and then it goes on to set forth much more ambitious, but realistic, goals: namely, to incorporate the use of standardized clients in all of the first‑year courses.

In very loose terms, Lawyering represented an effort to build on Professor Anthony Amsterdam’s groundbreaking first-year Lawyering course at New York University School of Law (“NYU”).[50] We also sought to demonstrate the integral connection between (1) fundamental legal reasoning and analysis skills and (2) applied lawyering skills. A critical difference was that we would teach our course only in large class sessions (100 to 120 students), whereas NYU teaches its course in sections of 15 to 20. At the outset, our course had no individual feedback and very little opportunity for students to apply the law in a simulated context. The NYU course had significant individualized instruction from full-time Lawyering faculty and many opportunities for students to conduct simulated lawyering exercises. Both the NYU course[51] and our course have undergone numerous changes in the last decade. While the differences in class size remain, we now offer substantial individualized feedback through the use of standardized clients.

We first adapted a breach of contract case file used at NYU to fit the needs of our large lecture course. Students worked on the case from the initial client interview through the filing of a summary judgment motion. When they finished, they had worked through the applicable substantive law, the pleadings, and the transcripts of depositions and interviews. We supplemented their written immersion into the case with videotaped depictions of various interviews and depositions in the case. At least once, we had the class “interview” someone role-playing a client or witness in front of the class.[52] Each student individually conducted a witness interview in which another student role-played the witness for the exercise. The videotapes, the simulated witness interview, and the assigned readings on lawyering skills provided a base for large class discussion of the challenges confronted by a lawyer developing a case.

Our first significant innovation in the Lawyering course was to have each student interview a standardized “client” (actually a witness) who would provide written feedback to the student interviewer, using a checklist form that the professors had drafted.[53] The final exam in the course required the student to synthesize new law and new facts in the same case file, to view a videotaped depiction of a lawyer performing one or more of the skills studied in the course, and to prepare a written critique of that performance.[54]

The current version of the course now includes three standardized client exercises arising out of case files in three different areas of law: torts, contracts, and real property. Whereas the earlier course was in the first semester of the first year, this revised course takes place in the second semester.[55] In the first case, the students conduct an initial client interview of a standardized client with a tort claim. The second case calls on the students to interview a standardized client who is a witness in a contract case.[56] Finally, in the third case, the students must counsel a client in an adverse possession case. As we had done previously, we show several videotaped demonstrations of the same kinds of skills the students will perform.[57] For example, in the first unit, which involves a claim based on the negligent infliction of emotional distress, the students view a demonstration initial interview of a client with a similar tort claim before they interview the client in their case. Likewise, in the second case, which involves a claim of undue influence, the students first observe an interview of another witness in the case before they interview their witness. Before they counsel their adverse possession client, students observe a tape of a counseling session with a client in a comparable situation. After the interview, the standardized client completes a detailed evaluation checklist assessing the student performance. While the students do not receive a grade on their performances, they could have their grade for the course reduced by as much as a third of a letter grade if they do not prepare and conduct the interview or counseling session appropriately.[58]

 One question arises immediately: Why use lay persons instead of professors to evaluate the law students’ performance? The answer is that the use of standardized clients seems to be a fair, reliable, and much more cost efficient method of providing individual feedback. Providing enough professors to give individual feedback to 500 first‑year students on three separate exercises is simply cost prohibitive, at least for most law schools.[59] NYU is an exception. While using standardized clients is not free of charge, it appears to be a manageable option.[60] Our proposal to establish a metropolitan training facility for all local law schools would lower costs even more.[61]

As in the earlier version of the Lawyering course, the videotaped demonstrations,[62] the standardized client exercises, and the substantive law and skills readings continue to provide the basis for classroom discussions of the development of case theories in each of the cases and of the challenges faced by the lawyers in the three cases. With respect to the standardized client feedback, the professors comment on their observation of a limited number of videotaped student exercises and also provide some statistical findings on the exercise. By compiling the results of the structured feedback forms, the professors have often been able to show that the problems and challenges faced by the students were typically experienced by many others.[63] Conversely, the statistics could show how nearly all of the students were doing something right, at least from the vantage point of the standardized clients performing the evaluations.

Finally, as it had done previously, the final exam calls on the students to analyze and incorporate into their case theory new facts and law and then to provide a written critique of a videotaped depiction of a lawyer’s performance. In the current version of the course, each student receives feedback from a standardized client on three different exercises. Thus, last year we completed more than 1500 standardized client exercises. The students have the opportunity to view and to analyze critically numerous other videotaped lawyer performances. The final exam then presents students with the challenge of putting those experiences together to synthesize their skills and case theory readings with substantive law and with the challenge of developing facts into viable case plans. Finally, students critically analyze a lawyer’s legal reasoning and oral lawyering skills.

The following briefly describes the process that we have been using in drafting the structured feedback forms and in training and preparing the standardized clients. The objective is to have all of the persons role-playing the client or witness portray the individual involved in a standardized fashion. This means that we train each person to assume the same profile, to know the same facts, and to respond appropriately to student questions and techniques so that the experience of each student is as close as possible to that of all other students.[64] Similarly, we want each standardized client to evaluate each of the student’s performances in a like manner. This is a critical training goal. Hence, the standardized client’s performance is standardized both as to role-playing and as to assessing. To do this, each of the four professors teaching the course conducts three two-hour training sessions with roughly ten to twelve potential standardized clients.[65] In our case, they are nearly all actors.[66] They need not be. The four professors spend approximately two-thirds of the training time on evaluation and grading issues.[67]

Using the medical educators’ experience with standardized patients as a model, we have drafted a written form that facilitates the completion of a structured evaluation by the standardized client.[68] It contains three parts: one addressing the substantive content of what the interviewing or counseling law student does, one addressing the communication skills of the law student, and one giving the standardized client the opportunity to include subjective comments. There is a need to raise the level of sophistication of these forms regarding the collection of data. The following section discusses this objective.

C. Evaluation of Our Use of the Standardized Clients

Our assessment of the success of this continuing experiment has taken many forms. Throughout, we have had the benefit of student feedback from our standard course evaluation forms. In the prior Lawyering class format, we also solicited specific written evaluations from the students of the standardized client exercises. We have also had the benefit of a small group of law professors who have worked together closely on this course. Several of us have sat in on one or another of our colleagues’ classes, and we have scrutinized the efficacy of these clinical teaching techniques in a large class setting.[69] Collaboration among law professors to design and implement a uniformly taught course is itself an unusual experience among law professors. In this instance, it has been a wonderfully productive enterprise. Finally, there has been no shortage of ad hoc student responses as to all aspects of their work with the standardized clients.

With respect to student response, the view is nearly all positive.[70] While we have not yet surveyed the students specifically on the current version of Lawyering (with three standardized client exercises), the standard course evaluations and the ad hoc comments have indicated quite clearly that the students have found the standardized client experiences very useful. Many students indicated that this was the first time they had the chance to talk to another person from the vantage point of a lawyer; they were appreciative. There have been many favorable observations about the value of viewing videotaped demonstrations of a particular lawyering function before attempting to perform the function themselves. One recurring comment, however, is that the students regret having put so much time into preparation for the exercises without getting a grade or some kind of credit for their work.[71] Earlier comments suggested that the experience might be more effective if it were not in the first semester; as noted above, students now take the course in the second semester. Some second-career students questioned the need for these exercises because they were quite experienced in dealing with people at different levels.[72] Two other recurring student comments have been: (1) standardized clients are not always consistent and (2) standardized clients do not always disclose information they possess.[73]

As set forth below, our goal is to work with psychometricians to design surveys and questionnaires that will provide the framework for empirical analyses regarding the appropriate use of standardized clients. Only after we can substantiate, as solidly as possible, the reliability and validity of these assessment methods, can we seriously consider them as significant components in the law school grading regime.

D.  Expansion of the Use of Standardized Clients

In our Lawyering course, we spend a good deal of time drawing connections between legal analysis and factual analysis (usually in the context of developing a case theory for a client). We also focus on the connections between a case theory and how the lawyer applies and utilizes that case theory in the context of performing a lawyering task such as interviewing a witness or counseling a client. This is the crucial means-ends analysis that is central to competent lawyering.[74] It becomes clear quite quickly to the student (or at least that is our hope) that lawyers cannot effectively perform such a lawyering task unless they have their legal analyses clear in their minds. By placing them in role, the students begin to gain an understanding of their duties and tasks as a professional and of the combination of skills necessary to carry out  those responsibilities  that come with that role.

When our first-year students sit down to counsel their client about the likelihood of success in asserting an adverse possession defense to a claim for an order evicting him from the plaintiff’s property, those students must know the doctrinal property law nuances and must have synthesized a workable case theory for their client. They must then be able to explain their legal opinions in clear and non-condescending language; that is transmitting their expertise to their clients in plain English.[75] Students cannot, and from everything I have seen do not, sit down until they feel they are capable of performing this task competently.[76] A critical component in their preparation is their analysis of the law as applied to the facts of their client’s situation. Placing the student in the role of the lawyer working with the client to solve the client’s problem is the experiential learning that is critical to good lawyering.[77] The student must work with the client to integrate law with all of the non-legal variables that affect the client’s decision.

If first-year students had to conduct similar standardized client exercises in their other first-year courses, it is likely that their view of the importance of interpersonal lawyering skills would change.[78] This would be especially true if legal educators graded and included the exercises as a component in the overall course grade.[79] If a metropolitan training center for all local law schools existed,[80] law schools could incorporate the exercises into these other first‑year courses quite easily and with almost no intrusion into the other professors’ class plans.[81] A Torts professor could select a key opinion that students will be reading in the traditional casebook and could ask the center to draft a standardized client counseling problem based on that case and an accompanying checklist for the standardized client to use to evaluate the performance of the student. That form would include both a substantive law component (for example, the explanation of the applicable law and a legal opinion on what the likely result will be in the case) as well as a communication skills component (that is, how well the student interacted with the client). The students would interview or counsel the standardized client in the week before the discussion of the key opinion in class.[82] During that class, the Torts professor would conduct the usual discussion of the doctrinal lessons from that case. In teaching Civil Procedure and in observing other first-year classes, my experience has been that a simulation exercise prior to the relevant class discussion considerably enhances the level of student engagement and the quality of the doctrinal discussion.[83] Contextualizing doctrinal learning is no longer unusual.[84] “Simulations . . . enhance student motivation to learn.”[85] Experiential learning can even “infuse[e] passion” into the doctrinal learning environment.[86] All of this can take place without spending a moment of traditional first-year courses’ class time on the lawyering skills related to the student experiences with the standardized client. Professors could repeat the same thing in each of the other first-year classes.[87]

The cumulative effect of integrating these exercises into traditional first‑year courses[88] could be significant.[89] First, were each exercise a component of the grade, students would understand the overall importance of mastering these oral lawyering skills. For better or worse, grades remain a primary motivator for students. The relevance of the ability to interview and counsel would compare to the importance of basic legal reasoning and analysis skills that are otherwise focused on and reinforced in all of the traditional first-year courses. Rather than communicating to the students that these lawyering skills are relatively unimportant by relegating them to a couple of credits in a thirty-credit first year, the students would be receiving the clear message that these skills are integral to all of what happens in the first year. Second, if there were a course like our Lawyering course (perhaps as part of the typical first‑year writing course), various skills themes could provide the larger context in which the students could place their various standardized client experiences. Third, the potential impact of this required first-year experiential learning on the content and sophistication of upper-class simulation courses and clinics is enormous. At New York Law School, the students’ completion of three exercises in the Lawyering course has called for re-examination of our upper-class simulation and clinical courses. If the standardized client experiences were a part of other first-year courses, the second- and third-year classes would be substantially different.

How might this type of system be implemented? Very briefly, my proposal is based on the medical education model established at the Mount Sinai Medical School Morchand Center in New York City.[90] A metropolitan training center would have ten to twelve interview rooms, and an educational director, who would hire and train the standardized clients to role-play and evaluate the students and would administer the program. That director would work with the law professors in the various subject matters to draft the problems and the evaluation forms and to schedule the exercises. In New York City, for example, the law schools in the metropolitan area could participate and could send their students to conduct sessions at the center in a manner similar to sending students to an externship field placement. The significant cost effectiveness of this system (based on the economy of scale) would benefit all of the cooperating law schools. The more often a lay person could reprise a particular standardized client role, the less cost to the center. This is, in fact, how the Morchand Center operates in New York City for all of the local medical schools. Any area with a number of local law schools could duplicate this process effectively.

An expansion of the use of standardized clients is dependent on the establishment of this type of center. We are in the process of seeking grants to do just that. Once established, this center could be self-sustaining, with the cooperating law schools sharing the costs. In addition to the intrinsic educational value of the exercises described above, a metropolitan center in New York or elsewhere would provide the vehicle for the completion of the kind of empirical studies that educators in medical schools have been conducting for decades.

III.  The Gap Between Medical Licensors and Bar Examiners

Can the legal profession make changes in the bar exam based on the experience of medical educators?[91] With respect to the potential analogous use of standardized clients on a bar exam, is the empirical data gap so wide as to suggest that the analogy is pure fantasy? Based on the foregoing, it would be easy to say yes and to put off any further consideration of changes to the bar exam until legal educators have conducted more experiments on the use of standardized clients. However, there is enough information to keep this discussion alive while continuing to pursue further empirical study.[92] In the meantime, as suggested above, the vast experience with standardized patients would seem to provide enough support for the use of low-stakes educational grading in the form of student evaluations completed by standardized clients.

A. The Similarity Between Lawyer and Physician Communication Skills     

There are striking similarities between the communication challenges facing a physician speaking with a patient and a lawyer talking to a client. As two of the leading clinical education law professors put it some time ago: “We have found [interviewing] analogies to medical diagnosis especially useful.”[93] While the differences are also clear,[94] a physician’s initial interview with a patient who seeks assistance for an ailment presents challenges quite similar to those facing a lawyer interviewing a client regarding the problem that brought the client to the lawyer’s office. In both cases, the professional must obtain enough information to make a diagnosis. Both professionals must relate to people who may be upset in varying degrees and who may be unable to communicate clearly. Beyond the task of obtaining facts, there is the challenge of applying the professional’s expertise to the specific facts of the case before them and then translating that expertise into clear, non‑condescending language so that the patient[95] or client can digest the information and apply it in his or her decision making process.[96] Once again, both the physician and the attorney confront this translation-communication challenge in remarkably similar circumstances. The process of deduction is the same for both doctor and lawyer.[97] It is because of these similarities that I first became interested in medical educators’ work with standardized patients.[98]

In an extensive and illuminating review of Donald Schön’s work, Professor Richard Neumann cataloged Schön’s insights and then suggested ways in which legal educators might act on those insights.[99] Among other suggestions, Neumann urged legal educators to “[s]ubject our theories of action [regarding lawyering notions] to empirical testing.”[100] Further, he suggested that legal educators “[d]ocument in convincing detail the ways in which legal education is far behind all the other professions in providing reflective practica.”[101] Finally, after looking at architecture and medicine, Neumann concluded that legal educators ought to disseminate that documentation to the widest possible audience.[102] Maybe then change will occur.

The medical establishment’s extensive research with standardized patients has enabled it to reach the point of including standardized patients as part of the licensing exam.[103] That medical experience provides pragmatic and moral support for ongoing efforts by the legal profession to conduct similar research and for the use of standardized clients in low-stakes law school grading. However, that support is insufficient grounds upon which to base any decisions to use a standardized client for a high-stakes exam. As Neumann pointed out, more work is necessary.[104]

B.  Empirical Analyses of Standardized Client Use

The use of non-lawyers and non-teachers to evaluate the performance of law students is both counterintuitive and, as noted above, contrary to long-standing legal education traditions. This does not mean that change is impossible. Rather, it means that, like medical educators, legal educators have to construct studies carefully to determine whether legal educators can effectively and efficiently train lay persons to give valid evaluations.

An expansion of the use of standardized clients, as described above, could provide the data for this needed empirical analysis. There are, broadly, two directions in which further examination of the use of standardized clients may proceed. First, there are empirical assessments that seek to measure the reliability, validity, and efficacy of performance and evaluations by standardized clients. Second, there should be surveys assessing the degree of satisfaction of the students. We did this previously at New York Law School, and with minor modifications of the questionnaire that we used then,[105] we intend to do this again. Legal educators should look for specific guidance from medical educators as to how and in what directions legal educators might proceed with respect to measuring the value of standardized clients. In addition, legal educators should focus on developing effective checklists—the questionnaires that the standardized clients complete in their evaluation of the law student performance. The goal here is to refine the formats as much as possible as to both the content and the methods of quantification for grading purposes. A related issue is setting the standard for a passing score.[106]

It is worth noting again the astonishing difference between what medical educators have done in terms of the quality and quantity of empirical research on methods of assessing professional competence and what legal educators have done or, more specifically, not done.[107] Exactly why this has been the case is an interesting issue, but it is beyond the scope of this Article.[108] For decades, medical professors, often jointly with psychometricians, have examined in great detail which testing methods have worked. Using the pyramid of knowledge, they have looked at alternative means to test pure knowledge (at the bottom of the pyramid), comparing multiple choice questions, true-false, and single-best-answer methods. For problem-solving and ultimate-performance assessments involving judgment, these medical educators have compared computerized problem-solving exercises, as well as observation of performance with real patients, and of course, they have engaged in extensive analysis of the reliability and validity of the use of standardized patients. For example, while recognizing the value of standardized patients, one study has also clearly acknowledged that “[p]erformance on [a standardized patient evaluation] might not be the same as performance in real life.”[109] Yet, it is the lack of consistency and, ultimately, reliability in evaluating medical students’ work with real patients (which is dependent on the happenstance of the illnesses of the patients who are available at the time of testing) that has been one of the driving forces for the use of standardized patients.[110]

Another lesson from medical educators is the importance of objectively assessing the value of the use of standardized patients or, in our case, of standardized clients. While there are benefits, there are also costs. Refined analysis might very well show differences between medical education and legal education insofar as the value of this technique or by revealing significant limitations on legal educators’ potential use of standardized clients. A recent article, for example, goes to some lengths to point out serious limitations in the use of standardized patients (as opposed to real patients) and the importance of using a variety of assessment methods.[111] Similarly, in another close analysis of the value of standardized patients, the author reached the clear conclusion that, while standardized patient encounters are excellent for assessing the ability to interact with a patient, medical educators should not use them to test clinical reasoning, which they could test much more effectively with written or computer-based exams.[112]

Thus, looking to the vast experience of medical education contemporaries for guidance on empirical research gives legal educators a very wide range of options to pursue. For example, with the cooperation of a first-year Contracts professor, we might ask for volunteers to participate in a standardized client experience. Assume that 80 students volunteer. We would ask 40 to participate in counseling a standardized client in a case that parallels an opinion that is the subject of class discussion. At the end of the semester (or perhaps at the conclusion of the unit in which the opinion is read), all 80 students would conduct a counseling session with a standardized client in a different case involving the same basic contract concepts. We could then compare the scores and evaluations of the two groups to determine if the results differed. The results may not conclusively establish a link between the additional counseling experience and the students’ ease in applying substantive knowledge because other factors might have an impact, such as the greater immersion into the subject matter by the experimental group as opposed to the control group. However, the results would be useful in determining whether there was an impact on interpersonal communication skills.[113]

One clear method of assessing the use of standardized clients is to compare the scores given by the standardized clients with those of professors who observe and evaluate videotapes of the same student exercises. I did this in an initial experiment involving a much smaller class of 45 students. The results were not, however, statistically significant. With respect to the Lawyering course, this would necessarily entail much larger numbers, perhaps a random sampling of 50 to 100 students on each of three or more standardized client exercises. This would allow inter-rating for multiple exercises and would approach what a typical Objective Structured Clinical Exam, which usually includes eight to ten standardized patient exercises, does for medical students. Some refer to this as comparing the standardized patient to a “gold standard”—the professorial evaluation. In the case of the Lawyering course, it would require the cooperation of several Lawyering professors viewing many videotapes and scoring the students observed.[114]

On a less scientific level, we might ask the students in a Criminal Law course to conduct a counseling session with a standardized client who is a criminal defendant to whom the students must give very bad news about the prospects of success at trial. A pre-counseling survey might ask the students: (1) What are your primary concerns? and (2) What is your level of comfort in giving such counseling? After the exercise, we might ask all of the participants the same questions as well as a question asking them to rate the educational value of the exercise on a scale of one to five.[115]

An interesting medical school project parallels this concept. Recognizing that first-year medical students were quite uncertain about their clinical skills, the Northwestern University faculty designed and implemented a program using five standardized patient exercises for first-year students.[116] The objective was to provide the new students with a non-threatening opportunity to identify their strengths and weaknesses as they begin work on clinical skills. The faculty did not grade students and even provided them with the checklists that the standardized patients would use in advance. These early formative experiences proved to be valuable introductions to clinical medicine.[117]

Changes in legal education, like the law itself, occur slowly and incrementally.[118] At the same time, a longer view provides a framework—a structure—within which change might take place. Here, the goal is twofold. First, from the outset, legal education should fully integrate doctrine and applied lawyering skills in the law school curriculum. Likewise, an emphasis on the role of the lawyer, along with the challenge of knowing what it means to be a professional, should be a more integral part of the first‑year experience. One vehicle to assist in this venture is the standardized client method. The means used to evaluate law students’ progress should also reflect a synthesized notion of theory, doctrine, and skills. Visual aids, interactive electronic programs, and standardized clients would all assist in the law student assessment effort. The second long-term goal is to match the means of evaluation applied in any high-stakes examination for entry to the profession with the integrated educative and evaluative methods, which the law schools will have established.[119]

Bar examiners (and the state courts and governments that oversee the bar examiners) should support the incremental approach necessary to produce a refined vision of how to ensure that lawyers are competent to practice law. There is implicit recognition that, for cost reasons, legal educators cannot replicate either the clinical model of medical education in law school or a meaningful apprenticeship system after law school. However, that does not mean that legal educators should do nothing toward pursuing a more well-rounded preparation for the practice of law. The objective here is to maintain and even speed up that movement.

Conclusion

Drawing the connection between initial efforts to use standardized clients in a legal education setting and improvements to the bar exam requires imagination or at least faith in long-term planning. However, that is exactly what is necessary if changes in bar admission screening are ever to take place. Inertia plays a large role in maintaining the status quo regarding bar exams. It is an especially powerful force in this area because most lawyers are so happy to put the bar exam experience behind them that they really do not want to address the more fundamental issues about the purposes of the exam and whether the current exams achieve those purposes. Thus, I welcomed the opportunity to tie my work with standardized clients into a far-reaching proposal to experiment with alternative ways to assess applicants for bar admission. It may very well be that the small pilot that the New York bar association committees have proposed could integrate the use of standardized clients. The collection of data from both the Committees’ PSABE and the law school programs could be valuable in assessing the overall worth of the standardized client method, which would benefit both programs. A bonus under the PSABE would be the provision of legal services by the bar applicants participating in the pilot.