The Effects of Psychotherapy—An Evaluation

THE recommendation of the Committee on Training in Clinical Psychologyof the American Psychological Association regarding the training of clinicalpsychologists in the field of psychotherapy has been criticized by the writerin a series of papers [10, 11, 12]. Of the arguments presented in favor ofthe policy advocated by the Committee, the most cogent one is perhapsthat which refers to the social need for the skills possessed by the psychotherapist.In view of the importance of the issues involved, it seemed worthwhile to examine the evidence relating to the actual effects of psychotherapy,in an attempt to seek clarification on a point of fact.

BASE LINE AND UNIT OF MEASUREMENT

In the only previous attempt to carry out such an evaluation, Landis haspointed out that "before any sort of measurement can be made, it is necessaryto establish a base line and a common unit of measure. The only unitof measure available is the report made by the physician stating that thepatient has recovered, is much improved, is improved or unimproved. Thisunit is probably as satisfactory as any type of human subjective judgment,partaking of both the good and bad points of such judgments" (26, p. 156).For a unit Landis suggests "that of expressing therapeutic results in termsof the number of patients recovered or improved per 100 cases admitted tothe hospital." As an alternative, he suggests "the statement of therapeuticoutcome for some given group of patients during some stated interval oftime."

Landis realized quite clearly that in order to evaluate the effectiveness ofany form of therapy, data from a control group of nontreated patientswould be required in order to compare the effects of therapy with the spontaneousremission rate. In the absence of anything better, he used the ameliorationrate in state mental hospitals for patients diagnosed under the headingof "neuroses." As he points out:

There are several objections to the use of the consolidated amelioration rate . . . ofthe . . . state hospitals . . . as a base rate for spontaneous recovery. The fact that psychoneuroticcases are not usually committed to state hospitals unless in a very bad condition ;the relatively small number of voluntary patients in the group ; the fact that such patientsdo get some degree of psychotherapy especially in the reception hospitals ; and the probablyquite different economic, educational, and social status of the State Hospital groupcompared to the patients reported from each of the other hospitals—all argue againstthe acceptance of [this] figure . . . as a truly satisfactory base line, but in the absence ofany other better figure this must serve (26, p. 168).

Actually the various figures quoted by Landis agree very well. The percentageof neurotic patients discharged annually as recovered or improvedfrom New York State hospitals is 70 (for the years 1925-1934); for theUnited States as a whole it is 68 (for the years 1926-1933). The percentageof neurotics discharged as recovered or improved within one year of admissionis 66 for the United States (1933) and 68 for New York (1914). Theconsolidated amelioration rate of New York state hospitals, 1917-1934,is 72 per cent. As this is the figure chosen by Landis, we may accept it inpreference to the other very similar ones quoted. By and large, we may thussay that of severe neurotics receiving in the main custodial care, and verylittle if any psychotherapy, over two-thirds recovered or improved to aconsiderable extent. "Although this is not, strictly speaking, a basic figurefor "spontaneous" recovery, still any therapeutic method must show anappreciably greater size than this to be seriously considered" (26, p. 160).Another estimate of the required "base line" is provided by Denker:

Five hundred consecutive disability claims due to psychoneurosis, treated by generalpractitioners throughout the country, and not by accredited specialists or sanatoria, werereviewed. All types of neurosis were included and no attempt made to differentiate theneurasthenic, anxiety, compulsive, hysteric, or other states, but the greatest care wastaken to eliminate the true psychotic or organic lesions which in the early stages of illnessso often simulate neurosis. These cases were taken consecutively from the files of theEquitable Life Assurance Society of the United States, were from all parts of the country,and all had been ill of a neurosis for at least three months before claims were submitted.They, therefore, could be fairly called "severe," since they had been totally disabled for atleast a three months' period, and rendered unable to carry on with any "occupation forremuneration or profit" for at least that time (9, p. 2164).

These patients were regularly seen and treated by their own physicianswith sedatives, tonics, suggestion, and reassurance, but in no case was anyattempt made at anything but this most superficial type of "psychotherapy"which has always been the stock-in-trade of the general practitioner. Repeatedstatements, every three months or so by their physicians, as wellas independent investigations by the insurance company, confirmed the factthat these people actually were not engaged in productive work during theperiod of their illness. During their disablement, these cases received disabilitybenefits. As Denker points out, "It is appreciated that this fact ofdisability income may have actually prolonged the total period of disabilityand acted as a barrier to incentive for recovery. One would, therefore, notexpect the therapeutic results in such a group of cases to be as favorable asin other groups where the economic factor might act as an important spurin helping the sick patient adjust to his neurotic conflict and illness"(9, p. 2165).

The cases were all followed up for at least a five-year period, and often aslong as ten years after the period of disability had begun. The criteria of"recovery" used by Denker were as follows: (a) return to work, and abilityto carry on well in economic adjustments for at least a five-year period;(b) complaint of no further or very slight difficulties ; (c) making of successfulsocial adjustments. Using these criteria, which are very similar to thoseusually used by psychiatrists, Denker found that 45 per cent of the patientsrecovered after one year, another 27 per cent after two years, making 72 percent in all. Another 10 per cent, 5 per cent, and 4 per cent recovered duringthe third, fourth, and fifth years, respectively, making a total of 90 per centrecoveries after five years.

This sample contrasts in many ways with that used by Landis. The caseson which Denker reports were probably not quite as severe as those summarizedby Landis; they were all voluntary, nonhospitalized patients, andcame from a much higher socioeconomic stratum. The majority of Denker'spatients were clerical workers, executives, teachers, and professional men.In spite of these differences, the recovery figures for the two samples arealmost identical. The most suitable figure to choose from those given byDenker is probably that for the two-year recovery rate, as follow-up studiesseldom go beyond two years and the higher figures for three-, four-, and fiveyearfollow-up would overestimate the efficiency of this "base line" procedure.Using, therefore, the two-year recovery figure of 72 per cent, wefind that Denker's figure agrees exactly with that given by Landis. We may,therefore, conclude with some confidence that our estimate of some twothirdsof severe neurotics showing recovery or considerable improvementwithout the benefit of systematic psychotherapy is not likely to be veryfar out.

EFFECTS OF PSYCHOTHERAPY

We may now turn to the effects of psychotherapeutic treatment. Theresults of nineteen studies reported in the literature, covering over seventhousand cases, and dealing with both psychoanalytic and eclectic types oftreatment, are quoted in detail in Table 1. An attempt has been made to reportresults under the four headings : (a) Cured, or much improved ; (b) Improved ;(c) Slightly improved; (d) Not improved, died, discontinued treatment, etc.It was usually easy to reduce additional categories given by some writersto these basic four; some writers give only two or three categories, and inthose cases it was, of course, impossible to subdivide further, and the figuresfor combined categories are given.f A slight degree of subjectivity inevitablyenters into this procedure, but it is doubtful if it has caused much distortion.A somewhat greater degree of subjectivity is probably implied in the writer'sjudgment as to which disorders and diagnoses should be considered to fallunder the heading of "neurosis." Schizophrenic, manic-depressive, andparanoid states have been excluded; organ neuroses, psychopathic states,and character disturbances have been included. The number of cases wherethere was genuine doubt is probably too small to make much change in thefinal figures, regardless of how they are allocated.

A number of studies have been excluded because of such factors as excessiveinadequacy of follow-up, partial duplication of cases with others includedin our table, failure to indicate type of treatment used, and otherreasons which made the results useless from our point of view. Papers thusrejected are those by Thorley and Craske [37], Bennett and Semrad [2],H. I. Harris [19], Hardcastle [17], A. Harris [18], Jacobson and Wright [21],Friess and Nelson [14], Comroe [5], Wenger [38], Orbison [33], Coon andRaymond [6], Denker [8], and Bond and Braceland [3]. Their inclusionwould not have altered our conclusions to any considerable degree, although,as Miles et al. point out: "When the various studies are compared in termsof thoroughness, careful planning, strictness of criteria and objectivity, thereis often an inverse correlation between these factors and the percentage ofsuccessful results reported" (31, p. 88).

Certain difficulties have arisen from the inability of some writers to maketheir column figures agree with their totals, or to calculate percentages accurately.Again, the writer has exercised his judgment as to which figures toaccept. In certain cases, writers have given figures of cases where there wasa recurrence of the disorder after apparent cure or improvement, withoutindicating how many patients were affected in these two groups respectively.All recurrences of this kind have been subtracted from the "cured" and"improved" totals, taking half from each. The total number of cases involvedin all these adjustments is quite small. Another investigator makingall decisions exactly in the opposite direction to the present writer's wouldhardly alter the final percentage figures by more than 1 or 2 per cent.

We may now turn to the figures as presented. Patients treated by meansof psychoanalysis improve to the extent of 44 per cent; patients treatedeclectically improve to the extent of 64 per cent; patients treated only custodiallyor by general practitioners improve to the extent of 72 per cent. Therethus appears to be an inverse correlation between recovery and psychotherapy;the more psychotherapy, the smaller the recovery rate. This conclusionrequires certain qualifications.

In our tabulation of psychoanalytic results, we have classed those whostopped treatment together with those not improved. This appears to bereasonable; a patient who fails to finish his treatment, and is not improved,is surely a therapeutic failure. The same rule has been followed with thedata summarized under "eclectic" treatment, except when the patient whodid not finish treatment was definitely classified as "improved" by thetherapist. However, in view of the peculiarities of Freudian proceduresit may appear to some readers to be more just to class those cases separately,and deal only with the percentage of completed treatments which are successful.Approximately one-third of the psychoanalytic patients listed brokeoff treatment, so that the percentage of successful treatments of patientswho finished their course must be put at approximately 66 per cent. It wouldappear, then, that when we discount the risk the patient runs of stoppingtreatment altogether, his chances of improvement under psychoanalysis areapproximately equal to his chances of improvement under eclectic treatment,and slightly worse than his chances under a general practitioner or custodialtreatment.

Two further points require clarification: (a) Are patients in our "control"groups (Landis and Denker) as seriously ill as those in our "experimental"groups? (b) Are standards of recovery perhaps less stringent in our "control"than in our "experimental" groups? It is difficult to answer these questionsdefinitely, in view of the great divergence of opinion between psychiatrists.From a close scrutiny of the literature it appears that the "control" patientswere probably at least as seriously ill as the "experimental" patients, andpossibly more so. As regards standards of recovery, those in Denker's studyare as stringent as most of those used by psychoanalysts and eclectic psychiatrists,but those used by the State Hospitals whose figures Landis quotesare very probably more lenient. In the absence of agreed standards of severityof illness, or of extent of recovery, it is not possible to go further.

In general, certain conclusions are possible from these data. They failto prove that psychotherapy, Freudian or otherwise, facilitates the recoveryof neurotic patients. They show that roughly two-thirds of a group ofneurotic patients will recover or improve to a marked extent within abouttwo years of the onset of their illness, whether they are treated by meansof psychotherapy or not. This figure appears to be remarkaby stable fromone investigation to another, regardless of type of patient treated, standardof recovery employed, or method of therapy used. From the point of viewof the neurotic, these figures are encouraging; from the point of view of thepsychotherapist, they can hardly be called very favorable to his claims.The figures quoted do not necessarily disprove the possibility of therapeuticeffectiveness. There are obvious shortcomings in any actuarial comparisonand these shortcomings are particularly serious when there is solittle agreement among psychiatrists relating even to the most fundamentalconcepts and definitions. Definite proof would require a special investigation,carefully planned and methodologically more adequate than these ad hoccomparisons. But even the much more modest conclusions that the figuresfail to show any favorable effects of psychotherapy should give pause to thosewho would wish to give an important part in the training of clinical psychologiststo a skill the existence and effectiveness of which is still unsupportedby any scientifically acceptable evidence.

These results and conclusions will no doubt contradict the strong feelingof usefulness and therapeutic success which many psychiatrists and clinicalpsychologists hold. While it is true that subjective feelings of this type haveno place in science, they are likely to prevent an easy acceptance of the generalargument presented here. This contradiction between objective fact and subjectivecertainty has been remarked on in other connections by Kelly andFiske, who found that "One aspect of our findings is most disconcerting tous : the inverse relationship between the confidence of staff members at thetime of making a prediction and the measured validity of that prediction.Why is it, for example, that our staff members tended to make their bestpredictions at a time when they subjectively felt relatively unacquaintedwith the candidate, when they had constructed no systematic picture of hispersonality structure? Or conversely, why is it that with increasing confidencein clinical judgment . . . we find decreasing validities of predictions?" (23,p. 406).

In the absence of agreement between fact and belief, there is urgent needfor a decrease in the strength of belief, and for an increase in the number offacts available. Until such facts as may be discovered in a process of rigorousanalysis support the prevalent belief in therapeutic effectiveness of psychologicaltreatment, it seems premature to insist on the inclusion of trainingin such treatment in the curriculum of the clinical psychologist.

SUMMARY

A survey was made of reports on the improvement of neurotic patientsafter psychotherapy, and the results compared with the best available estimatesof recovery without benefit of such therapy. The figures fail to supportthe hypothesis that psychotherapy facilitates recovery from neurotic disorder.In view of the many difficulties attending such actuarial comparisons,no further conclusions could be derived from the data whose shortcomingshighlight the necessity of properly planned and executed experimental studiesinto this important field.

REFERENCES

1. ALEXANDER, F . Five year report of the Chicago Institute for Psychoanalysis, 1932-1937.

2. BENNETT, A. E., and SEMRAD, Å. V. Common errors in diagnosis and treatment of thepsychoneurotic patient—a study of 100 case histories. Nebr. med. J., 1936, 21, 9 0 - 9 2 .

3. BOND, E. D., and BRACELAND, F. J. Prognosis in mental disease. Amer. J. Psychiat.,1937, 94, 2 6 3 - 2 7 4 .

4. CARMICHAEL, H. T., and MASSERMAN, T. H. Results of treatment in a psychiatricoutpatients' department. / . Amer. med. Ass., 1939, 113, 2 2 9 2 - 2 2 9 8 .

5. COMROE, Â. I. Follow-up study of 100 patients diagnosed as "neurosis." / . nerv.ment. Dis., 1936, 83, 6 7 9 - 6 8 4 .

6. COON, G. P., and RAYMOND, A. A review of the psychoneuroses at Stockbridge.Stockbridge, Mass.: Austen Riggs Foundations, Inc. 1940.

7. CURRAN, D. The problem of assessing psychiatric treatment. Lancet, 1937, II, 1 0 0 5 -1009.

8. DENKER, P. G. Prognosis and life expectancy in the psychoneuroses. Proc. Ass. LifeInsur. med. Dir. Amer., 1937. 24, 179.

9. DENKER, R. Results of treatment of psychoneuroses by the general practitioner.A follow-up study of 5 0 0 cases. N.Y. State J. Med., 1946, 46, 2164-2166.

10. EYSENCK, H. J. Training in clinical psychology: an English point of view. Amer.Psychologist, 1949, 4, 173-176.

11. EYSENCK, H. J. The relation between medicine and psychology in England. In W. Dennis(Ed.), Current trends in the relation of psychology and medicine. Pittsburgh: Univer.of Pittsburgh Press, 1950.

12. EYSENCK, H. J. Function and training of the clinical psychologist. / . ment. Sei., 1950,96, 1-16.

13. FENICHEL, Ï. Ten years of the Berlin Psychoanalysis Institute, 1920-1930.

14. FRIESS, C , and NELSON, M. J. Psychoneurotics five years later. Amer. J. ment. Sei.,1942, 203, 5 3 9 - 5 5 8 .

15. HAMILTON, D. M., VANNEY, I. H., and WALL, T. H. Hospital treatment of patientswith psychoneurotic disorder. Amer. J. Psychiat., 1942, 99, 2 4 3 - 2 4 7 .

16. HAMILTON, D. M., and WALL, T. H., Hospital treatment of patients with psychoneuroticdisorder. Amer. J. Psychiat., 1941, 98, 551-557.

17. HARDCASTLE, D. H. A follow-up study of one hundred cases made for the Departmentof Psychological Medicine, Guy's Hospital. / . ment. Sei., 1934, 90, 536-549.

18. HARRIS, A. The prognosis of anxiety states. Brit. med. J., 1938, 2, 6 4 9 - 6 5 4

19. HARRIS, H. I. Efficient psychotherapy for the large out-patient clinic. New England ËMed., 1939, 221, 1-5.

20. HUDDLESON, J. H., Psychotherapy in 2 0 0 cases of psychoneurosis. Mil. Surgeon,1927, 60, 1 6 1 - 1 7 0 .

21. JACOBSON, J. R., and WRIGHT, K. W. Review of a year of group psychotherapy.Psychiat. Quart., 1942, 16, 7 4 4 - 7 6 4 .

22. JONES, E. Decennial report of the London Clinic of Psychoanalysis, 1926-1936.

23. KELLY, E. L., and FISKE, D. W. The prediction of success in the VA training programin clinical psychology. Amer. Psychologist, 1950, 5, 3 9 5 ^ 0 6 .

24. KESSEL, L., and HYMAN, H. T. The value of psychoanalysis as a therapeutic procedure./. Amer. med. Ass., 1933, 101, 1612-1615.

25. KNIGHT, R. Ï. Evaluation of the results of psychoanalytic therapy. Amer. J. Psychiat.,1941, 98, 4 3 4 - 4 4 6 .

26. LANDIS, C. Statistical evaluation of psychotherapeutic methods. In S. E. Hinsie (Ed.),Concepts and problems of psychotherapy. London, Heineman, 1938, pp. 155-165.

27. LUFF, M. C , and GARROD, M. The after-results of psychotherapy in 500 adult cases.Brit. med. J., 1935, 2, 5 4 - 5 9 .

28. MAPOTHER, E. Discussion. Brit. J. med. Psychol., 1927, 7, 57.

29. MASSERMAN, T. H., and CARMICHAEL, H. T. Diagnosis and prognosis in psychiatry./. ment. Sei. 1938, 84, 8 9 3 - 9 4 6 .

30. MATZ, P. Â. Outcome of hospital treatment of ex-service patients with nervous andmental disease in the U.S. Veteran's Bureau. U.S. Vet. Bur. med. Bull., 1929, 5,829-842.

31. MILES, H. H. W., BARRABEE, E. L., and FINESINGER, J. E. Evaluation of psychotherapy.Psychosom. Med., 1951, 13, 8 3 - 1 0 5 .

32. NEUSTATTER, W. L. The results of fifty cases treated by psychotherapy. Lancet, 1935,1, 796-799.

33. ORBISON, T. J. The psychoneuroses : psychasthenia, neurasthenia and hysteria, withspecial reference to a certain method of treatment. Calif, west. Med., 1925, 23, 1132—1136.

34. Ross, T. A . An enquiry into prognosis in the neuroses. London, Cambridge Univer.Press, 1936.

35. SCHILDER, P. Results and problems of group psychotherapy in severe neuroses. Ment.Hyg., N.Y., 1939, 23, 8 7 - 9 8 .

36. SKOTTOWE, I., and LOCKWOOD, M. R. The fate of 1 5 0 psychiatric outpatients. / . ment.Sei., 1935, 81, 5 0 2 - 5 0 8 .

37. THORLEY, A . S., and CRASKE, N . Comparison and estimate of group and individualmethod of treatment. Brit. med. J., 1950, 1, 9 7 - 1 0 0 .

38. WENGER, P. über weitere Ergebnisse der Psychotherapie im Rahmen einer MedizinischenPoliklinik. Wien. med. Wschr., 1934, 84, 3 2 0 - 3 2 5 .

39. WILDER, J. Facts and figures on psychotherapy. J. clin. Psychopath., 1945, 7, 3 1 1 - 3 4 7 .

40. YASKIN, J. C. The psychoneuroses and neuroses. A review of 1 0 0 cases with specialreference to treatment and results. Amer. J. Psychiat., 1936, 93, 1 0 7 - 1 2 5 .