Your trusted source for ehealth, telehealth, home health and telemedicine information and networking.

Telehealth: PSYCHOTHERAPY USING DISTANCE TECHNOLOGY
A COMPARISON OF OUTCOMES

by: Paul L. Schneider, Ph. D. 01/21/01
Paper presentation: American Psychological Association 1999, Boston Abstract

The use of telehealth, the delivery of health services over a distance via technology, continues to expand throughout the field of health services (Grigsby, 1997). Very few studies have empirically examined the effectiveness of this method of delivery (Baer, Cukor, & Coyle, 1997). The purpose of this study was to examine one specific area of telehealth, psychotherapy and its effectiveness across three different modes of treatment delivery (face to face, two-way audio, and two-way video).

In this study, three treatment groups and one wait list control group were used. Clients in each of the three treatment groups received five sessions of psychotherapy. The clients communicated with their therapist through only one mode of communication, face-to-face, two-way audio, or two-way video.

Outcome scores were obtained from measurements on the Brief Symptom Inventory (Derogatis, 1993; Derogatis & Spencer, 1982), Target Complaints (Battle, Imber, Hoehn-Saric, Stone, Nash, & Frank, 1966), the Global Assessment Scale (Endicott, Spitzer, Fleiss, & Cohen, 1976), the Client Satisfaction Questionnaire (Tracey & Dundon, 1988) and the Therapist Satisfaction Questionnaire (Tracey & Dundon, 1988).

These measures were then analyzed across groups to determine if there are any differential effects. A final piece of this study examined the effects of comfort with communication technologies on treatment. The findings of this study will help to provide guidelines for the future use of these technologies in the delivery of psychotherapy.

Psychotherapy Using Distance Technology:

A Comparison of Outcomes

The expansion of telehealth, the delivery of health services over distances using technology, continues to grow in leaps and bounds (Grigsby, 1997). However, studies empirical examining the effectiveness of this method of delivery have not met this growth (Baer, Cukor, & Coyle, 1997). The purpose of this study was to examine one specific area of telehealth, psychotherapy and its effectiveness across three different modes of treatment delivery (face to face, two-way audio, and two-way video).

The American health care system continues to face three particularly tenacious problems: an uneven geographic distribution of health care resources; inadequate access to health care for certain segments of the population including those who are isolated or confined; and the continually rising costs of care (Bashur, 1997). One of the answers to these problems currently being advocated is telehealth. The force behind this trend has come from both industry and government. Industry continues to develop new technologies to more effectively reach distant populations, while, the government has begun to earmark large amounts of money for the development of telehealth projects as well as require that services delivered via these methods be reimbursed (Nickelson, 1996). If health services and specifically psychotherapy can be effectively delivered through distance technology, its use will, probably for the first time, truly address the problem of underserved populations, such as the geographically remote, handicapped, and homebound in an effective way.

Although telehealth and specifically psychotherapy through distance is relatively new in the public eye, it has in fact been conducted using various two-way audio and video systems for over four decades. Having been in use for such a long period one would think that its effectiveness had been studied. However, although there are a small number of studies that have examined the use of this technology, virtually none of them have empirically examined the effectiveness of psychotherapy delivered through two-way audio or video (e.g., Sleek, 1997).

The field of telehealth and psychotherapy in particular is approaching a turning point in history. Technology is advancing at a rate that will enable us to do what some had only dreamed of doing before. The main purpose of this study was to help close this gap in the literature by comparing various psychotherapy outcomes and moderating variables across three different modes of treatment delivery, face-to-face, two-way audio, and two-way video. Specifically this study sought to:

Method

Study Design

A three-group, between-subjects design was used, in which clients were randomly assigned to one of three psychotherapy delivery modes (face-to-face, audio, and video). Each of the clients received treatment via one of the three delivery modes. One third of these clients were assessed first and then placed on a wait list prior to treatment, at which time they were reassessed. This group was included as a control "no treatment" group.

Outcome Assessment

Over 25 years ago Bergin and Garfield (1971) noted that one of the main problems with assessing outcome, in general, is in the selection of outcome measures and the subsequent lack of agreement between researchers on which measures to use. Today, this is still true. When assessing outcome, the consensus appears to be measurement through multiple dimensions (Cartwright, 1975; Volsky, Magoon, Norman, & Hoyt, 1965).

In order to address these issues the assessment of outcome multiple sources of information (e.g., counselor and client) and multiple targets for change (e.g., symptom change, satisfaction level, problem resolution) were used (Strupp & Hadley, 1977). In choosing the source of outcome, inclusion of the therapists was important because of their ability to view significant behavioral change. An important second source of outcome is that of the client. The clients are the consumers and ultimately they are the ones who provide the impetus for seeking therapy and then returning if they were satisfied with their experiences.

In determining what to measure from these various sources of outcome a target must also be chosen. On a dimensional level this refers to whether the measure falls on the global end of the spectrum or the specific end. Some measures are designed to measure a very global trait or outcome (e.g., personality dimensions, global symptom distress level, satisfaction level), while others look at a specific problem (e.g., number of negative self statements, length of tolerance to a specific stimuli). Due to the heterogeneity of the population we used client specified specific problem inventories and global measures of overall adjustment (Gelso, 1979).

From both the client and clinician viewpoints global perspectives on symptoms, functioning, specific perspectives on client problems, and satisfaction were all assessed. The instruments used were as follows:

Each of these instruments was chosen for its ability to assess each of the areas previously outlined as well as demonstrated validity and reliability.

Aptitude by Treatment Effects

Matching the client to the treatment, or the treatment mode can be an important factor. Given a new and novel method of treatment it is important to assess as to whether certain clients have a greater aptitude, or a personality, that is more suited for a particular delivery mode. A measure previously developed the Distance Communication Comfort Scale (DCCS), was used to assess the clients' comfort level in audio, video, and face to face therapy.

Procedures

Clients contacted our clinic through the telephone and made initial assessment appointments. Each client was randomly assigned to a treatment mode and then assessed in that mode. If the clients met our specific inclusion and exclusion criteria they were scheduled for follow-up sessions for a total of five sessions including the initial assessment.

Throughout the treatment clients and clinicians were asked to fill out a variety of assessment instruments. At the conclusion of treatment, clients were debriefed about the goals of the research study and given referrals if appropriate.

Participants

Clients. The clients who completed treatment (n=80) ranged in age from 19 to 75 (mean = 39.35, SD = 15.88), included 52 women and 28 men, and reported ethnic identities of 66 White, eight African American, three Asian, and three Other. All clients received free psychotherapy. Clients had a wide variety of backgrounds and presenting problems.

Although most of the clients who attended the first session completed treatment there were some (n=11) dropouts. These clients attended between one to three sessions and dropped out for a variety of reasons. The small number of dropouts gave the study an overall attrition rate of less than 8%. However, rates for the different treatment modes did vary with five for audio, four for video, two for face to face.

Exclusion Criteria

Clients were excluded from the study if they did not meet a minimum distress level, were unable to generate problems to work on, or were unable to commit themselves to a full course of treatment. Subclinical levels of symptomology were accepted, since nonpathological people often need help with problems such as adjustment to new situations, grief, time management, romantic difficulties, and so on. Given that very brief cognitive-behavioral therapy was the mode of treatment, extreme cases, such as actively suicidal and psychotic clients were to be referred to another agency. However, this situation did not arise.

Therapists

The therapists were members of an advanced practicum group supervised by doctoral level psychologists. All the therapists had completed two academic years or more of practical experience and had master's degrees in clinical or counseling psychology.

Treatments

Therapists primarily used a brief cognitive-behavioral treatment approach in light of its efficacy over the very short term (Koss & Shiang, 1994). However, the therapists were encouraged to adjust their treatment style as appropriate, much like a real world situation. At the conclusion of the treatment sessions the therapists rated their own adherence to the suggested therapeutic orientation. This information--the average degree of perceived adherence to a cognitive-behavioral protocol--is provided in the final report to assist readers in interpreting results.

Locale

Although there are three different modes of delivery and one control group, all clients received treatment in the same building. Though the centralization of locale did not replicate the conditions of real-world distance technology, it did provide a control of the physical setting, which might otherwise introduce unmeasured variability among clients, such as privacy levels, differences in travel time, and equipment problems.

Treatment Delivery Modes

The technology used to represent each mode was selected to emulate an ideal technological environment. A closed circuit television system with two 20" television sets was used to simulate two way video delivery. This same system was also used, without a picture, to simulate the two way audio system. Although the picture and sound quality obtained through this method is greater than today's typical video teleconference situation, it did provide a ceiling level for treatment through this type of technology. In the face-to-face mode the client and therapist were in the same room. All of the sessions were videotaped.

Analyses

Multivariate analyses of variance (MANOVA) were performed to examine the outcome measures in relation to the three modes of delivery and wait list control group. For significant overall multivariate results, follow-up tests were performed to determine which pairs of groups differed significantly on the sets of variables and which variables contribute to the significant pairwise differences.

Following this analysis a second MANOVA was performed to compare the treatment groups. The same procedures were used for this analysis. A multivariate within subjects analysis was used to determine if levels of comfort changed over time as measured before treatment, after two sessions and after five sessions.

An aptitude by treatment interaction model and regression analysis was used to investigate whether levels of comfort with different types of technology had any main or interaction effects on outcome across the three treatment modes.

Results

The outcome variables, BSI (GSI), TC ­ Client, TC ­ Therapist, and GAF were entered into a multivariate analysis with the four dependent variables. The dependent variables were the three treatment groups, face (face to face), audio, and video. The fourth group is the wait-list control group (control). There was both a significant intercept (F = 2863 (4,100)) and groups (F = 1.82 (12,265)) effect. The observed power of .93 exceeds the estimated level of power needed to detect a medium effect size.

Due to the significance found in the MANOVA separate ANOVAs were performed on the different variables. The ANOVAs revealed significant differences between the groups on three of the four outcome variables (p <.10). The symptom inventory, GSI, was the only measure not to show a significant difference.

Finally, post hoc analysis was performed to analyze the differences between groups on these three significantly different variables. This analysis found significant differences between the Video and the Control on the GAF (p <.15), the Video and the Control on the Target Complaints ­ Client (p <.05), the Face to Face and the Control on the Target Complaints ­ Client (p <.10), and the comparison between each of the three treatment groups Face to Face, Video, Audio and the Control group (p <.15, p <.05, p <.01).

The results of this analysis demonstrate support for the hypothesis that treatment outcome significantly differs from no treatment. The first step of the analysis demonstrated this overall pattern difference. In the subsequent analysis this pattern was further examined. Looking specifically at the post hoc analysis a consistent pattern of differences can be found. For each of the treatment groups there was a significant difference between them and the control group on the TC therapist scale. Furthermore, although this significance is not as pronounced on the other two significant variables, TC client and GAF, there is a consistent pattern of differences for each of these groups. In sum, three of the four outcome variables demonstrated significant differences between the treatment groups and control group.

Treatment Effectiveness Across Groups

The next step was to look for differences between treatment groups. The control group was excluded from this MANOVA and two more outcome measures, the client and therapist satisfaction scales, were added. These, along with the original four outcome variables, were entered into a multivariate analysis with the three treatment modes as dependent variables.

There were no significant model effects. However, the observed power of .60 was below the estimated level of power needed to detect a medium effect size. In conclusion the null hypothesis could not be rejected, thus supporting the hypothesis of no differences in outcome among treatment groups.

Changes in Comfort Levels Over Time

In this analysis I sought to determine if comfort levels change over time and specifically which comfort variables, over what time period and in which groups. A multivariate analysis of variance of within subjects on the three DCCS scales over time with the between subject variable mode of treatment (Face to Face, Video or Audio) was performed next. The results of this analysis show that the profiles of the measures changed significantly, both when the treatment groups were analyzed as a whole (F=3.56 (6,68)), and separately (F=2.91 (12,136)).

Given these findings, the next step was to examine the pattern of change for individual comfort measures across treatment groups. The results of this test show that the pattern of the audio and video subscales changed significantly over time when analyzed separately across treatment groups (F = 4.06 (4,146), F = 6.78 (4,146)). Although there was change in the face to face scale, it did not reach significance (F = 1.80 (4,146)). This was the expected result in that people are likely to adapt to new communication mediums over time. In the case of face-to-face, this medium is not new and thus no change was expected.

The next step in the analysis breaks down this pattern of change and examines the time differentials individually across treatment modes. The pattern that emerges after this final analysis is as expected. The level of comfort with audio, as determined by the DCCS audio scale, changes significantly over time for those individuals who are in the audio treatment group (F=3.69 (2,24)). This comfort level does not change significantly in any of the other treatment groups. This identical pattern was found in respects to comfort with video and the individuals in the video treatment group. Comfort with video increased significantly over time, but only for those individuals who received treatment in the video group (F = 18.38 (2,23)).

The last step in this analysis was to specifically look at the changes in the audio and video measures between time 1 and 2 and time 2 and 3 within the audio and video treatment groups. The results support the hypothesis that people adapt to the communication medium with which they are presented. The Audio comfort increased in the Audio group between session one and two (F = 1.07) and had a significant increase between session 2 and session 5 (F = 4.36). The Video group followed a similar pattern, but with a stronger and significant initial gain (F = 20.19) followed by another significant gain between session 3 and 5 (F = 6.10). Furthermore, the final analysis demonstrated that this change in comfort occurs over a relatively short time period, particularly in respects to comfort with video.

Aptitude by Treatment Effects:

Contribution of Comfort Level to Outcome

In order to assess aptitude by treatment effects a composite outcome score was obtained. Z scores for each of the six outcome variables were computed. These scores we combined to for a single composite and used as the dependent variable in the three separate regression analysis. For each of these regression analysis the independent variables were a DCCS comfort scale (one for each analysis), treatment mode, and an interaction effect. Dummy coding was used for the analysis of the mode and interaction variables. The DCCS measurements used were those gathered at time 1. The results showed no significant interaction or main effects for the comfort with face to face variable. However there were significant interaction effects for the comfort with video (D R2.= .07) and audio (D R2.= .07) variables.

In order to determine the interaction effects the individual beta weights were examined. The first interaction variable contrasted the video treatment group with the face to face and audio groups. The second interaction variable provided a contrast between the audio treatment group and the face to face and video groups. In the case of the Video group the second interaction was significant (t = 2.35, p <.05). For the Audio group both interaction variables were significant (t = 1.71, p <.09 and t = 2.24, p <.03).

Initial examination of the video scale suggests that higher levels of comfort on the video scale lead to higher outcome levels in the audio treatment group while lower levels are better for the face to face and/or video groups. In terms of the audio scale, both contrasts are significant which would suggest that with low levels of comfort the face to face group does best and with high levels of comfort the audio groups do best.

Looking at the results it is apparent that the interaction is present for both technology groups as original hypothesized. In the case of the audio comfort scale, there is a significant aptitude by treatment effect, which suggests that comfort with audio play a significant role in certain treatment modes in the expected direction. However, in the case of the video scale, while it does have a significant aptitude by treatment effect, its effect is slightly different than originally hypothesized. In this instance higher levels of comfort are significantly better for the audio treatment group, not the video. However, although the significance in this interaction lies within the contrast between the audio treatment group and the video and face to face groups, the video group's overall aptitude by treatment pattern is similar to that of the audio group.

Conclusion

Do the Treatment Conditions Differ from the Control Group?

It has been established that people who receive therapy, on average, improve significantly more than those who do not receive therapy over a given time period (Lambert & Bergin,1994). Although this is the common wisdom, it is important when delving into any new area of research that its accuracy be reestablished. Several steps were taken to address this first hypothesis: Clients were randomly assigned to the treatment conditions. The length of time the control group stayed on the wait list was similar in length to the actual treatment period. Assessment of outcome was made on several variables, using both global and specific improvements from client and therapist perspectives.

In our analysis the pattern of the four outcome variables was first compared across groups. It was found that there was a significant difference between the four groups. Given the significance subsequent individual ANOVAs were performed to determine specifically where this difference lay. It was then established that there were significant differences across groups on three of the four outcome variables and that there was a consistent pattern among the treatment groups. These differences provide support for the hypothesis that treatment in each of the three conditions is better than no treatment.

Do the Three Treatment Conditions Differ?

As previously asserted, there have been virtually no empirical examinations comparing the effectiveness of audio, video, and face to face treatment in individual counseling. This second analysis found no significant differences among treatment groups. Thus the null hypothesis of no treatment differences could not be rejected. Although these are fairly strong findings there are limitations.

It appears that in a brief therapy situation therapy delivered over two-way audio or video is comparable to face-to-face treatment. However, although there were no significant differences between treatment groups in outcome levels it must be noted that there was differential drop out rates. Both the audio and video modes had larger drop out rates than the face to face mode and although the overall rates for each mode was quite low, this should not be discounted. It is quite possible that client variables such as comfort level, problems, or personality are important in determining whether a client is suitable for treatment over a particular delivery mode. In addition, a longer treatment period should be considered for future comparisons of these modes. Effectiveness of treatment might begin to differ over time or across client variables. In sum it appears that these three delivery modes can be used to provide similarly effective treatment, but that there are several variables that must still be examined before we can universally declare equivalent effectiveness.

Does Comfort with Communication Mediums Change Over Time?

One common assertion when discussing communication over two way audio or video is that people will be uncomfortable with this type of interaction and it will affect the communication process in a maladaptive manner. This is especially common in respect to beliefs about counseling. However, research into communication has established that people actually adapt to new mediums quite well and find effective ways to communicate (Heeter, 1992; Held & Durlach, 1992). Given these findings one would suspect that, on average, people will adapt. Still, many would argue that counseling is a different type of relationship; thus it is important to determine if in this type of relationship people adapt. To address this comfort level with different communication mediums in a counseling situation was examined over time.

Using a multivariate within subjects analysis, difference in comfort levels for the three scales were analyzed over the three time periods across treatment groups. The results found that there was both an overall significant pattern difference when the treatment groups were looked at as a whole and as individual groups. As expected, there is a significant difference in the pattern of the DCCS scales over time in the Audio and Video treatment groups and no significant difference in the Face to Face group. Finally the nature and pattern of these differences was established through a post hoc analysis.

It was found that audio comfort scale increased in time for the people in the audio treatment group. It did not significantly increase in any of the other treatment groups. The pattern found for the video scale was identical. Further analysis looking specifically at the differences between times 1 and 2 and times 2 and 3 found that this change became significant within the first time period for video and second period for audio. In conclusion, people do adapt to the medium in question. Last, comfort levels increase over time and do so fairly quickly.

Comfort with Communication Mediums: Aptitude by Treatment Effects

Although people adapt to comfort with technology over time, this does not mean that people will universally adapt, thus negating any mediator effects by comfort level. Examination of the previous within subjects analysis might lead one to conclude that, due to the increase in comfort levels, mode would not significantly affect treatment outcome. However, although most people adapt, some in all likelihood do not. For some this change might not be significant enough to eliminate aptitude by treatment effects. The analysis performed resulted in two significant findings and provided partial support for this hypothesis.

In the case of the audio scale, there was a significant interaction effect, suggesting that comfort with audio does contribute significantly to outcome levels. With the video scale there was a similar effect, however it was the comfort with audio which was significantly related to differential outcome levels. Comfort with face to face mode did not contribute significantly to outcome.

Given these findings it seems important that comfort with video and particularly audio be taken into consideration when providing treatment and that this area receives further research. Although the exact nature of this aptitude by treatment effect is not entirely clear, it is apparent that comfort with various communication modes does affect treatment outcome. In the clinical realm this can information can help clinicians to assure that a clients' comfort with a particular treatment mode is addressed, lest it hamper a potentially successful intervention.

References

Baer, L., Cukor, P., Coyle, J. T. (1997). Telepsychiatry: Application of telemedicine. In R. L. Bashshur, J. H. Sanders, & G. W. Shannon (Eds.), Telemedicine: Theory and Practice (pp. 265-290). Springfield, IL: Charles C Thomas.

Bashshur, R. L. (1997). Telemedicine and the health care system. In R. L. Bashshur, J. H. Sanders, & G. W. Shannon (Eds.), Telemedicine: Theory and Practice (pp. 265-290). Springfield, IL: Charles C Thomas.

Battle, C. C., Imber, S. D., Hoehn-Saric, R., Stone, A. R., Nash, E. R., & Frank, J. D. (1966). Target complaints as criteria for improvement. American Journal of Psychotherapy, 20, 184-192.

Bergin, A. E., & Garfield, S. L. (Eds.) (1971). Handbook of psychotherapy and behavior change. New York, NY: Wiley.

Cartwright, D. S. (1975). Patient self report measures. In I. E. Waskow & M. B. Parloff (Eds.), Psychotherapy change measures. Rockville, MD: National Institute of Mental Health.

Derogatis, L. R. (1993). Brief symptom inventory: Administration, scoring, and procedures manualžII. Minneapolis, MN: National Computer Systems.

Derogatis, L. R. & Spencer, P. M. (1982). The brief symptom inventory manual. Baltimore: John Hopkins School of Medicine.

Endicott, J., Spitzer, R. L., Fleiss, J. L., & Cohen, J. (1976). The global assessment scale. Archives of General Psychiatry, 33, 766-771.

Gelso, C. J. (1979). Research in counseling: Methodological and professional issues. Counseling Psychologist, 8, 7-30.

Grigsby, J. (1997). Telemedicine in the United States. In R. L. Bashshur, J. H. Sanders, & G. W. Shannon (Eds.), Telemedicine: Theory and Practice (pp. 265-290). Springfield, IL: Charles C Thomas.

Heeter, C. (1992). Being there: The subjective experience of presence. Presence, 1, 262-271.

Held, R. M., & Durlach, N. I. (1992). Telepresence. Presence, 1, 109-112.

Mintz, J. Kiesler, D. J. (1982). Individualized measures of psychotherapy outcome. In P. C. Kendall & J. N. Butcher, (Eds.), Handbook of research methods in clinical psychology (pp. 429-460). New York, NY: John Wiley & Sons.

Nickelson, D. W. (1996). Behavioral telehealth: Emerging practice, research, and policy opportunities. Behavioral Sciences and the Law, 14, 443-457.

Sleek, S. (1997). Providing therapy from a distance: Video conferencing helps practitioners maintain strong relationships with patients they can't see regularly. APA Monitor, 28(8), 1+.

Strupp, H. H., & Hadley, S. W. (1977). A tripartite model of mental health and therapeutic outcomes. Archives of General Psychiatry, 36, 1125-1136.

Tracey, T. J. & Dundon, M. (1988). Role anticipations and preferences over the course of counseling: An interactional examination. Journal of Counseling Psychology, 31, 13-27.

Volsky, T., Jr., Magoon, T. J., Norman, W. T., & Hoyt, D. P. (1965). The outcomes of counseling and psychotherapy. Minneapolis, MN: University of Minnesota Press.