INTERVENTION RESEARCH How can the evidence base for Psychological Interventions be assessed? MAP, WS3 Prof. Dr. Claus Vögele Aims of the session: • • • provide an overview of models of intervention research; introduce students to a phase model of intervention research; describe in detail methodological issues specific for each phase (baseline, enquiry, pilot, test, quality assurance); Learning outcomes of the session: • • • • • distinguish between various study designs used in intervention research; discuss advantages and disadvantages related to these study designs; describe assessment criteria for intervention studies; discuss methods of meta-analysis and related issues; describe the distinction between efficacy and effectiveness in relation to intervention research Models of intervention research: a historical perspective • General examination of efficacy - e.g. Institute for Psychoanalysis , Berlin (1930) • Controlled trials - H.J. Eysenck‘s Hypotheses (1952): spontaneous remission • Differential intervention research - Process-outcome research (Kiesler, 1969) • Perspektive approach - Micro- (single individual) vs Macro- (institution) approach Models of intervention research • • Phase models - Intervention research as sequence of distinct phases Meta-analyses - Integration of study results • Guidelines for interventions - Efficacy and indication for sub-groups of people, manuals • Quality assurance and management - Optimisation of phase IV Phases of intervention research • Baseline • Phase I: Enquiry phase • Phase II: Pilot phase • Phase III: Test phase • Phase IV: Quality assurance Evaluation criteria 1 Efficacy - Statistical significance of change - Clinical significance of change - Percentage of improved patients - Range of improvement (outcome parameters) - Pattern of change - Stability of improvement/change over time - Negative side-effects Evaluation criteria 2 Cost-Effectiveness, Cost-Benefit 3 Patient satisfaction 4 Effectiveness (Seligman, 1995) 5 Ethical considerations Back to ... Phases of intervention research • Baseline • Phase I: Enquiry phase • Phase II: Pilot phase • Phase III: Test phase • Phase IV: Quality assurance Methodology of enquiry phase • Analog studies ... may be partially different from the target group or context to be investigated in one or more of the following - species, disorder, examination procedure • Single case studies ... examine single individuals, single groups or single communities - Single case experiment, quantitative single case study, case study (time series analyses) Back to ... Phases of intervention research • Baseline • Phase I: Enquiry phase • Phase II: Pilot phase • Phase III: Test phase • Phase IV: Quality assurance Methodology of pilot phase • Study designs: Control group designs - No explicit control group or control condition: One group design · one measurement at the end of intervention · repeated measurement (Pre / Post) · within group comparison under condition of non-treatment Methodology of pilot phase • Study designs: controlled trials - Explicit control conditions · non-treated control group · waiting list control group · insignificant treatment of control group (placebo) · conventional treatment of control group (routine) · specific treatment of control group (alternative therapy) · specific treatment of control group with modification of treatment parameters (e.g. duration of session) Assessment criteria for intervention studies (APA, 1997) • Intervention: - Description, control group, instructions, setting, frequency and duration of therapeutic sessions, period of treatment, other interventions • Classification: - Reliability and validity of assessment procedures • Dependent variables - Spectrum and adequacy of parameters, length of catamnestic period • Therapist: - Qualification, traits (e.g. gender), expectation Assessment criteria for intervention studies (ctnd.) (APA, 1997) • Patients: - Recruitment of participants, expectation, clinical status, voluntary participation, allocation to treatment or control group • Procedure: - Drop-outs in general and group selective, criteria for exclusion of participants • Data analysis: - Statistical tests, significance levels, effect size calculations, uni- vs multivariat procedures etc. Back to ... Phases of intervention research • Baseline • Phase I: Enquiry phase • Phase II: Pilot phase • Phase III: Test phase • Phase IV: Quality assurance Methodology of test phase Aim: • Provision of a broad empirical basis, e.g. multicentre-studies Problem: • Aggregation of single study results Possible solutions: • Meta-analyses, normative criteria Exkurs: Was sind Meta-Analysen? Quantitative, statistische Verfahren, die die Zusammenfassung von verschiedenen Untersuchungen zu einem wissenschaftlichen Forschungsgebiet ermöglichen. Dabei werden die empirischen Einzelergebnisse statistisch ausgewertet, um zu einer Effektstärkeneinschätzung zu kommen. d = (MT - MK) / SK d = .20 d = .50 d = .85 Group activity: Please concentrate on the following: 1) What was the primary research question? 2) How were studies identified and selected? 3) What is your view on the quality / reliability / appropriateness of the studies included in the MA? 4) What meta-analytical statistics were used? 5) Give a brief summary of results and conclusions, including a critique. Back to ... Methodology of test phase Aim: • Provision of a broad empirical basis, e.g. multicentrestudies Problem: • Aggregation of single study results Possible solutions: • Meta-analyses, normative criteria Normative Criteria (APA, 1993) • Evaluation of therapy effectiveness using explicit guidelines - at least two well controlled group trials from different research teams - a multitude of experimental single case studies - Intervention strategy/techniques are based on intervention manual Back to ... Phases of intervention research • Baseline • Phase I: Enquiry phase • Phase II: Pilot phase • Phase III: Test phase • Phase IV: Quality assurance Methodology of quality assurance • Seligman (1995): effectiveness vs efficacy Seligman, M.E.P. (1995). The effectiveness of psychotherapy. American Psychologist, 50, 965-974. • Quality assurance by careful documentation of intervention Primary aim of intervention research: ... is the continuing optimisation of patient care Group activity: Please concentrate on the following: 1) Description of methodology (sample, therapy, therapists, manual, selection, statistical procedure etc.) 2) Efficacy or effectiveness? 3) What was the primary research question? http://www.youtube.com/watch?v=mljU_9AC964 Discussion 1. Is it OK to be eclectic? 2. Is it OK to refer to one‘s own personal experience and ignore the scientific evidence? 3. What would help you to take into account the scientific evidence? Additional literature Hunter, J.E. Schmidt, F.L. (1990). Methods of meta-analysis. London: Sage. Kazdin, A.E. (Ed.)(1992). Methodological issues and strategies in clinical research. Washington D.C.: American Psychological Association.