Q.  Do I need to seek written permission to use the appraisal tools?
A.  Permission is not required if you are to use the tools for your own personal development or if using them for a non-profit-making activity.  We do, however, ask that you acknowledge CASP at the Public Health Resource Unit, Oxford as the producers and providers of the tools.

Q. How were the CASP appraisal tools developed?
A. The CASP appraisal tools are based on the guides produced by the Evidence Based Medicine Working Group, a group of clinicians at McMaster university, Hamilton, Canada, and colleagues across North America, published in the Journal of the American Medical Association. 1
The CASP checklists were developed specifically for teaching purposes and all followed the same four-stage process of development, consisting of:

• Drafting of written materials by a multidisciplinary working group and CASP secretariat;
• Testing the critical appraisal tool by the working group and modifying the tool in the light testing;
• Piloting of the whole package with a knowledgeable audience and further modifying the tool;
• Use of package with non-expert health staff.

Members of the multidisciplinary working groups had backgrounds in public health, epidemiology or evidence based practice. CASP staff undertook a review of the literature on methodological issues relating to relevant  studies and on scales and checklists for critical appraisal of those studies. For each package the working group, supported by CASP staff, developed an appraisal checklist (the tool), a one hour talk, a small group exercise to appraise a  research paper and a glossary of terms.  The tools were designed to address the epidemiological principles behind the study types with particular attention to assessing study validity.  All the study tools are divided into three sections relating to internal validity, the results and the relevance to practice.  Each section contains several questions with supporting hints to explain the relevance of the question or to expand on a principle.
The tools were subject to an iterative process of piloting within workshops, with feedback and review of materials, using successively less epidemiologically aware audiences.  As a result the materials were suitable for a target audience of staff in service administration and health care delivery.  Each participant completed a confidential four-page questionnaire at the end of the workshop followed by verbal feedback from the group as a whole, which was written down publicly.
Oxman AD, Sackett DL, Guyatt G. Users' guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group.  JAMA 1993;270:2093-5
 
Q.  How many copies can I make?
A.  As long as they are not being used for any profit making activity, you can print up to 10 copies?
 
Q.  How should the CASP tools be referenced?
A.  You should follow your University recommended referencing system quoting the website address and pathname.
 
Q.  Can I use your appraisal tools to help with a literature review I’m doing?
A.  Yes.  Please take note of the answer above if referencing.
 
Q.  Is it possible to use the CASP tools within our journal club?
A.  Yes.  Please acknowledge us (CASP at PHRU) as the producers and providers of the tools.
 
Q.  I’m running a profit making workshop/training event and would like to use the CASP appraisal tools.  Do I need permission?
A.  Yes, the appraisal tools can not be copied multiple times for this purpose, without our permission. We do not allow the tools to be used freely if the training given is for profit.
 
Q.  Which appraisal tool should I use for Quantitative research?
A.  You can use either the systematic review or randomised controlled trial tool. 
 
Q.  What’s the difference between a Systematic Review and a Randomised Controlled Trial?
A.  A systematic review will take results from any trial or survey and measure the contrast between them, whilst the randomised controlled trial looks at a specific topic, using a randomly identified audience.
 
A systematic review is sometimes known as secondary research, it is a summary of appropriate available primary literature – often randomised controlled trials, but not always. The randomised controlled trial is the most robust research method for an investigation about the effectiveness of an intervention in health/social care.

Q. Is there any evidence available as to the tools’ effectiveness/validity?
A. As part of specific educational strategies, directed at improving general clinical behaviour, interactive educational workshops for continuing medical education have been shown to have a moderately large impact on professional practice when compared to didactic teaching or no teaching. 

On the other hand, systematic reviews comparing problem-based learning with “traditional” medical education, have only shown modest improvements in knowledge, skills and professional behaviour for undergraduate medical students, whereas the effect on postgraduate doctors is still uncertain.  However, satisfaction with learning seems to increase for both groups.

Looking at the effects of specific educational interventions, a systematic review of the literature by Peter Bradley found that interventions aimed at teaching critical appraisal skills generally have some positive effects, but the research findings are not yet conclusive for some important outcomes.  The strongest and most consistent effects concern knowledge transfer.  It is possible that critical appraisal and searching skills, attitudes to EBP, use of medical literature and reading habits also improve, but the standard and volume of existing research does not allow an accurate assessment of the extent of this effect.  There is no evidence on how educational interventions affect patient outcomes or the process of care and no studies from the educational or social services.  In addition, in the studies identified it is not possible to comment on whether educational interventions have been delivered to an appropriate standard, describe the process by which learning takes place or consider how specific learning contexts have affected results or identify causes of variation in study results.
(With thanks to Peter Bradley for preparing this statement based on work undertaken for his PhD)

i) Thomson O’Brien MA, Freemantel N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing educational meetings and workshops: effects on professional practice and health care outcomes (Cochrane Methodology Review). In: The Cochrane Library, Issue 4, 2003. Chichester, UK: John Wiley and Sons, Ltd.
ii) Dolmans D, Schmidt HG. The advantages of problem-based curricula. Postgrad Med J 1996; 72:535-8.
iii) Albanese MA, Mitchell S. Problem-based learning:a review of the literature on its outcomes and implementation issues.Acad Med 1993; 68:550-63.
iv) Vernon DT, Blake RL. Does problem based learning work? A meta analysis of evaluation research. Acad Med 1993; 68:579-98.
v) Berkson L. Problem based learning: have the expectations been met? Acad Med 1993: 68:579-88.
vi) Norman GR, Schmidt HG. The psychological basis of problem-based learning: a review of evidence. Acad Med 1993; 67:557-65.
vii)Smits PBA, Verbeek JHAM, de Buisonjé CD. Problem based learning in continuing medical education: a review of controlled evaluation studies. BMJ 2002; 324:153.
viii) Peter Bradley Personal communication

Q.  Which appraisal tool could I use for a Controlled Clinical Trial?
A.  Often, randomised controlled trials have not been conducted in the area of interest therefore the consumer must use the research that is available. If the research has a prospective experimental design the appraisal tool to use would still be the RCT. This will help the consumer understand the risks associated with interpretation of experimental trials with quasi or NO randomisation procedures.

Q.  Which appraisal tool could I use for a Case study?
A.  A case study usually does what it says on the tin… it studies a case (or small number of cases). It is not to be confused with a case-control study. Case study is a term that is often used for investigations that may not use robust enough qualitative or quantitative research methods and the results/conclusions of ‘case studies’ should be interpreted with caution. The contemporary thinking is that case studies are essentially qualitative in nature and hence we might advise that the CASP qualitative tool is used. Failing that there is some research on a checklist to appraise single case studies, which is based on the Trisha Greenhalgh (BMJ) checklist. Here’s the link

Q.  Can you recommend which CASP tool to use when appraising a study using a retrospective approach?
A.  It depends on the research design incorporating the retrospective approach.  It could be the case-control or (retrospective) cohort tool.

Q.  Do you know of an appraisal tool to use for guidelines?
A.  Here you go It’s quite comprehensive (25 Qs!) and comes with instructions for completion.

If you can’t find an answer for your question(s), please do not hesitate to contact us