What is critical appraisal and how should I consider this as part of my PhD thesis?

There are a number of questions you should pose, whilst reading a peer-reviewed manuscript. Here are a few:

1.            Who funded the study?

As an example, a recent publication by Tomljenovic and Shaw highlighted the dangers of the HPV vaccine, Gardasil (www.ncbi.nlm.nih.gov/pubmed/23061593 ). They claim that it is a dangerous vaccine and will not reduce cervical cancer. Aside from the fact that the authors work in the Department of Ophthalmology, they are funded by a number of families in America. Upon further reading, unfortunately these 3 families have had children die or suffer illness – the families believe that vaccines were the cause of death and/or illness and have devoted monies into research which show this. As such, there is a bias throughout this work, which the authors do not acknowledge.

2.            What kind of study is it?

Assess the methodology – is this a case study i.e. a report on one patient? Is it a case series i.e. a report into a number of patients? Or is it an analytical study such as a cohort study? A cohort study or randomised controlled trial (RCT) are generally much more robust and scientifically valid than a case series since they will often involve larger numbers, are well-controlled and have appropriate statistical analyses. In the aforementioned Tomljenovic paper, the authors conducted their analysis on 2 patients who died. Remember, that millions of young girls have been vaccinated with this vaccine and the stringent safety assessments believe this to be a very safe vaccine. In Scotland, over 100,000 doses of Gardasil have been given with no deaths.

3.            What is the sample size under scrutiny?

In order to assess statistical significance, it is important to consider sample size. The larger the study, the more likely it is that a difference will be observed, if one exists. This is to do with the power of the study. However, the great difficulty with Healthcare-associated infection (HCAI) is that it is often not possible to assess differences in treatment regimens or practice unless there are enough numbers. While the IPCN may have a hypothesis as to how people became infected, it is often difficult to test that hypothesis using an analytical study such as a case-control study. Therefore, many reports of outbreaks and incidents in the healthcare setting tend to be descriptive. Perhaps an important question to consider is can a study be performed over multiple sites or is this a single-centre study? The former may be logistically harder but it is much more robust, provided each centre adheres to the same methodology. You should bear this in mind, when reading your paper.

4.            What is the sample demographic?

Consider your study participants – are they a group of children on a paediatrics ward or a group of elderly cancer patients? Could the results of a study looking at burns patients and MRSA infection be applicable to the residents of a nursing home? Are the study findings generalisable to the population?

5.            Is there any bias in the study?

There are many forms of bias that can invalidate the findings of a study e.g. recall bias, investigator bias and selection bias. These will be explained in the ESHAI module. Once bias has been introduced into a study, it is very difficult to remove.

6.            Have the authors considered the weaknesses and strengths of the study?

Every good paper should have a section on this, in the discussion. No study is perfect and this is a good place to consider the pros and cons of the findings – it should also help you to formulate your answer to your assessment i.e. although you may have quoted a paper, you are also cogniscent of its limitations and strengths. See this paper, 4th para in the discussion, where weaknesses are described.

Final note

This list is not exhaustive. Consult STROBE and CASP (via Google) to see the types of questions you should be considering when reading your paper. It will of course be a pain to have to download the full paper, when you are doing your literature search. However, by simply reading the abstract, you cannot get a full picture of the study and it is likely that you will not be able to show whether your articles that you cite, have been critically appraised. In short, the more you put into your essays and thesis, the better it will be. This isn’t dogma; this is based on 8 years of personal experience in marking assessments and dissertations – those who pass or attain distinction, do so because they have thoroughly researched the pertinent literature.