Critical appraisal of your found literature is an important part of the research process. This ensures that your results are meeting the needs to answer your research question.
Depending on your specific question and scope you may have varying criteria when it comes to the level of validity, the uniqueness, the types of methods used, etc., of the studies you are looking at. This may vary from what you had hoped to examine at the outset of your literature review, and it will certainly differ from other colleagues.
Regardless of varying expectations in appraisals with other colleagues, it is most important to remain consistent across all of resources you consult.
One example tool you may find useful in evaluating literature is the acronym P.R.O.M.P.T. - for more information visit The Open University's detailed guide.
| PRESENTATION | Review the resource for proper use of language and effectual writing. |
| RELEVANCE | Review the resource against your inclusion and exclusion criteria - does the resource match what you set out to find? |
| OBJECTIVITY | Is the author's viewpoint expressed clearly and with neutral language. Are there competing objectives - i.e. has the author been sponsored or funded to come to a certain conclusion? |
| METHOD | Has the author been clear about why a method was chosen and how they utilized it? Did the author produce results that should be seen with that type of method? |
| PROVENANCE | Is the author qualified to research or comment on this topic? Were they funded by a group or company with specific interests on the topic? Was the research published via an avenue that was neutral (e.g. double blind peer-review)? |
| TIMELINESS | When was the resource published? Is it now considered to be obsolete or perhaps too old for the question you are posing? |
There are a variety of checklists available online for free from institutions around the world that promote critical appraisal as an important part of the research process. These may not always meet the specific guidelines you are after, but are designed to act as a template that you can make your own for your specific research question. You might find that you need to add more detailed questions, or you may need to simplify them.
Although these are often routed in evidence-based practice within health practice, that does not make them any less valuable to critically appraising other research.
SAIT (Southern Alberta Institute of Technology) RADAR for Evaluating Information is another framework that considers relevance, authority, date, appearance, and reason (for creation).
​University of Melbourne Literature Review Guide - select the Critical Reading tab to view possible questions to consider when critically appraising.
Develop a system for tracking common themes. This may be through your citation management software, an Excel sheet or a Word document, whichever works best for you. Record a summary of each work, and critically evaluate the ideas, methods and results.
Use a tool and terminology that will allow you to group themes together.
One popular method is the Synthesis Matrix, a table with headers to group identified themes.
Sample Synthesis Matrix*:
|
Source #1 (citation or author's last name) |
Source #2
|
Source #3
|
|
| Idea A | Note/information and page number | ||
| Idea B | Leave blank if a paper does not discuss a main idea |
*Adapted from: University of Western Ontario Library (n.d.). “Writing your literature review”. https://guides.lib.uwo.ca/mme9642/litreview