Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

SOCW 553: Research Knowledge and Evidence in Clinical Social Work Practice

Overview of evidence-based practice, literature review/systematic review workflows, and citation management for your written EBP assignment.

Title & Abstract Screening

The Title/Abstract screening stage is sometimes one of the longest, and you'll see many of the results you found in the databases excluded due to lack of applicability to your eligibility criteria. It is not uncommon to see your list of results cut down by 50% or more.

This is why it is particularly important to ensure you are finding enough results in the databases - if you're not, your question may be too specific, or your search may not be including the most relevant search terms. 

To Screen Title/Abstract in Covidence:

  • Open your review
  • Select the Title and Abstract Screening stage if it is not already open
  • Click the Continue button on the right-hand side
  • Read through the abstracts and titles - and select No, Maybe, or Yes depending on whether the publication meets the eligibility criteria for your group
    • NOTE: Your group may choose not to use Maybe, as it moves along with the Yes' and it can create more conflicts in the system to have to approve. Stick to using maybe if you are really unsure if a publication qualifies.
  • Continue through your list of publications to screen.
  • When completed - reconvene your group to talk through and approve any conflicts around inclusion.

Things to Remember:

  • Try and split up the work evenly amongst your group members, so that everyone gets the chance to try screening. 
  • Do not talk through decisions with other group members as you are screening - this will help to eliminate bias.
  • Some publications may be missing an abstract - leave those for the end and double check that it wasn't just missed when importing, or move it to the full-text review and review the content for inclusion there. 
  • If you come across additional duplicates - remember to note Manual Duplicate with a study tag, before excluding it from the review. 
    • Sorting by Title will help you to see if there were any duplicates the system missed.
  • If you can't remember your group's eligibility criteria, open the Show Criteria button at the top of the screening list to remind yourself.
  • Add Notes as you see fit based on what your group might want to track for later.

Full-Text Screening

Once your group approves publications to move to the next stage of screening, they will populate in the Full Text Review stage for you all to add the full text and screen again for eligibility.

Again, it is very common to see many of your remaining articles be screened out when reviewing the full text. This is because abstracts are notoriously lacking details that often relate to clinical questions and eligibility criteria - you'll find you can only truly know if an article will be relevant to answering your question by reviewing its full methods. 

To Add Full Text Documents:

  • Open your review
  • Select the Full Text Review stage of your review, if it is not already open
  • Click Continue to start reviewing
  • Open another browser window with the Library homepage: https://library.ok.ubc.ca/
  • Search for the article title in the main search box - Summon
  • Locate the article in the Summon results (it sometimes isn't first), and go to the full text of the article
  • Download the PDF to add in Covidence
  • Back in Covidence, click the Add Full Text button just below the title of the article
  • Upload the PDF you downloaded
  • Click Finished

To Screen in Full Text Review:

  • Open your review
  • Select the Full Text Review stage of your review, if it is not already open
  • Click Continue to start reviewing
  • Locate or open the full text - reading through it against your eligibility criteria
  • Select Include or Exclude depending on if the article fits your group's parameters
    • If you are excluding, you'll be asked to indicate why. Your group should be using standardized exclusion reasons, so that the PRISMA chart will populate with them at the end of your review. You should be pre-selecting these or remaining in close contact with group members as they are applying them to ensure that everyone is using the same reasons. 
  • Continue screening full text until the list is complete

Things to Remember:

  • Try and split up the work evenly amongst your group members, so that everyone gets the chance to try full text review. 
  • Do not talk through decisions with other group members as you are reviewing the full text - this will help to eliminate bias.
  • If you come across additional duplicates - remember to note Manual Duplicate with a study tag, before excluding it from the review. 
    • Sorting by Title will help you to see if there were any duplicates the system missed.
  • If you can't remember your group's eligibility criteria, open the Show Criteria button at the top of the screening list to remind yourself.
  • Add Notes as you see fit based on what your group might want to track for later.

Snowballing

Once your group has finalized the articles it will most likely or definitely use via the full text review in Covidence, you can review the reference lists and/or forward citations for each of the included studies - this is called Snowballing, and it is very common in systematic reviews. It is not required for your group assignment. 

Review each of the citations the articles for inclusion have listed to see if there are any more relevant articles to include in your review. This is good practice because sometimes articles are not published in the journals that get indexed in the subject databases you have chosen.

It is common to see that no new articles were found, but it is still good practice to go through and check and note in your methods that this was at least completed with nothing new located. 

Remember - always stick to your eligibility criteria though, especially date range.

Using either Google Scholar or Web of Science, check each of the included articles to review any future articles that have cited the one you are including. This can help to locate other publications that are newer, and to double check that nothing was missed in your original searches. 

It is common to see that no new articles were found, but it is still good practice to go through and check and note in your methods that this was at least completed with nothing new located. 

Remember - always stick to your eligibility criteria though, especially date range.

Exporting Your Included Studies

In exporting included studies, you'll be pulling from the system information related to the article, including citation information, the abstract (if available), any of the notes your group might have made on individual titles, and any study tags applied. You can also export your inter-rater reliability for your title & abstract screening and your full-text review phases. 


Exporting General Article Information:

The reason you are doing this, is to move to the next phase in the review process: Critical Appraisal. While the system allows for critical appraisal, you will not be completing this step in the Covidence system. 

  • Open the review you wish to export
  • Click the Export button in the top right corner
  • Under the References box on the left, select the Stage you want you export from (probably Included), and select CSV as your Format
  • Click Export
  • Wait for the export to appear below in Recent Exports, and click Download when the file is available
  • Open the CSV file with Excel - you'll now be able to see all of your included studies citation information, notes, tags, etc. 

NOTE: You may also want to export your article information into RefWorks or another citation management system, to allow for easy citation creation for your References list. It is suggested that you do this at the end before you begin writing your final assignment. 


Exporting Inter-Rater Reliability Information:

As good practice, it is common to see inter-rater reliability mentioned in a methods section of a review. This tells readers how often the reviewers agreed or disagreed on the inclusion of an article to the review. This information is often shared to disclose any potential biases in the process, and to promote transparency of the experience of completing that specific review. 

  • Open the review you wish to export
  • Click the Export button in the top right corner
  • Under Inter-Rater Reliability in the centre, select the stage you wish to export. You can export both to include. 
  • Click Export CSV 
  • Wait for the export to appear below in Recent Exports, and click Download when the file is available
  • Open the CSV in Excel - you'll be able to see the reliability calculations for all of the screening scenarios between reviewers
  • Calculate the overall average of the Proportional Agreement or use the Cohen's Kappa measure to represent the agreement on a sliding scale. 
    • "Cohen suggested the Kappa result be interpreted as follows: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement" (McHugh, 2012, p. 279).

For your group assignment, it is easy for you to include this information because Covidence creates it for you. Do try and include this in your methods section. 

**McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 267-282. https://hrcak.srce.hr/89395