When screening for a systematic review or meta-analysis, we conduct several pilot screening rounds. Pilot screenings help us refine our search string, decision tree, and increase the overall accuracy of our screening for literature reviews [check out this nice guide from the I-DEEL team for more info: Foo et al, 2021].
During a pilot screening, we want to select a random subset of references that would be a representative sample of the full set. When possible, screening rounds are conducted in collaboration with another reviewer. To speed up the screening process, we sometimes want to randomly allocate a subset of papers to a collaborator by splitting a reference list into subsets.
There are two reasons we’d want to automate the selection and splitting of a reference list:
- It is time consuming to randomly select papers (>100 papers is tedious to select by hand!)
- We are not really good at selecting things at random (actually computers aren’t really good at selecting truly at random either*)
Below is the R (www.r-project.org) code to run two functions that may come in useful when conducting your pilot and collaborative screenings with Rayyan (https://rayyan.ai/), or any other software where you can upload your pilot reference list.
1. Select random pilot set:
First, load the getpilotref function below in your environment:
Load example csv file that was exported from Rayyan (a reference list of papers in Ecology & Evolutionary Biology having the word “butterflies” in their title):
Load the splitref_prop function in your environment:
Using the example butterfly reference list, let’s first split the reference list in two equal splits (50% each):
Now let’s get 30% of references in the first subset (split1) and 70% in the second subset (split2), for example if one reviewer has more time to spend on the screening:
(Any comments, questions or feedback, you can reach me at: firstname.lastname@example.org)