Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

Abstract

Background: Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic’s complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision. Methods: An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance. Results: The gold standard set comprised 534 citations. The search filter optimised for recall (‘Broad Integrated Care Search’) achieved 86.0–88.3% recall with corresponding low precision (47–53%). The search filter optimised for precise searching (‘Narrow Integrated Care Search’) demonstrated precision of 73–95% with recall reduced to between 55.9 and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care. Conclusions: The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic. Keywords: Information retrieval, Integrated care, Literature searching, Precision, PubMed, Recall, Search filters