Systematic review and meta-analysis using SyRF- a tool to accelerate drug development

G. L. Currie, J. Liao, E. S. Sena, C. T. Sena, A. Bannach-Brown, K. Hair, S. K. McCann, Q. Wang, M. R. Macleod. CCBS, University of Edinburgh, Edinburgh, United Kingdom.

Introduction: The median delay between drug discovery and licencing is 24 years[1]. A major component of this is the time it takes for translation from preclinical to clinical research. Systematic review and meta-analysis are powerful, unbiased approaches to summarising research, but at present they take months or years to complete, and this limits their utility. SyRF (the Systematic Review Facility, app.syrf.org.uk) is an online platform that supports all stages of a systematic review and meta-analyses. SyRF levers artificial intelligence tools to enable a continually updated summary of what is known. Here we report the utility of this approach to drug development in chemotherapy-induced peripheral neuropathy (CIPN) and in stroke.

Method: As part of an ongoing systematic review of drugs tested in animal models of CIPN we assessed the performance of machine learning in citation screening; and we assessed the utility of text mining in risk of bias annotation in CIPN and in reports from animal models of stroke. We report the sensitivity and specificity, comparing the performance of the artificial intelligence tools against 2 trained human researchers.

Results: Citation screening: using 33,184 unique publications as a training set we applied a machine learning algorithm to 11,880 additional publications identified in an updated search. Compared with the human “gold standard”, the algorithm performed with a sensitivity of 97% and specificity of 67%. Risk of bias annotation: Text mining tools developed in a psychosis dataset achieved high sensitivity and reasonable specificity in CIPN and stroke validation datasets (Table). Future developments: We have previously used systematic review and meta-analysis to inform drug selection for the MS-SMART trial [2]. By automating key steps in the process (Figure) we will provide utterly contemporary information to inform drug selection for clinical trials.

Conclusions: SyRF - which currently hosts 220 projects, 390 users, 630,126 uploaded publications and 299,601 screening decisions - can dramatically reduce the time taken in systematic review and meta-analysis. Tools under development will facilitate living systematic reviews, provide curated current content for research areas and will help identify and rank new drugs for further investigation.

References

(1) Contopoulos-Ioannidis DG et al. (2008). Science 321:1298-9.

(2) Vesterinen HM et al. (2015). PloS One 10(4):e0117705.

Table: Performance of risk of bias assessment using Regex. n= number of publications.