In science and drugs, understanding the historic context and cornerstone findings of a discipline or subject is essential to creating new hypotheses and unlocking subsequent degree insights for scientific analysis to tell strategic planning akin to pipeline improvement. The main strategy within the scientific arsenal to discover previous and present state data is a scientific literature overview, nonetheless with the appearance of the web, and the overwhelming growth of for-profit-journals plus various publication strategies it has change into troublesome to determine the optimum inclusion standards for an impactful overview.
In a world migrating in direction of data-driven choices, it’s essential to have ‘good’ knowledge supporting these choices. Now, greater than ever, understanding “what ‘good’ means in science and knowledge, is an crucial query. This text seeks to judge the utility of generally used methods for selecting which publications to include right into a literature overview and supply steering on the very best practices for assessing publications and their knowledge for the aim of a literature overview.
Literature critiques: An summary
A literature overview is a compilation of key printed info relating to a particular subject, usually centered inside a particular time frame. Dissemination of data within the type of literature critiques is a pivotal a part of the scientific course of—it permits new researchers the power to rapidly acquire a high-level understanding of a discipline, and compiles and condenses doubtlessly a long time of analysis into an simply digestible format. For all times sciences firms (e.g., medical trial sponsors, CROS, and so forth.) literature critiques serve two predominant functions: 1) to ‘get a leg up’ on the competition by understanding current and potential future state of a discipline, and a pair of) to assist strategic choices inside pipeline improvement.
Systematic literature critiques are a particular kind of overview that makes use of a set of outlined and reproducible ‘guidelines’ developed previous to enterprise the overview to make sure a rigorous methodology and to attenuate bias. One strategy that has been used to this point to find out the worth of a publication is the calculated impression issue.
The issue with the Impression Issue
The Impression Issue was initially developed by Eugene Garfield on the Institute for Scientific Data (ISI). Conceptualized as a technique to monitor the citations of particular person publications, Garfield argued that the index would make it simpler for scientists to search out related articles and assess how the scientific neighborhood at giant acquired numerous publications.1 Impression Issue, as compiled at the moment by Clarivate Analytics, is the typical of the sum of the citations acquired in a given yr to a journal’s earlier two years of publications, divided by the sum of “citable” publications within the earlier two years.
In essence, if a journal printed papers that have been then cited by many different publications, the Impression Issue can be larger. Whereas Garfield’s intentions have been good—creating a scientific strategy to evaluate the inherent utility of printed works—there have been unintended penalties to Impression Issue recognition. The scientific neighborhood started to uphold the Impression Issue of a journal because the ‘gold normal’ metric when assessing the standard of analysis and researchers alike.
The heavy reliance on Impression Issue has drawn criticism through the years for a lot of causes, although essentially the most salient points are as follows:
Impression Issue calculation is time constrained and restricted
The Impression Issue calculation solely accounts for quotation frequency of articles inside the first two years submit publication.2 Fields and journals that generate articles that get cited over an extended time frame relatively than instantly are at a critical drawback based mostly on the two-year time restrict.
Article kind biases the calculation
There are a number of varieties of printed articles akin to overview articles and letters to the editor that aren’t accounted for in a journal’s whole article rely however do rely for the journal’s quotation common. If a journal tends to publish quite a lot of a lot of these articles, it could dramatically improve the Impression Issue.2
Faster to publish = higher Impression Issue
It’s maybe not shocking that some fields can generate giant datasets quicker than others. This will give researchers publishing in these fields a synthetic enhance within the variety of publications they will generate in a brief period of time. For example, researchers in computational modeling could possibly generate one giant knowledge set that generates a number of journal articles in fast succession. These researchers can be prone to publish in the identical journals and cite their very own earlier works together with the works of others within the discipline.
In the meantime, medical journals might have low quotation counts partly, as a result of the meant viewers will not be usually publishing prolifically, and since medical analysis timelines are typically longer, which implies we regularly see years between publications.
This turnaround time bias places many journals catering to particular fields at a drawback compared to fields which have larger quotation counts. The excellent news right here is that Clarivate Analytics is actively attempting to mitigate this explicit subject with the 2021 launch of the brand new Journal Quotation Indicator (JCI), meant to account for the totally different charges of publication and citations throughout fields, nonetheless critics of the Impression Issue preserve that this new strategy has its personal potential points and might be misused.3
The peer overview course of can inflate Impression Issue
Anybody who has gone by means of the peer-review course of for their very own printed work can inform you that the peer reviewers usually basically form the top product previous to publication. In some circumstances, reviewers might request sure citations be included into an article earlier than it may be printed. Researchers usually put on many hats of their careers. A reviewer may be a journal editor, or the creator of a advisable paper, which is an inherent battle of curiosity.
Impression Issue excludes the humanities
Whereas fields akin to mobile biology and physics are inclined to disseminate new info nearly solely by means of journal articles, social science fields akin to sociology and historical past are extra probably to make use of various dissemination methods. Books, wikis, and the like are fully excluded from Impression Issue calculations, which negatively impression humanities particularly.
Whereas the Impression Issue was a good suggestion in principle, the world of science has change into too huge and our publication methodologies too various to rely upon this singular metric in judging good science.
Whole quotation rely paints an incomplete image
A subset of the Impression Issue is the variety of citations a person publication might accumulate over time. Many researchers might take into account this a treasured metric of the utility of printed findings. The thought course of is that if many different researchers discover the work necessary sufficient to warrant a quotation, then it should be a pivotal work, in principle making it ‘good science.’
Sadly, this assumption reductions the human factor inherent in journal-based publication. Well-known or in style researchers will usually be cited much more usually than others. Papers that characteristic a ‘sizzling subject’ or new and thrilling methodology could also be included merely to capitalize on public curiosity. Pals and colleagues might cite one another as a courtesy. The human factor makes it particularly important to evaluate the analysis thoughtfully and rigorously.
Preliminary steps for systematic literature overview course of improvement
To conduct a productive systematic literature overview that considers extra than simply the Impression Issue or frequency an article has been cited, it is very important start with a transparent understanding of why the overview is required, such because the wants of a possible viewers or what knowledge is want for a pending resolution, and to have a well-structured analysis query formulated to start the method. Typically essentially the most difficult process of a literature overview is figuring out the overview’s focus and making a well-defined query the analysis goals to reply. This is a crucial step that may information many elements of the overview course of and streamline an often-overwhelming course of.
Whereas there are a number of methods for creating a transparent and centered analysis query, in medical analysis and improvement utilizing the PICOTS (Affected person Inhabitants, Intervention, Comparator, End result, Timing/Setting, Research Design/Research Traits) framework is useful.4-6 Every factor within the PICOTS framework focuses the query, finally yielding essentially the most relevant search outcomes. Moreover, well-defined PICOTS scale back the danger of potential bias. Widespread examples of bias in analysis embody choice bias, interview bias, recall bias, quotation bias, and efficiency bias. Widespread bias mitigation methods in analysis embody blinding, randomization, and elevated pattern sizes.5
After the query is clearly outlined, it’s essential to understand how and the place to seek for info. First the well-defined analysis query must be translated into key phrases (Trace: phrases utilized in your PICOTS framework work properly!).6 Having descriptive and related key phrases and phrases is the cornerstone to an efficient search and utilizing synonyms and various textual content usually elicits extra materials.
Subsequent it’s necessary to set some search boundaries. That is useful as a result of setting boundaries earlier than starting analysis helps to ascertain inclusion and exclusion standards for a literature overview. Growing inclusion and exclusion standards will assist decide the related info for the overview outcomes. Some researchers recommend that every PICOTS element has inclusion and exclusion standards to cut back potential bias. Normal inclusion and exclusion standards embody language, date, publicity of curiosity, geographic location, individuals, peer overview, reported consequence, setting, examine design, and publication kind.7-9
As soon as the standards are set, it’s time to begin looking. There are web-based search engines like google like PubMed or Google Scholar and quite a few databases out there for literature searches, together with evidence-based databases for built-in info out there as systematic critiques and databases for articles initially printed in peer-reviewed journals. On the identical time, it’s important to be as complete as doable; understanding the place to go and the place to not go for info is important. As nobody database can discover all of the literature in a discipline, a number of totally different databases ought to be utilized to supply essentially the most complete and balanced overview.6,9
The Nationwide Institutes of Well being (NIH) recommends that at a minimal, PubMed, or Medline, Embase, and the Cochrane Central Trials Registry must be searched.9 Having an inventory of journals and databases related to your analysis questions helps maintain the method organized and transferring ahead.
Greatest practices in assessing publications
Extracting info from analysis papers might be very tedious and time-consuming and requires steering on the best way to strategy and break down a analysis paper to effectively and successfully collect and document info. It’s pertinent to take notes and doc the method and progress whereas executing analysis. Beneath is a recommended methodology for the identification and validation of article choice for literature critiques.
1. Learn the title and summary if out there
Usually, the summary would have already been reviewed within the technique of deciding whether or not or not the article meets the literature overview inclusion and exclusion standards. Nevertheless, if the summary has not been reviewed but, it is very important learn the summary, if out there, to acquire a high-level overview of why and the way the analysis was carried out and the end result of that analysis. Between the title, summary and any listed key phrases the authors have chosen to supply, you must be capable to decide whether or not a publication could also be an excellent match to your overview.
We’d advocate skipping the introduction part when figuring out whether or not a publication ought to be included in a literature overview. The introduction of a analysis paper usually contains an outline of the analysis subject below investigation and why the analysis was carried out and will summarize prior analysis nonetheless, it should hardly ever give you evidentiary assist for the way properly the present analysis was carried out or how related the work is to your overview subject.
2. Briefly look at the strategies
At this level in evaluation of the article below overview, there’s a basic understanding of the subject of investigation, why and the way the analysis was carried out, and the implications of the findings. Whereas studying this part, it is very important search for crimson flags that point out poor high quality of analysis together with incomplete or unclear strategies, small pattern sizes, lacking description of the statistical evaluation carried out. Ask your self: Do I perceive what experiment or intervention was carried out sufficient to do the identical factor? The strategies part of a analysis paper ought to go into sufficient element to permit one other researcher to repeat the analysis. If the reply is not any, or unclear take into account excluding the publication out of your overview.
3. Overview tables and figures
Information visualization can present readers with basic understanding of what was measured throughout the investigation with out studying by means of a number of paragraphs. If the strategies seem like sound, the subsequent step is to have a look at the info representations within the article, together with tables and figures. Tables and figures usually comprise bulk knowledge with out the bias of interpretation from the investigators.
You will need to take into account whether or not the figures and tables match up with the strategies used—all strategies used ought to be mirrored someplace within the figures of the publication. As properly, figures ought to at all times mirror the statistical evaluation that has been accomplished, with error bars, p-values, or asterisks denoting significance within the footnotes. Overview all figures earlier than studying the researcher interpretation of the info to get an unbiased image of the outcomes. Solely after getting a strong understanding of the info must you transfer on to studying by means of the outcomes part to see in case your takeaways align with the authors.
4. Scan the outcomes part
The outcomes part ought to describe the info as it’s displayed within the figures and tables and the statistical evaluation used to interpret the info into findings. Learn the outcomes part to fill in any gaps in understanding high-level particulars of the analysis paper, and to find out in case your understanding of the info aligns with that of the authors. In case your interpretation of the info doesn’t match the authors, however you may have ample subject material understanding of the methodology utilized, it’s possible you’ll take into account not together with the article in your overview.
5. Assess the conclusion/dialogue
The conclusion, usually referred to as dialogue in a analysis paper, summarizes the outcomes of the analysis, discusses limitations, describes how the analysis was carried out, and the way the findings develop, oppose, or validate present data of the subject in investigation. Often the dialogue may even embody the implications of the findings, as properly. You will need to observe any limitations of the analysis described on this part as they can provide perception into the standard of the analysis. If as an illustration, a medical examine had began out with a various affected person inhabitants, however a lot of the sufferers who accomplished the examine have been white, non- Hispanic the dialogue part ought to denote this as a possible limitation to decoding the outcomes broadly. This is able to not be a ‘deal breaker’ for inclusion into your overview if the article meets your different standards, but it surely ought to be considered and doubtlessly famous in your overview.
6. Examine the citations/references
Lastly, overview the citations/references and take note of the references used within the introduction. References utilized in totally different sections of a publication usually signify various things to the reader. For example, references used within the introduction can provide a baseline for evaluating findings of the present analysis with historic analysis. The references/citations part generally is a good supply to search out extra articles to incorporate in your literature overview that you will have missed in your preliminary key phrase search.
After getting accomplished these 5 steps and finalized your quick record of publications that meet your literature overview standards, it’s essential to return and skim the papers of their entirety to make sure full comprehension. Taking notes relating to the important thing findings, limitations and implications of every article will give you an incredible place to begin to stipulate your overview.
Conclusion
Literature critiques are a pivotal a part of the scientific course of, permitting researchers the power to rapidly acquire a high-level understanding of a discipline, and life science companies the knowledge to make strategic, data-driven choices. Whereas there are ‘guidelines’ to comply with to make sure a rigorous methodology and minimized bias happens throughout the technique of conducting (or authoring) a literature overview, the background info, steps offered for making a well-structured analysis query, and methodology to successfully look at publications can help with adherence to the ‘guidelines’ and supply a straight-forward, structured strategy to the literature overview.
Emily Moyers, Useful resource Supervisor, April Purcell, Guide, Medical Operations, and Cortney Miller, Senior Supervisor, Consulting Operations; all with Halloran Consulting Group
References
- Baldwin, Melinda (2017) ‘Origins of the journal impression issue’(https://physicstoday.scitation.org/do/10.1063/PT.5.9082/full/)
- Grossman, Alexander. ‘Journal Impression Elements- Time to say goodbye?’ 2017 Sep 29. Revealed on Scienceopen.com. https://blog.scienceopen.com/2013/12/journal-impact-factors-time-to-say-goodbye/
- Ogrady, Cathleen. ‘Journal impression issue will get a sibling that adjusts for scientific discipline’ 2021 Jun 30 doi: 10.1126/science.abk2273 (https://www.science.org/content/article/journal-impact-factor-gets-sibling-adjusts-scientific-field)
- AHRQ. Utilizing the PICOTS Framework to Strengthen Proof Gathered in Medical Trials—Steering from the AHRQ’s Proof-based Observe Facilities Program (2022). (https://www.fda.gov/media/109448/download)
- McDonagh M, Peterson Ok, Raina P, et al. Avoiding Bias in Choosing Research. 2013 Feb 20. In: Strategies Information for Effectiveness and Comparative Effectiveness Evaluations [Internet]. Rockville (MD): Company for Healthcare Analysis and High quality (US); 2008-. Desk 2, Hypothetical examples of potential for bias based mostly on inadequately outlined PICOTs. Obtainable from: https://www.ncbi.nlm.nih.gov/books/NBK126701/table/cerguideavbias.t2/?
- Grewal A, Kataria H, Dhawan I. Literature seek for analysis planning and identification of analysis drawback. Indian J Anaesth. 2016 Sep;60(9):635-639. doi: 10.4103/0019-5049.190618. PMID: 27729689; PMCID: PMC5037943
- Hulley, Stephen B., ed. Designing medical analysis. Lippincott Williams & Wilkins, 2007
- The College of Melbourne. Library Guides: Systematic Evaluations: 5: Inclusion and exclusion standards. (2022). (https://unimelb.libguides.com/sysrev/inclusion-exclusion-criteria)
- Xiao Y, Watson M. Steering on Conducting a Systematic Literature Overview. Journal of Planning Schooling and Analysis. 2019;39(1):93-112. (https://journals.sagepub.com/doi/full/10.1177/0739456X17723971)
