Wikipedia:Identifying reliable sources (medicine)
 | This page documents an English Wikipedia content guideline. Editors should generally follow it, though exceptions may apply. Substantive edits to this page should reflect consensus. |
 | This page in a nutshell: Ideal sources for biomedical material include systematic and literature reviews in reliable, third-party, published secondary sources (such as reputable medical journals), recognised, standard medical textbooks, or medical guidelines and position statements from reputable national or international expert bodies. Cite reviews, don't write them. |
Biomedical information must be based on reliable, third-party published secondary sources, and must accurately reflect current knowledge. This guideline supports the general sourcing policy with specific attention to what is appropriate for medical content in any Wikipedia articles, including those on alternative medicine. Sourcing for all other types of content – including non-medical information in medical articles – is covered by the general guideline on identifying reliable sources.
Ideal sources for biomedical information include: review articles (especially systematic reviews) published in reputable medical journals, academic and professional books written by experts in the relevant fields and from respected publishers, and guidelines or position statements from national or international expert bodies. Primary sources should generally not be used for medical content, as such sources often include unreliable or preliminary information; for example, early lab results that do not hold in later clinical trials.
See the reliable sources noticeboard for questions about the reliability of specific sources, and feel free to ask at WikiProjects such as WikiProject Medicine and WikiProject Pharmacology.
Definitions
[edit]Types of sources
[edit]In the biomedical literature:
- A primary source is one in which the authors directly participated in the research and documented their personal experiences. They examined the patients, injected the rats, ran the experiments, or supervised those who did. Many papers published in medical journals are primary sources for facts about the research and discoveries made.
- A secondary source summarizes one or more primary or secondary sources to provide an overview of current understanding of the topic, to make recommendations, or to combine results of several studies. Examples include literature reviews or systematic reviews found in medical journals, specialist academic or professional books, and medical guidelines or position statements published by major health organizations.
- A tertiary source summarizes a range of secondary sources. Undergraduate- or graduate-level textbooks, edited scientific books, lay scientific books, and encyclopedias are tertiary sources.
Biomedical v. general information
[edit]
Biomedical information requires sourcing that complies with this guideline, whereas general information in the same article may not.
For example, an article on Dr Foster's Magic Purple Pills could contain both biomedical and non-biomedical claims:
- Dr Foster's pills cure everything. A biomedical claim! Strong MEDRS (MEDical Reliable Source) sourcing is definitely required here (see WP:MEDASSESS).
- The pills were invented by Dr Archibald Foster and released onto the market in 2015. This is not biomedical information, and it only requires ordinary RS.
- They are purple and triangular, packaged one to a box,[citation needed] as no-one ever manages to swallow a second one.[medical citation needed] Note the first "citation needed" is for general information, while the second is for biomedical information (a.k.a. MEDRS).
Basic advice
[edit]Avoid primary sources
[edit]Per the Wikipedia policies of neutral point of view, no original research, and verifiability, articles need to be based on reliable, independent, published secondary or tertiary sources. For biomedical content, the Wikipedia community relies on guidance contained in expert scientific reviews and textbooks, and in official statements published by major medical and scientific bodies. Note that health-related content in the general news media should not normally be used to source biomedical content in Wikipedia articles. (News sources may be useful for non-biomedical content, such as information about "society and culture" – see WP:MEDPOP.)
Primary sources should NOT normally be used as a basis for biomedical content. This is because primary biomedical literature is exploratory and often not reliable (any given primary source may be contradicted by another). Any text that relies on primary sources should usually have minimal weight, only describe conclusions made by the source, and describe these findings so clearly that any editor can check the sourcing without the need for specialist knowledge. Primary sources should never be cited in support of a conclusion that is not clearly made by the authors (see WP:Synthesis).
Respect secondary sources
[edit]Primary sources should not be cited with the intent of "debunking", contradicting, or countering conclusions made by secondary sources. Synthesis of published material advancing a position is original research, and Wikipedia is not a venue for open research. Controversies or uncertainties in medicine should be supported by reliable secondary sources describing the varying viewpoints. Primary sources should not be aggregated or presented without context in order to undermine proportionate representation of opinion in a field. If material can be supported by either primary or secondary sources, the secondary sources should be used. Primary sources may be presented together with secondary sources.
Findings are often touted in the popular press as soon as primary research is reported, before the scientific community has analyzed and commented on the results. Therefore, such sources should generally be omitted (see recentism). Wait for reliable secondary sources (not press releases or newspaper articles based on such sources). Resist the temptation to add content based solely on new primary sources especially when they report surprising results. For example, this content should not have been added to the encyclopedia:
"A large study published in 2010 found that selenium and Vitamin E supplements, separately as well as together, did not decrease the risk of getting prostate cancer and that vitamin E may increase the risk; they were previously thought to prevent prostate cancer." (citing PMID 20924966)
because later work failed to confirm the surprising results. Later the appropriate content could be added:
"Neither vitamin E nor selenium decreases the risk of prostate cancer and vitamin E may increase it." (citing PMID 29376219, PMID 26957512)
Wikipedia role is to focus on knowledge that is verifiable, not to present the newest results. Given time, confirmations or a review will be published and those secondary sources allow facts to be stated reliably.
A reason to avoid primary sources in the biomedical field – especially papers reporting results of in vitro experiments – is that they are often not replicable[1][2][3] (see also replication crisis) and are therefore unsuitable for use in generating encyclopedic, reliable biomedical content. Scientists at Bayer reported in 2011 that they were able to replicate results in only ~20 to 25% of prominent studies they examined;[4] scientists from Amgen followed with a 2012 publication showing that they were only able to replicate 6 (11%) of 53 high-impact publications and called for higher standards in scientific publishing.[5] Further, the fact that a claim is published in a refereed journal need not make it true. Even well-designed randomized experiments will occasionally produce spurious results. Experiments and studies can produce flawed results or even fall victim to deliberate fraud (e.g. the Retracted article on dopaminergic neurotoxicity of MDMA and the Schön scandal.)
Summarize scientific consensus
[edit]Scientific journals are the best place to find both primary and secondary sources. Every rigorous scientific journal is peer reviewed. Be careful of material published in journals lacking peer review or that report material mainly in other fields. Be careful of material published in disreputable journals or disreputable fields.
Wikipedia policies on the neutral point of view and not publishing original research demand that we present prevailing medical or scientific consensus, which can be found in recent, authoritative review articles, in statements and practice guidelines issued by major professional medical or scientific societies (for example, the European Society of Cardiology or the Infectious Disease Society of America) and widely respected governmental and quasi-governmental health authorities (for example, AHRQ, USPSTF, NICE, and WHO), in textbooks, or in scholarly monographs. Although significant-minority views are welcome in Wikipedia, such views must be presented in the context of their acceptance by experts in the field. Additionally, the views of tiny minorities need not be reported.
Finally, make readers aware of controversies that are stated in reliable sources. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers.
Assess evidence quality
[edit]When writing about any health effect, assessing evidence quality helps distinguish between minor and major views, determine due weight, and identify accepted evidence-based information. Even in reputable medical journals, different papers are not given equal weight. Studies can be categorized into levels in a hierarchy of evidence,[6] and editors should rely on high-level evidence, such as systematic reviews. Low-level evidence (such as case reports or series) or non-evidence (such as anecdotes or conventional wisdom) is avoided. Medical guidelines or position statements by internationally or nationally recognized expert bodies also often contain recommendations, along with assessments of underlying evidence (see WP:MEDORG).
Left: Procter & Gamble.[7] Right: Canadian Association of Pharmacy in Oncology.[8]
The best evidence for efficacy of treatments and other health interventions comes mainly from meta-analyses of randomized controlled trials (RCTs).[9] Systematic reviews of literature that include non-randomized studies are less reliable.[10] Narrative reviews can help establish the context of evidence quality.
Lower levels of evidence in medical research come from primary studies (see WP:MEDDEF). Roughly in descending order, these include: individual RCTs; quasi-experimental studies; prospective observational (non-experimental) studies, such as prospective cohort studies (one type of longitudinal study); case control studies; cross-sectional studies (surveys), and other correlation studies such as ecological studies; other retrospective analyses (including retrospective cohort studies); and non-evidence-based expert opinion or clinical experience. Case reports and series are especially avoided, as they are uncontrolled.
Speculative proposals and early-stage research should not be cited to imply wide acceptance. For example, results of an early-stage clinical trial would not be appropriate in the 'Treatment' section of a disease article because future treatments have little bearing on current practice. The results might – in some cases – be appropriate for inclusion in an article specifically dedicated to the treatment in question or to the researchers or businesses involved in it. Such information, particularly when citing secondary sources, may be appropriate in research sections of disease articles. To prevent misunderstanding, the text should clearly identify the level of research cited (e.g., "first-in-human safety testing").
Several formal systems exist for assessing the quality of available evidence on medical subjects.[11][12] Here, "assess evidence quality" essentially means editors should determine the appropriate type of source and quality of publication. Respect the levels of evidence: Do not reject a higher-level source (e.g., a meta-analysis) in favor of a lower one (e.g., any primary source) because of personal objections to the inclusion criteria, references, funding sources, or conclusions in the higher-level source. Editors should not perform a detailed academic peer review.
Avoid over-emphasizing single studies, particularly in vitro or animal studies
[edit]In vitro studies and animal models serve a central role in research, and are invaluable in determining mechanistic pathways and generating hypotheses. However, in vitro and animal-model findings do not translate consistently into clinical effects in human beings. Where in vitro and animal-model data are cited on Wikipedia, it should be clear to the reader that the data are pre-clinical, and the article text should avoid stating or implying that reported findings hold true in humans. The level of support for a hypothesis should be evident to a reader.
Using small-scale, single studies makes for weak evidence, and allows for cherry picking of data. Studies cited or mentioned in Wikipedia should be put in context by using high-quality secondary sources rather than by using the primary sources.
Use up-to-date evidence
[edit]Keeping an article up-to-date while maintaining the more important goal of reliability is important. These instructions are appropriate for actively researched areas with many primary sources and several reviews, and may need to be relaxed in areas where little progress is being made or where few reviews are published.
- In many topics, a review that was conducted more than five or so years ago will have been superseded by more up-to-date ones, and editors should try to find those newer sources to determine whether the expert opinion has changed since the older sources were written. The range of reviews you examine should be wide enough to catch at least one full review cycle, containing newer reviews written and published in the light of older ones and of more recent primary studies.
- Assessing reviews may be difficult. While the most recent reviews include later research results, this does not automatically give more weight to the most recent review (see recentism).
- Prefer recent reviews to older primary sources on the same topic. If recent reviews do not mention an older primary source, the older source is dubious. Conversely, an older primary source that is seminal, replicated, and often-cited may be mentioned in the main text in a context established by reviews. For instance, the article Genetics could mention Darwin's 1859 book On the Origin of Species as part of a discussion supported by recent reviews.
There are exceptions to these rules of thumb:
- History sections often cite older work.
- Cochrane Library reviews and NICE guidelines are generally of high quality and are periodically re-examined even if their initial publication dates fall outside the 5-year window.
- A newer source that is of lower quality does not supersede an older source of higher quality.
Use independent sources
[edit]Many treatments or proposed treatments lack good research into their efficacy and safety. In such cases, reliable sources may be difficult to find, while unreliable sources are readily available. When writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of marginal ideas may be used to describe personal opinions, but extreme care should be taken when using such sources lest more controversial opinions be taken at face value or, worse, asserted as fact. If independent sources discussing a medical subject are of low quality, then it is likely that the subject itself is not notable enough to have its own article or relevant for mention in other articles.
Symposia and supplements to academic journals are commonly sponsored by industry groups with a financial interest in the outcome of the research reported. They may lack independent editorial oversight and peer review with no supervision of content by the parent journal.[13] Such articles do not share the reliability of their parent journal.[14] Indications that an article was published in a supplement may be fairly subtle; for instance, a letter "s" added to a page number,[15] or "Suppl." in a reference.
Bias
[edit]Personal conflicts of interest
[edit].svg)
Use your best judgement when writing about topics where you may have a conflict of interest: citing yourself on Wikipedia is problematic. Citing your own organization, such as a governmental health agency or an NGO producing high-quality systematic reviews, is generally acceptable – if the conflict of interest is disclosed, it is done to improve coverage of a topic, and not with the sole purpose of driving traffic to your site. All edits should improve neutral encyclopedic coverage; anything else, such as promoting an organization, is not allowed.
According to the conflict of interest guideline – conflicts of interest (COI) must be disclosed. Editing on topics where one is involved or closely related, especially when there is potential financial gain, is discouraged. Medicine is not an exception. One way to contribute with a COI is to post on talk-pages, suggesting edits. Another alternative is the articles for creation pathway.
These methods are often best when writing about oneself, one's organization or company – but may be less so when there is a potential conflict of interest in a research field. For example, one may legitimately be an authority on a certain topic – a volunteer who reads the talk-page will not always have the knowledge to assess the sources properly. Then it is better to follow ordinary editing protocol, disclosing any COI and to be careful not to overemphasize your own sources.
Choosing sources
[edit]