Esta referencia está disponible actualmente en inglés. La versión en español está en preparación.

Risk of Bias Assessment: RoB 2 and ROBINS-I

Por Benn Bluestein-Veyra, M.Sc. en Química Orgánica · Actualizado el 21 de octubre de 2026

A risk of bias assessment judges whether flaws in the design, conduct, analysis or reporting of a study could push its result above or below the true effect. Cochrane's RoB 2 tool covers randomized trials and ROBINS-I covers non-randomized studies of interventions, and both work domain by domain.^[1][2][3]

Systematic reviewers assess each included study before they pool results, and the judgments then shape the GRADE rating of certainty for the body of evidence. This reference covers the tools, their signaling questions, the plots that display them and the empirical studies behind them.^[4][5]

Distinguishing risk of bias from imprecision and reporting quality

Higgins and colleagues define internal validity as the extent to which a study is free from bias, a systematic error. They separate it from precision, the freedom from random error, which depends on the number of participants and events.^[1]

A small trial at low risk of bias can give an imprecise result with a wide confidence interval, and a large trial can give a precise result that is still biased (Fig. 1).^[1] Adding participants narrows the interval, while the design and conduct of the study set where the estimate is centered.^[1]

Reporting quality describes how completely a paper documents its methods. The CONSORT 2025 statement sets a 30-item checklist and a participant flow diagram for reports of randomized trials, so readers can see how a trial was run.^[6]

A sound study can be reported sparsely, which leaves assessors without the facts a judgment needs. Lo and colleagues found low agreement between reviewers' and study authors' Newcastle-Ottawa assessments, and suggested asking authors for methodological details missing from publications.^[7]

Quality scales blend these properties into one number. Higgins and colleagues set scales aside because they combine reporting with conduct and weight items in ways that are hard to justify, and their associations with effect estimates have been inconsistent.^[1] Moher and colleagues found that the way quality scores entered a meta-analysis changed the interpretation of the pooled benefit.^[8]

Understanding the original Cochrane tool and why it was revised

The Cochrane Collaboration published its tool for assessing risk of bias in randomized trials in 2011, noting that flaws in design, conduct, analysis and reporting can make an intervention effect look larger or smaller than it is.^[1] The tool covered six domains: selection, performance, detection, attrition and reporting bias, plus other bias.^[1]

Each item had two parts. A support for judgment described the relevant trial feature, ideally with verbatim quotes, and a judgment rated the item at low, high or unclear risk of bias.^[1] Assessments were made for each main outcome or class of outcomes.^[1]

Sterne and colleagues released RoB 2 in 2019 to reflect how understanding of the causes of bias had advanced.^[2] The main changes from the first version are these:

• Each assessment targets a single numerical result from a trial, and a trial contributing several results may need several assessments.^[2]
• Domain names describe the issue they address, replacing labels such as selection bias and performance bias.^[2]
• Signaling questions and algorithms guide each domain judgment.^[2]
• The tool separates the effect of assignment to an intervention from the effect of adhering to it.^[2]
• The domain set is fixed, and an explicit process yields the overall judgment for the result.^[2]

Applying RoB 2 domain by domain to a randomized trial

RoB 2 has five domains, each tied to a stage of the trial where bias can enter a result (Fig. 2).^[2] The issues each domain addresses are summarized below.

• Bias arising from the randomization process: whether the allocation sequence was random and concealed, and whether baseline differences between groups suggest a problem.^[2]
• Bias due to deviations from intended interventions: whether participants and the people delivering interventions knew the assignment, and whether the analysis suited the effect of interest.^[2]
• Bias due to missing outcome data: whether data were available for nearly all randomized participants, and whether missingness could depend on the true value of the outcome.^[2]
• Bias in measurement of the outcome: whether the method was appropriate, whether it could differ between groups, and whether assessors knew the intervention received.^[2]
• Bias in selection of the reported result: whether the analysis followed a plan finalized before unblinded outcome data were available, and whether the result was chosen from several measurements or analyses.^[2]

Signaling questions lead to a proposed judgment

Each domain opens with signaling questions answered yes, probably yes, probably no, no or no information. Yes and probably yes carry the same implication for bias, as do no and probably no.^[2]

An algorithm maps the answers to a proposed domain judgment of low risk of bias, some concerns or high risk of bias. Assessors justify each answer in free text, may override the proposal with a reason, and can record the likely direction of bias.^[2]

The overall judgment for a result therefore matches the least favorable domain, with room to move to high when several domains raise some concerns together.^[2]

The effect of interest shapes the deviations domain

For the effect of assignment, often called the intention-to-treat effect, assessors ask whether deviations arose from the trial context and whether an appropriate analysis estimated that effect. For the effect of adhering, they ask about non-protocol interventions, implementation failures and adherence.^[2]

Regulators frame the same choice through estimands. The FDA guidance E9(R1), issued in May 2021, presents a structured framework for agreeing on the effect a clinical trial sets out to estimate.^[9]

Calibration supports consistent judgments

Minozzi and colleagues had four raters apply RoB 2 to 70 trial results. Agreement was moderate for the randomization domain and slight for the overall judgment, and the tool took a mean of 28 minutes per outcome.^[10]

The authors described RoB 2 as detailed, comprehensive and demanding to apply, even for raters with substantial expertise in systematic reviews, and recommended calibration exercises and intensive training before its application to improve reliability.^[10]

Assessing non-randomized studies with ROBINS-I

ROBINS-I, published by Sterne and colleagues in 2016, assesses risk of bias in estimates of comparative effectiveness from studies that did not use randomization to allocate individuals or clusters.^[3] It reads each study as an attempt to emulate a target trial, a hypothetical pragmatic randomized trial in the same participant group with no features putting it at risk of bias.^[3]

Its seven domains follow the timeline of the intervention (Fig. 3). Two address issues before the intervention starts, one addresses classification of the interventions themselves, and four address issues after the start.^[3]

• Confounding: a prognostic variable that predicts the outcome also predicts the intervention received at baseline.^[3]
• Selection of participants into the study: exclusion of eligible participants, or their start of follow-up, is related to both intervention and outcome.^[3]
• Classification of interventions: intervention status is recorded with error.^[3]
• Deviations from intended interventions, missing data, measurement of outcomes and selection of the reported result, which parallel the RoB 2 domains.^[3]

Signaling questions take the same five responses as RoB 2, and judgments run from low to moderate, serious and critical risk of bias. Low risk corresponds to the risk of bias in a high-quality randomized trial, and the authors expect a non-randomized study to reach low risk for confounding only exceptionally.^[3]

ROBINS-I also addresses time-varying confounding, which arises when participants switch between the interventions being compared or when prognostic factors after baseline affect the intervention received.^[3]

ROBINS-E extends the approach to exposures

Higgins and colleagues published ROBINS-E in 2024 for cohort studies of the causal effect of an exposure on an outcome, such as environmental, occupational or behavioral exposures. It also works through seven domains of signaling questions toward an overall judgment and a predicted direction of bias.^[11]

Matching the risk of bias tool to the study design

Diagnostic accuracy studies, animal experiments and observational cohorts each have a dedicated tool built on the same domain logic.^[7][12][13] The table pairs each tool with its design.

Whiting and colleagues developed QUADAS-2 for diagnostic accuracy studies. Its four domains cover how participants were selected, the index test, the reference standard, and flow and timing, and the first three also receive judgments of applicability.^[12]

Hooijmans and colleagues adapted the Cochrane tool for animal intervention studies as SYRCLE's risk of bias tool. Its 10 entries cover selection, performance, detection, attrition, reporting and other biases, and half of them match items in the Cochrane tool.^[13]

The Newcastle-Ottawa scale awards cohort and case-control studies up to nine points across selection of study groups, comparability of groups and ascertainment of exposure or outcome. Lo and colleagues found slight to poor agreement on its items between reviewers and study authors.^[7]

Presenting judgments in traffic-light and summary plots

The same judgments can be shown two ways: a grid with one row per study and one colored cell per domain, and a bar summary of the share of studies at each judgment in every domain (Fig. 4). McGuinness and Higgins built robvis, an R package and Shiny web app that turns an assessment table into publication-quality risk of bias figures.^[14]

The PRISMA 2020 statement guides how systematic reviews report the way they identify, select, appraise and synthesize studies, through a 27-item checklist and revised flow diagrams. Risk of bias methods and results belong to that appraisal step.^[5]

A plot summarizes, and the written support behind each judgment carries the reasoning. RoB 2 asks for free-text justification of every answer and judgment, so readers can trace how each rating was reached.^[2]

Meta-epidemiological studies show design features shifting effect estimates

Meta-epidemiology compares trials with and without a design feature across many meta-analyses, often as a ratio of odds ratios. In the analysis by Wood and colleagues, a ratio below 1 means that trials lacking the feature reported larger intervention effects.^[15]

Schulz and colleagues assessed 250 controlled trials from 33 meta-analyses in 1995. Odds ratios were exaggerated by 41% in trials with inadequate allocation concealment and by 17% in trials without double blinding.^[16]

Wood and colleagues combined 146 meta-analyses including 1,346 trials. For subjective outcomes, estimates were exaggerated when allocation concealment was inadequate or unclear or blinding was absent, with little evidence of bias for objective outcomes.^[15]

Savović and colleagues pooled 1,973 trials and estimated that lack of double blinding exaggerated intervention effects by an average of 13%, driven mainly by trials with subjectively assessed outcomes.^[17] The ROBES study of 2,443 trials in Cochrane reviews found exaggerated estimates in trials at high or unclear risk for sequence generation, allocation concealment and blinding.^[18]

Page and colleagues reviewed 24 meta-epidemiological studies and reported exaggerated estimates with inadequate or unclear sequence generation and allocation concealment, with the average bias largest for subjective outcomes.^[19]

Measurement and reporting carry the same pattern

Hróbjartsson and colleagues studied trials that assessed the same binary outcome with both blinded and non-blinded assessors. Non-blinded assessment exaggerated odds ratios by 36% on average, driven by the misclassification of a median 3% of assessed participants.^[20]

Chan and colleagues compared trial protocols with their publications and found that 62% of trials had at least one primary outcome changed, introduced or omitted.^[21] Kirkham and colleagues found that a third of Cochrane reviews contained at least one trial with high suspicion of outcome reporting bias for the review's primary outcome.^[22]

Registered outcomes give assessors a fixed point of comparison for the reported result domain. For applicable clinical trials, 42 CFR Part 11 requires a ClinicalTrials.gov registration to describe each primary outcome measure with its metric and the time points at which it is assessed.^[23]

Carrying risk of bias assessment into GRADE certainty

GRADE rates certainty for a body of evidence at four levels: high, moderate, low and very low. Randomized trials start high and observational studies start low, and risk of bias sits beside imprecision, inconsistency, indirectness and publication bias as reasons to rate down.^[24]

Guyatt and colleagues advise rating down when most of the relevant evidence comes from studies at high risk of bias. Reviewers judge each outcome separately, avoid averaging across studies, and may restrict an analysis to the studies at lower risk of bias.^[4]

Wood and colleagues recommended that reviewers report meta-analyses restricted to trials at low risk of bias, either as the primary analysis or alongside the analysis of all trials.^[15]

Schünemann and colleagues describe how ROBINS-I places randomized and non-randomized studies on a common metric for risk of bias within GRADE. Large effects, dose-effect relations and plausible residual confounding that would strengthen the conclusion can then raise certainty.^[25]

Where to go from here

The hierarchy of evidence reference places these tools within the full range of study designs, and the randomized controlled trials reference explains the allocation and blinding methods that RoB 2 examines. The sham and placebo controls reference covers the comparators that make blinding possible.

The systematic reviews and meta-analysis reference shows how these judgments feed a pooled estimate, and the clinical trial phases reference follows research from IND to FDA approval. The guide to reading the evidence and the exosome clinical trials reference apply these ideas, and our team is glad to talk through any study you are reading.