Analysts and lawyers check each statement in a report or brief against the filing it cites. The work is routine and high-volume, and an error that slips through matters more than a slow answer. Frontier language models do it well, but at volume their cost and delay add up.
A decision model is a small model that picks one answer from a fixed list, with a probability for each, in a single call. It is cheap and fast, and its probabilities show when it is unsure. The question is how much checking it can take over: the claims it is confident about, within a set error budget, with the rest sent to a person or a stronger model.
ClaimBench has 1,960 claims, each paired with an excerpt from a 10-K. A system says whether the excerpt supports the claim, contradicts it, or says nothing about it. Below: how many claims each decision model can answer alone within a 1% error budget, which kinds of claim trip it up, and what it costs to send the rest to a language model.
- Language models
- 100.0%
- Top accuracy, reached by 4 models from $0.0025 per claim
- Cascade
- $0.11
- Per 1,000 claims for Decider V1.2 in front of Claude Haiku 5.5, whose label call alone costs $1.19
Which claims a decision model can take over
At most 1% errors
| System | Direct support | Subtle contradiction | Numeric discrepancy | Missing evidence | Non-supporting citation | Overall |
|---|---|---|---|---|---|---|
 Decider V1.2 | 100.0% | 98.5% | 100.0% | 100.0% | 100.0% | 99.7% |
| Decider V1.1 27B | 100.0% | 93.9% | 100.0% | 100.0% | 100.0% | 98.8% |
 Microsoft-Decision-1 | 99.7% | 91.8% | 99.5% | 100.0% | 100.0% | 98.2% |
 Mercury Decide | 98.0% | 97.7% | 100.0% | 100.0% | 99.7% | 99.1% |
 d1 | 99.0% | 94.6% | 99.7% | 100.0% | 100.0% | 98.7% |
 Jev 1.14 | 99.5% | 92.6% | 98.7% | 100.0% | 99.5% | 98.1% |
 Drex v1.5 | 99.2% | 92.6% | 99.0% | 100.0% | 99.2% | 98.0% |
| Decider Flash V1.2 | 100.0% | 83.9% | 99.0% | 100.0% | 99.7% | 96.5% |
| Jev 1.13.0 | 99.5% | 92.1% | 98.0% | 100.0% | 98.7% | 97.7% |
 GPT-6 Luna Decisions | 98.2% | 88.0% | 98.2% | 100.0% | 100.0% | 96.9% |
 Tev1 4B Experimental | 98.5% | 68.6% | 84.0% | 99.7% | 98.2% | 89.8% |
 Clef Omni | 98.2% | 55.9% | 88.0% | 99.5% | 96.9% | 87.7% |
| Drex v2.0 | 98.7% | 94.1% | 98.0% | 98.2% | 97.4% | 97.3% |
 Classify-1 | 88.0% | 66.8% | 74.8% | 97.7% | 97.7% | 85.0% |
When a decision model is unsure
Across: share of held-out claims a model answers alone.Up: share of those answers that are wrong.Dot: its tuned threshold.Green: at most 1% wrong.
1,376 held-out claims. Below its threshold, a claim goes to Claude Haiku 5.5. Dashed: its label call alone.
Swipe for more columns →
| System | Cost / 1,000 claims ↓ | Accuracy ↑ | Automated ↑ |
|---|---|---|---|
| Decider V1.2 | $0.11 | 99.85% | 99.7% |
| Decider V1.1 27B | $0.11 | 99.27% | 99.3% |
| Mercury Decide | $0.12 | 99.42% | 98.8% |
| d1 | $0.13 | 99.42% | 97.7% |
| Jev 1.14 | $0.34 | 99.64% | 93.0% |
| Drex v2.0 | $0.39 | 99.49% | 95.5% |
| Jev 1.13.0 | $0.41 | 99.49% | 88.0% |
| Drex v1.5 | $0.47 | 99.85% | 81.0% |
| Decider Flash V1.2 | $0.64 | 99.78% | 55.5% |
| GPT-6 Luna Decisions | $0.77 | 99.13% | 79.4% |
| Tev1 4B Experimental | $1.29 | 99.85% | 15.6% |
| Microsoft-Decision-1 | $1.36 | 99.78% | 2.9% |
| Classify-1 | $1.40 | 100.00% | 2.3% |
| Clef Omni | $1.97 | 99.64% | 1.7% |
 Claude Haiku 5.5 | $1.19 | 100.00% | — |
Claims
A system gets a claim, a 10-K excerpt and the three labels below, and returns one label, with a probability for each label when it can. Every system gets the same instruction: judge only from the excerpt, not from outside knowledge of the company.
| Label | Meaning |
|---|---|
supported | The excerpt states or directly entails the claim. |
contradicted | The excerpt states something that conflicts with the claim. |
insufficient_evidence | The excerpt neither supports nor contradicts the claim. |
v1 has five categories of about 392 claims each. Gold labels come from the construction rule, never from a model. Excerpts are short (a paragraph), medium (about 2k to 4k tokens) or long (about 8k to 16k tokens).
| Category | Gold label | How the claim is built |
|---|---|---|
| Direct support | supported | A model paraphrases a sentence of the excerpt. |
| Subtle contradiction | contradicted | A model makes a small edit to a sentence of the excerpt. Numbers stay the same, a plain negation is rejected, and the wording stays close to the original. |
| Numeric discrepancy | contradicted | One amount or percentage in a sentence of the excerpt is changed to a plausible wrong value. Sentences with a bound, such as “more than” or “at least”, are excluded. |
| Missing evidence | insufficient_evidence | A sentence from a different passage of the same filing, with little wording in common with the excerpt. |
| Non-supporting citation | insufficient_evidence | A sentence from one company, paired with similar prose from another. |
The class mix is uneven: 392 supported, 785 contradicted and 783 with insufficient evidence, so always guessing one label scores about 40% accuracy.
Missing evidence and non-supporting citation use sentences copied from filings. The other three categories use edited or generated text, which can carry stylistic quirks, so read the two groups separately.
Examples
Three of the 10 public examples: the shortest for each label, picked by rule and not by reading them first.
Direct support
- Source
- Danaher Corp, 10-K filed 2026‑02‑24
- Claim
- Interest rate swap contracts were employed by the Company to hedge against cash flow fluctuations resulting from shifts in benchmark interest rates associated with a part of the debt issued by the Company.
- Excerpt
The Company used interest rate swap agreements to hedge the variability in cash flows due to changes in benchmark interest rates related to a portion of the debt the Company issued. These contracts effectively fixed the interest rate for a portion of the Company’s debt equal to the notional amount of the swaps to the rate specified in the interest rate swap agreements and were settled in November 2019 and December 2021. The changes in the fair value of these instruments were recorded in accumulated OCI prior to the issuance of the debt and are subsequently being reclassified to interest expense over the life of the related debt.
- Correct answer
supported.
Subtle contradiction
- Source
- Morgan Stanley, 10-K filed 2026‑02‑19
- Claim
- The protection afforded in those jurisdictions may be more established and/or predictable than in the U.S. or other jurisdictions in which we operate.
- Excerpt
We conduct business in various jurisdictions outside the U.S., including jurisdictions that may not have comparable levels of protection for their corporate assets, such as intellectual property, trademarks, trade secrets, know-how, and customer information and records. The protection afforded in those jurisdictions may be less established and/or predictable than in the U.S. or other jurisdictions in which we operate. As a result, there may also be heightened risks associated with the potential theft of their data, technology and intellectual property in those jurisdictions by domestic or foreign actors, including private parties and those affiliated with or controlled by state actors. Additionally, we are subject to complex and evolving U.S. and international laws and regulations governing areas such as cybersecurity, privacy and data governance, transfer and protection, which may differ and potentially conflict, in various jurisdictions or cause us to develop or enhance controls that may encumber
- Correct answer
contradicted. The claim changes "less established" in the filing to "more established".
Non-supporting citation
- Source
- Claim from BlackRock, Inc., excerpt from Nucor Corp, 10-K filed 2026‑02‑25
- Claim
- BlackRock FMC is also subject to the China Securities Law and various other financial laws and regulations.
- Excerpt
Show the medium excerpt
Trade remedies implemented by the U.S. International Trade Commission, (the "USITC”) such as antidumping duties ("AD") and countervailing duties ("CVD") and similar measures play a key role in allowing the American steel industry to compete on a level playing field against unfairly traded imports. There are currently 142 AD/CVD orders in place on core steel product lines made by Nucor, and sunset reviews are typically held with respect to specific exporters and products every five years.
During 2025, the U.S. Department of Commerce and the USITC made important rulings regarding unfairly traded imports of corrosion-resistant steel and rebar. The U.S. government kept in place duty orders on large diameter welded pipe from Japan and wire rod imports from China as the result of sunset reviews.
Circumvention of trade duties also continues to pose a threat. Besides producing over a billion net tons of steel in its own country, China continues to invest heavily in steel production in other countries to avoid being subject to trade duties on exports to the U.S. market. Most of the investment being done by Chinese state-owned companies outside of the country’s borders is occurring in other Southeast Asia countries and Africa.
After recent years of weakened effectiveness due to country exemptions and product exclusions, the Section 232 steel tariffs were fully reinstated in 2025 without exceptions or exclusions, and were broadened to cover approximately 600 fabricated steel products. These comprehensive tariffs have been more effective in combating circumvention behaviors and keeping unfairly traded imports out of the U.S. market.
We also experience competition from other materials. Depending on our customers’ end use of our products, there are often other materials, such as concrete, aluminum, plastics, composites and wood that compete with our steel products. When the price of steel relative to other raw materials rises, these alternatives can become more attractive to our customers.
In our steel products segment we manufacture a wide range of products that primarily have construction applications. In each of our product lines, we face competition from well capitalized domestic and international providers offering similar products and services. We compete on price, service (e.g., consulting on engineering requirements, facilitating logistics, and timeliness of order fulfillment) and quality (e.g., reliably producing to exacting custom specifications). We believe we have established a reputation as a market leader who can consistently meet customer needs for these products in a timely
Competition in our scrap and raw materials business is also vigorous. The scrap metals market consists of many firms and is highly fragmented. Firms typically compete on price and geographic proximity to the sources of scrap metal.
In the steel mills segment, Nucor’s backlog of orders was approximately $3.35 billion and $2.13 billion at December 31, 2025 and 2024, respectively. Order backlog for the steel mills segment includes only orders from external customers and excludes orders from other Nucor businesses. Nucor’s backlog of orders in the steel products segment was approximately $4.46 billion and $4.02 billion at December 31, 2025 and 2024, respectively. The majority of these orders are expected to be filled within one year. Order backlog within our raw materials segment is not meaningful because the vast majority of the raw materials that segment produces are used internally.
An ample supply of high-quality scrap and scrap substitutes is critical to support Nucor’s ability to produce high-quality steel. The goal of Nucor’s raw materials segment is to safely produce, source, trade and transport steelmaking raw materials. Nucor’s raw materials investments are focused on creating an advantage for our steelmaking operations, through a global information network and flexible approach to metallics supply.
Scrap and scrap substitutes are the most significant element in the total cost of steel production. The average cost of scrap and scrap substitutes used in our steel mills segment remained relatively stable from $394 per gross ton used in 2024 to $392 per gross ton used in 2025. On average, it takes approximately 1.1 tons of scrap and scrap substitutes to produce one ton of steel. We employ variable steel pricing mechanisms so that we are better able to maintain operating margins and meet our customer commitments as scrap and scrap substitute costs fluctuate.
Nucor remains focused on securing reliable access to low-cost raw material inputs as they are the Company’s largest expense. We believe Nucor’s broad, balanced supply chain is an important strength which allows us to reduce the cost of our steelmaking operations, create a shorter supply chain and have greater control over our metallic inputs. Our investment in DRI production facilities and scrap yards, as well as our access to international raw materials markets, provides Nucor with significant flexibility in optimizing our raw material mix. Additionally, having a significant portion of our raw materials supply under our control minimizes risk associated with the global sourcing of raw materials, particularly since a good deal of scrap substitutes comes from regions of the world that have historically experienced greater political turmoil, such as Ukraine, Russia and Brazil. We believe the continued successful implementation of our raw material strategy, including key investments in DRI production, as well as in the scrap brokerage and processing services performed by our team at DJJ, gives us greater control over our metallic inputs and thus helps us navigate significant fluctuations in the availability and costs of critical inputs.
Steel manufacturing is an energy-intensive, trade exposed industry. As a result, we continuously strive to make our operations in all three of our business segments more energy efficient. In addition, we proactively engage with suppliers, regulators and other energy industry participants to ensure the continued domestic availability of reliable, low-cost sources of energy in various forms.
Our steelmaking operations utilize EAFs for 100% of their production. The total energy consumed by Nucor includes electricity, natural gas, and other carbon raw material inputs. For the scrap melting process, electricity is the primary energy source, with natural gas combustion serving as the fuel for reheat furnaces and other pre-heating operations. Our DRI facilities in Trinidad and Louisiana are also large consumers of natural gas.
We closely monitor developments in public policy relating to energy production and consumption. We engage with policymakers to provide technical information that can inform policy decisions and avoid unintended adverse consequences of legislative and regulatory actions. We believe that a thoughtful approach to domestic energy policy can help ensure that steel and steel products manufactured in the United States remain competitive in the global marketplace.
As the leading U.S. supplier of numerous essential steel products using primarily recycled ferrous scrap metal, we believe we are in a competitively advantageous position. Our circular production process has approximately one-third the GHG emissions intensity of the average extractive steelmaking process using a blast furnace and basic oxygen furnace.
In addition, we are committed to further reducing our GHG footprint over time. After announcing our net-zero, science-based GHG targets for 2050 and our interim emissions-reduction target for 2030, we achieved third-party certification of these targets in December 2024. Our GHG intensity targets are defined in accordance with the Global Steel Climate Council’s (GSCC) Steel Climate Standard and have been verified to conform with the emissions-reduction glidepath required to achieve the Paris Climate Agreement’s 1.5°C goal by 2050.
Currently, we are a party to two Power Purchase Agreements (“PPAs”). Under these PPAs, we have agreed to purchase for a fixed price output from one solar and one wind project in the United States. The PPAs are structured for monthly financial settlement. We have undertaken these initiatives to support the ongoing diversification of generation technologies on the U.S. power grid. As part of these arrangements, we will also receive Renewable Energy Credits (“RECs”) commensurate with the renewable power we purchase. These RECs can be applied against a portion of our GHG emissions, enabling us to receive credit for reducing them. The pay-fixed, receive-floating nature of these arrangements also offsets a portion of our exposure to higher prices for electricity over the life of the contract.
In collaboration with our electric utility partner, we evaluated opportunities to deploy battery storage and solar generation in support of the expansion at our Kingman, Arizona bar mill. In 2025, we contracted with a third party to install and operate a 50 MW onsite battery energy storage system which became operational in the fourth quarter of 2025. The Kingman site will also host a 25 MW solar project, with construction planned for 2026. The goal of combining the two systems is to provide our Kingman division with access to reliable power for years to come.
- Correct answer
insufficient_evidence. The claim is about a different company than the excerpt, on a similar topic.
Methodology
Cost and latency are per claim. A language model makes two calls per claim, a label and then a probability for each label, so its figures cover both. Latency is the time the model took, as the gateway measured it, and leaves out the hop between the sandbox and the gateway.
A decision model answers a claim itself when its top probability reaches a threshold: the lowest one that keeps at most 1% of answers wrong on a fixed 30% of the claims. It is then frozen and applied to the other 70%, the held-out claims. In the cascade table the rest go to the language model’s label call, and the cost is the decision model’s on every held-out claim plus that call on the claims it sends.