Research

Tariff Classification Benchmark

Can an open model on a single GPU tell when a US tariff code is wrong? A public test, scored on customs rulings no model has seen.

Design published. Results pending.

As of November 15, 2026

The short version

Every product imported into the United States is declared under a ten-digit tariff code, and the code sets the duty. When an importer and US Customs disagree, Customs issues a ruling that states the correct code and, often, the code it rejected. That makes rulings a fair public test: each one is a real product, a real disputed code, and an official answer. This page fixes the test and the pass mark before the trained model is scored, so the numbers cannot be chosen after the fact.

The task

Input. A product description taken from a ruling, cut off before the ruling discusses classification, plus one declared code, the General Rules of Interpretation, the notes for that code’s chapter, and the tariff lines under its heading.

Output. A verdict (correct or incorrect), a confidence from 0 to 100, the model’s own best code, and a short quote from the tariff text that supports it.

The data

The main test is 237 questions drawn from US Customs rulings dated July 1 to October 22, 2026, after the published training cutoff of the frontier model compared. In 183 the declared code is the one Customs upheld. In 54 it is a code Customs rejected in that ruling. Rulings are public records of the US government. An extension set of rejected-code questions from later rulings is being collected and will be scored as its own group, separate from the main test.

The systems compared

SystemNotes
Claude Sonnet 5.5Frontier model.
Qwen3.8-27B, untunedOpen-weight model, as released, on a single GPU.
Trained open modelLater.

The score is the chance that a system gives a correct code a higher confidence than a wrong one. A coin toss scores half.

The bar, fixed in advance

The trained open model passes if it matches or beats the frontier model’s score on the main test. The bar was fixed on November 13, 2026, before any training run, and will not be changed after results.

What is and is not published

No ruling text is re-hosted here; a list of ruling numbers will be published so anyone can rebuild the test from the public source.

Results

As of November 15, 2026

SystemScore, main test
Claude Sonnet 5.5not yet measured
Qwen3.8-27B, untunednot yet measured
Trained open model (later)not yet measured

A negative result is published as such.

The same method applies to a firm’s own files, which never leave the firm. If you own a problem like this one, write to us.

Or email us directly at hello@avante.io

Changelog

  • November 15, 2026: Design and pass bar published.

Avante IO

© Avante IO Inc. · 🇨🇦 Built in Toronto

Office260 Adelaide St EToronto ON M5A 1N1