← Apps

Tiered taxonomy experiments · results to 14 November 2026

Which taxonomy taggers work best

We tested 23 ways of tagging documents with labels from a tiered taxonomy, on EU laws and on humanitarian excerpts. The most accurate was our combined method: several trained models each suggest labels, and a scorer trained on documents they never saw decides which to keep. Which models it draws on is chosen for each dataset. It led on the EU laws, and on HumSet 2.0.0 by the HumBERT paper's measure, though XR-Transformer edges it by exact labels. Methods that prompt an LLM came well behind, though on the EU laws they found more of the rare labels.

0.610 vs 0.484 published BERT benchmark Our combined method · EU laws

Mean R-Precision on the full test set, trained on the training laws only, as BERT was; trained on newer laws as well, it scores 0.640. BERT's score is from the MultiEURLEX paper (Chalkidis et al., 2021 (opens in a new tab)).

0.663 vs 0.648 published HumBERT benchmark Our combined method · HumSet 2.0.0

Mean macro F1 over five label groups on the full test set; HumBERT's is the mean of the five in its paper (Tamagnone et al., 2023 (opens in a new tab)). On that paper's own average it is 0.739 against 0.73.

0.340 vs 0.566 our combined method Best LLM method · EU laws

KI-FSPrompt, which prompts an LLM, by exact-label F1 on the EU laws.

Every score runs from 0 to 1, and higher is better.

Every method on the same documents

The results

Every method was run on the same 600 EU laws and on all 12,896 test excerpts of HumSet 2.0.0, the HumBERT paper's test set, and scored the same way. None of them saw these documents during training or tuning. Open a method's card, by its name, to read what it does and the papers behind it. Sort the table by any column heading.

What the numbers mean

F1 score
Balances finding the correct labels against adding wrong ones. 1.00 is perfect.
Exact labels
Credit only for the very label a person assigned. The main measure, and the hardest. On HumSet, a pillar given together with one of its own subpillars is dropped first, since the subpillar already names it; a pillar given alone stays.
Tier 1, 2, 3
Credit for landing in the right branch at that level of the taxonomy, even with the wrong label inside it. Tier 1 is the broadest, so the easiest.
Macro F1
HumSet only. The measure the HumBERT paper (Tamagnone et al., 2023 (opens in a new tab)) reports: F1 worked out for each label, so rare labels count as much as common ones, averaged in each of five groups of labels and then over the groups. Trained models get a threshold per label, chosen on development excerpts; LLM methods, which give labels without scores, count every label they return.
Rare labels
Recall, from 0 to 1: the share each method found of the correct labels in the least-used quarter of labels in training, and of those never given in training.
Cost
LLM methods: what the calls cost per 1,000 documents. Trained methods: one-off GPU time, or free on a laptop.

Kinds of method

LLM
Prompts a general-purpose LLM. Needs no labelled documents.
LLM + examples
Prompts an LLM and shows it labelled training documents.
LLM + trained model
An LLM chooses from a shortlist that a trained model helped make.
No training
Embedding search only. Needs no labelled documents and no LLM.
Trained
Learns from the dataset's labelled documents. Trains on a laptop, except XR-Transformer.
Fine-tuned
A pretrained neural model further trained on the dataset, on a rented GPU.
Combined
Our method. Several trained models each suggest labels, and a scorer trained on documents they never saw decides which to keep. Which models it draws on is chosen for each dataset.

Why the exact score is lower than the tier scores

Every box below is a label in our copy of EuroVoc, which arranges its labels in 8 tiers. It is not quite a tree: 244 labels have more than one parent. Tiers 1 and 2 only group the labels below them. Laws are tagged from tier 3 down (Chalkidis et al., 2021 (opens in a new tab)), and none of these test laws has a tier 1 or 2 label.

Say a person gave a law the tier 4 label labelling, and a method chose packaging, another tier 4 label. Both sit under the same tier 3 label, preparation for market.

  1. Tier 1tradeboth match
  2. Tier 2marketingboth match
  3. Tier 3preparation for marketboth match
  4. Tier 4 labelling person packaging method no match

A tier score replaces each label with the label or labels above it at that tier, through every parent, then compares those. At tiers 1, 2 and 3 both labels become the same label, so this near miss counts as correct. The exact score compares the labels themselves, at whatever tier they sit, here tier 4, so it scores nothing.

  • EU laws: our copy of EuroVoc has 7,372 labels, only 693 of them in tiers 1 to 3; the MultiEURLEX paper counts 7,390 in its version. Of the labels people assigned to these test laws, 94% sit deeper than tier 3, so every method's exact score is below all three of its tier scores.
  • Humanitarian excerpts: the taxonomy has only two tiers, and 44% of the labels people assigned are tier 1 sectors and pillars with no subpillar. The exact score mixes the two tiers, so for 20 of the 22 methods it falls between the tier 1 and tier 2 scores.

EU laws (MultiEURLEX): 600 test laws, EuroVoc's top three tiers

Where the labels come from. The laws and their labels are MultiEURLEX (Chalkidis et al., 2021 (opens in a new tab); Dataset (opens in a new tab)). The EU Publications Office tagged each law with labels from EuroVoc (opens in a new tab), the thesaurus it manages for the EU's work, in the 24 official EU languages and three more. Because the EU publishes and maintains it, and its own institutions use it, EuroVoc is the vocabulary EU documents are indexed with. Its users also include national and regional parliaments in Europe, national governments and private users.

The laws are split by date, so the test laws are newer than the training laws. Rows marked "with newer laws" were also trained on newer laws from the development split, which brings them closer in time to the test laws. The published BERT benchmark (Chalkidis et al., 2021 (opens in a new tab)) used training laws only. A rare label here was given to at most 3 of the training laws.

Tiers 1 to 3 here come from moving each label up our copy of EuroVoc, which is newer than the dataset (notebook 01). MultiEURLEX publishes its own labels for its levels 1 to 3 (Chalkidis et al., 2021 (opens in a new tab)), and ours match them for 99.4% of test laws at tier 1, 84.8% at tier 2 and 72.4% at tier 3. So these tier scores cannot be set beside the paper's results by level.

MethodKindF1 score, higher is betterRecall, higher is betterLabels per lawCost
Exact labelsTier 1Tier 2Tier 3Rarest quarterNever seen
Adds GLiNER2.5 and each label's recent historyCombined0.570.750.710.710.170.013.9$10.92 GPU, once
Adds each label's recent historyCombined0.550.770.720.720.090.005.1$3.85 GPU, once
Weighs fine-tuned Legal-BERT, similar documents, label search, fine-tuned small LLM and fine-tuned reranker, chosen on development lawsCombined0.550.730.700.700.120.023.8$8.46 GPU, once
Weighs fine-tuned Legal-BERT, embedding classifier, Omikuji and XR-Transformer, chosen on development lawsCombined0.530.730.690.680.050.003.9$3.85 GPU, once
A BERT-style model with one output per labelFine-tuned0.500.750.700.690.130.015.2$1.09 GPU, once
A BERT-style model with one output per labelFine-tuned0.470.730.680.670.060.004.6$1.03 GPU, once
Reads text and label names togetherFine-tuned0.470.740.680.670.190.064.6$2.46 GPU, once
Word counts, label treesTrained0.460.690.650.630.010.004.8laptop
Qwen2.5-7B, used as a classifierFine-tuned0.460.700.660.660.020.004.1$3.37 GPU, once
BGE-M3 embeddings, one output per labelTrained0.440.710.650.640.010.005.5laptop
The leading trained model in a 2026 benchmark (Kähler et al., 2026 (opens in a new tab))Trained0.430.700.650.640.000.004.0$2.82 GPU, once
Word counts, label treesTrained0.430.670.620.610.000.004.2laptop
Labels of the closest training documentsTrained0.430.710.640.600.110.005.0laptop
BGE-M3 embeddings, one output per labelTrained0.420.710.640.620.000.005.5laptop
Reads text and label togetherFine-tuned0.400.680.620.610.120.074.2$4.00 GPU, once
Labels of the closest training documentsTrained0.390.630.570.530.020.003.4laptop
LLM shown similar labelled documentsLLM + examples0.340.680.590.540.250.259.8$0.40 per 1,000
Shortlist from bothLLM + trained model0.240.540.420.360.150.204.4$0.61 per 1,000
LLMs write keywords; keywords are matched to labelsLLM0.210.610.460.380.350.4915.3$0.87 per 1,000
An LLM picks from the 100 closest labelsLLM0.190.430.320.270.150.213.9$0.46 per 1,000
Five LLMs write keywords, one rates themLLM0.170.500.370.300.180.2610.0$6.69 per 1,000
As above, matching written definitionsLLM0.170.410.310.260.180.233.8$0.59 per 1,000
No extra callLLM0.150.370.270.230.140.182.7$0.59 per 1,000
A second call checks each labelLLM0.150.360.260.220.150.203.0$0.80 per 1,000
Starts lower, looks aheadLLM0.140.390.250.170.120.172.9$0.88 per 1,000
Closest label names, no trainingNo training0.120.500.330.250.100.199.8laptop
Top-down, several pathsLLM0.050.320.140.080.080.104.2$1.35 per 1,000
Top-down, one pathLLM0.020.180.060.030.010.031.4$0.59 per 1,000

Humanitarian excerpts (HumSet 2.0.0): all 12,896 test excerpts, the HumBERT paper's test set

Where the labels come from. The excerpts and their labels are HumSet (Fekih et al., 2022 (opens in a new tab); Dataset (opens in a new tab); Project page (opens in a new tab)). Humanitarian analysts in 46 response projects picked them from crisis documents on DEEP, an analysis platform run by Data Friendly Space, a US-based international NGO, whose board includes OCHA, UNHCR, UNICEF and the IFRC. Each project tagged with its own analysis framework, 11 between them, and the HumSet authors merged these into one common framework of sectors, pillars and subpillars. Unlike EuroVoc, it is not a standard published on its own, but a merger of the frameworks these projects worked with.

Excerpts in English, French and Spanish. Tier 1 holds the sectors and pillars, and tier 2 the subpillars. A rare label here was given to at most 596 of the training excerpts. Every label in these test excerpts was given in training, so the never-seen column reads none. Every method, LLM methods included, was trained, tuned and run on this version's own splits (notebooks 15 and 17), so the macro F1 columns score the paper's own test set by its measure (Tamagnone et al., 2023 (opens in a new tab)) and can be set beside its printed figures; the note on the recipe's row says what the paper reported and how our cut-offs differ from its trainer's.

How the labels differ from HumSet 1.0.0

Version 2.0.0 uses 77 labels. Only it has Capacities & Response, Capacities & Response->People Reached/Response Gaps, Capacities & Response->Red Cross/Red Crescent, Casualties, Context, Covid-19, Covid-19->Prevention Campaign, Covid-19->Research And Outlook, Displacement, Humanitarian Access, Humanitarian Access->People Facing Humanitarian Access Constraints/Humanitarian Access Gaps, Humanitarian Conditions, Impact, Information And Communication, Priority Needs, Shock/Event and Shock/Event->Mitigating Factors; only 1.0.0 has Capacities & Response->Number Of People Reached/Response Gaps and Humanitarian Access->Number Of People Facing Humanitarian Access Constraints/Humanitarian Access Gaps. Version 2.0.0 also tags some excerpts with a pillar but none of its subpillars; that pillar is then a label of its own. The label browser and examples are from 1.0.0.

MethodKindF1 score, higher is betterRecall, higher is betterLabels per excerptCost
Macro F1, sectors and pillars (paper's Avg)Exact labelsTier 1Tier 2Macro F1Rarest quarterNever seen
The leading trained model in a 2026 benchmark (Kähler et al., 2026 (opens in a new tab))Trained0.490.720.760.690.510.44none2.1$1.38 GPU, once
Weighs fine-tuned HumBERT and similar documents, chosen on development excerptsCombined0.740.720.780.710.660.58none2.4$0.67 GPU, once
The paper's own trainer and settings, rerun (Tamagnone et al., 2023 (opens in a new tab))Fine-tuned0.72paper 0.730.690.770.680.63paper 0.650.33none2.3$1.26 GPU, once
Labels of the closest training documentsTrained0.650.690.730.650.590.41none2.2laptop
A BERT-style model with one output per labelFine-tuned0.720.680.760.680.640.45none2.2$0.67 GPU, once
Word counts, label treesTrained0.630.670.710.640.570.44none2.3laptop
BGE-M3 embeddings, one output per labelTrained0.490.630.680.590.470.47none2.2laptop
LLM shown similar labelled documentsLLM + examples0.580.580.650.570.530.59none2.8$0.15 per 1,000
Reads text and label togetherFine-tuned0.610.540.630.550.510.24none3.0$3.00 GPU, once
Reads text and label names togetherFine-tuned0.580.500.620.450.460.11none2.6$2.80 GPU, once
Qwen2.5-7B, used as a classifierFine-tuned0.570.490.620.440.450.12none2.4$1.67 GPU, once
Top-down, several pathsLLM0.400.410.470.260.300.42none1.8$0.16 per 1,000
The LLM sees every label at onceLLM0.430.390.480.320.350.44none1.9$0.15 per 1,000
As above, matching written definitionsLLM0.420.370.450.330.350.46none1.7$0.29 per 1,000
An LLM picks from the 100 closest labelsLLM0.430.360.460.300.350.54none2.4$0.18 per 1,000
Top-down, one pathLLM0.360.360.420.220.260.24none1.0$0.13 per 1,000
LLMs write keywords; keywords are matched to labelsLLM0.380.340.440.230.290.28none1.4$0.33 per 1,000
A second call checks each labelLLM0.400.340.420.300.330.39none1.5$0.39 per 1,000
No extra callLLM0.370.320.400.280.310.37none1.4$0.29 per 1,000
Five LLMs write keywords, one rates themLLM0.370.320.430.270.290.27none2.7$2.84 per 1,000
Starts lower, looks aheadLLM0.200.220.330.300.200.40none1.5$0.22 per 1,000
Closest label names, no trainingNo training0.330.110.250.070.250.52none7.5laptop

We first ran every method on 1,500 sampled test excerpts of HumSet 1.0.0, an earlier version with different splits. There the combined method had the top score by exact labels, 0.662 against XR-Transformer's 0.642, and on all 15,147 of that version's test excerpts it beat HumBERT's recipe, rerun by us, by macro F1, 0.574 to 0.560.

LLM methods used openai/gpt-oss-120b:novita and openai/gpt-oss-20b:deepinfra through Hugging Face Inference Providers, except LLM-Ensemble, which used the five open models listed in configs/experiment.yaml. GPU training ran on Hugging Face Jobs at November 2026 prices.

What the results show

Findings

  • The combined method is the most accurate. On the laws it scores 0.566 by exact labels, against 0.497 for the best single model, the fine-tuned Legal-BERT. On HumSet 2.0.0 it leads by the HumBERT paper's macro F1, 0.663 against 0.506 for XR-Transformer, the best single model, though XR-Transformer edges it by exact labels, 0.722 to 0.716. A version of the combined method also has the top score at every tier.
  • Trained models beat LLM methods, with exceptions on HumSet. On the laws every fine-tuned model scores above every LLM method; the best LLM method, KI-FSPrompt, scores 0.340. On HumSet 2.0.0 KI-FSPrompt scores 0.585: below every trained model that has no LLM inside it, but above the fine-tuned reranker (0.542), GLiNER2.5 (0.502) and fine-tuned small LLM (0.493). KI-FSPrompt also needs labelled documents, because it shows them to the LLM as examples.
  • LLM methods fall further behind on finer labels. On the laws, KI-FSPrompt trails the best single model by 0.07 at tier 1 (0.68 against 0.75), and by 0.15 at tier 3 (0.54 against 0.69).
  • LLM methods find labels the training data lacks. The LLM keyword pipeline finds 49% of the correct labels never seen in training; the combined method finds 1%. But the pipeline suggests 15 labels per law, where a law has about 6.5, so most of its suggestions are wrong.
  • Showing the LLM labelled examples is the best LLM recipe. KI-FSPrompt beats the plain LLM keyword pipeline by 0.13 (95% interval 0.12 to 0.15) on the laws and by 0.17 (95% interval 0.16 to 0.19) on the HumSet 1.0.0 sample, at about half the cost. LLM-Ensemble, with five LLMs, did worse than the plain pipeline and cost about eight times as much.
  • Search-then-choose needs a trained model in the shortlist. On the laws, search alone put only 41% of the correct labels in its top 100, so the LLM could not choose the rest. Adding the trained model's suggestions raised that to 75%, and F1 by 0.08 (95% interval 0.06 to 0.09).
  • Walking down the taxonomy one branch at a time fails. It scores 0.02 on the laws. Keeping three branches open helps on both datasets. Starting from search results instead of from the top helps on the laws but hurts on the excerpts.
  • Asking the LLM to double-check its labels made things worse. F1 fell by 0.02 (95% interval 0.01 to 0.02) on the laws and by 0.02 (95% interval 0.02 to 0.03) on the HumSet 1.0.0 sample, and the number of calls doubled. Keeping only labels with a supporting quote also lowered F1.

Each difference with an interval was tested on the same documents: the 600 laws, or the sample of HumSet 1.0.0 where every method was first run. Every one of those intervals excludes zero, so none of these differences is likely to be chance.

Full test sets

Against the published benchmarks

The published benchmarks use the full test sets: 5,000 EU laws, and 12,896 humanitarian excerpts in HumSet 2.0.0. Every method was run on all of HumSet 2.0.0's, as its table above shows; on the laws only methods without an LLM call per law were run on all of them.

EU laws: mean R-Precision

For each law, take as many of the top-ranked labels as the law really has, and count the share that are correct. 1.00 is perfect. The MultiEURLEX paper prints it as a percentage: 48.4 there is 0.484 here. That figure is from its Table 4: BERT on English, all of the original labels, trained on the training laws only, so our rows marked "training laws only" are the like-for-like comparison. The paper scores against all 7,390 of its labels; we leave out the 72 of the 31,178 labels given to these laws that are missing from our copy of EuroVoc. Counting them lowers our scores by less than 0.001, in a check made outside the notebooks.

Combined method, second attemptCombined method, other versionsSingle modelsPublished benchmark
  1. Combined method, second attemptWith newer laws0.640
  2. Combined method, first attemptWith newer laws0.624
  3. Combined method, second attemptTraining laws only0.610
  4. Combined method, first attemptTraining laws only0.600
  5. Fine-tuned Legal-BERTWith newer laws0.529
  6. Fine-tuned Legal-BERTTraining laws only0.500
  7. BERT, publishedThe benchmark to beat (Chalkidis et al., 2021 (opens in a new tab))0.484
  8. XR-TransformerTraining laws only0.474
  9. OmikujiTraining laws only0.471
  10. Embedding classifierTraining laws only0.459
  11. Similar documentsTraining laws only0.424
  12. Label searchNo training0.139

The second attempt added GLiNER2.5 and each label's recent history. It raised the score by 0.015 (95% interval 0.013 to 0.018) with newer laws, and by 0.010 (0.007 to 0.013) with training laws only. It was made after seeing the first attempt's test scores, so we report it as a second attempt.

Humanitarian excerpts: mean macro F1

F1 is worked out for each label and then averaged, so rare labels count as much as common ones. This is the measure the HumBERT paper (Tamagnone et al., 2023 (opens in a new tab)) reports, over five groups of labels, on all 12,896 test excerpts of HumSet 2.0.0.

  1. Combined methodWeighs fine-tuned HumBERT and similar documents, chosen on development excerpts0.663
  2. HumBERT, publishedThe benchmark to beat (Tamagnone et al., 2023 (opens in a new tab))0.648
  3. HumBERT recipe, rerunThe authors' code and settings0.643
  4. Fine-tuned HumBERT, oursThe best single source on development excerpts0.639
Label group HumBERT, published HumBERT recipe, rerun Combined method 95% interval
Sectors0.780.7780.7900.780–0.799
Pillars 1D0.700.6850.7050.696–0.718
Subpillars 1D0.540.5480.5740.562–0.588
Pillars 2D0.710.7050.7210.707–0.735
Subpillars 2D0.510.5010.5270.512–0.541
Five-group mean0.6480.6430.6630.657–0.670
Tier-1 average0.730.7220.739–

On HumSet 2.0.0, the version HumBERT's published figures come from, the combined method's 95% interval for the five-group mean lies wholly above the published figure. Its interval lies wholly above the published figure in Subpillars 1D and Subpillars 2D. It beats HumBERT's recipe, rerun with the authors' own code and settings, in every group.

Whether that rerun reproduces the published figures depends on how subpillar scores are averaged, which the paper (Tamagnone et al., 2023 (opens in a new tab)) does not say. Averaging every label once, as every score here does, it comes within 0.015 of every published group. Averaging subpillars within each pillar first, as the authors' code does, it falls short in Subpillars 1D (by 0.041), so notebook 13 records it as not reproducing the paper. The published figures have two decimals; their mean is worked out from them.

The HumBERT paper (Tamagnone et al., 2023 (opens in a new tab)) prints no mean of the five groups. Its own "Avg" column is the tier-1 average in the table's last row: the mean of Sectors, Pillars 1D and Pillars 2D, leaving out both subpillar groups, which score lowest.

The recipe rerun is one run with one seed, so it has no interval.

Long documents

Labelling long documents in pieces

Long documents were cut into 400-word pieces, an LLM method labelled each piece, and four rules combined the pieces' labels into labels for the whole document. Weighting each piece by how sure the model was gave the same result as keeping any label found in any piece. Which rule worked best depended on the dataset.

Documents First piece only Label in a share of pieces Label in any piece Weighted by certainty LLM cost
120 long EU laws0.164 labels0.223 labels0.1155 labels0.1155 labels$3.12
200 humanitarian reports0.162 labels0.273 labels0.5412 labels0.5412 labels$1.31

Exact-label F1, with the average number of labels each rule gave a document. The certainty-weighted rule follows (Wu et al., 2026b (opens in a new tab)), listed below.

Read with care

Limits

  • Most methods are simplified from their papers, so these results say little about the originals. Each method's card says how ours differs.
  • The EU laws table uses 600 laws, a sample of the test set; HumSet 2.0.0's table uses its whole test set. Small differences between neighbouring rows may not hold on other documents.
  • Some findings, marked with the HumSet 1.0.0 sample, were measured only on that earlier version, where every method was first run.
  • Results depend on the LLMs named above. Other LLMs could change the order of the LLM methods.
  • The second attempt was made after seeing the first attempt's test scores. The pass marks it was judged against were not changed.
Sources

Papers and software

  1. MultiEURLEX – A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer (opens in a new tab)Ilias Chalkidis, Manos Fergadiotis and Ion Androutsopoulos (2021). arXiv preprint.
  2. LEGAL-BERT: The Muppets straight out of Law School (opens in a new tab)Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis and others (2020). arXiv preprint.
  3. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (opens in a new tab)Jianlv Chen, Shitao Xiao, Peitian Zhang and others (2024). arXiv preprint.
  4. CTI2TTP: Evidence-Grounded TTP Extraction from Cyber Threat Intelligence via Diverge-then-Converge Verification (opens in a new tab)Yutong Cheng, Changze Li, Raihan Sultan Pasha Basuki and others (2026). arXiv preprint.
  5. Introducing GLiNER2.5: Efficient Span-Free Information Extraction with Schema-Driven Interface (opens in a new tab)Fastino (2026). Fastino blog.
  6. HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response (opens in a new tab)Selim Fekih, Nicolò Tamagnone, Benjamin Minixhofer and others (2022). arXiv preprint.
  7. LoRA: Low-Rank Adaptation of Large Language Models (opens in a new tab)Edward J. Hu, Yelong Shen, Phillip Wallis and others (2021). arXiv preprint.
  8. DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing (opens in a new tab)Lisa Kluge and Maximilian Kähler (2025). SemEval-2025.
  9. Does generative AI supersede supervised XMLC? A Benchmark Study on Automated Subject Indexing with German Scientific Literature (opens in a new tab)Maximilian Kähler, Katja Konermann, Lisa Kluge and Markus Schumacher (2026). arXiv preprint.
  10. DNB-AI-Project at the GermEval-2025 LLMs4Subjects Task: KIFSPrompt – Knowledge-Injected Few-Shot Prompting (opens in a new tab)Maximilian Kähler, Lisa Kluge and Katja Konermann (2025). KONVENS 2025 Workshops.
  11. MATCH: Multi-stage Article Tagging to Concept Hierarchies (opens in a new tab)Daniele Raimondi, Feichi Lu, Oliver Grun, Mariia Eremina and Andrea Perlato (2026). arXiv preprint.
  12. Graph-Constrained Policy Learning for Extreme Clinical Code Prediction (opens in a new tab)Amritpal Singh, Sebastian Torres, Khawar Shakeel and Syed Ahmad Chan Bukhari (2026). arXiv preprint.
  13. classifier-trainer, the trainer behind HumBERT's classifier (opens in a new tab)Data Friendly Space, the team behind DEEP (2023). GitHub.
  14. Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification (opens in a new tab)Nicolò Tamagnone, Selim Fekih, Ximena Contla, Nayid Orozco and Navid Rekabsaz (2023). IJCAI 2023.
  15. Constraint-Aware Hierarchical Search for Regulation-Driven Fine-Grained Classification (opens in a new tab)Siyu Wang, Wei Tan and Lulu Chen (2026). arXiv preprint.
  16. Label-Centric Knowledge Retrieval for Zero-Shot Text Classification (opens in a new tab)Daoping Wu, Dong Wang, Siyu Liu and others (2026a). arXiv preprint.
  17. Count Evidence, Not Sentences: Tempered Evidence Fusion of LLM Judgments for Long-Text Value Measurement (opens in a new tab)Yuhe Wu, Rui Qian, Guangyu Wang and others (2026b). arXiv preprint.
  18. Travel on the ICD Tree: Benchmarking Agentic Reasoning for ICD Coding from Chinese Electronic Medical Records (opens in a new tab)Xinjie Xu, Yongqi Fan, Shuang-shuang Chen and others (2026). Findings of ACL 2026.
  19. LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification (opens in a new tab)Hui Ye, Jing Zhang, Xiulong Yang and Rajshekhar Sunderraman (2026). arXiv preprint.
  20. Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification (opens in a new tab)Jiong Zhang, Wei-cheng Chang, Hsiang-fu Yu and Inderjit S. Dhillon (2021). arXiv preprint.
  21. Qwen2.5 Technical Report (opens in a new tab)Qwen team (2024). arXiv preprint.
  22. PECOS: Prediction for Enormous and Correlated Output Spaces (opens in a new tab)Amazon. Software, GitHub.
  23. bge-reranker-v2-m3 (opens in a new tab)BAAI. Model card, Hugging Face.
  24. Omikuji: an efficient implementation of Partitioned Label Trees (opens in a new tab)Tom Dong. Software, GitHub.

Paper details checked against arXiv and the ACL Anthology on 10 November 2026. Built by scripts/make_report.py from outputs/results/*.json.

    Models it combines

      The clues it weighs

      F1 score here bar: this method · tick: best in the table

      Broad to narrow. A tier counts a label in the right branch; exact needs the very label a person chose.

      Ours against the source

      Sources

        Notebook

        The labels

        How often each label was given in training

          Open a label to see the ones under it. The share beside a label is of the training documents people gave it to; hover for the count. Broad labels are rarely given directly.

          Example records

          Each record as it appears in the test set, and the labels people gave it, each shown on its path from tier 1 down.

          label given by a person label a branch above it