Cell-Based Representation of Relational Binding in Language Models
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Benjamin Heinzerling
I study how AI systems represent and use information, with a particular focus on language models. My research asks how they encode numbers, keep track of entities, and store facts about the world. As these systems become more complex, I also investigate how AI can help us understand their internal workings.
At RIKEN AIP, I’m Deputy Team Director of the Natural Language Understanding Team, and I also hold an appointment as Specially Appointed Associate Professor at Tohoku University’s Center for Language AI Research. I received my PhD in computational linguistics from Heidelberg University.
News
-
I gave a guest lecture on “Analyzing and Interpreting the Internals of Language Models” at the University of Tokyo.
-
I gave a talk on “Linear Representations in LLM Interpretability: Claims and Evidence” at the TU Darmstadt–RIKEN AIP joint workshop.
-
Our paper “A Robust Evaluation of Probe Robustness: Lessons for Reliable OOD Uncertainty Quantification” has been accepted to EMNLP 2026.
-
Our new preprint “Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics” is now available on arXiv.
-
I'll be a tutorial co-chair at ACL 2027.
Publications
2026
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
arXiv preprint, arXiv:2606.03982
A Robust Evaluation of Probe Robustness: Lessons for Reliable OOD Uncertainty Quantification
EMNLP 2026 (accepted)
Linear Representations of Hierarchical Concepts in Language Models
arXiv preprint, arXiv:2604.07886
2025
Can Language Models Handle a Non-Gregorian Calendar? The Case of the Japanese wareki
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
Findings of the Association for Computational Linguistics: EMNLP 2025
Understanding the Side Effects of Rank-One Knowledge Editing
Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
On Entity Identification in Language Models
Findings of the Association for Computational Linguistics: ACL 2025
Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
TopK Language Models
arXiv preprint, arXiv:2506.21468
Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
arXiv preprint, arXiv:2505.21458
Understanding Fact Recall in Language Models: Why Two-Stage Training Encourages Memorization but Mixed Training Teaches Knowledge
arXiv preprint, arXiv:2505.16178
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Findings of the Association for Computational Linguistics: NAACL 2025
2024
Representational Analysis of Binding in Language Models
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation
Conference on Language Modeling (COLM 2024)
Monotonic Representation of Numeric Attributes in Language Models
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
The Curse of Popularity: Popular Entities have Catastrophic Side Effects when Deleting Knowledge from Language Models
arXiv preprint, arXiv:2406.06032
2023
Tracing and Manipulating Intermediate Results in Neural Math Problem Solvers
Journal of Natural Language Processing 30(4), 1151–1171
Test-time Augmentation for Factual Probing
Findings of the Association for Computational Linguistics: EMNLP 2023
Can LMs Store and Retrieve 1-to-N Relational Knowledge?
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)
Prompting for explanations improves Adversarial NLI. Is this true? Yes it is true because it weakens superficial cues
Findings of the Association for Computational Linguistics: EACL 2023
Examining the effect of whitening on static and contextualized word embeddings
Information Processing & Management 60(3), 103272
2022
Universal Graph based Distantly Supervised Relation Extraction
Journal of Natural Language Processing 29(4), 1138–1164
Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Tracing and Manipulating intermediate values in Neural Math Problem Solvers
Proceedings of the 1st Workshop on Mathematical Natural Language Processing (MathNLP)
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
arXiv preprint, arXiv:2211.05100
COPA-SSE: Semi-structured Explanations for Commonsense Reasoning
Proceedings of the Thirteenth Language Resources and Evaluation Conference
2021
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL)
Learning to Learn to be Right for the Right Reasons
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
2020
NLP's Clever Hans Moment has Arrived
Journal of Cognitive Science 21(1), 161–170
2019
On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages
Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)
When Choosing Plausible Alternatives, Clever Hans can be Clever
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing
Riposte! A Large Corpus of Counter-Arguments
arXiv preprint, arXiv:1910.03246
Fine-Grained Entity Typing in Hyperbolic Space
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL)
Aspects of Coherence for Entity Analysis
PhD thesis, Heidelberg University
2018
What's important in a text? An extensive evaluation of linguistic annotations for summarization
2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS)
BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
2017
Revisiting Selectional Preferences for Coreference Resolution
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Trust, but Verify! Better Entity Linking through Automatic Verification
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers
2015
Visual Error Analysis for Entity Linking
Proceedings of ACL-IJCNLP 2015 System Demonstrations
HITS at TAC KBP 2015: Entity discovery and linking, and event nugget detection
Proceedings of the Text Analysis Conference
2014
HITS' Monolingual and Cross-lingual Entity Linking System at TAC 2014
Proceedings of the Text Analysis Conference
Grants
- BOOST; JPY 49,960,000
- ISPF — International Collaboration Awards 2024; GBP 215,560 (project total)
- ABCI 3.0 開発加速利用
- Computational Modeling of Argumentation Understanding (Apr 2022–Mar 2027); JPY 260,000 (FY2022 allocation)
- Knowledge-Base-Grounded Language Models (Apr 2021–Mar 2024); JPY 4,550,000
Talks
Analyzing and Interpreting the Internals of Language Models
University of Tokyo, Applied Natural Language Processing; Guest lecture;
Linear Representations in LLM Interpretability: Claims and Evidence
TU Darmstadt & RIKEN AIP Joint Workshop; Research talk;
Talk
ICONIP 2025, AMED workshop;
Guest lecture
MBZUAI;
Research talk
LLM-jp;
Research talk
Nagoya NLP Seminar;
Research talk
KAIST;
Finding and Editing Representations of Numerical World Knowledge in Language Models.
MBZUAI; Research talk;
Prompting: Querying World Knowledge from Pretrained Language Models.
Stockmark Inc.; Invited talk;
Service
Conference reviewing
- 2025, 2026, 2027: Reviewer, ICLR
- 2025, 2026: Reviewer, COLM
- 2020, 2022, 2026: Reviewer, LREC
- 2020, 2022, 2025: Reviewer, COLING
- 2025: Reviewer, NeurIPS
- 2021, 2023: Reviewer, EACL
- 2018, 2021, 2022, 2023: Reviewer, EMNLP
- 2020, 2021: Reviewer, ACL
- 2021: Reviewer, IJCAI
- 2018, 2021: Reviewer, NAACL
Conference organization
- 2027: Tutorial Co-Chair, ACL
- 2025: Tutorial Chair, Organizing Committee, IJCNLP-AACL
Area chair
- 2024, 2025, 2026: Area Chair, ACL Rolling Review (ARR)
- 2026: Area Chair, LREC
- 2023: Area Chair, IJCNLP-AACL
Workshop reviewing
- 2025: Reviewer, ACL Student Research Workshop (SRW)
- 2020, 2021, 2023: Reviewer, SustaiNLP
Journals and review platforms
- 2021, 2022, 2023, 2024, 2025: Reviewer, ACL Rolling Review (ARR)
- 2021, 2022: Reviewer, Journal of Artificial Intelligence Research (JAIR)
- 2020: Reviewer, Journal of Cognitive Science
Reviewer awards
- 2021: Outstanding Reviewer, EACL
- 2021: Outstanding Reviewer, EMNLP
- 2020: Outstanding Reviewer, ACL
Other reviewing
- 2020: Reviewer, Book chapter: Word Classes in Computational Linguistics and Artificial Intelligence
Writing
NLP's Clever Hans Moment has Arrived
A review of Timothy Niven and Hung-Yu Kao, 2019: Probing Neural Network Comprehension of Natural Language Arguments.