Publications

You can find the complete list of articles on my Google Scholar or DBLP profiles.

2026

• Hakimov, S., Bernard, R., Leiber, T., Osswald, K., Richert, K., Yang, R., Bernardi, R., Schlangen, D. (2026). The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models. Findings of the Association for Computational Linguistics: EACL 2026, pages 529–570, Rabat, Morocco
  Paper
• Hakimov, S., Osswald, K., Psurek, J., Bukovszky, E., Lüser, A., Schlangen, D. (2026). Language Equality has a Price: A Systematic Investigation of Multi-turn LLM Performance for EU-24+. Preprint
  arXiv Git repo Leaderboard
• Mohiuddin, S., Chalamalasetti, K., Hakimov, S., Schlangen, D. (2026). Don’t Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference. Preprint
  arXiv
• Chalamalasetti, K., Hakimov, S., Schlangen, D. (2026). Multi-Turn Multi-Agent Dialogue for Collaborative Reconstruction Improves VLM Performance on Spatial Reasoning, But Only Barely. Preprint, under review
  arXiv
• Hakimov, S., D’Agostini, M., Samodelkin, I., Schlangen, D. (2026). The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue. Preprint, under review
  arXiv
• Yang, J., Feldhus, N., Mohtaj, S., Hennig, L., Wang, Q., Metheniti, E., Hakimov, S., Jakob, C., Solopova, V., Rieck, K., Schlangen, D., Möller, S., Schmitt, V. (2026). Order in the Evaluation Court: A Critical Analysis of NLG Evaluation Trends. Preprint, under review
  arXiv
• Hakimov, S. (2026). TurkicNLP: An NLP Toolkit for Turkic Languages. Preprint
  arXiv Git repo Project site

2025

• Hakimov, S., Abdullayeva, Y., Koshti, K., Schmidt, A., Weiser, Y., Beyer, A., Schlangen, D. (2025). Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models. Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), pages 5686–5718, Abu Dhabi, UAE
  Paper Leaderboard (tab Multimodal)
• Horst, N., Mazzaccara, D., Schmidt, A., Sullivan, M., Momentè, F., Franceschetti, L., Sadler, P., Hakimov, S., Testoni, A., Bernardi, R., Fernández, R., Koller, A., Lemon, O., Schlangen, D., Giulianelli, M., Suglia, A. (2025). Playpen: An Environment for Exploring Learning From Dialogue Game Feedback. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), Suzhou, China
  Paper Git repo
• Chalamalasetti, K., Hakimov, S., Schlangen, D. (2025). clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisations. Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2025), Avignon, France
  Paper
• Chalamalasetti, K., Hakimov, S., Schlangen, D. (2025). From Templates to Natural Language: Generalization Challenges in Instruction-Tuned LLMs for Spatial Reasoning. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL 2025)
  Paper
• Hakimov, S., Pfennigschmidt, L., Schlangen, D. (2025). Ad-hoc Concept Forming in the Game Codenames as a Means for Evaluating Large Language Models. Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²) at ACL 2025, Vienna, Austria
  Paper
• Hsu, Y.-S., Feldhus, N., Hakimov, S. (2025). Free-text Rationale Generation under Readability Level Control. Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²) at ACL 2025, Vienna, Austria
  Paper
• Jordan, J., Hakimov, S., Schlangen, D. (2025). Plant in Cupboard, Orange on Rably, Inat Aphone. Benchmarking Incremental Learning of Situation and Language Model using a Text-Simulated Situated Environment. Text, Speech, and Dialogue (TSD 2025), Lecture Notes in Computer Science vol. 16030, pages 51–63, Springer, Cham
  Paper arXiv
• Chalamalasetti, K., Hakimov, S., Schlangen, D. (2025). Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programming. ITL4HRI Workshop at IEEE RO-MAN 2025, Eindhoven, Netherlands
  PDF arXiv

2024

• Chalamalasetti, K., Hakimov, S., Schlangen, D. (2024). Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft. Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11159–11170, Miami, Florida, USA
  Paper
• Hakimov, S., Cheema, G.S. (2024). Unveiling Global Narratives: A Multilingual Twitter Dataset of News Media on the Russo-Ukrainian Conflict. Proceedings of the 2024 International Conference on Multimedia Retrieval (ICMR 2024), pages 1160–1164, Phuket, Thailand, ACM
  Paper Git repo Dataset
• Thakkar, G., Hakimov, S., Tadić, M. (2024). M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10833–10845, Torino, Italy
  Paper Git repo
• Sadler, P., Hakimov, S., Schlangen, D. (2024). Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 14770–14783, Torino, Italy
  Paper
• Sadler, P., Hakimov, S., Schlangen, D. (2024). Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game. Proceedings of the 4th Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP 2024) at ACL 2024, pages 17–29, Bangkok, Thailand
  Paper Git repo
• Hakimov, S., Weiser, Y., Schlangen, D. (2024). Evaluating Modular Dialogue System for Form Filling Using Large Language Models. Proceedings of the 1st Workshop on Simulating Conversational Intelligence in Chat (SCI-CHAT 2024) at EACL 2024, pages 36–52, St. Julians, Malta
  Paper Git repo
• Kastner, M.A., Cheema, G.S., Hakimov, S., Garcia, N. (2024). MUWS 2024: The 3rd International Workshop on Multimodal Human Understanding for the Web and Social Media. Proceedings of the 2024 International Conference on Multimedia Retrieval (ICMR 2024), pages 1342–1344, Phuket, Thailand, ACM
  Paper Workshop page
• Beyer, A., Chalamalasetti, K., Hakimov, S., Madureira, B., Sadler, P., Schlangen, D. (2024). clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents. Preprint, under review
  arXiv
• Bhavsar, N., Jordan, J., Hakimov, S., Schlangen, D. (2024). How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics. Preprint, under review
  arXiv

2023

• Chalamalasetti, K., Götze, J., Hakimov, S., Madureira, B., Sadler, P., Schlangen, D. (2023). clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), pages 11174–11219, Singapore
  Paper Git repo Clem Leaderboard
• Hakimov, S., Schlangen, D. (2023). Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks. Findings of the Association for Computational Linguistics: ACL 2023, pages 14196–14210, Toronto, Canada
  Paper Git repo
• Sadler, P., Hakimov, S., Schlangen, D. (2023). Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers. Findings of the Association for Computational Linguistics: ACL 2023, pages 9228–9239, Toronto, Canada
  Paper Git repo
• Cheema, G.S., Hakimov, S., Müller-Budack, E., Otto, C., Bateman, J.A., Ewerth, R. (2023). Understanding image-text relations and news values for multimodal news analysis. Frontiers in Artificial Intelligence, volume 6
  Paper
• Tahmasebzadeh, G., Hakimov, S., Ewerth, R., Müller-Budack, E. (2023). Multimodal Geolocation Estimation of News Photos. Advances in Information Retrieval – 45th European Conference on Information Retrieval (ECIR 2023), Lecture Notes in Computer Science vol. 13981, pages 204–220, Dublin, Ireland, Springer
  Paper Git repo
• Tahmasebzadeh, G., Müller-Budack, E., Hakimov, S., Ewerth, R. (2023). MM-Locate-News: Multimodal Focus Location Estimation in News. MultiMedia Modeling – 29th International Conference (MMM 2023), Lecture Notes in Computer Science vol. 13833, pages 204–216, Bergen, Norway, Springer
  Paper Git repo
• Tahmasebi, S., Hakimov, S., Ewerth, R., Müller-Budack, E. (2023). Improving Generalization for Multimodal Fake News Detection. Proceedings of the 2023 ACM International Conference on Multimedia Retrieval (ICMR 2023), pages 581–585, Thessaloniki, Greece, ACM
  Paper Git repo
• Cheema, G.S., Hakimov, S., Kastner, M.A., Garcia, N. (2023). MUWS’2023: The 2nd International Workshop on Multimodal Human Understanding for the Web and Social Media. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM 2023), pages 5263–5266, Birmingham, United Kingdom, ACM
  Paper Workshop page
• Alam, F., Barrón-Cedeño, A., Cheema, G.S., Shahi, G.K., Hakimov, S., Hasanain, M., Li, C., Míguez, R., Mubarak, H., Zaghouani, W., Nakov, P. (2023). Overview of the CLEF-2023 CheckThat! Lab Task 1 on Check-Worthiness in Multimodal and Multigenre Content. Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), CEUR Workshop Proceedings vol. 3497, pages 219–235, Thessaloniki, Greece
  Paper

2022

• Cheema, G.S., Hakimov, S., Sittar, A., Müller-Budack, E., Otto, C., Ewerth, R. (2022). MM-Claims: A Dataset for Multimodal Claim Detection in Social Media. Findings of the Association for Computational Linguistics: NAACL 2022, pages 962–979, Seattle, WA, USA
  Paper Git repo
• Hakimov, S., Cheema, G.S., Ewerth, R. (2022). TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) at NAACL 2022, pages 756–760, Seattle, WA, USA
  Paper Git repo
• Hakimov, S., Cheema, G.S., Kastner, M.A., Shah, R.R., Sikka, K. (2022). MUWS’22: 1st International Workshop on Multimodal Understanding for the Web and Social Media. Companion of The Web Conference 2022, pages 692–693, Lyon, France, ACM
  Paper Workshop page
• Demidova, E., Hakimov, S., Winters, J., Tadić, M. (2022). CLEOPATRA’22: 3rd International Workshop on Cross-lingual Event-centric Open Analytics. Companion of The Web Conference 2022, pages 497–498, Lyon, France, ACM
  Paper Workshop page

2021

• Müller-Budack, E., Theiner, J., Diering, S., Idahl, M., Hakimov, S., Ewerth, R. (2021). Multimodal news analytics using measures of cross-modal entity and context consistency. International Journal of Multimedia Information Retrieval, 10(2), pages 111–125
  Paper Git repo
• Müller-Budack, E., Springstein, M., Hakimov, S., Mrutzek, K., Ewerth, R. (2021). Ontology-driven Event Type Classification in Images. IEEE Winter Conference on Applications of Computer Vision (WACV 2021), pages 2927–2937, IEEE
  Paper Open access PDF Git repo
• Ilkou, E., Abu-Rasheed, H., Tavakoli, M., Hakimov, S., Kismihók, G., Auer, S., Nejdl, W. (2021). EduCOR: An Educational and Career-Oriented Recommendation Ontology. The Semantic Web – ISWC 2021, 20th International Semantic Web Conference, Lecture Notes in Computer Science vol. 12922, pages 546–562, Springer
  Paper Ontology page Git repo
• Tahmasebzadeh, G., Kacupaj, E., Müller-Budack, E., Hakimov, S., Lehmann, J., Ewerth, R. (2021). GeoWINE: Geolocation based Wiki, Image, News and Event Retrieval. SIGIR ‘21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2565–2569, ACM
  Paper Git repo
• Ghauri, J.A., Hakimov, S., Ewerth, R. (2021). Supervised Video Summarization Via Multiple Feature Sets with Parallel Attention. 2021 IEEE International Conference on Multimedia and Expo (ICME 2021), Shenzhen, China, IEEE
  Paper Git repo
• Kanafani, H., Ghauri, J.A., Hakimov, S., Ewerth, R. (2021). Unsupervised Video Summarization via Multi-source Features. ICMR ‘21: International Conference on Multimedia Retrieval, pages 466–470, Taipei, Taiwan, ACM
  Paper Git repo
• Pustu-Iren, K., Müller-Budack, E., Hakimov, S., Ewerth, R. (2021). Visualizing Copyright-Protected Video Archive Content Through Similarity Search. Linking Theory and Practice of Digital Libraries – 25th International Conference on Theory and Practice of Digital Libraries (TPDL 2021), Lecture Notes in Computer Science vol. 12866, pages 123–127, Springer
  Paper
• Cheema, G.S., Hakimov, S., Müller-Budack, E., Ewerth, R. (2021). A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods. Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding (MMPT ‘21) at ICMR, pages 37–45, Taipei, Taiwan, ACM
  Paper Git repo
• Hakimov, S., Ewerth, R. (2021). Combining Textual Features for the Detection of Hateful and Offensive Language. Working Notes of FIRE 2021 – Forum for Information Retrieval Evaluation, CEUR Workshop Proceedings vol. 3159, pages 412–418, Gandhinagar, India
  Paper Git repo
• Cheema, G.S., Hakimov, S., Müller-Budack, E., Ewerth, R. (2021). On the Role of Images for Analyzing Claims in Social Media. Proceedings of the 2nd International Workshop on Cross-lingual Event-centric Open Analytics (CLEOPATRA 2021) at The Web Conference 2021, CEUR Workshop Proceedings vol. 2829, pages 32–46
  Paper Git repo
• Demidova, E., Hakimov, S., Winters, J., Tadić, M. (eds.) (2021). Proceedings of the 2nd International Workshop on Cross-lingual Event-centric Open Analytics (CLEOPATRA 2021) co-located with the 30th The Web Conference (WWW 2021). CEUR Workshop Proceedings vol. 2829
  Proceedings Workshop page

2020

• Cheema, G.S., Hakimov, S., Ewerth, R. (2020). Check_square at CheckThat! 2020 Claim Detection in Social Media via Fusion of Transformer and Syntactic Features. Working Notes of CLEF 2020 – Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings vol. 2696, Thessaloniki, Greece
  Paper Git repo
• Cheema, G.S., Hakimov, S., Ewerth, R. (2020). TIB’s Visual Analytics Group at MediaEval ‘20: Detecting Fake News on Corona Virus and 5G Conspiracy. Working Notes Proceedings of the MediaEval 2020 Workshop, CEUR Workshop Proceedings vol. 2882
  Paper Git repo
• Ghauri, J.A., Hakimov, S., Ewerth, R. (2020). Classification of Important Segments in Educational Videos using Multimodal Features. Proceedings of the CIKM 2020 Workshops, CEUR Workshop Proceedings vol. 2699, Galway, Ireland
  Paper Git repo
• Tahmasebzadeh, G., Hakimov, S., Müller-Budack, E., Ewerth, R. (2020). A Feature Analysis for Multimodal News Retrieval. Proceedings of the 1st International Workshop on Cross-lingual Event-centric Open Analytics (CLEOPATRA 2020) at ESWC 2020, CEUR Workshop Proceedings vol. 2611, pages 43–56
  Paper Git repo
• Tavakoli, M., Hakimov, S., Ewerth, R., Kismihók, G. (2020). A Recommender System For Open Educational Videos Based On Skill Requirements. 20th IEEE International Conference on Advanced Learning Technologies (ICALT 2020), pages 1–5, Tartu, Estonia, IEEE
  Paper
• Demidova, E., Hakimov, S., Winters, J., Tadić, M. (eds.) (2020). Proceedings of the 1st International Workshop on Cross-lingual Event-centric Open Analytics (CLEOPATRA 2020) co-located with the 17th Extended Semantic Web Conference (ESWC 2020). CEUR Workshop Proceedings vol. 2611
  Proceedings Workshop page

2019

• Hakimov, S. (2019). Learning Multilingual Semantic Parsers for Question Answering over Linked Data. A comparison of neural and probabilistic graphical model architectures. Doctoral dissertation, Bielefeld University, Germany. Graded *magna cum laude*
  Record PDF Slides
• Hakimov, S., Jebbara, S., Cimiano, P. (2019). Evaluating Architectural Choices for Deep Learning Approaches for Question Answering Over Knowledge Bases. 13th IEEE International Conference on Semantic Computing (ICSC 2019), pages 110–113, Newport Beach, CA, USA, IEEE
  Paper Git repo

2017

• Hakimov, S., Jebbara, S., Cimiano, P. (2017). AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. The Semantic Web – ISWC 2017, 16th International Semantic Web Conference, Lecture Notes in Computer Science vol. 10587, pages 329–346, Vienna, Austria, Springer
  Paper Git repo
• Ell, B., Hakimov, S., Braukmann, P., Cazzoli, L., Kaupmann, F., Mancino, A., Altaf Memon, J., Rother, K., Saini, A., Cimiano, P. (2017). Towards a Large Corpus of Richly Annotated Web Tables for Knowledge Base Population. Proceedings of the 5th International Workshop on Linked Data for Information Extraction at ISWC 2017, CEUR Workshop Proceedings vol. 1946, pages 2–13, Vienna, Austria
  Paper Git repo

2016

• Hakimov, S., ter Horst, H., Jebbara, S., Hartung, M., Cimiano, P. (2016). Combining Textual and Graph-Based Features for Named Entity Disambiguation Using Undirected Probabilistic Graphical Models. Knowledge Engineering and Knowledge Management – 20th International Conference (EKAW 2016), Lecture Notes in Computer Science vol. 10024, pages 288–302, Bologna, Italy, Springer
  Paper Git repo
• Ell, B., Hakimov, S., Cimiano, P. (2016). Statistical Induction of Coupled Domain/Range Restrictions from RDF Knowledge Bases. Knowledge Graphs and Language Technology – ISWC 2016 International Workshops (KEKI and NLP&DBpedia), Revised Selected Papers, Lecture Notes in Computer Science vol. 10579, pages 27–40, Kobe, Japan, Springer
  Paper Git repo

2015

• Hakimov, S., Unger, C., Walter, S., Cimiano, P. (2015). Applying Semantic Parsing to Question Answering Over Linked Data: Addressing the Lexical Gap. Natural Language Processing and Information Systems – 20th International Conference on Applications of Natural Language to Information Systems (NLDB 2015), Lecture Notes in Computer Science vol. 9103, pages 103–109, Passau, Germany, Springer
  Paper Git repo

2014

• Dogdu, E., Hakimov, S., Yumusak, S. (2014). A data-model driven web application development framework. Proceedings of the 2014 ACM Southeast Regional Conference, article 47, Kennesaw, GA, USA, ACM
  Paper

2013

• Hakimov, S., Tunc, H., Akimaliev, M., Dogdu, E. (2013). Semantic question answering system over linked data using relational patterns. Joint EDBT/ICDT 2013 Workshops, pages 83–88, Genoa, Italy, ACM
  Paper
• Hakimov, S. (2013). Named Entity Disambiguation using Linked Open Data. Master’s thesis, TOBB University of Economics and Technology, Ankara, Turkey
  PDF

2012

• Hakimov, S., Oto, S.A., Dogdu, E. (2012). Named entity recognition and disambiguation using linked data and graph-based centrality scoring. Proceedings of the 4th International Workshop on Semantic Web Information Management (SWIM 2012), article 4, Scottsdale, AZ, USA, ACM
  Paper