Publications

You can find the complete list of articles on my Google Scholar or DBLP profiles.

2026

• Hakimov, S. (2026) TurkicNLP: An NLP Toolkit for Turkic Languages
  Arxiv prerprint Git Repo

• Hakimov, S., Bernard R, Leiber T, Osswald K, Richert K, Yang R, Bernardi R, Schlangen D. (2026). The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models. The 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL)
  PDF

• Yang, J., Feldhus, N., Mohtaj, S., Hennig, L., Wang, Q., Metheniti, E., Hakimov, S., Jakob, C., Solopova, V., Rieck, K., Schlangen, D., Möller, S., Schmitt, V. (2026). Order in the Evaluation Court: A Critical Analysis of NLG Evaluation Trends.
  PDF

2025

• Hakimov, S., Abdullayeva, Y., Koshti, K., Schmidt, A., Weiser, Y., Beyer, A., & Schlangen, D. (2025). Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models. The 31st International Conference on Computational Linguistics (COLING 2025)
  PDF Leaderboard (tab Multimodal)

• Horst, N. and Mazzaccara, D. and Schmidt, A. and Sullivan, M. and Momentè, F. and Franceschetti, L. and Sadler, P. and Hakimov, S. and Testoni, A. and Bernardi, R. and Fernández, R. and Koller, A. and Lemon, O. and Schlangen, D. and Giulianelli, M. and Suglia, A. (2025). Playpen: An Environment for Exploring Learning Through Conversational Interaction. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP 2025)
  PDF Git repo

• Kranti, C., Hakimov, S. and Schlangen, D. (2025). clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisations. Special Interest Group on Discourse and Dialogue (SIGDIAL 2025)
  PDF

• Kranti, C., Hakimov, S. and Schlangen, D. (2025). Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programming. ITL4HRI Workshop co-located with International Conference on Robot and Human Interactive Communication (RO-MAN)
  PDF

• Jordan, J., Hakimov, S., Schlangen, D. (2025). Plant in Cupboard, Orange on Rably, Inat Aphone. Benchmarking Incremental Learning of Situation and Language Model using a Text-Simulated Situated Environment. In: Text, Speech, and Dialogue. TSD 2025. Lecture Notes in Computer Science(), vol 16030. Springer, Cham. https://doi.org/10.1007/978-3-032-02551-7_6
  PDF

• Hakimov, S., Pfennigschmidt, L., & Schlangen, D. (2025). Ad-hoc Concept Forming in the Game Codenames as a Means for Evaluating Large Language Models. GEM2 Workshop: Generation, Evaluation & Metrics co-located at ACL 2025
  PDF

• Hsu, Y., Feldhus, N., & Hakimov, S. (2024). Free-text Rationale Generation under Readability Level Control. GEM2 Workshop: Generation, Evaluation & Metrics co-located at ACL 2025
  PDF

• Kranti, C., Hakimov, S. and Schlangen, D. (2025). From Templates to Natural Language: Generalization Challenges in Instruction-Tuned LLMs for Spatial Reasoning. IJCNLP-AACL 2025.
  PDF

2024

• Kranti, C., Hakimov, S. and Schlangen, D. (2024). Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)
  PDF

• Bhavsar, N., Jordan, J., Hakimov, S., & Schlangen. D. (2024). How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics.
  arXiv preprint arXiv:2406.14051

• Beyer, A., Chalamalasetti, K., Hakimov, S., Madureira, B., Sadler, P., & Schlangen, D. (2024). clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents.
  arXiv preprint arXiv:2405.20859

• Hakimov, S., and Cheema, G.S., (2024). Unveiling Global Narratives: A Multilingual Twitter Dataset of News Media on the Russo-Ukrainian Conflict. ACM International Conference on Multimedia Retrieval (ICMR) 2024
  PDF Git repo Dataset

• Thakkar, G., Hakimov, S., and Tadic, M., (2024). M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets. LREC-COLING 2024
  PDF Git Repo

• Sadler, P., Hakimov, S., and Schlangen, D., (2024). Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies. LREC-COLING 2024
  PDF

• Sadler, P., Hakimov, S., and Schlangen, D., (2024). Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game. Work presented at the “Cooperative Multi-Agent Systems Decision-making and Learning” workshop (AAAI’24)
  PDF Git repo

• Hakimov, S., Weiser, Y., and Schlangen, D., (2024). Evaluating Modular Dialogue System for Form Filling Using Large Language Models. In Proceedings of the 1st Workshop on Simulating Conversational Intelligence in Chat (SCI-CHAT 2024) at EACL 2024, pages 36–52, St. Julians, Malta. Association for Computational Linguistics.
  PDF Git repo

2023

• Chalamalasetti, K., Götze, J., Hakimov, S., Madureira, B., Sadler, P., and Schlangen, D., (2023). clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)
  PDF Git repo Clem Leaderboard

• Hakimov, S., and Schlangen, D., (2023). Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks. Findings of the Association for Computational Linguistics: ACL 2023
  PDF Git repo

• Alam F, Barrón-Cedeño A, Cheema GS, Hakimov S, Hasanain M, Li C, Míguez R, Mubarak H, Shahi GK, Zaghouani W, Nakov P., (2023). Overview of the CLEF-2023 CheckThat! Lab Task 1 on Check-Worthiness of Multimodal and Multigenre Content. Working Notes of CLEF
  PDF Challenge Website

• Tahmasebi, S., Hakimov, S., Ewerth, R., and Müller-Budack, E., (2023). Improving Generalization for Multimodal Fake News Detection. International Conference on Multimedia Retrieval (ICMR 2023)
  PDF Git repo

• Sadler, P., Hakimov, S., and Schlangen, D., (2023). Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers. Findings of the Association for Computational Linguistics: ACL 2023
  PDF Git repo

• Cheema, G.S., Hakimov, S., Müller-Budack, E., Otto, C., Bateman, J., and Ewerth, R., (2023). Understanding Image-Text Relations and News Values for Multimodal News Analysis. Frontiers in Artificial Intelligence, Sec. Language and Computation, Volume 6
  PDF DOI

• Tahmasebzadeh, G., Hakimov, S., Ewerth, R., and Müller-Budack, E., (2023). Multimodal Geolocation Estimation of News Photos. Advances in Information Retrieval - 45th European Conference on Information Retrieval (ECIR 2023)
  PDF Git repo

• Tahmasebzadeh, G., Müller-Budack, E., Hakimov, S., and Ewerth, R., (2023). MM-Locate-News: Multimodal Focus Location Estimation in News. 29th International Conference on Multimedia Modeling (MMM 2023)
  PDF Git repo

2022

• Hakimov, S., Cheema, G.S., Kastner, M.A., Shah, R.R., Sikka, K., (2022). MUWS’22: 1st International Workshop on Multimodal Understanding for the Web and Social Media. WWW (Companion Volume) 2022: 692-693
  URL

• Cheema, G.S., Hakimov, S., Sittar, A., Müller-Budack, E., Otto C. and Ewerth, R., (2022). MM-Claims: A Dataset for Multimodal Claim Detection in Social Media. Findings of the North American Chapter of the Association for Computational Linguistics (Findings of NAACL 2022)
  PDF Git repo

• Hakimov, S., Cheema G.S. and Ewerth, R., (2022). TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes. SemEval-2022, Task 5: MAMI - Multimedia Automatic Misogyny Identification co-located with NAACL 2022
  PDF Git repo

• Demidova, E., Hakimov, S., Winters, J. and Tadic, M., (2022). CLEOPATRA’22: 3rd International Workshop on Cross-lingual Event-centric Open Analytics. WWW (Companion Volume) 2022: 497-498
  URL

2021

• Hakimov, S. and Ewerth, R., (2021). Combining Textual Features for the Detection of Hateful and Offensive Language. HASOC 2021 - Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages, co-located with Forum for Information Retrieval Evaluation (FIRE)
  PDF Git repo

• Pustu-Iren, K., Müller-Budack, E., Hakimov, S. and Ewerth, R., (2021). Visualizing Copyright-Protected Video Archive Content Through Similarity Search. In the Proceedings of the International Conference on Theory and Practice of Digital Libraries (TPDL 2021)
  PDF

• Cheema, G.S., Hakimov, S., Müller-Budack, E. and Ewerth, R., (2021). A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods. In the Proceedings of the Multi-Modal Pre-Training for Multimedia Understanding (MMPT 2021), co-located with ICMR
  PDF Git repo

• Ilkou, E., Abu-Rasheed, H., Tavakoli, M., Hakimov, S., Kismihók, G., Auer, S. and Nejdl, W., (2021). EduCOR: An Educational and Career-Oriented Recommendation Ontology. In the Proceedings the 20th International Semantic Web Conference (ISWC 2021)
  PDF Ontology page Git repo

• Kanafani, H., Ghauri, J.A., Hakimov, S. and Ewerth, R., (2021). Unsupervised Video Summarization via Multi-source Features. In the Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR)
  PDF Git repo

• Tahmasebzadeh, G., Kacupaj, E., Müller-Budack, E., Hakimov, S., Lehmann, J. and Ewerth, R., (2021). GeoWINE: Geolocation based Wiki, Image,News and Event Retrieval. In the Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval 2021 (SIGIR)
  PDF DEMO Git repo

• Müller-Budack, E., Theiner, J., Diering, S., Idahl, M., Hakimov, S. and Ewerth, R., (2021). Multimodal news analytics using measures of cross-modal entity and context consistency. *International Journal of Multimedia Information Retrieval*, pp.1-15.
  PDF Git repo

• Ghauri, J.A., Hakimov, S. and Ewerth, R., (2021). Supervised Video Summarization via Multiple Feature Sets with Parallel Attention. In the Proceedings of IEEE International Conference on Multimedia and Expo (ICME)
  PDF Git repo

• Cheema, G.S., Hakimov, S., Müller-Budack, E. and Ewerth, R., (2021). On the Role of Images for Analyzing Claims in Social Media. In the Proceedings of the CLEOPATRA workshop co-located with The Web Conference (WWW)
  PDF Git repo

• Demidova, E., Hakimov, S., Winters, J. and Tadic, M., (2021). Proceedings of the 2nd International Workshop on Cross-lingual Event-centric Open Analytics co-located with the 30th The Web Conference (WWW 2021).
  CEUR Vol-2829 Workshop page

• Müller-Budack, E., Springstein, M., Hakimov, S., Mrutzek, K. and Ewerth, R., (2021). Ontology-driven Event Type Classification in Images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
  PDF Git repo

2020

• Demidova, E., Hakimov, S., Winters, J. and Tadic, M., (2020). Proceedings of the 1st International Workshop on Cross-lingual Event-centric Open Analytics (CLEOPATRA 2020) co-located with the 17th Extended Semantic Web Conference (ESWC 2020).
  CEUR Vol-2611 Workshop page

• Cheema, G.S., Hakimov, S. and Ewerth, R., (2020). TIB’s Visual Analytics Group at MediaEval’20: Detecting Fake News on Corona Virus and 5G Conspiracy. MediaEval workshop FakeNews task
  PDF Git repo

• Ghauri, J.A., Hakimov, S. and Ewerth, R., (2020). Classification of Important Segments in Educational Videos using Multimodal Features. In Proceedings of the CIKM 2020 Workshops
  PDF Git repo

• Tahmasebzadeh, G., Hakimov, S., Müller-Budack, E. and Ewerth, R., (2020). A Feature Analysis for Multimodal News Retrieval. In the Proceedings of the CLEOPATRA workshop co-located with Extended Semantic Web Conference (ESWC)
  PDF Git repo

• Cheema, G.S., Hakimov, S. and Ewerth, R., (2020). Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Features. In Proceedings of the Eleventh International Conference of the CLEF Association
  PDF Git repo

• Tavakoli, M., Hakimov, S., Ewerth, R. and Kismihók, G., (2020). A Recommender System For Open Educational Videos Based On Skill Requirements. In Proceedings of International Conference on Advanced Learning Technologies (ICALT)
  PDF

2019

• Hakimov, S. (2019). Learning Multilingual Semantic Parsers for Question Answering over Linked Data. A comparison of neural and probabilistic graphical model architectures. Bielefeld University, Germany (Doctoral Dissertation) Graded as *magna cum laude*
  URI PDF Slides

• Hakimov, S. , Jebbara, S., Cimiano, P. (2019). Evaluating Architectural Choices for Deep Learning Approaches for Question Answering over Knowledge Bases. In Proceedings of the 13th International Semantic Computing Conference (ICSC)
  PDF Git repo

2017

• Ell B, Hakimov, S. , Braukmann, P., Cazzoli, L., Kaupmann, F., Mancino, A., Altaf Memon, J., Rother, K., Saini, A., Cimiano, P. (2017). Towards a Large Corpus of Richly Annotated Web Tables for Knowledge Base Population. In Proceedings of 5th International Workshop on Linked Data for Information Extraction, co-located with the 16th International Semantic Web Conference (ISWC)
  PDF Git repo

• Hakimov, S., Jebbara, S., Cimiano, P. (2017). AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. In Proceedings of the 16th International Semantic Web Conference (ISWC)
  PDF Git repo

2016

• Ell, B., Hakimov, S., Cimiano, P. (2016). Statistical Induction of Coupled Domain/Range Restrictions from RDF Knowledge Bases. In Proceedings of 4th NLP and DBpedia Workshop, co-located with the 15th International Semantic Web Conference (ISWC)
  PDF Git repo

• Hakimov, S., ter Horst, H., Jebbara, S., Hartung, M., Cimiano, P. (2016). Combining textual and graph-based features for named entity disambiguation using undirected probabilistic graphical models. In Proceedings of 20th International Knowledge Engineering and Knowledge Management Conference (EKAW)
  PDF Git repo

2015

• Hakimov, S., Unger, C., Walter, S., Cimiano, P. (2015). Applying semantic parsing to question answering over linked data: Addressing the lexical gap. In Proceedings of International Conference on Applications of Natural Language to Information Systems (NLDB)
  PDF Git repo

2014

• Dogdu, E., Hakimov, S., Yumusak, S. (2014). A data-model driven web application development framework. In Proceedings of the 2014 ACM Southeast Regional Conference
  PDF

2013

• Hakimov, S. (2013). Named Entity Disambiguation using Linked Open Data. TOBB University, Ankara, Turkey (Master’s Thesis)
  PDF

• Hakimov, S., Tunc, H., Akimaliev, M., Dogdu, E. (2013). Semantic question answering system over linked data using relational patterns. In Proceedings of the Joint EDBT/ICDT 2013 Workshops
  PDF

2012

• Hakimov, S., Oto, S. A., Dogdu, E. (2012). Named entity recognition and disambiguation using linked data and graph-based centrality scoring. In Proceedings of the 4th international workshop on semantic web information management
  PDF