Aspect-Based Sentiment Analysis Using Large Language Models on Museum Visitor Reviews

Anastasia V. Kolmogorova; Elizaveta R. Kulikova; Vladislav V. Lobanov

doi:10.14529/jsfi250309

Authors

Anastasia V. Kolmogorova HSE University, Saint-Petersburg, Russian Federation https://orcid.org/0000-0002-6425-2050
Elizaveta R. Kulikova HSE University, Saint-Petersburg, Russian Federation https://orcid.org/0000-0003-1996-737X
Vladislav V. Lobanov HSE University, Saint-Petersburg, Russian Federation https://orcid.org/0009-0005-3472-5696

DOI:

https://doi.org/10.14529/jsfi250309

Keywords:

museum reviews, aspect-based sentiment analysis, LLM, thematic categorization, prompting

Abstract

Museum reviews provide rich insight into visitor preferences and can drive useful change within institutions, yet they have attracted little attention in sentiment research owing to limited commercial interest and the multi-thematic nature of reviews. In this study we analysed over 12 000 reviews in Russian for 15 museum sites collected from nine different platforms. Methodologically, we first evaluated traditional approaches: a lexicon-based method utilising sentiment dictionaries and a neural network approach leveraging open-source pre-trained models such as RuBERT. While such methods can be applied to document-level sentiment analysis, where the text is labelled simply as positive or negative, they cannot uncover the specific topics that give rise to these sentiments. Finally, we implemented large language models (LLMs) for aspect-based sentiment analysis to discover positive and negative aspects visitors mention. Our system uses a two-step pipeline that initially extracts positive and negative keywords about each museum site and subsequently categorises these keywords into 14 predetermined categories, enabling the reader to effortlessly discover strong points and areas for improvement. Results include 15 csv tables of positive and negative keywords and 15 year-wise text reports for all objects. While some LLM hallucinations were observed, the outputs were largely realistic. We conclude that LLMs are well suited to this task and offer substantial scope for future research and practical applications in museum evaluation and service improvement.

References

Blinov, P.D., et al.: Research of lexical approach and machine learning methods for sentiment analysis. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference "Dialogue-2013". vol. 12(19), pp. 51–61 (2013)

Brauwers, G., Frasincar, F.: A survey on aspect-based sentiment classification. ACM Computing Surveys 54(1), 1–35 (2021). https://doi.org/10.1145/3503044

Choi, Y., Wiebe, J.: +/-effectwordnet: Sense-level lexicon acquisition for opinion inference. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1181–1191 (Oct 2014). https://doi.org/10.3115/v1/D14-1125

Feng, X., Wang, C., Zou, T.T.: Visitor experience of the grand canal national cultural park museum based on sentiment analysis algorithm. SSRG International Journal of Electrical and Electronics Engineering 11(9), 142–150 (2024). https://doi.org/10.14445/23488379/IJEEE-V11I9P112

Gao, Y., Wang, R., Hou, F.: How to Design Translation Prompts for ChatGPT: An Empirical Study. In: Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops. Association for Computing Machinery (2024). https://doi.org/10.1145/3700410.3702123

Gatti, L., Guerini, M., Turchi, M.: SentiWords: Deriving a high precision and high coverage lexicon for sentiment analysis. IEEE Transactions on Affective Computing 7(4), 409–421 (2015). https://doi.org/10.1109/TAFFC.2015.2476456

Hugging Face: MBARTRuSumGazeta-RuSentiment-RuReviews. https://huggingface.co/sismetanin/mbart_ru_sum_gazeta-ru-sentiment-rureviews

Hugging Face: Mistral-7B-v0.1-Q4_K_M-GGUF. https://huggingface.co/3dsabh/Mistral-7B-v0.1-Q4_K_M-GGUF

Hugging Face: RuBERT Conversational Cased Sentiment. https://huggingface.co/MonoHime/rubert_conversational_cased_sentiment

Hugging Face: RuBERT-Tiny2 Russian Sentiment. https://huggingface.co/seara/rubert-tiny2-russian-sentiment

Hugging Face: saiga_llama3_8b-Q4_K_M-GGUF. https://huggingface.co/itlwas/saiga_llama3_8b-Q4_K_M-GGUF

Hugging Face: SOLAR-10.7B-Instruct-v1.0-Q4_K_M-GGUF. https://huggingface.co/solxxcero/SOLAR-10.7B-Instruct-v1.0-Q4_K_M-GGUF

Hugging Face: Vikhr-7B-instruct_0.2-Q4_K_M-GGUF. https://huggingface.co/itlwas/Vikhr-7B-instruct_0.2-Q4_K_M-GGUF

Hugging Face: YandexGPT-5-Lite-8B-instruct-GGUF. https://huggingface.co/yandex/YandexGPT-5-Lite-8B-instruct-GGUF

Hugging Face: YandexGPT-5-Lite-8B-instruct-Q8_0-GGUF. https://huggingface.co/BoloniniD/YandexGPT-5-Lite-8B-instruct-Q8_0-GGUF

Koltsova, O.Y., Alexeeva, S.V., Kolcov, S.N.: An opinion word lexicon and a training dataset for russian sentiment analysis of social media. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference "Dialogue-2016". vol. 15(22), pp. 277–287 (2016). https://doi.org/10.5281/zenodo.4084953

Kulagin, D.: Russian word sentiment polarity dictionary: a publicly available dataset. Poster, Artificial Intelligence and Natural Language (AINL 2019) (2019). https://doi.org/10.28995/2075-7182-2021-20-1106-1119

Loukachevitch, N., Levchik, A.: Creating a General Russian Sentiment Lexicon. In: Proceedings of the Language Resources and Evaluation Conference (LREC 2016). pp. 1171–1176. European Language Resources Association (ELRA) (2016)

Loukachevitch, N., Tkachenko, N., Lapanitsyna, A., et al.: RuOpinionNE-2024: Extraction of Opinion Tuples from Russian News Texts (2025)

Pang, B., Lee, L.: Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval 2(1–2), 1–135 (2008). https://doi.org/10.1561/1500000011

Pang, B., Lee, L., Vaithyanathan, S.: Thumbs Up? Sentiment Classification Using Machine Learning Techniques. In: Mooney, R.J. (ed.) Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 79–86. Association for Computational Linguistics, Philadelphia, USA (2002). https://doi.org/10.3115/1118693.1118704

Qi, P., Zhang, Y., Zhang, Y., et al.: Stanza: A Python natural language processing toolkit for many human languages. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. pp. 101–108. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-demos.14

Qin, L., et al.: Large Language Models Meet NLP: A Survey. Frontiers of Computer Science (FCS) (2024). https://doi.org/10.1007/s11704-025-50472-3

Sheremetyeva, S.: An efficient patent keyword extractor as translation resource. In: Proceedings of the MT Summit XII: Third Workshop on Patent Translation. pp. 25–32. Ottawa, Canada (2009)

Tabularisai, Gyamfi, S., Borisov, V., Schreiber, R.H.: Multilingual-sentiment-analysis (revision 69afb83) (2025). https://doi.org/10.57967/hf/5968

Vatolin, A.: Structured sentiment analysis with large language models: A winning solution for RuOpinionNE-2024. In: Proceedings of the International Conference on Computational Linguistics and Intellectual Technologies (Dialogue 2025). FRC CSC RAS, Moscow, Russia (2025)

Wang, Z., Xie, Q., Feng, Y., et al.: Is chatGPT a good sentiment analyzer? In: First Conference on Language Modeling (2024), https://openreview.net/forum?id=mUlLf50Y6H

Xu, Q., Shih, J.Y.: Applying text mining techniques for sentiment analysis of museum visitor reviews. In: 2024 IEEE 4th International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB). pp. 270–274. Taipei, Taiwan (2024). https://doi.org/10.1109/ICEIB61477.2024.10602556

Yadav, A., Vishwakarma, D.: Sentiment analysis using deep learning architectures: A review. Artificial Intelligence Review 53(6), 4335–4385 (2020). https://doi.org/10.1007/s10462-019-09794-5

Zhang, T., Irsan, I., Thung, F., et al.: Revisiting sentiment analysis for software engineering in the era of large language models. ACM Transactions on Software Engineering and Methodology 34(3), 1–30 (2025). https://doi.org/10.1145/3697009

Água, M., Antonio, N., Carrasco, M.P., et al.: Large language models powered aspect-based sentiment analysis for enhanced customer insights. Tourism & Management Studies 21(1), 1–19 (2025). https://doi.org/10.18089/tms.20250101