Olivera Grljević, Zita Bošnjak, Saša Bošnjak

DOI Number
First page
Last page


Knowing what attracts or deters tourists to/from a tourist visit and what products to offer them and to pay special attention to is crucial for good economic results. Such knowledge can be obtained by analysis of online comments and reviews that tourists leave on travel websites (such as Booking, TripAdvisor, Trivago, etc.). This paper describes the value which information about opinions and emotions hidden in online reviews has for managers who receive it, especially the knowledge of (dis)satisfaction of users with certain aspects of the tourist offer. Uncovered knowledge from online reviews provides a chance to take advantage of the strong points, and correct the shortcomings through timely corrective measures and actions. Contemporary approaches and methods of analyzing online reviews and the opportunities for development they provide in the tourism industry are described through a case study conducted over a subset of 20491 hotel reviews from TripAdvisor. We have conducted sentiment analysis of reviews with the goal of building an automated model which will successfully distinguish positive from negative reviews. Logistic Regression classifier has the best performance, in 90% of reviews it has correctly classified positive reviews and in 83% negative. We have illustrated how association rules can help management to uncover relationships between concepts under discussion in negative and positive reviews.


online reputation management, е-word-of-mout, online reviews, automated classification, sentiment analysis, association rules

Full Text:



Agarwal, A., Biadsy, F. & Mckeown, K.R. (2009). Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, 24-32.

Alam, M.H., Ryu, W.-J. & Lee, S. (2016). Joint multi-grain topic sentiment: modeling semantic aspects for online reviews. Information Sciences, 339, 206–223.

Alm, C.O., Roth, D. & Sproat, R. (2005). Emotions from text: Machine learning for text-based emotion prediction. In Proceedings of the Joint Conference on HLT–EMNLP, Vancouver, Canada.

Aye, Y.M. & Aung, S.S. (2018). Senti-Lexicon and Analysis for Restaurant Reviews of Myanmar Text. International Journal of Advanced Engineering, Management and Science (IJAEMS), 4 (5).

Ballabio, D., Grisoni, F. & Todeschini, R. (2018). Multivariate comparison of classification performance measures. Chemometrics and Intelligent Laboratory Systems, 174, 33-44. doi:

Berrar, D. (2019). Performance Measures for Binary Classification. Encyclopedia of Bioinformatics and Computational Biology, 1, 546-560. doi:

Berthon, P.R., Pitt, L.F., Plangger, K. & Shapiro, D. (2012). Marketing meets Web 2.0, social media, and creative consumers: Implications for international marketing strategy. Business Horizons, 55 (3), 261-271.

Bibi, M. (2017). Sentiment Analysis at Document Level. Retrieved from, uploaded on 31 October 2017.

Bing, P. & Yang, Y. (2016). Monitoring and Forecasting Tourist Activities with Big Data. In: Management Science in Hospitality and Tourism. Theory, Practice, and Applications, New York, Apple Academic Press.

Broß, J. (2013). Aspect-Oriented Sentiment Analysis of Customer Reviews Using Distant Supervision Techniques, Dissertation at Freie Universität Berlin, Berlin.

Brooks, M., Kuksenok, K., Torkildson, M.K., Perry, D., Robinson, J.J., Scott, T.J., Anicello, O., Zukowski, A., Harris, P. & Aragon, C.R. (2013). Statistical affect detection in collaborative chat. In Proceedings of the 2013 conference on Computer supported cooperative work, pp. 317–328. ACM.

Bucur, C. (2015). Using Opinion Mining Techniques in Tourism, 2nd Global Conference on Business, Economics, Management and Tourism, Prague, Czech Republic. Procedia Economics and Finance, 23, 1666-1673.

Dave, K., Lawrence, S. & Pennock, D.M. (2003). Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews. In WWW '03 Proceedings of the 12th international conference on World Wide Web. ACM New York, 519-528.

Davidow, M. (2003). Organizational responses to customer complaints: What works and what doesn’t. Journal of Service Research, 5, 225-250.

Dos Santos, C.N. & Gatti, M. (2014). Deep convolutional neural networks for sentiment analysis of short texts. In COLING, 69–78.

Gligorijevic, B. & Luck, E. (2012). Engaging Social Customers – Influencing New Marketing Strategies for Social Media Information Sources. In Contemporary Research on E-business Technology and Strategy, 25-40. Springer Berlin Heidelberg.

Grljević, O. (2016). Sentiment u sadržajima sa društvenih mreža kao instrument unapređenja poslovanja visokoškolskih institucija. Univerzitet u Novom Sadu, Ekonomski fakultet u Subotici, doktorska disertacija.

Grljević, O. & Bošnjak, Z. (2018). Sentiment analysis of customer data. International Journal of Strategic Management and Decision Support Systems in Strategic Management, 23(3), 38-49.

Grljević, O. & Bošnjak, Z. (2018). Evaluating customer satisfaction through online reviews and ratings. In V. Bevanda & S. Štetić (Eds.) 3rd International Thematic Monograph – Thematic Proceedings: Modern Management Tools and Economy of Tourism Sector in Present Era. Belgrade. Belgrade, Serbia: Association of Economists and Managers of the Balkans in cooperation with the Faculty of Tourism and Hospitality, Ohrid, Macedonia. ISBN: 978-86-80194-14-1

Gruen, T.W., Osmonbekov, T. & Czaplewski, A.J. (2006). eWOM: The impact of customer-to-customer online know-how exchange on customer value and loyalty. Journal of Business Research, 59 (4), 449-456.

Holzman, L.E. & Pottenger, W.M. (2003). Classification of emotions in internet chat: An application of machine learning using speech phonemes. Tech. rep., Leigh University.

Hu, M. & Liu, B. (2004). Mining Opinion Features in Customer Reviews. Proceedings of the 19th International Conference on Artificial Intelligence AAAI'04, 755-760.

Ikeda, D., Takamura, H., Ratinov, L.-A. & Okumura, M. (2008). Learning to Shift the Polarity of Words for Sentiment Classification. Transactions of the Japanese Society for Artificial Intelligence, 25 (1), 50-57.

Jimenez, A., Berzal, F. & Cubero, J.C. (2010). Interestingness Measures for Association Rules within Groups. In E. Hullermeier, R. Kruse, and F. Hoffmann (Eds.): IPMU 2010, Part I, CCIS 80, pp. 298–307. Available at:

Jurafsky, D. & Martin, J.H. (2018). Speech and Language Processing. An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Draft of September 23, 2018.

Kim, S.M. & Hovy, E. (2004). Determining the Sentiment of Opinions. In COLING '04 Proceedings of the 20th international conference on Computational Linguistic.

Lee, Y.L. & Song, S. (2010). An empirical investigation of electronic word-of-mouth: Informational motive and corporate response strategy. Computers in Human Behavior, 26 (5), 1073-1080.

Lei, S. & Law, R. (2015). Content Analysis of TripAdvisor Reviews on Restaurants: A Case Study of Macau. Journal of tourism, 16 (1), 17-28.

Leung, D., Law, R., van Hoof, H. & Buhalis, D. (2013). Social media in tourism and hospitality: A literature review. Journal of Travel and Tourism Marketing, 30 (1-2), 3–22.

Li, X., Bing, L., Li, P., Lam, W. & Yang, Z. (2018). Aspect Term Extraction with History Attention and Selective Transformation. IJCAI 2018, Computation and Language, arXiv:1805.00760

Litvin, S., Goldsmith, R.E., Pan, B. (2008). Electronic word-of-mouth in hospitality and tourism management. Tourism Management, 29 (2008), 458–468.

Ma, Y., Xiang, Z., Du, Q. & Fan, W. (2018). Effects of user-provided photos on hotel review helpfulness: An analytical approach with deep leaning. International Journal of Hospitality Management, 71, 120-131.

Marrese-Taylor, E., Velásquez, J.D., Bravo-Marquez, F. & Matsuo, Y. (2013). Identifying Customer Preferences about Tourism Products using an Aspect-Based Opinion Mining Approach, Procedia Computer Science 22, 182-191, 17th International Conference in Knowledge Based and Intelligent Information and Engineering Systems - KES2013.

Medhat, W., Hassan, A. & Korashy, H. (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5, 1093-1113.

Mohammad, S. (2012). Portable features for classifying emotional text. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 587–591, Montreal, Canada.

Mohammad, S.M. (2012a). #emotional tweets. In Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, SemEval ’12, pp. 246–255, Stroudsburg, PA.

Nakagawa, K. Inui. & S. Kurohashi. (2010). Dependency tree-based sentiment classification using CRFs with hidden variables. In NAACL, HLT

Öğüt, H. & Onur, T.B.K. (2012). The influence of internet customer reviews on the online sales and prices in hotel industry. The Service Industries Journal, 32 (2), 197 – 214.

Pang, B. & Lee, L. (2008). Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval, 2 (1-2), 1-135.

Pang, B., Lee, L. & Vaithyanathan, S. (2002). Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing, Volume 10. Association for Computational Linguistics, 79–86.

Park, D.-H., Lee, J. & Han, I. (2007). The Effect of On-Line Consumer Reviews on Consumer Purchasing Intention: The Moderating Role of Involvement. International Journal of Electronic Commerce, 11 (4), 125-148.

Phillips, P., Zigan, K., Santos Silva, M.M. & Schegg, R. (2015). The interactive effects of online reviews on the determinants of Swiss hotel performance: A neural network analysis. Tourism Management, 50, 130-141

Pitt, L.F., Berthon, P.R., Watson, R.T. & Zinkhan, G.M. (2002). The Internet and the birth of real consumer power. Business Horizons, 45 (4), 7-14.

Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Mohammad, A. S. & Hoste, V. (2016). SemEval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 19-30.

Puri, C.A., Kush, G., Kumar, N. (2017) Opinion Ensembling for Improving Economic Growth through Tourism. Procedia Computer Science 122, 237-244.

Racherla, P., Connolly, D.J. & Christodoulidou, N. (2013). What Determines Consumers' Ratings of Service Providers? An Exploratory Study of Online Traveler Reviews. Journal of Hospitality Marketing and Management, 22 (2), 135-161, doi: 10.1080/19368623.2011.645187

Socher, R., Pennington, J., Huang, E.H., Andrew Y. Ng & Manning, C.D. (2011). Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions, Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK, July 27–31, 151–161,

Schuckert, M.L. & Law, X.R. (2014): Hospitality and tourism online reviews: Recent trends and future directions. Journal of Travel and Tourism Marketing, 32 (5), 608-621, doi:10.1080/10548408.2014.933154

Sparks, B.A., FungSo, K.K. & Bradley, G.L. (2016). Responding to negative online reviews: The effects of hotel responses on customer inferences of trust and concern. Tourism Management, 53, 74-85.

Strapparava, C. & Mihalcea, R. (2007). Semeval-2007 task 14: Affective text. In Proceedings of SemEval-2007, pp. 70–74, Prague, Czech Republic.

Suttles, J. & Ide, N. (2013). Distant supervision for emotion classification with discrete binary values. In Computational Linguistics and Intelligent Text Processing, pp. 121– 136. Springer.

Tjahyanto, A. & Sisephaputra, B. (2017). The Utilization of Filter on Object-Based Opinion Mining in Tourism Product Reviews. Procedia Computer Science, 124, 38-45.

Tripp, T. M. & Grégoire, Y. (2011). When Unhappy Customers Strike Back on the Internet. MIT Sloan Management Review, 52 (3), 37-44.

Turney, P.D. (2002). Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In proceedings of the 40th annual meeting on association for Computational Linguistics (ACL'02), Philadelphia, Pennsylvania, USA.

Visa, G.P. & Salembier, P. (2014). Precision-Recall-Classification Evaluation Framework: Application to Depth Estimation on Single Images. In D. Fleet et al. (Eds.): ECCV 2014, Part I, LNCS 8689, pp. 648–662. Available at:

Xie, K.L., Kam Fung So, K. & Wang, W. (2017). Joint effects of management responses and online reviews on hotel financial performance: A data-analytics approach. International Journal of Hospitality Management, 62, 101-110.

Yang, C.-S., Chen, C.-H. & Chang, P.-C. (2015). Harnessing consumer reviews for marketing intelligence: a domain-adapted sentiment classification approach. Information Systems and e-Business Management, 13 (3), 403-419.

Yang, Y. & Loog, M. (2018). A benchmark and comparison of active learning for logistic regression. Pattern Recognition, 83, 401-415.

Zhang, Y., Lai, G., Zhang, M., Zhang, Y., Liu, Y. & Ma, S. (2014). Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, 83-92.



  • There are currently no refbacks.

© University of Niš, Serbia
Creative Commons License CC BY-NC-ND
ISSN 0354-4699 (Print)
ISSN 2406-050X (Online)