nueva página del texto (beta)
Inglés (pdf)
Artículo en XML
Referencias del artículo
Traducción automática
Enviar artículo por email
Citado por SciELO 
Similares en
SciELO 
Permalink
Introduction
A chatbot is a computer program that responds as an intelligent entity when spoken to via text or voice. The use of artificial intelligence (AI) techniques is essential in the natural language processing (NLP) capabilities of this software, as it allows computers to understand, generate, and manipulate human language (Adamopoulou & Moussiades, 2020). Currently, AI has positioned itself among the leading digital marketing trends since 2017, including the tourism sector as an application area, one of the most communicative industries with its users (van Esch & Black, 2021). As the adoption of chatbots increases, mobile applications lose weight. This interaction technology is emerging as a platform used by millions of consumers around the world due, in part, to the commodification of natural language services, which provide developers with many building blocks to build chatbots inexpensively (Adamopoulou & Moussiades, 2020; Brandtzaeg & Følstad, 2017). However, it is still challenging to build and implement chatbots, especially when they are not developed for English-speaking users. Developers must handle the coordination of cognitive services to build the chatbot interface, integrate the chatbot with external services, and worry about extensibility, scalability, and maintainability.
Chatbots can be implemented for online orders, product suggestions, customer service, climate, personal financial assistance, and other options, which causes companies to trust their implementation. According to Rivero (2019), in 2018, 15% of consumers in 195 countries of the world already used chatbot channels, 38% had already started conversations with a commercial website using chat, 30% used the mobile application (which also usually includes chat and chatbot functions), and 28% used social networks (which can also incorporate chatbot functions). The reason is that chatbots allow direct interaction with the consumer and provide marketing teams with valuable data about their preferences, making them a fascinating tool for companies; additionally, users are happy with their service. 68% value them positively for being a service open 24 hours, and 55% prefer them for the speed of response (Rivero, 2019).
At present, clients demand more efficiency; therefore, they opt for a self-service model to obtain autonomy to carry out procedures without interacting with another person. As data availability increases for consumer-facing platforms like blogging, speech recognition, or social media, marketing professionals must learn to use artificial intelligence tools to classify data automatically and spend their valuable time capitalizing on the resulting knowledge (Ukpabi et al., 2019). On the other hand, the growth in recent years of this type of technology has considerably increased the intention of users to use chatbots in different contexts, including travel and tourism (Melián-González et al., 2021).
Interaction with chatbots is an excellent opportunity to explore new forms of digital advertising and connect with potential customers in a new way. The chatbot generates a conversation that is usually more creative than simply viewing a video or an advertisement. This interaction with the brand, through chat, can generate an entertaining experience for the user, a positive emotional feeling towards the brand, and a revaluation of the brand's values. Within marketing, chatbots will save money and incentivize customer purchases. This tool will direct customers to the products that have been recommended for them (Chung et al., 2020; Nair & Gupta, 2021).
In the hospitality industry, technology has played an increasingly important role in attracting and retaining guests, particularly with the Millennial generation. Hotels today must invest heavily in technology solutions to create a more personalized experience. Reservations must be made effortlessly through intelligent systems, guestrooms must encourage any sort of content, the internet must be stable and secure, and customer information services must be available at any time and without delay (Dash & Bakshi, 2019).
The use of chatbots has been growing in electronic commerce, especially in those products and services that by their nature represent a more significant number of doubts or questions from users (Nichifor et al., 2021); in this sector we can encompass the tourism sector, representing our project's development area. In particular, the use of chatbots in hotels, according to the perspective of technology providers, has received positive feedback and the benefits of chatbots outweigh the challenges. This scenario is leading to boost the chatbots' full potential in the hotel industry (Buhalis & Cheng, 2020).
Nowadays, the hospitality and tourism industry has adopted and evaluated the use of AI technology-based services in many aspects (Lehto et al., 2021; Li et al., 2021; Ma et al., 2021), and chatbots are not the exception. They are a recent technology that hotels, travel agencies, and airline companies are adopting. For example, Melián-González et al. (2021) revealed that the factors that influenced the adoption of a chatbot mainly are: the chatbots’ expected performance, the habit of using chatbots, the hedonic component in using them, the predisposition to using self-service technologies, the social influences, and the fact that the chatbot behaves like a human. Pillai & Sivathanu (2020) showed that the predictors of chatbot adoption intention are the perceived ease of use, perceived usefulness, perceived trust, perceived intelligence, and anthropomorphism. Also, they indicated that technological anxiety does not influence the adoption intention. Similarly, Cheng & Jiang (2020) indicated that utilization, entertainment, technology, and social presence from chatbot use are positively predicted users’ satisfaction. In contrast, perceived privacy risk associated with chatbot use reduced user satisfaction.
It is essential to mention that the studies presented above are mainly addressed to analyze the usage and adaption of a chatbot in the hospitality industry. Some studies address the challenges of chatbot development (Chiaráin & Chasaide, 2016; Tavanapour & Bittner, 2018). Additionally, even though the most common language used to develop a chatbot is English, there are some studies for Spanish chatbots (Herrero-Diz & Varona-Aramburu, 2018; Segura et al., 2019; Vanjani et al., 2019; Taylor et al., 2021); however, to the best of our knowledge, there are no works on the design, development, and evaluation of a chatbot for hospitality services in Spanish.
This work aims to develop a natural language dialogue system that allows guests, customers, and the public to interact with an automatic system specialized in the Spanish language, which provides relevant information on products, services, events, and sites of interest in a hotel. The user interface is based on text interaction, and the response is obtained in real-time. The project focuses on creating a platform that large hotels could benefit from. We performed interviews with Mexican hoteliers to define the platform's objectives, requirements, and features. In this way, we developed a web interface where the information provided by the chatbot is real-time updated with the information provided by the hotel staff. Based on an interview conducted with several hoteliers in Nayarit, Mexico, the benefits that can be obtained with the implementation of a chatbot oriented to the hotel sector are the following:
Faster decision-making: customer prospects can obtain the information they want, encouraging an immediate reservation.
Personalization: customer or potential customer service is personalized whenever they wish.
Speed-up administration: reducing the time of customer service processes.
Availability: eliminating restrictions for customer service hour.
Multi-platform: allowing users to access through a website and different mobile applications.
Service quality: improving the communication with potential customers and hotel guests.
The proposed software will automatically interact with customers and prospects of hotels, providing relevant information about products, services, events, and sites of interest. This proposal offers new ways to design an interactive means of contact with clients of hotels in Spanish and provides the most relevant information on each of the services managed by the hotel.
Materials and methods
Modularization based on conversation topics
The first step in this research was to validate the need for automated customer service tools among hotels in Mexico. To accomplish this, we conducted a study in the state of Nayarit, Mexico. We interviewed staff members of six hotels in the region and asked a series of 24 questions about their means of interacting with guests. Our questions covered topics such as the hotels' use of text messages, the type of information they provided through chat, their hours of service, their customer service improvement strategies, the resources invested in customer service, and their interest in adopting new technology for customer service. The complete list of questions can be found in Appendix A.
After analyzing the answers to this questionnaire, we concluded that an automated written interaction software is a viable option since hotels frequently interact with customers by chat. This mode of interaction is the most used after the telephone and help desk service. Currently, all the interviewed staff members of hotels use WhatsApp, Telegram, or Facebook to communicate (Figure 1). Four out of the six interviewed hotels have been using this interaction for more than 10 years. Five out of six hotels consider that they have lost customers due to the current performance of their customer service (for instance, how quickly they offer information and how accurate the answer is). Four out of six are looking for a solution to make their service more efficient. The six hotels are willing to acquire and invest in technology depending on its performance and the price (Table 1).
Table 1 Hoteliers’ answers to some questions related to their need for technology to provide customer service.
| Hotels | Years of using interaction |
Lost customers due to customer service |
Looking for a solution |
Willing to invest |
| 1 | More than 10 | Yes | Yes | Yes |
| 2 | More than 10 | Yes | Yes | Yes |
| 3 | Less than 10 | Yes | Yes | Yes |
| 4 | More than 10 | Yes | Yes | Yes |
| 5 | More than 10 | Yes | No | Yes |
| 6 | More than 10 | No | Yes | Yes |
Source: Author’s own elaboration.
Likewise, through the analysis of the hoteliers' answers, the following problems related to the customer service process were detected:
Having fixed office schedule conditions means that clients must wait for availability in order to have their requests or concerns addressed.
Since the customer service office has a frequent staff turnover, the training is not constant, causing that the person who assists the clients does not fully know the main topics of the hotel and, therefore, the staff is not able to quickly meet customer needs.
Sometimes, the employees do not have a sense of "loyalty" towards the hotel. Therefore, when feeling an unpleasant communication tone, the customers may not want to have more contact with the hotel.
The task of attracting clients to the hotel depends significantly on the employee's emotional stability. On many occasions, customers complain because they were attended by people who used an unpleasant tone and language.
Once we validated the relevance of an automated chat service, the next step was to identify the most recurrent requests for information through WhatsApp, Telegram, and Facebook to establish the structure and architecture of the dialogue system. We performed the analysis based on data since the hotels allowed us access to their history of conversations with customers who requested information via chat. From this research, the main topics were broken down into seven modules:
Vacancies: People interested in working at the hotel ask about available opportunities.
Price and reservation: People looking for accommodation ask about the rooms' price and other services.
Restaurants: Guests request information on availability, hours, and other information about the hotel restaurant(s).
Pools: Guests or those interested in staying ask about the existence of pools and their characteristics.
Recreational activities: Guests ask about events at the hotel for fun, such as exhibitions, recitals, concerts, dances, among others.
Rooms: People interested in staying at the hotel ask about the characteristics of the rooms, number, and size of beds, internet availability, air-conditioned, among others.
Event rooms: People organizing events look for information about rooms for conferences, banquets, presentations, among others.
These modules were defined to establish the scope, functionality, and structure of the web-based management interface and the dialogue system.
Dialogue design
The role of a dialogue designer is to map what users can do, considering user needs and technological limitations. The designer curates the dialogue, defining the flow and its underlying logic in a detailed design specification that represents the entire user experience. We designed our conversational agent based on the methodology proposed by Giangola & Cao (2017). Through this methodology, users will project a person in their action. Therefore, it was convenient to design the experience that we wanted the users to perceive rather than leaving it to chance. Our goal was to generate a character that users feel comfortable communicating naturally, as they do with a human.
Dialog system implementation using Dialogflow
Two of the most powerful tools for its implementation were analyzed: Rasa (Bocklisch et al., 2017) and Dialogflow (Sabharwal & Agrawal, 2020). After analyzing both tools, we decided to use Dialogflow because the natural language understanding component in Spanish is more robust. It should be emphasized that the modules responsible for implementing artificial intelligence algorithms for the chatbot's natural language processing tasks are readily available within the Dialogflow tool. Therefore, the development, training, and implementation of these features are not among the contributions made in this work.
The first step was to formulate each intent and its training phrases, actions, and parameters to interpret the user's queries, phrases, and words. This procedure was done for each topic: vacancies, price, reservation, restaurants, pools, recreational activities, rooms, and event rooms. The "action and parameters" trigger the events of each intention made by the user, thus sending a response request through a JSON-type text file. In Dialogflow, we can use a webhook to fetch data from a server whenever an intent having webhook enabled is invoked. The information from the intent is passed to the webhook service to receive the result. The webhook analyzes and answers this request to provide the appropriate response according to the action triggered in the requested intention. In addition, we specified the type of entities used in the training phrases. This specification helps to handle queries with synonyms, typos, and word abbreviations, improving the interpretation of training phrases. We carried out this conversation design process for each main topic addressed in the hotel. The interconnection of the chatbot with the instant messaging application Telegram is very simple from the Dialogflow project. It is possible to indicate Telegram as an integration option for the project.
Back-end implementation
We developed a webhook to generate the answers to the users' queries, which receives, processes, and responds to the requests sent by the Dialogflow front-end. The webhook sends HTTP requests to the REST API that gets the required data from the database and sends it back to the webhook, responding to the Dialogflow initial request. The information exchange is done through a JSON-type text file containing all the information necessary to respond to the user. The responses of each triggered event were established in specific modules of the main topics of the hotel. They contain the functions that respond to each of the requests made by Dialogflow. The flow of the processes explained above is shown in Figure 2.
The user writes a query on a topic related to the hotel in the Telegram application (Figure 3).
Dialogflow matches the user's expression with intent and extracts the parameters.
Dialogflow interprets the query and sends an information request message to the webhook service. This message contains information about the matching intent, action, parameters, and response defined for that intent.
The webhook performs the query management actions as required and sends an HTTP Request type request to the REST API to obtain the necessary information from the database.
The Web API sends a response message to the HTTP JSON type webhook with the requested information.
The webhook sends a response message via an HTTP JSON document to Dialogflow. This message contains the response to be sent to the end-user.
Dialogflow receives and sends the response to the end-user.
Web interface
We developed a web interface to manage the data retrieved by the chatbot from a relational database (Figure 4). Through this web interface, the hotel administrator can make insertions, modifications, and deletions of data regarding the themes or modules established for the hotel. The hotel administrators can update the information delivered to the chatbot users in real-time. The REST API communicates with the webhook through an HTTP Request. The API provides the necessary information from the database to the webhook; the webhook answers to the Dialogflow request (interpreter of the intention user). This process is also observed in Figure 2.
1. The administrator logs into the web-based system and authenticates.
2. Within the system, the administrator performs the process of consulting the data, inserting, modifying, deleting, and displaying the information regarding the main modules of the hotel established and used in the chatbot.
3. This process is reflected in the database, which communicates directly with the REST API.
4. The webhook sends a request of type HTTP Request to the REST API to obtain the necessary information from the database.
5. The REST API sends a response message to the webhook of type HTTP JSON with the requested information.
System evaluation
We evaluated the system with end-users older than 18 years old. The design of the evaluation instrument was based on the chatbots evaluation framework described in Peras (2018). In this framework, the different perspectives of evaluation of a chatbot are raised, such as user experience, information retrieval, linguistics, technology, and business. At the same time, we tried to keep the evaluation instrument short and as simple as possible to ensure that the participants could answer it easily and quickly.
The chatbot evaluation consists of five questions for which five different scenarios were proposed. Each evaluator was asked to stand in a hypothetical scenario where it was required to interact with the chatbot to obtain information. This part of the assessment aims to measure information retrieval, linguistics, and technology. The evaluators were asked to answer a Google form by answering questions related to the interaction. The instructions given to the appraisers were as follows:
Before starting, you must install the Telegram mobile application. Please download it from the App Store if you do not have it. Once the App is installed, enter and search the user HotelCiceseBot, then a conversation with our chatbot will appear. Write any greeting to start the interaction.
Subsequently, the following cases were presented to the user:
Case 1: You are a woman, and you want to go to breakfast with three friends, but first, you want to know if there is a promotion at the hotel. Talk to the chatbot to answer the following three questions:
6. What is the name of the most suitable promotion for case 1?
7. What days is the promotion valid?
8. What is the price of the promotion?
Case 2: You are a mother/father, and before deciding to book at the hotel, you want to know if your children, who are under ten years old, can enjoy the pool. Talk to the chatbot to answer the following questions:
Are there pools in the hotel?
How many swimming pools does the hotel have?
What swimming pool would you choose following case 2?
How deep is it?
What clothing is required?
Case 3: In your work, you are tasked with organizing the presentation of a product to the public. You need to rent a room with an auditorium-shaped setup. Talk to the chatbot to answer the following three questions:
Does the hotel rent rooms?
How many and which rooms have an auditorium-type setup?
How many people fit in the room?
Case 4: Imagine you want to apply for a job at the hotel:
Case 5: You arrive at the hotel, and you need the Wi-Fi password:
The objective of using these scenarios was for the evaluator to interact with the chatbot to obtain the requested information. In this way, it is possible to verify the number of interactions that were carried out successfully and not successfully and thus have an accurate evaluation of the chatbot by quantifying the average success rate with each user who uses it.
Once the questions about each case were answered, the evaluators answered a questionnaire to evaluate user experience. The questions used were the following:
How easy was it for you to use the chatbot?
Do you consider this to be an efficient means of obtaining information?
What do you prefer to do to request information?
Did you have a problem with the chatbot?
If the answer was YES, indicate what was the problem you found and what the chatbot needs to offer a better service.
Results
In this section, we analyzed the responses obtained through the evaluation instrument. Likewise, some performance metrics obtained from the Dialogflow logs are presented concerning each of the attempts made by users when interacting with the chatbot.
The evaluation was carried out by 52 people who answered the questions referring to the five scenarios and evaluated ease in information retrieval. Likewise, they responded to questions focused on user experience. The evaluation was done in two stages, 30 participated in the first and 22 in the second. The first stage served to make a preliminary evaluation and detect the main aspects to improve. This section presents the results obtained in the final assessment, and a comparison is made with the initial assessment to observe the improvement.
The error percentage per case out of the total questions asked by each of the cases is shown in Figure 5.
We obtained this percentage by considering each of the incorrect answers given by the user in any question. For example, in scenario 1, the user must answer three questions, multiplied by 22 users, we have 66 answers. From these answers, only three gave incorrect information. This number represents 5% of errors of the total of questions asked.
Regarding user experience evaluation, 31.8% of the people who carried out the assessment consider that the chatbot is easy to use. 27.3% consider that it is very easy to use. For 36.4% of the people, using the chatbot was neither easy nor difficult. Finally, for 4.5% of users, using the chatbot was very difficult.
It is important to emphasize that 100% of evaluators considered that the chatbot is an efficient means to obtain information. 72.7% of people who carried out the evaluation would prefer to use the chatbot to request information regarding the hotel instead of other communication options. On the other hand, 18.2% prefer to make a phone call, 9.1% would look for information through the hotel's website. 72.7% of the users who used the chatbot had no problem when evaluating the chatbot, while 27.3% of the people did have a problem.
Regarding the analysis of the chatbot's performance based on the system logs, each of the user's attempts during the evaluation was verified. We accessed the attempts logged in the section called “History”, found in the Dialogflow web interface. This section shows all the evaluator's conversations with the chatbot organized by date. For the analysis of this information, the following indicators are available:
Failed Attempts: those in which a favorable response was not obtained from the chatbot to the question asked by a user.
Complete attempts: those in which a favorable response was obtained from the chatbot to the question asked by the user.
Active attempts: the total of attempts used per conversation, the sum of failed attempts, and full attempts.
Figure 6 displays the average number of activated intents, failed intents, and completed intents, along with their respective standard deviations. This graph allows us to observe the variation in the data around the mean values. By examining the standard deviation bars, we can see how much the data points diverge from the mean, providing insight into the improvement from the preliminary to the final evaluation. The total score was obtained by dividing the number of completed intents by the number of activated intents. Figure 6 also shows the comparison with the first evaluation stage to quantify the improvement in the final version of the chatbot. The mean total score of all participants in the final evaluation was 93.83 with a standard deviation of 0.06, while in the preliminary it was 86.58 with a standard deviation of 0.09.
Discussion
The chatbot presented in this work seems to be an essential tool that can benefit all the actors involved with the hotels that have participated. The results show that only a few questions were answered incorrectly by the chatbot. Scenarios 2 and 5 obtained a perfect rating from the evaluators. These scenarios concern whether the pools are appropriate for children and information about Wi-Fi. Scenarios 1 and 4 have a 5% error rate. These scenarios concern when breakfast information is required and when a vacancy is applied for. These scenarios appear to be more complex than scenarios 1 and 5, which could explain why there are more errors in these scenarios. Finally, the scenario with the more significant number of errors is scenario 3, which refers to the questions about reservations and information about rooms availability. This scenario seems to be the most complex.
Through the evaluation, we observed that users adapt to the functioning of the chatbot and end up understanding each of the answers and indications that are shown, although it could also be noted that the chatbot is not perfect. There are areas for improvement that must be considered, for example, to include more training phrases, synonyms, misspellings, and abbreviations that users commonly use to communicate.
By doing a manual review of the interactions we could see that the users who adhered to the instructions provided by the chatbot were the fastest to find the information they were looking for. Almost all participants in the evaluation perceived value and benefit in using this technology to interact with hotel customer services. We observed that to deliver the chatbot to users, it is necessary to train it and correct the details found in the evaluation carried out, but it can also be said that the chatbot can show its functionality as a demo or a very established prototype, and that could undoubtedly help the Mexican hotel sector for which it was made. As a user interaction technology, chatbots are an excellent alternative. In the case of the implemented chatbot for hotels, it only requires a little more fine-tuning in its training to achieve excellent performance.
Conclusions
In this work we designed, developed, and evaluated a chatbot to give information to hosts and customers. One of the main contributions of this work is the methodology to be able to carry out an improvement of the interface of a chatbot specialized in Mexican Spanish for a local hotel. This study shows the design, development, and evaluation of a support system for a hotel, which is the first with these characteristics to our best knowledge. Through these results, the opportunity to develop, evaluate, and improve other chatbots dedicated to the tourism sector opens. Tourists today are increasingly related and more comfortable with digital technologies and tools to improve the conditions of their stay. This work demonstrates how to improve the interaction between the service, tourists, and hotels. However, this proposal can be extended to other services in the tourism industry.
As future work, it is proposed to emphasize the aspects where the evaluation is not the best. Also, it is interesting to explore different ways to improve chatbot training, considering that there is not much data for Mexican Spanish. Finally, it is crucial to evaluate when the system is already in production with real end-users, always keeping in mind the possibility of scaling the system to cover a vast number of hotels and Spanish language variants.
