nueva página del texto (beta)
Inglés (pdf)
Artículo en XML
Referencias del artículo
Traducción automática
Enviar artículo por email
Citado por SciELO 
Similares en
SciELO 
Permalink
1 Introduction
Coronavirus 2019 (COVID19) [1, 2, 3, 4] is a global pandemic since past two years. It has almost ruined mankind with catastrophic implications [5] having major impact on global economy. This has led to unprecedented rise in unemployment, psychological issues and depression of people around world. The abrupt social, economic and travel changes have motivated research in several domains [6, 7, 8, 9]. In India COVID19 has adversely affected its economy in past two years [10]. It has battered Indian economy in two major COVID19 waves as shown in Fig. 1 which country has seen.
As of January 2022 considering oficial figures, India stands at second highest number of confirmed cases in world after United States and third highest number of deaths after United States and Brazil. However it is assumed that there has been high degree of under reporting in COVID19 cases.
During this phase of 2 years there has been unprecedented growth of social media usage such as Twitter by people where they have expressed several concerns related to their living conditions, psychology [11, 12, 13, 14] and mental health [15]. This data has been used for research in behavioural sciences [16]. It has also been used for personality prediction [11, 12] as well as understanding trends and backgrounds of users online [17, 18].
In view of this research on sentiment analysis of COVID19 reviews has become centrestage of attention for social media analytics.Sentiment analysis involves natural language processing (NLP) [19, 20, 21] in order to extract systematically affective states, attitudes and opinions of individuals or social groups [22, 23, 24] across various domains such as politics, sociology, business intelligence, etc. [25].
Classification of sentiments within any specific domain involves any event, item, topic, product, application or others. The importance of sentiment analysis lies in many tasks such as digital mental health [26], recommendation systems [27] as well as intelligent cognitive assistants [28].
The major prima face in using sentiment analysis is towards text classification having polarities [29]. Polarity comes in several forms including sentiment class having several values such as very positive, positive, neutral, negative and very negative. However, polarity values and classes differ from one application to another. Sentiment expressions with happy, sad, angry etc emotions alongwith sentiment classes can be analyzed through sentiment analysis.
Text classification can be either subjective or objective [30] in nature. Sentiment analysis approaches are basically classified as three major groups [29] viz (a) lexicon-based approach (b) machine learning approach and (c) hybrid approach. In lexicon-based approaches, polarity is extracted with respect to predefined lexicon or dictionary. Sentiment analysis model is trained using dataset commonly known as corpus. In machine learning approaches, algorithm is trained considering given dataset in order to build classification model which extracts sentiment polarity of given text. Hybrid approach is most suitable for real life applications which combine both lexicon-based and machine learning methods to perform sentiment analysis activity.
One of the commonly used machine learning approaches is artificial neural networks (ANN) [31, 32]. Closely associated with ANN are deep learning networks [33, 34, 35].
In recent past both these methods have been used in abundance in sentiment analysis research [36]. Deep learning networks are more powerful than ANNs. They are blessed with better data representation capabilities with good features and multiple representation levels than ANNs. This allows different computer recognition tasks including text mining, image processing and pattern recognition to be performed more smoothly and efficiently. Deep learning networks allow representation of words considering textual data and produce word embedding which could be used by different machine learning approaches.
Several types of deep learning networks have been used for NLP such as feedforward neural networks, convolutional neural networks (CNN) and recurrent neural networks (RNN) [33, 34]. Deep learning networks have outperformed other machine learning methods in several NLP tasks including machine translation and named-entity recognition [37, 38]. Deep learning-based approaches for sentiment analysis are further classified in accordance to deep learning networks being used. There are basically three different categories viz: (a) feed forward neural network-based sentiment analysis, (b) CNN based sentiment analysis, and (c) RNN based sentiment analysis approaches. The majority of sentiment analysis approaches in previous few decades have been focused towards identification of text polarity. Deep learning models have played a significant role in forecasting COVID19 infection trends [39] in various parts of world.
In this research work, novel sentiment analysis methods have been used considering state-of-the-art methods towards understanding people’s behavior arising from COVID19.
This work provides motivational direction towards making society aware of the fact that dissemination of useless information in a sensitive topic like COVID19 is harmful to mankind. It considers psychological well-being with respect to constant rise in number of active COVID19 cases during peak periods.
Considering abundant corpus of text data available for this research, here computational modeling and machine learning methods [20, 21, 36] have been primarily used. However, there have been number of challenges arising due to testing and reporting of COVID19 cases [39].
It has been observed that an appreciable amount of available reported data is plagued with false figures. This issue is addressed by re-collecting that portion of data from other reliable sources [39].
The computational framework highlights multi-label sentiment classification with more than one sentiment expressed at once. In order to achieve this deep learning based language models considering delayed recurrent neural networks (d-RNN) and hierarchical version of d-RNN (Hd-RNN) are used to address rise of COVID19 cases in different parts of India.
All sentiments are reviewed with respect to time window spread across January 2020 and June 2021 when India witnessed most active cases.
Both d-RNN and Hd-RNN are optimized by fine tuning different network parameters. The experimental hypothesis is made stronger by performing comparative performance analysis of d-RNN and Hd-RNN models with some traditional machine learning models like naive bayes (NB), support vector machine (SVM), k-nearest neighbor (kNN), random forest (RF), gradient boosting (GB), ada boost (AB) and decision trees (DT) [36].
We use different variants of BERT [36] in order to evaluate test datasets with prediction validation. LSTM [20, 21] and BD-LSTM [20, 21] models are also used in comparing performance of d-RNN and Hd-RNN.
The results on experimental datasets highlight Hd-RNN’s superiority with respect to other techniques.
The novelty of this work is attributed through following points: (a) Preparation and analysis of COVID19 sentiment data spanning across time period of 1.5 years, (b) Computational framework comprising of novel RNN models such as d-RNN and Hd-RNN, (c) Comparative analysis with other models relevant to COVID19, (d) All methods are evaluated with highly skewed data where precision, recall and F1 scores are used.
This paper is organized as follows. In section 2 work related to sentiment analysis is presented. The computational methodology is discussed in section 3. In section 4 experiments and results are highlighted. Finally, conclusion is given in section 5.
2 Related Work
The process of feature extraction from text for NLP related tasks is known as word embedding [40, 41, 42]. Some of the common examples of word embedding include sentiment analysis. As such significant amount of research work has been performed on sentiment analysis using machine learning. It is basically obtained using methods where words or phrases from vocabulary are mapped to real number vectors. It involves mathematical embedding from large corpus with multi-dimensions per word to a vector space. The lower dimension here is used by machine learning or deep learning models for text classification [40].
Basic word embedding methods such as bag-of-words (BOW) [43] and term frequency inverse document frequency (TF-IDF) [44] do not have context awareness and semantic information in embedding. This problem is related to skip-grams [45] which use n-grams involving bigrams and tri-grams to develop word embedding. This allow adjacent word token sequences which needs to be skipped [46].
There has been considerable progress in word embedding and language models since last few decades. In [47] word2vec embedding is proposed which uses feedforward neural network model. They learn association between words from text dataset which detects synonymous words or suggest additional words with respect to a partial sentence.
It uses continuous bag-of-words (CBOW) or continuous skip-gram model architectures in order to produce distributed words’ representation. The method creates large vector which represent each unique word in corpus. Here semantic information and relation between words are preserved. For any two sentences which do not have much common words, their semantic similarity is captured using word2vec [47].
However, word2vec does not well represent word context. To obtain vector representations for words, in GloVe [48] words are mapped into meaningful space where word distance is related to semantic similarity. It uses matrix factorization in order to construct large matrix of co-occurrence information. This results in representation which shows linear substructures of word vector space. With top list words embedding feature vectors match within certain distance measures.
The relations between words such as synonyms, company-product relations etc can be found using GloVe. A gender-neutral GloVe has been proposed [49] where it has gender biased information.
In [50] word embedding methods evaluation is provided methods including GloVe [54], skip-gram and continuous space language models (CSLM) [51]. It has been observed that in all language related tasks skip-gram and GloVe have outperformed CSLM.
In [52], an evaluation of word embedding methods have been performed for biomedical text analysis applications. Here it has been observed that word embedding trained from clinical notes and literature better captured word semantics.
In [53], classification of twitter data using machine learning is performed with 89.47% accuracy.
In [54], sentiment analysis on Uri attack has been performed in order to mine emotions and polarity on Twitter data. The dataset comprised of about 5000 tweets. The experimental results showed Uri attack disgusted 94.3% of individuals.
In [55], sentiment analysis using machine learning for business intelligence has performed.
In [56], analytical categorization and evaluation of prevalent testing techniques and deployment of sentiment analysis machine learning techniques on different applications have been performed.
In [57], Naïve Bayes and OneR have been used for sentiment analysis.
Another work from [58] also uses various machine learning algorithms such as naïve bayes, support vector machine (SVM), logistic regression, decision trees, k-nearest neighbor and random forest for sentiment analysis.
In [59], distribution of COVID19 vaccines implies an urgent need to track and understand public opinion on an ongoing basis to establish baseline vaccine confidence levels in order to detect early confidence loss warnings.
Research on public attitudes with facebook and twitter towards COVID19 vaccinations uses Artificial Intelligence analysis [60] in UK and US.
In [61], a study is performed on social network analysis of COVID19 sentiments using machine learning methods with twitter data. This study creates sentiment analysis through large number of tweets. The results are categorized considering consumers' viewpoint into positive and negative with respect to tweets [62].
Another significant work on sentiment analysis for COVID19 and infectious diseases can be found [63].
In [64], public sentiments on COVID19 using machine learning for tweet classification is performed where classification methods like naive bayes and logistic regression are used.
LSTM and BERT language models have also provided promising results [65, 66] for sentiment analysis. Both LSTM and BERT belong to family of RNNs [67, 68].
LSTMs are characterised by two important gates viz forgot and output gates which makes them efficient sentiment analysis models. LSTM has feedback relation which helps in processing both single and sequential data points. Many versions of LSTM have been developed till date. For smaller datasets bidirectional LSTM models produce better results than BERT models.
These models are trained in lesser time than their pre-trained counterparts [69]. In LSTM are that words are passed in and generated sequentially. For capturing true meaning of words even bi-directional LSTMs do not have good performance.
These issues are resolved by BERT efficiently [70]. BERT is based on transformers where each output element is connected to each input element. They have been unveiled by Google in 2017. BERT architecture proved to be game changer in NLP which allow transfer learning usage in several tasks. It uses adjacent text in order to assist machines in interpretation of ambiguous language in text.
Some recently proposed significant sentiment analysis approaches are presented in Table 1.
Table 1 Recently proposed significant sentiment analysis approaches
| Paper | Year | ANN Models | Datasets | Results |
| [75] | 2020 | CNN | 8000 comments and posts | Accuracy 90.9% |
| [83] | 2020 | word embedding | Tweets | Accuracy 62.8% |
| [76] | 2019 | CNN and LSTM | Lithuanian Internet comments | Accuracy 70.6% |
| [80] | 2019 | LSTM | Militarylife PTT | Accuracy 85.4% F1-Score 88.41% |
| [81] | 2019 | LSTM | Roman Urdu datasets | Accuracy 95.2% |
| [74] | 2018 | CNN | Manually annotated dataset | Accuracy 95% |
| [79] | 2018 | LSTM | 504 news headlines and 675 microblog messages | |
| [78] | 2018 | RNN | Twitter posts and news headlines in financial domain | |
| [71] | 2017 | Feed Forward | Arabic tweets | Accuracy 90% Precision 93.7% |
| [77] | 2017 | RNN | Amazon health product reviews, SST-1 and SST-2 | GRU is best (Accuracy) |
| [72] | 2016 | CNN | Movie reviews and IMDB | |
| [73] | 2015 | CNN | SentiStrength (text) SentiBank (visual) | Accuracy 79% |
| [82] | 2015 | word embedding | SemEval 2013 | |
| [84] | 2015 | auto-encoder | Arabic Tree Bank | Accuracy 73.5% |
In [71], feed forward neural network is used on Arabic tweets with accuracy and precisión of 90% and 93.7% respectively.
Appreciable results are available in [72] where CNN is used on movie review datasets.
In [73] and [74], CNN have been used on SentiStrength (text) and SentiBank (visual) as well as manually annotated datasets with accuracy values of 79% and 95% respectively. With 8,000 comments and posts CNN [75] have produced accuracy of 90.9%.
In [76] CNN and LSTM models on Lithuanian internet comments have provided an accuracy of 70.6%. In [77] RNNs with Amazon health product reviews, SST-1 and SST-2 considering accuracy have given best results for GRU. Another significant results can be found in [78] and [79] with RNNs.
In both of these Works datasets considered constituted new headlines with microblog messages as well as twitter posts. In [80] LSTM on Militarylife PTT datasets have produced accuracy and F1 Score of 85.4% and 85.4% respectively. In [81] LSTM on roman urdu datasets have produced accuracy of 95.2%. In [82] and [83] word embedding have been used on SemEval 2013 and tweets. In [84] auto-encoders have been used on Arabic tree bank.
3 Computational Methodology
The computational methodology for sentiment analysis of COVID19 reviews is presented in this section. The section starts with discussion on COVID19 datasets. This is followed by brief introduction on RNN and delayed RNN. Based on RNN and d-RNN, Hd-RNN is presented.
Next an approximation of hierarchical delayed RNN with hierarchical bidirectional RNN is performed. The section concludes with twitter-based sentiment analysis framework for COVID19 in India.
3.1 Datasets
Due to non-availability of any standard COVID19 datasets, in this research we have developed datasets considering various factors. Some of the significant factors considered while preparing this dataset include time window, geographical region, age group, gender, testing frequency and COVID19 vaccine taken. The datasets are prepared considering tweet data available from Twitter with respect to abovementioned factors.
The prepared datasets are validated against available benchmark datasets [36]. The time window here spans from January 2020 to June 2021 when there were most active cases were detected in India. It is during this period India had two COVID19 waves, first one from March 2020 to September 2020 and second one from February 2021 to May 2021.
The geographical region included those states where most COVID19 cases are detected during stated time window.
In view of this, 5 top Indian states viz Maharashtra, Delhi, Karnataka, Kerala and Tamil Nadu are considered. The age groups are divided into 5 intervals viz 0-20, 20-40, 40-60, 60-80 and 80+. The gender considered as male and female. The testing frequency involves COVID19 tests performed per day by both government and non-government testing agencies. The COVID19 vaccine taken highlights those people in abovementioned states who have been vaccinated.
3.2 Recurrent Neural Networks
RNNs are an important category of deep learning networks [85] with infinite impulse response. The different computational units of RNNs are connected together to form directed circle. As a result of this, these networks create an internal state which highlights dynamic temporal behavior.
The arbitrary input sequences are processed through internal memory. This makes them readily applicable for non-segmented handwritten or speech recognition tasks. They are basically Turing complete [86] which provides capability to run arbitrary programs in order to process arbitrary input sequences.
Since RNNs can be trained in either supervised or unsupervised manner they are more efficient than traditional ANNs and SVMs [85, 86]. RNNs learn intrinsic characteristics about data without target vector's help. This learning capability is stored as network weights. The network's unsupervised training has similar input as target units.
In deep learning network's architecture is optimized through several routines. The network is treated as directed graph where different hidden units are connected to each other. Each hidden layer in network is non-linear combination of layers.
This is because combination of outputs from all previous units works with their activation functions. Each hidden layer becomes optimally weighted and non-linear, when optimization routine is applied to network. Each hidden layer becomes low dimensional projection of below layer, when each sequential hidden layer has fewer units than one below it.
The recurrent structure of network allows modeling of contextual information for temporal sequences. Due to issues of vanishing gradients and error blowing up problems [86], it is very difficult to train these networks with commonly used activation functions.
This is addressed through LSTM architecture [85, 86] which replaces non-linear units in traditional RNNs. LSTM memory block with single cell is highlighted in Fig. 2. It has one self-connected memory cell and three multiplicative units viz input, forget and output gates. These gates store and access long range temporal sequence based contextual information. The activations of memory cell and three gates are available in [85]. It has been shown that topological enhancements RNNs increases their expressive power and representation capacity [87].
The two most common enhancement strategies are: (a) stacked RNNs which increases learning non-linear functions capacity and (b) bidirectional processing which uses acausal information in sequence. The basic mathematical background to increase network’s depth for single layer RNN is presented here. In view of this, let us consider an input sequence
Here
The parameters include input weights
Here activation function and parametrization abide by single layer RNN. The weights and bias terms for each layer 𝑗 are represented as
3.3 Basic d-RNN
An alternative means to increase RNNs’ depth is to consider time within single layer RNN. Single layer RNNs are restricted considering number of non-linearities applied to recent inputs. [87] have addressed this restriction by adding intermediate non-linearities between input elements. Here computational steps are added between elements in sequence which increases runtime complexity. The delayed recurrent neural networks (d-RNNs) addresses this by increasing effective depth through introduction of delay between input and output.
The d-RNN can be defined as single layer RNN such that for any input respective output is obtained
Depending on task being solved this can be achieved by adding null input element or including
Another RNN very similar in structure to single layer RNN is stacked RNN where additional in between layer connections are placed which adds depth in network. Any stacked RNN can be configured into a single-layer d-RNN which produces exact hidden states and output sequences. The depth from in between layer connections are replaced with temporal depth applied through output delays. Considering above equations parameters of single layer RNN using weights and bias terms of
The structure of recurrent weight matrix
This buffering mechanism processes information which eventually arrives at output after
It has been proved that d-RNN parameterized by above equations is exactly equivalent to stacked RNN in above equations. This proposition can be extended towards recurrent cells with more complexity. A
3.4 Hierarchical d-RNN
Considering RNN and d-RNN in previous sections, we now discuss Hd-RNN [36] for semantic analysis of COVID19 reviews. The better approximation of network topologies like stacked RNN, bidirectional RNN and stacked bidirectional RNN by d-RNN with faster runtimes [36] serve major motivation. Hd-RNN differs from its non-hierarchical counterparts with respect to better classification accuracy taking similarities and running time parameters as data corpus grows [36].
A schematic representation of architecture of Hd-RNN is shown in Fig. 4. Here, d-RNN is used to model temporal sequences in COVID19 reviews. The results from d-RNN are combined together to form Hd-RNN. Hd-RNN comprises of 7 layers d-RNN1→d-RNN2→d-RNN3→d-RNN4→d-RNN5→fc→sm.
Here, d-RNNi with
In order to recover any single hierarchy, we can run split d-RNN on small subset of reviews having few words [36]. This helps in computation of seed classification value.
The input dataset subsets are developed randomly which is initiated at layer d-RNN1. With initial classification value remaining part of data corpus is placed into seed class for which average similarity is present.
This leads to classification of entire dataset using only similarities to words with respect to small subset. This process is applied recursively to each class such that Hd-RNN is build up considering only small similarities fraction. The classification process continues till d-RNN5.
This recursive phase has no measurements between classes at previous split. This results in robust version of Hd-RNN which aligns its measurements
Hd-RNN is specified with respect to success probability of success in recovering true hierarchy 𝐶𝑆∗, measurement 𝑚𝑠 and runtime complexity. Certain restrictions are placed on similarity function 𝑆𝑀 such that similarities agree with hierarchy up to some random noise:
P1 For each
Here expectations are taken with respect to possible noise on
P2 For each
Here
The condition P1 highlights similarity from word
Considering feature learning d-RNN extracts temporal features for sentiment data sequences. After obtaining sentiment sequence features, fully connected layer fc and softmax layer sm performs classification. This architecture addresses vanishing gradient problem [36]. Network neurons are adopted in last recurrent layer d-RNN5. The first four d-RNN layers use tanh activation function. This is trade-off between improving representation ability and avoiding any over fitting. The number of weights in network is more than in tanh neuron.
The network can be overfitted with limited data training sequences. Specifically, when
In averaging phase if words in data are not similar to any represented class, new word class is developed.
3.5 Approximation with Hierarchical Bidirectional Recurrent Neural Networks
It is very well known that d-RNN can have equivalent structure as stacked RNN when its weight matrices are constrained. If these constraints are ignored, d-RNN peeks at future inputs. It computes delayed output considering time using also inputs which are beyond specified timestep.
An analogous idea has been used as benchmark for bidirectional recurrent neural networks (BRNN) [36, 88]. It has been shown that BRNN are superior to d-RNN considering relatively simple problems. However, it is not clear that this comparison holds true for problems requiring more non-linear solutions. The similar proposition holds for hierarchical bidirectional recurrent neural networks (HBRNN) [36].
If a recurrent network computes its output for specified time by exploiting future input elements, what are necessary conditions in order to approximate its BRNN and HBRNN. Moreover, can d-RNN and Hd-RNN have similar results. And with these conditions, is it more beneficial to use d-RNN and Hd-RNN instead of BRNN and HBRNN. There are number of non-linear transformations [36] where each network applies to any input element prior to computation of output at initial timestep. Only past inputs are processed by generic RNN where number of non-linearities decrease when inputs are close to initial timestep.
BRNN has similar behavior considering causal inputs. For acausal inputs it is symmetrically augmented. For casual inputs Hd-RNN has similar behavior with higher number of non-linearities. This remains for first d acausal inputs with non-linearities decreasing. For Hd-RNN to have at least similar number of non-linearities as BRNN for every sequential element, a delay is required which is twice as sequence length. Hd-RNN can superceed BRNN when non-linear influence of nearby acausal inputs on learned function is superior than farther elements. When Hd-RNN is used in order to approximate BRNN, it also decreases computational cost. For length of considerable sequence stacked BRNN computes both forward and backward RNNs for each layer prior to computation of next layer. As a result of this synchronization parallelization is not allowed which increases runtime. Forward passes for d-RNN takes additional steps, but synchronization does not affect it. In highly parallel hardwares, runtime of k layer stacked BRNN is at least k times slower than d-RNN or Hd-RNN. In línes with d-RNN, Hd-RNN can also be used in critical output values in near real time applications [89, 90].
3.6 Twitter Based Sentiment Analysis Framework for COVID19 in India
In this study a novel framework is presented which uses twitter information in order to understand public behavior in India during COVID19 pandemic. This research addresses sentiments of people in different parts of India during outbreak of pandemic in 2020 and 2021.
Considering datasets defined in section 3.1 our analysis is mainly concentrated in states of Maharashtra, Delhi, Karnataka, Kerala and Tamil Nadu where maximum number of COVID19 active cases are detected. Fig. 5 highlights major components of this framework. It is to be noted that framework is attributed with multi-label classification with multiple outcomes. The social media language has been rapidly evolving. As a result of this special phrases, emotion symbols and abbreviations are present in tweets. These are transformed for building language models [36]. The languages predominant in stated 5 Indian states are Marathi, Hindi, Kannada, Malayalam and Tamil which are used in combination with English. Hence, a transformation is also performed for certain words, emotions and character symbols highlighted in these stated languages.
The computational framework for sentiment analysis involves following steps: (a) tweet extraction, (b) tweet pre-processing, (c) model development with training, and (d) prediction. Tweet extraction involves processing on COVID19 dataset mentioned in section 3.1. During this process special symbols and abbreviations present in tweets are transformed towards building language model. Few such instances are represented in Table 2.
Table 2 Instances of special symbols and abbreviations present in tweets
| Original Phrases | Transformed Word |
| Facemasks | face masks |
| Socialdistancing | social distancing |
 |
smiling faces |
 |
beds |
 |
sanitisors |
 |
winks |
It is to be noted that transformation is performed for specific words, emotions and character symbols which are represented in Marathi, Hindi, Kannada, Malayalam and Tamil.
The experimental dataset features 17 different sentiments [36], which are labelled by a group of 1000 experts for 100,000,000 tweets during COVID19 in 2020 and 2021. In tweet pre-processing each word is converted into its corresponding GLoVe vector.
Here, each word is converted into a 500-dimensional vector. The main reason behind selection of GLoVe embedding involves good results it has shown for sentiment analysis [48].
From each word, GloVe vector is passed towards respective d-RNN and Hd-RNN models for training. The trained models are evaluated initially. After successful evaluation they are used for COVID19 sentiment analysis considering test data from states of Maharashtra, Delhi, Karnataka, Kerala and Tamilnadu.The trained models are used for classification of 17 sentiments.The evaluation metric is so chosen such that it evaluates trained models in best possible way [36].
Hence, it is required that metric captures correct loss [36] arising from misclassification and gives best representative score. As such classification here is of multi-label in nature and is based on binary cross entropy loss, hamming loss, jaccard coefficient score, label ranking average precision score and F1 score [36]. Binary cross entropy loss represents softmax activation alongwith cross entropy loss.
Hamming loss generates bit string of class labels using XOR between actual and predicted labels and averages with respect to dataset instances. Jaccard coefficient score measures overlap between actual and predicted labels with respect to similarity and diversity attributes. Label ranking average precision score calculates percentage of higher ranked labels which resemble true labels considering given samples. F1 score highlights balance between precision and recall measures.
Since classification here is of multi-label in nature, combination of two variants of F1 score viz F1 macro and F1 micro are used towards evaluation of training models.
Considering experimental framework highlighted in Fig. 5 multi-label classification is performed with respect to d-RNN and hierarchical d-RNN models. We use 80:10:10 ratio of dataset for training, validation and testing. The aforementioned models are trained using COVID19 dataset [36] stated in section 3.1. This dataset has 100,000,000 tweets collected between March 2020 to June 2021.
In d-RNN and Hd-RNN hyperparameters are determined considering experiments performed. GloVe embedding uses word vector of size 500 in order to provide data representation [48].
A dropout regularization probability of 0.75 is used for d-RNN and hierarchical d-RNN models which feature 500 input units, two layers with 128 and 64 hidden units and an output layer with 17 units for sentiment classification. Here, BERT model is used as benchmark to validate results obtained from d-RNN and hierarchical d-RNN models. The main reason for considering BERT models lies in success obtained by using these models in sentiment analysis [36]. In BERT default hyperparameters are used. For BERT base uncased model learning rate is tuned.
BERT architecture has dropout layer and linear activation layer with 17 outputs corresponding to 17 sentiments. Table 3 shows model training results for 10 experiments with different initial weights and biases for respective models with different performance metrics. In India given huge population with large number of densely populated cities [36, 39] COVID19 management is plagued with major challenges.
Table 3 Analysis of training performance for d-RNN, Hd-RNN and BERT for COVID19 datasets
| Metric used | d-RNN | Hd-RNN | BERT |
| Binary Cross Entropy Loss | 0.279 | 0.260 | 0.370 |
| Hamming Loss | 0.160 | 0.155 | 0.145 |
| Jaccard Score | 0.435 | 0.437 | 0.530 |
| Label Ranking Precision Score | 0.507 | 0.517 | 0.779 |
| F1 Macro | 0.449 | 0.460 | 0.570 |
| F1 Micro | 0.500 | 0.505 | 0.589 |
The first COVID19 case in India was detected on 30th January 2020. Thereafter India started lockdown as a result of which situation gradually changed. On 22nd March 2021 India had more than 11.6 million confirmed cases with more than 160,000 deaths. This made India third highest with confirmed cases after United States and Brazil. India was 8th in world with more than 300,000 active cases prior to second wave. India witnessed its first major peak around middle of September 2020 with close to 100,000 cases daily. This gradually decreased to around 11,000 cases daily by end of January 2021. In March 2021 India witnessed its second peak when cases began rising faster and by 22nd March, 2021 India had 47,000 daily new active cases [39].
In first six months, states of Maharashtra, Delhi and Tamil Nadu led COVID19 infections [39] with city of Mumbai having highest number of active cases. In later half of 2021, Delhi cases reduced but still remained in leading 8 states [39]. In 2021, Maharashtra continued as state of highest infections and in March, 2021 it featured more than half of new active cases on a weekly basis. Delhi continued with less than a thousand daily active cases. Alongwith this Karnataka, Kerala and Tamilnadu continued to show good number of daily active cases.
We have applied COVID19 datasets from different parts of India including nationwide COVID19 sentiments with 5 major states having majority of COVID19 cases.
The trends have shown that these 5 states had two major peaks followed by several minor ones. However, India had first major peak around mid-September, 2020 and second major peak around mid-May, 2021 as shown in Fig. 6 [36].
4 Experiments and Results
Considering computational methodology in section 3, in this section experiments and results are presented.
4.1 Analysis of COVID19 Results
The test dataset [36, 39] contains COVID19 tweets between March 2020 to June 2021.
It comprised of more than 750,000 tweets from India. Five more datasets are generated considering states of Maharashtra, Delhi, Karnataka, Kerala and Tamilnadu with around 22,000 tweets each.
It is observed that number of tweets in India follows identical trend as number of COVID19 cases increase till July, 2020 after which number of tweets decline.
Again, number of COVID19 cases increase till April, 2021 after which number of tweets decline. There is similar pattern for Maharashtra and Kerala.
For Delhi, Karnataka and Tamilnadu situation is slightly different as first peak was observed in July, 2020 with increasing tweets that declined afterwards and did not keep up with second peak of cases in September, 2020. Again, number of COVID19 cases increase till April, 2021 after which number of tweets decline. This indicates that as more cases were observed in early months, there was much concern which eased before major peak was reached and number of tweets were drastically decreased. There could be signs of fear, depression and anxiety as tweets decreased drastically after July, 2020 with increasing cases. Fig. 7 and Fig. 8 show number of bi-grams and tri-grams prevalent in India. In bi-grams it is observed that novel corona virus is most used followed by covid19. Similarly, namaskar is most used followed by covid19 positive cases and infected cases. In order to provide better understanding of tweets some examples are highlighted as shown in Table 4. In case of tri-grams we can find more information in tweets as shown in Table 5.
Table 4 Certain situations of tweets which are captured in most prominent bi-grams
| Month | Tweets | Bi-gram |
| May 2020 | …….#lockdown times#.......... | pointing up |
| July 2020 | ……face masks mandatory…… | pointing up |
| September 2020 | …..positive cases increasing…. | Pointing up |
| March 2021 | ……..vaccination necessary….. | with folded hands |
| April 2021 | …….#lockdown times#.......... | pointing up |
| May 2021 | …..vaccination mandatory….. | with folded hands |
| June 2021 | ……..partial lockdown………. | Pointing up |
Table 5 Certain situations of tweets which are captured in most prominent tri-grams
| Month | Tweets | Tri-gram |
| September 2020 | …..positive cases increasing…. | Pointing up |
| March 2021 | ……..vaccination necessary….. | with folded hands |
| April 2021 | …….#lockdown times#.......... | pointing up |
| May 2021 | …..vaccination mandatory….. | with folded hands |
| June 2021 | ……..partial lockdown………. | Pointing up |
Fig. 9 shows number of occurrences of a given sentiment in relation to rest of tweets sentiments in training datasets [36, 39]. Now we present results of COVID19 tweets prediction in India, Maharashtra, Delhi, Karnataka, Kerala and Tamilnadu by considering them as individual datasets. The dataset mentioned in Section 3.1 has been used here for training. Fig. 10 above presents distribution of sentiments predicted by Hd-RNN for respective datasets considering stated span of time. Hd-RNN has provided best results for training data. In Fig. 11 sentiments predicted are reviewed considering heatmap in order to examine number of occurrences of given sentiment with respect to rest sentiments. These heatmaps indicate how two sentiments have been expressed and provides more insights regarding positive and negative sentiments. Fig. 12 provides visualisation of tweets distribution with number of combination sentiments.
Fig. 9 Heatmap depicting occurrences of given sentiment with respect to remaining sentiments for tweets from training dataset [36]
Fig. 10 Heatmap showing number of occurrence of given sentiment with respect to remaining sentiments for India datasets using Hd-RNN [36]
Fig. 11 Heatmap showing number of occurrence of given sentiment with respect to remaining sentiments for Maharashtra datasets using Hd-RNN [36]
4.2 Validation of COVID19 Results
After an initial analysis of results in previous section, we present further validation of obtained results here. In order to achieve this cross validation is used. This is a resampling method which evaluates developed models on small data samples. The given data samples are divided into number of groups such that we have
The success of
Since COVID19 datasets have an uneven class distribution, most intuitive performance metric to be used here is precision. Closely associated with precision is recall performance metric which also works well here as well as F1 Score which is weighted average of precision and recall. Another important performance metric which could have been used here is accuracy but because datasets are not symmetric, it will work well here.
In order to make our experimental hypothesis stronger we now present comparative performance analysis of d-RNN and Hd-RNN models with some traditional machine learning models such as NB, SVM, kNN, RF, GB, AB and DT. Table 6 highlights
Table 6 Comparative analysis of
| Models |
|
|
|
|
|
test level |
| NB | 0.79 | 0.80 | 0.82 | 0.79 | 0.84 | 0.84 |
| SVM | 0.82 | 0.84 | 0.85 | 0.87 | 0.87 | 0.87 |
| kNN | 0.75 | 0.76 | 0.77 | 0.78 | 0.79 | 0.79 |
| RF | 0.79 | 0.77 | 0.78 | 0.80 | 0.79 | 0.80 |
| GB | 0.84 | 0.85 | 0.87 | 0.84 | 0.85 | 0.87 |
| AB | 0.85 | 0.84 | 0.82 | 0.84 | 0.85 | 0.85 |
| DT | 0.77 | 0.78 | 0.79 | 0.80 | 0.80 | 0.80 |
| d-RNN | 0.95 | 0.96 | 0.97 | 0.96 | 0.96 | 0.96 |
| Hd-RNN | 0.96 | 0.97 | 0.98 | 0.99 | 0.97 | 0.99 |
Table 7 Comparative analysis of
| Models |
|
|
|
|
|
test level |
| NB | 0.80 | 0.80 | 0.82 | 0.84 | 0.85 | 0.85 |
| SVM | 0.84 | 0.85 | 0.87 | 0.88 | 0.89 | 0.89 |
| kNN | 0.77 | 0.78 | 0.79 | 0.80 | 0.80 | 0.80 |
| RF | 0.80 | 0.79 | 0.79 | 0.82 | 0.80 | 0.82 |
| GB | 0.85 | 0.87 | 0.87 | 0.88 | 0.85 | 0.88 |
| AB | 0.87 | 0.85 | 0.84 | 0.85 | 0.87 | 0.87 |
| DT | 0.79 | 0.79 | 0.82 | 0.82 | 0.82 | 0.82 |
| d-RNN | 0.96 | 0.97 | 0.97 | 0.97 | 0.97 | 0.97 |
| Hd-RNN | 0.97 | 0.98 | 0.98 | 0.99 | 0.99 | 0.99 |
Table 8 Comparative analysis of
| Models |
|
|
|
|
|
test level |
| NB | 0.82 | 0.84 | 0.84 | 0.80 | 0.85 | 0.85 |
| SVM | 0.84 | 0.85 | 0.87 | 0.88 | 0.89 | 0.89 |
| kNN | 0.77 | 0.78 | 0.79 | 0.80 | 0.80 | 0.80 |
| RF | 0.82 | 0.79 | 0.80 | 0.82 | 0.80 | 0.82 |
| GB | 0.85 | 0.87 | 0.88 | 0.85 | 0.88 | 0.88 |
| AB | 0.87 | 0.85 | 0.84 | 0.85 | 0.87 | 0.87 |
| DT | 0.79 | 0.80 | 0.80 | 0.80 | 0.80 | 0.80 |
| d-RNN | 0.96 | 0.97 | 0.98 | 0.97 | 0.98 | 0.98 |
| Hd-RNN | 0.98 | 0.98 | 0.98 | 0.99 | 0.99 | 0.99 |
Table 9 Comparative analysis of
| Models |
|
|
|
|
|
test level |
| NB | 0.79 | 0.79 | 0.80 | 0.82 | 0.84 | 0.84 |
| SVM | 0.82 | 0.84 | 0.85 | 0.86 | 0.88 | 0.88 |
| kNN | 0.76 | 0.77 | 0.78 | 0.79 | 0.79 | 0.79 |
| RF | 0.79 | 0.78 | 0.78 | 0.80 | 0.79 | 0.80 |
| GB | 0.84 | 0.85 | 0.85 | 0.87 | 0.84 | 0.87 |
| AB | 0.85 | 0.84 | 0.82 | 0.84 | 0.87 | 0.87 |
| DT | 0.78 | 0.78 | 0.80 | 0.80 | 0.79 | 0.79 |
| d-RNN | 0.95 | 0.96 | 0.96 | 0.96 | 0.96 | 0.96 |
| Hd-RNN | 0.96 | 0.97 | 0.97 | 0.98 | 0.98 | 0.98 |
As BERT model has been used here for evaluating test datasets with prediction validation, we use different variants of BERT [36] as well as LSTM [36] and BD-LSTM [36] models as shown in Table 10 in order to compare performance of d-RNN and Hd-RNN models.
Table 10 Comparative analysis of BERT variants, LSTM and BD-LSTM with d-RNN and Hd-RNN models (L: hidden layers, H: hidden size, A: attention heads) [36]
| BERT/Other Models | Accuracy | Precision | Recall | F1 Score |
| L-2 H-128 A-2 | 0.61 | 0.60 | 0.57 | 0.61 |
| L-2 H-256 A-4 | 0.82 | 0.80 | 0.77 | 0.79 |
| L-2 H-512 A-8 | 0.84 | 0.82 | 0.85 | 0.85 |
| L-2 H-768 A-12 | 0.84 | 0.80 | 0.85 | 0.85 |
| L-4 H-128 A-2 | 0.80 | 0.79 | 0.78 | 0.80 |
| L-4 H-256 A-4 | 0.82 | 0.84 | 0.85 | 0.84 |
| L-4 H-512 A-8 | 0.84 | 0.82 | 0.80 | 0.82 |
| L-4 H-768 A-12 | 0.87 | 0.84 | 0.85 | 0.84 |
| L-6 H-128 A-2 | 0.82 | 0.80 | 0.84 | 0.82 |
| L-6 H-256 A-4 | 0.82 | 0.84 | 0.82 | 0.82 |
| L-6 H-512 A-8 | 0.85 | 0.82 | 0.82 | 0.82 |
| L-6 H-768 A-12 | 0.87 | 0.85 | 0.84 | 0.84 |
| L-8 H-128 A-2 | 0.80 | 0.80 | 0.82 | 0.82 |
| L-8 H-256 A-4 | 0.82 | 0.84 | 0.82 | 0.82 |
| L-8 H-512 A-8 | 0.85 | 0.82 | 0.82 | 0.82 |
| L-10 H-128 A-2 | 0.82 | 0.80 | 0.84 | 0.82 |
| L-10 H-256 A-4 | 0.84 | 0.80 | 0.85 | 0.85 |
| L-10 H-512 A-8 | 0.84 | 0.82 | 0.84 | 0.85 |
| L-12 H 128 A-2 | 0.82 | 0.80 | 0.87 | 0.87 |
| L-12 H-256 A-4 | 0.85 | 0.80 | 0.80 | 0.85 |
| L-12 H-512 A-8 | 0.89 | 0.84 | 0.89 | 0.89 |
| LSTM | 0.90 | 0.93 | 0.94 | 0.95 |
| BD-LSTM | 0.93 | 0.94 | 0.95 | 0.96 |
| d-RNN | 0.96 | 0.95 | 0.97 | 0.97 |
| Hd-RNN | 0.99 | 0.96 | 0.95 | 0.99 |
5 Conclusion
In this work we have presented sentiment analysis of COVID19 infections with d-RNN and Hd-RNN as major computational models. The time span considered is from January 2020 to June 2021. We considered tweets from different regions of India. We reviewed tweets from specific regions such as Maharashtra, Delhi, Karnataka, Kerala and Tamilnadu. The models are trained with COVID19 datasets hand labelled tweets. The majority of tweets have highlighted optimism, fear and uncertainty during infections of COVID19 cases in India. There has been variability in number of tweets during peak of new cases.
The predictions indicate that although majority of population have been optimistic, a significant group has been disturbed by way pandemic was handled by Indian government.
This computational framework can be used for better COVID19 management in order to support cases of depression and mental health issues. The experimental results are validated considering various traditional machine learning as well as different variants of BERT models. The results with d-RNN and Hd-RNN highlight superiority of proposed methods.
The computational model can be used for different regions, countries, ethnic and social groups. This can also be extended to understand reactions towards vaccinations with rise of antivaccine sentiments given fear, insecurity and unpredictability of COVID19 situations. This computational framework incorporates topic modelling with sentiment analysis which provides more details during COVID19 cases with respect to various government rules and regulations. As concluding remarks we would like to mention that present Indian government has been unsuccessful in addressing stress and strain through which country’s economy is passing through.
