Introduction and The State of the Question

People’s well-being, beyond the economic aspect, depends largely on access to knowledge (^{Bernal, Frost & Sierra, 2014}). In particular, scientific knowledge can be extremely useful for academics as well as anyone who can benefit from it. According to ^{Ruíz-Pérez, Marcos-Cartagena and López-Cózar (2014)}, periodicals are the primary means of communication among the scientific community, which makes them the public record of knowledge. However, their use had been limited almost exclusively to academic circles that had the opportunity to access specialized journals.

Many studies have highlighted the importance of open access for scientific progress (^{Más-Bleda, Thelwall, Kousha & Aguillo, 2014}). With the explosion of Open Access Journals, the economic problem of accessing scientific knowledge is overcome (^{Nazim & Husain, 2013}). Presently, anyone with Internet connectivity and basic computer skills can access a large number of journals and scientific articles.

Open access takes scientific knowledge beyond academic boundaries and makes it available to all sectors (^{Berlin Declaration, n.d.}; ^{Budapest Open Access Initiative, n.d.}; ^{Gul, Shah & Nisa, 2014}; ^{Morrison, 2009}). Despite this, having access to scientific articles is not necessarily reflected in their use. One of the possible reasons for not reading a useful article could be because its existence is unknown.

Along with the growth of open access journals, there has been a growing incorporation of digital social tools within the academic and scientific world (^{Crawford, 2011}). Despite their original recreational nature, some authors acknowledge that if they are used well, they can be very important in disseminating intellectual production by academics (^{Bik & Goldstein, 2013}; ^{Darling, Shiffman, Cȏté & Drew, 2013}; ^{Osterrieder, 2013}; ^{Rogel-Salazar, Santiago-Bautista & Martínez-Domínguez, 2017}).

Not only are social networks, such as Twitter, Facebook, or LinkedIn, that are typically considered recreational used but also an increasing number that focus on the scientific-academic field are being developed. It is not uncommon for professors and researchers to use academic networks such as Researchgate and Academia.edu, or reference managers such as Mendeley and CiteULike, to conduct some activity related to their academic or research role. According to ^{Owens (2014)}, sites like Researchgate or Academia.edu are started due to the frustration caused by the complexity of sharing research results. These tools have the advantage of specialization and the existence of a greater link between researchers, facilitating communication, and collaboration among colleagues (^{Arévalo, Cordón, Gómez & García, 2014}; ^{Campos, Rivera & Rodríguez, 2014}; ^{Crawford, 2011}; ^{Martorell & Canet, 2013}; ^{Merlo et al., 2010}; ^{Thelwall & Kousha, 2014}; ^{Van Noorden, 2012}).

In contrast, it is not only researchers and academics that use these social and recreational tools but also the journals themselves. In fact, ^{López-Ornelas, Osuna and Díaz (2017)} claim that the fact that journals have social networks as distribution channels for their articles is part of the quality criteria, in terms of technology, of an academic publication. Whether you are an individual, typically an academic or researcher, or a specialized journal, the reasons for using this type of tool can be very diverse. Researchers can use them to establish collaborative networks, keep up with new research, or promote their work. Furthermore, journals can use them to give their readers attention or to disseminate the articles that are published in their journals.

The impact of a research study depends primarily on how well publications (journal articles, lectures, and books) are disseminated to the final user (^{Allen, Stanton, Pietro & Moseley, 2013}) . In this regard, this study establishes the objective of identifying the possible effect of the dissemination of scientific articles published in open access journals on the impact indicators of these articles.

Journals’ adoption of open access provided the opportunity to increase citations and improve the visibility of academic publications (^{Gul, Shah & Nisa, 2014}). According to ^{Priem, Taraborelli, Grot and Neylon (2010)}, as academic literature grows exponentially, academics rely on certain filters to select the most relevant publications. One of those filters is the number of citations. However, authors find this variable useful but insufficient, among other factors, because in addition to ignoring the impact of works outside the academic field, it is possible that some very influential works remain uncited

Thus, with changes in traditional academic publication, an important challenge is to identify the impact of scientific publications beyond the classical measures, such as counting citations or the journal’s impact factor. According to ^{Más-Bleda, Thelwall, Kousha and Aguillo (2014)}, in the last two decades they have attempted to develop indicators to evaluate research beyond the traditional bibliometric measurements. For ^{Uribe-Tirado and Alhuay-Quispe (2017)}, these traditional bibliometric metrics do not allow for a full understanding of the presence or influence of articles in the social web environment. According to these authors, it is this dynamic that has driven the movement known as altmetrics.

Altmetrics intend to capture online activity surrounding a scientific publication by tracking metrics such as the following: number of downloads, recommendations, number of readers, and discussion in social networks (^{Osterrieder, 2013}; ^{Priem, Taraborelli, Groth & Neylon, 2010}; ^{Uribe-Tirado & Alhuay-Quispe, 2017}).

According to ^{Eysenbach (2011)}, one of the disadvantages of citations as a variable of an article’s scientific impact is that it takes long for them to be reflected. In addition, citations only measure their significance in the scientific community but not in the general public. In this sense, ^{Smith (2001)} claims that in the real world, scientific quality and social impact are not always in close association, which is why concepts such as social impact should be part of any research study’s assessment.

Altmetrics offer a promising outlook that is worth considering when measuring the impact of scientific and academic work. However, it is so novel that it must still solve theoretical (meaning), methodological (validity of sources), and technical (normalization) problems (^{Torres-Salinas, Cabezas-Clavijo & Jiménez-Contreras, 2013}). Considering this, the research objective was to analyze the potential of online social networks as tools for article design, taking into account the indicators found within the spectrum of altmetrics.

Materials and methods

This study analyzes the relationship between altmetrics and other impact indicators associated with scientific articles published in open access journals. Given the nature of this question, a quantitative study was proposed. The research design is based on ^{Rogers’s concept of digital methods (2015)}, which consists of a set of techniques for the study of social and cultural phenomena through data available on the web. These techniques use digital resources such as websites, URLs, hyperlinks, tags, likes, and tweets among other tools created on various Internet platforms (content managers, blogs, social networking sites, and directories).

To address this question, articles published by PLOS ONE and Scientific Reports in 2015 were used. This choice was made because these academic journals had the highest volume of publication in that year (^{Wakeling, Willett et al., 2016}). It should be noted that these are open access publications. The reference period was selected in order to allow the articles to have a sufficient level of maturity, especially regarding the number of citations received.

To collect data from the journal’s websites, the web scraping technique was used. The hierarchical structure of the web pages allows for collection through a sequence of commands (^{Marres & Weltevrede, 2013}).

With PLOS ONE, metrics were obtained for 29 808 (99.9%) of the 29 835 articles indexed in Scopus for 2015, while for Scientific Reports, metrics were obtained for 10 718 (97.7%) of the 10 967 articles indexed in Scopus for that same period. Metrics for the missing items were not available at the time of collection. The data used for this analysis was collected in July 2017.

On PLOS ONE, a list of comprehensive impact assessment metrics at the article level is available. Specifically, there are four metrics: a) the number of times the article is saved as reference on Mendeley and CiteULike, which are reference management systems; b) the number of times an article is cited in the Scopus database; c) the number of times the article is visited and downloaded on the journal’s site; and d) the number of times the article is shared on Facebook and Twitter. The publication date was also extracted as a control variable, which is expressed as a publication date number in ascending order. On Scientific Reports, there are two metrics available: a) the number of times an article is cited from information on Web of Science, CrossRef, and Scopus; and b) altmetric, which consists of an indicator composed of mentions and number of times an article is shared on social networks (including Facebook and Twitter), blogs, news sites, and bookmarking. The date of publication is also available.

Analysis and results

Is there a relationship between the number of times an article is shared through the journal’s social networks and citation volume?

To examine the relationship between the metrics at the article level, the respective Pearson correlation coefficients are presented in Table 1. Regarding the PLOS ONE metrics, the positive and moderate correlation between the number of times an article is shared through Facebook and Twitter with the number of times the document visited and downloaded stands out. There is also a positive and moderate correlation between the number of citations an article receives and the number of times it is saved as reference (Mendeley and CiteULike).

The number of times an article is shared through Facebook and Twitter is weakly correlated with the number of citations. On Scientific Reports, there was a weak correlation between the altmetric indicator and the number of citations. In both journals, the publication date reported weak and negative correlations with all the metrics analyzed.

For the PLOS ONE metrics, two linear regression models were proposed. In both models, the publication date and the number of times the article is shared on social networks act as predictor variables. In the first, the response variable is the number of times the article is cited (F(2, 29805)= 1 086, p< 0.001 with an R2 of 6.79%, Cite= 18 698.45 + 22.17*Share - 31.99*Date, with both significant predictors). In the second model, the number of times the article is visited and downloaded acts as a response variable (F(2, 29805)= 3 475, p< 0.001 with an R2 of 18.90%, View= 1 799.94 + 10.85*Share - 2.36*Date, with both significant predictors).

For Scientific Reports, a linear regression model was proposed. In that the article’s number of citations was assigned as a response variable, while the altmetrics indicator and the date of publication were predictor variables (F(2, 10 715)= 1 404, p< 0.001 with an R2 of 20.77%, Cite= 3 674.12 + 0.43*altmetrics - 0.12*Date, with both significant predictors). All the variables from the regression analysis previously received a rank transformation to reduce the non-normality effect.

Discussion and conclusions

Regarding the possible relationship between an article’s digital social impact and the number of citations it generates, although the social indicators used by journals are not precisely equal to each other, positive but weak correlations were found (r= 0.10). These results coincide with previous studies (^{Allen, Stanton, Pietro, & Moseley, 2013}; ^{Eysenbach, 2011}; ^{Haustein Peters, Sugimoto, Thelwall, & Larivière, 2014}; ^{De Winter, 2015}) where they found essentially the same patterns, although in several only a single social network was analyzed: Twitter. ^{Eysenbach (2011)} examined 4 208 tweets in which 286 different articles were cited. In performing the correlation analysis, the researcher identified that citations in Google Scholar are more closely correlated with the number of tweets than Scopus citations. In contrast, Haustein and colleagues (2014) found something very similar; the correlations between the number of tweets sharing an article and the number of citations generated were similarly low (r= 0.183). ^{De Winter (2015)}, who analyzed 27 856 articles from PLOS ONE, found an equally weak predictive power of the number of tweets regarding the number of citations in Scopus (β= 0.10) and a stronger prediction between the number of tweets and the number of views the article receives (β= 0.38). Finally, after sharing 16 PLOS ONE articles on Facebook, Twitter, LinkedIn, and a blog, ^{Allen, Stanton, Pietro and Moseley (2013)} found that both views and downloads increased, but there was no effect on the number of citations one year later.

Even though their use is not widespread, both academic networks and social reference management systems, as well as recreational social networks, can be important tools to increase the impact of articles published in open access journals.

Recreational networks such as Facebook and Twitter may not have an impact (at least in the short term) on the number of citations in Scopus, that is, in the scientific environment, but they do have an impact on the number of times the general public is exposed to the publication, in contrast to activity with reference managers (such as Mendeley and CiteULike), which can impact the number of citations and, to a much lesser extent, on the general public’s exposure.

However, there is the possibility that although the articles are widely disseminated through social networks (for example, they may be shared and forwarded many times), in reality they are not read that many times. It is likely that many people who disseminate articles through social networks never actually read them, and therefore it is logical that they are not cited because they are not even read. This behavior, if true, could be related to the general public’s tendency to share information that seems appealing. This possibility encourages the completion of possible future research that helps to identify what percentage of the articles disseminated by final users of online social networks is actually read.

In this sense, an explanation of the difference between academic citations and altmetrics could lie in the difference between preferences of researchers and the general public, as proposed by ^{Haustein et al. (2014)}. According to ^{Van Noorden (2012)}, the general public, which tends to make an article popular with social tools, prefers more “catchy” topics, while scientists look for articles that are useful for their research work.

However, according to ^{De Winter (2015)}, the lack of a relationship between the number of times an article is shared through online social networks and the number of citations does not imply that these tools are useless for the scientific field. The relationship with altmetrics allows for the assumption that this dissemination facilitates scientific knowledge being accessed in greater volume by a public distinct from academics. This can generate another type of social impact or simply increase interest in a particular topic.

The results of this study should be interpreted according to the limitations that the study may have. An important limitation is the control over time. The time it takes for an article to have an impact in the academic and scientific world can be prolonged. The review periods of some indexed journals can be more than a year. Thus, the time that may elapse from when academics receive an article through social networks, download it, read it, use it for their own research, send their own publication for review, pass the review period, make the adjustments, and finally publish the work, providing the original article with a citation, could be very long. A longitudinal study could deal with this limitation more effectively.

By contrast, beyond the time required for an article to establish itself, a possible reason for the lack of relationship between altmetrics and academic citations may lie in the limited use of social tools by researchers themselves. It is likely that researchers do not use social digital tools within their intellectual production process. Actions such as following profiles on social networks (Twitter, Facebook, etc.) of the most important journals in their field, making note of references using tools such as Mendeley, or participating in academic social networks could be productive in their process of intellectual creation. “To what extent do researchers use digital social tools in their process of developing scientific articles?” is a possible question for future research. “Do academic researchers have the necessary competencies to take advantage of social tools in their scientific production process?” could be another question.