2.1. Mexican Spanish resources

In the previous section, one can notice that there are several options available for the Spanish language, but when one focuses in a dialect such as Mexican Spanish, the resources are scarcer. In the literature one can find several articles dedicated to the creation of speech resources for the MexicanSpanish, (^{Kirschning, 2001}; ^{Olguín-Espinoza, Mayorga-Ortiz, Hidalgo-Silva, Vizcarra-Corral, & Mendiola-Cárdenas, 2013}; ^{Uraga & Gamboa, 2004}). However, researchers usually create small databases to do experiments so one has to contact the authors and depend on their good will to get a copy of the resource (^{Audhkhasi, Georgiou, & Narayanan, 2011}; de ^{Luna Ortega, Mora-González, Martínez-Romo, Luna-Rosas, & Munoz-Maciel, 2014}; ^{Moya, Hernández, Pineda, & Meza, 2011}; ^{Varela, Cuayáhuitl, & Nolazco-Flores, 2003}).

Even though the resources in Mexican Spanish are scarce,we identified 7 corpora which are easily available. These are presented in Table 2. As shown in the table, one has to pay inorder to have access to most of the resources. The notable exception is the DIMEx100 corpus (^{Pineda et al., 2010}) which was recently made available.⁷ The problem with this resource is that the corpus is composed for reading material and it is only 6 h long which limits the type of acoustic phenomena present. This imposes a limit on the performance of the speech recognizer created utilizing this resource (^{Moya et al., 2011}). In this work we present the creation of the CIEMPIESS corpus and its open resources. The CIEMPIESS corpus consists of 17 h of recordings of Central Mexico Spanish broadcast of interviews which provides spontaneous speech. This makes it a good candidate for the creation of speech recognizers. Besides this, we expose the methodology and tools created for the harvesting of the different aspects of the corpus so these can be replicated in other underrepresented dialects of Spanish.

3.1. Audio collection

The original source of recordings used to create the CIEM-PIESS corpus comes from radio interviews in the format of a podcast.⁸ We chose this source because they were easily available, they had several speakers with the accent of Central Mexico, and all of them were freely speaking. The files sum together a total of 43 one-hour episodes.⁹ Table 3 summarizes the main characteristics of the original version of these audio files.

3.2. Segmentation of utterances

From the original recordings, the segments of speech need to be identified. To define a “good” segment of speech, the following criteria was taken into account:

At the end, 16,717 utterances were identified. This is equivalent to 17 h of only “clean” speech audio. 78% of the segments come from male speakers and 22% from female speakers.¹⁰ This gender imbalance is not uncommon in other corpora (for example see ^{Federico, Giordani, & Coletti, 2000}; ^{Wang, Chen, Kuo,& Cheng, 2005} since gender balancing is not always possible as in ^{Langmann, Haeb-Umbach, Boves, & den Os, 1996}; ^{Larcher,Lee, Ma, & Li, 2012}). The segments were divided into two sets: training (16,017) and test (700), the test set was additionally complemented by 300 utterances from different sources suchas: interviews, broadcast news and read speech. We added these 300 utterances from a little corpus that belong to our laboratory, to perform private experiments that are important to some ofour students. Table 4 summarizes the main characteristics of the utterance recordings of the CIEMPIESS corpus.¹¹

The audio of the utterances was standardized into recordings sampled at 16 kHz, with 16-bits in a NIST Sphere PCM mono format with a noise removal filtering when necessary.

3.3. Orthographic transcription of utterances

In addition to the audio collection of utterances, it was necessary to have their orthographic transcriptions. In order to createthese transcriptions we followed these guidelines: First, the process begins with a canonical orthographic transcription of every utterance in the corpus. Later, these transcriptions were enhanced to mark certain phenomena of the Spanish language.

The considerations we took into account for the enhanced version were:

3.4 Enhanced transcription

As we have mentioned in the previous section, the orthographic transcription was enhanced by adding information of the pronunciation of the words and the disfluencies in the speech.The rest of this section presents such enhancements.

Most of the words in Spanish contain enough information about how to pronounce them, however there are exceptions. In order to facilitate the automatic creation of a dictionary, we added information of the pronunciation of the x’s which has several pronunciation in the Mexican Spanish. Annotators were asked to replace x’s by an approximation of its possible pronunciations. Doing this, we eliminate the need for an exception dictionary. Table 5 exemplifies such cases.

Another mark we annotate in the orthographic transcription is the indication of the tonic vowel in a word. In Spanish, a tonic vowel is usually identified by a raise in the pitch; sometimes this vowel is explicitly marked in the orthography of the word by an acute sign (e.g. “acción”, “vivía”). In order to make this difference explicit among sounds, the enhanced transcriptions also marked as such in both the explicit and implicit cases (Table 6 exemplifies this consideration). We did this for two reasons, the most important is that we want to explore the effect of tonic and nontonic vowels in the speech recognition, and the other oneis that some software tools created for HTK or SPHINX do not manage characters with acute signs properly, so the best thingto do is to use only ASCII symbols.

Finally, silences and disfluencies were marked following this procedure:

Table 7 compares a canonical transcription and its enhanced version.

Both segmentation and orthographic transcriptions in their canonical and enhance version (word alignment) are very time consuming processes. In order to transcribe and align the fullutterances, we collaborated with 20 college students investing 480 h each to the project over two years. Three of them made the selection of utterances from the original audio files using the Audacity tool (^{Team, 2012}) and they created the orthographic transcriptions in a period of six months, at the rate of one hour per week. The rest of collaborators spent one and a half year aligning the word transcriptions with the utterances for detecting silences and disfluencies. This last step implies that orthographic transcriptions were checked at least twice by two different person. Table 8 shows a chronograph of each task done per half year.

3.5. Distribution of words

In order to verify that the distribution of words in CIEM-PIESS corresponds to the distribution of words in the Spanish language, we compare the distribution of the functional words in the corpus versus the “Corpus de Referencia del Español Actual”(Reference Corpus of Current Spanish, CREA).¹² The CREA corpus is conformed by 140,000 text documents. That is, more than 154 million of words extracted from books (49%), news-papers (49%), and miscellaneous sources (2%). It also has more than 700,000 word tokens. Table 9 illustrates the minor differences between frequencies of the 20 most frequent words inCREA and their frequency in CIEMPIESS.¹³ As one can see, the distribution of functional words is proportional between both corpora.

We also calculate the mean square error (MSE = 9.3 × 10−8) of the normalized frequencies of the words between the CREA and the whole CIEMPIESS so we found that it is low and the correlation of the two distributions is 0.95. Our interpretation of this is that the distribution of words in the CIEMPIESS reflects the distribution of words in the Spanish language. We argue that this is relevant because the CIEMPIESS is then a good simple that reflects well the behavior of the language that it pretends to model.

4.1. Phonetic alphabet

In this work we used the MEXBET phonetic alphabet thathas been proposed to encode the phonemes and the allophones of Mexican Spanish (^{Cuetara-Priede, 2004}; ^{Hernández-Mena, Martínez-Gómez, & Herrera-Camacho, 2014}; ^{Uraga & Pineda, 2000}). MEXBET is a heritage of our University and it has been successfully used over the years in several articles and thesis. Nevertheless, the best reason for choosing MEXBET is that thisis the most updated phonetic alphabet for the Mexican Spanish dialect.

This alphabet has three levels of granularity from the phonological (T22) to the phonetic (T44 and T54).¹⁴ For the purpose of the CIEMPIESS corpus we extended the T22 and T54 levels to what we call T29 and T66.¹⁵ In the case of T29 these are the main changes:

All these changes were motivated by the need to produce more accurate phonetic transcriptions following the analysis of ^{Cuetara-Priede (2004)}. Table 10 illustrates the main difference between T54 and T66 levels of the MEXBET alphabets.

Table 11 shows examples of different Spanish words tran-scribed using the symbols of the International Phonetic Alphabet (IPA) against the symbols of MEXBET.¹⁶

Table 12 shows the distribution of the phonemes in the automatically generated dictionary and compares it with the DIMEx100 corpus. We observe that both corpora share a similar distribution.¹⁷

4.2 Characteristics of the pronouncing dictionaries

The pronunciation dictionary consists of a list of words forthe target language and their pronunciation at the phoneticor phonological level. Based on the enhanced transcription, we automatically transcribe the pronunciation for each word.For this, we followed the rules from ^{Cuetara-Priede (2004)} and ^{Hernández-Mena et al. (2014)}. The produced dictionary consists of 53,169 words. Table 13 shows some examples of the automatically created transcriptions.

The automatic transcription was done using the fonética 2¹⁸ library, that includes transcription routines based on rules for theT29 and the T66 levels of MEXBET. This library implements the following functions:

6.1. Automatic transcription

In these evaluations we measure the performance of different functions of the automatic transcription. First we evaluated the performance of the vocal tónica () function which indicates the tonic vowel of an incoming Spanish word. For this, we randomly took 1452 words from the CIEMPIESS vocabulary (12%of the corpus) and we predicted their tonic transcription. The automatic generated transcriptions were manually checked byan expert. The result is that 90.35% of the words were correctly predicted. Most of the errors occurred in conjugated verbs and proper names. Table 18 summarizes the results of this evaluation.

The second evaluation focuses on the T29 () function which transcribes words at the phonological level. In this case we evaluated against the TRANSCRÍBEMEX (^{Pineda et al., 2004}) and the transcriptions done manually by experts for the DIMEx100 corpus (Pineda et al., 2004). In order to compare with the system TRANSCRÍBEMEX, we took the vocabulary of the DIMEX100 corpus but we had to eliminate some words. First we removed entries with the archiphonemes²⁰: [-B], [-D], [-G], [-N], [-R] since they are not a one-to-one correspondence with a phonological transcription. Then, words with the grapheme x were eliminated since TRANSCRÍBEMEX only supports one of four pronunciations. After this, both systems produced the same transcription 99.2% of the times. In order to evaluate against transcriptions made by experts, we took the pronouncing dictionary of the DIMEX100 corpus and we removed words with thex phonemes and the alternative pronunciations if there was any.This shows us that the transcriptions made by our T29 () function were similar to the transcriptions made by experts 90.2% of the times.

Tables 19 and ²⁰ summarizes the results for both comparisons. In conclusion, besides the different conventions there isnot a noticeable difference, but when compared with humanexperts, there still room for improvement of our system.

6.2. Benchmark systems

We created three benchmarks based on state of the art systems: HTK (^{Young et al., 2006}), Sphinx (^{Chan et al., 2007}; ^{Lee et al., 1990}) and Kaldi (^{Povey et al., 2011}). The CIEMPIESS corpus is formatted to be used directly in a Sphinx setting. In thecase of HTK we created a series of tools that can read directlythe CIEMPIESS corpus²¹ and finally for Kaldi we created the setting files for the CIEMPIESS. We set up a speech recognizer for each system using the train set of the CIEMPIESS corpus and we evaluated the performance utilizing its test set. Every system were configured with their corresponding default parameters and a trigram-based language model. Table 21 shows the performance for each system.

6.3. Tonic vowels

Given the characteristics of the CIEMPIESS corpus, we decided to evaluate the effect of the tonic vowels marked in the corpus. For this we trained four acoustic models for the Sphinx system. These were tested in the corpus using the standard language model of the corpus. Table 22 presents the Word error rate (WER, the lower the better) for such cases. We can observe that the distinction of the tonic vowel helps improve the performance.²² However, the use of phonetic transcriptions (T66 level of MEXBET) does have a negative effect on the performance of the speech recognizer.

Using the same configurations found with the experiment ofthe tonic vowels, we created four learning curves. Fig. 3 presentsthe curves, we also can notice that a phonetic transcription (T66) was not beneficial while using a phonological (T29) even with a small amount of data yields to a better performance.

Fig. 3 Learning Curves for different training conditions.