1 Introduction

This article is focused on the use of technology to identify people in collapsed buildings. The objective is to develop RICB, a prototype of a telemetry device with sensors and Artificial Neural Network (ANN) model to predict the presence of people in collapsed buildings. It must be taken into account that this work contributes in an original way the application in a system with commercial sensors and a sensor of our own elaboration that is described in Section 3.1 and that it does not exist in other works already published.

There are three main levels of challenges in the topic about rescuing people using Artificial Intelligence:

RICB's contribution is about the challenge 1 mentioned above, mainly in the management of the data that the sensors identify, by applying an ANN in which an algorithm learns through the data training set, speeding up the search time. Another contribution is the insertion of the device (Raspberry Pi Zero W) in an acrylic sphere that protects the RICB. The objective of the RICB project is not to make a robot, but to continue strengthening the management of information in the core of a robot.

Telemetry devices are being used in various areas such as health, for example, in [¹] a system for the analysis and recognition of human activities using a ANN model and integrated into a Raspberry Pi to predict these activities in real time. In [²], a development of an autonomous mobile garbage collector that detects objects. Other work that uses pedestrian recognition [³], includes a comparison of machine learning and deep learning algorithms for pedestrian recognition in an autonomous vehicle.

Other works such as [⁴-⁶] are Systems for Internet of Things (IoT) used to locate victims in the disaster mitigation, for fire rescue and to detect landslides.

For the development of RICB, the methodology of prototypes has been used. The platform on which it has been implemented is Unix. Mostly open source software tools are used and it is coupled with the architecture of the single-board computer.

White box tests were performed, errors were corrected. A classroom in the Applied Sciences building of the Universidad Autónoma del Estado de México (UAEMex) was taken as the proof scenario. Different tests were carried out with 19 different situations, for example: under, and on furniture, with noise and movement. The results are presented in the corresponding section.

The article is organized as follows: Section 1 describes the theoretic frame. Section 2 has a brief analysis of the related works that solves problems in different fields. In Section 3 the description of the RICB is presented and its details. In Section 4 the tests are presented showing the functionality of the model and Section 5 contains the conclusions and future works.

2 Theoretical Framework

A sensor is an object that can measure some physical or chemical quantity and transform it into electricity through a transducer. The chemical or physical quantities that can be measured are called instrumentation variable.

These can be of different types and can be: luminosity, temperature, speed, distance, acceleration, inclination, displacement, force, torsion, humidity. A sensor is in contact with the instrumentation variable that conditions it, unlike the transducer that only converts these signals into voltage.

There are several characteristics and types of sensors. Those that have been used in the telemetry device are those classified according to the nature of their operation, such as: position, photoelectric, magnetic, temperature, humidity, pressure, movement and chemicals.

The Raspberry Pi Zero W is a Simple Board Computer (SBC) because on a single board it contains all the necessary components of a computer. Its main characteristic is the small size, 66.0 mm x 30.5 cm x 5.0 cm that are the dimensions of the Raspberry Pi Zero W. There are mini-ITX boards and that in turn have more powerful processors such as Intel. SBC such as the Raspberry Pi Zero W do not subtract merits [⁷].

3 Related Works

Mexico is a country that is constantly affected by natural disasters, specifically earthquakes, due to five tectonic plates interacting and three geological faults that cover many parts of Mexican territory. Southeast Mexico is one of the parts with more seismic activity. This difficult feature has caused serious damage: houses, buildings and victims.

The lack of an efficient contingency system for disasters causes a waste of time in recovering victims. All types of communication links are collapsed after earthquakes.

When the General Hospital of Mexico City collapsed due to an earthquake that occurred on September 19, 1985 almost the 64% of people inside died, and only 24% were rescued [⁸].

In the Ecuador earthquake, of 2016, there were 659 victims and only 17 % were rescued [⁹].

Having a high level of control of technologies is important when a disaster occurs, because it represents a key to measure damages. The use of technologies in the victim search would increase the response time of the rescue people.

In spite of there being institutions with the aim of dealing with disasters in Mexico, there are few technology projects that decrease the time of rescuing people. So, RICB is a telemetry device with sensors that interacts between each other and artificial intelligence tools for rescuing victims inside collapsed buildings. RICB thus offers important contributions for the natural disaster problems. In this paper how RICB works is explained; its design using ANN, and many tests to measure its functionality are provided.

ANN has been used in applications of many areas such as health and ecology authors use a Recurrent Neural Network (RNN) with internal memory that makes it robust and powerful for Human Activity Recognition (HAR). The main component of the data acquisition phase, which uses a sensor BlueNRG-Tile that measures various attributes such as acceleration and velocity. The other component is the ST-BLE sensor application, that is used to collect and preprocess the sensor signal, the network communication and a server to save the data. The sensors used are the following: tri-axial accelerometer sensor and the velocity tri-axial gyroscope. A data set with three activities was used: sitting, walking and running. The accelerometer and gyroscope were placed on the right foot of two boys and three girls. It was also implemented in an ANN and a Raspberry Pi to validate the simulations obtained. The similarity of this project and RICB is that they both detect movement in real time. At the end of this test, an accuracy of 99% was obtained. In a real time environment, the accuracy was 86%.

Other related work is about pedestrian recognition [³], author present a comparison of machine learning and deep learning algorithms for pedestrian recognition in an autonomous vehicle. The system proposed uses ultrasonic sensors to detect obstacles, it was implemented in a mobility scooter that was modified to be fully autonomous using Raspberry Pi 3 as a controller. Two computer vision algorithms of Histogram of Oriented Gradient (HOG) descriptor and Haar classifier were trained and tested.

HOG descriptor was the superior algorithm for detecting pedestrians compared to Haar-classifier with an accuracy of around 83%. The deep learning had an accuracy of around 88%. Ultrasonic sensors were tested for time delay for obstacle avoidance and were found to be reliable at ranges between 100 cm and 500 cm at small angles from the acoustic axis, and at delay periods over two seconds. This work is similar to RICB although RICB doesn’t use an ultrasonic sensor; its sensors are enough to identify people.

In the ecology field, an intelligent garbage collector system has been designed [²], that uses Keras, phyton language and OpenCV to collect objects. The model without pre-training uses a Convolutional Neural Net (CNN) with a fully connected and dense layer. CNN performs well and is better in real time simulation, but it is complex and uses a lot of runtime.

The components of this system: CNN network, Arduino and Raspberry Pi 3 board and camera, servo motor, DC motor, ultrasonic sensor and battery.

The types of garbage that it detects: paper, glass and plastic, with 150 images each, and it collects them in a mounted container. The robot has an algorithm that separates garbage from other objects. It has a sonar sensor that detects the existence of an object near the robot. If there is, then the camera sends a signal to the Raspberry Pi to verify if the objects are garbage or not.

The operation is divided into 3 parts:

Systems for Internet of Things (IoT), frequently use small microcontrollers and microcomputers, such as Raspberry. IoT have attracted strong interests from disaster mitigation stakeholders to rapidly locate victims [⁴]. In [⁵], authors proposed a system based on a Raspberry Pi to collect data from IoT enabled Arduino based sensors for fire rescue. The landslide monitoring system shown in [⁶] was developed using a Raspberry Pi coupled to a camera and vision-based algorithms to detect landslides.

4 Description of the RICB System

The RICB is implemented on a Raspberry Pi Zero W, with movement sensor, temperature connected to it. The Raspberry PI is inside a small transparent sphere made of resistant material to get inside collapsed buildings. This entry is in accord with the strategy of rescuing people, mainly in places where rescuing people supposed there are victims. Fig. 1 shows the general diagram of RICB.

Fig. 1 General diagram of RICB that describes the protocols of the sensors 

First, the sensors that RICB has are described. A temperature sensor without contact is used; this model is: MLX90614 of Melexis enterprise. This sensor is used to measure temperatures of objects at long distance. This sensor is connected through SMBus that is a subset of I2C. Its reading is easy because it has a GPIO development board [⁷]. This sensor includes a voltage regulator with five volts. This temperature sensor has many applications in several areas such as buildings, factories and some health applications.

The low-cost infra-red movement sensor HC-SR501 is used. It uses two potentiometers and one jumper that modifies distance and time of activation. Also, one jumper for calibrating data in case of repetitive response. The FC-04 model sound sensor is used. It needs voltage between 3.3 to 5 VDC. Their output is analog.

To calibrate the sensibility of the sensor it uses a potentiometer.

4.1 Design of Human Radiation Sensor

The Human Radiation Sensor (HRS) is made nearly completely from the chemically inert polymer PolyVinyliDene Fluoride (PVDF). The polymer fibers to make PVDF membranes were electro-spun using a laboratory-made apparatus, composed of DC power supply 0–25 kV, a syringe (volume of 3 cm³, hypodermic needle (0.8×40 mm), and a collector screen (copper plate) that sustained the substrates (10 cm×15 cm). The precursor solutions were prepared using 10 PVDF pellets of 0.06 gr each, dissolved in 2.4 gr of solvent (N, N dimethylformamide-DMF) to obtain concentration of 20%.

The solution was heated (110 ⁰C) during 1.5 hours before deposition. The distance between tip and collector plate ranged from 5 to 15 cm. The samples were subjected to heat treatment for 24 hours at 100 ⁰C to change the crystallinity of the polymer. The thickness and pore size of PVDF membranes were controlled by the electrospinning time. The PVDF membrane prepared by electrospinning presents a phase β [¹⁰]. The membrane has a thickness of 90–100 µm and an elliptical shape whose major axis is 9 cm and minor axis is 7.5 cm as shown in Fig. 2.

Fig. 2 Fiber mats produced by electrospinning machine 

Micrograph was made in Centro de Investigación y Estudios Avanzados (Cinvestav) of Instituto Politécnico Nacional (IPN).

These fibers have diameters between 50 nm and 700 nm as shown in Fig. 3.

Fig. 3 Fiber mat sample 16.5 KV ejection 

For the application of the HRS, a sample of 0.5 cm² was extracted from the membrane described above, to be positioned between aluminum electrodes of sensor to have a sensing area of 5 mm² as shown in Fig. 4.

Fig. 4 PVDF polymer Fiber mat with aluminum electrodes on both sides, housing PLA 3D printing 

In order to measure the capacitance given by the HRS, a LM555 oscillator was applied. Using this device, a change in the capacitance affects in a direct way to an output frequency. Thus, this frequency can be more easily measured using a Raspberry Pi. Fig. 5 shows how an application of HRS based on a PVDF fiber mat was made, using the LM555 manufacturer’s datasheet. This tells that the oscillation frequency is given by Eq. (1). Thermal noise and electrical noise are the most important factors in the detection of living beings since this is a source of infra-red capture.

Fig. 5 Electrical diagram using LM555 circuit and PVDF polymer mat 

Where: Ra = Rb = 10KΩ and Capacitance = PVDF mat:

f=1T=1.44Ra+2Rb. (1)

5 Interoperability Between Sensors

Interoperability occurs through messages. It will start describing the MQTT structure message.

5.1 MQTT Message

The MQTT structure is composed of two parts: Topic and Data.

For this project the topic used is individual for each telemetry device. It depends on the name specified inside the SSL certificate. In addition to the name of device, the topic contains a subtopic. This subtopic delimits the information that come from sensors.

This subtopic is: /sensors. The topic is useful to identify each message sent from RICB. It comprises the device’s name that sent the message, including it subtopic, see Fig. 6.

Fig. 6 Structure of the message topic 

The information of each sensor is delivered in JSON format. This format is used by many standards for interchange of information using APIs. It is thus very popular. This format defines an object that contains two elements: key - value that each sensor sends. This data structure of the message received by each sensor is: {“sound”: 0, “longitude”: “-”, “temp_amb”: 2169000000000 0055, “obj_amb”: 12.270000000000039, “latitude”: “-”, “speed”: “-”, “movement”:0}.

Once RICB is connected it is turned on, and the service that will send the messages from the sensors to the broker will automatically start. The message sending speed is arbitrarily configured to be a message every second.

Fig. 7 shows the physic telemetry device running without the PVDF acrylic sphere.

Fig. 7 Telemetry device working 

The main screen of RICB shows data of the sensor send to the Raspberry Pi. Fig. 8. shows its interface with four quadrants.

Fig. 8 Results screen that shows four quadrants with data of sensors. Quadrant 4 is still not used 

5.4 Prediction Model

An ANN was used to predict the presence or absence of a person, using the readings of the three sensors as inputs. For the architecture chosen, several configurations were tested to select the best one. The parameters varied were the number of hidden layers, the number of neurons in each hidden layer, the type of activation functions (linear, sigmoid, ReLu, and hyperbolic tangent), the number of epochs, and the learning rate. The grid-search method was applied to find the parameters that would reduce the root mean squared classification error. Fig. 9 shows the best configuration found.

Fig. 9 Architecture of the neural network 

The synaptic weights for this architecture shown in Fig. 12 are the shown in Fig. 10. 𝑊₁ =

Fig. 10 Synaptic weights for the architecture 

Fig. 11 Response frequencies for the sound sensor 

Fig. 12 Response frequencies for the motion sensor 

The best values of parameters for the architecture summarized in Fig. 9 and 10, are shown in Table 2, for 10-fold cross validation.

Table 2 Best parameters of the neural network 

Parameter	Best value
Number of hidden layers	2
Activation function	Logistic
Epochs	7727
Learning rate	0.23

The implementation of the ANN and all tests were using the scikit-learn library [¹⁴].

For large data sets, data collected is split in a training subset (80% of data) used to fit the prediction model, and test subset (20% of data), used to measure the performance of the model. In the case presented here, the size of data collected is very small, therefore, data was split randomly into training set (75%) and test set (25%).

6 Results

In RICB movement, sound and temperature sensors are used because if a sensor does not have the scope to identify people, other sensors will identify it. For this reason, the ANN was included.

RICB was tested with 200 cases in a controlled environment in two classrooms at UAEMex University. One classroom was empty, other was with plenty of furniture. RICB was located under and upon a furniture. A person was in the classroom. Figures 11, 12 and 13 show a summary of the distribution of frequencies for each sensor, respectively.

Fig. 13 Response frequencies for the temperature sensor 

The performance of ANN was measured with testing set using 10-fold cross validation. The class to be predicted is the presence or absence of a person in the environment. The confusion matrix Table 3 shows the performance achieved.

The explanation in Table 3 is as follows: The test data set is 50 instances in size. Of these, 21 correspond to measurements in which there is no person in the classroom.

The other 29 instances correspond to measurements with a person in the classroom. The ANN correctly identifies 20 of the 21 cases in which a person is not present.

On the other hand, the ANN correctly predicts correctly the 29 cases in which there is no person in the classroom. In short, the ANN makes only one mistake in one case, indicating that there is a presence of a person when in fact there is not.

Based on this table, we compute the classification accuracy, Kappa statistics sensitivity and specificity.

The overall performance of the ANN is summarized below. The classification accuracy was 0.98, Kappa statistics 0.9587, sensitivity 0.9524, specificity 1.0000. According to the results obtained, the model is suitable for the detecting the presence of people in a room.

The RICB design is functional according to the tests carried out and a close space. The acrylic sphere measures 8 centimeters (See Fig. 14).

Fig. 14 shows a Raspberry Pi Zero inside an acrylic sphere of approximately 8 cm diameter 

The model HC-SR501 infra-red movement sensor detects human presence from 3 to 7 meters. The non-contact temperature sensor MLX90614 model has detection capacity in a diameter of 0.83 meters as shown in Fig. 15.

Fig. 15 shows the telemetry device (RICB) and its components 

The location of RICB has been manually set under furniture in a classroom where it can fit, to interfere between RICB and a person.

Several functionality tests have been carried out through various obstacles: chairs, tables, and other objects. Below are the results of the tests performed. All the tests in Table 4 were made in the same building.

7 Conclusion and Future Works

Due to the earthquakes that occurred in Mexico during the past 50 years, the desperation of the population to recover its living victims trapped in the rubble caused difficulties in locating victims.

According to [¹⁴] people alive who are trapped, die due to lack of oxygen and water. Without serious injuries they can survive up to three to seven days. RICB, has been developed to contributes to those cases.

It is a telemetry device with the interaction of sensors on a Raspberry Pi board. It has been installed inside an acrylic sphere with the measurements to be collected from the rubble of the buildings.

Furthermore, the interaction between the sensors is supported by a model that uses an ANN that helps to identify living people after the incident.

There are some works related, these are aimed at identifying human activities [¹], identification and garbage collection [²] that use ANN with sensors. Some of them, as in [²] propose a cheap model of implementation.

However, RICB in addition, proposes a cheap model, implements an HRS that is a non-commercial product and is it our own elaboration and without implementation in other works.

A set of 200 data has been developed and a prediction model has been applied with a classification accuracy: 0.98, thus the model is suitable for the analyzed data of RICB. Since post-earthquake recovery time is crucial, the use of RICB will speed up victim recovery time. It is important to mention that the system only identifies the presence of victims, giving rescuers the opportunity to recover them.