2.1. Gray level co-occurrence matrix (GLCM)

In the early 1970s ^{Haralick, Shanmugam, and Dinstein (1973)} proposed the extraction of fourteen features, from the GLCM of a gray level, to characterize the image texture. The computation of GLCM depends on two parameters: the orientation θ formed by the line-segment connecting the two considered pixels, and the distance (d) [ number of pixels ] between them. The direction θ is usually quantized in 4 directions (horizntal - 0^o, diagonal - 45^o, vertical - 90^o, anti-diagonal - 135^o).

To compute the gray-level co-occurrence matrix of a windowin an image, the following parameters are considered:

We adopt the formulation used in ^{Bastos, Liatsis, and Conci, 2008} and ^{Eleyan and Demirel (2011)}) to present the calculation of GLCM. The GLCM is defined as the probability of occurrence of two gray levels at a given offset (with respect to given distance and orientation). Given the image I, of size N _x × N _y , the value of the co-occurrence for the gray values i and j, at the distance (d) and direction θ, P _d,θ (i, j) can be defined as

Where

The offset (π _x (d, θ), π _y (d, θ)) is used to compute the positionof (x, y) with respect to its neighbor at the distance (d) and direction θ. For the 4 directions (0^o, 45^o, 90^o, and 135^o) and the offsets are given in Table 1.

2.1.1. Haralick’s features

Given an image I with G gray levels, an angle θ anda distance (d), after the gray level co-occurrence matrix, (Pd,θ(i, j)) _{0≤i,j≤G−1}, number of features can be extracted, amongst which the most popular are the 14 Haralick features (energy or angular second moment (ENR), contrast (CON), correlation (COV), variance (VAR), inverse different momento (IDM), sum average (SAV), sum variance (SVA), sum entropy (SEN), entropy (ENT), difference variance (DIV), difference entropy (DEN), information measures of correlation (IMC1,IMC2), maximum correlation coefficient (MCC)). The computation of Haralick features is done using a normalized GLCM.The (i, j) th normalized entry

P _d,θ (i, j), of P _d,θ (i, j) is defined as

where = || P d,θ ||= ∑i=0G-1∑j=0G-1P(d,θ)(i, j). Details on the calculation of all these features can be found in ^{Haralick et al. (1973}).For each texture, T, a chosen distance (d) and a direction θ, 14 Haralick features can be extracted.

2.2. Local directional pattern (LDP)

The local binary pattern (LBP) operator depends on the change of the intensity around the pixel to encode the micro-level information of spot, edges and other local features in the image (^{Jabid et al., 2010}a). The gradient is known to be more stable than the gray level; that is why some researches have replaced the intensity value at a pixel position with its gradient magnitude and calculated the LBP (^{Ferrer et al., 2010}). The local directional pattern (LDP) was proposed by ^{Jabid et al. (2010b)} to resolve the problem with LBP, mentioned earlier. Since the LBP depends on the neighboring pixels’ intensity which makesit unstable. Instead, LDP considers the edge response value indifferent direction. LDP features are based on eight bit binary codes assigned to each pixel of an input image. It is composed of three steps (^{Kabir, Jabid, & Chae, 2010}):

• Calculation of eight directional responses of particular pixels using the Kirsch compass edge detector in eight orientations (M ₇, . . ., M ₀) centered on its own position as shown in Figure 1. Given a pixel (x, y) of an image, I, for each direction i, and using the corresponding mask M _i the ith directional response m _i can be computed as

Fig. 1 The 8 Krisch masks. 

For the 8 directions a vector (m ₇, . . ., m ₀) is obtained. Figure 2 shows the Kirsch directional responses of a pixel (x, y).

Fig. 2 Krisch directional response. (a) This figure shows a pixel (x,y) that has a gray level 60. (b) Directional responses, together with the ranking of those responses, and the associated bit significance, with m ₀ being at the less significant position and m ₇ at most significant position. Note that the ranking of responses is done on absolute values. 

• LDP code generation of the directional responses obtained in the previous step. It is based on the selection of k most significant responses and set the corresponding bit to 1 leaving other (8 − k) bits to 0. Finally, the LDP code, LDP _x,y (m ₀, . . .,m ₇), of the pixel (x, y) with directional response (m ₀, . . ., m ₇),is derived using Eq. (5).

where, m _k is the kth most significant response and s(x) isdefined as

Given the directional responses generated by the Kirsch convolution on pixel (x, y) presented in Figure 2, the LDP code for k = 3 is computed as follows:

o m ₅= 270 is the 3rd most significant directional response.

o The code of the LDP code of the pixel (x, y) is then

o The LDP code, LDP _x,y , of the pixel (x, y) is then 49.

• Construction of LDP descriptor which is carried out after the calculation of the LDP code for each pixel (x, y). The input image I of size M × N is then represented by a LDP histogram using Eq. (7), that is also called LDP descriptor. In this case k = 3, is used; It means, ⁸C₃= 56 distinct values are generated and used to encode the image. The histogram H obtained from the transformation has 56 bins and can bedefined as

where C _i is the ith LDP pattern value, i = 1, . . ., ⁸C₃ and the definition of p is given in Eq. (9).

Given a texture, T, and the number of significant bits k, afeature vector LDP _k,T can be extracted and represented as

2.3. Classifiers

Four classifiers are used to evaluate the power of LDP to characterize textures and compared Haralick features extracted from GLCM.

2.3.1. Support vector machines

SVM is a learning technique for pattern classification and regression (^{Cortes & Vapnik, 1995}; ^{Vapnik, 2013}). It was originally designed as two-class classifier, but many versions have been proposed to perform multi-class classification (^{Crammer& Singer, 2002}; ^{Hsu & Lin, 2002}). The principle is, given a labeled set of M training samples (x _i , y _i ), where x _i ∈ R and y _i is the associated label (y _i ∈ {−1, 1}), i = 1, . . ., M. A SVM classifier finds the optimal hyperplane that correctly separates the largest fraction of data points while maximizing the distance of either class from the hyperplane. The discriminant hyperplaneis defined by the level set function

where k(·, ·) a kernel function and the sign of is f(x) indicates the membership of x. Constructing an optimal hyperplane is equivalent to finding all nonzero α _i . Kernel function K _{(xi, xj)} is the inner product of the features space K _{(xi, xj)} ≤ ∅ _(xi) , ∅ _(xj) .The three following kernel functions are often used:

• Polynomial kernel

where c is a positive constant and d is the dimension of feature space in question.

• Linear kernel

• Radial basic function kernel (RBF)

where ||x ₁ − x ₂|| is the distance between the vectors x ₁ and x ₂, σ ∈ R is the bandwidth of a gaussian curve.

In our case, the M feature vectors(x _i = 1, . . ., M) are extracted, from texture images that need to be classified, using the local directional pattern, or GLCM.

2.3.2. Naïve Bayes classifier

One of the most popular and most simplest classification models is the naive Bayes classifier (^{Friedman, Geiger, &Goldszmidt, 1997}). The principle of Naive bayes is: given the training data T which contain a set of samples, each simple X = (x ₁, . . ., x _n ) and there are k classes C ₁, . . ., C _k . Each sample is labeled by one of these classes. Naive Bayes predicts a given sample X belongs to the class that has the highest posterior probability conditioned on X. Therefore, sample X is predicted to belong to class C _i if and only if P (C _i |X) > P(C _j |X), for all j such that 0 < j < (m − 1) and j ≠ i. By Bayes’ theorem

If the data set has many attributes, it would be expensive to compute P(X|C _i ). To solve this problem, naive assumption assumes that the value of the attributes are conditionally independent of one another. This means that

2.3.3. K-Nearest neighbor (k-NN)

k-NN is a non-parametric classification method introduced in the early 1970s by ^{Fix and Hodges (1951)}. The process isdone by computing the similarity between the sample and the different classes. Let C ₁, . . ., C _k be the classes of our samples.Given a new sample X = (x ₁, . . ., x _n ), to find to which class it belongs, the distance d(x, C _j ) between x and C _j , for j = 1, . . .,k, is calculated. Sample X is assigned to class C_i0 to which it is closest. Index i ₀ is calculated as presented in the following equation:

2.3.4 Multilayer perceptron (MLP)

MLP is well known and widely used in different detection and estimation applications (^{Burrascano, Fiori, & Mongiardo,1999}; ^{Gati, Wong, Alquie, & Fouad Hanna, 2000}; ^Kasabov,1996). The principle of MLP is that the input layer in MLP is considered as layer 0. Assume that the total number ofthe hidden layers is L. In the hidden layer l the number ofnode is Nl, l = 1, . . ., L. Let w_ij be the weight of the connection between the jth nodes of (l − 1) th hidden layer and i _th nodes of the l ^th hidden layer, and let x _i be the ith input factor to the MLP. Let y ^l _i represent the output of the ith node of the l ^th hidden layer, which can be calculated by the following equation:

where θ^l _i represent the bias factor of the ith node of the lth hidden layer, y⁰ _i= x_i, i = 1, . . ., N₀ and f(.) is the active function. Let υ _ki be the weight of the connection between the kth node of theoutput layer and the ith node of the Lth hidden layer. The MLPoutput can calculated as

where the β _k is the bias factor of the output layer. The MLP algorithm compares between the network output with the desire output which measures the error in the network. To correct the output layer, this algorithm updates the weight until the outputof network gets closer to the desire output.

3.1. Data set

Settlements image categories, shown in Figures 3-8, were identified in Soweto (Gauteng province, South Africa) to work

Fig. 3 Samples FT1 and FT2 are extracts of the formal township, where buildings are placed in a planned manner. This type contains stable structure. The different between T_1 and T_2 is on the houses. 

Fig. 4 Informal squatters (IS type1): the structure of the informal squatters is not stable. The dwellings of this category are shack type (made out of cardboard, wood, tin, etc.). Typically characterized by high building densities. 

Fig. 5 Formal township + informal squatter (FTIS Type1, FTIS Type2, and FTIS Type3): In this category we can find any type of any density, of residential unit, but buildings appear in pairs a larger building will be accompanied by a backyard shack. 

Fig. 6 Informal Township: the structure of the informal township is recognized as constant or semi-constant structure. The dwelling of this category is shack type and located on serviced and un-serviced sites. The dwelling densities vary from low to high. 

Fig. 7 Formal: the structure of the formal residential is a constant structure, located near well-established buildings areas. 

Fig. 8 Non-urban: shows samples of this type. All of them were gray level images with 8 bit per pixel. 

with as target classes. Data used are gray-level images with 8 bit per pixel, and a size of (200 × 200) pixels. Table 2 shows the number of samples in each category.

3.1.1. Choice of parameters

The computation of GLCM is done using the following parameters:

The computation of the local directional pattern is based on the number of significant bits, and in this work four alternative values are considered: 2, 3, 4, 5.

3.2. Result and discussion

Performance of the various classifiers using various sizeof training samples, with various values for k are considered.k-NN and MLP achieve reasonably good results, even when only a small proportion of the data is used for training. The superiority of the k-NN and MLP over both SVM and NB is easily perceptible. Moreover, it can be noticed that when the number of significant bits (k) changes from 2 to 4 the accuracy improves and declines when k = 5. Classifiers have the best performance for k equal 4 and the worst performance is registered for k equal to 2. The performance of LDP is evaluated for different values of k: The average accuracy of LDP with different k values shows that for LDP when k equal 4 is the highest performance (85.6%) compared to the rest. For LDP, the value 2 for k is the lowest performance (77.7%).The best achievement was obtained by k-NN and MLP 98.9%,98.19% in LDP (4) and LDP (3) when the data proportion is80% as training 20% testing. NB performances are remarkably low ranging between 55% and 70%. Another observation is the fact that the accuracy of SVM is not very high either. With the gray-level co-occurrence matrix, different combinations of the 14 original Haralick features (entropy, (entropy + IDM), (energy + contrast + correlation), (entropy + energy + contrast), (energy + correlation + entropy + IDM), (entropy + energy + contrast + correlation + IDM), and all 14 features are used to characterize textures. The presentation of different feature combinations to characterize informal settlements reveals that the best performance is achieved by k-NN with value (86.04%) using the 14 original Haralick features in 45 ^o direction. With NB, the performance is remarkably low ranging between 29% and 53%. Another observation is that both SVM and MLP are not very effective classifying informal settlements, with accuracies ranging from 30% to77%. Usingk-NN, the best achievement was achieved with 45 ^o direction at 91%. With NB, the performances are remarkably low ranging between 50% and 55% The accuracy of both SVM and MLP are not very high range between (48%-60%) and (64%-80%).The best performance was achieved in 45 ^o direction with value (80%) using MLP and (60.81%) using SVM. The training set percentage: the average accuracy of the best direction changes from 62% when the training set is 10%-71.17% when the training set is 80%. With the knowledge that the best performance is achieved with GLCM and this when the 14 original Haralick features at 45^o direction are chosen, it can be compared to the performance of LDP when k equals 4. The average performances of each classifier in both feature method sare such that, with NB and SVM, the performance is remarkably low in both LDP (4) and GLCM (45 ^o), the best achievement is (83.06%) by SVM using LDP (4). With MLP, LDP (4) achieved 20% more with value (92.77%) compared to GLCM (45 ^o) with value (77.48%). With k-NN, The best result was achieved by LDP (4) with value (92.7%), compared to GLCM (45 ^o) with value (86.03%). For GLCM (45), we find the performance from 20% to 80% increase by approximately 1% in each step. The best achievement obtained when the training set is 80% witha value (71.17%). The worst achievement obtained when the training set is 10% with a value (62.6%). For LDP (4), we find the performance almost constant from 50% to 80% with value (86.81%). The best achievement obtained when the training setis 80% with a value (86.81%). Figures 9-10 below summarize the comparison between these two methods (LDP, GLCM). In conclusion, the best combination used in our approach is using the local directional pattern with k equals four applying on nine categories of image, and the k-nearest neighbor being the preferred classifier.

Fig. 9 Classification accuracy vs. the number of training samples using both features method LDP when k = 4 and GLCM with distance 1 and 45 ^o direction.