nova página do texto(beta)
Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Enviar este artigo por email
Citado por SciELO 
Similares em
SciELO 
Permalink
1. Introduction
Superpixels are homogeneous regions of clustered pixels, taking into account low level features such as intensity, color, texture, spatial relationship, among others. They provide a simple, more compact and efficient representation of an image (Stutz et al., 2018). Some desired properties in the algorithms to extract superpixels are edge adhesion, precision and minimum runtime. To address this task, different approaches have been addressed in the state-of-art, e.g., those based on watersheds (Hu et al., 2015; Machairas et al., 2014; Neubert & Protzel, 2014), graph theory (Felzenszwalb & Huttenlocher, 2004; Humayun et al., 2015; Shi & Malik, 2000; Zhang et al., 2011), contour evolution (Buyssens et al., 2014), clustering (Achanta et al., 2010 Levinshtein et al., 2009; Li & Chen, 2015) and energy optimization (Yao et al., 2015). The simple linear iterative clustering (SLIC) algorithm is one of the most efficient for extracting superpixels (Achanta et al., 2010); however, like other methods, it has an insufficient spatial relationship, which can reduce the adhesion to the edges of the image as well as have an influence on the compactness and regularity of the superpixels.
Most of the aforementioned algorithms have outstanding performance; especially, when it comes to grayscale images. However, they require local or global features (texture, statistics, among others) to improve their performance. These algorithms are also sensitive to atypical information during the initialization stages and to the noise in the clustering stage; in addition, some of them require a parametric adjustment, processing steps or training phases. In contrast to these, in this paper, a proposal is given for extracting superpixels, which does not require a training stage; it works with only the intensity of the pixels or the color of the images as the only feature; and it does not require preprocessing to increase its performance. Beyond this, the only required parameter is the number of regions in which the image is oversegmented, although it makes a dynamic adjustment to obtain a correct number of superpixels depending on the size of the image.
The proposal consists of an adaptation of the intuitionistic fuzzy clustering algorithm engineered to extract superpixels, considering the properties of superpixels, such as the homogeneity of the segments as well as the adaptation to the natural limits of the image. The proposed approach considers pixels as intuitionist fuzzy sets; with this, each pixel belongs to all initial superpixels with different degrees of belonging. The homogeneity of the superpixels is refined in the clustering stage, in which the membership of each pixel for a certain superpixel is calculated iteratively. The labeling stage is done based on the membership matrix, which is obtained at the end of the clustering stage.
This paper is organized as follows: in Section 2, the relationship of intuitionist fuzzy numbers with image processing is described; in Section 3, the extraction of superpixels with the proposed algorithm is described in detail, for both grayscale and color images; in Section 4, the experimental configuration is indicated, in terms of the datasets used, metrics to quantify performance, and comparative methods; in Section 5, an analysis of the results obtained is shown; finally, conclusions and future work can be found in Section 6.
2. Intuitionistic fuzzy numbers applied on images
Let us X be a finite subset of the real p-dimensional vector space
where μ(xij) is the membership degree, min(xj) and max(xj) are functions that allow to compute the minimum and maximum values of each color component, respectively. By using the Chaira intuitionistic fuzzy generator (Chaira, 2015), non-membership degree ν(xij) can be computed based on the following expression:
where λ is a constant parameter. The hesitancy degree π(xij) can be determined from (1) and (2) as follows:
A formal definition of
subject to the constraints a) μ(xij)+ν(xij)+π(xij)=1 and b) 0 ≤ π (xij) ≤ 1, for each xij ∈ X. In the context of image segmentation, there is the need to compute the similarity between pixels; in this regard, the knowledge of a metric distance is required. Let us two pixels be expressed as intuitionistic fuzzy data
3. Superpixels extraction by an intuitionistic fuzzy clustering
3.1. Gray scale
To extract superpixels in grayscale images, an adaptation of the intuitionistic fuzzy C-means (IFCM) algorithm (Xu, 2013) is proposed. If each initial superpixel is considered to have representative data, then the objective function can be denoted as:
where XIFS are the pixels stated as intuitionist fuzzy sets,
In this implementation, the search space is limited to a region of size 2R ⨯ 2R, in order to increase the speed of the algorithm and preserve the homogeneity of the segments. The conventional algorithm calculates the membership of a given data xj, taking into account the centroids
Expression to update the membership matrix is stated as follows:
where uij is the membership, Kp is a vector with the three closest centroids Kp ⊂ K and
where
Subsequently, the oversegmentation is reinforced using a postprocessing step that links small superpixels with their neighbors. The union of the superpixels can be controlled through a parameter α; it is necessary to compute the average intensity sk of the superpixels and the euclidean distance d(sk,sk+1) < α is satisfied.
Algorithm 1 summarizes the steps for the implementation of the proposed algorithm. The input is a grayscale image with size W⨯H, where W and H are the width and height of the image. The number of superpixels K is also requested as input. In step 1, the image is transformed into a column vector; then, in step 2, the vector is taken to the intuitionistic fuzzy space by means of (1). VIFS is initialized in step 3 (Algorithm 2), using the average value of the region, this process allows the region to be better represented since local information is taken into account. Before entering the iterative process,
Initialization
The IFCM algorithm can perform better with a good initialization of the centroids. The conventional algorithm uses a random initialization. In this proposal (Algorithm 2), the arithmetic mean is used as the initialization form of the IFS in each initial region. The inputs are images represented as intuitionistic fuzzy sets XIFS and number of superpixels K. The algorithm computes the arithmetic mean of each fuzzy index
3.2. Color
The proposed method can be extended to process color images, for this the three channels of the image must be included I: W ⨯ H → R3. Each channel can be treated as an independent feature, so it is necessary to transform the input to the intuitionistic fuzzy domain by separating the channels. In this paper, the IJK color space is used since it is easy to convert; in addition to that, the IJK color space separates the brightness from channel I to the texture information in channels J and K. The conversion of the RGB color space to IJK is done with the following equations (Pătraşcu, 2007):
The proposed method for the color images is summarized in the Algorithm 3. The algorithm does not have a significant change with respect to the version for grayscale images, since to process the three channels, they can be interpreted as independent vectors. In step 4, the expression is modified by adding
4. Experimental setup
4.1. Datasets
The performance of the proposed algorithm is evaluated against magnetic resonance imaging (MRI), for which a study was simulated in Brain Web (Web, 2004); it was considered in T1 modality and corrupted with a density of 5% Gaussian noise. The study consists of 182 images with a size of 181⨯217 pixels. Additionally, the BSDS500 database was contemplated (Arbelaez et al., 2011), this is divided into a training subset of 200 images, 100 for validation and 200 testing images, all of them with sizes of 481⨯321 or 321⨯481 pixels.
4.2. Evaluation
In order to evaluate objectively the quality of the superpixels, the metrics boundary recall (BR) were considered (Martin et al., 2004), Error of subsegmentation (UE) (Achanta et al., 2010) and the variation explained (VE) (Moore et al., 2008). Runtime measured in seconds was also included, in order to know how fast the evaluated algorithms are in the superpixel extraction task.
Boundary recall is used to evaluate the adherence to the boundaries with respect to ground truth (Martin et al., 2004). This index can be computed by:
where FN(G,S) and TP(G,S) are the number of False-Positives and True-Positives, respectively; with respect to the boundaries given by an over segmentation obtained S and the ground truth G.
Error of subsegmentation describes the deviation of a superpixel with respect to a specific ground truth segment (Achanta et al., 2010), that is, it measures how each super pixel is superimposed with a ground truth segment.
where gi is the segmentation ground truth and sj the obtained superpixel. M represents the number of ground truth segments, N is the total number of pixels in the image and B is a minimum number of pixels that overlap gi.
Variation explained measures the quality of a superpixel segmentation without using the ground truth segmentation (Moore et al., 2008), it is defined as:
where xi is the value of the real pixel, μ is global mean is the global average of pixels and μi is the average value of the pixels assigned to the super pixel that contains xi.
4.3. Comparative algorithms
The performance of the proposed algorithm SP-IFCA (superpixel extraction by an intuitionistic fuzzy clustering algorithm) is compared with W (watershed) (Hu et al., 2015), SLIC (simple linear iterative clustering) (Achanta et al., 2010), LSC (linear spectral clustering) (Li & Chen, 2015), TP (turbopixel) (Levinshtein et al., 2009) and PB (pseudo-boolean superpixel) (Zhang et al., 2011) algorithms. These algorithms were considered because they can work with grayscale and color images, unlike others presented in the state of the art, which can only work with a specific type of images.
5. Experimental results
In these experiments the performance of the algorithms was evaluated before different number of superpixels; hence, it was assumed that the size of grid R depends on the total number of pixels in the image and the number of superpixels K required. To prevent the size of the grid range from being too small, the value of superpixels was bounded to 200 ≤ K ≤ 1450 for the RMI gray scale and 200 ≤ K ≤ 6000 for the color images. Therefore, the minimum interval size that is reached is
5.1. Grayscale images: MRI of the brain
In this experiment, the algorithms developed the task of extracting superpixels from all images of the simulated MRI study. In Figure 2, the average performance against different number of superpixels K Is depicted.
Specifically, in terms of the level of adhesion to the edges (Figure 2a), a superior performance of the proposed method (SP-IFCA) can be observed in all number of superpixels considered. W and SLIC show similar performance even though they use different approaches to generate superpixels, both outperform the other comparative algorithms from K ≥ 1000. LSC produces good results up to K ≤ 1000 but its yield tends to decrease from this value. TP reaches its best performance in K = 800, while PB has the lowest performance but with an upward improvement. In terms of the UE metric (Figure 2b), it can be seen that the proposed method obtained the minimum subsegmentation error in the range of 350 ≤ K ≤ 1450, LSC has the best performance in 200 ≤ K ≤ 300; however, from K = 350 its error has a growing behavior, SLIC shows a linear trend just like SP-IFCA, TP tends to decrease in 200 ≤ K ≤ 600 and to increase in 1000 ≤ K ≤ 1450, W and TP present greater error, although their behavior decreases.
Figure 2c shows the level of adherence to the edges of the image, the highest value for all K values was obtained by SP-IFCA, followed by SLIC, LSC showed a decreasing behavior and the worst result in 850 ≤ K ≤ 1450, PB presented the lowest values in 200 ≤ K ≤ 800. Figure 2d reveals a similar trend in the processing time of most algorithms, which is interpreted as a proportional increase between the execution time and the number of superpixels. It highlights the proposed method SP-IFCA with the shortest processing time (0.005 ≤ Runtime ≤ 0.073 seconds), followed by the SLIC algorithm, and so on. Instead, the TP algorithm required the longest processing time (0.020 ≤ Runtime ≤ 0.251 seconds).
In Figure 3, different representations of K = 800 superpixels are shown. In the form of contours (first row), as the average intensity of the segments (second row), was well comparison of the extracted segments with respect to ground-truth for Rec and UE metrics, in the third and fourth row, respectively. Given the amount of superpixels, the representation of the average intensity is not very supportive since it shows similar results. On the other hand, the representation of contours exposes the W, LSC and SP-IFCA algorithms as those that generate irregular superpixels with better adaptation to the edges, and therefore, a better preservation of details. The other two representations make it possible to highlight the superpixels that fail to adapt correctly (regions in red). With this representation it can be seen that SP-IFCA has the best performance with respect to the ground truth segments for both Rec and UE metrics.
5.2. Color images: BSDS500
For the second experiment, the test subset of the database BSDS500 was used; the number of superpixels considered was in the range 200 ≤ K ≤ 6000. The results obtained with respect to the Rec, UE and EV metrics are plotted in Figure 4. Adherence to limits is illustrated in Figure 4a, it is possible to see the best performance of SP-IFCA algorithm for all K values, followed by LSC, the remaining algorithms present an ascending behavior as K is greater. The subsegmentation error can be seen in Figure 4b, the best performance of SP-IFCA is ratified with a range of values 0.07 ≤ UE ≤ 0.157, which decreases as the number of K increases, the other algorithms have the same decreasing trend, although with a greater error. The level of adhesion to the edges is shown in Figure 4c, the results of EV suggest that the best performance is that of SP-IFCA, followed by LSC and TP, SLIC stands out from K ≥ 4300; the other algorithms have lower performance.
Figure 4 Average evaluation of the metrics, for the test subset of the BSDS500 database, and different K values.
On the other hand, the graphical trend of the processing time illustrated in Figure 4d confirms the increase in the processing time of all the algorithms, when it comes to color images; which is obvious since in this case three information channels are being processed. It can be seen that for K ≥ 250 superpixels, the proposed method required the least processing time, i.e., 0.075 ≤ Runtime ≤ 0.166 seconds. The rest of the algorithms showed good performance in terms of processing time.
To exemplify the extraction of superpixels in their different representations, image 230063 was oversegmented in K = 800 superpixels (Figure 5). A smaller amount of regions in red color can be observed by the proposed SP-IFCA algorithm, which implies a better adaptation of the extracted superpixels compared to those suggested in the ground truth.
6. Conclusions
In this paper, an algorithm to extract superpixels in grayscale and color images was introduced, it is based on an adaptation of the IFCM algorithm. The characteristics of this clustering algorithm allowed it to handle uncertainty more efficiently, resulting in a better allocation of pixels to the superpixels. The performance of the proposed algorithm against magnetic resonance images degraded with noise, allowed to obtain average values of Rec = 0.957, UE = 0.059 and EV = 0.988, per study, as well as all the values of K.
In contrast, for the test color images, the average values obtained were Rec = 0.849, UE = 0.108 and EV = 0.907, for all K values. A better performance of the proposed SP-IFCA algorithm compared to comparative algorithms was demonstrated through quantitative and subjective evaluations, as well as a shorter processing time.
Future work will focus on incorporating spatial information as another feature for local clustering, as well as texture information to generate more homogeneous superpixels.
