2.1 ADditive CLUStering (ADCLUS)

One of the first works is [¹] where a new model of clustering is described, in this model, the restrictions of the clustered objects in an exhaustive or mutually exclusive categories is relaxed allowing the establishment of overlapping clusters. It is notorious that many datasets to be grouped do not require exclusive clusters, it is from there the need of a solution with overlap, but create all possible overlapping sets gives a total of 2n−1 clusters, therefore, heuristics are needed to select potential groups.

ADCLUS is the proposed model, which cluster elements that meet some property, with a certain weight. ADCLUS considers n objects to be grouped and a symmetric proximity matrix M=n(n−1)/2. The data is transformed into similarities in the [0,1] interval. The underlying basic equation of ADCLUS is:

where S^ij is theoretically the reconstructed similarity in the objects between i y j, wk is a non-negative weight. So, the similarity between a pair of objects is the sum of the weight of those groups that contain both objects. In addition, the MAPCLUS algorithm builds a matrix starting of a set of weights and defined subsets, surpassing ADCLUS.

2.2 Overlapped K-Means (OKM)

The reason for performing an overlapping algorithm based on K-Means[⁴], is due to different applications in information retrieval, natural language processing, chemistry, biology, medicine; among others, where an overlapping data coverage is required, so an objective criterion is proposed associated with the OKM algorithm that generalizes the K-means algorithm.

The objective criterion is defined as follows: Given a set of data vectors X={xi}i=1n with xi∈ℝn, to find a k-way coverage {πc}c=1k, where πc represents the cth group; such that the following goal is minimized:

where each xi must be in at least one group, so: ∪c=1kπc=X y ϕ(xi) denotes the image of xi defined by the combinations of the (mc) prototypes for the group xi as:

where Ai the assignment set of: xi:{mc|xi∈πc}. So an heuristic for obtaining the optimal coverage was developed.

2.3 Dynamic Overlapping Algorithm based on Relevance (DClustR)

The DClustR algorithm [¹⁸] is an algorithm that allows the overlap between its groups, as an alternative for analysis in social networks, information retrieval and bioinformatics, this algorithm is based on graph theory by introducing strategies for the construction of more precise overlapping clusters or when the collection changes.

The main idea is to generate a set of clusters that are a coverage G˜β using ws-graphs and subsequently improve the initial clusters to obtain the right one, so “improve” means reducing both the number of clusters as the overlap between them.

To define the ws-graph, let Gβ=〈V∗,E∗,S〉 a graph of similarity threshold with weight. A star shape weighted subgraph (ws-graph) in G˜β, denoted by Gβ∗=〈V∗,E∗,S〉 is a subgraph of G˜β having a vertex c∈v∗ such that there is an arch between c and another vertex un V∗. The vertex c is called the center of the ws-graph and the rest of the vertexes are called satellites. Isolated vertexes are considered degenerate ws-graphs.

Being the ws-graph determined by its center, the problem is to build the set W={Gc1∗,Gc2∗,…,Gck∗} of ws-graphs, such that W if a coverage of G˜β, can be seen as the problem of reconstructing the set X={c1,c2,…,ck} such that ci∈X is the center of G˜ci∈W,∀i=1,…,k.

To avoid analyzing all vertexes in v∗ and delimit the search space, a selection criterion is established, DClustR introduces the concept of relevance of a vertex, for which it can be selected those vertexes with the highest degree, and its about maximizing the number of added vertexes to the G˜β coverage in each iteration.

2.4 Overlapping Clustering based on Density and Compactness (OCDC)

The OCDC algorithm [¹⁹], introduces a new graph coverage and a new filter strategy, with which a small set of overlapping clusters are can be obtained. The collection of objects are represented as a graph of similarity threshold with weight G˜β, the overlapped clustering is realized in two phases: the initialization and the improvement.

In the initialization phase, an initial set of groups of coverage vertexes is built G˜β, using the ws-graphs, in this context, each of these graphs make up an initial group. In this step, the algorithm seeks to reduce the search space, and OCDC introduces the concepts of density and compactness of a vertex v. The density of a vertex v∈V, is calculated by using the following equation:

where v.pre_dens is the number of adjacent vertexes to v, having a degree not greater than the degree of v, and v.Adj is the total of adjacent vertexes to v.

The compactness of a vertex v∈V, is estimated as follow:

where v.pre_compt is the number of vertexes u∈v.Adj such that Aprox_Intra_sim(Gv∗)≥Aprox_Intra_sim(Gu∗), where Gv∗ and Gu∗ are ws-graphs determined by v y u, respectively. The greater value of the compactness is included in coverage and therefore it is the best coverage graph.

2.5 MCLC Algorithm

The MCLC algorithm is proposed to discover overlapping communities [⁶], for which is used a random path in a line graph and attraction intensity. Unlike the traditional random path that starts from a node, it starts from a link. In the first instance, a network graph is transformed into a weighted linear graph, and the random path in this linear graph is associated with a string of Markov. In order to obtain the probability of the Markov’s chains, a similarity between the pair of leagues is obtained. Then, the leagues can be grouped into “league communities” where those nodes can be overlapped.

The league communities become “node communities”, and a attraction intensity is defined for the control of the overlap’s size. Finally the communities that allow overlapping are detected.

The distance or similarity between pairs of leagues is obtained by calculating the probability transition of random paths in the linear graph. A matrix of M×M, can be associated to a chain of M-Markov’s states, the transition matrix P=[Pαβ], is defined as:

The number of repetitions of random paths that start from the league α, [Pt]αβ should be considered and is the probability that the path starts from α and remain in β which is ∑t=1T[Pt]αβ(1≥t≥T)

The cluster analysis can use “peer league pairs” in communities of candidate leagues, so a similarity (symetric)ϕαβ is proposed as follow:

The distance dαβ between pair of leagues (α,β) is obtained by the complement of the similarity and normalization of the results between 0 and 1:

2.6 Clustering Method with Incremental Overlapping based on Trees

The clustering method with incremental overlap based on trees [²⁰], uses the tripartite decision theory. A tree is represented by points that can improve the relevance of the search result. The overlapped clusters are represented by the tripartite decision with a set of intervals. Tripartite decision strategies are designed for the update of clusters, at the moment that the data increases. Further, with this method is possible to determine the number of clusters during the process.

To define the tripartite decision clustering, be U={x1,…,xn,…,xN} the universe, and the resulting clusters C={C1,…,Ck,…,CK} a family of clusters of the universe. xn is an object, which has D attributes, xn=(xn1,…,xnd,…,xnD), where xnd represents the value of the d-th attribute of the object xn, where n∈{1,…,N} y d∈{1,…,D}.

The algorithm starts calculating the distance(Euclidean) between objects. The similarity between the objects is obtained with the complement of distance. Subsequently, the representative points are calculated using the following condition: if |Neighbor(r)|≥ζ, r is a representative point and represents the object in the area where r is centered and with radius δ.

The next step is the construction of an indirect G graph based on the R representation points, this is achieved using the tripartite decision and the calculated representative points. Finally, the algorithm search in the subgraph those who are strongly connected in the graph G. This design allows the growth of the data, but when increasing, it is required to simulate different situations to evaluate the performance of the method.

2.7 INDCLUS

In this section is examined the scalability of the ADCLUS and INDCLUS models [⁸], which are techniques that can be used to extract overlapping clusters with similar data. In this paper was taken the models ADCLUS and INDCLUS appropriately and were designed different metaheuristics extensions to have more relaxed models.

For the INDCLUS model, N elements are considered, with a similarity matrix S=(sij){N×N}, and is required a group of a known number of M overlapped clusters possibilities. The INDCLUS model requires to minimize the optimization functions:

where K is the number of subjects, N is the number of elements to be stored and skij is the

similarity of the elements i and j del sujeto k. If K=1 the model is reduced to ADCLUS.

The used heuristics with these algorithms are: alternating approach of minimum squares (SINDCLUS), a symmetric approach applied to SINDCLUS (SYMPRES), simulated annealing (SA-SINDCLUS), tabu search (TABU-SINDCLUS), and relaxed solution space (SMC-Relax). The tests were realized with medium size real datasets, SMC-Relax had the a better execution than SINDCLUS and SYMPRES. The use of heuristics makes the ADCLUS and INDCLUS models scalable.

2.8 Hybrid K-Means

In [¹⁵] is described an algorithm (HKM-OKM) that combines harmonic k-means with overlapped k-means. By making use of the overlapped k-means algorithm; which is an extension of k-means is sensitive to the centroid of the initial cluster but when is combined with harmonic k-means this limitation can be overcome.

The main idea in this method is to use the output HKM method to initialize the centroids of the OKM method. The OKM method was explained in section 2.2. The HKM algorithm introduces a bias(using the weight) to move the cluster centers to the data points that are most important according some criteria.

Similar to the k-means algorithm, the HKM method can be formulated as an optimization problem where the objective is to minimize:

where p is a free parameter (typically p≥2), and the expression (k∑j=1k1‖xi→−zj→‖p) is the harmonic media. To calculate the harmonic media, the algorithms needs to calculate the cluster centroid zj→ by using:

where m(zj→|xi→) is a member of the data point xi→ to the cluster centroid j calculated by:

And w(xi→) is the associated weight which each xi→ point, calculated by:

The HKM-OKM algorithm starts by finding centers, using the HKM, initializes OKM using the found centers. A set of medical data is used, because it is required to model elements with overlap. It improves the obtained results by OKM.

3 Algorithms Analysis

The analyzed algorithms have a maximum time complexity of the quadratic order as they are: OCDC, MDLC based on trees, INDCLUS with hybrid heuristics and k-means; the particular case of OKM maintains the order of the algorithm on which it was based (k-means), and only the ADCLUS algorithm is of the cubic order. This information can be reviewed in table 1.

In the experiments the number of instances that were used with these algorithms varies, being the minimum of 105 and a maximum of 102, 294 instances. Further, the objects are of different types: discrete, qualitative or documents. In all experiments were tested stable datasets, for example, the repository UCI Machine Learning^fn is used, testing the cancer dataset, heart disease, parkinson, among others, also the dataset on the karate Zachary^fn, KDD, ISOLET were used; Reuters-21578^fn, TDT2^fn, were mainly used for the dataset with documents.

In general, clustering algorithms with overlap are based on others algorithms that do not support overlap and even improve some aspects of the same.