Negations of Probability Distributions: A Survey

Batyrshin, Ildar Z.; Kubysheva, Nailya I.; Bayrasheva, Venera R.; Kosheleva, Olga; Kreinovich, Vladik; Batyrshin, Ildar Z.; Kubysheva, Nailya I.; Bayrasheva, Venera R.; Kosheleva, Olga; Kreinovich, Vladik

doi:10.13053/cys-25-4-4094

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

Otros
Otros

Permalink

Computación y Sistemas

versión On-line ISSN 2007-9737versión impresa ISSN 1405-5546

Comp. y Sist. vol.25 no.4 Ciudad de México oct./dic. 2021 Epub 28-Feb-2022

https://doi.org/10.13053/cys-25-4-4094

Articles

Negations of Probability Distributions: A Survey

Ildar Z. Batyrshin¹^*

Nailya I. Kubysheva²

Venera R. Bayrasheva²

Olga Kosheleva³

Vladik Kreinovich³

¹ Instituto Politécnico Nacional, Centro de Investigación en Computación, Mexico City, Mexico, batyr1@gmail.com

² Kazan Federal University, Kazan, Russia, aibolit70@mail.ru

³ University of Texas at El Paso, El Paso, USA, olgak@utep.edu, vladik@utep.edu

Abstract:

In recent years many papers have been devoted to the analysis and applications of negations of finite probability distributions (PD), first considered by Ronald Yager. This paper gives a brief overview of some formal results on the definition and properties of negations of PD. Negations of PD are generated by negators of probability values transforming element-by-element PD into a negation of PD. Negators are non-increasing functions of probability values. There are two types of negators: PD-independent and PD-dependent negators. Yager's negator is fundamental in the characterization of linear PD-independent negators as a convex combination of Yager's negator and uniform negator. Involutivity of negations is important in logic, and such involutive negator is considered in the paper. We propose a new simple definition of the class of linear negators generalizing Yager's negator. Different examples illustrate properties of negations of PD. Finally, we consider some open problems in the analysis of negations of probability distributions.

Keywords: Probability distribution; negation; linear negator; involutive negation

1 Introduction

In recent years many papers have been devoted to the analysis and applications of negations of finite probability distributions (PD) [²-¹⁴], first considered by R. Yager in [¹].

This paper gives a brief overview of some formal results on the definition and properties of negations of PD. Negations of PD are generated by negators of probability values transforming element-by-element PD into a negation of PD [²]. Negators are non-increasing functions of probability values that can be PD-independent or PD-dependent.

It was shown that Yager's negator [¹] is fundamental in the characterization of linear PD-independent negators as a convex combination of Yager's negator and uniform negator [²].

Negators give new tools for the transformation of probability distributions. They are similar to negations in fuzzy logic [¹⁵, ¹⁶]. The concepts of contracting and involutive negations studied in fuzzy logic [¹⁵,¹⁶] can be extended on negators of probability values [³].

Involutive negations are important for logic. Such involutive negation can be introduced also for probability distrbutions [³].

The paper proposes a new simple representation of linear negators generalizing Yager's negator, and compare them with other known negators of PD. The paper discusses some open problems in the analysis of negations of probability distributions.

The paper has the following structure. Section 2 gives some basic definitions of negators of probability values in PD and negations of PD. Sections 3 and 4 consider PD-independent and linear negators. Section 5 considers the influence of negations of PD on the entropy of PD. Sections 6 and 7 consider PD-dependent and involutive negators. Section 8 introduces a new, simple representation of linear negators. Section 9 contains examples. Section 10 contains the conclusion and discusses future work.

2 Negations of Probability Distributions

A probability distribution (PD) of length n is a sequence $P=(p1,…,pn)$ of n real values satisfying for all $i=1,…,n$ , $(n≥2)$ , the properties:

$0≤pi≤1, ∑i=1npi=1.$ (1)

Let $Pn$ be the set of all probability distributions of the length n. A negation of probability distribution is a function $neg:Pn→Pn$ such that for any PD $P=(p1,…,pn)$ in $Pn$ the probability distribution

$neg(P)=Q=(q1,…,qn).$ (2)

satisfies for all $i=1,…,n$ , the following properties:

$if pi≤pj, then qi≥qj.$ (3)

From (2), it follows:

$if pi=pj then qi=qj.$ (4)

The property (3) is similar to the property of negation of membership and truth values in fuzzy logic [¹⁵, ¹⁶], defined as a decreasing function on [0,1]. The essential difference between negation (complement) of fuzzy sets and negation of PD is that the negation $Q=neg(P)$ of PD $P$ should satisfy the following properties of probability distributions:

$0≤qi≤1, for all i=1,…,n,$ (5)

$∑i=1nqi=1.$ (6)

A function $N$ , transforming elements $pi$ of PD $P$ into elements $qi=N(pi)$ of PD $Q=neg(P)$ is called a negator.

From (5),(6), and (3), it follows that for all $i=1,…,n$ the negator satisfies the following properties:

$0≤N(pi)≤1,$ (7)

$∑i=1nN(pi)=1,$ (8)

$if pi≤pj, then N(pi)≥N(pj).$ (9)

In terms of negators, a negation of probability distribution $P=(p1,…,pn)$ can be presented as follows:

$negN(P)=(N(p1),…,N(pn)),$ (10)

and we will say that PD $P$ is generated by negator $N$ . The general method of constructing negators was proposed in [²].

Theorem 1 [²]. Let $P=(p1,…,pn)$ be a probability distribution and $f(p)$ be a non-negative non-increasing real-valued function satisfying the property: $∑j=1nf(pj)>0$ , then the function N defined for all $i=1,…,n$ by:

$N(pi)=f(pi)∑j=1nf(pj)$ (11)

is a negator.

Generally, a negator depends on the set of values $pi$ in $P$ , but in the following Section, we consider PD-independent negators $N(p)$ defined on [0,1] and depending only on value $p$ in [0,1].

The probability distribution

$PU=(1n,…,1n).$ (12)

is called the uniform distribution.

A probability distribution $P$ is called a fixed point of a negation neg of PD if

$neg(P)=P.$

Theorem 2 [²]. The uniform distribution is a fixed point of any negation neg of probability distributions, that is:

$neg(PU)=PU.$ (13)

3 PD-Independent Negators

A PD-independent negator $N$ is a non-increasing function $N:[0,1]→[0,1]$ such that for any $p,q$ in [0,1], it is fulfilled:

$if p≤q, then N(p)≥N(q),$ (14)

and for any PD $P=(p1,…,pn)$ in $Pn$ it is fulfilled:

$∑i=1nN(pi)=1.$ (15)

PD-independent negator $N(p)$ depends only on the value of $p∈[0,1]$ , and at first glance, it isn't easy to expect the existence of a PD-independent negator such that for any PD $P$ in $Pn$ the property (15) will hold. But formula (11) can also be used for constructing PD-independent negations, as shown in Section 8.

It is easy to see that Yager's negator [¹]:

$NY(p)=1−pn−1,$ (16)

is a PD-independent negator. It is a decreasing function of $p$ on [0,1]. The Yagers's negator defines the following negation of a probability distribution $P=(p1,…,pn)$ :

$negY(P)=(NY(p1),…,NY(pn))=(1−p1n−1,…,1−pnn−1).$

and $∑i=1nNY(pi)=1$ .

Another PD-independent negator called uniform negator was proposed in [²]:

$NU(p)=1n.$ (17)

This negator defines the transformation of any PD $P$ into the uniform distribution:

$negU(P)=(NU(p1),…,NU(pn))=(1n,…,1n)=PU.$

Theorem 3 [²]. Any PD-independent negator $N$ has the unique fixed point $p=1n$ , and any PD-independent negation of probability distributions $negN$ has a unique fixed point $PU$ .

Hence, for any PD-independent negator 𝑁 we have:

$N(1n)=1n.$

Theorem 4 [²]. For any PD-independent negator $N$ , the following properties are satisfied:

$N(p)∈[1n,1n−1] if p≤1n,$ (18)

$N(p)∈[0,1n] if p≥1n.$ (19)

For example, we have for $p=0$ and $p=1$ :

$N(0)∈[1n,1n−1],$ (20)

$N(1)∈[0,1n].$ (21)

4 Linear Negators

A PD-independent negator $N$ is called linear if $N(p)$ is a linear function of $p∈[0,1]$ . A negation of PD generated by a linear negator is called a linear negation of PD. One can see that Yager's negator and uniform negators are linear negators. The convex combination of Yager's negator and uniform negator was used in [²] for constructing a class of linear PD-independent negators. This class of negators is considered below.

Theorem 5 [²]. A PD-independent negator $N(p)$ is a linear negator if and only if it is a convex combination of negators $NU$ and $NY$ , i.e. for some $α∈[0,1]$ for all p in [0,1] it holds:

$N(p)=αNU(p)+(1−α)NY(p)=α1n+(1−α)1−pn−1.$ (22)

From (21) it follows: $nN(1)∈[0,1]$ , and using $α=nN(1)$ from (22), we can obtain:

$N(p)=N(1)+(1−nN(1))1−pn−1.$ (23)

From (23) and (21), we see that any linear negator can be obtained from Yager's negator by suitable selection of the value of $N(1)$ in the interval $[0,1n]$ . When $N(1)=0$ , formula (23) gives the Yager's negator. When $N(1)=1n$ , we obtain the uniform negator $N(p)=N(1)=1n$ .

Using $p=0$ in (23), we can obtain:

$N(0)=1−N(1)n−1,N(1)=1−(n−1)N(0),$

and represent linear negator as a function of $N(0)$ :

$N(p)=N(0)+(1−nN(0))p.$ (24)

In [²], one can find a graphical representation of linear negators.

5 Linear Negations and Entropy of PD

Theorem 6 [³]. For linear negator $N$ for any $p$ in [0,1], it holds

$limk→∞(Nk(p))=1n,$

where $N1(p)=N(p)$ , $Nk(p)=N(Nk−1(p))$ for $k=2,…$

Theorem 7 [³]. If $N$ is a linear negator, then for the corresponding linear negation of PD $negN(p)=(N(p1),…,N(pn))$ for any PD P in $Pn$ it holds:

$limk→∞(negNk(p))=negU(P)=PU,$

where $negNk(P)negN(negNk−1(P))$ for $k=2,…$ and $negN1(P)=negN(P)$ .

As it follows from Theorem 7, the multiple linear negations of probability distributions have as the limit the uniform distribution $PU=(1n,…,1n)$ with the maximal entropy [¹] value:

$H(P)=∑i=1n(1−pi)pi.$

The similar result was obtained for PD-dependent negators $NZ$ [¹⁴] and $NW$ [¹⁰] considered below.

6 PD-Dependent Negators

PD-independent negator $N:[0,1]→[0,1]$ is a function of probability value $p$ from [0,1], and $N(p)$ depends only on the value $p$ , but not depends on PD where this value appears. Hence, if $N$ is PD-independent negator, then for any PD $P=(p1,…,pn)$ and $Q=(q1,…,qn)$ in $Pn$ , it holds:

$if pi=qj, then N(pi)=N(qj).$ (25)

A negator $N$ that is not PD-independent is referred to as PD-dependent. Generally, for PD-dependent negator, the values $N(pi)$ for probability values $pi$ in a PD $P=(p1,…,pn)$ depend on other values $pj$ in $P$ , and (25) does not fulfill. Below are examples of PD-dependent negators of PD. Note that both of them have the form (11).

Negator of Zhang et al. based on Tsallis entropy [¹⁴]:

$NZ(pi)=1−pikn−∑j=1npjk,k≠0.$ (26)

Exponential negation of Wu et al. [¹⁰]:

$NW(pi)=e−pi∑j=1ne−pi.$ (27)

7 Involutive Negations and Negators

Considered above negators $N$ define negations $neg(P)$ transforming probability distributions $P$ in $Pn$ into PD $neg(P)$ in $Pn$ . If PD $P$ simulates the term $HIGH PRICE$ then its negation $neg(P)$ can simulate the term $NOT(HIGH PRICE)$ . It is reasonable like in logic to have involutive negations satisfying to: $NOT(NOT(HIGH PRICE))=HIGH PRICE$ .

A negation $neg$ of probability distributions is involutive if for any PD $P$ in $Pn$ it is fulfilled:

$neg(neg(P))=P.$

A PD-independent negator $N$ is involutive if for any $p$ in [0,1] it is fulfilled:

$N(N(p))=p.$

Theorem 8 [³]. Any PD-independent negator $N$ is non-involutive.

As follows from the Theorem 8, involutive negator should be PD-dependent. It is easy to show that PD-dependent negators considered in the previous Section are non-involutive.

Such an involutive negator was introduced by Batyrshin [³]. Let $P=(p1,…pn)$ be a probability distribution. Denote $max⁡(P)=max⁡i{pi}=max⁡{p1,…,pn}$ , $min⁡(P)=min⁡i{pi}=min⁡{p1,…,pn}$ and $MP=max⁡(P)+min⁡(P)$ .

Theorem 9 [³]. Let $P=(p1,…pn)$ be a probability distribution. Then the function:

$NB(pi)=max(P)+min(P)−pin(max(P)+min(P))−1=MP−pinMP−1,$ (28)

$i=1,…,n$ , is an involutive negator.

We have $NB(1n)=1n$ , i.e. $1n$ is the fixed point of Batyrshin's negator $NB$ . Note that negator (28) is PD-dependent. When one calculates $Q=negN(P)=(NB(p1),…,NB(p1))$ the negator $NB(pi)$ uses in (28) maximal and minimal values of P. But when one will calculate $negN(negN(P))=negN(Q)=(NB(q1),…,NB(q1))$ in the formula (28) instead of $max(P)+min(P)$ one should calculate $max(Q)+min(Q)$ .

The simplified version of involutive negation (28) for PD with $min⁡(P)=0$ was independently proposed by Pham et al. [⁸].

8 A New Representation of Linear Negators

Here we propose a new representation of linear negators:

$NL(p)=1−dpn−d, 0≤d≤1.$ (29)

This negator can be considered as a straightforward generalization of Yager's negator that will be obtained from (29) when $d=1$ . Below we show how it can be constructed from the general method of construction of negators (11).

Using (11), we can represent linear negator as a linear function as follows:

$NL(pi)=a+bpi∑j=1n(a+bpi)=a+bpina+b∑i=1npi=a+bpina+b.$

It is clear that $a≠0$ , otherwise $N(pi)=pi$ and N will be increasing, which contradicts (9). Hence we can represent $N(pi)$ as follows:

$NL(pi)=a+bpina+b=1+bapin+ba=1+cpin+c.$

Since a negator is a non-increasing function, parameter $c$ is non-positive: $c≤0$ . Replacing it by a non-negative parameter $d≥0$ , we obtain:

$NL(pi)=1−dpin−d.$ (30)

From (18) and (19) it follows $NL(p)∈[0,1n−1]$ that gives: $d≤1$ . Replacing (30) by the function defined on [0,1], we obtain a new representation for linear PD-independent negators:

$NL(p)=1−dpn−d, 0≤d≤1.$ (31)

Changing in (31) parameter $d$ from 0 to 1, we obtain different linear negators. For $d=0$ , we have uniform negator: $NL(p)=NU(p)=1n$ , and when $d=1$ , we obtain Yager's negator: $NL(p)=NY(p)=1−pn−1$ . We obtain (24) and (23) from (31) after substitutions:

$d=n−1N(0)=1−nN(1)1−N(1).$ (32)

(32) together with (20) and (21) again gives $d∈[0,1]$ .

For $p=1n$ we have in (31):

$NL(1n)=1−d1nn−d=1n,$

i.e., $p=1n$ is a fixed point of linear negators.

In the following Section we consider examples of negations of PD using different negators.

9 Examples

Consider probability distributions from $P5$ , $(n=5)$ : $P=(0,0.1,0.2,0.3,0.4)$ and $Q=(0.1,0.1,0.1,0.3,0.4)$ , with uniform distribution $PU=(0.2,0.2,0.2,0.2,0.2)$ .

Compare the results of negation of probability distributions $P$ and $Q$ using different negators.

For uniform negator $NU$ (17) we have:

$negU(P)=negU(Q)=(0.2,0.2,0.2,0.2,0.2)=PU.$

For Batyrshin's involutive negator $NB$ (28) we have:

$negB(P)=(0.4,0.3,0.2,0.1,0),negB(negB(P))=(0,0.1,0.2,0.3,0.4)=P.negB(Q)=(0.267,0.267,0.267,0.133,0.067),negB(negB(Q))=(0.1,0.1,0.1,0.3,0.4)=Q.$

For Yager's linear negator $NY$ (16) we have:

$negY(P)=(0.250,0.225,0.2,0.175,0.150),negY(negY(P))=(0.188,0.194,0.2,0.206,0.213).$

For linear negator $NL$ (31) with $d=0.5$ we have:

$negL(P)=(0.222,0.211,0.2,0.189,0.178),negL(negL(P))=(0.198,0.199,0.2,0.201,0.202),negL(Q)=(0.211,0.211,0.211,0.189,0.178).$

For negator $NZ$ of Zhang et al. (26) with $k=2$ we have:

$negZ(P)=(0.213,0.211,0.204,0.194,0.179),negZ(negZ(P))=(0.199,0.199,0.200,0.201,0.202),negZ(Q)=(0.210,0.210,0.210,0.193,0.178).$

For negator $NW$ of Wu et al. (27), we have:

$negW(P)=(0.242,0.209,0.198,0.179,0.162),negW(negW(P))=(0.192,0.196,0.200,0.204,0.208),negW(Q)=(0.219,0.219,0.219,0.180,0.162).$

These examples illustrate the main properties of negators and corresponding negations of PD:

We see that the values of $P$ and $Q$ are increasing and the values of $neg(P)$ and $neg(Q)$ are decreasing because the negators are decreasing functions.
The uniform negator transforms any PD into the uniform PD $PU$ .
For the Batyrshin's involutive negator $NB$ we have: $negB(negB(P))=P$ , but for other negators the involutivity property does not fulfil: $neg(neg(P))≠P$ .
The involutive negator $NB$ and linear negators $NY$ and $NL$ keep the fixed point: $N(1n)=1n$ . In our examples we have: $n=5$ , $1n=0.2$ , and $N(0.2)=0.2$ . For PD-dependent negators $NZ$ and $NW$ this property generally does not fulfil.
The values $N(pi)$ of PD-independent linear negators depend only on negated probability value $pi$ but for PD-dependent negators they depend also on other probability values $pj$ in PD. Compare negations of values 0.1, 0.3 and 0.4 in PD $P$ and $Q$ by linear negator $NL$ and PD-dependent negators $NZ$ and $NW$ .
All considered non-involutive negators transform PD $P$ into $neg(P)$ and $neg(neg(P))$ approaching uniform distribution, in our case $PU=(0.2,0.2,0.2,0.2,0.2)$ . This property proved for linear negators, see Theorems 6 and 7, and for $NZ$ and $NW$ , but generally not necessary fulfilled for PD-dependent negators.

10 Conclusion and Future Work

The paper observed the main definitions and formal properties of negators and corresponding negations of probability distributions. The examples illustrate these properties. We did not consider here contracting negations of PD [³] also studied in fuzzy logic [¹⁵, ¹⁶].

In [³], it was proved that any linear negator except for uniform negator $NU$ is strictly contracting. As a result, multiple linear negations increase the entropy of probability distributions and converge to the uniform distribution $PU$ . We plan to study negations of PD as transformations of PD-dependent negations also in terms of similarity and distance between PD [¹⁷,¹⁸].

Another direction of possible research is related to the linguistic interpretation of PD proposed by Yager [¹]. We can use similar approaches to linguistic descriptions of other types of data [¹⁹].

Also, we plan to consider possible applications of negations of PD, in Dempster-Shafer theory, like in [¹] and some other papers from the list of references.

The open problems are:

To prove or disprove the hypothesis that any PD-independent negator is a linear negator [²].
To find contracting or expanding PD-dependent negations of PD [³].
To find negators for which the multiple negations decrease (increase) the entropy of PD.

Acknowledgments

This work has been partially supported by the project IPN SIP 20211874 and by the program № 075-02-2020-1478 of developing the Scientific-Educational Mathematical Center of Volga Federal District.

References

1. Yager, R. R. (2014). On the maximum entropy negation of a probability distribution. IEEE Transactions on Fuzzy Systems, 23(5), 1899–1902. [ Links ]

2. Batyrshin, I., Villa-Vargas, L. A., Ramírez-Salinas, M. A., Salinas-Rosales, M., Kubysheva, N. (2021). Generating negations of probability distributions. Soft Computing, 25(12), 7929–7935. [ Links ]

3. Batyrshin, I. Z. (2021). Contracting and involutive negations of probability distributions. Mathematics, 2021, 9, 2389. arXiv preprint arXiv:2103.16176. [ Links ]

4. Gao, X., & Deng, Y. (2019). The generalization negation of probability distribution and its application in target recognition based on sensor fusion. International Journal of Distributed Sensor Networks, 15(5), 1550147719849381. [ Links ]

5. Gao, X., Deng, Y. (2019). The negation of basic probability assignment. IEEE Access, 7, 107006– 107014. [ Links ]

6. Li, S., Xiao, F., Abawajy, J. H. (2020). Conflict Management of evidence theory based on belief entropy and negation. IEEE Access, 8, 37766– 37774. [ Links ]

7. Luo, Z., Deng, Y. (2019). A matrix method of basic belief assignment's negation in Dempster–Shafer theory. IEEE Transactions on Fuzzy Systems, 28(9), 2270–2276. [ Links ]

8. Pham, U., Batyrshin, I., Kubysheva, N., Kosheleva, O. (2021). Estimating a probability distribution corresponding to the negation of a property. Soft Computing, 25(12), 7975–7983. [ Links ]

9. Sun, C., Li, S., Deng, Y. (2020). Determining weights in multi-criteria decision making based on negation of probability distribution under uncertain environment. Mathematics, 8(2), 191. [ Links ]

10. Wu, Q., Deng, Y., Xiong, N. (2020). Exponential negation of a probability distribution. arXiv preprint arXiv:2010.11533. [ Links ]

11. Xie, D., Xiao, F. (2019). Negation of basic probability assignment: Trends of dissimilarity and dispersion. IEEE Access, 7, 111315–111323. [ Links ]

12. Xie, K., Xiao, F. (2019). Negation of belief function based on the total uncertainty measure. Entropy, 21(1), 73. [ Links ]

13. Yin, L., Deng, X., Deng, Y. (2018). The negation of a basic probability assignment. IEEE Transactions on Fuzzy Systems, 27(1), 135–143. [ Links ]

14. Zhang, J., Liu, R., Zhang, J., Kang, B. (2020). Extension of Yager's negation of a probability distribution based on Tsallis entropy. International Journal of Intelligent Systems, 35(1), 72–84. [ Links ]

15. Batyrshin, I.Z. (2001). Basic Operations of Fuzzy Logic and their Generalizations. Otechestvo Publisher: Kazan, Russia [in Russian]. [ Links ]

16. Batyrshin, I. (2003). On the structure of involutive, contracting and expanding negations. Fuzzy Sets and Systems, 139(3), 661–672. [ Links ]

17. Cha, S. H. (2007). Comprehensive survey on distance/similarity measures between probability density functions. International journal of mathematical models and methods in applied sciences, 1(2), 1. [ Links ]

18. Batyrshin, I. (2019). Towards a general theory of similarity and association measures: similarity, dissimilarity and correlation functions. Journal of Intelligent, Fuzzy Systems, 36(4), 2977–3004. [ Links ]

19. Batyrshin, I., Sheremetov, L., Herrera-Avelar, R. (2007). Perception-based patterns in time series data mining. In Perception-based Data Mining and Decision Making in Economics and Finance, pp. 85–118, Springer, Berlin, Heidelberg. [ Links ]

Received: September 02, 2021; Accepted: October 03, 2021

^* Corresponding author: Ildar Z. Batyrshin, e-mail: batyr1@gmail.com

This is an open-access article distributed under the terms of the Creative Commons Attribution License