Reaching High Interactive Levels with Situated Agents

 

Alcanzando altos niveles de interacción con agentes situados

 

Ibarra-Martínez S.¹, Castán-Rocha J.A.², Laria-Menchaca J.³, Guzmán-Obando J.⁴ y Castán-Rocha E.⁵

 

¹ Facultad de Ingeniería Arturo Narro Siller Centro Universitario Tampico-Madero, México Universidad Autónoma de Tamaulipas. E-mail: sibarram@uat.edu.mx

² Facultad de Ingeniería Arturo Narro Siller Centro Universitario Tampico-Madero, México Universidad Autónoma de Tamaulipas. E-mail: jacastan@uat.edu.mx

³ Facultad de Ingeniería Arturo Narro Siller Centro Universitario Tampico-Madero, México Universidad Autónoma de Tamaulipas. E-mail: jlaria@uat.edu.mx

⁴ Departamento de Ingeniería Eléctrica y Electrónica Instituto Tecnológico de Ciudad Madero. E-mail: jguzmano@uat.edu.mx

⁵ Departamento de Ingeniería Eléctrica y Electrónica Instituto Tecnológico de Ciudad Madero. E-mail: ecastanr@yahoo.com.mx

 

Information on the article: received: February 2010.
Reevaluated: April 2011.
Accepted: January 2012

 

Abstract

The paper presents a coordinated process through which the situated agents are capable of working jointly using knowledge about their skills to interact with other similar entities. A good coordination is reached because each situated agent is able to represent its suitability rates to perform any action in an effective way. Besides using such rates, an agent can coordinate its actions with other agents. In this sense, each situated agent tries to select and perform only the actions with the highest execution rates. Some experiments in a real robotic soccer test-bed are designed to put in practice the proposed coordinated approach. Final remarks conclude the effectiveness and advantages of the presented work in order to increase the performance of a team composed by situated agents when they must solve complex tasks in a dynamical, competitive and unpredictable scenario.

Keywords: situated agents, autonomous mobile robots, interaction, cooperative methods, robot soccer, complex tasks.

 

Resumen

Este artículo presenta un proceso de coordinación a través del cual, los agentes situados son capaces de trabajar conjuntamente utilizando conocimiento acerca de sus capacidades para interactuar con otras entidades similares. Una buena coordinación se puede lograr debido a que cada agente situado es capaz de representar, de manera efectiva, sus rangos de efectividad para ejecutar cualquier acción. Además, usando la información puede coordinar su trabajo con otros agentes. En este sentido, cada agente situado trata de seleccionar y ejecutar únicamente las acciones para las que tiene un alto nivel de efectividad. Algunos experimentos en el escenario de futbol robótico se desarrollaron para poner en práctica el método de coordinación aquí propuesto. Los comentarios finales concluyen la efectividad y ventajas del trabajo presente para incrementar el rendimiento de un equipo de agentes situados cuando ellos deben solucionar tareas complejas en escenarios dinámicos, competitivos e impredecibles.

Descriptores: agentes situados, robots cooperativos móviles, interacción, métodos cooperativos, futbol robótico, tareas complejas.

 

Introduction

Simply stated, decision-making structure is a process where agents should be able to select a specific action out of multiple alternatives. Such process occurs continuously in daily life. Human beings, for instance, usually must take decisions about what cloths to wear, what food to eat, etc. Likewise, an agent is defined as anything that is situated in a specific environment and acts based on its observation, interpretation and knowledge about the situation of such environment, to select a particular action. Therefore, to take decisions, agents must obtain knowledge that allows them to be aware of what actions can or cannot perform. Agents who should work jointly must be able to converse with other agents in order to communicate information related to the actions that each can do. Methods for cooperative multi-agent decisions are therefore, in most cases, intensive software applications and highly sophisticated algorithms that use advanced design technologies. In particular, some research trends have led to managing complex and cooperative problems using agents. A situated agent is presented as a physical and mobile entity capable of flexible and autonomous actions in dynamic, unpredictable and typically cooperative environments (Luck et al., 2005). In this sense, several results to solve coordinated actions have been obtained using agent technology (Bogdan et al., 2008; Zhang and Parker, 2010; Barret et al., 2010). In particular, relevant knowledge is not appropriately reflected and communicated by the agents. These deficiencies do not allow agents to make reliable collective decisions when these are requested (Ibarra, 2008). Obviously, lack of the appropriate reasoning on these knowledge results in a lower cooperative performance among agents, where a proper managing of such knowledge is quite relevant to achieve sure and trustworthy commitments. Achieving cooperative agents is still a challenge, because many issues must be considered in order to develop a working cooperative team, such as task allocation, conflict of interest, communication, etc. (Fang and Parker, 2007; Parker, 2008; Jeong et al, 2006; Benson et al., 2007). In such a case, cooperative agent systems often work in dynamic and unpredictable scenarios, for thus, the members of a team-agent must respond robustly and adaptively to unexpected environmental changes, such as unsuitable communication among them or modification in the agent system configuration due to failures of performance. In particular, cooperative systems of situated agents are characterized by distributed control of heterogeneous agents. A number of interesting reasons exist to support the cooperation among these agents. In fact, the objective of the cooperation is to maintain maximum utilization of multi-agent resources while ensuring job performance at the highest productive level.

 

How to reach interaction

High interactive levels are desired due to the fact that the solution to a more complex problem could be more successfully performed if multiples autonomous and intelligent entities can work together with the same global objective in mind. To reach interaction, first, an agent must be able to differentiate from each other in order to perceive its situation. Such information allows agents to know what action they can do by estimating their operatively rates from all the possible alternatives.

Let us define a situated agent (SA_j) as an intelligent entity with a physical representation and through which the system can perform physical changes on the environment. These situated agents are embodied by considering the knowledge involved in their capability to execute an action within their knowledge base. Let us suppose that a SA_j is part of a cooperative group of situated agents (G_SA). A group of situated agents must generally involve more than one situated agent for the fulfillment of a particular task.

According to this, an agent uses the information provided by three (3) parameters, called decision axes, where each axis provides situated agents with knowledge related to its capability to execute any determined action with a particular kind of information in a determined time (t) inside a specific spatial region, called scene (S). Axis 1, environmental conditions (EC), is composed by information related to the state of the environment directly involved in the performance of a particular action. Axis 2, physical knowledge (PK), is represented by the specification, the structure and other relevant details related to the agents' physical skills and characteristics. Finally, axis 3, trust value (TV), is related to the capability of an agent to communicate, to interact and other relevant details to entrust in other agents. In this light, the situated agent's knowledge base KB(SA_j) is therefore based on the combination of the three axes above (EC, PK and TV) directly implicated in the execution of any action, such as it is described by (1).

In particular the situated agent's knowledge base for the execution of a specific role (R_β) in a given time (t), in a determined scene (S_α), is given by (2).

In particular, this paper assumes that each situated agent is capable to evaluate its range of aptitude to perform any action. Such evaluation is performed by a match that includes two important aspects used to calculate their suitability rate to all the proposed action. These aspects are:

• The capacity of a situated agent taking into account the information provided by the three axes.

• The influence degree that each axis has over the execution of any determined action.

In particular, the influence degree y refers to the relevance that decision axes have in the selection of a determined action in a particular scene. Such influence aims to calculate critically the suitability of a situated agent to execute any role R_β from all the roles R in a successful and reliable way. In this sense, such influence degree y is represented by (3).

and

is the relevance of the environmental conditions,

is the relevance of the physical knowledge and

is the relevance of the trust value.

In particular the influence degree for the development of any specific role in any determined scene is given

Then, the suitability rate S, of any situated agent is obtained by a match between the requirement of the roles and the capability of the situated agents . Let us suppose that a situated agent SA_j is capable of executing a role R_β with a suitability rate ξ in a time t, within a scene S_α as it is described in (5).

To the end, this match ensures that any situated agent is capable of knowing which roles it can do in a suitable and reliable way. This fact guarantees an increase in the performance of the team-agent because almost all the time each role is performed by a trustworthy situated agent.

 

Some experiments and results

Experiments are devoted to prove the efficiency of the proposed approach. Hence, some features have been fixed to do an implementation in the real robotic soccer test-bed. Particularly, each axis is implemented using different information of the field, as it is described. Environmental conditions (axis 1), considered here as a parameter of proximity P, are related to the real distance between the current location of a situated agent and the current location of the proposed role, and it is provided by (6).

Where d max means the maximal distance of a situated agent with the proposed roles in a scene S_α as it is described in (7)

Physical knowledge (axis 2) refers to the cognitive ability of each situated agent to estimate the knowledge related to the capabilities of its body involved in the execution of a proposed action, represented as a capability of introspection I. Introspection parameter is calculated implementing feed-forward back-propagation neural networks.

Trust value (axis 3), here called trust T, refers to the social relationship among agents taking into account both the amount of "good" actions which are actions executed in a suitable way and the amount of "bad" actions which are actions that are executed in a negative way. So that, the trust measure of a situated agent is provided by (9).

Once the implementation of each axis has been introduced, an agent is capable to define how the relevance of the set of decision axes can influence in the calculus of the situated agents' suitability rates. We have designed a classification of a binary combination of the three axes (i.e., [0, 0, 0] where 0 means that the parameter is disable and [1, 1, 1] where 1 means that the parameter is used). From this, we have obtained eight study cases. In particular, each study case denotes the behavior of each one of the team that we have used in the empirical experiments. It means that each team-agent uses one of the binary combinations to allow agents self-calculate their suitability rates. In particular, Figure 1 shows the general scheme of the robot soccer test-bed used in this experimental phase.

These experiments are related to eight different teams of agents which use each one of the eight possible influence degree. These teams compete among themselves in ten (10) episodes of only one game with a predefined period of five (5) minutes. In particular, the overall expected team's performance is related to reaching the greater number of achieved points and goals. For example, Figure 2 illustrates the obtained results in one of the episodes of the experiments. Here, the overall expected performance by each team is measured using the total score of points to obtain a goal average (G_AVE). The goal average is calculated by the difference between the obtained goals (G+) and the received goals (G-).

In addition, the Figure 3 illustrates the teams' performance taking into account the successful number of points obtained based on the won and tied games along all the episodes. The performance does not improve significantly somewhere beyond the 6 episodes of the experiment. The number of trials (episodes) to initially confirm the systems performance was fixed in 10.

As a conclusion, these statistical results confirm clearly the idea that situated agents can make sure decisions, and carry out a better performance when they use knowledge related to their situation to perform any proposed action. Due to the difficult, in the development of these experiments, the number of games is small but not less important for the aims of this research.

So, the exposed results confirm clearly the difference between the system performances where it uses coordinated task problem-solving strategy. In particular, there is an improvement rate of around an 63% in this scenario between the best team (using the selection of parameters [1, 1, 1]) and the worst team (using the parameters [0, 0, 0]).

 

Final remarks

In this paper it is possible to argue how situated agents that use information about their rates of actions are capable to reach better decisions out of all their possible alternatives. In this light, the decision performance (successful decisions) of the situated agents is better when they are able to estimate their situation (i.e., their knowledge base) related to the execution of any proposed actions, than when the agents do not use it. Successful decisions in the experiments are related to the ratio between the number of successful tasks performed by the situated agent and the total number of actions performed by the same agent. According to this, the situated agents increase significantly the number of successful actions if they are able to base its decisions on their situation considering the three proposed parameters over the environment.

The data from the experiments discloses that the implementation of the three parameters of the decision axes combined within the agents' decision-making structure, produces best performance in all the experiments. But more importantly, the system performance (successful performance) is significantly better when the agents increase the information (i.e., when the agents use grater amount of knowledge) involved in their decision-making to perform any action. However, the remaining cases show interesting results but not an optimal strategy for the present domains at all. This fact illustrates that the choice of a strategy that includes knowledge in the agents' decision-making structure is far from trivial. In this case, the obtained results are significant and show the need for further investigation about the agents' situation and its effect in the performance of complex problems in dynamic and cooperative environments.

This is a complicated process because the number of roles grows exponentially. Therefore, each agent is able to perceive and interpret the information involved in the proposed actions to generate its knowledge base. In this sense; redundancy in the tasks execution is then avoided. Finally, this new and effective approach contributes to enhance multi-agent efficiency and performance in dynamic and cooperative environments because the agents are capable of knowing if they can perform the proposed action.

 

Acknowledgments

This work is supported by the Facultad de Ingeniería "Arturo Narro Siller" at the Universidad Autónoma de Tamaulipas.

 

References

Barrett S., Taylor M.E., Stone P. Transfer Learning for Reinforcement Learning on a Physical Robot, on: Ninth International Conference on Autonomous Agents and Multiagent Systems - Adaptive Learning Agents Workshop (AAMAS-ALA), Toronto, Canada, May 2010.         [ Links ]

Benson E., Berliner J., Brinn M. Framework for Multiresolutional Knowledge Dissemination and Collection in Dynamic Agent Societies, on: Proc. of the International Conference on Integration of Knowledge Intensive Multi-Agent Systems (KI-MAS'07), 2007, pp. 97-102.         [ Links ]

Fang T., Parker L.E. A Complete Methodology for Generating Multi-Robot Task Solutions using ASyMTRe-D and Market- Based Task Allocation, on: Proc. of the IEEE International Conference on Robotics and Automation, pp. 3351-3358, April 2007.         [ Links ]

Ibarra M.S. A Formalization for Multi-agent Decision Support in Cooperative Environments. A Framework for Situated Agents, (Doctoral Thesis), Girona, Catalonia, Spain, 2008.         [ Links ]

Jeong-Ki Y., Yong-Duk K., Bum-Joo L., In-Won P., Kuppuswamy N.S., Jong-Hwan K. Hybrid Architecture for Kick Motion of Small-sized Humanoid Robot, HanSaRam-VI, on: Proc. of the International Joint Conference SICE-ICASE'06, October 2006, pp. 1174-1179.         [ Links ]

Luck M., McBurne P., Shehory O., Willmott S. Agent Technology: Computing as Interaction, In A Roadmap for Agent Based Computing, Compiled, written and edited by Luck M., McBurney P., Shehory O., Willmott S. and the AgentLink Community, pp. 11-12, 2005.         [ Links ]

Parker L.E. Distributed Intelligence: Overview of the Field and its Applications in Multi-Robot Systems. Journal of Physical Agents, volume 2 (issue 1), 2008: 5-14.         [ Links ]

Zhang Y. and Parker L.E. A General Information Quality Based Approach for Satisfying Sensor Constraints in Multirobot Tasks, on: Proc. of IEEE International Conference on Robotics and Automation, Anchorage, Alaska, 2010.         [ Links ]

 

About the authors

Salvador Ibarra-Martínez. He holds the doctor degree on computer science (focused on cooperative robotics) from the University of Girona, Cataolina, Spain. He is a lecturer at the Universidad Autonoma de Tamaulipas, Mexico. He is member of the SNI. His research interests include intelligent agent systems, coordination mechanisms for autonomous mobile cooperative robots with application for both real and simulated robot soccer and rescue test beds.

José A. Castán-Rocha. Received the master degree in science by the Universidad Autonoma de Tamaulipas. He is, currently, PhD candidate in the program of organization engineering, of the Escuela Superior de Ingenieros at the University of Sevilla, Spain. He is a lecturer at the Universidad Autonoma de Tamaulipas. His research interests are related to intelligent transit systems, models of vehicles flow and their implementation with intelligent agents.

Julio Laria-Menchaca. He holds the doctor degree in physics by the CICATA-IPN, Mexico. He is a lecturer at the Universidad Autonoma de Tamaulipas. His research interests involve electronic designs, automation and intelligent control.

Javier Guzmán-Ovando. Received the doctor degree on computer science from the University of Girona, Catalonia, Spain. He is a lecturer at the Universidad Autonoma de Tamaulipas, Mexico. His research interests include distributed artificial intelligence and recommender systems

Emilio Castán-Rocha. Received the master degree in ???. He is a lecturer at the Universidad Autonoma de Tamaulipas and at the Instituto Tecnologico de Ciudad Madero. His research interests are related to control theory, electrical and engineering component systems.