Previous column

Next column

Modeling Agents and their Environment: The Physical Environment

James Odell, James Odell Associates, Ann Arbor, U.S.A.
H. Van Dyke Parunak, Mitchell Fleischer, Sven Brueckner, Altarum, Ann Arbor, U.S.A.


PDF Version


Without an environment, an agent is effectively useless. Cut off from the rest of its world, the agent can neither sense nor act. An environment provides the conditions under which an entity (agent or object) can exist. It defines the properties of the world in which an agent will function. Designing effective agents requires careful consideration of both the physical and communicational aspects of their environment. This paper is the first of two part and deals with the physical environment aspect of agents.


Agents need to operate and exist within an environment. Figure 1 illustrates a common view that agents perceive their environment though sensors as well as effect actions on it. [Pfeifer, 1999; Weiss, 1999; Russell, 1995]. For example, a Stock agent can receive an event indicating that quantities of a particular part are low. The agent then decides whether more parts need to be ordered and, if so, put out a general call-for-proposal so that interested vendors can reply. When proposals arrive, the Stock agent will choose and notify the winning vendor. This model implies that agents interact via an environment. Even direct communications (such as vender notification) must occur through some medium. In other words, the environment provides the appropriate conditions that enable interaction among agents. This insight, largely overlooked in the design of purely electronic agents, is particularly critical for managing agents that are situated in the physical world.

Fig. 1. Agents interact with and through their environment.


One of the key properties of agents is their autonomy. However, autonomy is not an all-or-nothing issue. Practically speaking, agents can neither be totally free of external dependencies nor completely reliant on them. They always depend on external factors to some degree.

An environment provides the conditions under which an entity (agent or object) exists.

In other words, it defines the properties of the world in which an agent can and does function. An agent’s environment, then, consists not only of all the other entities in its environment, but also those principles and processes under which the agents exist and communicate. Designing effective agents requires careful consideration of all of these factors when designing their environment.

A canonical example of agents situated in an environment is an ant colony. Ants interact with one another largely through chemicals, called pheromones, that they deposit in the environment and then sense to guide their actions. Numerous individual interactions yield the emergent development of paths through the environment. However, the environment is more than just a communication channel. Agents depend both on tangible, physical support and on other agents. Two aspects, then, are critical for agent environments (and the formation of paths): the physical and the communicational. This paper discusses with the physical environment; the next paper discusses with the communication environment.


The particular kind of environment that biological agents (animals and plants) require for survival is referred to as their ecological niche. Edward O. Wilson defines ecological niche as: “The range of each environmental variable such as temperature, humidity, and food items, within which a species can exist and reproduce.” [Wilson, 1975]. While artificial agents can have different requirements for survival, they still require an ecological niche, or physical environment, to support them.

The physical environment provides those principles and processes that govern and support a population of entities.

Principles of a physical environment

The laws of physics provide us with the fundamental truths that are essential to the world in which we live. For example, a physicist could use the study of particle dynamics to describe the causes for motion and the way in which bodies influence each other. For such descriptions, we obtain principles such as the conservation of energy, gravity, sound waves, and fluid dynamics. In Karl Sims’ agents, the same principles apply because his “creatures” were bred to swim, run, and fly in a world whose laws of physics are almost identical to ours. [Sims, 1994a, 1994b] In contrast, the ant’s environment has its own particle dynamics. For example, ants may only move from one place to an adjacent place; no two ants may occupy the same place at the same time; and yet pheromones may be aggregated when separate ants deposit them at the same place. The concepts of diffusion and evaporation are also part of the agent environment. This makes it possible for pheromones to spread to neighboring places as well as evaporate over time. Similarly, a statement of fundamental qualities is also required for agent environments. Here, each agent-based system must identify and define those fundamental truths forming the ground of its system.

For agents, principles of the physical environment can be thought of as laws, rules, constraints, and policies that govern and support the physical existence of agents and objects. Basic characteristics for an agent environment can include [Weiss, 1999; Russell, 1995]:

  • Accessibility. To what extent is the environment known and available to the agent? An environment is effectively accessible if the agent can access the environmental state relevant to the agent’s choice of action. Another consideration is whether the available resources are ample or restricted.
  • Determinism. To what extent can the agent predict events in the environment? The environment is deterministic when the next state of the environment can be determined by the current state and the actions selected by the agents.
  • Diversity. How homogeneous or heterogeneous are the entities in the environment?
  • Controllability. To what extent can the agent modify its environment?
  • Volatility. How much can the environment change while the agent is deliberating?
  • Temporality. Is time divided in a clearly defined manner? For example, do actions occur continuously or discrete time steps or episodes?
  • Locality. Does the agent have a distinct location in the environment which may or may not be the same as the location of other agents sharing the same environment. Or, are all agents virtually collocated? Also, how is a particular locality expressed (e.g., coordinate system, distance metrics, relative positioning)?

Processes of a physical environment

Formally, an environment can be expressed as a two-tuple [Parunak, 1997]:

Environment = <Statee , Processe>

where Statee is a set of values that completely define the environment. The structure, domains, and variability of these values are not constrained by this definition, and differences in these features are responsible for much of the interesting variation among different kinds of environments. The state also includes the agents and objects within the environment. Processe is an autonomously executing mapping that changes the environment’s state, Statee. “Autonomously executing” means that the process runs without being invoked from any outside entity. In computational terms, an environment has its own virtual CPU. The important feature of this definition of environment is that the environment itself is active. It has its own process that can change its state—which includes the agents and objects within the environment—independently of the actions of its embedded agents1.

In an agent environment, the primary purpose of these processes is to implement the environmental principle. For example, the gravitational field is a principle that can be implemented with a process that attracts entities in a prescribed manner. In other words, the falling of an apple to earth can be regarded as the process of gravity in action.

In the case of ants, the environment is not a passive conduit for information. Instead, it actively processes pheromones in three ways. It aggregates pheromone deposits from separate ants at the same place (thus realizing a primitive form of information fusion). It evaporates pheromones over time (thus providing a novel form of truth maintenance). Finally, it propagates pheromones to neighboring places (thus disseminating information). Experiments show that these mechanisms are critical to the formation of paths. More generally, environmental activity means that the environment may change even when the agents living in the environment do not take action.

Different physical environments will be required for different kinds of agents—and vice versa. With artificial agents, much more than physics is happening because much of the environment is information intensive. In ant-based environments, the pheromones are information. In many defense-related agent systems, the information-intense environment includes satellite telemetry, body- and vehicle-based communications technology, and geographic positioning grids. In agent-based supply chains, information about orders and resources is a major component of the system.

To support the varied information requirements of such agent-based systems, a common processing platform would be useful. This platform would provide a foundation upon which agent applications could build to leverage their own specific environmental requirements. Such a platform—whether the agents are implemented as software, hardware, or a combination of both—would consist of:

  • Application Support contains the applications, as well as all management and support services for the entities supported by the environment, such as directory and ontology services, query, mobility, security, and firewalls.
  • Communication and Transportation packages, routes, verifies, and transmits data required for the application support layer. It provides a general-purpose service that has no application dependencies and the type of data does not matter.
  • Physical Linkage specifies the physical and electrical characteristics of the bus. Typically, this involves the hardware that converts the characters of a message into electrical signals for transmitted messages and electrical signals into characters for received messages. This can include standard physical interfaces such as controllers, actuators, sensors—as well as road networks and pallets.

The processes for an agent’s physical environment may be implemented in either hardware or software; however, at some point (Physical Linkage) the environment must be realized in some material form. For example, CAN (Controller Area Network) has developed hardware for Physical Linkage layer. They have also developed software for the Application Support layer that supports CAN controllers and interface devices [].

Fig. 2. The agent platform specified by FIPA.

Some work has already been done to define the standard services required for agent-based physical environments. The FIPA (Federation of Intelligent Physical Agents) Agent Platform defines an abstract architecture for agent deployment and is summarized in Fig. 2. [FIPA, 1998] The existence of layered protocol such as FIPA and ISO shows that people already have an intuition about the importance of relating agents to the rest of the world.

  • Agent management system (AMS) can be implemented as a single agent that supervises access to and use of the agent platform. The AMS maintains a directory of logical agent names and their associated transport addresses for an agent platform. The AMS is responsible for managing the lifecycle of the agents on the platform and actions such as authentication, registration, de-registration, search, and mobility requests.
  • Agent platform security manager (APSM) is responsible for maintaining security policies for the platform and infrastructure. The APSM is responsible for run-time activities, such as communications, transport-level security, and audit trails. Security cannot be guaranteed unless, at a minimum, all communication between agents is carried out through the APSM.
  • Agent platform communication channel provides a path for basic interchange between agents, agent services, AMS, and other agent platforms. It must at least support IIOP. Agents can reach agents on any number of other platforms through the Agent Communication Channel. Ways of communicating include using blackboard or message-based communication; point-to-point, multicast, or broadcast; push or pull; and synchronous or asynchronous.

In spite of the acronym, the FIPA architecture focuses almost entirely on the electronic environment, and does not address the physical environment. As such, it does not address the real potential of an active environment to provide emergent system-level behavior. As stated earlier, every agent has an environment. However, such environment can be consciously used in special ways to get more powerful interaction.

A standard that does address the physical environment is the ISO/OSI model, depicted in Fig. 32. This model describes how communications should occur between computers on any network, and has been adopted as a general "open" network communication standard. In principle, anything that conforms to the standard can communicate, electronically, with anything else that conforms to the standard.

Fig. 3. The ISO 7498 Open Systems Interconnection (OSI) model [ISO 1994].

Population of a communication environment

An environment is an inhabited place; i.e., it is populated. An agent’s environment might or might not contain other entities, and it might be open or closed. An environment’s population is the totality of entities under its consideration. For the environment of the canonical software ant, this population would consist of food, pheromones, and other ants. For a real-world ant, it would also include earth, twigs, trees, and picnics. For a stock agent in a supply network, it would include physical inventory, road and rail networks, packaging conventions, and so on.


By 2015, the social computing is expected by some to morph into ecological or symbiotic computing. John Seely Brown, chief scientist of Xerox suggests that structural matter (atoms) and computing (bits) will become inseparable.

Zillions of sensors, effectors and logical elements (made of organic and inorganic materials) will be interconnected via wireless, peer-to-peer technologies, producing smart, malleable stuff used to build smart appliances, buildings, roads and more. It is during this era that computers disappear. In their place, nearly every physical artifact harbors some computationally based brainpower that helps it know where it was, what was near it, when it was moved and so on. In a way, the inorganic world took on organic properties, using computing to transparently modulate responses to the environment. [Brown, 2001]

1 The exact nature of the coupling between agents and their environment depends on how state and process are modeled in each: as a discrete-event or time-based dynamical system. The former involves a discrete state with a symbol-manipulation processing style; the later, a continuous state with difference or partial differential equations. See [Parunak, 1996] for more details.
2 Guy Genilloud, Guy has proposed a flexible translation for linking FIPA to OSI via CORBA in [Genilloud, 1997].


[FIPA 1998] Foundation for Intelligent Physical Agents FIPA98. Agent Management Specification, Geneva, Switzerland, Oct. 1998. (

[Geni 1997] Guy Genilloud: “Flexible Translation for Integrating CORBA and OSI,” Proceedings of Distributed Object Computing in Telecommunications (DOCT 97), Object World, Frankfurt, Germany.

[ISO 1994] ISO/IEC and ITU-T, “Information Processing Systems, OSI Reference Model: The Basic Model,” Standard 7498-1, Recommendation X.200, 1994.

[Paru 1997] H. Van Dyke Parunak: "’Go to the Ant’: Engineering Principles from Natural Agent Systems," Annals of Operations Research, volume 75, pp. 69-101.

[Pfeif 1999] Rolf Peifer and Christian Scheier: Understanding Intelligence, MIT Press, Cambridge, MA.

[Russ 1995] Stuart Russell and Peter Norvig: Artificial Intelligence: A Modern Approach, Prentice-Hall, NJ.

[SRI 2000]

[Sims 1994a] Karl Sims: "Evolving Virtual Creatures," Siggraph Proceedings, July 1994, pp. 15-22.

[Sims 1994b] Karl Sims: "Evolving 3D Morphology and Behavior by Competition," Artificial Life I. Proceedings, R. A. Brooks and P. Maes, eds., MIT Press, Cambridge, MA, pp. 28-39.

[Weiss 1999] Gerhard Weiss, ed.: Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, MIT Press, Cambridge, MA.

[Wils 1975] Edward O. Wilson: Sociobiology, (abridged edition), Belknap Press, Cambridge, MA

About the authors

James J. Odell is a consultant, writer, and educator in the areas of object-oriented and agent-based systems, business reengineering, and complex adaptive systems. He has written four books on object orientation and has two books in progress on agent-based system design. His website is

Dr. Van Parunak is Altarum's Chief Scientist. He is currently working on the applications of complex adaptive systems, with special emphasis on fine-grained agent software architectures for modeling, control, and collaboration. He has written numerous seminal papers in this area, which are available at

Dr. Mitchell Fleischer is a Scientific Fellow at Altarum and Practice Area Leader for Supply Chain Engineering. He is an Organizational Psychologist by training and has spent the past 18 years working on issues related to the adoption of new technologies in manufacturing settings. His most recent book is Concurrent Engineering Effectiveness: Integrating Product Development Across Organizations, written with Jeffrey Liker. His website is

Dr. Sven A. Brueckner is a member of the Technical Staff at Altarum. His website is

Cite this column as follows: James Odell et al.: "Modeling Agents and their Environment: The Physical Environment, in Journal of Object Technology, Vol. 2, No. 2, March-April 2003, pp. 43-51.