Agent Models

The most effective way to organize work under partial observability is for an agent to track that part of the world that he perceives at the moment. This means that the agent must maintain a kind of internal state , which depends on the history of acts of perception and therefore reflects at least some of the unobservable aspects of the current state.

To solve the braking problem, supporting the internal state is not too expensive - it’s enough to save the previous frame taken by the video camera so that the agent can determine the moment when two red lights on both sides of the rear of the vehicle in front light up or go off at the same time. To solve other driving tasks, such as moving from one lane to another, the agent must keep track of where other cars are located if he cannot see all these cars at the same time.

To ensure that this internal state information can be updated over time, it is necessary for the agent’s program to encode two types of knowledge. First, we need certain information about how the world changes independently of the agent, for example, that a car going for overtaking usually becomes closer than at some previous moment. Secondly, some information is required on how the agent’s own actions affect the world, for example, that when the agent turns the steering wheel clockwise, the car turns to the right or that after driving along the motorway five minutes to the north, the car is five miles to the north. from where he was five minutes ago.

This knowledge of “how the world works” (which can be embodied in simple logic circuits or in complex scientific theories) is called the model of the world. An agent that uses such a model is called a model-based agent.

The figure shows the structure of a reflex agent acting with regard to the internal state, and shows how the current perception is combined with the previous internal state to produce an updated description of the current state. The program of such an agent is shown in the listing. In this listing, of interest is the Update-State function, which is responsible for creating a new description of the internal state. This function not only interprets the results of a new perception in the light of existing knowledge about the state, but also uses information about how the world is changing to track invisible parts of the world, and therefore should take into account information about how the agent’s actions affect the state of the world.

function Reflex-Agent-With-State ( percept ) returns action action
static: state , description of the current state of the world
rules , a set of rules conditional action
action , last action;
originally undefined
state <- Update-State ( state , action , recept )
rule <- Rule-Match ( state , rules )
action <- Rule-Action [ rule ]
return action