Neisser’s Cycle of Perception: Formal Representation and Practical Implementation

This article presents results of a theoretical investigation of the application of the objectoriented Unified Modelling Language to a formal description of Neisser’s cycle of perception. The article includes class diagrams, modelling the spatial structure of Neisser’s cycle of perception and demonstrates how these formal descriptions can be applied to the area of modelling the problemindependent dialogue process. The basis of such applications is the assumption that the dialogue process in relation to either of the dialogue agents is similar to the process of routine perception of the environment in accordance with Neisser’s understanding of perception.


INTRODUCTION
This article is devoted to one of classical models of Cognitive Psychology know as Neisser's cycle of perception [1] . Besides a conceptual description of the model the article proposes its formalization by means of the Unified Modelling Language (UML) and discusses the ability of this model to serve as a psychological basis for design of a problem-independent dialogue agent. One of the goals of the article is to demonstrate the applicability of modern object-oriented technology to the area of Cognitive Psychology.
Cognitive Science in general and Cognitive Psychology in particular is one of the most significant and growing areas of research at the moment. For instance, the bibliography of Eysenck and Keane's textbook on Cognitive Psychology [2] includes more than 1300 references. Models developed by cognitive psychologists embrace all aspects of human mental activity, from perception, to decision-making processes and models of emotional states. However, from a pragmatic viewpoint, the models offered by cognitive psychologists seldom translate into computer simulations and so these models quite are often practically unknown to researchers working in the area of the theory and practice of artificial intelligence.
Today, the depth and universality of the concepts of object-oriented computer programming systems and object-oriented modelling is so prevalent that we can consider them a general theory for all complex natural and artificial systems [3] .
Such modeling is becoming practicable with the advent of the Unified Modelling Language, which has acquired the features of a strict and formalized theory based on the conception of object-oriented modelling. The attractiveness of UML is in its diagrammatical way of formal descriptions of systems and, therefore, its 'formulas' are various types of diagrams, which depict various aspects of the system: its structure, its behaviour, etc. We believe that the expressive power of UML is enough to build sophisticated 'formulas of mind' describing the structure and behaviour of psychological phenomenon. This brief article does not allow us to include either speculations regarding peculiarities of the underlying cognitive models nor the basic concepts of UML. Therefore, to better understand the basic concepts and definitions of these subjects, we refer readers to other books [4,5] .

FORMAL REPRESENTATION OF NEISSER'S CYCLE OF PERCEPTION
The model of perception offered by Ulric Neisser in 1976 is a well-known model in Cognitive Psychology. The model integrates 'bottom-up' (from sensory system to the long-term memory) and 'topdown' (from long-term memory to the motor system) processes into one cyclically repeated process. According to this model, the main cognitive structure, which determines such processes as perception, attention and categorization, is a set of anticipatory schemata, within a given cycle/step of perception. We would like to note that we are using the term schema in its meaning from Cognitive Psychology, namely as a fragment of knowledge. Simple classification of schemata divides the set of schemata as frames and scripts [6] .
The set of anticipatory schemata prepares the mind for perception of the subsequent sensory events from the environment and can be considered as a control structure for the processes of perception, attention and categorization. Thus, a perception is a constructive process because at every step of perception the consciousness forms a new set of anticipatory schemata.
Generating the new set of anticipatory schemata initiates the process of perceptual exploration of the environment, directing a search for a sensory event corresponding to one of the schemata from the set of anticipatory schemata. We use the term sensory event to denote a fragment of the environment, which is unambiguously categorized as one of the schemata. Several sensory modalities form a sensory event, usually vision and hearing. Hence, schemata forming the anticipatory set, do not represent visual or auditory stimuli, but rather a certain integration of multimodal sensory information. People focus their attention on objects and events, but not on sensory inputs.
The purpose of perceptual exploration is a search for the sensory event relevant to a schema from the set of anticipatory schemata. This, as a rule, entails motor reactions, such as moving of head, extremities or the whole body. A good example of perceptual exploration is the process of palpation of a coin during its tactile perception. By means of perceptual exploration the correspondence between a sensory event and one schemata is established. It is clear that the process of perceptual exploration should include the process of categorizing sensory events. The sensory system sequentially focuses on sensory events, which it then categorizes and compares with schemata from the anticipatory set. Perceptual exploration is complete when a chosen sensory event is perceived. Being perceived, a sensory event becomes the trigger to change the current set of anticipatory schemata into a new one. Usually the new set of anticipatory schemata is similar to the previous set and is formed by its modification.
The set of anticipatory schemata is formed from schemata stored in a long-term memory (i.e. previous perceptual experiences) and is a part of a cognitive structure named by Tolman [7] a cognitive map. Thus, we can say that according to Neisser's model, the process of perception is determined by two factors: the current state of the environment and previous perceptual experience.
From this brief description of Neisser's model of perception it follows that the spatial structure of the perceptual system can be represented by at least the following classes of objects: * AnticipatorySchemata -The class of sets of anticipatory schemata. At every perceptual step only one object from this class is 'working'; * CognitiveMap -The class of cognitive maps; * SensoryEvent -The class of sensory events in the environment. An object of this class is a concrete sensory event, which was initially encoded in the sensory system and then recognized.
We will not consider a cognitive map as a simple set of schemata but rather as a script of perception represented by sets of anticipatory schemata and relationships between them. Therefore, instead of the class CognitiveMap we will consider the class PerceptualScript and will treat objects from this class as control structures, which control the transition from the current set of anticipatory schemata to the subsequent set of anticipatory schemata. An objectscript receives an identifier of a perceived sensory event and an identifier of the current set of anticipatory schemata and returns the identifier of the subsequent set of anticipatory schemata.
We have to distinguish the process of categorizig the current sensory event from the process of its identification with one of the schemata, from the set of anticipatory schemata. The task of categorization presupposes that for a certain object-represented by its properties (attributes) and behaviour (operations) -it is necessary to define the class to which this object belongs. In the simplest case categorization can be accomplished only on the basis of object's attributes.
Let the operation recognition(out eventType) be responsible for the process of recognizing sensory events. This operation returns the value of the attribute eventType. Let the class SensoryEvent be a container for the operation recognition and the attribute eventType.
In the process of identification, the type of categorized sensory event is sequentially compared to schemata from the anticipatory set of schemata. Identification is complete when the type of sensory event is equal to one of the anticipatory schemata. Figure 1 depicts a UML class diagram modelling the spatial structure of Neisser's cycle of perception in accordance with speculations given above.
The spatial structure of Neisers's model of perception can be considered from two points of view: (1) from the point of view of its physical components (visual and auditory sensors, sensory buffer, sensory system, etc.); and (2) from the point of view of its information components (sensory event, array of sensory events, categorized event, schema, etc.). The model in Fig. 1 depicts the second point of view because as it seems to authors this point of view is closer the goal of a program simulation of the model.
The cyclical nature of Neisser's model is expressed in Fig. 1 by the directions of navigation of associations between the classes AnticipatorySchemata and SensoryEvent. The class AnticipatorySchemata models a fragment of a perceptual script, which controls the process of There are two associations, named identification and renovation, between the classes SensoryEvent and AnticipatorySchemata. The relationship identification models the fact that during the process of perception the current sensory event is identifies with one of the schema from the set of anticipatory schemata; and the relationship renovation models the fact that after the process of identification the current set of anticipatory schemata is renewed. In the relationship identification, objects of the class SensoryEvent play the role of a current event, whereas objects of the class AnticipatorySchemata play the role of a pattern. Expression 1..1 means that this relationship permits the link of only one object from class AnticipatorySchemata (one set of anticipatory schemata) with one object from class SensoryEvent (one sensory event).
The relationship aggregation between the classes AnticipatorySchemata and PerceptualScript, models the fact that the object of the class AnticipatorySchemata is a part of the object of class PerceptualScript.
The subclasses Visual and Sound model the multimodal nature of a sensory event, which includes visual and sound components. This set of subclasses is incomplete because a sensory event can include other components, for example an olfactory component.
The formal representation of the structure of Neisser's cycle of perception in the form of UML class diagram -depicted in Fig. 1 -does not take into account some peculiarities and details of its original description [1] . Simple introspection allows us to conclude that one sensory event must correspond to several sets of anticipatory schemata from different perceptual scripts. For example, if we observe somebody's a smile then depending on the context we expect to perceive: (a) a shape of teeth; (b) an articulation of lips; (c) relation to a certain event (a smile can be sardonic, polite, or offensive), etc.
Apparently our consciousness uses several perceptual scripts, which differ from each other by the goal of the perception. Therefore, the model must be complicated and take into consideration such things as: a set of goals, a set of perceptual scripts (linked with the set of goals) and means for selection one of the perceptual scripts. We can model the goal-oriented character of the perceptual script by an attribute describing the goal of a perception in the class PerceptualScript.
Neisser's theory draws upon a highly conditional boundary between the environment and its reflection in the consciousness of a human. The majority of people share the viewpoint that perception of the environment does not change the environment but that perception does change the subject of perception. An object of class AnticipatorySchemata, which works on a certain cycle of perception, is a "product" of current cycle of perception, but at the same time it is a product of all previous cycles in which this object was used. From cycle to cycle a schema evolves and this is called a schema accommodation [8] by Piaget in his theory of cognitive development. Since a schema is an element of a more complex cognitive structure, namely a script , it is clear that the process of accommodation is inherent in the script. In some sense schemata and perceptual scripts 'keep' the history of the development of a subject's consciousness. Therefore, the model must be complicated and take into consideration the possibility of accommodation of schemata and perceptual scripts. We can account the accommodation features of schema and script by including the operations schemaAccomodation() and scriptAccomodation() into corresponding classes. The problem of modelling the accommodation of schemata and scripts is very close to the problem of differentiation of schemata and scripts on: (1) innate/genetic and (2) acquired in the process of development of the organism. In other words, we can say that our model must account for the typology of classes AnticipatorySchemata and PerceptualScript. Figure 2 depicts the structure of Neisser's cycle of perception, which takes into account some details of its original description.
In the model depicted in Fig. 2, the typology of classes AnticipatorySchemata and PerceptualScript is modelled by the relationship generalization. Each of these classes has the status of a super-class and falls into two subclasses: Native (innate schemata/script) and Acquired (acquired schemata/script). The set of subclasses characterized as a complete set, which means that among perceptual scripts there can be only innate or acquired schemata and no other.
We added some attributes and operations into classes AnticipatorySchemata and PerceptualScript to specify their properties and behaviour. Attributes typeOfSchema and typeOfScript characterize types of schema and scripts respectively and can take values native or acquired. The operations schemaAccomodation() and scriptAccomodation() model the ability of accommodation for schemata and scripts. In the class PerceptualScript we added the attribute goalOfPerception and the corresponding operation setGoalOfPerception, which model the goal-oriented nature of the script.

MODELLING OF THE DIALOGUE PROCESS
The dialogue process in relation to either of the dialogue agents is similar to the process of routine perception of the environment and therefore it is worthwhile to investigate the applicability of Neisser's model of perception to the area of modelling of problem-independent dialogue processes. Besides similarity, there is a distinction between the dialogue process and the process of perception: during the dialogue, a sensory system is not connected to the 'natural environment' but to the artificial environment formed by flows of visual and auditory stimuli generated by the opposite dialogue agent. Thus, in the dialogue process a real environment is substituted by an artificial one represented by a perceptual script of the dialogue agent. However, it is clear that the perception and subsequent processing of sensory events generated by an artificial environment and by the natural environment are carried out by the same psychological 'laws and rules'. Figure 3 depicts a model of the dialogue system formed by two interacting dialogue agents.
Objects of the class SensoryEvent represent the environment and are generated in turn by both dialogue agents. Apparently, for a given step of perception and in relation to the process of generating objects of the class SensoryEvent, an agent can have one of two different statuses. Let us call an agent, which is generating an object of the class SensoryEvent an active agent and an agent, which perceives this object a reactive agent. Thus, the middle part of the diagram in Fig. 3 (between two dashed lines) 'belongs' to an active agent. In the process of routine perception of a human, by means of motor reactions, affects on the environment with the aim to perceive a desired sensory event on the subsequent step of perception. In the process of dialogue interaction, an active agent acts upon the reactive agent with the aim of perceiving a desired portion of sensory information on the subsequent step of the dialogue. It is clear that such influence of an active agent upon the reactive can be accomplished only by generating a relevant sensory event.
If the role of an active agent of the dialogue is played by a program agent, then its capabilities generate sensory events dependent on and constrained by computer architecture. Modern computers with standard architectures have relatively well developed capabilities for the presentation of visual and audio information, but usually do not have equipment for generating information oriented towards other the sensory organs of a human. Figure 4 depicts a structure of the sensory event generated by an active program dialogue agent, which uses the standard computer peripheral equipment.
The class diagram in Fig. 4 models an object data base for a computer simulator of an active dialogue agent. Since the term sensory event as a rule is not used in descriptions of program systems, we have substituted for this term the semantically equivalent term scene. Class S ene models a set of scenes needed for building perceptual scripts by an active agent. Each scene consists of ordered set of scene's objects. We model this by means of the relationship aggregation between the class-aggregate S ene and the class-constituent SceneObj. The ordering of a scene's objects means that they come out of the scene sequentially and in the case when the tempo of its appearance from the scene is low, a reactive agent can perceive them as separated sensory events. The class of scene objects falls into two subclasses VideoObj and AudioObj, which form a complete set of subclasses and which are in analogy with classes Visual and Sound in Fig. 3.
Each object of the scene can be described by two groups of descriptors: attributive descriptors and nonattributive descriptors. Attributive descriptors are the classical object's attributes. For instance, if a videoobject is represented by a rectangular window, which can appear at any place on the screen, then the list of its attributive descriptors can include the window's location on the screen, width and height. As an example of an attributive descriptor in the case of an audioobject, which represents, for instance, human's speech, we can consider an attribute giving the speech the modality of male or female voice.
Non-attributive descriptors are information, which are hard or even impossible to break down into a set of properties. An example of non-attributive descriptor is a realistic picture of a landscape in the case of videoobject.
The diagram in Fig. 4 presupposes that an object from class VideoObj consists of objects from the class Attributive and the class NotAttributive. We model this fact by the relationship composition in which the class VideoObj is a composite and classes Attributive and NotAttributive are components.

DISCUSSION
This article utilizes only one type of UML diagram, namely class diagrams, which models the spatial structure of a system and our efforts, were focused on obtaining the structure of Neisser's cycle of perception. However, modelling the behaviour of a system is also very important. In the process of solving the problem of modelling the dynamic nature of Neisser's cycle, it is worthwhile to start from the investigation of two cases: (1) the discrete or step-by-step cycle of perception; and (2) the continuous cycle of perception.
Neisser's cycle is a good model for the process of routine perception, i.e. the process of successive focusing of a sensory system on external and nonsuspicious sensory events. Perception of suspicious events obviously requires certain modifications of the model. For instance, as the processing of suspicious sensory event must be accomplished during a short period of time, the detection of suspicious stimulus can interrupt the process of routine perception at any stage.
In summary the authors would like to note that the diagrams given in the article were created with the general aim of producing 'diagrammatical formulas' effective first for teaching and only secondly with the aim of specifying a computer program simulator. Therefore, we included in these diagrams only those attributes and operations, which we have discovered in the original description of the model, or which directly follow from the original description [1] .