Representations of Reality Enable Control – Part 2

Part 1 looked at language and thought, mental models and computational approaches to how the mind represents what it knows about the world (and itself). Part 2 contrasts thinking in words with thinking in pictures, looking first at how evidence from brain studies inform the debate, and then concludes how all these approaches – linguistic, psychological, computational, neurophysiological and phenomenological are addressing much the same set of phenomena from different perspectives. Can freedom be defined in terms of our ability to reflect on our own perceptions and thoughts?

The Flexibility of Thought

Although we often seek order, certainty and clarity, and think that the world can be put in neat conceptual boxes, nothing could be further from the truth. Our thoughts and our language are full of ambiguity, flexibility and room for interpretation. And this is of great benefit. Just like a building or a bridge that cannot flex will be brittle and break, our thinking (and our social interaction) is made less vulnerable and more robust by the flexibility of language and thought.

Wittgenstein realised that categories do not really exist in any absolute sense. A particular concept, such as ‘furniture’, does not have necessary and sufficient defining features so that we can say definitively that any one object, say a piano or a picture, is furniture or not. Rather pieces of furniture have a ‘family resemblance’ that makes them similar, but without any hard boundaries on what is inside or outside the category. Steven Pinker describes a man who was unable to categorise but nevertheless had amazing feats of memory.

YouTube Video, Professor Steven Pinker – Concepts & Reasoning, NCHumanities, October 2014, 1:10:40 hours

Pinker also considers reasoning – both deductive and inductive. Deductive reasoning is where a conclusion necessarily follows from a set of premises or assumptions – all men are mortal and Socrates is a man, leads inevitably to the conclusion that Socrates is mortal. Inductive reasoning is where we generalise from the particular – so we encounter five white swans and this leads us to the generalisation that ‘all swans are white’ even though this may not necessarily follow. He concludes that people can do deductive reasoning so long as they are dealing with concrete and familiar content, but easily go awry when the content is abstract. As for inductive reasoning, people are generally not very good, and thinking is subject to all manner of biases (as described by Kahneman).


Representation of Concepts in the Brain

Since technology has become available to scan brain activity, there has been a spate of studies that look at what is happening in the brain as people perform various mental tasks.

TED Video, Nancy Kanwisher: A neural portrait of the human mind,TED , March 2014, 17:40 minutes

Control Systems in the Brain

As well as looking at individual functional components it is possible to identify some of the gross anatomical parts of the brain with different forms of control.

http://totalbraintotalmind.co.uk/architecture

  • Cerebrum – Control mediated through conscious abstract thought and reflection
  • Cerebellum – Learned control and un/sub-consious processes
  • Brain stem – Innate level control

These ideas and a more fully elaborated nine-level brain architecture can be found in a free downloadable ebook available from:

http://totalbraintotalmind.co.uk


For more on the imaging techniques see:

YouTube Video, Magnetic Resonance Imaging Explained,ominhs, October 2011, 5:30 minutes

If you want to find out more about magnetic imaging techniques then there are several videos in the following Youtube playlist:

Using Functional Nuclear Magnetic Imaging (FNMI) techniques on people as they look at pictures of different objects (faces, road signs etc.) reveals not only something about object recognition in the brain’s visual system but also says something about how we may form categories and concepts. Interestingly, it appears to validate the more armchair philosophical speculations about the ‘fuzziness’ of concepts (e.g. Wittgenstein’s notion of ‘family resemblance’). For example, in his research, Nikolaus Kriegeskorte investigates patterns of neural activity in humans and monkeys. The neural activity suggests conceptual clusters such as animate ‘bodies’ (e.g. a human or animal body) and inanimate objects, despite visual similarities between the members in each group. If we consider the complexity of these patterns of activity and the way in which the patterns overlap, it is possible to see how concepts can, at the one time, be both ‘fuzzy’ (i.e. have no necessary and defining features) and yet distinct (i.e. given separate linguistic labels such as animate or inanimate).

TSN Video, Representational similarity analysis of inferior temporal object population codes – Nikolaus Kriegeskorte, The Science Network, August 2010, 23:11 minutes

In fact, brain and cognitive scientists have made considerable progress in bridging between our understanding of brain activity and more symbolic representation in language.

TSN Video, Emergence of Semantic Structure from Experience – James McClelland, The Science Network, August 2010,1:16 hours


The eventual direction of this type of work will be to integrate what we know about the brain into a simulation of how it works.

https://www.humanbrainproject.eu/brain-simulation-platform


Goals, Tasks and Mental Representation

Whilst both language and patterns of neural activity can be considered as mental representation, somehow neither really capture the level of representation that we intuitively feel underlie the performance of tasks and the ‘navigation’ towards goals.

When people perform tasks they have a model in their mind that guides their behaviour. To illustrate this, imagine going from one room to another in your house at night with the lights turned off. In your mind’s eye you have a mental map of the layout of the house and you use this to help guide you.

As you stumble about in the dark you will come across walls, pictures, doorways, stairways, shelves, tables and so on. Each of these will help reinforce your mental image and help validate your hypotheses about where you are. If you come across objects you do not recognise you will start to suspect your model. Any inconsistencies between your model and your experience will cause tension and a search for a way of reconciling the two, either by changing your model or by re-interpreting your experience.

It is often the case that mental representations are vague and fragmentary, needing reinforcement from the environment to maintain and validate them. Even so, conceptual models create expectations which guide the interpretation of experience and tension is created when the internal representation and the external experience are out of step.

In this example, by turning out the lights, we remove a major element of external feedback from the environment. All that is left is the conceptual or mental model supported by far less informative sensory mechanisms. Because you know your house well, the mental model acts as a strong source of information to guide behaviour. Even if you are in a strange house, your knowledge about how houses are typically designed and furnished will provide considerable guidance.

Now consider an example where there is still a strong mental model that drives task performance, except it is less obvious because it does not involve the disabling of any sensory feedback from the environment.

Imagine performing the task of putting photographs in a physical album. You are driven by a view of what the finished product will look like. You may imagine the photographs organised by date, by place, or by who or what is shown in them. Alternatively, you may organise the album to tell a story, to be a random collection of pictures in no particular order, or to have all the better shots at the front and the worse ones at the back. Perhaps you have some constraints on the photos you must include or leave out. All these factors and visualisations form the conceptual model that stands behind the performance of the task. The activity of conceptual modelling is to capture this ‘mind’s eye’ view.

The mental model is not the task itself. The task of putting photographs in the album might be done in many different ways. For example, the behaviour would be quite different if the album were on a computer, involving mouse clicks and key presses rather than physical manipulation of the photographs. The task behaviour would also be different if you were instructing somebody else to put the photographs in the album for you.

The model is the internal mental representation that guides the task behaviour. It can be seen to be different from the behaviour itself, because the behaviour can be changed while keeping the model the same. If instructing somebody to put photographs in the album a particular way is not working effectively, you can take over the job yourself. You have the same image of the end product even though you achieve it in a different way.

A mental model need not necessarily be a goal. The model of the house was simply a representation that allows many different tasks to be performed and many different goals to be achieved. The goal may be to get out of the house, to get to the fuse box, or to check that somebody else in the house is safe. The same mental representation may support the achievement of many different goals.


Imagination, Envisioning and Visualisation

From the above it will be clear that although the mind can respond in an immediate and simple way to what is going on around it, for example by pulling back a hand when it touches something hot, it is also capable of sophisticated modelling of what might happen in the future. This is imagination or envisioning.

http://plato.stanford.edu/entries/aristotle-psychology/suppl4.html

Francis Galton in 1880 published a classic paper in the journal Mind called the Statistics of Mental Imagery in which he set out some of the main characteristics of the ‘mind’s eye‘, in particular how people vary in the vividness of their mental images.

http://psychclassics.yorku.ca/Galton/imagery.htm

Jean Paul Sartre in ‘The Psychology of Imagination’ distinguishes between perception, conceptiual thinking and imagination.

The following playlist from the Open University looks at imagination and envisioning from perspectives from art through to neurophysiology.

https://www.youtube.com/playlist?list=PLBFE8D91E196C83B5

Stephen Kosslyn has been researching mental imagery since the 1970’s and argues that people have and can inspect internal mental images when performing tasks. They form a model or representation of reality in addition to propositional representations.

Youtube Video, 12. The Imagery Debate: The Role of the Brain, MIT OpenCourseWare, August 2012, 55:11 minutes (Embedded under policy of Fair Use)

However, the psychology of imagination is somewhat out of fashion at the moment as neurological approaches come to the fore. But talking about the mind in terms of mentalistic concepts like imagination is under-exploited, both as a means of understanding mental representation and as a therapeutic tool.

Youtube Video, Interview Ben Furman 2 – Imagination in modern psychology, MentalesStärken, October 2014, 6:43 minutes


Phenomenology

One approach to understanding how we think is phenomenology (Edmund Husserl). This focuses on subjective experience. It is looking inside our own heads rather than trying to construct an objective and theoretical account. Philosophers (Heidegger, Jean Paul Sartre, Simon de Beauvoir) and psychologists (Amedeo Giorgi) have taken this approach. The focus of phenomenology is on being, existence, consciousness, meaning, and intuition. This, in some sense, comes before the great philosophical questions like what is truth and why are we here. It is the sheer realisation that we exist at all and concerns fundamental ideas like the nature of the self and the relationship of self to reality – what we perceive and how we interpret it, before we start to analyse it, put linguistic labels on it or think about it in any logical sense.

BBC Radio 4, In Our Time, Phenomenology, January 2015, 43 Minutes
http://www.bbc.co.uk/programmes/b04ykk4m

An idea that comes out of phenomenology is the notion of the gap between what we perceive and our reflections on our perceptions. So, we see a glass of water, but the content of our thought can be about our perception of the glass of water as well as the perception itself. That we can reflect upon what we are seeing is well and simply just seeing it. So much is obvious. Indeed when I ask you to pass the glass of water I am making a reference to my perception of it and the assumption that you can perceive it too. If I ask, “where is the glass of water?” I am making a reference to a belief that the glass of water exists for both you and me even though I am unable to perceive it.

The interesting idea is that the notion of freedom derives from this ability to not just perceive but to be able to reflect on the perception. This removes us from responding to the world in a purely mechanical way. Instead, there are intermediary states that we can consult when making decisions.

It turns out that what the phenomenologists referred to as the gap between perception and reflection, the psychoanalysts have referred to as the distinction between the id, ego and super-ego, the psychologists have developed into the notion of mental models, Kahneman refers to as system 1 and system 2 thinking, linguists think of in terms of semantic structure, and the neurophysiologists have identified as being associated with higher layers of the brain such as the cortex, are all pretty much the same thing!

Mind the Gap

How the mind represents reality can be described at different levels from patterns of neural activity through to mentalistic concepts like imagination.

In reading the following very general and abstract account of mental processes, it is useful to think of an example, like driving a car. For an experienced driver it is almost automatic and requires little conscious thought or effort (until a child unexpectedly runs into the road). For a new driver it is a whole series of problems to be solved.

We can think of a person experiencing the world as a sensory ‘device’ attuned to monitoring our state of internal need and the gap between expectations and experience (our orientation). If all our needs are met, by default we coast along on automatic pilot simply monitoring the environment and noting any differences with our expectations (maintaining orientation). Expectations tune our sensory inputs and the inputs themselves activate neural pathways and may elicit or pre-dispose to certain outputs (behaviours or changes to internal states). Where we have needs, but know how to satisfy them (i.e. we have mastery), we engage appropriate solutions without effort or thought. The outputs can be behaviours that act on the world or changes to internal states (e.g. the states in our internal models). Some circumstances (either internal or external) may trigger a higher level control mechanism to over-ride default responses. When needs are met and experience and expectation are more or less aligned, our autonomic and well-learned responses flow easily. This, in Kahneman’s terms is relatively effort free, automatic and more or less subconscious, system 1 thinking.

Dissonance occurs when there is an unmet need or a difference between expectation and experience e.g. when there is a need to deal with something novel or some internal state is triggered to activate some higher level control mechanism (e.g. to inhibit an otherwise automatic reaction). If sufficient mental resources are available the mind is triggered to construct a propositional, linguistic or quasi-spatial/temporal representation that can then be internally inspected or consulted by the ‘mind’s eye’ in order to envisage future states and simulate the consequence of different outputs/behaviours before making a decision about the output (e.g. whether to act on the outside world or an internal state, and if so how). This is what Kahneman refers to as system 2 thinking. When we have done some system 2 thinking we sometimes go over it and consolidate it in our minds. These are the stories we construct to explain how we met a need or managed the difference between expectation and experience. The stories can then act as a shortcut to retrieving the solution in similar circumstances.

In a very simple system there is a direct mapping between input and output – flick the switch and the light comes on. In a highly complex system like the human brain the mapping between input and output can be of extra-ordinary complexity. At its more complex, an input might trigger an internal state that creates an ‘on the fly’ (imaginary) model of the world which is then used to mentally ‘test’ different possible response scenarios before deciding which response, if any, to make.

As we experience the world (through learning and maturation) we adjust our expectations in line with our experience. Our brains and and expectations become a progressively more refined model of our experience. When we are ‘surprised’, and recruit system 2 problem-solving thinking, we produce solutions. Solutions are outputs – either behaviours that act on the world or changes to internal states. Problem solving takes effort and resource but results in solutions that can potentially be re-used in similar circumstances in the future. This type of learning is going on at all levels of experience from the development of sensory-moror skills like walking or driving a car through to high level cognitive skills such as making difficult decisions and judgements in situations of uncertainty (e.g. a surgeon’s decision to operate on a life-threatening condition). System 1 and system 2 thinking are really just extremes of a spectrum. In practice, any task involves thousands of separate sub-processes some of which are highly learned and automatic and some of which require a degree of problem solving. To an outside observer these processes often appear to mesh seamlessly together.

The learning we do and the models we construct in our minds are very dependent on our own experiences of the world (and this accounts for many of the biases in the way we think). Although our models can be influenced by other people’s stories about how the world works e.g. though our education, peers, family, media etc. (or observing what happens to others), the deepest learning takes place through our own direct experience, and because our experiences are all just different samples of a larger reality, we are all different from each other. Each one of us has merely sampled an infinitely small fraction of an omniscient reality but because of the consistencies in the underlying reality (for example, we all experience the same laws of physics) there are sufficient commonalities in our models that we understand each other to a greater or lesser extent.

Need and maintaining orientation drive us all, and when under normal (but not total) control, we have wellbeing. However, we must always have some manageable gap, so that the system is at least ticking over. This is easily achieved because as lower level needs are satisfied we can always move to others further up the hierarchy, and constant change in the world is usually enough to drive the maintenance of our orientation.


Radio programme links

An index of BBC Radio programmes on cognitive science can be found at:
http://www.bbc.co.uk/programmes/topics/Cognitive_science

An index of BBC Radio programmes on Mental processes can be found at:
http://www.bbc.co.uk/programmes/topics/Mental_processes


This Blog Post: ‘Representations of Reality Enable Control’ shows how different levels of description can be used to represent the knowledge that enables us to meet our needs and deal with the unexpected.

Next Up: ‘Are we free?’ delves deeper into freewill, consciousness and moral responsibility. If we are free, then in what sense is this true?

Share Button

Rod Rivers' passions include writing about economics, psychology, and philosophy; listening to Radio 4 and watching TED and YouTube videos; engaging in conversations with friends and colleagues, and re-experiencing the world through the eyes of his two teenage sons. Living in the 21st century is a huge privilege.

Posted in Control, Mind, Uncategorized Tagged with: ,

Leave a Reply

About This Blog

This series of blog postings takes a multi-disciplinary approach to social policy, bringing together ideas from psychology, economics, neuroscience, philosophy and related subjects to inform policy makers and other professionals about how we might think in new ways about the individual and society . There are some easy ways to read it:

• Very Easy – Just read the blog titles: Most blog title are propositions that the blog content attempts to justify. Just reading the names of the blogs in order from first to last will provide an overview of the approach.

• Quite Easy - Just read the text in bold. This brings out the main points in each posting.

• Easy - Just watch the videos. This is easy but can take a while. The running time of each video can be seen in the caption above it. Hover over the video to see the controls – play and pause, large screen, and navigate around.

• Harder – Read the whole blog. Useful if you are really interested, want to learn, or want to comment, disagree with the content, have another angle or whatever. The blog is not being publicised yet but please feel free to comment and I will try to respond if and when I can.

The blog attempts not to be a set of platitudes about what you should do to be happy. In fact, I would like to distance myself from the ‘wellbeing marketplace’ and all those websites/blogs that try and either sell you something or proffer advice. This is something quite different. It takes as its premise that there is a relationship between wellbeing, needs and control in both the individual and society. If needs are not being met and you have no control to alter the situation, then wellbeing will suffer.

While this may seem obvious, there is something to be gained by understanding the implications of this simple idea. We are quite used to thinking about wellbeing in terms of specifics like money, health, relationships, work and so on, but less familiar with dealing with the more generic and abstract concepts of need and control.

Taking a more abstract approach helps filter out much of the distraction and noise of our usual perceptions. It focuses on the central issues and their applicability across many specifics that affect how we think and feel.

The blog often questions our current models of the way we think about the human condition and society. It looks at the things we all know and talk about – decisions and choices, relationships and loss, jobs and taxes, wealth and health but in a way in which they are not usually described. It tries to develop a new account, that draws on a broadly based understanding of what we now know from science, culture and common sense.

If you are looking for simple answers you will not find them here. This is not because the answers are complex. It is because the answers are not necessarily what you expect.

If you are looking to explore in some depth the nature of wellbeing and how it is influenced by what you can control, and what others can control that may affect you, then read on. Playing through some of these ideas into the specifics of policy, at the level of society and the individual, will take time but I hope you will see the virtue of working from first principles.

When walking through any landscape different people will see different things. A geologist might see an ice-age come and go, forming undulations in its wake. A politician might see territorial boundaries. Somebody else may see a hill they have to climb together with the weight of their back-pack.

Taking a perspective of wellbeing and control is different from how we normally look at the world. It's a deeper look at why and how things happen as they do and the consequences on wellbeing. It questions the relationship between intention and outcome.

We normally see and act through the well-worn habits of our thoughts and behaviours as they have evolved to deal with things as they are now. We mainly chose the easy options that require the least resource. As a survival strategy this generally works well, but it also entrenches patterns of thought, behaviour and emotion that sometimes, for the benefit of our wellbeing, need to be changed. When considering change, people often say ‘well, I wouldn’t start from here’. And that’s the position I take. I am not starting from the ways things are or have evolved, but from the place they might have been had we known what we know now and had designed them.

The blogs argue that, in an era of specialisation, we have forgotten the big picture – we act specifically and locally within the silos of our specialised education and experience. We check process rather than outcomes. We often fail to integrate our knowledge and apply it to the design of our social and work systems (as well as our own thoughts and behaviours).

To understand society we first need to understand the individual and to this end, a psychological account of how we feel, think and behave based on notions of wellbeing and control is proposed. And not in an abstract airy-fairy kind of way, but as a more or less precise theory that forms the basis of a predictive and testable computational model. The theory is essentially about how, both as individuals and society we manage multiple (and often conflicting) intentions in real time within limited resources. I call this model 'the human operating system'. This is like a computer operating system except that it is motivated by emotions, modulated by reason and is expressed in the language of mind and its qualities of agency and intentionality.

Just as in the mathematics of fractal geometry, complex structures can emerge from simple rules. The explanation given of the interplay between emotions, physical bodily states, thoughts and behaviours shows how much of the complexity in the individual can be accounted for by a set of relatively simple rules. This can be modelled using a system of symbolic representation and manipulation involving intentions and priorities operating in a complicated and changing environment.

The language and models that we use to understand the individual can also be applied to organisations and other structures in society. Through an understanding of what makes for wellbeing in the individual we can also understand what makes for better wellbeing in society generally. The focus, therefore, is on understanding the individual and then using that understanding to inform how we might think about other structures in society and how all these structures relate to each other from the point of view of wellbeing, shifting patterns of control and the implications for social policy.