Is the Referendum Vote Definitive?
I have heard several politicians say that UK citizens cast a definitive vote in favour of leaving the EU. But this position is easily challenged. Firstly, there were 48.1% who wanted to remain. I would call that a narrow margin rather than a definitive vote. Secondly, it is only based on the turnout, meaning that the vote for leaving was only 51.9% of 72.2% (Ref: 2). That means the decision was made by only about 37.5% of the electorate. And, lastly, and most importantly for the argument I set out here, the electorate is subject to ‘cognitive bias’ in much the same way as individuals are, when making decisions.
Research that has looked at how citizens discuss political issues in online forums reveals some interesting findings (Ref: 3). We do not tend to seek out or bring to bare evidence on issues in anything resembling a scientific way and we tend to go off at tangents about personal issues and hobby-horses. You only have to go to any online forum, or to listen to any pub argument to realise that most positions are driven by emotion, personal experience, alignment with a favoured in-group, inaccurate estimates of baseline probabilities etc. rather than any in-depth analysis or structured process working from hypothesis, to evidence gathering, to testing and validation and on to conclusion.
I am not saying that the democratic process is wrong, or emotion and personal experience should not drive the process. How political decisions are made is dependent on a complex set of value judgments we hold with respect to democracy, equality, fairness, freedom of speech and so on. These values tend to over-ride those of science or more formal (normative) models of decision-making. And so they should, because much of what claims to be scientific and rationale is equally value-laden.
What I am saying is that, as with all complex decisions, there are dangers in thinking of them as ‘black or white’ – ‘leave or remain’, having a system in which they are made as ‘on-offs’, instead of an iterative process where there is an opportunity to see what other people think and consider alternative positions before a final decision.
Last month I made an application to do a Phd sponsored by an organisation called the Behavioural Insight Team. This is sometimes called the ‘nudge unit’. It was set up by Cameron and spun out from the cabinet office in 2010. There were over 90 competing applications and mine did not succeed but, addressing as it does, the issue of citizens’ participation in policy and political decision making, it seems even more pertinent in the light of a referendum result which seems to have surprised everybody, including its advocates. See the outline proposal in the next section below.
References
REF 3 – John, P. 2011. Taking Political Engagement Online: An Experimental Analysis of Asynchronous Discussion Forums.
Can citizens collaboratively formulate coherent policy with appropriate types and levels of support and structure?
Background:
There have been recent initiatives to improve policy-making (REF1) and involve citizens more in policy formulation. One objective of the Open Government Partnership (OPG) is that the public ‘can influence the workings of their government and society by engaging with policy processes’ (‘participation’) (REF2). However, John (2011) found problems with participation using asynchronous online discussion forums (REF3). John (2011) identified two problems (amongst others) with online asynchronous participation:
- (1) A lack of engagement with information salient to the policy issues
- (2) a skewed distribution of comments amongst participants. In this study
(1) is addressed by using facilitation, support, guidance and tools and (2) by creating specific roles and tasks for participants to take on.
Rationale:
Better understanding of how to involve citizens in policy is more democratic, may lead to policy that takes into account more factors, better bridges between the intentions and consequences of policy, better anticipates unintended consequences, and better addresses diversity. This study may indicate direction.
Hypothesis:
The study is designed to test the hypothesis that ‘a randomly selected set of citizens, using appropriate types and levels of facilitation, support, guidance and tools, and given appropriately defined tasks and roles, could collaboratively formulate policy recommendations that would provide useful input to government policy making’. ‘Usefulness’ is the main dependent variable (V5). Part of the study would be to identify and assess different criteria for what constitutes ‘useful’.
Design:
In total 5 variables are the subject of study. V1 – level of support, V2 – level of structure/definition, V3 – policy type, V4 – dropout rates/characteristics, V5 – the usefulness of the policy recommendations. V1, V2 and V3 are experimentally manipulated (independent variables) while V4 and V5 are dependent variables.
The study uses randomly controlled trials to investigate the 2 main independent variables: (V1) the type/degree of facilitation, support, guidance and tools provided’ and (V2) the effect of providing more structure, better defined tasks and roles for participants. V1 concerns the type/level of support available, while V2 concerns the type/degree of self-organisation expected vs prescriptive structure provided. So, in condition V1a/V2a there would be no support to perform a loosely structured task, while in condition V1d/V2d a lot of support is given to perform a highly defined task using a prescribed method.
Experimental conditions to address V1(support) would range in 4 conditions from ‘almost none’ to ‘highly facilitated, guided and supported’. V2(structure/definition) would include 4 conditions ranging from minimal structure, loose task definition and no roles through to highly structured, with well defined tasks and roles. All combinations of conditions would be included to determine interaction effects. In order to assess any interaction with policy area and learning/practice effects, 4 different policy questions (V3 – Policy) would be used that attempted to sample from a space of policy dimensions.
The experimental design is therefore a 4 x 4 x 4 matrix of ‘type/degree of support’ by ‘type/degree of definition’ by ‘policy question’. The study would invite 16 random samples of 64 people (to populate 4 support levels x 4 definition levels) on the UK electoral register (giving a total study size of 16 x 64 = 1,024), the opportunity to work together in their respective groups, initially online. Each group would be asked to each design 2 sets of policy recommendations. Half of each group (i.e. 32 people) would tackle 2 out of 4 policy questions and the other half the other 2. Within each half, the question order would be reversed to counterbalance for order effects.
Detailed Design:
All groups would be asked to perform the same task with the same output. The task is to consider the policy questions and to set out the group’s top five policy recommendations together with their rationales. V1a receives no support. V1b receives training in collaborative working and the online guidance and tools available. V1c receives no training but receives ‘on the job’ support in carrying out the task. V1d receives both training and support.
For V2 (structure/definition), V2a receives no further task definition and no role definitions. V2b receives role and task definitions as implied by Positioning Theory (e.g. Barnes 2004). V2c receives role and task definition as implied by Argumentation Theory (Toulmin 1969). V2d receives role and tasks definitions combining Positioning Theory, Argumentation Theory and Mediation Theory (e.g. Noll 2010) and are asked to use the Delphi method (Alder 2002) of multiple rounds with feedback, as an underlying process. Positioning theory can be used to clarify the viewpoints of participants as individuals and as occupying roles in relation to policy. Argumentation theory can be used to elaborate particular positions, and mediation theory can be used to help reconcile positions and identify the precise points of difference between them. The Delphi method facilitates convergence on positions.
Policy areas (V3) would range from specific to broad/complex and from politically neutral to politically charged. The precise policy questions will be formulated in consultation with the academic supervisor and the Behavioural Insights Team.
All groups would have access to a variety of online tools and guides to techniques. These might include familiar tools such as those found in office applications (email, Word, spreadsheets, Powerpoint etc.) as well as more specialized tools and techniques such as rules of collaborative engagement, open data sources, stakeholder analysis, Delphi technique, consultation processes, an argumentation browser, mediation processes, policy evaluation and modeling tools, collaborative work tools, mediation processes and processes from systemic approaches to change. Where some individuals do not have internet or computer access or the necessary skills they would have to rely on other members of their group. All groups would have ‘rules of collaborative engagement’ and be encouraged to be inclusive.
There would inevitably be dropout from those that did not wish to (or could not) take part, those that start but do not finish etc.. A detailed analysis of dropouts at each stage will be treated as findings from the study (V4) rather than a challenge to its validity. It is expected that ‘leaders’ of the activity would emerge and that in some cases, possibly only one or a few people would be left at the completion of the task.
Subjects would most likely start by working remotely online, but some groups might choose to meet or communicate in some other way. In some conditions there would be no intervention and the research would simply observe.
Analysis:
As far as possible, all outputs and interactions would be captured for analysis, including the history of the development of the policy recommendations and associated rationale. This is easier for online activity (e.g. using version control, email trails etc.), but other forms of data capture might also be encouraged in some conditions. Crowd-sourcing would be considered for some aspects of the analysis. This design is expected to result in a rich dataset that can be used to investigate a variety of hypotheses in relation to citizen participation in policy formulation and inform participation design.
References:
REF 1 – Policy Making in the Real World (2011), Institute for Government.
http://www.instituteforgovernment.org.uk/sites/default/files/publications/Policy%20making%20in%20the%20real%20world.pdf
REF 2 – Open Government Partnership
http://www.opengovpartnership.org/country/united-kingdom
REF 3 – John, P. 2011. Taking Political Engagement Online: An Experimental Analysis of Asynchronous Discussion Forums.
Positioning Theory:
Barnes, M. 2004. ‘The use of positioning theory in studying student participation in collaborative learning activities’, University of Melbourne
Harré, R. 2015. Positioning Theory. The International Encyclopedia of Language and Social Interaction. 1–9.
Argumentation Theory:
Toulmin, S. 1969. The Uses of Argument, Cambridge, England: Cambridge University Press
https://en.wikipedia.org/wiki/Argumentation_theory
Mediation Theory:
McCorkle, S. and Reese M.J. 2014. Mediation Theory and Practice, Sage Publications Inc.
Noll, D. 2010. ‘A Theory of Mediation’, Chapter 2 in ‘AAA Handbook on Mediation’, JurisNet
Delphi Method:
Adler, M. and Ziglio, E. 2002. ‘Gazing in the Oracle: The Delphi Method and its application to Social Policy and Public Health’, Jessica Kingsley Publishers
Leave a Reply