A research design is a plan for collecting evidence that will be used to answer a research question. Selecting the design is the most important choice in planning a research project, because it determines all subsequent choices of methods. The design should be selected principally on the basis of how well it addresses the research question and enables the researcher to resolve a research problem. (Vogt, 2008, p. xxiv)
The aims of the research study were to explore the phenomenon of informed choice for disabled young people in transition to adult services/supports. This information has developed our understanding of informed choice in the context of self-directed support, especially with regard to perceived facilitators and barriers to informed choice.
The research strategy was based on a two phase qualitative exploratory sequential design. The first phase involved the secondary analysis of qualitative longitudinal data with the aim of identifying key themes that were used to inform the research questions for the primary stakeholder interviews. This strategy allowed the researcher to further explore the phenomenon of informed choice with a range of stakeholders to elicit multiple perspectives and identify emerging theories. The study explored the phenomenon of informed choice and has begun to develop broad hypotheses (‘folk theories’) based on the accounts (‘folk wisdom’) of key participants.
Theoretical approach
To ensure a strong research design, researchers must choose a research paradigm that is congruent with their beliefs about the nature of reality.” (Mills et al, 2006)
My own ontological and epistemological position is best represented by critical realism and the chosen research design is influenced by realistic evaluation (Pawson & Tilley, 1997). Realism accepts that there exists an objectively knowable, mind-independent reality, whilst acknowledging the roles of perception and cognition. Critical realism aims to identify structures that generate the social world in order to challenge inequalities and injustices. Advocates of critical realism describe it as the ‘third way’ beyond the limitations of positivism and interpretivism (Bergin et al, 2008). There are also claims as to its ‘double inclusiveness’:
[…] critical realism is […] the ontologically least restrictive perspective, insofar as it is maximally inclusive as to causally relevant levels of reality and additionally maximally inclusive insofar as it can accommodate the insights of other metatheoretical perspectives.” (Bhaskar & Danermark, 2006, p. 294)
Realism seeks to position itself as a model of scientific explanation which avoids the traditional epistemological poles of positivism and interpretivism. The researcher needs to be able to move between the different research paradigms in response to the situation. Realism’s key feature is its stress on the mechanics of explanation. Critical realism is an integration of realist ontology (there is a real world that exists independently of our perceptions and theories) with an interpretivist epistemology (our understanding of this world is inevitably a construction from our own perspective). Some realists describe their approach as being neither inductive or deductive, but ‘retroductive’ in that realists ask why things happen including the impact of the researcher on the production of the data (Olsen, 2007).
Realistic evaluation argues that it is not ‘programs’ per se that ‘work’; rather they contain certain ideas and resources which work for certain individuals in certain situations, and it is the task of the researcher to test theories to identify what works for whom in what circumstances. The basic model is: Context + Mechanism = Outcome. The realist evaluation cycle starts with theories about how mechanisms are fired in contexts to produce outcomes; specific hypotheses are derived from these theories in terms of what might work for whom in what circumstances; these hypotheses are tested through observations; and these observations lead to empirical generalisations (program specification) about what actually works; and these feed back into theory refinement (figure 1).
Figure 1: The realist evaluation cycle (Pawson & Tilley, 1997, p. 85)
Realistic evaluation suggests outcomes are characterised by the equation: C (Context) + M (Mechanism) = O (Outcome) and seeks to answer the question ‘what works for whom in what circumstances?’ (Pawson & Tilley, 1997, p.109). ‘Realistic evaluation’ is not an evaluation technique as such, but is a framework for programme evaluation (Pawson, 2002). Realistic evaluation can be used with any legitimate research method: both quantitative and qualitative, and does not favour any one technique (Pawson & Tilley, 1997).
The importance of social context in understanding how complex programmes lead to changes in outcomes are emphasised in realistic evaluation (Blamey & Mackenzie, 2007). People generate change in social contexts through their actions on the basis of their interpretations, capacities, and liabilities (Bonner, 2003). This approach to programme evaluation engages stakeholders in the generation and ownership of theories, requiring a more intensive relationship between researcher and stakeholders than would be found in other some other approaches.
Pawson & Tilley (1997) do not refer explicitly to grounded theory in their work, but this approach, which attempts to explain as well as describe, can be seen to have influenced realistic evaluation.
The theoretical approach adopted for this study is in many ways closely related to constructivist grounded theory, but with some distinct differences:
- Grounded theorists use theoretical sampling whereas I have sampled on the basis of including all the significant stakeholders in informed choice and self-directed support for disabled young people.
- Constructivist grounded theorists state that all reality is socially constructed whereas critical realists accept that there is an objective reality but we do not have direct access to it.
- Grounded theorists tend to use memos (notes) rather than retrospectively study interview transcripts in detail.
- Coding for grounded theorists is emergent and interactive, whereas I have coded after the interviews were completed: emergent but not interactive.
- Realistic evaluation uses the teacher – learner approach with the researcher articulating his/her theories to interview subjects and asking participants to share their own theories (‘folk wisdom’) and refine or refute the researcher’s theories.
Realistic evaluation is based on theory-led evaluation and such approaches have become increasingly popular way to study partnership evaluations (Dickinson, 2006); with a Theories of Change approach increasingly espoused as an evaluative approach to policy initiatives (Sullivan & Stewart, 2006). Both ‘theories of change’ and ‘realistic evaluation’ emphasise the importance of programme context in understanding how complex programmes lead to changes in outcomes.
Pawson & Tilley (1997) explain that all social programmes are introduced into pre-existing social contexts made up of social rules, norms, values and interrelationships. A key distinction between the two approaches is their interaction with stakeholders: in theories of change approaches, the theory is ideally articulated, owned and approved by a wide range of stakeholders; while the realist evaluator articulates the theory through interviews with a more limited selection of stakeholders, but retains control of the theory (Blamey & Mackenzie, 2007).
The research study on which the dissertation is based has included all four categories of key stakeholders distinguished by Pawson & Tilley (1997): subjects, practitioners, evaluators, and policy-makers. Each stakeholder group has valuable but partial knowledge and experiences of the phenomenon and it is the researcher’s task to draw on these diverse perspectives in order to refine concepts and develop theories.
[…] the goal has never been to construct theory per se, rather it has been to develop the theories of practitioners, participants and policy makers. (Pawson & Tilley, 1997, p.214)
Research questions
Given that self-directed support is a recent policy initiative with a lack of an established practice or research literature base, the research questions themselves were open and with the flexibility to explore emerging topics as the interviews progressed. Qualitative research questions were focused on developing theories from stakeholder’s knowledge, attitudes and experience. Core areas were asked of all participants, such as what they consider to be the facilitators and barriers to informed choice, and some specific questions were asked of individuals depending on their role in the process. As an exploratory study the interviews were semi-structured with fairly open questions to allow the researcher and participants to explore topics as they arose. The research questions were broad and based around who influences informed choice and what are the key factors in disabled young people’s decision-making:
- What information and assistance is available to disabled young people in making choices? • Who is involved in assisting disabled young people with informed choice? •
- What are the facilitators and barriers to informed choice for disabled young people?
Data collection methods
The study used a multiple qualitative design consisting of two strands with sequential timing: firstly secondary analysis of interviews held with disabled young people and their parents, and, secondly, primary research interviews with a range of stakeholders in Scotland.
1. Secondary analysis of interviews with disabled young people and their parents
Secondary analysis is a key research council funding priority in the UK as it is seen as a way of maximising the utility of existing research in a time of financial constraint. Secondary analysis can be defined as the (re)using of data produced on a previous occasion to glean new social scientific and/or methodological understandings. The re-use of previously collected qualitative data can, according to Irwin & Winterton (2011a), be carried out to:
- assess the credibility of new research
- supplement one’s own primary data
- reveal new methodological insights
- provide rich descriptive information
- generate new findings
- gain further insight on hard to reach populations without further intrusion into vulnerable populations
Secondary analysis should not be considered an easy option when choosing a research method:
Secondary analysis is a challenging undertaking. It is time consuming. It requires great persistence in ensuring an adequate understanding of details which may be tacit for primary researchers. […] However, as many insightful secondary analyses stand testament, there is a depth of social scientific insight and progress which can be achieved. (Irwin & Winterton, 2011c, p.14)
Debates on the challenges of secondary analysis tend to be focused on two methodological issues: the original data not fitting the new research questions, and the lack of direct access to the detailed contextual knowledge about the circumstances of the data collection possessed by the primary researcher (Coltart et al, 2013).
Interview data from the Social Policy Research Unit, University of York: Choice and Change: extending choice and control over the life course – a qualitative longitudinal panel study were added in 2012 to the ESRC Timescapes archive hosted by the University of Leeds. Choice and Change: extending choice and control over the life course – a qualitative longitudinal panel study examined the realities of choice in the context of changing circumstances, including changes in illness, disability or social transitions (Mitchell, Maddison & Beresford, 2011). The original Choice and Change study included 27 disabled young people with degenerative conditions and their parents and focused on the processes by which choices are made within families, exploring the respective roles and experiences of young people and their parents. Data from 20 of these 27 cases were archived with Timescapes.
Irwin & Winterton (2011b) caution against sampling from within project data sets as there is a risk that this further decontextualises data. However, with regard to my own research questions it seemed relevant to limit the cases included to have some commonality in terms of a statement of special educational need (SEN). An SEN is applied in England to children and young people with a learning disability who require additional help at school. Only individuals were included in the sub-sample where both the young person and their parent(s) interview transcripts were archived. This left a cohort of 6 disabled young people and their parents (mainly, but not exclusively, mothers) to be included in the secondary data analysis. One case consisted of two young disabled people from the same family, resulting in 6 disabled young people and their parents included in the sample from 5 households (table 1). The following table presents a pen picture of each individual. All the young people were White British, and all families had access to a car.
Table 1: Details of study participants:
1. Interviews with stakeholders
Semi-structured interviews were held with a range of stakeholders including a policy maker from the Scottish Government SDS team, the co-ordinator of a national transitions forum, and a number of people from ‘Urbal Region’ (an anonymised region of Scotland with a mix of urban and rural population), including a disabled young woman, a third sector manager of support services, a manager of an independent college providing residential and day support for young people with learning disabilities, a social worker, a social work manager, and an independent consultant with experience in SDS and participation for disabled young people.
Urbal Region was selected as an appropriate location for the research interviews as a region with a mix of urban and rural settings, and because it was a Scottish Government test site between 2010 and 2011for self-directed support for disabled young people. Purposeful sampling was used to recruit participants in that a ‘gateway’ professional contact in Urbal Region was asked to identify suitable participants representing the range of stakeholders involved in transition planning for disabled young people. Relevant national stakeholders were identified and approached directly by the researcher. The focus of the stakeholder recruitment was on recruiting both ‘active stakeholders’ who influence the policy at different stages, and ‘passive stakeholders’ who are affected by the policy (Hanberger, 2001).
Semi-structured interviews offered the advantages of a degree of shared focus across the interviews with predetermined topics to be covered, but also allowed scope for flexibility to pursue certain subjects in greater depth. Informed consent to participate was gained from all the subjects and all interviews were audio recorded on a digital recorder. Transcripts of the interviews were written on word documents on a personal computer by the researcher. These were anonymised, in that the name and locality of the participant was not stated. The interviewees’ professional background or role was tied to the transcripts for analysis purposes. Any subsequent publications will ensure anonymity by not identifying participants by name or indicating the geographical region in which the research was carried out.
The potential risks or hazards in terms of data collection are limited but included the loss of the digital recorder containing recordings of interviews. The substance of the recordings is considered to be of low risk in terms of any data protection issues or potential harm to the interviewees. Risks of loss of equipment were mitigated by keeping the digital recorder in a locked briefcase and written records are kept in a password protected personal computer. Transcripts will be deleted after a period of 6 months.
Other data collection methods were considered: participant observation, documentary analysis, and focus groups but these methods were not pursued on the basis of ethical and pragmatic grounds. Firstly, direct observation (either as a participant or non-participant) is a standard ethnographic method that can be used to understand behaviours such as the interaction between practitioners and service users. It is argued that behaviours are more stable over time than are attitudes and opinions (Gobo, 2011).
Secondly, the use of documents, such as minutes of transition planning meetings, could offer a unique perspective on the study phenomena in terms of their content and how they are manufactured (Prior, 2003). For example, a study of the minutes of transition planning meetings could reveal patterns of who attends such meetings, the issues discussed, whose ‘voice’ is recorded, and who is involved in informing and making choices. Documentary analysis could have been particularly useful as a means of identifying themes and topics to be pursued in more depth in interviews with the participants involved in transition planning meetings. The content of the documents and the way they are produced (i.e. who writes them, who verifies the content, what participants feel was said but not recorded, etc.) could be used to structure discussions with practitioners, family members, and disabled young people.
Thirdly, focus groups have been promoted as an efficient and empowering approach in disability research. Focus groups are sometimes referred to as focused group interviews as the researcher decides on the membership of the group, usually based on the members sharing certain characteristics of relevance to the study, and facilitates the discussion on a pre-selected topic. Focus groups are particularly useful when the researcher wants to explore people’s experiences, opinions and feelings (McLaughlin, 2012). They are often used at an early stage of research to discover general viewpoints which can be used to inform the design of larger studies (Smithson, 2008), but can also be used in mixed method research with disabled people to explore or clarify the context of quantitative findings (Kroll et al, 2007). Focus groups are particularly useful for exploratory research that addresses broad questions about ‘why’, ‘how’, ‘when’, ‘where’, and ‘what kind’ (Stewart et al, 2009).
Participant observation, documentary analysis, and focus groups are all valid approaches that could have been deployed within a realistic evaluation framework but pragmatically and ethically these methods posed additional challenges for the researcher in terms of time, cost, informed consent, and confidentiality that could not be justified in terms of the overall research strategy.
Data analysis strategy
Analysis is the interplay between the researcher and the data. (Strauss & Corbin, 1998, p.13)
Secondary analysis
In considering which methodology to apply to the analysis of the secondary data, I was aware that the interview transcripts, as documents, could be analysed using a number of approaches including grounded theory, critical discourse analysis, interpretative phenomenological analysis, and narrative analysis. However, the interview transcripts did not employ the detailed transcription notations (e.g. Jeffersonian transcription system) that would enable approaches that relied on detailed transcription of every word, utterance, and silence.
One should be cautious of treating transcripts as ‘true’ representations of speech. Despite technical guidelines transcription remains an imperfect process of constructing a textual version of the original interaction (Nikander, 2008). Transcripts are constructions and as such are products shaped by the researcher’s theories (Skukaskaite, 2012). A symbolic interactionist approach stipulates that the researcher must enter the everyday world of study participants to appreciate their experience (Milliken & Schreiber, 2012). Critical realists argue that there is a material dimension to people’s lives that is separate from language, and that language which describes our social realities is constrained by material conditions.
Sims-Schouten, et al (2007) state that while the interviewees discourse can help to illuminate the constraints of the ‘real world’, the researcher needs to explore material concerns outside the boundary of the interview by means of a literature review, and by exploring the local social and physical environment in which the respondent lives. However, there are critics of this approach who claim that a reflexive approach would recognise how the researcher is helping to socially construct the interview, and how a skilled analysis of the interview will reveal in the discourse, patterns relating to social and political structures (Speer, 2007).
The secondary analysis of the Timescapes data archive (Choice & Change Project) qualitative longitudinal dataset was carried out using thematic analysis based on the work of Attride-Stirling (2001). A thematic network analysis consists of three broad stages: (a) the reduction or breakdown of the text; (b) the exploration of the text; and (c) the interpretation of the exploration. These stages can be understood in six steps (Box 1).
Box 1: Stages and steps in thematic network analysis
Thematic analysis involves a search for patterns of categories that help to describe and interpret a phenomenon. While grounded theory follows an inductive approach in which theories emerge from the collected data, my own approach to the research design has taken a ‘general inductive approach’ in which data analysis is determined by both the research objectives (deductive) and multiple readings and interpretations of the interview data (inductive):
Data analysis is determined by both the research objectives (deductive) and multiple readings and interpretations of the raw data (inductive). Thus the findings are derived from both the research objectives outlined by the researcher(s) and findings arising directly from the analysis of the raw data. (Thomas, 2003, p.3)
Step 1: code material: The interview transcripts from the secondary data were read fully. The texts of the transcripts were printed and blocks of text that appeared to have relevance to the research questions were highlighted using a highlighter pen. The highlighted texts were read again and salient statements representing relevant beliefs and processes were identified and given codes. For example the code ‘experiential knowledge’ included text segments such as “We had a walk round the school and…looked and I, doubted the toilet” (mother of disabled young man moving schools). Texts were then dissected using the coding framework with manageable quotations and short segments of the interviews ‘cut and pasted’ from the word documents into a separate word document.
Step 2: identify themes: Once the text had been coded, more abstract themes were applied to the coded text segments. The aim was to reduce the data into a more manageable set of significant themes that concisely summarised the main issues. In this case 12 codes were grouped into 4 clusters and led to 9 themes being identified (table1).
Table 1: From codes to themes (CHECK this seems to be the 2nd table 1)
Step 3: constructing the networks: The 9 Basic Themes identified in table 1 were arranged into 4 groups based on conceptual correspondence and these were interpreted as Organising Themes. Global Themes unifying the organising themes were then produced (table 2).
Table 2: From Basic to Organising to Global Themes
Step 4: describe and explore the thematic network: The Global Theme of information content, accessibility and trustworthiness will be used to demonstrate this step (figure 1). This constitutes one thematic network comprising two Organising themes and 5 Basic Themes and explores the content of information and the ways in which it is made available to disabled young people and their parents.
The interview transcripts of the secondary data revealed the importance of disabled young people and parents having direct experience of a service in informing choice as evidenced by a disabled young woman being asked about the helpfulness of having information, who repeatedly uses the phrase “going through it”:
You could have a whole book on it, but it’s still unknown to you, because until you are there going through it, you don’t know.
The Organising Themes in the network suggest that disabled young people and their parents benefit from information that is meaningful to them and correct. A lack of trust in the information provided by professionals is a barrier to informed choice. The following is a quote from a mother giving her perception of the information provided by social workers:
I do think me husband will tell you, if he was sat here now he would say that it’s a bit, he will say it’s scary because … they were giving us sometimes the wrong information, time and time and time again.
Figure 1: Thematic network for information format and style
Step 5: summarise the thematic network: The thematic network revealed two distinct components of informed choice: the role of others in terms of trust and emotional support, and how the actual information was presented/made available to disabled young people and their parents. It was not sufficient that the information was in an accessible format, disabled young people also needed to trust the source of the information and receive emotional support from a family member (generally mothers) in a way that empowered their involvement in choice-making.
Step 6: interpret patterns:
In step 6, the researcher must relate the principal themes and patterns that emerged in the analysis to the original questions; and propose some explication of the questions grounded on the content and exploration of the texts, and on the theoretical constructs guiding the research. (Attride-Stirling, 2001, p. 402)
The theoretical constructs of social capital and bounded agency (Evans et al, 2001) can be used to inform the analysis of the other Global Theme (empowerment and support). Analysis of the text provides evidence to support the hypothesis that a disabled young person’s sense of individual agency (power to make decisions) and social capital (trust in parents, professionals, and others) have an impact on informed choice and involvement in choice-making. The following transcript extracts reveal significant differences in feelings of self-empowerment and support from others. In the first example a disabled young woman explains that although she may seek information and emotional support from others, important life decisions are made by her alone:
[Local Hospice] is good for the information and mum’s good with the support, but at the end of the day it’s my life isn’t it?
In the second case, the parents of a disabled young man are asked if they feel their son has the right level of involvement in decisions regarding his care needs:
[Mother]: I mean it’s funny because if the doctor comes here, or any professional comes here, [Young Person] will say “I’ll go in me room where you can talk.” So he sort of knows but he doesn’t, he’s, it’s funny really when he says that cos you think well what’s going through his mind, thinking well they’re talking about me. [Father]: But we have a saying like “Eh, it’s adults now.”
It should be recognised that interviews are themselves socially constructed contexts producing narratives that are context specific to the interview and may or may not provide access to meanings relating to a wider ‘reality’ (Miller & Glassner, 2011). The interview should be regarded as a product of a situated interaction, rather than as the embodiment of the interviewee’s pre-existing knowledge, beliefs and values. The research interview has been described as a social encounter whose meaning is negotiated between the participants, and Scott et al (2012) explore the ‘dramaturgical complexity’ of this relationship and how the interview can be shaped by the personality of the researcher.
Both Hammesley (2010) and Seale (2011) emphasise the distinction between data and evidence or, put another way, data collection and data analysis. Data is collected as a resource but in using or re-using data the researcher analyses and reconstitutes the data as evidence to support a theory or illuminate a research conclusion. Their central argument is that selection and interpretation of the ‘raw’ data is always involved, so there is no absolute distinction between primary and secondary data analysis.
Analysis of primary stakeholder interview data
In phase two of the research strategy I adopted a looser, more fluid approach to data analysis than had been the case in phase 1. The emphasis in this phase was on discovering more specific hypotheses regarding context, mechanism and outcomes relating to the barriers and facilitators of informed choice for disabled young people. Disaggregating the data into small chunks for coding purposes may distort meaning by removing extracts of speech from the original context and selective quotations from interviews can therefore be biased or misleading (Taylor, 2012). By attentive listening to the interview audio recordings and re-reading the typed transcripts (see appendix 2 for an example of a transcript) I identified ‘folk theories’ and ‘folk wisdom’ of the stakeholders that could be used to develop more specific hypotheses of ‘what works for whom, in what circumstances’.
Ethical issues
The fundamental principle in ethics is that no one should be harmed by research. Who has the authority to make this judgement: should the researcher act as ‘judge and jury’? (Robson, 1993). This is rarely the case nowadays as most research has to satisfy the requirements of some ethical body. In the case of this dissertation, approval was gained from the faculty ethics committee prior to any contact with participants.
The application provided details of the practical means by which the issues of informed consent, non-harm, and confidentiality would be addressed. Information sheets providing details of the researcher and the research project were given to all participants (appendix 1). Interview subjects received further verbal information immediately prior to the interview and were asked to sign consent forms regarding their participation. All participants gave their written consent for interviews to be audio-recorded.
In addition to the process of the research the author reflected on the interpersonal relationships involved in the interaction with participants and maintained an awareness of the question of power and having a relationship with all participants based on trust, respect and reciprocity (Barton, 2005). Social science is not ‘value-free (Padgett, 1998).