Family Practice Vol. 17, No. 1, 71-75
© Oxford University Press 2000
Methodologies |
The ‘doctor’ or the ‘girl from the University’? Considering the influence of professional roles on qualitative interviewing
Department of General Practice, University of Glasgow, Glasgow, UK.
Helen Richards, Department of General Practice, University of Glasgow, 4 Lancaster Crescent, Glasgow G12 0RR, UK.
Richards H and Emslie C. The ‘doctor’ or the ‘girl from the University’? Considering the influence of professional roles on qualitative interviewing. Family Practice 2000; 17: 71–75.
Received 21 June 1999; Accepted 6 September 1999.
Abstract
Background. Qualitative research methods are now recognized as valuable tools for primary care. With the increasing emphasis on evidence-based medicine and critical appraisal of published work, it is important that qualitative researchers are transparent about their methods and discuss the impact of the research process on their data.
Objectives. To consider the impact of the professional background of researchers on in-depth interviewing in primary care.
Methods. We compare interactions between the interviewer and respondents in two qualitative interview studies of heart disease. Both samples consisted of 60 middle-aged men and women from a range of social backgrounds living in the West of Scotland. One study was conducted by a GP and the other by a sociologist.
Results. Some interview interactions were common to both researchers; for example, interviews were often regarded by respondents as therapeutic. However, some interactions seemed to be related to the researcher's professional background. The GP's perceived higher status led to obscuring of her personal characteristics. The sociologist was often perceived as a ‘young woman’ rather than defined by her professional role. Thus respondents' perceptions of the interviewer influenced the interview interactions.
Conclusions. Appraising qualitative research depends on the transparency with which the research process is described. Awareness of professional background is particularly important for university departments of primary care (which often include doctors, nurses and social scientists) and should be considered carefully in designing, carrying out and disseminating the results of qualitative studies.
Keywords. Heart disease, methodology, qualitative research, reflexivity..
"The scientific observer is part and parcel of the setting, context and culture he or she is trying to understand and represent."1 (p. 486).
Introduction
Qualitative methods make a valuable contribution to primary care research.2–4 However, assessing the standard of qualitative research depends on the transparency with which the research process, and its impact on the data, are described.5–8 One part of this process which is rarely considered in primary care research is the interaction between the interviewer and respondent. In this paper, we consider the impact of the professional background of the researcher on this interaction. This is particularly relevant for university departments of general practice which tend to include researchers from such diverse professional backgrounds as general practice, nursing and sociology, who may be working as part of a team.9
Social scientists have considered how contextual details, such as the interview setting and the interaction between their own and the respondent's personal characteristics, may influence their data. Awareness of these factors is sometimes called reflexivity. Researchers have been encouraged to reflect upon their own gender, ethnic and class identities and "to abandon the illusion that researchers, their informants, and the research setting do not influence each other reciprocally"10 (p. 882). Similarly, feminist writers have stressed that researchers' own experiences and interests influence their research11–13 from the choice of research topics to the way in which it is disseminated. However, ‘technical’ aspects of the interview (e.g. the number conducted, their length and content, how they were recorded) are still much more likely to be included in the literature than information about the nature of interactions between the interviewer and respondents.14,15
The absence of contextual detail in qualitative research published in general practice has been highlighted recently.16 This may be due largely to the restrictions that medical journals place on the length of articles: few allow more than 2500 words. The result is that information about the professional background of researchers, the relationship between the interviewer and respondents and the context in which interviews take place is rarely provided, making it difficult both to evaluate studies and for new qualitative researchers to gain guidance from published accounts. Hoddinott, a GP researcher, has discussed the influence of professional background on the qualitative interview. In a study on attitudes to breast feeding,17,18 she carried out some interviews stating she was a GP, and others saying she was a ‘researcher’. She found that her interviewing skills were better when she declared that she was a GP, and concludes that in future she would make her professional background clear to all respondents.
Here, we wish to extend Hoddinott's account of the influence of professional background on interviewing by comparing our experiences as two researchers from different professional backgrounds (general practice/ sociology), working on similar qualitative studies in the same university department of general practice. First, we provide some background about the studies and about ourselves. Then, we reflect upon types of interview interactions common to both researchers. Finally, we provide some examples of the ways in which professional background and personal characteristics may influence interview interactions.
Background
The studies
Both studies involved in-depth interviews in the West of Scotland with 60 middle-aged men and women of varied occupational social class, classified as ‘middle-class' if they were engaged in non-manual work (I, II or IIINM) and ‘working-class' if engaged in manual work (IIIM, IV or V). The family history study,19 carried out by C Emslie, explored respondents' perceptions of whether they had a family history of heart problems, and the chest pain study, carried out by H Richards, explored perceptions of chest pain. Both qualitative samples were drawn from previous large population studies20,21 which involved structured questionnaires and clinical measurements (electrocardiographs and blood tests). Respondents in the chest pain study were slightly older (between 45 and 64 years, compared with between 41 and 51 years in the family history study) and, because experience of chest pain was a selection criterion, they were more likely to have had significant medical problems. Respondents in both studies were sent a letter, on the university department of general practice letterhead, explaining the purpose of the study, followed by a telephone call to arrange the interview at a place convenient to them. Interviews were tape-recorded and transcribed in full. Extensive field notes (including observations about the respondent and their setting, initial ideas about emerging themes for analysis, methodological issues and personal notes) were recorded after each interview.
The researchers
C Emslie is a sociologist and H Richards is a GP. Both are white, female and middle-class, and have similar experience of qualitative research. C Emslie is Scottish and in her late twenties, H Richards is English and in her mid-thirties.
Reflections on interview interactions
While certain interactions were noted by both researchers, some seemed to be associated with being a doctor or a sociologist. We discuss these in turn. Respondents' words are presented in italics and labelled with their gender, social class and ID number, and the interviewer's initials where necessary.
Interactions common to both researchers
The similarity between research interviews and therapeutic interactions has been widely discussed.22–25 Qualitative interviews are often described as ‘non-directive’, a term which originates from psychotherapy, where patients are encouraged to express problems in their own terms "stimulated by an interested and sympathetic listener"26 (p. 111).
Some respondents in both studies stated that they found interviews therapeutic. For example, when asked to describe her health problems, one respondent replied: "Oh, I love this. I love it. I think that once you get to a certain age, your doctors are not interested in you" (female, working-class, R31, HR). Another respondent commented that an interview had been "cathartic", like "sitting in the psychiatrist's chair" (male, middle-class, R56, CE).
Respondents also disclosed sensitive and confidential information. One respondent who still had chest pain despite a recent coronary bypass operation said "I don't mention it to anyone, if I mentioned it to S (wife), she'd just get excited about it" (male, middle-class, R1, HR). In the family history study, men and women talked about their difficult childhoods, and alcoholism, illegitimacy and suicide amongst family members. A number of respondents became upset when talking about painful or early deaths of family members (CE/HR) and about experiences of caring for elderly relatives (HR). For example, one woman became tearful when talking about her elderly mother: "It ended up anyway, she was the one sitting on my knee" (female, working-class, R32, HR). Considering respondents' emotions in primary care research is important because respondents often talk about sensitive subjects such as personal relationships and health, and yet, with few exceptions,14,22,27 this issue is not addressed in the literature.
Interactions specific to each researcher
The GP researcher..
I decided, for methodological and ethical reasons, to be explicit about my professional background, and introduced myself as a GP involved in research. I made it clear that I was not practising currently. I gave no medical advice; general queries were dealt with at the end of the interview and respondents were referred to their own GP as appropriate.
Many of the working-class respondents were deferential: the title ‘Doctor’ was often used and I was introduced by several interviewees to family members as ‘the doctor’. One respondent apologized for taking up my time, even though the interview took place at my request, and one revealed his view of doctors by saying of his own GP: "He's the same sort of man as a priest type thing ... they're not far away from the same profession" (male, working-class, R46). Another respondent referred to his GP as "a saint" (male, working class, R56).
Deference was not observed among the middle-class respondents who, in contrast, tended to align themselves with me by assuming commonality of opinions and experience. In particular, there were references to friendships and insider knowledge which had led to special medical treatment. For example, one respondent who was in a profession allied to medicine said: "There's a waiting list for bypasses obviously but, in a sense, ...I'm on an inside track" (male, middle-class, R18). Another middle-class man reported how he was able to avoid the GP appointment system by knocking on the surgery door when his GP was working late, and there were several examples of friendships between respondents and their GPs and consultants. The assumed commonality of experience was stronger in male than female respondents.
Respondents from all social backgrounds asked clinical questions. For example, with regard to cholesterol, one respondent said: "You might be able to tell me something about that. Somebody was telling me that they've reduced the upper limit from ... six down to four point six" (male, middle-class, R29). In relation to heart disease and the menopause, a woman asked: "Once that [periods] stops like, you're at higher risk ... Is this true?" (female, working-class, R31). Respondents frequently gave detailed accounts of hospital appointments and medical treatments. Another example of the interview taking on elements of the consultation was when respondents talked about sexual health problems. One response to the question "Have you had any other sort of health problems?" was: "Ehh, the sexual one are you talking about?" (male, working-class, R49), and a woman, talking about her hysterectomy, said: "Even sexually it 's changed me, I can't be bothered to be touched, I just don't have the same urges any more" (female, middle-class, R2).
When asked at the end of the interview whether there was anything they would like to add or ask, respondents expanded on their health problems, asked health-related questions and gave their opinions of doctors and other aspects of the health service. One woman said: "Excuse me saying it, I don't think [doctors] realise how frightened people are when having this sort of thing [operations]" (female, working-class, R13). In this case, the respondent apologized for expressing a negative view of doctors.
The sociologist. . I chose to introduce myself to respondents as a ‘researcher’ rather than as a ‘sociologist’ because the role of a researcher is more commonly recognized and is easier to explain. I felt, for a number of reasons, it was vital to distance myself from medical professionals. First, the primary focus of the study was on lay perceptions of family history of heart disease; secondly, respondents had already provided clinical measurements for the earlier epidemiological study; and thirdly, respondents knew that the study was based in a university department of general practice. To reinforce this, I did not use my title (‘Doctor’) on the introductory letter, stressed at the beginning of the interview that I was not medically qualified, and asked respondents to explain even common medical conditions to me. This was important both ethically and practically, as I did not wish respondents to focus narrowly on medical issues.
I arrived on respondents' doorsteps, a young, fairly casually dressed woman without medical qualifications or the usual signs of professional status (suit and car). On the other hand, I was associated with a well-known health study at the university and produced my ‘tools of the trade’ (a tape recorder and topic guide). These seemingly contradictory signs of status seemed to make me rather hard to place, particularly for some male respondents. It has been noted that gender is often highlighted when women interview men because the qualitative researcher "is required to take on an acquiescent, attentive, and assenting role very close to traditional notions of femininity"15 (p. 630). A number of (mainly male) respondents attempted to resolve some of these contradictions in status by asking me at the end of the interview if I was a student, assuming that the research was for a thesis. On another occasion, a respondent (female, working-class, R33) introduced me as "the girl from the University" to each member of her family as they arrived home from work. When her husband arrived, he voiced his concerns to both of us that market researchers would contact them as a result of the interview. The respondent was clearly embarrassed and assured her husband that I was a university researcher and would not pass on their details to anyone else.
In contrast, my professional status was much more apparent when respondents chose to be interviewed at the university. Similarly, a few middle-class women emphasized my professional status by asking me about my career. One woman was very eager to introduce me to her family after the interview (female, middle-class, R4). She informed her husband that I had a PhD, and introduced me as a potential role-model to her teenage daughter, who was interested in medicine as a career. These examples illustrate how contextual details of the interview, such as the gender and age of the interviewer and respondent and where the interview takes place, interact and influence the interview process.
Differences in social class were referred to obliquely. Respondents asked me where I lived, or where I had been brought up. One respondent referred to her "broad Scottish voice" on the tape (female, working-class, R44). While most respondents expected me to understand colloquial Scottish phrases and expression, others checked that I followed what they were saying.
Although I did not ask respondents anything about their doctors or their views of medical professionals, they often commented on them. These comments were usually unfavourable, perhaps because I had stressed that I was not medically qualified. For example "I wish doctors would listen to you more, because you're not medically minded, so when you tell them something they have this preconceived idea of what is wrong with you, and they're not listening to what you're actually trying to tell them" (female, middle-class, R15). One man said that were it not for the hospital health checks his workplace provided, "I'd be relying on my GP for my health-check, and you really don't want to know what I think about GPs!" (male, middle-class, R17).
At the end of the interview, respondents were asked what they had thought of the interview and if they had any questions. A number of respondents referred back to the clinical measurements taken as part of the general population survey. I reassured respondents that any abnormal results would have been passed on to their GP and stressed that I did not have access to these results. Most respondents had no previous experience of an in-depth interview and many said they had not been sure what to expect. This was illustrated by the comment of a busy teacher who asked me during the initial telephone call if I could just "send her a questionnaire instead" (female, middle-class, R54). Respondents from all backgrounds wanted to know what would happen to the interview material and were interested to know more about the research process. Finally, when interviews were conducted in respondents' homes in the evening, they showed concern about me getting home by public transport.
Discussion and conclusions
In this paper, we compared the interview process in two qualitative studies of heart disease. Both authors found that interviews were described by some respondents as therapeutic. Qualitative interviews are likely to be perceived as therapeutic by respondents regardless of the professional background of the interviewer. However, the authors found some notable differences in interview interactions. Doctors have a more clearly defined role and higher social status than sociologists. This difference in status led to the more frequent observation by HR than CE of deference amongst working-class respondents and social alignment amongst middle-class respondents.
While we have concentrated on the impact of professional background on interviewing, personal characteristics (such as gender, age, ethnicity and social class) of the researcher and the respondent are equally important. These characteristics are not discrete and unchanging, but dynamic and interdependent, constantly constructed during the interview. Morgan28 argues that "gender (in common with other personal characteristics) is a latent variable, exaggerated in some cases and relatively muted in others" (p. 46). We found that this was also true of professional background. The professional identity of ‘the doctor’ is so well known that it became an exaggerated variable in HR's interviews, overshadowing her personal characteristics. In contrast, professional background was much more muted for CE, so gender and age were key variables in identifying her as ‘the girl from the University’.
Respondents' preconceptions of the roles of doctors and university researchers also influenced the content of the interviews, particularly the topics mentioned spontaneously by respondents. For example, respondents asked HR health-related questions (including those relating to sexual health), whereas they tended to talk to CE about broader, non-health-related topics.
We are not suggesting that social scientists collect ‘better’ data than GPs, or vice versa, but that it is important that researchers reflect upon their professional backgrounds and personal characteristics and consider how these influence the way in which they gather and analyse data. Respondents have definite preconceptions about the role of a GP; in contrast, a sociological ‘researcher’ is an unknown quantity and respondents took more time to place the interviewer. GP researchers must first decide whether or not to declare their professional background. If they do make it clear that they are medically qualified, they should be aware of respondents' possible preconceptions and take care to explain their current role as researcher. Sociologists and other social scientists, who have a less clearly defined professional role and status, face the similar challenge of making their professional background and interests clear.
The main limitation of this paper is that the two studies had different recruitment criteria. Participants in the chest pain study were more likely to have had significant medical problems and undergone investigations and treatment than those in the family history study. However, the similarity of the studies in terms of sample composition, interview topic and geographical area provided an unusual opportunity to examine the influence of professional roles on the interview process.
One of the strengths of university departments of general practice is their eclectic composition: anthropologists, psychologists, nurses, sociologists and GPs all bring different perspectives to research. Because of this diversity, the impact of professional background should be considered carefully when designing, carrying out and disseminating qualitative research. It has been argued that "who you are affects what you get told"29 (p. 114); we suggest that who respondents think you are affects what you get told. In this study, we have focused solely on data collection. Future work is planned to explore the influence of professional roles on other stages of the research process.
Acknowledgments
We would like to thank Rosaline Barbour, Marina Barnard, Kate Hunt, Margaret Reid, Graham Watt and an anonymous referee for their helpful and perceptive comments, Karen Kane for her careful transcription, and the respondents for their valuable assistance. The chest pain study was funded by the Wellcome Trust and the family history study was funded by the UK ESRC Social Variations Programme, grant number L128251028.
References
1 Altheide DL, Johnson JM. Criteria for assessing interpretive validity in qualitative research. In Denzin NK, Lincoln YS (eds). Handbook of Qualitative Research. London: Sage Publications, 1994: 485–500.
2
Kinmouth A. Understanding and meaning in research and practice. Fam Pract 1995; 12: 1–2.
3
Britten N. Qualitative interviews in medical research. Br Med J 1995; 311: 251–253.
4
Jones R. Why do qualitative research? Br Med J 1995; 311: 2.
5
Freer CB, Coles CR. Family practice research: quantity and quality. Fam Pract 1987; 4: 316–317.
6
Seale C, Silverman D. Ensuring rigour in qualitative research. Eur J Publ Health 1997; 7: 379–384.
7
Mays N, Pope C. Rigour and qualitative research. Br Med J 1995; 311: 109–112.
8
Britten N, Jones R, Murphy E, Stacy R. Qualitative research methods in general practice and primary care. Fam Pract 1995; 12: 104–114.
9
Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res 1999; 9: 26–44.
10 Ambert A, Adler PA, Adler P, Detzner DF. Understanding and evaluating qualitative research. J Marriage Fam 1995; 57: 879–893.[Web of Science]
11 Morgan DHJ. Discovering Men: Critical Studies of Men and Masculinities. London and New York: Routledge, 1992.
12 Maynard M. Trend report—the re-shaping of sociology? Trends in the study of gender. Sociology 1990; 24: 269–290.[Web of Science]
13 Scott S. Feminist research and qualitative methods: a discussion of some of the issues. In Burgess RG (ed.). Issues in Educational Research. Philadelphia: Falmer, 1985: 67–85.
14 Oakley A. Interviewing women: a contradiction in terms. In Roberts H (ed.). Doing Feminist Research. London: Routledge, 1981.
15 Green G, Barbour RS, Barnard M, Kitzinger J. Who wears the trousers? Sexual harassment in research settings. Women's Stud Int Forum 1993; 16: 627–637.
16
Hoddinott P, Pill RM. A review of recently published qualitative research in general practice. More methodological questions than answers? Fam Pract 1997; 14: 313–319.
17
Hoddinott P, Pill RM. Qualitative research interviewing by general practitioners. A personal view of the opportunities and pitfalls. Fam Pract 1997; 14: 307–311.
18
Hoddinott P, Pill RM. Qualitative study of decisions about infant feeding among women in east end of London. Br Med J 1999; 318: 30–34.
19 Emslie C, Hunt K, Watt G. Barriers rooted in biography. In Graham H (ed.). Understanding Health Inequalities. Buckingham: Open University Press, forthcoming.
20 Tunstall Pedoe H, Kuulasmaa K, Mahonen M, Tolonen H, Ruokokoski E, Amouyel P. Contribution of trends in survival and coronary-event rates to changes in coronary heart disease mortality: 10-year results from 37 WHO MONICA project populations. Lancet 1999; 353: 1547–1557.[Web of Science][Medline]
21 Hawthorne VM, Watt GCM, Hart CL, Hole DJ, Smith GD, Gillis CR. Cardiorespiratory disease in men and women in urban Scotland: baseline characteristics of the Renfrew/Paisley (MIDSPAN) study population. Scott Med J 1995; 40: 102– 107.[Web of Science][Medline]
22 Bourne J. Researchers experience emotions too. In Barbour RS, Huby G (eds). Meddling with Mythology: AIDS and the Social Construction of Knowledge. London and New York: Routledge, 1998: 90–104.
23 Gatter P. Policing boundaries: linking the theory and experience of psychotherapy in HIV/AIDS research. In Barbour RS, Huby G (eds). Meddling with Mythology: AIDS and the Social Construction of Knowledge. London and New York: Routledge, 1998: 54–75.
24 Collins P. Negotiating selves: reflections on ‘unstructured’ interviewing. Sociol Res Online. 1998. <http://www.socresonline.org.uk/socresonline/3/3/2.html>
25 Small N. The story as a gift: researching AIDS in the welfare market place. In Barbour RS, Huby G (eds). Meddling with Mythology: AIDS and the Social Construction of Knowledge. London and New York: Routledge, 1998: 127–146.
26 Whyte WF. Interviewing in field research. In Burgess RG (ed.). Field Research: A Sourcebook and Field Manual. London: George Allen and Unwin, 1982: 111–122.
27 Masters H. It's a family affair: on public and private accounts of HIV/AIDS. In Barbour RS, Huby G (eds). Meddling with Mythology: AIDS and the Social Construction of Knowledge. London and New York: Routledge, 1998: 75–90.
28 Morgan DHJ. ‘Gender’. In Burgess RG (ed.). Key Variables in Social Investigation. London and New York: Routledge and Kegan Paul, 1986: 31–53.
29 Deverell, K. Fact-finder, fag hag, fellow and funambulist: research as a balancing act. In Barbour RS, Huby G (eds). Meddling with Mythology: AIDS and the Social Construction of Knowledge. London and New York: Routledge, 1998: 104–124.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C. Emslie, F. Whyte, A. Campbell, N. Mutrie, L. Lee, D. Ritchie, and N. Kearney 'I wouldn't have been interested in just sitting round a table talking about cancer'; exploring the experiences of women with breast cancer in a group exercise trial Health Educ. Res., December 1, 2007; 22(6): 827 - 838. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Hewitt Ethical Components of Researcher Researched Relationships in Qualitative Interviewing Qual Health Res, October 1, 2007; 17(8): 1149 - 1159. [Abstract] [PDF]
|
|
|
|
|
|
E. Peel, O. Parry, M. Douglas, and J. Lawton "It's No Skin off My Nose": Why People Take Part in Qualitative Research Qual Health Res, December 1, 2006; 16(10): 1335 - 1349. [Abstract] [PDF]
|
|
|
|
 |
|
 S Reventlow and C Tulinius The doctor as focus group moderator--shifting roles and negotiating positions in health research Fam. Pract., June 1, 2005; 22(3): 335 - 340. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Lipman, M. J Murtagh, and R. Thomson How research-conscious GPs make decisions about anticoagulation in patients with atrial fibrillation: a qualitative study Fam. Pract., June 1, 2004; 21(3): 290 - 298. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K.-D. B. Nielsen, L. Dyhr, T. Lauritzen, and K. Malterud "You can't prevent everything anyway": A qualitative study of beliefs and attitudes about refusing health screening in general practice Fam. Pract., February 1, 2004; 21(1): 28 - 32. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Richards, M. Reid, and G. Watt Victim-blaming revisited: a qualitative study of beliefs about illness causation, and responses to chest pain Fam. Pract., December 1, 2003; 20(6): 711 - 716. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Aira, J. Kauhanen, P. Larivaara, and P. Rautio Factors influencing inquiry about patients' alcohol consumption by primary health care physicians: qualitative semi-structured interview study Fam. Pract., June 1, 2003; 20(3): 270 - 275. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Emslie, K. Hunt, and G. Watt A chip off the old block? Lay understandings of inheritance among men and women in mid-life Public Understanding of Science, January 1, 2003; 12(1): 47 - 65. [Abstract] [PDF]
|
|
|
|
 |
|
 H. M. Richards, M. E. Reid, and G. C. M. Watt Socioeconomic variations in responses to chest pain: qualitative study BMJ, June 1, 2002; 324(7349): 1308 - 1308. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. A Chew-Graham, C. R May, and M. S Perry Qualitative research and the problem of judgement: lessons from interviewing fellow professionals Fam. Pract., June 1, 2002; 19(3): 285 - 289. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. M. Richards and L. J. Schwartz Ethics of qualitative research: are there special issues for health services research? Fam. Pract., April 1, 2002; 19(2): 135 - 139. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Steven, J. Morrison, and N. Drummond Lay versus professional motivation for asthma treatment: a cross-sectional, qualitative study in a single Glasgow general practice Fam. Pract., April 1, 2002; 19(2): 172 - 177. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||