Family Practice Vol. 19, No. 6, 623-631
© Oxford University Press 2002
What shapes GPs’ work with depressed patients? A qualitative interview study
a The NEPI Foundation, Malmö and
b the Department of Family Medicine, Lund University, Malmö, Sweden.
Dr Stig Andersson, Hermes Vårdcentral, Industrigatan 10, SE-661 33 Säffle, Sweden; E-mail: stig.j.andersson{at}swipnet.se
Andersson SJ, Lindberg G and Troein M. What shapes GPs’ work with depressed patients? A qualitative interview study. Family Practice 2002; 19: 623–631.
Received 18 October 2001; Accepted 16 July 2002.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
Background. The ways that GPs treat depressed patients have been criticized in a number of studies.
Objective. To explore factors that shape how GPs work with depressed patients.
Methods. Seventeen GPs from the county of Örebro, Sweden participated in a qualitative semi-structured interview study. GPs’ conceptions of factors shaping their way of working with depressed patients, especially continuing medical education (CME), commercial information, inter-collegial support, collaboration with psychiatrists and GPs’ gender were recorded.
Results. Private life experiences as well as professional experiences from family medicine were more often stressed as formative factors than university education and training in psychiatry. Groups of GPs discussisng the doctor–patient relationship set out from real cases (Balint groups) and CME groups were regarded as good forms of education. Most GPs considered company-sponsored lectures valuable. Commercial drug information was seen as more powerful than non-commercial information and GPs wished for more non-commercial information. Collaboration with psychiatry consultants was perceived as insufficient, and GPs felt a need for more inter-collegial support. Traditional female qualities were generally seen as advantageous in the work with depressed patients.
Conclusions. Many GPs consider personal qualities and experiences, including those of gender, to be more influential than academic education and professional literature. This reflects a preference for individual ‘tacit knowledge’. Although tacit knowledge is indispensable in consultations, the low priority given to theoretical CME may make GPs less inclined to make optimal use of different therapeutic alternatives and also less critical of commercial marketing. CME on depression should start with GPs’ individual tacit knowledge and assume a more independent stance from the drug industry.
Keywords. Depressive disorder, general practitioner, interviews.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
The way a physician works is influenced not only by formal education and professional guidelines, but also by individual experiences from their private and professional life. This is most apparent when examining GPs dealing with complex human situations, e.g. treating depression.
In Sweden, postgraduate training to become a GP requires work in primary care as well as different clinical specialities for five years. Most GP training programs also include six months of residency in psychiatry. Throughout their professional lives, GPs in Sweden are offered various forms of continuing medical education (CME). Drug companies organize and sponsor many CME courses, and there are also courses organized by the county councils, the largest employer of GPs.
Most patients with depressive disorders are treated by GPs. However, the ways that GPs treat depressed patients have been criticized in a number of studies.1–8 It has been argued that, despite the high number of patients with depression in primary care, the way that GPs work is not adapted to the needs of such patients. Reports of the insufficient implementation of guidelines have raised questions about the ability of CME to influence GPs’ working methods in a more scientifically-based direction.9–11
These circumstances have fostered interest in GPs’ conceptions of factors that shape how they work with depressed patients. The aim of the present study was to survey the conceptions of a selected number of GPs.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
We used a qualitative approach with a semi-structured interview scheme of open questions (see Appendix). The sampling procedure aimed to achieve a broad variation in conceptions. Consequently, the sample was intended to represent GPs with different preconditions. Looking for female and male GPs of different ages working in urban and rural areas as well as in different forms of primary care, we selected 17 informants from a list of all the 130 GPs working in the county of Örebro. The informants included eight women and nine men aged 40 to 60 years. Three of the selected physicians refused participation and were replaced by three others. Fourteen of the chosen GPs were employed by the county council; three others were in private practice. Four GPs worked in the city of Örebro (population 130 000), while the rest practised in different smaller communities. Two GPs were immigrants. All but one of the interviewed GPs were unknown by the authors. The interviews were performed between August and November, 1997.
The interview questions were principally the result of the interviewer’s understanding of the research field. The interviewer (SA) has a long experience as a GP and a special interest in psychiatric issues in the practice of family medicine. The questions introduced different issues in the interviews. The informants interpreted the questions in their own way, and the interviewer encouraged the informants to clarify their thoughts in a number of follow-up questions.
The interviews were tape-recorded and transcribed verbatim. All interviews were performed and transcribed by one of the authors (SA). This study was approved by the regional research ethics committee before being conducted.
Data analysis
The database is composed of dialogue texts. These texts were first scrutinized by one of the authors (SA) for any statements related to GPs’ work with depressed patients and also to work-influencing conditions. Such statements were selected in the form of dialogue excerpts serving as examples of GPs’ conceptions. A conception has been regarded as a fundamental notion with components of personal experience, actual ideas and cultural descent.12 Informants’ conceptions were grouped into categories according to type, and each category was named after the common essence of the type of conception represented. These categories constituted the final result of the study, and they were grouped according to different issues in the interviews.13
To verify that the conceptions and findings described were an accurate reflection of the database, the second author (MT) independently scrutinized the texts of every second interview and conducted the same form of data analysis as the first author.14 A few subsequent ambiguities in the analyses were discussed until the authors reached consensus.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
The informants’ conceptions, grouped into categories, are presented in six tables, one for each issue of the research area. Sample quotations demonstrating informants’ conceptions are also included.
Table 1
contains conceptions of the general factors shaping the way a physician works. The life experiences of individuals as well as professional experiences from family medicine were stressed more often than university education and vocational training in psychiatry. Some informants considered their university education and training in psychiatry as relevant to some extent, but there were also informants who considered both to be worthless and even a hindrance. Working conditions in practice and the introduction of new drugs were also mentioned as formative elements.
|
Professional literature as well as non-medical literature were reported to influence working methods. One informant mentioned the importance of a fellow colleague’s mentorship. Two informants viewed religious faith as a possible influence.
Table 2 comprises conceptions of CME. Groups of GPs discussing the doctor–patient relationship set out from real cases (Balint groups) and CME groups were regarded as a good forum for further education because participants got the opportunity to exchange experiences along with exposure to additional theoretical knowledge and support. Courses on therapeutic attitudes and mentorship were other examples of valuable education mentioned.
|
Professional literature was reported as an important CME source. The most frequently mentioned journals and books were The Journal of Swedish Medical Association (Läkartidningen) and Information from the Swedish Medical Products Agency (Information från Läkemedelsverket), The Swedish Therapy Handbook from the National Corporation of Swedish Pharmacies (Läkemedelsboken), as well as periodicals and books published by pharmaceutical companies.
Some informants expressed a desire for more non-commercial medical education. Several informants expressed a need for review of the basic literature on depression, but two informants felt they knew enough. For some, knowledge about antidepressant drugs was regarded as a small fraction of the knowledge necessary to treat depressive disorders. Only one informant mentioned information overload as a problem.
Table 3 shows conceptions of the information and courses provided by pharmaceutical companies. Most informants considered company-sponsored lectures and courses as a valuable part of their education on depression management. Other informants regarded these lectures and courses mainly as examples of commercial promotion. Several pointed out that pharmaceutical company representatives are the first to inform about new drugs. Some viewed this commercial information as their main education on anti-depressive drug therapy, and one informant stated that such information formed the whole of his postgraduate education about depression.
|
Only one GP considered himself free of the influence of commercial information, whereas several others expressed difficulty in remaining objective. These informants also stressed that commercial drug information is much more powerful than non-commercial information.
Some informants agreed that the media’s great attention to new anti-depressants had influenced patients’ attitudes and, indirectly, physicians’ treatment strategies.
Table 4 presents the informants’ conceptions of clinical practice with depressed patients. All informants perceived this work in a predominantly positive way. Several regarded patient confidence as one benefit. Other informants emphasized the experience of reward when a depressed patient recovers, and some also felt that they had learned a great deal from their patients. For one informant, work with depressive patients gave him a sense of identity as a doctor. There were also informants who experienced strain and anxiety as a result of their contacts with depressive patients, especially those patients who also suffered from chronic pains. One informant said that a patient’s problems sometimes reminded him of his own painful experiences and thereby made problem-solving difficult.
|
Table 5
shows conceptions of support between colleagues as well as the level of collaboration between GPs and psychiatrists. Informants with experience from CME groups and Balint groups noted a sense of collegial support in these groups. There were also informants from health centres in which physicians supported each other. Other informants had a colleague to go to for support. Staff meetings with the health centre’s social worker were also reported to be supportive. One informant said that his assignment as a tutor had created a supportive relationship between him and his assistant physician. However, others reported very little contact with colleagues and several expressed desire for better support.
|
Informants’ conceptions of GP–psychiatrist co-operation were often negative, and most informants thought that psychiatrists offered little help. Regular meetings or consultations with psychiatrists were uncommon. Some had tried to consult with a psychiatrist regularly, but with poor results. Even in cases where a practice was situated close to a psychiatric centre, there were few contacts between GPs and psychiatrists. However, some informants believed that a GP’s ability to treat would be enhanced by regular support from psychiatrists.
Table 6 demonstrates conceptions of the significance of gender in treatment situations. A small number of male informants stated that the gender of the physician was of no importance. Most informants, however, considered gender to play a role in consultations. One female physician voiced a feministic perspective on gender.
|
A common view was that qualities defined as ‘female’ —a greater capacity for empathy and better listening skills—are advantageous in consultations. Still, both female and male informants believed that many male GPs may have these same qualities while certain female GPs may not.
Informants of both genders believed that patients generally expect more empathy from female physicians, and that this expectation influences the patient’s selection of physician. It was also pointed out that men’s and women’s different ways of presenting their problems may hinder communication when a patient and a doctor are of different genders. A male informant speculated that female patients who had experienced sexual or physical assault by men would have difficulty communicating with a male physician.
Certain male informants and one female informant believed that it might be advantageous for the GP to be a man as men are more capable of setting limits and organising their work and are thus more able to fend for themselves. One male informant claimed that a strong biomedical attitude is most often a characteristic of male physicians.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
Method considerations
We chose a qualitative approach in order to obtain knowledge about the GPs’ personal experiences and ideas, and we relied upon semi-structured interviews in order to assure a broader range of responses.15–17 Our strategic sampling procedure aimed to include representatives from different types of GPs, taking into consideration factors as gender, age, type of community of working place, form of primary care and native-country. We think the procedure enabled us to achieve a broad variation in conceptions.18 Three invited GPs refused interviews and were replaced by other GPs from the list. We have no idea what differences of conceptions this exchange created. Because our purpose was not comparison of conceptions but variation and difference of conceptions, the conceptions of any GP have the same value for the result.
Open interviews rely on narrative information, which, in turn, relies on the willingness of informants to reveal their feelings, opinions and beliefs to the interviewer. It is likely that the interviewer’s position as a colleague increased informants’ confidence and hence their willingness to communicate. On the other hand, the interviewer’s experience of primary care and his peer status may have created a shared conceptual blindness allowing the interviewer’s attitudes to govern the dialogue and interpretations and thus influence the result.14,18
The interview schedule, constructed by the authors who are all GPs, may have entailed a similar risk. However, the great variation in conceptions would seem to indicate that the informants were not unduly influenced by their interviewer.
We have used the word ‘conception’ in a rather broad way. In some instances, the word suggests vague or less than well thought-out opinions, in others, ‘conception’ signifies more fundamental presumptions or attitudes. Some conceptions may also be rationalizations based on informants’ existent clinical habits.
The tables offer rather detailed information of the conceptions identified and add to the transferability of the study. To verify that the conceptions described were derived from the database, two of the authors independently made the same content analysis and coding procedure on eight interview texts. Full consensus was achieved on all eight interview texts, thus indicating good confirmability.14
Factors shaping GPs’ work
The importance that informants place on individual attitudes and experiences suggests that GPs’ work with depressed patients is characterized by a high degree of personal commitment. As several informants indicated, this may make it difficult for the physician to set boundaries between the professional and the private. When professionals rely primarily on personal experiences as grounds for clinical assessment and intervention, they demonstrate tacit preference for ‘knowledge in action’ over inquiring scientific knowledge.19 This behaviour may be interpreted as an expression of the practising physicians’ scepticism regarding the usefulness of scientific literature20 and, perhaps, as a feature of amateurism. Schön’s observation that practitioners are often confronted with problems which are not easily translated into medical problem settings, makes the informants’ scepticism about scientific literature understandable.21 This scepticism is also understandable when considering that the result of a consultation depends on successful communication with the patient.22 The ability to communicate is not acquired through reading scientific articles, but rather through individual experience, guided training, reflective reading and so forth. The significant role of humanism in medical practice was reaffirmed by the informants who emphasized non-medical reading and religious faith as formative agents in the practice of medicine.
CME
Swedish medical journals as well as periodicals and books from pharmaceutical companies were mentioned as examples of professional reading whereas no international medical journal was mentioned. One informant commented on the overload of information, but no informant demanded improved methods for the systematic assessment and management of information or mentioned their view on evidence-based medicine. Some informants’ rather weak emphasis on reading accords with research showing that physicians seek colleagues’ advice more frequently than they consult printed sources.23,24 This would strongly suggest that physicians are looking not only for medical knowledge, but also for guidance, feedback and psychological support. It is important to satisfy these needs when introducing new clinical behaviour.25 Presumably, a CME group represents a proper forum to discuss complex clinical questions.
The role of commercial information
Physicians often deny the influence of commercial sources on their decisions to prescribe. However, these sources have more influence on physicians’ conceptions of the efficacy of drugs than non-commercial science-based information.26 The considerable influence of drug advertising on the prescribing habits of American physicians has been shown in the case of calcium channel blockers.27 Moreover, a recent meta-analysis confirmed the influence of the pharmaceutical industry on the prescribing habits and professional behaviour of physicians.28
Informants’ conceptions illustrated the impact of commercial marketing on GPs’ continuing medical education. For physicians, this produces information overload and may create a dilemma about the objectivity and validity of their knowledge. Several informants pointed out quantitative superiority of the commercial information.
Inter-collegial support and collaboration
Schön proposed in 1983 that better correspondence between the practitioner’s problem situations and scientific-based knowledge also presupposes inter-collegial discussions and co-operation.21 In 1996, Smith suggested stronger collegial support to meet the needs of practising physicians in their work.25 The timelessness of both proposals is illustrated by informants’ expressed need for more support and greater co-operation. Strong support was reported only by a few informants.
Negative conceptions regarding collaboration between GPs and psychiatrists may reveal a serious problem. It is possible that the informant’s disappointment is based on unrealistic ideas about the sort of problem solving psychiatrists may offer. Nevertheless, the difficulties involved in collaborations between GPs and psychiatrists may equally reflect insufficient organization and a mutual lack of understanding between GPs and psychiatrists about each other’s working conditions. In comparison with the patients of psychiatrists, the depressed patients of GPs often have less serious depressions but more co-morbidity and often negative attitudes towards psychiatric treatment.2,29–31
Gender perspective
Informants generally viewed traditionally ‘female’ characteristics as advantageous in consultations, but a number of informants of both genders emphasized the overriding importance of the physician’s personal qualities, regardless of gender. This is generally in agreement with published studies. Female physicians give more time to each patient, and their consultations contain more affective talk, emotional responses, social concern and partnership building than men’s consultations.32,33 A Californian study demonstrated that female physicians’ consultations contained more preventive services and family information than the consultations of male physicians.34 In addition, there are studies reporting more patient satisfaction with female physicians.34,35 There is, however, also a study that failed to demonstrate any difference in the communication skills of female and male physicians.36
The conception that consultations across the gender border may complicate communication has received support from a study,37 but it is not supported in another study on co-operative language in consultations.36,38
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
The aim of the present interview study was to survey GPs’ conceptions of what forms the way they work with patients with depression. The results indicate that many GPs consider personal qualities and experiences, including that of gender, to be more influential than academic education and professional literature. Such a result reflects a preference for individual ‘tacit knowledge’. Although tacit knowledge is indispensable in consultations, the low priority given to theoretical CME may make GPs less inclined to make optimal use of different therapeutic alternatives and also less critical of commercial marketing.
The results also support proposals that CME on depression should start with GPs’ individual tacit knowledge and assume a more independent stance from the drug industry. Finally, the results of this study suggest that collaboration between GPs and psychiatrists needs to be improved.
|
Appendix |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
Interview schedule, translated from Swedish.
How interested are you in the work with depressed patients?
How do you feel about meeting depressed patients?
What has shaped the way you work with patients with these problems?
What sort of information and education influence your way of working?
Which influence has the drug companies?
Do you discuss problems and patients with your colleagues?
How is co-operation with psychiatrists?
Do you think your gender has importance for this kind of work?
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendixReferences |
|---|
1 Bodlund O. Anxiety and depression go unrecognized in primary care. (In Swedish with summary in English). Läkartidningen 1997; 94: 4612–4618.[Medline]
2 Davis P, Yee RL, Millar J. Accounting for medical variation: the case of prescribing activity in a New Zealand general practice. Soc Sci Med 1994; 39: 367–374.[Web of Science][Medline]
3 MacDonald AJD. Practice observed: Practice Research: Do general practitioners ‘miss’ depression in elderly patients? Br Med J 1986; 292: 1365–1367.
4 Matthews K, Eagles JM, Matthews CA. The use of antidepressant drugs in general practice. Eur J Pharmacol 1993; 45: 205–210.
5 Orrell M et al. Management of depression in the elderly by general practitioners: Use of antidepressants. Fam Pract 1995; 12: 5–11.
6 Pharoah P, Melzer D. Variation in prescribing of hypnotics, anxiolytics and antidepressants between 61 general practices. Br J Gen Pract 1995; 45: 595–599.[Web of Science][Medline]
7 Rosholm J-U, Gram L, Damsbo N, Hallas J. Antidepressant treatment in general practice—An interview study. Scand J Prim Health Care 1995; 13: 281–286.[Web of Science][Medline]
8 Wells K, Katon W, Rogers B, Camp P. Use of Minor Tranquilizers and Antidepressant Medications by Depressed Outpatients: Results From the Medical Outcomes Study. Am J Psychiatry 1994; 151: 694–700.
9 Kendrick T. Why can’t GPs follow guidelines on depression? We must question the basis of the guidelines themselves. Br Med J 2000; 320: 200–201.
10 Thompson C, Kinmonth AL, Stevens L et al. Effects of a clinical-practice guideline and practice-based education on detection and outcome of depression in primary care: Hampshire Depression Project randomized controlled trial. Lancet 2000; 355: 185–191.[Web of Science][Medline]
11 Paykel ES, Priest RG. Recognition and management of depression in general practice: consensus statement. Br Med J 1992; 305: 1198–1202.
12 Dahlgren LFM. Phenomenography as a Qualitative Approach in Social Pharmacy research. J Soc Adm Pharm 1991; 8: 150–156.
13 Alexandersson M. Analysis and interpretation of the interview. (Analys och tolkning av intervjun.). In Starrin B, Svensson PG (eds). Kvalitativ metod och vetenskapsteori. 1994. Lund: Studentlitteratur, 125–131.
14 Hamberg K, Johansson E, Lindgren G, Westman G. Scientific rigour in qualitative research—examples from a study of women’s health in family practice. Fam Pract 1994; 11: 176–181.
15 Britten N, Jones R, Murphy E, Stacy R. Qualitative research methods in general practice and primary care. Fam Pract 1995; 12: 104–114.
16 Green J, Britten N. Qualitative research and evidence based medicine. Br Med J 1998; 316: 1230–1232.
17 Faltermaier T. Why public health research needs qualitative approaches. Subjects and methods in change. Eur J Pub Health 1997; 7: 357–363.
18 Malterud K. Hva er kvalitative metoder? Kvalitative metoder i medicinsk forskning. Aurskog: Tano Aschehoug, 1996: 30–63.
19 Molinder B. Tacit Knowledge and Silenced Knowledge: Fundamental Problems and Controversies. In Göranzon B, Florin M (eds). Skill and Education. Reflection and Experience. London: Springer Verlag, 1992: 9–31.
20 Lennarson Greer A. The state of the art versus the state of the science. Int J of Technology Assessment in Health Care 1988; 4: 5–26.
21 Schön DA. The reflective practitioner. How professionals think in action. London: Basic Press, 1983.
22 Pendelton D, Schofield T, Tate P, Havelock P. The Consultation. An approach to learning and teaching. Oxford General Practice Series 6. Oxford: Oxford University Press, 1984.
23 Covell D, Uman GC, Manning PR. Information Needs in Office Practice: Are They Being Met? Ann Intern Med 1985; 103: 596–599.
24 Timka T, Ekström M, Bjurulf P. Information Needs and Information Seeking Behaviour in Primary Health Care. Scand J Prim Health Care 1989; 7: 105–109.[Medline]
25 Smith R. What clinical information do doctors need? Br Med J 1996; 313: 1062–1068.
26 Avorn J, Chen M, Hartley R. Scientific versus Commercial Sources of Influence on the Prescribing Behaviour of Physicians. Am J Med 1982; 73: 4–8.[Web of Science][Medline]
27 Wang TJ, Ausiello JC, Stafford RS. Trends in Antihypertensive Drug Advertising 1985–1996. Circulation 1999; 99: 2055–2057.
28 Wazana A. Physicians and the Pharmaceutical Industry. Is a Gift Just a Gift? JAMA 2000; 283: 373–380.
29 Chinburapa V, Larson LN, Brucks M, Draugalis JL, Bootman JL, Puto CP. Physician prescribing decisions: the effects of situational involvement and task complexity on information acquisition and decision making. Soc Sci Med 1993; 36: 1473–1482.[Web of Science][Medline]
30 Schwenk TL, Klinkman MS, Coyne JC. Depression in the family physician’s office: what the psychiatrist needs to know. The Michigan Depression Project. J Clin Psychiatry 1998; 59: 94–100.
31 Priest RG, Vize C, Roberts A, Roberts M, Tylee A. Lay people’s attitudes to treatment of depression: Results of opinion poll for Defeat Depression Campaign just before its launch. Br Med J 1996; 313: 858–859.
32 Maheux B, Dufort F, Béland F, Jaques A, Léversque A. Female medical practitioners. More preventive and patient oriented? Med Care 1990; 28: 87–92.[Web of Science][Medline]
33 Roter D, Lipkin M, Korsgaardt A. Sex differences in patients’ and physicians’ communication during primary care medical visits. Med Care 1991; 29: 1083–1093.[Web of Science][Medline]
34 Bertakis K, Helms LJ, Callaham EJ, Azari R, Robbins JA. The influence of gender on physician practice style. Med Care 1995; 33: 407–416.[Web of Science][Medline]
35 Delgado A, Lopez-Fernandez LA, de Dios Luna J. Influence of the doctor’s gender in the satisfaction of the users. Med Care 1993; 31: 795–800.[Web of Science][Medline]
36 Skelton JR, Hobbs FDR. Descriptive study of cooperative language in primary care consultations by male and female doctors. Br Med J 1999; 318: 576–579.
37 Kelly JM. Sex preference in patient selection of a family physician. J Fam Pract 1980; 11: 427–430.[Web of Science][Medline]
38 Wynn R. The number of word spoken by male doctors and male and female patients in a sample of British general practitioner-patient consultations. Scand J Prim Health Care 1997; 15: 184–187.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C. E Newman, S. C Kippax, L. Mao, G. D Rogers, D. C Saltman, and M. R Kidd Features of the management of depression in gay men and men with HIV from the perspective of Australian GPs Fam. Pract., February 1, 2009; 26(1): 27 - 33. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Iliffe Review: screening instruments had sensitivities of 67-100% and specificities of 53-98% for detecting major depression in older primary care patients Evid. Based Med., July 1, 2004; 9(4): 124 - 124. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||