How do we know the diagnostic manuals used in mental health are reliable and valid? Are there ways in which we can improve the reliability and validity of diagnosis and classification in schizophrenia? Both the DSM (Diagnostic and Statistical Manual of Mental Disorders) and the ICD (International Statistical Classification of Diseases and Related Health Problems) require high levels of reliability and validity in diagnosing and classifying schizophrenia.
Explore our app and discover over 50 million learning materials for free.
Lerne mit deinen Freunden und bleibe auf dem richtigen Kurs mit deinen persönlichen Lernstatistiken
Jetzt kostenlos anmeldenHow do we know the diagnostic manuals used in mental health are reliable and valid? Are there ways in which we can improve the reliability and validity of diagnosis and classification in schizophrenia? Both the DSM (Diagnostic and Statistical Manual of Mental Disorders) and the ICD (International Statistical Classification of Diseases and Related Health Problems) require high levels of reliability and validity in diagnosing and classifying schizophrenia.
Reliability and validity in diagnosis in psychology are essential for the DSM and ICD to be effective tools for psychiatrists to diagnose patients.
Schizophrenia is usually diagnosed between the ages of 20 to 30 for both men and women, although symptoms can appear long before then, sometimes even in early childhood.
Fig. 1 - Reliability and validity are important aspects of a diagnostic manual.
Reliability in research refers to the consistency of achieving the same results if different researchers were to perform the same experiment over and over again. A study is considered reliable if it can be repeated and give the same results.
Reliability in diagnosis refers to the level of agreement that different psychiatrists can achieve on a single diagnosis for an individual, both over time and across cultures, provided that the disorder’s symptoms have not changed.
It must be repeatable and consistent.
The diagnosis must be stable over time if the definition of the disorder has not changed.
To ensure a diagnosis and its classification are reliable, test-retest reliability is usually measured by the intraclass correlation (ICC) and the inter-rater reliability test, also measured as the kappa score. Both are statistical results that examiners can use to confirm the reliability of their results.
If the kappa classification has a score of 0.7 or more, it is considered reliable.
Let's explore an example of test-retest reliability in schizophrenia tests.
To measure the reliability of something, the test-retest method is often used. In this method, a test is performed twice on one person at two different times.Lee et al. (2011) examined two tests used to measure switched and sustained attention in patients with schizophrenia for test-retest reliability.
147 Participants were tested with an interval of 1 week between tests. The results indicated both tests had good test-retest reliability and were stable measures. They both had high ICC values, a commonly used measure of test-retest reliability in studies.
Let's explore an example of inter-rater reliability measures in schizophrenia.
Inter-rater reliability is when different psychiatrists should come to the same conclusion or diagnosis when diagnosing a person. The kappa score is a measure of inter-rater reliability.
In Cheniaux (2009), two psychiatrists assessed 100 patients using the DSM and the ICD. They found that inter-rater reliability was poor, as one psychiatrist diagnosed 26 patients with schizophrenia using the DSM while diagnosing 44 patients with schizophrenia using the ICD.
As a result, schizophrenia is underdiagnosed or overdiagnosed depending on which system is used.
Similarly, Regier (2013) found the DSM edition at that time had a kappa score of 0.45, which is quite low and below the satisfactory requirement of 0.7, indicating a weakness in schizophrenia diagnosis.
Rosenhan’s (1973) study, one of the most influential studies affecting the diagnosis and treatment of psychiatric disorders today, highlighted the problems with reliability in the diagnosis of psychiatric disorders. Rosenhan emphasised that psychiatric hospital staff were somewhat desensitised and unaware of how drastic the experience of being admitted as a patient can be.In this study, Rosenhan himself and seven healthy individuals went to 12 different psychiatric hospitals.
The pseudopatients asked the hospitals for an appointment and then complained upon arrival that they heard unfamiliar but same-sex voices, specifically:
These words were explicitly chosen to show the pseudopatient has symptoms such as hallucinations, and to suggest that the pseudopatient believes their life is ‘empty and hollow’.
Their history showed no signs of pathological problems.
Once admitted, the pseudopatients immediately ceased all symptomatic behaviours. They behaved as they normally would and talked to other patients and staff as they would before they were admitted as pseudopatients. They were not told when they could leave the clinic. Instead, they were told they would have to convince the staff that they were sane to leave the clinic.
Pseudopatients who were asked how they were doing told staff they were fine and no longer had symptoms. They behaved accordingly and followed the rules. They also noted observations about the hospitals, secretly at first, but when it was obvious that no one cared or took note, they wrote them down without secrecy.
Fig. 2 - In Rosenhan's study, patients behaved normally after entering the hospital.
All but one of the patients were discharged with a diagnosis of schizophrenia in remission. Their length of stay ranged from 7 to 52 days (an average of 19 days). They were not classified as ‘sane’; the label of schizophrenia remained. The pseudopatients were not detected during their stay, although they remained sane throughout. There was a lack of adequate observation, but Rosenhan attributes this to a failure of the hospital rather than the patient and the lack of opportunity.
Interestingly, actual patients at the hospital recognised the pseudopatients as sane. About 35 of 118 patients recognised the sanity of the pseudopatients and even suggested their real occupation:
Patients were more likely than staff to recognise normality, even though the pseudopatient insisted he was sick.
Rosenhah suspects this failure to recognise sanity is due to type 2 errors, also known as false-positive diagnoses. Rosenhan explained that diagnoses in this area of health care are neither useful nor reliable and represent a depressing and frightening reality that we must combat. Hospitals of this type cannot identify and treat these behaviours, and they also convey a sense of powerlessness.
Rosenhan stated:
We now know that we cannot distinguish sanity from insanity.¹
The study impacted the way we treat patients with mental disorders today. It led to a condemnation of the use of institutionalised hospitals and a focus on care in the community.
Validity is the legitimacy of a test, i.e., whether what the psychiatrist uses to diagnose a person measures what it purports to measure.
If it is valid, the diagnosis represents something real and is different from other disorders.Cronbach and Meehl (1955) established several types of validity.
Construct validity – one can determine if a measurement instrument represents what it intends to measure.
Content validity – the test represents all construct areas, i.e., all areas of what one measures.
The following issues cause problems with validity and reliability:
Comorbidity can be a problem in any type of disorder, but what exactly is comorbidity?
Comorbidity is when two or more disorders coincide in a person.
It is not to be confused with symptom overlap.
Fig. 3 - Comorbidity in schizophrenia can be a common problem in diagnosis.
If two disorders can occur in one person, the validity of the diagnosis begs the question, who is to say that it is only one disease and not several?Unfortunately, in the case of schizophrenia, there are many comorbidities with disorders such as OCD (obsessive-compulsive disorder) and depression. As a result, there are significant problems with the disorder’s validity.Buckley et al. (2009) found that comorbid depression occurs in 50% of patients with schizophrenia.
This finding raises some concerns:
Is it because psychiatrists or diagnostic manuals are poor at recognising the difference between these two disorders?
If they are so similar that there are problems with comorbidity, why can not they be considered a single disorder?
Since there is no truly distinct disorder, it can be difficult to diagnose and treat schizophrenia with confidence.
As there are an abundance of symptoms in schizophrenia as well as other mental health disorders, diagnosis and classification can be tricky.
Symptom overlap is the significant overlap between the symptoms of schizophrenia and other disorders.
Schizophrenia and bipolar disorder are good examples.Bambole et al. (2013) noted that patients diagnosed with schizophrenia and bipolar disorder have both positive and negative symptoms, making it difficult to distinguish between them.If these two disorders have too many symptoms in common, it questions the validity of the classification system and thus the validity of the diagnosis.For example, if a patient has these symptoms:
Avolition
Delusions of grandeur
The ICD may diagnose schizophrenia, while the DSM suggests bipolar disorder, indicating an apparent problem in the way these manuals claim to measure a disorder, and, therefore, a weakness in their validity.
Gender bias is problematic in many areas of research, and it is not exempt from psychological research.
Gender bias in the classification and diagnosis of schizophrenia occurs when the diagnosis is dependent on the patient’s gender. What symptoms they display and explain is usually irrelevant, as judgments are made based on their gender.
There may be gender-specific criteria, or there may be a bias based on stereotypes that the clinician believes in.
In the study by Loring and Powell (1988), 290 male and female psychiatrists were presented with two case studies of patient behaviour. When the patient was described as male, 56% of them were diagnosed with schizophrenia. However, only 20% diagnosed the patient as schizophrenic when described as female.There was a clear gender bias, but it was not present in the female psychiatrists when they made the diagnosis, suggesting that the gender of both the patient and the psychiatrist is a factor.
Longenecker (2010) found that more men than women have been diagnosed with schizophrenia since 1980.Are men genetically more prone to schizophrenia? Or do women function better in their daily lives than men with the disorder? If women present with the same symptoms but cope better because they have more support or continue to work and maintain relationships, there could be a bias among treating physicians and underdiagnosis in women.
Cotton et al. (2009) hypothesised that women cope better with their symptoms because they function better and may have more support. They function better on an interpersonal level and may not be diagnosed where men would be diagnosed with similar symptoms.Better interpersonal functioning may lead psychiatrists to underdiagnose women because the symptoms are masked and buried under-functioning routines. Some thus believe the case is too mild for a specific diagnosis, even though men would have been diagnosed with the same level of symptoms.
Cultures differ across the world, and research has long had issues with the generalisability of data because of this.
Fig. 4 - Culture across the world differ in their approaches to diagnosing schizophrenia, flaticon.com/authors/flat-icons
Cultural bias exists because some psychiatrists diagnose a patient differently if the patient is from a different cultural background. They judge the patient according to what they think is acceptable in their own culture, rather than looking at the patient objectively and considering the patient’s culture and what is acceptable in the patient’s cultural views.
Tortelli et al. (2015) noted that despite improved study methods over time, there has been a higher rate of psychotic disorders in Black Caribbean ethnic groups in England over the past 60 years.
Ineichen et al. (1984) suggested people of West Indian origin were overdiagnosed with schizophrenia when assessed by White doctors in Bristol, possibly due to their ethnic background and culture.
Minsky et al. (2003) found that African Americans were more likely to be diagnosed with schizophrenic disorder than Latinos and European Americans.
Copeland (1971) also found that when a large group of psychiatrists from the US and the UK were presented with descriptions of a patient who exhibited certain behaviours and symptoms, American psychologists used a much wider variety of clinical conditions than British psychologists, often overdiagnosing them in comparison.
Why is that so?
There may be a cultural problem with understanding what is ‘normal’ behaviour because in some cultures. In Africa, hearing voices and doing things that are normally considered positive symptoms of schizophrenia are more accepted than in other cultures where this is considered bizarre and irrational.
Racial biases could also play a role. When certain ethnic minorities are overdiagnosed compared to White people, especially by White psychiatrists, it indicates a high degree of miscommunication, bias, and lack of understanding of the patient and their symptoms.
Since countries of origin generally do not support these high diagnosis rates, there is no genetic susceptibility. Why are African Americans in another country more likely to be diagnosed with schizophrenia than people in Africa and the West Indies?
In light of the above, it is also important to acknowledge that:
The reliability of schizophrenia diagnosis depends on the psychiatrist diagnosing the person and the manual used. Overall, diagnosing schizophrenia reliably has improved over time as the guides have been revised, but some issues still require consideration, such as biases and issues with comorbidity and symptom overlap.
Following the DSM or ICD by a certified, experienced psychiatrist would be the best, most reliable method to diagnose schizophrenia. However, both are still subject to biases, comorbidity, and symptom overlap.
Reliability is the level of agreement that different psychiatrists can achieve on a single diagnosis for an individual, both over time and across cultures, provided that the disorder’s symptoms do not change.
Validity is the legitimacy of a test, i.e., whether what the psychiatrist uses to diagnose a person measures what it purports to measure.
Multiple aspects can worsen schizophrenia, such as unstable, hostile households and family relationships (family dysfunctions), a lack of appropriate treatment, or not seeking treatment at all.
Schizophrenia is usually diagnosed between the ages of 20 to 30 for both men and women, although symptoms can appear long before then, sometimes even in early childhood.
What is the definition of reliability in psychology?
Reliability is the level of agreement different psychiatrists can reach on a single diagnosis for one individual, across both time and cultures, given that there is no change in the symptoms of the disorder.
What is the definition of validity in psychology?
Validity is the legitimacy of a test, i.e., whether what the psychiatrist uses to diagnose a person measures what it purports to measure. If it is valid, the diagnosis represents something real and is different from other disorders.
Name two types of validity in psychology.
Construct validity and content validity.
What is the definition of cultural bias in psychology?
Cultural bias exists because some psychiatrists diagnose a patient differently if the patient is from a different cultural background. They judge the patient according to what they think is acceptable in their own culture, rather than looking at the patient objectively and considering the patient’s culture and what is acceptable in the patient’s cultural views.
If a disorder has issues with reliability and validity, what could that mean?
It can mean the disorder may not be justified in being a diagnosable disease.
Already have an account? Log in
Open in AppThe first learning app that truly has everything you need to ace your exams in one place
Sign up to highlight and take notes. It’s 100% free.
Save explanations to your personalised space and access them anytime, anywhere!
Sign up with Email Sign up with AppleBy signing up, you agree to the Terms and Conditions and the Privacy Policy of StudySmarter.
Already have an account? Log in
Already have an account? Log in
The first learning app that truly has everything you need to ace your exams in one place
Already have an account? Log in