DISC Research and Validation
DISC is one of the most researched and studied behavioral assessments available. It’s also the oldest personality assessment.
William Moulton Marston outlined a theory of behavior in 1928. Marston theorized that he could categorize people based on learned behavioral traits. In his book, Emotions of Normal People, he first identified four behavioral styles (Dominance, Submissiveness, Inducement, and Compliance).
However, Marston didn’t create an assessment from his theory, and his work is now in the public domain. For that reason, you will find many DISC Profiles and assessments with varying levels of research.
We offer two versions of DISC that two different organizations publish, and this article goes into specifics for each product we offer:
These two assessments measure the four primary styles of DISC, but offer their own nuances in terms of how they relate the concepts into a person’s personality profile.
In this article, we will outline how DISC assessments are researched. We will cover:
- What is being researched?
- What you should look for when evaluating DISC Research reports
Our goal overall is that you understand a little more about the work behind creating an experience where you learn more about yourself and others when you take this assessment.
DISC Research: What does it mean?
Assessment research can cover lots of different areas. Often, the use of the assessment will determine what research is needed for it to go to market.
Most DISC research focuses on two areas of research: Reliability and Validity.
- Reliability is determined by understanding if an instrument measures consistently and diligently. Ideally, you don’t want to see a huge change in your score if you take the assessment on Monday and then again on Friday.
- Validity is determined by identifying if the tool measures what it is supposed to measure.
While no DISC Assessment will ever be perfectly reliable or perfectly valid, it’s important the weaknesses of the tool are understood. When looking at assessment research, the best question is, “How much evidence is there for the reliability of this tool?” rather than, “Is this tool reliable?”
Reliability
Is the tool consistent? Are the measurements dependable?
Validity
Does the tool measure what it's supposed to measure?
DISC seeks to measure a person’s response to a series of statements about themselves. Their responses are used to calculate a score based on the below four scales:
- D-Style: Dominance
- I-Style: Influencing
- S-Style: Steadiness
- C-Style: Conscientious
When conducting research for reliability and validity, the researchers were looking to understand if the assessment correctly identified a person’s DISC style and if it was identified correctly consistently.
DISC Research: Testing for reliability
When testing for reliability for DISC or any four-quadrant models, it’s important to recognize that these instruments are self-reported. That is, the person taking the assessment identifies their own traits (qualities).
This is important to keep in mind because a person’s final assessment score will never be absolute. As our perception of ourselves changes, so will the final score of the assessment. That said, DISC instruments or any other similar type of personality instrument doesn’t measure quantities (like blood pressure or cholesterol). Instead, it measures qualities that individuals report about themselves.
You probably know this, but people are notoriously inconsistent in their perceptions. We change our view when we are hungry, sad, or because we didn’t get enough sleep.
So, how do researchers determine the right level of inconsistency when trying to determine if an assessment is consistent?
Statistics.
Specifically a specific type of statistical analysis called Cronbach’s Alpha or Coefficient Alpha.
This statistical tool works by finding how a large number of people’s scores correlate between each of the scales. It is measured by correlating the score for each scale item with the total score for each test taker.
The Cronbach’s Alpha / Coefficient Alpha represents the number of items in a test, the average covariance between pairs of items, and the variance of the total score.
All of this is represented as a number between 0 and 1. The closer Cronbach’s Alpha is to 1 the higher the covariances are on the assessment. Most researchers recommend a minimum score between 0.70 and 1. Here are the following guidelines:
- Below .70 = Questionable
- Between .70 to .80 = Acceptable
- Between .80 to .90 = Very Good
- Between .90 to 1 = Excellent
So how does DISC Basic measure? Below are the scores for the four primary styles (Dominance, Influencing, Steadiness, Conscientious). You will see that each style except for Influencing was tested with 10,000 unique assessments. Influencing was tested against 3,212 unique assessments:
DISC Basic Research
DISC Basic’s reliability falls within the Very Good to Excellent range because its alpha was found to be between .83 and .93 for each style. Compare these scores to other DISC Reliability Scores.
DISC Research: Testing for validity
How do we know what an assessment is really measuring? Researchers use a measurement that is able to find a pattern of correlations. The measurement is called construct validity.
Construct validity essentially seeks to understand if a test or an assessment measures what it’s supposed to measure. It answers the question: “Does the measure behave like the theory says a measure of that construct should behave”?
In regards to DISC, you would hope to see near-perfect alignment (strong correlation) when you are comparing two test results for the same style. Likewise, you would hope to see different styles have a weak correlation with each other. Afterall, if a person takes a DISC assessment and they go from having a strongly correlated D-Style to a strongly correlated S-Style, the assessment isn’t valid.
The validity correlations are measured between the numbers 0 and 1. When two correlated styles score closer to 0, they are considered to be weakly correlated. The opposite is true when the correlation score is closer to the number 1.
Here is what DISC Basic’s validity correlations showed:
The research shows that there is alignment when it’s needed and weaker ties when comparing the various style to each other. Overall, DISC Basic is a valid tool and has proven that it measures what it intends to measure.
DISC Research: Testing for discrimination
The Equal Employment Opportunity Commission (EEOC) is a federal agency responsible for setting guidelines for assessment use in the workplace. Often, their regulations are used in hiring and selection, but their rules are also used to ensure assessments don’t show any demographic, racial, or disability bias.
DISC was researched for disparity impacts in the following areas using the EEOC guidelines:
- Gender (Male / Female / LGBTQ)
- Race
- Age
- Veterans or Individuals with disabilities
The disparate impact study analyzed the effects on these groups using the 80% Rule or the 4/5ths Rule. Typically, the 80% rules look to understand if the protected classes are hired by less than 80% of the group with the highest rate. In this study, the 80% rule was applied to the assessment data.
The study compares the mean scores by each protected class and is reviewed to see if its mean ratio values are greater than or less than the 80% guideline. If the comparative scores are less than 80% the study would assume that the assessment disparitely impacts the group that is being review. If the scores are above 80%, the study would conclude that no disparative impact was found.
Below is the list of protected groups compared to the control group (non-protected group):
- Gender: Female, LGBTQ compared to Male
- Ethnicity: African American, Asian, Hawaiian, Latino, Middle Eastern, and Native American compared to Caucasian
- Age: Born before 1945, Baby Boomer, Gen X compared to Under 40
- Veteran or Disabled Status: Disabled, Disabled Veteran, Other Veteran, Vietnam Veteran compared to non-veteran or non-disabled
The study reviews each group for each style (Dominance, Influencing, Steadiness, and Conscientious).
DISC Findings by GENDER
Female & LGBTQ assessment data compared to Male assessment data. Each style was found to have no disparate impact on Gender (ratio scores were higher than 80%).
DISC Findings by ETHNICITY
African American, Asian, Hawaiian or Pacific Islander, Latino or Hispanic, Middle Eastern, and Native American group’s individual assessment data compared against Caucasians assessment data. Each style was found to have no disparate impact on ethnicity. Ratio scores for each style was above 80%.
DISC Findings by AGE
African American, Asian, Hawaiian or Pacific Islander, Latino or Hispanic, Middle Eastern, and Native American groups individual assessment data compared against Caucasians assessment data
DISC Findings by Veteran or Disabled Status
Disabled, Disabled Veteran, other veterans, Vietnam veteran individual assessment data compared against non-veteran or non-disabled assessment data
DISC Ai Research
Crystal Knows publishes the content and manages the technology behind DISC Ai. The tool leverages both a standard assessment process like what is involved with the DISC Basic assessment, and it offers a complex algorithm that allows users to predict a person’s DISC Style based on social media profiles, writing samples, or other text communication (i.e., email).
DISC Ai uses two techniques to determine and predict a person’s profile with around 80% accuracy:
- Text Sample Analysis
- Attribute Analysis
Below is how DISC Ai uses these algorithms to determine a person’s profile.
DISC Ai and Text Sample Analysis
Text sample analysis might be an unfamiliar term for you, but you benefit from this technique every day or even multiple times a day when you use Google or Bing to search for something on the internet.
Text sample analysis is just one part of a section of very vast computer science field called Natural language processing.
This tool detects someone’s personality type by using a n-gram analysis. A ‘gram’ is a sequence of adjacent symbols in a particular order in computer science. This could be words, groups of words, punctuation marks, or more.
The programmers use millions of text samples to train the algorithm to detect patterns and predictable differences that translate into personality types. This can be easier to identify with structured writing samples like resumes or social media profiles.
The trick to using this technique is to sort through various n-grams that don’t offer a definitive clue about someone’s personality. Statements like “managed a team” or words like “customers” aren’t strongly correlated with a particular DISC Style, but here are a few statements that are strongly correlated:
Writing Samples and DISC Correlations:
“Return on Investment”
Strong correlation with the D Personality Type.
“Plain Awesome”
Strong correlation with the I Personality Type.
“Weekly Meetings”
Strong correlation with the S Personality Type.
“Continuously evaluated”
Strong correlation with the C Personality Type.
Text-sample analysis is not immune to bias, just like any standard assessment.
The biggest issue with this technique is ensuring that the sample being evaluated was written by the person you are predicting. Fortunately, this isn’t the only technique used by the algorithm, and another layer helps increase the accuracy of the prediction.
DISC Ai and Attribute Analysis
DISC Ai also uses a method called attribute analysis that, in theory, analyzes more structured data that is about the person. Here are some examples:
- Past and current job titles
- Pas and current employers
- Industries in which the person has worked
- Places where the person has lived
- Schools the person has attended
- Interests the person has
- Topics the person has written or posted about.
Now, it’s important to clarify that taking individual characteristics from this list doesn’t create someone’s personality. As an example, people who post about the NFL, live in Philadelphia, and work for Amazon do not all have the same personality. However, these data points can be combined and weighted to determine where a person might score on the DISC Personality scale.
Below is one example of how this data is used as a single touchpoint. Of the thousands of founders who have taken the DISC Ai personality assessment, 15% had the DI or Initiator archetype. The least common was the Cs or Editor archetype which made up about 1% of respondents.
If the population of founders were evenly distributed among personality types, we would see each type with 6.25% of the sample. That is far from the reality, but it allows us to calculate the probability that any given founder has a particular personality archetype. This allows us to infer that anyone who assigns themself the title of ‘founder’ is twice as likely to be an Initiator (DI) than the average respondent.