How A Career Assessment Test Works
This post first appeared on LinkedIn.
At their most basic level a career assessment test has you (the person trying to figure out what job you are most suited to) answer a series of questions, after which you are presented with one or more occupations that are deemed as ‘suitable’ to you!
These career assessment tests have been around for decades. Perhaps the most popular of such tests are those that fall into the ‘interests inventories’ category; essentially tests that focus on your interests and matching those to occupations.
The four most popular interest inventories are as follows:
- Strong Interest Inventory (SII)
- Self-Directed Search (SDS)
- Campbell Interest and Skill Survey (CISS)
- Kuder Occupational Interest Survey (KOIS)
All of the above career assessment tests can be related to the work by John Holland who developed a theory of careers and vocational choice based upon personality types.
Holland’s theory is based on four main assumptions:
- In our culture, most people can be categorized into six Themes and each person may be characterized by one Theme or some combination.
- Job environments can be divided into these same six Themes and each environment is dominated by a particular type of person. Thus, the personality types of co-workers, as much as job requirements, establish the working tenor of a given occupation.
- People search for environments that let them exercise their skills and abilities, express their attitudes and values, take on problems and roles they find stimulating and satisfying, and avoid chores or responsibilities they find distasteful or formidable.
- Behavior is determined by an interaction between a person’s personality and the characteristics of his or her working environment. Factors such as job performance, satisfaction, and stability are influenced by this interaction.
How a Career Assessment Test really works
Let’s get under the hood of the Strong Interest Inventory to see how these career assessment tests really work.
The person taking the ‘strong’ career assessment test answers 291 questions with each question requiring you to select one of 5 responses (strongly dislike, dislike, indifferent, like, strongly like). These questions are deemed to capture the essence of you; your personality and interests.
Then the magic happens. The tool will then match you to certain themes and ultimately spit out some recommended occupations that are worth further exploration.
How does it do this?
This is where it gets a little bit trickier.
Essentially the tool holds data for 260 different occupations (it’s really only 130 occupations as male occupations and female occupations are separated). Example occupations include:
- Human Resources Specialist
- Personal Financial Advisor
- Elementary School Teacher
These occupations are selected based on data collected ‘from the population of employed adults in the United States’. Forty Six new occupations were added in 2012 (previous list of occupations was set in 2004). “Several of these (new) occupations were based on popular press articles and online reports featuring “best jobs” and “hot career fields,” such as those found in U.S. News and World Report, Fast Company, the Bureau of Labor Statistics, and CNN Money.”
In order to obtain a definition of these occupations that relates to the 291 questions that candidates will answer, people employed in these occupations are given the same questionnaire. In order to participate, people employed in these occupations need to meet the following criteria:
- They could not indicate any degree of dissatisfaction with their work.
- They must have had at least three years of experience on the job.
- They must have been at least 18 years of age.
The above bullet points are critical pieces in understanding this tool. Essentially, it says the following:
If your answers to the 291 questions are somewhat similar to those answers from people in a specific occupation, then given that these people are not dissatisfied with their occupation and have been in that occupation for over 3 years, then there is a good chance that you will also find a decent level of satisfaction in the same occupation.
This poses a lot of additional questions:
- How happy are these people that took the questionnaire with their occupations?
- Who are these people that took the questionnaire for specific occupations?
- Of all the people that took the questionnaire for a given occupation, surely they couldn’t have answered the 291 questions in the same manner?
- If they didn’t answer the questions in the same manner then how is ‘an occupation’ defined in terms of responses to the 291 questions?
All very good questions. Let’s deal with each in turn.
1.How happy are these people that took the questionnaire with their occupations?
No idea. All we know is that they have been in their occupation for over three years and have not expressed dissatisfaction with that occupation.
2.Who are these people that took the questionnaire for specific occupations?
The Strong Interest Inventory Manual Supplement provides details on the number of people (who took the questionnaire) for each occupation, the year the data was collected, the average age of those taking the survey and the average number of years that people had been in that particular occupation (see below for example for ‘Accountant’ occupation).
3. Of all the people that took the questionnaire for a given occupation, surely they couldn’t have answered the 291 questions in the same manner?
They didn’t. Take a look around your department at work. Do you think you would answer a 291 question survey about detailed interests and preferences the same as one of your colleagues who has a similar role to you (i.e. who would be classed as having the same occupation)?
However, what is interesting is that there are some definitive patterns of answers for people in the same occupation. This brings us to the 4th question.
4. If they didn’t answer the questions in the same manner then how is ‘an occupation’ defined in terms of responses to the 291 questions?
Essentially the responses of people in a specific occupation are compared with responses from those of a general sample (called the General Representative Sample (GRS) – currently consists of 2250 individuals). The way in which the responses differ for those in the specific occupation is deemed to be characteristic of that occupation.
As an example to help visualise this, let’s assume that the ‘majority’ of accountants answer questions 10, 20, 30, 40 and 50 with the response ‘ strongly like’. This is different to how the majority of the General Representative Sample answered those questions, therefore these set of responses is part of what makes up the definition of ‘Accountant’.
So there you have it. You take the career assessment test and for each occupation you get a score (occupational scales). The higher the score, the higher degree of similarity you have to responses from people in that occupation (well, in theory the above sentence should probably be replaced with the more complicated sentence which reads… the higher degree of similarity you have to the responses from people in that occupation that separated them from responses from the general population).
I think you probably get it by now. Take the career assessment test. Get a score for each occupation. The highest scores indicate the most similarity to people who have been in those occupations for more than 3 years (and are therefore presumably satisfied with their jobs).
Does a Career Assessment Test Work?
So how do you evaluate the effectiveness of such a career assessment test?
Ideally you would find a bunch of people who took the career assessment test and then chose an occupation based on that test and find out whether they are still in that occupation and how satisfied they are with it.
Simple right? Not so fast. That is not how the effectiveness of such tools is measured given the practical difficulties of achieving the above (not sure you are going to get people to sign up for the test based on the condition that they then need to choose the occupation that they score the highest in, stay in that occupation for 10 years, just so you can gather more data on the tool? Sign me up!)
Instead, the effectiveness is usually measured by testing the ability to predict occupations that people will eventually enter. Simply put – get people in certain occupations to take the test and see if their occupation (or one similar to it) gets the highest score.
Miller goes through some of the history of this testing and presents data on numerous tests.
A few definitions are needed before we look at the results.
A ‘Good Hit’ is when an individual’s current occupation matched one of their three highest OS (occupation) scores (scoring greater than 45 for that occupation)
A ‘Direct Match’ means that occupation scored on the survey was ‘exactly’ the same as the person’s actual occupation.
An ‘Indirect Match’ required some inference by the researcher (i.e. we can take that to mean to the occupation scored on the survey was somewhat similar in definition to the person’s actual occupation).
Taking all the data shown by Miller (see Miller – table 1 pages 23-25) provides the following result.
So about 50% of the time the Strong Interest Inventory (SII) did identify a person’s occupation (or one similar to it) as one of the highest scoring occupations for that person.
In summary, the top career assessment tests have had a significant amount of time, money and effort poured into them over the years to make them as useful as possible. A career assessment test continues to be a valuable asset in providing career guidance to individuals, but it is important to understand exactly what you are getting.