A Chat with CFA Institute About the New CFA Exam Results Format

Want to pass your exams? Start preparing the right way.
My email is and I'm preparing for
By Zee, interviewing CFA Institute

As you probably know, the CFA Institute recently changed their results format to a new, visual format (here’s our guide on how to interpret the new results charts). Naturally this generated a lot of interest among ourselves and our readers, so we reached out to CFA Institute to speak to them about this.

Here’s what we found out about the new CFA results format, including the reason behind the change, the process, and how the new format improves over the previous one.

The Reason Behind the CFA Results Format Change

CFA Institute enhanced the CFA Program results format in response to candidates’ feedback and to provide fuller transparency into their exam results.  The most important thing that a score report can do is to help a candidate who did not pass to look ahead and develop an effective study plan. That’s what drove the vision for the new results format.

​As a reminder, the prior format presented the candidates performance per topic as falling into one of three score buckets:

The Old CFA Results Format
 
>50%
51-70%
>70%
Topic 1
 
*
 
Topic 2
 
 
*

This approach and data didn’t provide a candidate with the fullest picture possible to improve study habits or better plan for the next exam, if needed.

Issues with the old CFA results format:

  1. There was no reference to the minimum passing score, which is the most important data point from a candidate’s perspective. 
  2. There was no indication of topic difficulty – scoring 70% in a difficult topic is different from the same score in an easier topic.
  3. Too broad. The 51-70% range was especially challenging, as the average candidate’s score and the minimum passing score typically fall within that range. The difference between 52% and 68% is highly significant, but candidates were unable to discern the difference. 
  4. Low correlation. Because the passing score is based on the total performance, high or low scores in a topic may not correlate to overall performance. There may be more room for improvement in taking a highly-weighted topic score from 70% to 80% than from taking a topic with a low exam weight from 40% to 80%.

The primary goal with the new format is to provide candidates with greater transparency so they can develop the best possible study plan. We think we have achieved that objective, and according to CFA Institute, early candidate feedback validates that claim! 


The Process of Creating a New CFA Results Format
​This specific project took several years of development to design, test with candidates, run a pilot, and develop an IT infrastructure that could deliver and render it.  

But CFA Institute have been considering making improvements for some time. For example, a few years ago they introduced the concept of a “score band” to address how close the candidate’s score was in relation to the minimum passing score. Other task forces have also been convened from time to time to suggest possible improvements.

One development was the publication of the 2014 edition of the Standards for Educational and Psychological Testing by the American Educational Research Association (AERA). These standards are considered to represent best practices for credentialing organizations, and a new standard recommends publishing an error measurement to indicate score reliability. A desire to comply with this standard helped spur some creative thinking about how the scores were represented, and helped inform the final format of the results. 


Challenges in Implementing and Testing the New CFA Results Format
​It is always a challenge when changing something people are familiar with. With that in mind, CFA Institute has tried to be as thoughtful and thorough in managing the project.

​While planning the new format, CFA Institute performed two rounds of market testing to gauge candidate response and to address the feedback. A pilot test was conducted in September 2017 with a subset of Level III candidates who had already received their results in the old format. Some adjustments were made based on their feedback in time for the full rollout in January 2018, and further improvements will continue to be considered going forward. 


Three Ways the New CFA Results Format is Better Than the Previous One

According to CFA Institute, the new CFA results format has at least three main advantages over the old format:

  1. Candidates can now see how close their overall exam scores are in relation to the minimum passing score. This is the first time we have given a direct reference to the MPS and represents a significant improvement.
  2. The new format shows performance of the 10th percentile and 90th percentile candidate, which is an indication of the relative difficulty between topics. 
  3. It is somewhat easier to gauge topic weights, because the confidence interval is partly related to the number of questions asked. The more highly-weighted topics will typically have more reliable scores. Candidates can factor this into their study plans.

Were There Any Alternatives Considered?

Many! CFA Institute continues to look at potential improvements, although in the near term they are likely to take the form of tweaks rather than the major changes just introduced in January 2018.


Would CFA Institute Ever Consider Releasing Actual Candidate Results, or the Minimum Passing Score?

We asked, folks, but unfortunately the response we got was “it’s unlikely”. CFA Institute’s stance is that credentialing programs are designed to measure whether a candidate has the necessary knowledge, skills, and abilities to perform a certain job function. They are not designed to indicate the relative performance within those skills.

From CFA Institute’s perspective, if candidate scores were published, there would be a temptation by candidates and employers to compare the scores of different candidates. The exam was not designed to rank candidate abilities so this would be a misleading comparison.

Likewise, although the minimum passing score can vary somewhat from year to year based on the difficulty of the questions asked, it is designed to be a consistent benchmark of the necessary knowledge, skills, and abilities. 

If you want to know what should you be aiming to score in your practice exams, you can learn more about the MPS and our data-backed recommendations here:


Wouldn’t Removing the Band Scores Make the New CFA Results Format More Opaque?

Quoting CFA Institute directly:

Even with a score band, there was no way to tell how much of a difference there was between one band and another. Originally, score bands were created to address the issue of the average candidate having most scores in the 51%-70% range, which also typically included the MPS.

The biggest advantage of the new format is that both the MPS and the candidate score are marked. So, to find out how close you were to passing, all you need to do is see how close those lines are to each other. Although candidates who are accustomed to the score bands may need time to adjust to the presentation, in the long run it should be much more informative.   

If you have any thoughts or questions for CFA Institute, let us know in the comments below. To accurately interpret the new format’s results charts, read our guide here.

Zee Tan
Author: Zee Tan

 

Leave a Comment