Why Is The CFA Pass Rate So Low & the MPS So High?

View Comments

• CFA Institute is a business, exam fees and annual dues are their revenue source, and their primary asset is the brand value of the CFA charter. Only way to ensure continued value of the brand is through scarcity. If every jackass out there could earn the designation there would be no incentive to go and commit to earning it.

  Its no coincidence that the MPS is rising immediately following the significant decline in the number of candidates taking the exam since the pandemic, one of their their primary revenue sources is down like +40% so their need to encourage more people to take it in the short term. Once it recovers you will see that MPS dip as they are eroding the brand value by passing too many people and diluting the value.

• Hey Everyone,
  Thanks for the great work that you do!

  I've been thinking about the CFA pass rates / MPS and wanted to get your view on whether you think that the CFA Institute has made appropriate and fair adjustments to the MPS to account for the switch from paper-based to computer-based testing. I personally think this will have an impact at all levels, but particularly levels 2 & 3.

  The vast majority of preparation questions are going to be completed via paper (EOCQs, BB, Q-Banks, most mocks, etc) and so the difference between that and the exam day is significant. Not only does it seem likely that the time to complete the CBT will be longer that PBT (or put another way, candidates will be under more time pressure thus increasing the overall difficulty), but also, I personally highlight, scribble, cross out, etc, parts of the questions that are relevant. Removing the ability to do this in an easy and meaningful way creates additional difficulty.

  What are your thoughts on this please? Would this be something you could liaise with the CFAI on to get their official input on please?

  Thanks so much :-)

• I don't buy the logic of deferrals factor, there were no deferrals in 2h21 sittings and candidates still failed badly. Being deferred is being more prepared, imo negative aspects of being deferred is overestimated by CFAI.

  I think that their CBT exam design is flawed: less questions > higher cost of error, plus new design introduced much greater factor of luck of getting question for which candidate well-prepared and not getting one for which candidate poorly prepared.

  • I agree...ish. Just my opinion, but I think that there is a deferral effect, but it's one of many. I believe that there are a general myriad of teething issues with the transition that need ironing out. It should stabilize at a norm (old or new, doesn't matter) but meanwhile we'll have to deal with the uncertainty. ☹️

• I don't agree with the impression being given by Zee Tan. However, agree with the points have been put forward. To my understanding, if pandemic had to be a real cause behind the low pass rates, than we would have witnessed the same in Dec 2020 exams. Rather, candidates performed well during such sessions. The only reason for the low pass rate I believe is the unreasonably higher MPS all of a sudden since May 2022. My back of envelope analysis for level 2 pass rate and MPS can be validated through the fact that when MPS first raised in May level 2 exam (i.e. from a range 63-64% to 65-66.5% or 67%) pass rate immediately dropped to 40% (i.e down from 55% in Dec 2020). Again for August 2021 level 2 exam, the MPS has been raised arbitrarily to 69-70% which led to a significant drop in pass rate to 29% (i.e ever low in CFA's history). CFAI's explanation with regard to pandemic does not go hand-in-hand as to the effect that at one end it says passing rate has gone down due to covid and at the other end raising mps not once but twice I believe. If candidates, during the pandemic already performing poorly (which I don't believe is the case) why raising the MPS?. Thus, to my understanding, the higher MPS is the sole reason for significantly low pass rate. Further, if CBT exams are easier to score and MPS supposed to be raised to equate the difficulty level of previous standards, than why we didn't we see the passing rate around 40-41%? (which is still low than 45 to 47% in pre covid periods). So, in a nutshell, this shows that CBT exams are now more difficult for that matter, unless the mps adjusted to a reasonable level, which I believe is the 65% to 66.5% range, where we can see the pass rate converging to 40%. Lastly, I also agree to Casey, commented above with regard to (i.e conflict of interest) charterholders involvement in arbitrarly setting the exam and passing standards. We need investment professionals rather machines to score 90% MPS, if, ever, going down the road, existing charterholders desire for it. Concluding my remarks, I don't see passing rate going up for longer periods at existing mps.

  Still hoping best and keeping my fingers crossed for a better dawn,soon.

  • The important point to know is that CFA Institute has two levers to pull to control the pass rate - the difficulty of the exams (i.e. how difficult it is to score in the exam) and the MPS. Since we don't have official insight into either, it is impossible to be certain of what's happening exactly. Hope that helps.

• There are too many CFA Charters in the world, i believe thats the reason

  • Whoops, wasted all that time replying just to you...ha!

  • John's got it . . . CFA Institute is a private organization, its a business. their brand value is their product and most important asset, in order to maintain the value of their brand they need to ensure it remains scarce. I earned the designation in '06, and guess what Ive had to do each year since then...pay the $400 annual fee, so exam fees and member dues are their primary revenue source. They must walk a fine balanced line of maintaining a degree of scarcity which makes the designation desirable, balanced with not making it so difficult that nobody will want to take it (leading to their revs drying up as people stop taking the exam and fewer new members are paying the annual fee).

    Do you think it is a coincidence the MPS is rising since the dramatic drop in the number of candidates sitting for the exam since the pandemic? they need to get more people in to take the test and pass because it threatens their top line.
    Certainly not hating on the program, dedicated 4 yrs of my life to it. each time i interview someone who has the designation or who has completed at least 2 levels of the test, I know immediately that this person means business, isnt afraid of hard work and knows what it takes to grind.

    That said, I would never wish having to endure the process upon anyone at this age in my early 40's!!!

• It's very easy for the CFAI to blame the low pass rate on deferred candidates without taking responsibility for the errors in their new system. “The degree of difficulty of the May Level I exam was consistent with previous Level I exam administrations, and this is the case whether we look back to paper-based testing or computer-based testing, which we introduced in February this year,” CFA Institute spokesman Matthew Hickerson said.

  If this statement by the CFA Institute holds true, then the MPS should not have been as high as it was, considering traditional MPS has been in the mid-sixties. This statement is literally contradictory. Additionally, I completely disagree with the THEORY that the test being shorter makes it easier. It depends on test taking ability and the amount of material that is covered. I know average is your average, but misses on easy questions such as definitions now count for more, and there is less opportunity to display knowledge in difficult areas where much time was spent (further clarification, this is solely based on me taking 240 question 2020 mocks in the old format, and comparing them to my online 180 question mock scores. My 240-question scores were higher on average btw).

  I understand the aforementioned has always been an issue, but with less questions and the amount of material staying the same, it becomes even more of an issue. It's very frustrating to receive a question asking the definition of a minute detail, and not receive a single question on a difficult concept that seems more important to the core knowledge of the curriculum. This is a pure example, and definitely not a reflection of my test, but missing a question because of forgetting the difference between Keynesian and New Keynesian school of thought seems very minor, but will be considered an "easy" question by CFA Standards. Whereas, a candidate that might forget such a minute detail, could have a full understanding of the different measures of convexity and duration of a bond, and not receive a single question on this topic. This might be considered a "harder" concept by CFA, but candidates were unable to show their abilities in that area due to a smaller sample size. Again, I realize average is average, but if CFAI is subjectively determining what they feel to be an "easy" of "difficult" question/exam, the lack of transparency truly disadvantages new candidates. All new candidates should be held to the same standards as current charter holders, and shouldn't have their exams scores SUBJECTIVELY determined by a group that clearly feels that it should be more difficult to obtain the charter than it was for them.

  Imagine taking a STANDARDIZED test that is SUBJECTIVELY graded. I'm not saying this is a money grab, but I think it is a direct conflict of interest to have Charter holders determining who is "just competent". I truly can't believe that the phrase "just competent" is the bar for passing this exam. What a joke.

  • Thanks for the well-thought out comment, Casey! I'll try to respond to a few of your points the best I can:

    Degree of difficulty being the same across paper and CBT exams: It is our theory (through our MPS deductions) that the CBT exams are easier to score in. This does not necessarily mean that the exam is now easier to pass. We believe the increase in MPS is to maintain a consistent difficulty in passing. So in broad terms, candidates get higher average scores in CBT exams, but the MPS is higher so the overall difficulty in passing remains consistent.

    The theory that the test being shorter makes it easier: When launching CBT, CFA Institute stated that in their studies, going past 180 questions does not increase accuracy of determining the competency of the candidate. Using the analogy of a thermometer, exposing a thermometer long enough for the mercury to adjust is sufficient to obtain an accurate temperature reading. Keeping the thermometer exposed for longer does not lead to a superior measurement. More in our CBT exams guide.

    Subjective grading: Multiple-choice exam scores may be objectively graded, but a pass mark will always have a subjective element to it. If the MPS is fixed (say 70%), then the subjective element is the exam difficulty level from year to year. I recognize that this means there is an element of 'taking CFA Institute's word for it', but I don't see how we can ever avoid an element of that, when the test matter is knowledge. Universities, for example, employ similar systems (arguably in an even less standardized manner since it's up to individual professors).

    • I hate that thermometer reference, it makes no sense because the exam questions cover an extremely small % of the curriculum. The analogy assumes that the area surrounding the thermometer is uniform and all encompassing. The CFA exam is more like walking into a building with 10 different rooms (topics), each subdivided into 5 sections (readings), and testing the temperature in a random selection of these areas for various periods of time. There could be extremely hot and cold sections within those rooms but if you don’t randomly select those or spend enough time to get a good reading then you wouldn’t know. If you spent time going to each one you’d have a more accurate result.

      • Fair point. However, perhaps CFA Institute's point is that after 180 questions, there is a fair representation of the curriculum.

        Whether or not CFA Institute and candidates agree on that point is another matter, but again worth mentioning that this conclusion is a result of studies specifically conducted for this purpose.

• I think it’s reasonable for deferred and especially twice+ deferred to get a significant discount on next attempt.

  • Feel free to suggest that to info@cfainstitute.org 🤷‍♂️

• How do we know that the computer-based test (CBT) exams are fair? Looking at the minimum passing score and the fact that they start to increase from the beginning of CBT approach. What if people get in the exams room and memorize the content then sell that out to candidates who take the exams at a later date? This is no joke or conspiracy, we cannot guarantee this is not happening. Unless the exams paper/questions changed (which is also not fair as people are not tested on the same areas), then there is always possibilities of cheating. Just never say never.

  • My understanding is that the individual tests are not identical. Hence the MPS are also individualized to an extent to each candidate/test.