Updated: Dec 11, 2022
Do you ever wonder how you compare to other mental health professionals within your field? How would you rate your overall clinical skills and effectiveness? What if we could line up all the talk therapy professionals in your area and rank them from least effective to most effective, where would you stand? Now can you imagine what would happen if we converted the effectiveness rankings of those professionals into percentiles?
What percentile would represent your effectiveness compared to colleagues who have similar credentials? Keep in mind that the 25th percentile represents below average, the 50th percentile average and the 75th percentile above average.
Now let’s turn to outcomes in your own case load. Do you know on average how many of the people who see you get better, get worse or stay the same?
a) What percentage of the people you see experience significant symptom relief or reduction?
b) What percentage stay the same?
c) Now for the tough one, what percentage do you estimate or know got worse during treatment?
Adjust the three until added together they equal 100%.
Want to know how your responses compare to your colleagues?
Have a look at the results of a study done by Walfish, McAllister, O’Donnell, and Lambert in 2012. They asked a diverse group of psychologists, psychiatrists, social workers marriage and family therapists from all over the United States these same questions. Here is what they found.
No matter the profession or where in the country they were from, they all rated their overall skill level and effectiveness about the same. Almost every single one rated themselves above the 75th percentile. The average rating was around the 80th percentile. So, if your response was similar, you are in good company, however it begs the question, where are all the average or below average providers? Did they get missed in the study?
By now you are probably gathering that this study was an investigation of mental health provider self-assessment bias. Let’s move on to question 2 and compare your responses to the study participants.
When asked, “On average how many get better?”, I doubt you will be surprised to learn the most common response was 80%, or that almost 1/4 of the respondents thought the number who improved in their care was closer to 90%. What you may find surprising is that almost half of them reported no one (0%) had ever deteriorated in their care.
Before we get to your responses, let’s look at what the outcome research tells us about what % you should expect will get better, get worse or stay the same.
Are you are starting to become disengaged because you know these averages don’t apply to you? One thing to consider is that while we all understand effectiveness rates vary tremendously between providers, meta-analysis in tightly controlled studies of effectiveness, indicate the majority only achieve a reliable change with about 50% of the people they see.
So how did you break down your answers to Question 2? Without your own outcome data to refer to, it’s likely you have scored your percent improving above 50%, perhaps well above. The higher the percentage you reported, the more likely it is that you could benefit from routinely monitoring and quantifying progress using more than your professional expertise.
This becomes even more important when it comes to who got worse, 25 years of data indicate that about 10 percent of our clients deteriorate while in our care, and we are notoriously bad at identifying it. With this in mind, look at your number. Is it lower than 10%, was it 0%? If so, you may want to consider it a sign that you are missing clients who are deteriorating.
Ignorance may be bliss when it comes to estimating how many people you help, but no one wants to do harm, so if only to help identify when the people you are trying to help are getting worse consider implementing a systematic way of collecting feedback and using the resulting data to identify at risk cases.
Ready to Get FIT NOW?
Sign up today for a free trial and receive lessons and group activities to help develop or improve your FIT Skills.
Feedback-Informed Treatment and Deliberate Practice Training
Lectures from Scott D. Miller, Founder of the ICCE
Proven FIT Practitioner Development Activities & Exercises
Videos of FIT Best Practice Demonstrations
Unlimited Access to live support from Senior FIT Trainers
Access to a Private FIT Practitioner Development Group
FIT Group Training
Looking for an engaging one day online group training to help your staff become more feedback informed? After completing this training participants will be familiar with research findings in behavioral health outcomes and will be able to identify:
- What is working in the field of psychotherapy (i.e. treatment is efficacious, cost effective etc.).
- What is not working in the field (i.e. high drop outs, stagnant outcomes, etc.).
- Identify and define the factors that account for the efficacy of all talking therapies (i.e. extra-therapeutic factors, therapist factors etc.) and what proportion each of these factors contribute to change.
- Identify what does not account for the efficacy of psychotherapy (i.e. model, and technique).
- Define therapeutic alliance; describe the components that make up the therapeutic alliance.
- Define Feedback Informed Treatment (FIT) and articulate how FIT differs from treatment models and approaches.
- Identify what predicts and what does not predict outcome.
- Explain why it is important to gather formal client feedback on outcome and alliance (i.e. early change, client experience of alliance).
- Describe what outcome and alliance measures are measuring.
- Describe who developed the Outcome Rating Scale (ORS) and the Session Rating Scale (SRS); why these measures were developed.
- Identify the characteristics of a culture of feedback.
- Articulate why a culture of feedback in therapy is important.
- Be familiar with a number of strategies for enhancing the development of a culture of feedback in therapy.
- How and when to introduce the ORS to clients
- How to score the ORS
- How to track the ORS on a graph
- How to be able to administer the ORS to their clients, through practice exercises.
- How and when to introduce the SRS to clients.
- How to score the SRS.
- How to track the SRS on a graph.
- How to administer the SRS to their clients, through practice exercises.
Book a free FIT Implementation Consultation to learn more about this engaging one day online group training.
Interested in becoming a Certified FIT Practitioner or Trainer?
The FIT Supervision Intensive is one of the requirements for certification and the next online training is coming up in April 2023.
After completing this training the health care professional will be able to:
-describe why FIT supervision is different from conventional supervision
-describe the most common challenges in FIT supervision and how to address them
– identify and address “at risk” cases in supervision
-describe the empirical evidence that supports the use of client feedback and deliberate practice strategies to improve outcome and retention rates
-describe how technology can be used to support and improve services
-outline FIT, its research foundations, and core competencies.
-describe current literature on supervision.
-utilize outcome and alliance data and deliberate practice strategies to identify supervision needs and guide supervision discussions.
-identify potential adjustments to supervisory practices to align with FIT/deliberate practices.