Don't Confuse the Map with the Journey: Why Assessments Aren't Programs and What We Need to Do About It
One of the recurring themes I've heard via fellow clinicians revolves around how new or inexperienced clinicians develop programming for their learners. Often, these new BCBAs run assessments, such as the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP), and use the milestones extracted from these assessments as their primary programming tool. You may have seen it before.
This practice, however, presents a significant problem: the misuse of assessments as curricula or programming.
This week, I’ll explore this issue a little more in depth and discuss the challenges faced by new BCBAs. Further, I’ll propose a few potential solutions that might help ensure more effective programming for learners.
First, what’s the big deal? Why not use the milestones for programming?
To explain it clearly, let’s dig in to the problem more in depth, first.
The issue arises when inexperienced BCBAs use an assessment as a curriculum instead of an assessment, extracting the milestones and assigning them to the Registered Behavior Technicians (RBTs) to work on with the client daily.
For example, in assessing a learner using the VB-MAPP, a BCBA realizes that the learner does not meet the milestone for Imitation 3M: Imitates 8 motor movements, 2 of which involve objects. As a result, they use the milestone as a program target for their learner. In other words, they literally have their RBT work on 8 different motor imitations.
That’s all well and good, after all the assessment results indicated that this was a weak area for the learner. However, it’s what happens when the learner masters out these 8 imitations where the larger problem lies. It’s at this point that the new BCBA simply inserts the next milestone in as a program: Spontaneously imitates the motor behaviors of others on five occasions (Imitation 4M).
The first problem is that neither of these two milestones are designed to be used as targets. The second problem is that there’s a lot of ground to cover in between each milestone. If the learner has mastered the eight imitations highlighted in 3M, it does not mean that the learner is automatically ready for spontaneous imitation. In fact, there’s usually a vast expanse of skills to cover between the two, and many more variations of targets to introduce. In our example, you may need to work on having a learner imitate in a contrived setting with a peer, first. Or, you may need to work have them imitate their therapists across a variety of settings. There are a lot of possibilities.
Assessments like the VB-MAPP are designed to evaluate a learner's knowledge, skills, or abilities. It measures the learner's progress, identifies areas of strength and weakness, and helps determine if the learner is achieving specific learning objectives. Assessments provide a zoomed out view of what we’re doing every day and help keep us on track. However, assessments are not meant to serve as programming tools. They aren’t curricula.
Targets (or programs) are a little different. They’re a “zoomed in” view on things and they’re specific to the learner’s needs. Further, they’re more sequenced, with each skill building off the other in difficulty and complexity. For example, a target might start with having the learner imitate clapping their hands and eventually evolve to having the learner imitate clapping their hands when modeled by a peer during circle time. Eventually, over time, targets evolve toward the milestone that the assessment highlights.
Easier said than done, I know. Heck, it’s even hard to explain in a written fashion.
Ultimately—however—this practice of using milestones as programs can result in a falsification of progress, as the BCBA is essentially teaching to the test. While the learner may technically meet the criteria, they often miss out on developing many essential component skills. Consequently, the progress reported to insurance companies may not accurately reflect the learner's true abilities and growth.
So why does it happen?
The misuse of assessments as programs can be attributed to several factors:
Lack of understanding. Some new clinicians may not understand the difference in purpose between assessments and curricula due to a lack of proper education or guidance from their supervisors or their master's programs. This lack of understanding can lead to the assumption that the learner is progressing when, in fact, they are only being taught to pass the test. Additionally, they likely aren’t being shown how to break down or expand on particular targets.
Limited access to resources. Another contributing factor is the limited access to resources for new BCBAs when it comes to programming and problem-solving. As a result, they may struggle to develop innovative and effective programs for their learners. Even when they understand the difference between assessments and curricula, coming up with new programs requires a degree of innovation. Innovation usually comes with experience.
Pressure to show progress. New BCBAs may also feel pressured to demonstrate progress to insurance companies, leading them to focus on specific milestones rather than addressing the broader developmental needs of the learner.
So what do we do about it?
Clear communication and better training. If you’re a supervisor, clinical director, or a BCBA with experience and leadership initiative, speak on this with your colleagues. Supervisors and educators must emphasize the difference between assessments and curricula and ensure that new clinicians understand that using assessments as curricula is unacceptable.
To anyone who has been reading my stuff over the last 6 months, it’ll come as no surprise to you that I also recommend a concise and thorough training on component-composite analysis. Follow the links below for a quick primer.
Improve resources. Finally, I’ll harp on this until…well, until something changes. The field needs to invest in providing better resources and technologies to assist clinicians in making informed decisions during therapy sessions. This includes moving beyond Continuing Education Units (CEUs) and journal articles as the only sources of information, and recognizing that clinicians often lack the time to read or attend CEU events on relevant topics.
Let me be clear, I’m not saying that it should be this way. I’m just saying that it currently is this way. The idea that BCBAs should have the time to consume mass quantities of journal articles to solve a programming issue is a mere apparition. The expectation that there’s a CEU event out there that perfectly addresses and solves for the particular stuck they are having with a learner is wishful thinking.
That said, the solution is for ABA companies and ABA researchers to look toward resources that bring quality programming ideas to clinicians quickly and on the spot. So much so that the BCBA has the resources and capabilities to make the correct—or at least—most informed programming decision on the spot, as well. At this point, nothing like this is readily accessible to these clinicians.
Certainly this is all simpler in theory than in practice. So much of this problem transpires due to a lack of resources and training for clinicians. An investment in these resources, technologies, and trainings will radically improve our field, increase progress with our learners, and eliminate headache and fatigue with our clinicians. Tell me what you think!
Have a clinical question and want to nerd out about it? Join our exclusive Facebook Group!
Sundberg, M. L. (2008). VB-MAPP: Verbal Behavior Milestones Assessment and Placement Program: A Language and Social Skills Assessment Program for Children with Autism or Other Developmental Disabilities. AVB Press.