Years ago, I wrote that many coaches would be better off putting a basketball at mid-court and leaving, allowing players to play their own pickup games for the duration of practice, rather than continuing with their current practices. This was not a popular statement, especially amongst coaches.
We attribute improvement to practice or in today’s era, to workouts with an individual skills trainer. The purpose of practice or a workout is to improve, so the practice or workout must cause any improvement.
How do we know that practice causes the improvement as opposed to playing games? How do we know that year-to-year improvement is not due to growth, maturation and strength development rather than practice or individual training? How do we know that it’s not a coaching change, improved shot selection, or a different role?
We don’t.
We assume because practice is supposed to improve performance, but we never know for sure because we do not have a control group.
There are several methods we could use to test the efficacy of Program A (practice, individual training, a specific coach/trainer, etc).
We could pre-test a group, have the group train with Program A for a period of time, and post-test the group. This is essentially how we conclude that practice caused the improvement: We see a player today, see the player next month, and attribute the improvement to the practices or training that occurred during that month.
Of course, practices or individual workouts are not the only activities in which the player participated during that month that could conceivably cause or impact improvement or skill development.
To draw more reliable conclusions, and to determine causation not correlation, we have to control for variables that could impact the results. For example, if Program A is a 3x per week individual training program, does the player practice with his team? Does he lift weights? Does he play games? How can we conclude that Program A caused the improvements instead of the team practices, games, or strength training?
A player once attended a weekly clinic that I directed. She also worked with an individual trainer, a strength & conditioning coach, and her high school and AAU teams at the same time. How can we possibly determine which activity caused any improvement?
To control for these confounding variables, we could use a control group. The experimental group uses Program A, and the control group does not. Everything else — lifting weights, playing games, team practices, etc. — remains the same between the two groups. Therefore, if the experimental group improves, and the control group does not, we can suggest that Program A improved performance: Players who engage in individual training 3x per week improve performance more than players who do not participate in any additional training.
Of course, this only demonstrates that Program A is better than nothing; it does not prove anything about the actual training. Does Program A cause the differences between Program A and the control group or is it the extra practice hours? Is any program better than no program? Would the player have improved equally or more by playing pickup games 3x per week instead of the individual training? We do not know because we compared 3x per week of individual training to doing nothing.
To demonstrate that Program A caused the improvements, we need the control group to spend the same amount of time doing something related: Program A does individual training 3x per week, and Program B plays pickup games 3x per week. Now, if Program A improves performance more than Program B, we know that the training had some effect, and it was not just the extra hours on the court that caused the improvement. We can state that three weekly sessions of individual training improves performance more than three weekly sessions of pickup games over the same time period.
Now, coaches do not care about the cause of the improvements; they care about the improvement. If one player works out 3x per week in the offseason with am individual skills trainer, and the other goes to the beach, and they improve to the same degree, does the coach care that one worked out with a trainer and one played at the beach? No. The coach cares that his players improved.
Of course, because we assume that the trainer caused the improvement, the coach would be disappointed by the player who went to the beach and would imagine how much more he would have improved by working out with the trainer.
However, if the two players improved to the same degree, why attribute any improvement to the training? Did the training cause the improvement when a player who went to the beach improved just as much? Maybe the coach should encourage all the players to go to the beach. Each one — going to the beach and working with an individual skills trainer — are correlated equally with the improvement. It is only our perception of the right or best way to improve that makes us believe that the training caused the improvement, but not the beach. Maybe, in reality, it had nothing to do with the beach or the individual skills trainer, and the players improved because they lifted weights 5x per week before heading off in different directions; maybe it was the weekend summer league playing games on Saturdays and Sundays against better competition that caused their improvement.
Basically, the beach is the control group, and the training is Program A. If there is no difference between the two groups, the training program did not cause the player’s improvement. Instead, the improvements were caused by lifting weights, playing in the summer league, or some other mechanism.
When we evaluate coaches and practices, we should not compare the practices to no practice. Instead, the control group should be playing pickup games or a player shooting on his own. To demonstrate that a training program works or that a coach’s practices develop skills, players would have to improve more than if they simply played pickup games or shot by themselves.
My first paragraph essentially suggested that few coaches would demonstrate skill development during their practices that surpassed the skill development that players would derive through playing games or shooting on their own for the same amount of time.
Therefore, our question should be:
Is _______ better than the same amount of time spent playing games?
The control group — playing pickup games — allows us to conclude that the experimental group (Program A) has some effect beyond just the extra hours. Without a control group, who knows what really drives improvement and how much a trainer, coach, a set of drills, etc. affect development?