As a child growing up in Western NY in the mid 1980s it was inevitable that In would fall in love with the New York Mets. My father was a bit of a front runner and I loved baseball so Doc and Darrell were my heroes. As the 90s rolled around I moved away from baseball and watched more football but that seed of baseball love was planted. It lay dormant until I moved to the Jersey shore in the early 2000s. As I drove to and from my grad classes I was able to catch sports radio on WFAN and was reintroduced to baseball. This was the era of the Jeter Yankees but it was the loveable and after the 2000 World Series loss) struggling Mets that still owned my heart.
I had moved to New Jersey to get a Masters Degree in History and,while there mistake of getting a job in a chain bookstore. The job was great but most of my paycheck went to new books. That’s only a slight exaggeration. As a baseball fan and lover of books in the early 2000s Micheal Lewis’ Moneyball was a must read.
The book tackled the Sabermetrics (baseball analytics) revolution of the 80s & ’90s as practiced by the Oakland A’s Billy Beane. The book was profound and really changed the way I thought about baseball. This was helped by the fact that I had modest understanding of statistics, an open mimd because of my Hiatory training, and the theory of market inefficiencies just made sense. I don’t want to get into details but one part of the Beane’s philosophy has been on my mind while I consider Standards Based Grading in general and the Power Law in particular.
It’s been over a decade since I read the book so I am massively paraphrasing but Michael Lewis tells a story about Billy Beane attempting to trade for a relief pitcher. The story attempts tp explain a “prisoner of the moment” problem where other GMs and fans often get caught up in small sample sizes and get trapped in the most recent example of performance. Beane was attempting to “buy low” on a pitcher that had just gone through a rough patch, giving up a bunch of home runs over the last few games. Beane was working based on the theory that the most recent example is not always a true measure of players’ skills. Instead to see actual skill level the evaluator should look over the player’s long history. Beane analyzed that the player’s homerun problem was relatively small blip in an overall record of outstanding performance. That said, the Moneyball philosophy also suggests that early demonstrations of mastery don’t always reflect overall skill either. A small burst of high performance followed by a prolonged stretch of poorer performance does not necessarily indicate the high skill potential. Sometime luck or poor competition can mislead the observer. Baseball is full of rookies that get off to a hot start, hit .350 with a few homers for a month, and then flame out with .230 average 1for the rest of the year.
The point of all this baseball reflection is that in terms of education Billy Beane was looking for a way to judge the mastery level of a player. In doing so he was attempting to remove outlier statistics to gauge player growth (or decline) in order to determine what the player might do in the short and medium term future. When Moneyball is put in these terms it sounds an awful lot like what the Power Law function is attempting to do. (Here is my breakdown of the different SBG Mastery Level Calculation Methods) The Power Law calculation smoothes out the perceived growth curve of a student in an attempt to show the most likely path the student is on in achieving mastery of a particular skill. A student might get lucky on an early assessment but then not do well in later assessments. The Power Law function will factor this in and generate a curve that quickly flattens out and might even begin to show a decline. A student that starts off without demonstrating much mastery but shows consistent growth will see a fairly steep growth curve. A student with consistent performance will see a fairly flat curve that projects little growth in the future.
My love of baseball has only been enhanced by my increased understanding of Sabermetrics. Keith Law and his Smart Baseball book is a great place to learn about some of the key elements of the modern analytics revolution in baseball. What I love about that book is the Keith Law seems open to change and understands that there is always a neew way to evaluate a player. Moreover even the best analytics have some level of subjectivity and are open to interpretation. In education teachers week to treat grades as sacred and inevitable. The traditional gradebook hasn’t changed much in the decades since I was a student and is treated much like baseball Oldel Timers treat the Batting Average. Both seem to see their chosen statistic as the ONLY way to evaluate a player but neither stat provides a very accurate way of measuring mastery of skill.