Jane D'oh

I've been away from TxBands for a couple days and had some catching up to do. I'm going to skip the quote process tonight and just share a few thoughts directly about what I just read. Some of it may repeat earlier comments to bring them back to the forefront because this monster conversation has been wondering and wagging its tail all through the weeds.
 
First, Mr. Move on, move on, move on... Just because this has happened before doesn't mean it has to happen again. If you're not contributing in a positive way or enjoying this difficult topic, maybe you would be happier if you just take your own advice and move on to another topic. Some of us are trying to explore possible solutions. Others are working through their grief and frustration. Your contributions are often helpful but you missed the mark this time. The same to others of a similar ilk.
 
Second, this is not about Round Rock or just this year. The Round Rock parents seem to understand they are just the latest example of a bigger issue. That point seems to be lost on some of you. Others are discouragingly indifferent.
 
Currently there is no "bad judge rule". That derogatory slang is part of what is confusing this issue. Yes, we need a solution to judging deviation but the perspective that accompanies that slang won't be well received by the UIL. The rule many of you are actually referring to is called the "judges preference rule" by the UIL. It has two main components. Application of one of them correctly advanced Leander to the SMBC where they made Finals. A perfect example of why the rule exists. The other component was rather haphazardly applied at Area B Finals due to the unclear way it was written. It was initially applied in a way that seemed consistent with the intent but was reversed the next day because making the rule scaleable was not expressly directed. Neither Keller or Waxahachie, the two bands involved, seemed overly concerned as both bands were advancing either way. It has be reported that UIL is reviewing and clarify whether or not application of the rule is scalable to include Area results. Understanding the rule and its intent would be helpful for those interested in how UIL currently deals with judging deviation.
 
Saying UIL doesn't care is both inaccurate and unfair. It also reveals a fundamental misunderstanding of what and who the UIL is.
 
Saying Directors who just advanced to the 6A SMBC Finals don't care is also wildly inaccurate. These are the Directors most likely to support change. They have all felt the sting of judging deviation many times.
 
It would help if people stopped using the term UIL without understanding it and how rules get changed. For example when you blame "The Government" you are really blaming "We The People". That's who the government is in this country. If you don't like what's happening, get involved and vote. Similarly, when you blame "The UIL" you are really talking about "We The Directors" and how they vote. All the Directors in Texas have an opportunity to fix this in the spring. Some are very concerned about this and other big picture issues. Others could be described as indifferent. A third group is openly hostile to change of any kind. That's why I explained in an earlier post how important it is to know where your Director stands and what they are doing to either influence or impede change.
 
And yes, there is a decoder ring. But even if you have access to one it takes forever to learn how to use it correctly.

Add one other point about directors' likelihood of bringing this up to UIL:
4) The directors that HAVE had this happen to their programs (the ones with the strongest motivation and the best case to make for change) aren't likely to complain either because it's perceived as whining or sour grapes, and I'm certain they feel strong pressure to take the high road and "just move on" (as we're seeing here on this thread).

One of my suggestions for similar circumstances in the future was to modify/extend the so-called "Bad Judge Rule" to address extraordinary spreads in ordinals between judges who are evaluating the same caption, which in this year's situation might have sent 13 bands through to finals and would not have cost the 12th-place band a thing.
 
I don't think anyone said TWHS didn't deserve to be in finals; in fact, several of us acknowledged repeatedly that every one of the 12 that went to finals absolutely 100% deserved to be there. In fact, there were about 17 bands by my count that could have been in finals and I would not have questioned their place there for a moment. TWHS puts on a consistently amazing show every year. I was in awe of how well your program recovered after Harvey last year. You could not tell by the end of the season that TWHS and many other Houston-area schools had gone through such a devastating natural disaster because the quality of the performances remained so high.
 
Again, this discussion isn't really about RRHS or TWHS or any other specific band or year or event -- it's about the entire UIL competition and judging system going forward. It may be in the directors' hands, but I'd say it's a foolish director who doesn't acknowledge the importance of having the parents', district's, and community's full support for and belief in the integrity of the competitions they attend.

One of my suggestions for similar circumstances in the future was to modify/extend the so-called "Bad Judge Rule" to address extraordinary spreads in ordinals between judges who are evaluating the same caption, which in this year's situation might have sent 13 bands through to finals and would not have cost the 12th-place band a thing.
 
I don't think anyone said TWHS didn't deserve to be in finals; in fact, several of us acknowledged repeatedly that every one of the 12 that went to finals absolutely 100% deserved to be there. In fact, there were about 17 bands by my count that could have been in finals and I would not have questioned their place there for a moment. TWHS puts on a consistently amazing show every year. I was in awe of how well your program recovered after Harvey last year. You could not tell by the end of the season that TWHS and many other Houston-area schools had gone through such a devastating natural disaster because the quality of the performances remained so high.
 
Again, this discussion isn't really about RRHS or TWHS or any other specific band or year or event -- it's about the entire UIL competition and judging system going forward. It may be in the directors' hands, but I'd say it's a foolish director who doesn't acknowledge the importance of having the parents', district's, and community's full support for and belief in the integrity of the competitions they attend.

I think it's because people think those of us expressing concern and proposing solutions are just Round Rock supporters whining about what happened to Round Rock. They're missing the main point that most of us have made, which is that this should be regarded as a problem that can and does affect any band program in Texas at any time and shouldn't.
 
In the end, it's really not about Round Rock or any other specific band that has received wildly varying ordinals in a UIL competition. Round Rock's 20-ordinal spread between marching judges this year was just a shockingly clear example of what detractors have admitted HAPPENS TO MANY BANDS, EVERY YEAR, to one degree or another. That is exactly why we're urging directors to call for real change -- it's not just Round Rock and one undesired outcome (or even two, though that does establish a concerning pattern, as you pointed out earlier). It's about the integrity of the whole competition and the need for some kind of calibration and accountability of judges in the system moving forward. TxDragonDad has put it in clear scientific/mathematical terms several times now, and if we all take our emotions and biases about our own kids and programs out of the equation, it seems fundamentally clear that there IS a consistent and provable problem, and it SHOULD be addressed for everyone's sake in the future.

I have two ways to eliminate "outlier" scores!
 
1) Have just one judge and his/her score is final - no outliers possible.
2) Have an infinite number of judges - outliers will be randomly distributed and average out among the bands.  
 
Obviously, neither of those is practical.  So we settle on a finite number of judges, greater than one.  How many?  Well, I would look at other subjectively judged competitions, like figure skating, diving, or gymnastics, which typically have about five judges.  Sometimes they throw out the high and low, sometimes they don't.  Even in head-to-head sports, like football, the referees' judgment can influence the result.  Often we blame the ref for the bad call and losing the game for us, but how about the quarterback that threw three interceptions or the point guard that missed five free throws?  
 
I think "outliers" frequently do happen in those events, so how do they cope?  If we expect all the scores to be consistent, then are we telling a judge that his/her opinion has to be be the same as the others or it doesn't count?  Might that judge have caught something the others didn't?  
 
I think the issue with throwing out data is drawing the line and agreeing on a philosophy.  Consider a three-judge system with throwing out high and low (taking the median) versus taking a average.  Do bands that have consistently good scores (say 7, 7, 7) deserve a lower ranking than bands that are all over the map (say 1, 6, 15)?  What makes the "middle" judge right and the other two "wrong?"  
 
Though the ordinal scores may amplify small differences in points, what is to say a pure points system (like BOA) are immune from bias?  I can give someone few enough points to guarantee they miss finals or a medal the same as I can flame them on ordinal rankings.  
 
As I wind down this lecture, I think it comes down to training and quality control on judges.  If one judge frequently displays a bias or uses bizarre reasoning (inconsistent with training and goals of competition) to justify a score, then we have to manage him or her out of the system - just as they do in other subjective competitions like ice dancing.  Mistakes and differences are a part of the game, but bias or negligence should be rooted out.  Unfortunately, there are victims along the way and there are always new judges coming into the system.  Hopefully we improve over time, but I don't know that a new scoring system fixes the problem.  It reminds me of work - put whatever system in place you want, but bad management can always screw it up.
 
In our lives, we've all suffered and benefited from clear official errors and biases.  The only consistent way I've found to overcome the occasional bad or unfair call is to be so good it doesn't matter.  Certainly, it never helps to get a reputation as a complainer.  
 
Thanks for reading, hope no one hates me now!

So, I'll come back to some key points I made earlier.   
 
1) Outliers - do we think that they are a problem?
2) Outlier needs to be statistically defined, not emotionally.
3) Outlier corrective/prevention should not fully negate the judge's input, but it should also not over-correct their score, either.
 
Key points from others:
1) Directors vote, so this is on their shoulders
2) Directors of top winners and bottom losers don't care
3) Too many directors have never had it happen to them, so they don't comprehend the problem.
 
To me, the next steps are:
1) Gather real examples from history of these issues.
2) Document several solution ideas
3) Educate all TX directors on the issue, examples, and proposed solutions.
4) Use Survey Monkey to ask the director to rank the proposed solutions.
5) UIL to tell directors that a solution will be implemented, but they get to pick the which one.
 
Of course, any outlier survey responses will be hotly debated here.   LOL.

I feel like a lot of the directors believe it is fruitless to even go up against UIL so why even try? Its probably why some bands don't really care much about even going to state. They know they'll get a more fair shake at BOA.
 
Maybe we do need to move on, but why can we not have a conversation about it first? Educate people such as myself on what happens behind the scenes. I was certainly unaware before this season of some of the scoring issues within UIL. Maybe if we educate more people on this stuff, more parents will ask directors why it isn't changing. More voices would equal change, surely. Having a conversation is a good place to start.

If I may weigh in on the Round Rock and judge 5 issue, these are my thoughts:
 
1) Although I was disappointed for Round Rock at first, and my kids went there at one time, I know the staff and kids have moved on. It was a bad thing for them and is something they will have to live with. I am not going to tell them in any way that it doesn't matter as it certainly does matter.
 
2) I know judge 5. I know what kind of person he is/was and I know his past as a director and judge. All I am going to say is I am not the least bit surprised. I will stop at that and will say that he did what he was hired to do - give his assessment of the bands that he judged. I am in no way saying that he was correct, but UIL chose to hire him and paid him. They knew his past as well. One other thing - there were other issues with his comments that could easily be regarded as trivial, heavy-handed, and inconsistent as well.
 
3) What really bothers me is the fact that a) one judge was allowed to dictate the placement of one band far beyond what the other 4 did and the fact that in the end, almost no one outside of the affected band program cares about this, including UIL.
 
The fact that the judging system doesn't have a safeguard in place to keep a judge that is significantly different than the other 4 (20 places in this case) from having that much of an influence should be a serious concern of every band director, student, and band parent or fan. I don't care who this happens to, I don't want to see it happen to anyone! This is a flaw in the system and needs to be addressed for ALL bands who compete in this arena. If this isn't allowed to happen, then the problem no longer exists. The rules can be changed to deal with this in fair and appropriate way.
 
And that brings me to what really bugs me most of all - why this isn't likely to change. First, the people who made the finals don't care about this. It didn't affect their placement and it either didn't or never has happened to them so why go to bat for some band that they are in competition with who was less fortunate than them? No issue for them at all. Second, the other bands that didn't make the finals don't care about this. They didn't make it anyway so why would they care about one band that ended up ahead of them didn't make it? Big deal, they should just be happy to be as far up as they were, right? Third, the band directors that didn't compete this year or don't ever make it to State don't care about this. Doesn't affect them, either. Not their pain, not their problem. Fourth, UIL doesn't care. This band wasn't one of the very top bands and they know no one will even comment or complain about it. Nothing for them to do. Fifth, there is no accountability for the offending judge. No one questions him or her ever and there isn't even a discussion about it. Check written, deposited, and spent like there was never an issue at all. He or she is ready to judge again and likely will.
 
I think it is a shame that this can still happen again and could once again be allowed and forgotten. This judge will very likely be hired again and the potential for this to happen again not only exists, but is likely as well. Sadly, no one really cares enough to do anything about it. At least there are not enough of us who do and are willing to try.

Reviewing ordinal scores/placements should be triggered by a percentage deviation. An ordinal point swing from 1 to 10 as happened at Area B or 1 to 9 at another Area is much more statistically significant than a swing from 74 to 84 or 100 to 110 as could theoretically happen at BOA SA or Indy Grand Nationals.
 
Percentage statistics are almost always more informative, revealing, and helpful than counting statistics.

It was the same judge, who did the exact same thing to Round Rock this year as he did six years ago. It seems like a personal vendetta to me. (I had a kid in the Dragon Band in 2012, and my second kid is in Dragon Band now. It was devastating for them both times.)
 
In my opinion, the fastest and easiest "fix" for such circumstances would be to slightly revise the "bad judge rule" (the same one that saved Leander at Area this year) to protect bands in the future from one really, really, really off-base ranking like this. If the "bad judge rule" were extended or modified to address a huge discrepancy like a 20-place difference between two judges supposedly judging the same thing, 13 bands would have gone to finals this time. Yes, UIL judging needs far more reform than that; obviously there needs to be better clarification of judging criteria and improved training/calibration of judges. But anything at all to protect bands from such wild variations in ordinals would help. It would be a start, and a relatively quick and easy one at that.
 
Please know that in my opinion all 12 bands that went to finals this year absolutely deserved to be there, and Round Rock was not positioned to medal this year regardless, so this is not sour grapes or undue bias on my part. In fact, This was my last UIL year as a band parent, so I no longer have a horse in this race. But the statistician who posted earlier is absolutely right -- you other parents and directors and fans of the marching arts need to carry this torch, or it will indeed keep happening every year, and it will continue to delegitimize the competition in the public's minds.

I think a not insignificant number of directors would defect from UIL if it weren't for administration/school board/community pressure to stay in.

It was the same judge, who did the exact same thing to Round Rock this year as he did six years ago. It seems like a personal vendetta to me. (I had a kid in the Dragon Band in 2012, and my second kid is in Dragon Band now. It was devastating for them both times.)
 
In my opinion, the fastest and easiest "fix" for such circumstances would be to slightly revise the "bad judge rule" (the same one that saved Leander at Area this year) to protect bands in the future from one really, really, really off-base ranking like this. If the "bad judge rule" were extended or modified to address a huge discrepancy like a 20-place difference between two judges supposedly judging the same thing, 13 bands would have gone to finals this time. Yes, UIL judging needs far more reform than that; obviously there needs to be better clarification of judging criteria and improved training/calibration of judges. But anything at all to protect bands from such wild variations in ordinals would help. It would be a start, and a relatively quick and easy one at that.
 
Please know that in my opinion all 12 bands that went to finals this year absolutely deserved to be there, and Round Rock was not positioned to medal this year regardless, so this is not sour grapes or undue bias on my part. In fact, This was my last UIL year as a band parent, so I no longer have a horse in this race. But the statistician who posted earlier is absolutely right -- you other parents and directors and fans of the marching arts need to carry this torch, or it will indeed keep happening every year, and it will continue to delegitimize the competition in the public's minds.

It was the same judge, who did the exact same thing to Round Rock this year as he did six years ago. It seems like a personal vendetta to me. (I had a kid in the Dragon Band in 2012, and my second kid is in Dragon Band now. It was devastating for them both times.)
 
In my opinion, the fastest and easiest "fix" for such circumstances would be to slightly revise the "bad judge rule" (the same one that saved Leander at Area this year) to protect bands in the future from one really, really, really off-base ranking like this. If the "bad judge rule" were extended or modified to address a huge discrepancy like a 20-place difference between two judges supposedly judging the same thing, 13 bands would have gone to finals this time. Yes, UIL judging needs far more reform than that; obviously there needs to be better clarification of judging criteria and improved training/calibration of judges. But anything at all to protect bands from such wild variations in ordinals would help. It would be a start, and a relatively quick and easy one at that.
 
Please know that in my opinion all 12 bands that went to finals this year absolutely deserved to be there, and Round Rock was not positioned to medal this year regardless, so this is not sour grapes or undue bias on my part. In fact, This was my last UIL year as a band parent, so I no longer have a horse in this race. But the statistician who posted earlier is absolutely right -- you other parents and directors and fans of the marching arts need to carry this torch, or it will indeed keep happening every year, and it will continue to delegitimize the competition in the public's minds.

I was even going to say I could see them in tears....but that's not immediately obvious from the videos that are up either.   It's been a number of years since I saw this show, I'm wondering if maybe there was a better quality video up at some point that eventually got taken down.

Check out the 2004 show (Ballet for Martha), the Simple Gifts finale is spellbinding, and probably the best performed rendition of Simple Gifts by a HS marching band.  They may have placed 4th that year at Nats, but the show was polished enough to where they could've won with the right panel (a 1.2 point gap from 1st-4th).  They won Outstanding Music that year as well as placing 1st in Vis GE, and it's clear why.  But more than that, this is the most intense show I've ever seen as far as the level of buy-in from the kids is concerned...it just oozes passion and energy.  Their dance team always had a sense of poise and stage presence back then that is similar to bands like Broken Arrow in the modern era....that rare feeling just from looking at their eyes that the kids really know what they're doing and why they're there.

Yes. Yes they do.
 
Mad respect for what Vandegrift is accomplishing by the way. I'm just curious about what seems like an apples and oranges comparison between dance teams and color guards.

I'd say so.  However a program decides to create effect is up to them.  I can see why it would be difficult to convey visual effect with a dance team alone when you have guards tossing rifles that nearly hit the ceiling.  Then again, Vandegrift seems to be doing just fine

Stephen F. Austin only had a dance team and was extremely successful at Grand Nationals:
 
2002 - 5th Place
2004 - 4th Place 
2008 - 8th Place
2010 - 9th Place
 
No "medal" placements, but their integration of the dance team was second to none and watching those ladies do the 2004 show was inspiring.  It can totally be done.  

Yeah, she made it all the way to the end through sheer determination, and then she just fell over backwards during the final pose and had to be carried out. They said she's fine now and performed in finals without mishap.

Round Rock made Lucas Oil Stadium its own personal Minotaur and bashed its head in.  GO ROCK BAND!

Facts. 
 
Marian is amazingly consistent and have some of the most different and unique shows year after year.

They fully earned their final spot with an extremely complex and well-designed show, and their students achieved a lot on the field and had the run of their lives.
 
Period.

They do too! One guard girl passed out at the very end of semis!

Texas bands are also at a disadvantage nationally because of our 8-hr/wk practice rule.