2018 6A State

[principalagent]

Is 6A state not being factored into Hornrank? Doesn’t make sense for them to use area results but not state results, as I believe that Vista Ridge should be higher than what they’re currently at.

For one, it’s a little more BOA focused a website to make national comparisons easier. And second, the ranking contributions were due before 6A finals ends. Expect some more changes in next week’s ranking. I don’t think Vista will end up lower than 6th.

[TxDragonDad]

Congrats to all the bands who qualified for 6A UIL State competition in 2018!   Wow.  So many talented performers and programs.   Texas reins above all in this regard.   The top bands brought amazing shows and performed them splendidly.  Wow, just wow!    My comments below should not take anything away from those bands.  

 

I've been waiting to decide what to post about all this.  

 

It is no surprise that Round Rock kids are working through the emotions of the reality of the judging scores.   The fact is that 1 judge was 23 rank points away from the average of the other 4 judges and that significance happened to cause them to cross over the finals qualifier threshold.  In fact if J5 had agreed with J4's score, RR would have ended up 10th at Prelims.  Round Rock knew that a Championship was a LONG shot.  But their goal was to make finals at state.   

 

From what I saw, they had a lot of work to do to achieve that goal.  Each performance they brought more to the field.  Their State Prelims run was their best yet.  They truly out marched all the other prior performances.   Those 4 judges' scores validate this truth.   Given BOA Austin Area, BOA SA, they weren't expected to get much higher than 17th at State.  The band had their sites set on 12th, to make finals.   It is such a travesty to watch these excellent performers continue to push and improve toward their goal.   They made huge strides each day.   They were on path to reach their goal   They knew it, too.   Despite being predicted to be somewhere near 17th place, they knew they could achieve higher.  They worked for it.  They had drive and passion.   And according to 4 of the judges, they succeeded!

 

So there we are.  An outlier score.  "Happens every year", so I hear.   Maybe somewhat but 27 rank places delta across the judges, every year?   I may come across emotional.  I am.   But there is some mathematical significance to this situation (see more data below).    This should never happen, to any band, ever again.

 

And if nothing changes in UIL scoring, your child is next to fall victim.    Every parent on here needs to know that it is easy to dismiss as "just how it goes" until it happens to your only child on their senior year.   Just take a moment and let that sink in.   It doesn't matter that it was Round Rock who got hurt this year.  It matters that some band full of seniors got hurt and your child and your band could be next...  unless there is enough support to demand a change.

 

Some have offered up the solution of more judges and throw out top/bottom scores.   The concern with this is cost.   Let's consider this more fully.  Say we want to retain 3 music and 2 marching scores, after throwing out top/bottom from each category.  That means we need to add 2 more music judges and 2 more marching judges, four total.    Just estimating lodging, airfare, and meals, plus compensation for time, I estimate that those four judges would cost about $6k.  If UIL 6A competition has about 1500 paid ticket sales, then ticket prices would need to go up by $4 to cover that additional cost.   So, Moms and Dads - would you pay $4 more per ticket to prevent the outlier problem from happening to your kids' band?    (If ticket sales are 3,000, then that is only $2 more per ticket to cover costs.)   Seems like a feasible suggestion.  

 

Another idea is to use a statistical formula to identify outlier scores and reduce their weight when calculating the final ranking score.   This would require a threshold of defining an outlier and then a sliding scale such that the correction does not overly compensate for the outlier but softens its impact.   This type of approach requires no more judges and basically a single Excel spreadsheet template to interpret the raw scores into judge ranking and adjusted ranking.   I've already drafted one, but need to tweak it some.  (My day job is data modeling and analytics.)   Would UIL be willing to take such a tool and simply enter judges scores (not the rankings) and let the tool soften outlier impact?   They could key in several past event scores to test the tool for "face validity", etc.   Do they want help solving the problem?  Do you want to encourage them to seek a solution?

 

So it comes down to these simple questions:  

1) Are significant outliers a problem?

2) What is significant?  

3) Should UIL Address it?  

4) Should there be an outlier test performed before final ranking?

5) Does it matter more when outliers impact placement across the finals qualifier threshold?

 

IMHO, every band parent and director should want UIL to pursue a solution to prevent outlier impact in the future, especially when it causes finals qualification issues.

 

Respectfully,

TXDragonDad

 

Some data from this year's prelim scoring: 

When comparing the highest judge rank to the lowest judge rank for each band, we get the rank delta for each band.   The average rank delta in prelims was 10.39, and the median was 10.  The top 8 prelim bands all had rank deltas below 10, which suggests high agreement on the quality of their performances.  

 

Round Rock had a rank delta of 27, followed by John Horn with a rank delta of 21.   They were the only two bands with a rank delta above 17.   In this event, anything above 17 appears to be significant.   Round Rock's rank delta was 2.6 times the average & 2.7 times the median.    This is not just a mild outlier situation, but an extreme one.

 

Also, I took each judge's rank and compared it with the average of the other four judges.  I did this for every judge for every band.

J5 was the most significantly out of line.    

   J5's max deviation from average:  23.25 rank places.   (That means J5's rank was 23.25 places away from the average of the other 4 judges!)

   J2 max deviation from average:  12.50  

   The other judges were lower.  

J5 was nearly 2x the outlier of the next judge and of each other judge. 

 

Recall that Round Rock had a rank delta of 27, followed by John Horn with 21 rank spots.   Round Rock lost placements while John Horn gained placements due to outlier impact.

These two bands were exceptionally skewed by a single judge.  In both cases it was J5.  

School    	Judge 1	Judge 2	Judge 3	Judge 4	Judge 5
Round Rock HS 	7	11	11	14	34
John Horn HS  	26	19	31	25	10

Outside of these observations, the rest of the scoring had no hugely significant deltas or outliers.  There were additional "normal" outliers.   

[Mash]

Dragon dad, that seems very thorough and makes sense.  It sounded like Danpod had some connections that could help in the UIL committee.  My gut tells me the committee will just say "This has always worked, why should we change"  but I hope that they will look into how to continually improve the judging. 

 

I am not sure if there are any other ways to make this know in a calm and rational discussion like yours seems to be.   We obviously want to keep the human part of judging in so there has to be a fine line about what can change.  Hopefully they will be able to take your suggestions and from others to tweak the system for the betterment of all the kids in the future.

 

On a side note: Congrats to a great season this year.  The band should hold their head up high no matter what a single judge says.  I know a few kiddos in the Dragon band and I know they are upset and confused about the situation.  

[PSHSMom]

So there we are.  An outlier score.  "Happens every year", so I hear.   Maybe somewhat but 27 rank places delta across the judges, every year?   I may come across emotional.  I am.   But there is some mathematical significance to this situation (see more data below).    This should never happen, to any band, ever again.

 

 

Back in 2012 when PESH made it to state (they made BOA SA Finals that year) they had a judge that marked them next to last in music I believe (so maybe 30th, I don't have the exact data) .  Had it not been for that one score they would have made the finals there too.   It knocked them down to 19th I believe.   Yeah, It would be nice not to see this kind of situation again.   

[TWHSParent]

Since we were the 12th place band in prelims, I've been reluctant to comment. I also have my biases, and I will clearly state that in my opinion Round Rock should not have been in finals over us this year. We were fully expecting that going first with such a deep field in 6A (and I recognize that Cedar Park won 5A last year going first, but 5A lacks the depth of 6A) would be almost problematic. So what's the answer? Take 11th, 12th, 13th, and 14th and have a march off on an even playing field? Top 2 advance? That gets rid of the timeslot issue (and it is an issue - going first will always push a score down in a strongly competitive contest), and allows direct comparison to get the right bands in finals. Congratulations to Round Rock for a very strong push at the end of the season. I've been a fan since I started paying attention to marching band again in 2015. It was especially great to see your program make finals at GN last year - the excitement from your kids was a sight to see. I was excited that our kids put a strong enough performance out there from the 1st slot to stick in the top 12 throughout prelims, and back that up by jumping to 9th in finals (interesting that we were 12th after both prelims this weekend, and jumped to 8th at BOA and 9th at UIL after finals). Everyone did a great job, and it was a pleasure watching the programs put out their best efforts.

[bingogooberman]

i will just say that in a field so deep, so rich in ability, technique, and flat out raw talent, it's incredibly difficult to discern who's better than who or to say that because one judge out of five, and I'll remind everyone that each of them have had long, successful careers in music education and adjudication (one of them was even my high school band director and I trust his opinion implicitly), doesn't agree with the others that they are wrong, or that anyone was wronged. Yes, in an ordinal scoring system, one lower ranking can have a significant impact on your final placement, particularly in preliminary competition where those ordinals can be well into the double digits, but that's just the rules of the game, and looking at the 12 programs that made finals, it's difficult to disagree with the results. I know it stings to be so close to finals and miss, I've definitely been in that spot, but just barely missing finals at this, probably the most challenging marching contest in the nation due to its emphasis on execution and perfection, says more about how amazingly talented the Round Rock band program is than it does about UIL's imperfect scoring system. Also just a side note in J5's defense, it isn't at all surprising to me to see John Horn score in the top 10 visually- that show has compelling, challenging drill that they executed near-flawlessly. 

[LostChoirGuy]

I think it is important to realize that the judges aren't necessarily outliers when they are judging different things. Based on the entire season's results, Round Rock was significantly stronger in music than marching, mostly due to their show not doing them any favors. Also, the two marching judges are positioned on opposite sides of the stadium to watch the performance, so who knows what differences they see in the show.

 

I do think there should be something done to limit these occurrences, but I also think we shouldnt assume that all judges shoukd be scoring the bands the same or even near each other.

 

My freshman year of high school choir, after making it past District, Region, and Pre-Area rounds of All-State, I went to the final round to try to get into the all-state choir. There were 20 people competing for 8 spots and 5 judges scoring us. I got placed 4th, 5th, 11th, 19th, and 20th. I came away very confused about how two of the judges thought I was pretty awful (the last place judge gave me lower scores than I had ever seen before) and how two of the judges thought I was great and easily worthy of making the all-state choir. My choir director talked with me and helped me realize how judges have many different opinions on what makes a beautiful sound and what they are looking for and why that is one of the marverlous things about music. If there was one specific "way to sound good" then we would all be bored and music would lose its power to affect us. She encouraged me to study the judges comments for necessary changes, but to mostly focus on the things that the two judges liked about me because those were my strengths as a musician. This was some of the most valuable advice I ever received and I would not have learned it, at least as early in life, if it wasnt for discrepancies in adjudication.

[Dband2018]

An update: judge 5s main comments on our (round rocks) Tape recording were that we "fluttered too much" and that our skirt "covered our marching".

[Somethingbandrelated]

Honestly, I dont think Vista Ridge was as clean as Flower Mound, the two bands were so close I believe it came down to Flower Mound performing so much earlier.

[LiciaMommycott]

Honestly, I dont think Vista Ridge was as clean as Flower Mound, the two bands were so close I believe it came down to Flower Mound performing so much earlier.

 

 

Proud VRHS mom here and I'm no judge of marching skills and obviously I love my kid's show. However, I really believe the Vista show gripped people this year in such an emotional way that it may have put us over the top. We were glued to the live blog and the love we received from Landry and the Daniels was so amazing to see.  But when Daniel Sanchez said, "That transition back into lost destroys me every single time. There are multiple people up here in the press box with tears," my next thought was "OMG we might have just won it!" That was the point where I thought we could actually take home the gold.  Today the kids listened to the recording of the judges comments in class and the judges were crying during our show. How could a show that moves you so much not get points for that? Maybe I'm wrong, because I know nothing but dang that was one fine show and those kids marched it beautifully! And we started the competition just wanting to make it to finals (speaking as a parent). Being a medal contender was so off my radar.

[scoobydoo]

Area D 2012 a judge made the same comment about Round Rock's leg cape. I hope it wasn't the same judge. RR has had leg capes for 9 years. Maybe a judge like that shouldn't judge marching.

[Jeffrey L. Gorman]

I am sorry that a High School Band Marching Contest could bring on such frustration.  I do not know why Round Rock had one judge not care for  their Marching(Judges 4 & 5 Did Marching and Maneuvering) in 6A.   I am sorry that you feel that someone else,s child will be punished because of what one judge thought of their Band. Every judge at the event has taken a Band to the SMC. They all have had to deal with situations like yours, because Judges are humans also. To be honest Round Rock was between Vista Ridge and Flower Mound. They were not in the same class as either of those two Bands. I am sorry but that happens, if they had been between two Bands of lesser quallity, they very well might have had higher Ordinals. 

 

I have one other comment and you probably will not like this. I would like to tell you a story of a Judge who took a way a Placling for the Band I was assistant director for and what the rest of the story was.

 

In Nov 1973, I was he Assistant Band Director at a major High School in Southern CA which had an outstanding Marching Band. In CA at that time the All Western Band Review was the unofficial State Championship matching the top 72,yes I said 72 High School Bands in Street Marching competition.  The Bands were divided into 6 Classes of 12 Bands Each by School enrollment. Besides the Sweepstakes and Music Awards the Top 5 Bands in each class were recognized with place trophies.  We had done such a great performance on the Street to Barnum and Baileys Favorite March, we were very sure we would finish with the 2nd Place Trophy in Class

 

The awards ceremony was held in the Long Beach Arena before 12,000 screaming Band members and Parents and Fans.  It was also held with TV cameras and all sorts of dignitaries.     When the awards were given in our class  we got a 4th Place trophy instead of the 2nd place we thought we had one.  In fact the Band that got the 2nd Place Award had played the Same March and we were convinced we had beaten this school. Our Head Director, Ross Davis, one of the most decorated Judges in the SCSBOA went to pick up our score sheets as I took the Band back to the buses. We were not happy campers.

 

Ross met me at the Buses and said there had been a major problem in the music scoring.  In CA you have to provide copies of the music score to the music judges. It seems that one of the 3 music judges had wrongly placed our Score of the March in another schools packet and sent it to the judges for tabulation. We received the other schools score for the music instead.  They also played Barnum and Baileys Favorite. 

 

Let this be clear to our Band and 12,000 people we were 4th Place not the 2nd place we should have been. How did this happen.

A week before the All Western  one of the music judges contacted SCSBOA and informed the chief judge that he had been given a diagnosis of an inoperable Brain Tumor which would take his life in 6 months. The judge had been given medicine to handle the blackouts and headaches that were causing him a great deal of pain. . The Honor of judging at All Western in CA was a High Honor for those selected for this event. The Chief Judge was asked to allow the Judge in Question to serve as one of the Music Judges. The Medical Dr. serving as the Judges Physician felt that there would be no problems him judging.     That day when we marched past the Judges we received the 3rd Highest music score of 72 Bands, however when the tabulators got the scores, in our packet  were the results of another Band who did not score anywhere near us.  The Judge in question had a blackout after the other Band went through and when he came out of the blackout he changed the score sheets for two schools. The Judge was contacted by both Schools and the SCSBOA. He admitted that this error had been his and he blamed the blackout for the mistake.  The directors of both schools and the SCSBOA agreed that no good could come of announcing that a Judge with a Terminal Illness had made a mistake.  As Ross reminded me. THIS IS HIGH SCHOOL BAND, A persons life and career mean more to us than a Trophy.

 

When we got back to the Bus, we told the Band this.   "You Did an Outstanding Job.  We feel that you should have finished higher. It did not happen today.  However we are proud of each and every one of you"   Some day you will understand things we cannot tell you"

In 1985 at the Band Alumni dinner for the Band of 73 and 74, shortly before Ross Davis passing, he told the Band what happened. To a person they agreed that the trophy was not worth ruining the legacy  of a  judge

 

This is how you should look at this event.

[Dband2018]

 

Area D 2012 a judge made the same comment about Round Rock's leg cape. I hope it wasn't the same judge. RR has had leg capes for 9 years. Maybe a judge like that shouldn't judge marching.

Our band director said it was (I dont want to name the judge however)

[Bandwoww]

 

I think it came down to the fact that Vista Ridge had THE BEST designed show. It effectively showcased the band’s talents in exactly the right ways so that there was very little room for error. Vista did not have the most difficult show. By any means. But the students were given exactly what they needed to win. Yeah, they had to work extremely hard for it, but the reality is that all of these finalist students spent the same amount of time rehearsing, etc. It’s blatant that there was a major shift in direction and design this year for Vista. Not to take anything from them, but it’s the truth. Vista had the perfect show, and that’s RARE.

 

Now, did the staff at Vista go into the season knowing they were going to win? No.

 

But was this direction and design change inevitably going to put out a program that was pretty unstoppable? Yes.

[vincentlee1220]

Proud VRHS mom here and I'm no judge of marching skills and obviously I love my kid's show. However, I really believe the Vista show gripped people this year in such an emotional way that it may have put us over the top. We were glued to the live blog and the love we received from Landry and the Daniels was so amazing to see.  But when Daniel Sanchez said, "That transition back into lost destroys me every single time. There are multiple people up here in the press box with tears," my next thought was "OMG we might have just won it!" That was the point where I thought we could actually take home the gold.  Today the kids listened to the recording of the judges comments in class and the judges were crying during our show. How could a show that moves you so much not get points for that? Maybe I'm wrong, because I know nothing but dang that was one fine show and those kids marched it beautifully! And we started the competition just wanting to make it to finals (speaking as a parent). Being a medal contender was so off my radar.

 

wow, I wish I could hear some of these tapes. this is like the time a judge exclaimed "holy brass" at our first hit at 2013 Area D finals, but at a whole 'nother level.

[Mash]

wow, I wish I could hear some of these tapes. this is like the time a judge exclaimed "holy brass" at our first hit at 2013 Area D finals, but at a whole 'nother level.

I remember those tapes, I seem to remember them saying good things about a certain soloist as well :-)  

 

I would love to hear the tapes, I bet they are interesting.

 

The difference between FloMo and VR was very close.  It could have easily gone the other way between the two schools.  Timing in finals could have made a difference, it also  could be other small factors.   The differences between the performances were just that close.   Both schools should be extremely proud of their program.

[Somethingbandrelated]

 

Proud VRHS mom here and I'm no judge of marching skills and obviously I love my kid's show. However, I really believe the Vista show gripped people this year in such an emotional way that it may have put us over the top. We were glued to the live blog and the love we received from Landry and the Daniels was so amazing to see. But when Daniel Sanchez said, "That transition back into lost destroys me every single time. There are multiple people up here in the press box with tears," my next thought was "OMG we might have just won it!" That was the point where I thought we could actually take home the gold. Today the kids listened to the recording of the judges comments in class and the judges were crying during our show. How could a show that moves you so much not get points for that? Maybe I'm wrong, because I know nothing but dang that was one fine show and those kids marched it beautifully! And we started the competition just wanting to make it to finals (speaking as a parent). Being a medal contender was so off my radar.

Oh I wouldn’t dispute that for a second. Absolutely incredible show design from whomever designs for VRHS. I would’ve preferred some more challenging music, but really thats my only complaint. Even so, the music was extremely well executed. However, UIL does not factor GE into their scoring. They are all about execution, and performance time is HUGE in UIL’s horribly flawed scoring system. After watching both FloMo and VRHS performances, I believe FloMo simply performed better than them, however they went extremely early, and VRHS went close to last. And again to clarify, I support the heck out of VRHS. They should have beaten hebron at BOA easy.

[Somethingbandrelated]

Cont. thoughts:

That show had a similar effect on both the crowd and judges that Leonardos Dream (FloMo 2016) had. That is quite an acheivement.

[LeanderMomma]

I am sorry that a High School Band Marching Contest could bring on such frustration. I do not know why Round Rock had one judge not care for their Marching(Judges 4 & 5 Did Marching and Maneuvering) in 6A. I am sorry that you feel that someone else,s child will be punished because of what one judge thought of their Band. Every judge at the event has taken a Band to the SMC. They all have had to deal with situations like yours, because Judges are humans also. To be honest Round Rock was between Vista Ridge and Flower Mound. They were not in the same class as either of those two Bands. I am sorry but that happens, if they had been between two Bands of lesser quallity, they very well might have had higher Ordinals.

 

I have one other comment and you probably will not like this. I would like to tell you a story of a Judge who took a way a Placling for the Band I was assistant director for and what the rest of the story was.

 

In Nov 1973, I was he Assistant Band Director at a major High School in Southern CA which had an outstanding Marching Band. In CA at that time the All Western Band Review was the unofficial State Championship matching the top 72,yes I said 72 High School Bands in Street Marching competition. The Bands were divided into 6 Classes of 12 Bands Each by School enrollment. Besides the Sweepstakes and Music Awards the Top 5 Bands in each class were recognized with place trophies. We had done such a great performance on the Street to Barnum and Baileys Favorite March, we were very sure we would finish with the 2nd Place Trophy in Class

 

The awards ceremony was held in the Long Beach Arena before 12,000 screaming Band members and Parents and Fans. It was also held with TV cameras and all sorts of dignitaries. When the awards were given in our class we got a 4th Place trophy instead of the 2nd place we thought we had one. In fact the Band that got the 2nd Place Award had played the Same March and we were convinced we had beaten this school. Our Head Director, Ross Davis, one of the most decorated Judges in the SCSBOA went to pick up our score sheets as I took the Band back to the buses. We were not happy campers.

 

Ross met me at the Buses and said there had been a major problem in the music scoring. In CA you have to provide copies of the music score to the music judges. It seems that one of the 3 music judges had wrongly placed our Score of the March in another schools packet and sent it to the judges for tabulation. We received the other schools score for the music instead. They also played Barnum and Baileys Favorite.

 

Let this be clear to our Band and 12,000 people we were 4th Place not the 2nd place we should have been. How did this happen.

A week before the All Western one of the music judges contacted SCSBOA and informed the chief judge that he had been given a diagnosis of an inoperable Brain Tumor which would take his life in 6 months. The judge had been given medicine to handle the blackouts and headaches that were causing him a great deal of pain. . The Honor of judging at All Western in CA was a High Honor for those selected for this event. The Chief Judge was asked to allow the Judge in Question to serve as one of the Music Judges. The Medical Dr. serving as the Judges Physician felt that there would be no problems him judging. That day when we marched past the Judges we received the 3rd Highest music score of 72 Bands, however when the tabulators got the scores, in our packet were the results of another Band who did not score anywhere near us. The Judge in question had a blackout after the other Band went through and when he came out of the blackout he changed the score sheets for two schools. The Judge was contacted by both Schools and the SCSBOA. He admitted that this error had been his and he blamed the blackout for the mistake. The directors of both schools and the SCSBOA agreed that no good could come of announcing that a Judge with a Terminal Illness had made a mistake. As Ross reminded me. THIS IS HIGH SCHOOL BAND, A persons life and career mean more to us than a Trophy.

 

When we got back to the Bus, we told the Band this. "You Did an Outstanding Job. We feel that you should have finished higher. It did not happen today. However we are proud of each and every one of you" Some day you will understand things we cannot tell you"

In 1985 at the Band Alumni dinner for the Band of 73 and 74, shortly before Ross Davis passing, he told the Band what happened. To a person they agreed that the trophy was not worth ruining the legacy of a judge

 

This is how you should look at this event.

Wow, that is just a truly incredible story. Incredibly sad for that judge and for your band. Also incredibly moving that you and the head director would make the hard choice of saving a man’s reputation over getting justice for your band. Impressive stuff, there. I feel like those are the truly valuable lessons that we learn in life. Having integrity. That’s what matters at the end of the day.

[TxDragonDad]

Back in 2012 when PESH made it to state (they made BOA SA Finals that year) they had a judge that marked them next to last in music I believe (so maybe 30th, I don't have the exact data) .  Had it not been for that one score they would have made the finals there too.   It knocked them down to 19th I believe.   Yeah, It would be nice not to see this kind of situation again.   

 

I think if enough people see the risk, change is possible.   I am trying to show to my child that sometimes bad thing happen to good people through no fault of their own.  The best reaction is to look for ways to make things better, especially if you won't ever benefit from the improvement.  Leave a legacy that helps others.    Hopefully this example will add yours to make the point that change is needed.

[Mmalpica9]

There are two quotes from Randy Pausch's "the Last Lecture" that I think applies here:

 

"The brick walls are there for a reason. The brick walls are not there to keep us out. The brick walls are there to give us a chance to show how badly we want something. Because the brick walls are there to stop the people who don’t want it badly enough. They’re there to stop the other people."

 

and...

 

"Experience is what you get when you didn't get what you wanted. "

 

with 3 kids we have experienced all kind of situations on any kind of environments (the referee annulling a valid goal in state finals, a judge not noticing points scored in a tae kwon do sparring tournament or not making the dance team) they were powerful drawbacks that somehow defined their next steps...

 

I guess eventually a video analytics program will be able to analyze a marching band and score it in the most un-biased way... I'm not sure if we would want that. Music is like food or other artistic activities... are subjective in nature. It sucks when the subjectivity affect us but this is the life of the artists... and all the kids marching this weekend, putting countless hours on practice... they are all artists. They'll be hurt, learn and come back stronger.

[Rudedog34]

 

Congrats to all the bands who qualified for 6A UIL State competition in 2018! Wow. So many talented performers and programs. Texas reins above all in this regard. The top bands brought amazing shows and performed them splendidly. Wow, just wow! My comments below should not take anything away from those bands.

 

I've been waiting to decide what to post about all this.

 

It is no surprise that Round Rock kids are working through the emotions of the reality of the judging scores. The fact is that 1 judge was 23 rank points away from the average of the other 4 judges and that significance happened to cause them to cross over the finals qualifier threshold. In fact if J5 had agreed with J4's score, RR would have ended up 10th at Prelims. Round Rock knew that a Championship was a LONG shot. But their goal was to make finals at state.

 

From what I saw, they had a lot of work to do to achieve that goal. Each performance they brought more to the field. Their State Prelims run was their best yet. They truly out marched all the other prior performances. Those 4 judges' scores validate this truth. Given BOA Austin Area, BOA SA, they weren't expected to get much higher than 17th at State. The band had their sites set on 12th, to make finals. It is such a travesty to watch these excellent performers continue to push and improve toward their goal. They made huge strides each day. They were on path to reach their goal They knew it, too. Despite being predicted to be somewhere near 17th place, they knew they could achieve higher. They worked for it. They had drive and passion. And according to 4 of the judges, they succeeded!

 

So there we are. An outlier score. "Happens every year", so I hear. Maybe somewhat but 27 rank places delta across the judges, every year? I may come across emotional. I am. But there is some mathematical significance to this situation (see more data below). This should never happen, to any band, ever again.

 

And if nothing changes in UIL scoring, your child is next to fall victim. Every parent on here needs to know that it is easy to dismiss as "just how it goes" until it happens to your only child on their senior year. Just take a moment and let that sink in. It doesn't matter that it was Round Rock who got hurt this year. It matters that some band full of seniors got hurt and your child and your band could be next... unless there is enough support to demand a change.

 

Some have offered up the solution of more judges and throw out top/bottom scores. The concern with this is cost. Let's consider this more fully. Say we want to retain 3 music and 2 marching scores, after throwing out top/bottom from each category. That means we need to add 2 more music judges and 2 more marching judges, four total. Just estimating lodging, airfare, and meals, plus compensation for time, I estimate that those four judges would cost about $6k. If UIL 6A competition has about 1500 paid ticket sales, then ticket prices would need to go up by $4 to cover that additional cost. So, Moms and Dads - would you pay $4 more per ticket to prevent the outlier problem from happening to your kids' band? (If ticket sales are 3,000, then that is only $2 more per ticket to cover costs.) Seems like a feasible suggestion.

 

Another idea is to use a statistical formula to identify outlier scores and reduce their weight when calculating the final ranking score. This would require a threshold of defining an outlier and then a sliding scale such that the correction does not overly compensate for the outlier but softens its impact. This type of approach requires no more judges and basically a single Excel spreadsheet template to interpret the raw scores into judge ranking and adjusted ranking. I've already drafted one, but need to tweak it some. (My day job is data modeling and analytics.) Would UIL be willing to take such a tool and simply enter judges scores (not the rankings) and let the tool soften outlier impact? They could key in several past event scores to test the tool for "face validity", etc. Do they want help solving the problem? Do you want to encourage them to seek a solution?

 

So it comes down to these simple questions:

1) Are significant outliers a problem?

2) What is significant?

3) Should UIL Address it?

4) Should there be an outlier test performed before final ranking?

5) Does it matter more when outliers impact placement across the finals qualifier threshold?

 

IMHO, every band parent and director should want UIL to pursue a solution to prevent outlier impact in the future, especially when it causes finals qualification issues.

 

Respectfully,

TXDragonDad

 

Some data from this year's prelim scoring:

When comparing the highest judge rank to the lowest judge rank for each band, we get the rank delta for each band. The average rank delta in prelims was 10.39, and the median was 10. The top 8 prelim bands all had rank deltas below 10, which suggests high agreement on the quality of their performances.

 

Round Rock had a rank delta of 27, followed by John Horn with a rank delta of 21. They were the only two bands with a rank delta above 17. In this event, anything above 17 appears to be significant. Round Rock's rank delta was 2.6 times the average & 2.7 times the median. This is not just a mild outlier situation, but an extreme one.

 

Also, I took each judge's rank and compared it with the average of the other four judges. I did this for every judge for every band.

J5 was the most significantly out of line.

J5's max deviation from average: 23.25 rank places. (That means J5's rank was 23.25 places away from the average of the other 4 judges!)

J2 max deviation from average: 12.50

The other judges were lower.

J5 was nearly 2x the outlier of the next judge and of each other judge.

 

Recall that Round Rock had a rank delta of 27, followed by John Horn with 21 rank spots. Round Rock lost placements while John Horn gained placements due to outlier impact.

 

These two bands were exceptionally skewed by a single judge. In both cases it was J5.

 

School    	Judge 1	Judge 2	Judge 3	Judge 4	Judge 5
Round Rock HS 	7	11	11	14	34
John Horn HS  	26	19	31	25	10

Outside of these observations, the rest of the scoring had no hugely significant deltas or outliers. There were additional "normal" outliers.

Very similar to the analysis I did for Area D. Inconsistencies abound. The anomalous scores "Do Not" reflect what should be an accurate measurement based on the governing body's judging rubric.

The way I see it, in BOA it's all about GE and the number of decibles created and is subject to a judges preference. Or what I call "Pie in the sky scoring".

In UIL it is supposed to be about the quality of music and marching and the combination there of. A judges preference should not come into play. At all. And UIL ranking should not be determined by people's choice.

Again. UIL Adjudicators should be held accountable and the systems for scoring should be updated.

 

Congrats to all the programs that competed for the big prize. They were all amazing shows and extremely deserving for placement in finals.

[PSHSMom]

Just wanted to say congrats to all of the bands who performed. It has been a challenging season between early school start dates in August to rain. rain, and more rain. To put out shows of this caliber is purely amazing. Although in order to compete these days it appears I need to move from PISD to one of the LISD areas!

[Parkwoodmom]

When will this system change??????!!! Been band watching since 2002...

[1998-2018]

There has been a great deal of concern expressed about UIL scoring, accountability, and whether or not UIL leadership is also concerned or even interested. I know the head of UIL Judging and I can assure you they are extremely concerned. However, UIL is not a dictatorship. When it comes to changing rules, it's more like a democracy. In fact you could argue it even has similarities to an electoral college. In the spring a presentation will be made to all the Directors regarding proposed changes. The significant opposition to change is coming from Directors, not leadership. The presentation will not be everything UIL leadership wants. It will represent a compromise they hope will pass. A Director from a small school with a handful of kids in their band program that never achieves a Division I rating at the Region level will have the same vote as the Director of Allen High School, representing over 700 band kids, or the Directors of the SMBC Finalist. Maybe that's fair or maybe not. Either way that's how it works.

 

Blaming "UIL" for this is a bit too nonspecific. This issue is squarely in the hands of the Directors. They need to stop complaining and start influencing!

 

Do you know if your Director plans to vote for change? What have they done to persuade others, particularly at small schools, to also support change? Have they told Directors who are resistant to change the stories of bands negatively affected by judging deviations? Have they listened to the concerns of those small school Directors and helped them overcome misunderstandings regarding how change will affect them? Have they mentored those Directors through the changes that will affect them?

 

You want accountability? It start with Directors. I have a question for all of you. What is your Director doing about this?