Design a Better Sprit System

33 posts / 0 new
Last post
#1

Now that we've seen the perennial spirit system critiques start to bloom, I am reminded that
the definition of insanity is doing the same thing over and over, while expecting a different
result. Chances are low that anything will improve, if all we do is make the same complaints
as always.

Instead, the 'combined genius' of the forum could put their efforts towards figuring out
aspects of a better spirit system, if not a better system itself. At the very least, we could
discuss what we like and dislike about different aspects of ranking spirit.

I don't have a lot of suggestions on how to improve the system, because I don't personally
have much of a gripe with it. Most of the teams I've been on have seen the ugly side of the
system (skewing score in both directions). Ultimately though, for a variety of reasons, I don't
place a lot of value in the current system.

What say you? What would you like to see?

Transparent and accountable critiques between teams?
Completely anonymous critiques?
An accurate multi-dimensional survey?
A simple question: "Would you want to play this team again? yes/no/only if we win"?
A system where scoring was optional?
A system where scoring is mandatory before seeing your score?
Public rankings so you can see what you're up for next game?
Private rankings where the best are given kudos and the worst are privately 'dealt with'?

Before launching into the enevitable and yearly debate on this issue....
I seem to remember signing up at the captain's meeting to join a group that was suppose to work on league-wide spirit. I would assume working with the current spirit system was a part of that groups responsibilities.

However, I haven't heard anything from the league in regards to the spirit group (or any other group I signed up for).

G5 By G5

I would certainly like to know what parts of my team's spirit are lacking and/or strong. For example, do we consistently get docked points for a dangerous player, or on the other half of the ledger, are we good at playing games like "Splat"?

What about....
If you give an opposing team a really poor spirit score (Say 5 or under within our current system) you also HAVE to send a message explaining your reasoning. Perhaps this would help in opening a dialogue between the teams to help resolve the issue(s).

The few times my team has been assessed poor scores over the years, I have always contacted the other captain to ask for an explanation (especially when my team thought the game went well). More often then not I have found it to be misunderstandings and simple things that could have been resolved easily had the "spirit-offended" team bothered to communicate with the "spirit-offender" team.

Just a thought.

Hey G5, Captains have access to this information. You can get a summary of all individual check boxes, and you can find out what your opponents checked on a game by game basis also.

Hey G5,
You know captains can see what other teams answered for each spirit question, right? In the Team Admin section, click 'view spirit rating' in the sidebar. You'll see the totals for each question, and you can select a specific game to see what the other team submitted for that game.

Far Side,
Apologies for the lack of communication. I'll be in touch before the end of the month.

Craig.

I think there's a bit of a dilemma with the spirit rating system in that many want a granular
system whereby specific criteria are measured, but most often just give out spirit based on a
completely subjective out-of-11 score.

Frequently if a team or just a captain really liked or really disliked a game, they will give a
spirit score based only on that, and somewhat spuriously check or uncheck the criteria boxes
to add up to that score.

Personally, I don't think it will be possible to change the behaviour of captains to get them to
give scores solely based on criteria. So the hopes are pretty minimal that we'd ever have a
system that actually represents a team's meeting set criteria.

Further, I don't actually like that a spirit score is based on the addition of specific criteria. I
don't think this works because not all criteria are created equal. Why should "knowing the
rules", "showing up on time", "being helpful during disagreements", "team cheered us or
played a game", etc all carry equal weight on a team's spirit? Sure, you could weight each of
those differently, but I think that's just asking for trouble.

Since I just don't think you can have an accurate spirit system made up of more granular
criteria, I think the best would be an 'overall spirit' score. The trouble with this was that
there's room for huge variance in interpretation between team to team. A game that one
team gives a perfect score might be given a significantly lower score by another, because the
two captains have different idea of what spirit is. This was apparent when spirit scores were
out of 7, and it would be much more so if they are out of 11. There are ways to minimize this
though.

Maybe we should think about what we want out of a spirit system, and that will help us solve
the problems with the current system.

Here's what I'd want out of the spirit system.

R1. Recognize teams that show good spirit.
R2. Recognize teams with truly exceptional spirit.
R3. Provide constructive feedback to all teams.
R4. Privately identify and initiate Coord mediation to resolve/diffuse any conflicts.
R5. Privately identify to the Coord teams which may need League attention regarding spirit.
R6. Provide scores as consistent with the criteria as possible from captain to captain.
R7. Keep scores private throughout the season.
R8. Scoring will best reflect true meaning of SOTG, and minimize the effect that winning,
giving treats, or playing games, etc have on the score.

Perhaps if we can get a complete list of requirements for our spirit system, we'll be able to
craft a better system. Please chime in with suggestions/comments.

I've got an idea for a system percolating, I'll share it tonight after I have time to write it
down.

--

Note on R2: We really need a system that differentiates between "I liked that team, no
problems at all: 11." and "Wow! That team was a shining example of how ultimate should be
played! Why can't I give them a 12?".

Note on R7: This would be a leap for most in the league, but I really don't see any value in
highlighting teams with great/poor spirit during the season. Actually I see problems with it in
that it can often act as a self-fulfilling prophecy when a team goes into a game knowing that
the other team has 'bad spirit'. There's zero value in knowing that before the game (if there
were, the league should have already stepped in), and it can only add to the potential
problems.

Note on R8: Too often teams are penalized or rewarded based on the wrong criteria. It's
nearly impossible to get top marks in spirit if you win by a wide margin. Unfortunately it
seems under our current system, usually the only way to get a perfect score is to be an
overly friendly team who plays a silly game after loosing. There's no reason that a team that
does that shouldn't get a top mark (depending on the game), but there's plenty of good
reason why a team shouldn't have to do that to get a top mark. That isn't SOTG, it's an ugly
caricature of SOTG.

Here's an idea I've been kicking around. The idea could use a lot of discussion, suggestion,
and/or refinement.

1. Games ranked on a three point scale of opponent's sportsmanship: Opponent showed good
sportsmanship, Opponent showed neutral sportsmanship, or Opponent showed poor
sportsmanship.

2. A (short!) survey of some form will provide specific feedback on certain aspects (like the
current checkboxes), eg: on time, handled disputes, knowledge of the rules, etc, etc. This
survey does not factor into the Spirit score, it is only for constructive comment purposes.

3. A team can give a 'kudos' to their opponent to highlight truly exceptional SOTG. Teams
have a limited number of 'kudos'. Perhaps limited to 1 per term, 5 over the whole year, etc.

4. As always, if there's a serious concern about at team's play, the Coord is to be contacted
directly.

--

Rationale:

1. Having a wide scale for a purely subjective spirit score tends towards greater inconsistency.
One captain's 8 out of 11 is another captain's 10 out of 11. You'll get much greater
consistency with a good/neutral/poor scale. And really, is there effectively any difference in a
team with an 8 out of 11 and a team with 9 out of 11? What really matters is if a team
displays good sportsmanship or not, not where a team is on some artificial gradient of
sportsmanship.

I think using "sportsmanship" (itself the core of SOTG) as the rating criteria will offer the
closest measurement to true SOTG. Too often the word "Spirit" is confused with 'happy-
clappy fun-time', so that even using the word "spirit" would only serve to measure the wrong
thing.

2. I think every team would want constructive feedback on how they can improve in certain
areas. However, when you factor the feedback on those specific areas into the score, there is
a certain amount of incentive to lie, whether to be nice or spiteful to the reviewed team.

If it is clear that the feedback not only doesn't affect the score, but is only seen by the other
captain, then it makes lying pointless.

3. Having a limited number of 'kudos' that a team can hand out will eliminate the 'everybody
is excellent, so nobody is' problem with the current system. Teams will still identify the
teams that show sportsmanship and SOTG through the regular spirit system, but will be
forced to be selective before giving recognition of truly excellent SOTG. I'd imagine that
picking one team in their Div that deserves the recognition each term would be best. They
can have a fun name like: "Gold Stars", "Golden Cleats", "Spencies", "VULValorAwards", etc.

4. As always, serious problems should be brought to the Coord right away. I don't think
there's any point in being able to differentiate between a 1/11 and a 3/11. More often than
not all that signifies is that somebody's nose got bent out of joint, and there's no added
information beyond 'the team was not sportsmanlike'.

--

So, I think a system that worked like that would do well to serve what we want out of a spirit
system, and would eliminate a lot of the common complaints. I'd love to see some
comments, suggestions, alternative ideas, so fire away.

Rob By Rob

Temple, this is very similar to what I was thinking. A stripped down system with three options: "Great", "Normal" and "Needs Improvement", or something along those lines.

I would like the system to be optional (with no entry being assumed to be a "Normal" game), and for there to be text boxes, like you mention, that must be filled in with the rational for why a game was either "Great" or "Needs Improvement". I would also want there to be an "I communicated my concerns to the opposing captain during or immediately following the game" check box. All this together I think would help avoid having people just carelessly fire off a lousy score.

I think the scores should be kept private except for end of the year awards and for negative situations in which coordinators need to intervene. In this way, coordinators could use common sense to weed out illegitimate complaints from legitimate ones (for instance, a team that gives "Needs Improvement" to every other team when none of the other teams have a problem with one another), and 9 times out of 10 no one would have to hear anything about spirit scores all season (which is not to say that they shouldn't hear anything about spirit, which should be emphasized early and often as being central to the game - just no scores).

To me, spirit scores shouldn't be used to encourage sportsmanship - all that leads to is people performing to a check list of actions instead of being fully spirited (I see this often enough). Instead, spirit scores should serve to identify the truly exceptional (who would be exceptional with or without the spirit score) and report the problem teams to the coordinators - I think a three-point scale does this best.

I agree, strip it down to a three, or possibly five point system. Five would fit that fun loving Strongly Agree-Agree-Neutral-Disagree-Strongly Disagree format.

Teams put too much emphasis on winning, and far too often use a poor spirit score as redemption for losing a lopsided game. "We lost, so they must be jerks". I've been a captain for 6 years now, and with out a doubt, almost every single low spirit score received was after a blowout win. When you ask the captain directly why they gave such a poor score, and point out that yes, you did indeed play "SPLAT" with them, I get a:

"Oh, I forgot to check that box."
"What about the disputes, there weren't any!"
"Oh, yeah, sorry. I can change that if you want."
"You thought we didn't know the rules? When there weren't any disputes, and we had to explain what a strip was?"
"I didn't have fun."
"But everyone else seemed to!"
"..."

If that's the case, this system is "too complex", and too dependant what one person (the captain) felt like putting. It should be a rating based on what *The Team* thought you deserved, not one person.

Maybe I've had a lot of bad experiences. For the mostpart, I've found teams to be very genuine about the spirit system. In a lot of cases, too generous, which skews the rankings. A 5 was told to be an "average" rating with this system. 5s get raked over the coals!

Spirit matters. A league this large with this many players and teams could not exist if we just played to win and let things get out of control. You can't really ask for a better sport in a better setting with better people, and you should let them know that. If you do have a problem with a team, try to work it out on the field. Let them know via the spirit system constructively. Communicate!

G5 By G5

Right now, the spirit ratings are effectively a four-point scale (from 8 to 11). Is there value in advertising that almost every team scores at least 8/11 in spirit? The league looks more spirited that way, rather than having scores of 1/4, for example. In particular, it gives newbies the message that everyone is spirited, and they're expected to be spirited too. Without a doubt it's artificial to take a four-point scale and add seven points to everyone's score, but as I said, is there value in it?

Temple. You are having a lot of fun with this eh?

"Wow! That team was a shining example of how ultimate should be
played! Why can't I give them a 12?".

"A team can give a 'kudos' to their opponent to highlight truly exceptional SOTG. Teams have a limited number of 'kudos'. Perhaps limited to 1 per term, 5 over the whole year, etc."

"They can have a fun name like: "Gold Stars", "Golden Cleats", "Spencies", "VULValorAwards", etc."

I feel like I am in grade school again... Sorry. I just wanted to point that out.

--------------------------------------------------------------------------------

Well. It is quite complex. I completely agree with the eight points that you have stated in a spirit system.

In Edmonton there is a very simple method of awarding spirit scores. I believe it would achieve most of what is required. I do believe a few things will need to be tweaked and I will state them later.

1. The spirit score is out of ten. Simple enough.

2. The captains were then given a criteria and defined what would be deemed a ten out of ten, eight out of ten, and so on... This message was relayed to the players.

3. At the end of the game, all the players put there fingers and the average was taken.

Note: If large issues occur the captains have the right to send there opinions to the league coordinators

Pros

- The definitions allowed the players/captains to make an informed choice or opinion of the spirit score
- All the players at the game weighed in making it an average (A larger 'n' meaning it more robust)

Disadvantages

- No constructive feedback
- Doesn't reward or recognize good spirit (It does for exceptional)

Now, to address the disadvantages. It wouldn't be too hard.

1. Have a way for the captain to convey there thoughts about the game. I believe this shouldn't be forced. If the game was okay, is there anything you really need to be so specific about.

The team belongs in VUL; however, nothing is really good or bad about them.

2. I enjoy the idea that Temple mentioned about recognizing teams. I thought the names and all that were really cute too :) However, I believe this is a better method.

a. If the teams spirit score improves. Give them a small prize like a VUL disc.
b. If they do well... (In this example: maybe average of 8.5/10) they can get a fifty dollar rebate.
c. If they do extremely well... Yeah, you guys have this covered already. (This is when comments are important)

There could also be a method for those who's spirit scores are lacking.

I hope this make sense.

m2c By m2c

Lots of fun here, but might I add that WFDF took 2 years and talked to a lot of people to come up with what I think is a better, if not perfect, system.

As Temple posted a few back, it is based on a three-tiered rating system (0,1,2) with 8 questions. So the final rating is between 0 and 16. The big improvement (IMO) over the current VUL system is that the catagories are much more broad so, as has been mentioned, there is less chance that you can't "check a box" because something didn't occur. I think the limited scope of the VUL system questions is the major drawback.

As has also been mentioned, if the Captains don't buy in and just give teams a somewhat random score out of X then it doesn't matter what system you have.

So the WFDF catagories are:

1. Respect - Communicated Objectively, Showed up On time...
2. Fair-Mindedness - Pointed out their own fouls, willing to accept opponents view of calls, game situation
3. Positive Attitude - Introduced themselves, left a positive impression
4. Emotional Management - Reactions to game situations was appropriately mature
5. Avoiding Body Contact - Aware of other players' body location and avoided dangerous play
6. Avoiding Violations and Fouls - Avoided fouls and violations
7. Knowledge of the Rules - Knew the rules and/or had willingness to learn them
8. Their Spirit Compared to Ours - How was their spirit compared to ours? (0 = Worse, 1 = Same, 2 = Better)

The actual WFDF rule set has more detail behind each section. Overall I like the idea of more broad catagories. I also really like the last question as it is a good indicator. Also, I think that the data generated (again, assuming the input was thought out) could help the league direct low spirited teams. If a team always got a 0 for Avoiding Body Contact that tells you something different then if they always got a 0 for Respect.

Try thinking about your games this week and use this chart to rank your opponents. Maybe the VUL could adjust some of these areas, but I like the direction this goes in terms of ranking spirit. A step up from "Out of 10" or the current VUL system I think.

Brian

Brian, I like it, but I've got a question for you.

I still see the likelihood that teams are going to give out a lot more 2s 'to be nice' (or 0s to
be spiteful). Having those feedback questions count towards the score will always add this
skewing. And having a large scale (16 in this case) will open the door for greater
inconsistency between captain to captain). The ones that think they're being 'nice' might give
out a 14/16 to a team that really isn't better then a team that got 10/16 from a more
objective captain.

This is mostly why I thought that separating this feedback from the actual spirit score would
help give more consistent results and most importantly better feedback.

My question is, why factor those individual questions into the score? Definitely have them required when filling out spirit, but do you think it's better to be able to differentiate teams
that have 14/16 and 12/16, if the cost is less consistency in those scores and less reliability in
the feedback itself?

Yes, with a single 3-point scale question that affects spirit, you lose the ability to identify that
Team A was 37th in league spirit with 13.6/16 and Team B was 43rd in spirit with 13.5, but
I don't even think it's worthwhile publishing that, let alone calculating it.

One other benefit to an overall 3-point spirit scale is that when somebody is being spiteful,
they'll often give a much lower score than the team deserves. Under our current system,
they'll give a 1 or 2 out of 11. If say that team should have deserved a 7/11 that's a drop of
6 or 7 spirit points out of spite. That single team's score will affect that team's spirit average
more than the next 5 games they play.

The reverse is true to a certain extent. No longer will captains giving out perfect scores skew
the results too often. A perfect score under a 3-point scale means "good". Most teams should
get those perfect scores, because the other team enjoyed playing them (personally I think
identifying that is 90% of what I'd want out of a spirit system).

Under a 3-point overall scale, one good game will make up for one bad game. I kind of like
that. You lose the ability to highlight truly exceptional teams, but that's where the
bonus/kudos come in. With a limited number of these to hand out, teams will be forced to be
judicious, and they will identify only the best they play.

m2c By m2c

If the majority of Captains are not going to care about the data input then no system will really help. I agree that 8 questions may not be the answer, but what I do like about a system like this is that it might make the Captain think about different aspects of Spirit, rather than just giving a 1, 2 or 3 each week. And of course with the Captains that don't really care, a 3 each week.

This is why I like the last question of the WFDF system. Do you think the team you played against was better or worse in terms of Spirit than your team. Again, if we discount the Captains who just don't care, then this might actually have an impact if a Captain is putting down a 2 (they were better than us) each week. Maybe not.

I do agree, however, that using a year long system that is not taken seriously by the majority of Captains is no way to single out either good or bad spirited teams. For awards, maybe we should allow teams to vote (or have a few votes) for the best team(s) they played all year long. But then the problem there is that the typical team only plays between 8-10 of the other 239 VUL teams during a season.

To help with data collection how about making the Spirit score system optional? Don't make recording spirit scores manditory. Like the provincial election, only the people who bother to show up should get to have a say in things. So maybe we would only get 20% of teams recoding data, but wouldn't a small amount of accurate data be better than a large amount of inaccurate data?

B

"If the majority of Captains are not going to care about the data input then no system will
really help. I agree that 8 questions may not be the answer, but what I do like about a
system like this is that it might make the Captain think about different aspects of Spirit,
rather than just giving a 1, 2 or 3 each week. And of course with the Captains that don't
really care, a 3 each week."

I don't think that captains not caring is really a concern. Rather, it's a question of sometimes
misunderstanding the criteria or being misguided in the scoring. It's inevitable that people's
subjective scores will vary from the stated criteria, but that's not a reason to just give up.
There is a solution to that issue.

I like that question of yours, I'd love to see that in the survey. I just don't like that extra
questions count (and are weighted equally) in the score.

The problem with having more questions *count* towards the score (I like that there are
several questions *asked*), is that, with more questions counting, the variance of the scores
will be higher. Also, the more individual misunderstandings (or more commonly misguided
attempts to positively/negatively skew the score), the less worthwhile the overall spirit score
becomes.

Ultimately the spirit system has two main goals: 1) recognize good spirit and 2) give
constructive feedback to the team. I don't think that combining those two into the same
questions works (for all my stated reasons above).

I think if you separate out the scoring from the feedback, you'll get equally good (or even
better) scoring, but more importantly, you'll get more valuable feedback.

--

"I do agree, however, that using a year long system that is not taken seriously by the
majority of Captains is no way to single out either good or bad spirited teams. For awards,
maybe we should allow teams to vote (or have a few votes) for the best team(s) they played
all year long. But then the problem there is that the typical team only plays between 8-10 of
the other 239 VUL teams during a season."

We do that now, but I agree it's kind of pointless. We pick the 'best' based on the spirit
scores, but rarely do those teams play more than 10 other teams all year, so it's only those
teams' scores that are counting.

Granted it's early yet, but as of this writing 225 out of 238 teams have a spirit of 9/11 or
greater, and 138 have 10/11 or greater. That will change somewhat, but our system will
never be able to differentiate great spirit from good spirit. Really our current system can't
adequately identify the 'best teams'. All it does is show us that the majority of teams have
good spirit.

I think having a limited number of 'kudos' per season or term (kind of like a vote I suppose)
will at least allow for selectivity. Teams won't be able to hand out 11s to every team. I think
the number of times each team was selected 'most spirited team in this div, this term' would
be a pretty good indicator of which teams deserve prizes/accolades.

--

I don't think either the spirit scoring, or the feedback should be optional. I also don't think the
numbers would be more accurate (actually I think they'd be less, as only the people with
problems speak out for the most part).

Not only would the data be even less useful, but the real problem would be that it would
serve to remove SOTG from the collective consciousness.

I like that rating spirit and giving feedback is required, even if that system is flawed.

Wouldn't the 'kudos' probably end up being hoarded until the end of the season, as they are
viewed as being scarce? Then you'd find yourself with handing out kudos at the end of the year
just to use them up. That's my initial impression of a potential downside.

You could solve that by having to give out 1 or 2 per semester.

--

I really like the WFDF feedback/spirit score. It is more informative and seems much less arbitrary. It also seems easier to give general guidelines of what good spirit would be for each category. I.E. Avoiding fouls and violations: 0 - any intentional or reckless fouling/violations, 1 - some fouls/violations, though no apparent intent or recklessness there is room for improvement in avoiding them, 2 - visible effort to avoid fouling and violations / rare, if any, violations/fouls, all unintentional. Or something like that anyway.
Followed by a comment line for constructive criticisms __"your team chronically fouls the thrower, maybe discuss marking technique with them"___

m2c By m2c

If we were in a reality show situation (God forbid) then I could understand limiting the Kudos, but what is the point of that in real life. As Biker said, a limited number would impact their use, maybe not for the best.

Let teams send in a Kudos (text, not just a checkbox) any time they want. We WANT to hear from the Captains who would bother to do this, as I think most would not, no matter if they had a limited or unlimited number.

So if you play a great team in week 1, with only 3 kudos do you save it? No, just send a note saying that your game against the Farm Accidents was fantastic and they even baked your team a cake.

Over the course of the year it would be interesting to see what happened. Maybe one or two teams get many, many kudos. Maybe some teams abuse the system. However, if team X gave a kudo to both teams they played each week it would be easy to discount their input.

One other thought on the overall system. I don't think a very simple system helps matters. Design a system that an idiot can use and only an idiot will use it (See FPP). So if we have a system that was just -1, 0 or +1 because we want to simplify responses, but then we suddenly don't get any useful data.

Again, I would like to see a system that, assuming we got good input, gave the league and teams some good feedback. I think that would go along with not making the system manditory.

Brian

In addition to a honed -1/0/+1 set of checkboxes (the WFDF categories look solid), I like the idea of a text area for additional comments (both positive and negative). I think that all comments entered should be publicly visible on the team page, along with the associated spirit scores. The result would be a feedback 'stream' that everyone could see - somewhat akin to ebay's feedback mechanism. This would help hold captains accountable for both their team and the feedback they give.
There are other elements that could be borrowed from ebay - badges for teams that consistently rate highly, persistence between seasons/years so teams that display exceptionally good spirit for five years in a row or reach 100 games with positive feedback or win the spirit award, etc. all get some kind of badge on their team page.

Brian: "Over the course of the year it would be interesting to see what happened. Maybe one
or two teams get many, many kudos. Maybe some teams abuse the system. However, if
team X gave a kudo to both teams they played each week it would be easy to discount their
input."

There's the rub, how do you know when a team's 'kudos' should be discounted? What you'll
get is what you have now, everybody giving that extra bonus for adequate spirit. If that's the
case, then there's no point in having that extra bonus.

I think maybe using the word 'kudos' is a bad idea in the discussion. You always want to give
constructive comments/thanks/kudos to a good team. Rather the concept I envision is a
method of giving out a 'bonus mark' or 'gold star' what have you. The only purpose for that is
to recognize teams that are truly superior in spirit (if that is indeed even a desire). If a team
gives out 15 gold stars in a season (which some absolutely will) then it makes the entire
process pointless, because you won't be able to identify the best.

I don't think limiting the number is going to lead to problems. If each team picks the best
team in their div at the end of every term, it would make a fair amount of sense. That is of
course in addition to the weekly ranking where a team can still give top marks for spirit, give
constructive feedback, and write a shining note to the team that you really liked playing
them.

--

Brian: "One other thought on the overall system. I don't think a very simple system helps
matters. Design a system that an idiot can use and only an idiot will use it (See FPP). So if
we have a system that was just -1, 0 or +1 because we want to simplify responses, but then
we suddenly don't get any useful data."

Suddenly don't get any useful data? The current system does not give any useful data.
Currently 95% of the teams in the league have a spirit score of 91% or higher!

My point is that the score itself is not useful. All it does is show whether a team has good
spirit or not. We shouldn't expect the actual score to do anything more than that, because in
the real world, it never will. What we can do, by scaling the system down to a 3-point scale is
at least make the scores more consistent.

Regardless of whether that is a less consistent out-of-11 scale or a more consistent out-of-3
scale, there's never going to be much value in the spirit score beyond showing that the team
has good or poor spirit.

Let's forget about the scale of the spirit score itself. Where the *real value* comes in is the
constructive feedback questions. If you factor those questions into the spirit score, you'll
always cause those answers to be skewed. That's just reality. We can hope that captains
would 'follow the rules' but that's a bit futile.

By separating the feedback questions from the actual numeric score, you cut out almost all
the incentive to fudge (for good or bad) the feedback. This I think would be the single most
useful change to our spirit system.

--

Brian: "Again, I would like to see a system that, assuming we got good input, gave the
league and teams some good feedback. I think that would go along with not making the
system manditory."

By changing the structure of the spirit system, I think you can make the feedback much more
valuable. Making it voluntary would be terrible in my opinion. It would be a big step towards
the obsolescence of spirit.

Rob By Rob

"Making it voluntary would be terrible in my opinion. It would be a big step towards the obsolescence of spirit. "

--

I really disagree with this. I have played in leagues with and without spirit scoring and it has never had an effect on the spirit on the field - which is central to the game whether you give a score to it or not.

If anything, when I've played in "scored" games I've seen people perform to the score card instead of being genuinely spirited, which has contributed to a more negative atmosphere on the field.

To me, the system of giving people a score to ensure that they are spirited is much like parents paying their kids to do well in school - it's hollow.

That said, I think we all agree on the values of the spirit score for informing coordinators of great or problematic teams - which is why I still support the idea of a spirit score, so long as it is part of a simplified, voluntary reporting system focused on text boxes, not checklists.

What leagues don't have a spirit system that you've seen play with spirit? Were they large in
size? I've only played league ultimate in the VUL, but I've played in a lot of other sport
leagues (various hockey/softball/jai alai/volleyball/etc). SOTG/respect/etc was relatively non-
existent in those leagues.

As I mentioned way back at the top of the thread, I don't put much value on our current spirit
system. There's too much encouragement to play to the spirit score as you say. While I don't
think that's inherently a bad thing, it is when the wrong behaviour gets rewarded with the
higher spirit score.

I see no problem with teams playing with better sportsmanship than they would merely for
the sake of increasing their spirit score (that would be amazing!). The problem is that under
our current system, where high spirit scores mean "we like you, because you gave us treats
and played a fun mini-game with us after we beat you", that 'popularity contest' behaviour is
what's being encouraged by the spirit system.

Sure c/ultimate players will never need a spirit score to have them play with spirit, but the
average player? I'm not so sure that without having a spirit score in the back of their mind,
that they wouldn't forget about spirit in relatively short order.

If you make rating somebody's spirit voluntary, only the c/ultimate players and those with a
gripe are going to use it. The c/ultimate players (and the teams they've probably played
against for years) aren't really going to need the guidance of a spirit system, and the rest will
likely only see it as a mechanism for bitching about the other team.

I'd sooner see voluntary score reporting than voluntary spirit reporting.

"To me, the system of giving people a score to ensure that they are spirited is much like parents
paying their kids to do well in school - it's hollow."

Rewarding achievement is the linchpin of our culture AFIAK.

Rob By Rob

Sorry, should have been more specific - I meant different ultimate leagues. Comparing ultimate to other sports re: spirit doesn't get you very far... And no, none of them were large leagues, which might be an issue, though personally I don't think a very big one.

I hear what you are saying re: a voluntary system, but I think you are misrepresenting what a voluntary system would look like. It wouldn't have to be some obscure form that you'd have to go through 12 pages of the website to find - it could be a page that came up every time you logged a score, asking you if the game was exceptionally good or bad. If not, you just move along and it gets reported as a "normal" game.

The only difference would be the wide scoring range that has been discussed, and the "everyone gets 9.5/11" spirit rankings - both of which, it seems from the discussion here, we generally agree aren't necessary.

I also think we are putting too much emphasis on the role of the scoring system in teaching VUL spirit. When I first came to VUL my captains explained spirit to the team at the start of the season, and once I was a captain myself I attended the captain's training session where understanding spirit and communicating the importance of it to your team were presented as being paramount. It's these kinds of interactions (in addition to, of course, witnessing spirited play on the field) that taught me about spirit in the VUL, not a checklist.

m2c By m2c

Ivar – The eBay idea is one of the better ones I have heard so far. The further point that a History of comments could be created could be a big improvement over the current state. Having this list come with a team from season to season gives some additional ownership to a teams Spirit. Even if you had some bad comments to start, I as a Captains would be happy to keep those available if there was then a steady improvement over time, it would show an effort was being put forward. A trend can be a powerful indicator/motivator. With the current system our spirit last year was a 9.2 and this year it’s a 10.4 – but who knows we got better?

Temple – “What you'll get is what you have now, everybody giving that extra bonus for adequate spirit”. --I think that most Captains enter spirit scores now because they must, and extra spirit scores happen because people don’t really care, and again, the system sucks. The idea that most/any Captains would bother to write up false spirit statements each week is farfetched. So let people who want to enter comments do so, most won’t. I can’t prove this, I just think it based on my experience dealing with many many Ultimate players over many many years. It would be interesting to see what happens in practice.

Temple – “Suddenly don't get any useful data? The current system does not give any useful data. Currently 95% of the teams in the league have a spirit score of 91% or higher!” -- Hey, calm down, nobody is defending the current system, I’m not. Actually the fact that there seems to be nobody on this thread that has even hinted that the current system should be kept or isn’t all that bad says something. Something chilling! What I'm saying is that a very simple system doesn't really help IMO. In actual fact, I think it would be easier for teams to just give a +1 each time out and you would still have the current problem that 90%+ of the teams would have a 90%+ spirit rankings. The ones with lower rankings might have bad spirit, or maybe they just played against Captains who take the system seriously.

Temple – “Regardless of whether that is a less consistent out-of-11 scale or a more consistent out-of-3 scale, there's never going to be much value in the spirit score beyond showing that the team
has good or poor spirit” -- With the current system that might be true, but I don’t think we can say that there is no yes/no multiple question system that can’t give good feedback. There is a whole industry that calls me every night to ask questions about all kinds of stuff that seems to think it is possible to get useful feedback this way. It’s the design of the system, not the overall system itself that is currently flawed. Or to put it another way, just because the current system is badly designed doesn’t mean that a similar system with a better design couldn’t give us better feedback.

Temple – “What leagues don't have a spirit system that you've seen play with spirit?” -- Here is a league that I have played in that didn’t record Spirit – the VUL! For the first 10-15 years of the league there was no talk about a quantitative spirit system. And before someone claims this was just when there were 4 teams in the league, Spirit scores didn’t come along until much later than that. Also, once the league had more than 25 teams each division was about the same size as they are now. In many practical matters, the VUL is just a bunch of 8-10 team leagues anyway.

Brian

m2c By m2c

So what would I like to see?

1. A system where Captains need to enter some Spirit Data as part of recording scores. A WFDF like 0/1/2 ranking over a number of different areas. This would create a "Spirit Score".

2. An addional optional feedback option for comments that would then be placed on the teams pages (feedback should not be anonymous - If I say you suck, it should have my name on it).

Would it be perfect? No. Would it be better than the current system? Yes.

BTW - Looking at Temples ideas, the final WFDF survey question, or an additional one, could actually be used to generate his suggested spirit score data.

Brian

Brian, I think you're missing what I think is the most important change. At least you haven't
commented on it.

I like the exact format of what you suggest, but factoring the feedback into the overall score
will lead to skewing of that feedback. The current system right now is not far from what you
suggest. The difference being yours has a 0/1/2 rating and a comment box for each criteria
and the current system has a 0/1 rating via checkboxes for each criteria (we all agree the
questions can change).

In practice how those are used is captains want to give a high or low score to a team, so they
check boxes to match that. Captains just can't rely on the feedback on the individual areas,
it's skewed horribly by the desire to give a certain spirit score. This isn't something that's
going to change just because you have a 0/1/2 scale instead of a 0/1 toggle.

However, if you display the exact same page to the captain, but make it clear that the
responses to the survey don't have an effect on the spirit score, then you'll get more accurate
survey responses.

Another downside is that all survey questions would likely count equally towards the numeric
spirit score (unless you added weighting, which would probably be too confusing). Eg: I don't
think the 'positive attitude' score nearly weighs as much as the 'avoids body contact' score. I
absolutely think they are important feedback to give, but why use that feedback
mathematically to craft the numeric spirit score?

I think we all agree that all a numeric spirit score is going to tell you is whether a team is
good or bad. It will never be able to differentiate between that 95% of the league in the top
9% of spirit. That's not a problem, no need to change that. What we do want to get out of
the spirit system is valuable feedback (in addition to the good/bad distinction).

What's the value-add of mathematically including the survey questions in the overall numeric
spirit score? Does that value make up for the down sides?

It really comes down to which do you prefer?

A) More valid feedback and a completely subjective spirit score.
B) Skewed feedback and a spirit score mathematically derived from that skewed feedback.

Obviously my preference is A.

m2c By m2c

It really comes down to which do you prefer?

A) More valid feedback and a completely subjective spirit score.

B) Skewed feedback and a spirit score mathematically derived from that skewed feedback.

+++

Like you I would pick "A" obviously, you convinced me! But hang on...something about your choices don't seem fair. I guess when you graduated from the Republican school of push polling that is what you get.

"but factoring the feedback into the overall score will lead to skewing of that feedback" - Your opinion, not supported by anything at all except that the current system, which we all agree is flawed, isn't providing the feedback we want/need. I attribute that more to the questions asked then the system itself.

"The current system right now is not far from what you suggest" - Wrong. The current system has questions that, IMO, lead to Captians not using it as intended but rather as just a mechanism to arrive at a score out of 11 that is meaningless (we agree here). The EXACT same system with better questions could give us better results.

"and a comment box for each criteria" Just for the record, I don't think 8 comment boxes would be a good idea. I would suggest just one that is optional to fill out. Teams can make free form comments about other teams. These comments would remain part of the teams records, season over season. - Props to Ivar again for this suggestion.

"This isn't something that's going to change just because you have a 0/1/2 scale instead of a 0/1 toggle" Again, you missed the point because you want to support your arguments. Just changing the scale won't change the results. Changing the questions being answered could.

"Another downside is that all survey questions would likely count equally towards the numeric spirit score" Agtain, just a downside in your opinion to support your argument. It's hard to calculate how much better one aspect of Spirit is from another. Personally I'm annoyed when a team doesn't have 7 players ready to go until 6:52 or start every game with only 2 women.

"What's the value-add of mathematically including the survey questions in the overall numeric spirit score?" To increase the overall score so we have a chance of getting a range of scores that will allow us to rank teams. You seem to be quick to assume that any such system will fail because one similar system failed. Not good thinking.

BTW - At Worlds when the WFDF system was used the range of scores was much lower and broader the the VUL system tends to show. The average was around 11 or 12 out of 16, rather then the 10/11 we tend to see at the VUL.

It really comes down to which system you prefer?

A) An improved system that allows us to better rank teams in terms of Spirit and provides some general guidance to problems and specific feedback

B) A simplistic system that is basic enough for 2 year olds to understand but tells us nothing even when used correctly.

Again, my preference is A. I will assume that yours is as well.

Brian

P.S. Hey Stumpy, this is a lot of work, do you do any special exercises that enable you to keep arguing with Temple all of the time?

I hadn't thought there was an argument. Most of what you took strong issue with was the
assertion that factoring the feedback into the score skews that feedback. I think this is a
generally held belief (doesn't mean it is true), and since I'd stated it repeatedly, and you'd
not hinted that you disagreed with it, I thought we were on the same page. Obviously we're
not.

Let's start with what (I think) we agree on.

A. I think we agree that the current VUL system results in skewed feedback, because captains
regularly check or uncheck boxes in order to give a good or bad score. Correct?

B. I also think we agree that the fundamental difference between the WFDF system and the
VUL system is that the feedback questions are of a higher quality. The fact that the scale is
changed from a 0/1 to a 0/1/2 is not that significant. Yes the WFDF system has a blank text
box for added feedback, but essentially, they are similar in that they gather feedback on
individual areas and an overall score is derived from that. Correct?

Now, you've said that if the feedback questions are of a better quality, then it will eliminate
the phenomenon where captains rank feedback areas spuriously in order to increase or
decrease the overall spirit score. I just don't see this as realistically occurring in the VUL.

For example:

VUL: Opponents know the rules, or have a willingness to learn them.
WFDF: Knowledge of the Rules - Knew the rules and/or had willingness to learn them.

I suggest that if a captain is using the above VUL question "as just a mechanism to arrive
at a score", then they're going to use that WFDF question the same way. I can't see how
arriving at better wording is going to change that particular behaviour. I see how arriving at
better wording will *improve* the feedback, but I don't see how better wording will change
how captains use the scoring system.

Now maybe you fault my example, but unless you think VUL captains *never* spuriously
mark the rules questions (those questions amount to 15% of the VUL feedback form), then
you should admit that some captains will continue to spuriously mark that question on a
WFDF-style spirit feedback.

Maybe you can give an example of how behaviour will change. I'm just not seeing it.

--

"Agtain, just a downside in your opinion to support your argument. It's hard to
calculate how much better one aspect of Spirit is from another. Personally I'm annoyed when
a team doesn't have 7 players ready to go until 6:52 or start every game with only 2
women."

Ok, this is interesting. You point out that having them weighted all equally might not be a
downside, but then you mention how some teams will care more about certain aspects than
others. Do you see there's inconsistency? The league would have to decide that they are all
equally weighted (or choose a different weighting system, but I think that would be too
complex).

--

I admit that my previous A/B question was loaded (although I had thought it was loaded only
with assumptions that we agreed upon). Here is what I believe is an objective distillation of
the issue.

A) Factoring feedback into the spirit score results in two options for the captain:
1. Give accurate feedback, which results in a mathematically derived spirit score which they
may or may not agree with.
2. Manipulate the feedback to arrive at a spirit score which they desire.

B) Feedback separate from the numeric spirit score results in two options:
1. Give a spirit score desired, and give accurate feedback.
2. Give a spirit score desired, and give inaccurate feedback.

I really think that A2 is much more likely in the VUL than A1. Additionally, I think B1 is much
more likely than B2. With A, you often have to make a choice between the score and the
feedback. With B, you never have to make that choice.

"BTW - At Worlds when the WFDF system was used the range of scores was much lower and
broader the the VUL system tends to show. The average was around 11 or 12 out of 16,
rather then the 10/11 we tend to see at the VUL."

As an aside, I don't think you can draw any conclusions on how the VUL might use a spirit
system based on how ultimate players from around the world used that spirit system
differently than the VUL uses its system. The culture of ultimate and presumably spirit is so
different in other regions that there's simply no way to draw conclusions on the VUL based on
experiences in those other regions. The same goes for comparing elite level tournament play
to local league play.

There is just so much difference between Worlds and VUL League play, that we can't assume
differences in the spirit results to be due to the differences in the systems themselves.

"P.S. Hey Stumpy, this is a lot of work, do you do any special exercises that enable you to keep
arguing with Temple all of the time?"

Believe it or not, I'm the quiet, un-opinionated one in my family.