Monday, June 22, 2009

Question Quest Issues

Question Quest is a new feature at, allowing one to submit individual questions. These questions then get rated by the players in a special "new question game", and you get feedback on the rating and the % of players getting each question correct.

What is striking is the correlation between the rating and getting a question correct. My math isn't sophisticated enough to be able to calculate the probability that this correlation would occur by chance, but it must be lower than .01%. Of my 4 highest-rated questions, 3 of those are the 3 highest in terms of % who got them right. The 4th one comes in at #7 on the % correct chart. Of the top 9 on each list (out of 18 questions total that have been rated so far), 6 questions are on both lists. The 6 common ones are Chad, Gran Columbia, Medici, Hawaiian holiday, Oscar winner, and Gene McCarthy. The 3 which were favored with good ratings despite a % correct around what a pure guess would give (1 in 4) are a Burt Lancaster q, an Aesop's Fable q, and a Louis Armstrong q. The 3 downgraded below what the % correct would predict are Bloomsday, Escher, and Olympics q's.

All of my q's are rated "good", meaning above the .2 threshold. However, to receive special consideration they must also score above 55% correct, and only 3 of mine so far have done that. I am trying to work in hints in order to improve future q's in this regard.