The following is a document by Andrew Hart which is intended to help teams write difficulty appropriate questions for ACF Winter. It can alternatively be viewed on the hsquizbowl.org forums at this location.

Target difficulty of packet submission

Since ACF Winter has very little history to go on, the editors are concerned that teams will miss the target difficulty. The conception of Winter has varied, but this year’s iteration will be “regular” difficulty. Teams writing packets for ACF Winter 2010 are instructed to follow the difficulty levels of past regular-difficulty tournaments like ACF Regionals and Penn Bowl.

I will attempt to elucidate just what the editors envision by “regular difficulty:”

Before we start, I want to produce some understanding about the basic distinction between a novice tournament and a regular tournament. There is a basic threshold of knowledge that will be required to get ten points on a bonus, or to convert a tossup, and some teams will probably not be up to that threshold. Some tossups and some easy bonus parts will be on very easy answers, and some will be on less easy answers, and that’s not a problem.

There are going to be teams playing this tournament who simply fall below the basic threshold of knowledge so much that they are not part of this tournament’s target audience; we surely encourage them to play if they are enthusiastic about doing so, but we cannot write the tournament with them in mind. When you consider how easy or difficult a tossup answer or bonus part is, it is not necessary to take into consideration these teams.

For tossups:

1. Tossups will not exceed seven lines in the final tournament; submissions may exceed this amount, because the editors appreciate having more quality clues to work with.
2. Tossup answers should be chosen to maximize an ideal distribution of buzzes. This means that there should be a few very knowledgeable people buzzing on the first few clues, a significant number of players buzzing on the middle clues, and that a vast majority of teams will have buzzed by the end of the question. Tossup answers should not be chosen such that only a few very knowledgeable players are buzzing on the middle clues, and most teams are waiting until the end to buzz.
3. Do not pick tossup answers that will result in the tossup going dead for a significant number of rooms.
4. Your clues should gently slope from difficult to the giveaway; there should be no drastic difficulty cliffs. In order to accomplish this, you should always describe something before you give its name. The larger the contingent of players you think will be buzzing on the name, the more detailed the description of the thing should be.

For bonuses:

1. This tournament has specific conversion goals for bonuses as a whole, as well as specific difficulty standards to ensure that each bonus will be uniform.
2. On the whole, we want the very best teams to be able to well exceed 20 points per bonus. All well-balanced and knowledgeable teams should be able to get 20 points per bonus. The median bonus conversion should be around 15, and we don’t want many, if any, teams in the target audience to fall below 10 points per bonus.
3. For each individual bonus, we want to see a clear easy, middle, and hard part.
4. The easy part should be something that any team with basic knowledge of the subject at hand will know. Note that this means that the subject you pick has to somehow lend itself to an acceptably easy part (we’re going off of the standard that 90% of teams in the target audience should convert the easy part).
5. The middle part should test for beyond-basic knowledge of the subject at hand. It should also be eminently answerable to any team with more than the bare minimum of knowledge. Numerically, the median team in the target audience should be converting just over half of these parts, and the best teams should be converting this part almost all the time.
6. The hard part should test for deep knowledge of the subject at hand, but it should not be impossible. It should also be important and not trivial to the subject at hand. It should also not be a “stock” answer that is very guessable based on recent packets. Numerically, the median team should be getting these parts sporadically when they have deep knowledge of a subject, while the best teams convert them regularly but not perfunctorily.

Please follow these guidelines, which I think are useful not only for Winter, but for writing good and fairly standardized questions in general.

A general rule to sum this all up: When in doubt, err on the side of making your answers easier.