THE RUBRIC: Looking Into a Festival Producer’s Mind, or How Do They Decide Who to Choose?
In the years I’ve produced the Alaska State Improv Festival (AS IF), I’ve heard performers ask “What can I do to improve my submission?” I’ve also heard my fellow producers bemoan that their 1 to 5 judging scale leads to a massive swath of “3” scores from their reviewers, making it impossible to figure out whether an ensemble is in the 25th percentile or the 75th percentile of the submission pool. More rarely, I hear of a rubric that takes a background in advanced statistics to comprehend where even the person creating it can only hope that it reflects more than a mathematical curiosity.
We do things a little differently in Alaska. The way we review submissions has made it much easier for our production team to determine where in the pool each ensemble fits. By describing the AS IF way, I hope that it helps performers understand what goes on in the mind of a producer. I also hope it helps other producers create a meaningful rubric for their own submission review processes.
Our Rubric:
The AS IF production team has always valued variety as well as skill. This became central to our scoring rubric which is divided into four parts, each scored on a 1 to 5 scale (minimum score of 4; maximum score of 20).
Originality – Does the show contain original elements? If we read the description of your show and say, “We’ve never seen that before. This sounds great!” then you’ve done well on originality.
Execution of Concept – Does the show deliver on its intended premise? If the show is described as an improvised detective show in the style of Columbo but the video contains a montage of disconnected scenes, then you probably lost almost all of these points.
Marketability – Can we describe your show to a lay person in one or two sentences and get them excited to see it? Think of a show like Jill Bernard’s Drum Machine – It’s a one-woman improvised musical based on a historical event. Her accompanist is the Zoomtronic 123 drum machine. Two sentences, you know the gist of the show, and you want to buy a ticket.
Skill – Regardless of whether the show was original or delivered on its promise, was it skillfully performed?
What’s The Reasoning Behind the Rubric?
We decided early that we wanted to provide not only quality shows, but to a broad spectrum of shows to introduce our audience to what improv can be.
Instead of asking “On a scale of 1 to 5, how were they?” the four parts of the rubric force the reviewers to consider more deeply what the show’s strengths and weaknesses, and whether the show is a fit for AS IF.
How Does It Work in the Real World?
Our results have been remarkably consistent. Groups that have received scores above 15 have proven to be exceptional additions to the festival. Groups in the 13 to 15 range have mostly been either placed on the waitlist or accepted as “last group in.” A couple of these groups have underwhelmed, but several have flashed brilliance that was not seen on the submission video. Groups who score under 13 are generally not considered for inclusion
Most importantly, we have been able to retain a consistently high bar for the festival. The sets at the festival are of high quality, yet are quite varied in style, composition, geography, etc., and the whole package reflects the overall vision of AS IF.
Anything Else?
The numbers don’t tell the whole tale. There are times when a producer makes a choice that goes against the numbers. This can happen based on personal knowledge of the performers, a reference from a trusted insider, a desire to not have too many of a particular type of show, or just a gut instinct regarding whether the submission video reflects the ensemble’s ceiling or their floor. If you are a producer, you have a right and a responsibility to look beyond the numbers to include the acts that best represent the festival’s vision.
If you are a performer who is caught on the wrong end of a producer’s decision, please understand that the number of submissions often greatly exceeds the number of available slots and that in a different pool of submissions you may have made the cut. If there is a festival you have targeted and you have not been accepted, don’t presume that’s the end of the story. Submit with another video, submit with a different ensemble, ask the producer where the submission fell short – and be ready to take the hard note.
So that’s a look inside the head of a producer heading into the submission period. Alaska State Improv Festival is in its open submission period through the end of September, and we hope to see many of you join us in Alaska next April!
Eric Caldwell is the Producer of the Alaska State Improv Festival, entering its fifth year in 2017.
