How I rate talks for conferences and events

Joe • June 13, 2017

community conferences speaking

"If you've given a talk that's been recorded more than once. it's probably time to stop submitting that to conferences." -- an offhand comment I made on twitter. I made this comment after going through the first round of rating talk submissions for a conference I have the privilege of helping with talk selection. I feel like there was a lot of back and forth and it's obvious people have strong opinions on this, so I thought I should elaborate.

I'm No One Special

I've been really lucky to have opportunities and day jobs that were fully supportive of letting me go to conferences and speak. Very few people have this ability, and it's a shame because I know there are a ton of amazing speakers out there. I've been lucky enough to have some really awesome people mentor me on getting talks accepted, crafting abstracts, and so much more.

What I'm describing in this process is absolutely nothing special. This process is a combination of what conferences state the review process will be and what I've learned from other organizers. Nothing I tell you here will magically get you accepted to a conference because I'm telling you some inside scoop on what to do. Take what you want from this post. I lay all of this out here so maybe it'll help inspire someone to submit, spice up an abstract, or never accept a talk I submit again.

How I Rate Talks

I've run a couple of small conferences in Memphis, and I've helped organize larger regional language specific conferences. I've spoken at a ton of conferences big and small. I'm lucky enough that other conference organizers value my input and they invite me to join in on their selection process. It's an honor. It's a TON of reading and work, but I love every minute of it.

Blind Voting (At Least First Pass)

I prefer to do the first pass without knowing who the speaker is. This way I'm judging the talk completely based on the title, abstract, notes to the organizers, and maybe some keywords that were submitted by the speaker. If the speaker doesn't include anything with URLs that contain recognizable usernames I often can't tell you who the speaker is. I always wonder if a Docker talk is from my good friend Chris Tankersley and inevitably I always see DragonBe's name in a URL somewhere, so I can sometimes tell. I try not to let this affect my judgment. One of the worst things you can do as a talk-rater is upvote your friends, just because you can. And blind readings keep us from being unconsciously biased as well.

Rating Scale

There's a scale, and it's not the same for every event, but speakers all get rated. Sometimes I'm a 9. Sometimes I'm a 3. As a speaker, please find some comfort in knowing you will never find out what your talks are rated; it can be hard to swallow. When I'm looking over the first blind pass I rate based on title, topic, and description. Would *I* attend this talk? Is it technical enough? Is it applicable to the guidelines given to me by the conference organizers? Does it describe a problem and entice me to attend to find a solution? Talks that do these things are instantly intriguing to me, so I often will rate these higher. If a submission has a buzzword style title and a 1-2 sentence description that just repeats buzzwords, that's an instantly low score. If you're going to use the Dr. Strangelove line in your talk title your description damn well better live up to the hype. (I'm guilty of doing this and not living up to the hype, probably why you've never seen me give this talk.) Stay away from cliches unless it ties your talk together better than The Dude's rug ties his room together.

Submissions I rate highly need to describe a problem and a offer a potential solution. Even if I Don't care about the subject, I'll rate it higher because it's a well thought out and crafted abstract.

Lower rated submissions tend to be half-cocked ideas and obviously not well thought out. If a talk comes across to salesy or pushy, it's an instant low score and flagged for later review (maybe it was just the writing style/approach).

You've Never Been Declined by Me

I *rate* talks; I'm not the one that picks the schedule. I've never been the only reason you've been declined (as far as I know). After talks are rated the even harder job goes to the organizers to come up with a cohesive schedule that makes sense for the events. Even if a talk has a 10/10 rating, it doesn't mean it will get selected. Maybe it doesn't fit the theme that year. Maybe it's too expensive to bring the speaker. This happens a lot.

Submit a Lot

My good friend Cal Evans has a post <a href="" target="_blank>When it comes to submitting talks how many is too many. There are a lot of good points there, read it and apply it to your process. Where I disagree with Cal is that I Don't believe there is a cap of too many submissions. If you have been speaking for a number of years, there may be a large number of things you want to talk about. Keep in mind if you are submitting 20+ talks that you may be shotgunning the organizers a good bit. I normally submit 5-10 talks, tutorials, and trainings to conferences. I won't submit the same talk that I gave the previous year (as much as I can help it, sometimes I goof on this). I Don't expect a conference that brought me to do a specific talk will want that talk again, so I Don't waste their time (or my time submitting it).

Submit Videos

Now we're getting to the original intent of this post. Submitting videos with your talk abstracts is something you should absolutely do. This gives the organizers real-world, first-hand experience of your abilities and is great for organizers that Don't know you. This is exactly why we record NomadPHP lightning talks and push aspiring speakers to submit and include these videos in their abstracts.

The original tweet that sparked this post was in reference to a submission that I saw that had the same talk that had been recorded three times and was publicly available on YouTube. This is hard for me to rate high because why would someone pay to come to a conference to see a talk they could easily watch online? Obviously, not everyone shares this opinion. My point with the statement is that as a talk reviewer my number one priority is to provide a value for the person buying a ticket and attending the conference. Ticket buyers make conferences possible, and organizers owe it to them to provide the best content possible. Just like you hear me talk about software decisions, "There are no solutions, there are only tradeoffs.” The same goes for talk selection.


Submit to conferences, include videos. Don't be scared of events recording your talks. Get help with your abstracts from the amazing people at