Calculating a survey response rate is one division. Answers on top, people asked underneath, multiply by one hundred. The reason it still causes arguments in review meetings is that both halves of that fraction are judgement calls, and a single send can produce five numbers that are all correct and all different.
This walks through the arithmetic on a real-shaped send, shows where the figures diverge, and settles which one to put in the report.
The formula, and the decision that actually sets the answer
The base calculation is fixed:
Response rate = (responses / people asked) x 100
Everything difficult sits inside "people asked". The candidates, in descending order of size:
- Invitations sent. The number of records in the list at the moment of sending.
- Invitations delivered. Sent minus hard bounces and rejected addresses.
- Invitations opened or forms viewed. The number of people who saw the request at all.
- Known eligible people. Delivered minus the people the survey itself disqualified.
The numerator has a smaller but real spread. A completed submission clearly counts. A submission abandoned halfway may or may not. A person who submitted twice should count once. A person screened out at question one is not a response and arguably should not be in the denominator either.
None of these choices is wrong. The only wrong move is switching between them, because a response rate is meaningless except as a comparison with a previous one calculated identically. The Standard Definitions report published by the American Association for Public Opinion Research exists for this reason: it assigns a final disposition code to every case in a sample and defines outcome rates built from those codes, for telephone, in-person, mail to named persons, and web surveys. Most teams do not need that level of formality. Every team needs its own version of it, written down.
A worked example
Take a customer satisfaction survey sent to a list once, with no reminder. These are the raw counts a send like that produces.
| Count | Number |
|---|---|
| Records in the list, invitations sent | 2,400 |
| Hard bounces and rejections | 180 |
| Delivered | 2,220 |
| Invitations opened | 1,150 |
| Started the survey | 310 |
| Screened out as ineligible at the first question | 22 |
| Abandoned part way | 42 |
| Completed | 246 |
| Of those completes, duplicate submissions by the same person | 6 |
Now the same survey, calculated five ways.
Completes over sent. 246 / 2,400 = 10.3 percent. The most conservative figure. It treats an undeliverable address as a person who declined, which is not true, but it keeps the denominator honest about list quality.
Completes over delivered. 246 / 2,220 = 11.1 percent. The common default for email surveys, and the one most reporting tools mean by response rate. It separates a bad list from a bad survey.
Completes and partials over delivered. (246 + 42) / 2,220 = 13.0 percent. Defensible when partial answers are analysed, which is normal for surveys where the important question is asked early.
All starts over delivered. 310 / 2,220 = 14.0 percent. This is participation, not response. It flatters the survey and should be labelled clearly if used at all.
Completes over opened. 246 / 1,150 = 21.4 percent. Useful as a diagnostic, misleading as a headline. It measures the form's persuasiveness among people who already engaged with the invitation, and open tracking is unreliable enough that this denominator moves for reasons unconnected to the survey.
Deduplicated, the completes become 240, and completes over delivered falls to 10.8 percent.
The same send therefore supports figures from 10.3 to 21.4 percent. That is a factor of two, with no dishonesty anywhere. This is exactly why comparing a rate against an industry average found online tells you almost nothing about the survey.
Which figure to report
For a repeated business survey, report unique completes over delivered, and report the others as supporting detail. That combination answers the question people actually have.
| Figure | What it diagnoses | When it belongs in the report |
|---|---|---|
| Completes over sent | List decay and address quality | When the list has not been cleaned in a while |
| Completes over delivered | The invitation and the form together | Always, as the headline |
| Starts over delivered | Whether the invitation worked | When the headline rate fell and the cause is unclear |
| Completes over starts | Whether the questionnaire is too long or too intrusive | When abandonment is the suspected problem |
| Completes over opened | The gap between interest and action | As a diagnostic only, never as the headline |
The second and fourth rows together localise almost every problem. If starts over delivered is healthy and completes over starts is poor, the questionnaire is the fault. If starts over delivered is poor, nobody got far enough for the questionnaire to matter, and the work is in the invitation, the sender, the timing or the list.
Reminders and waves move the denominator
A second send complicates the arithmetic, and the mistake is almost always the same: adding the responses from both waves and dividing by the size of the second send.
The rule is that the denominator belongs to the population, not to the mailing. If 2,220 people were reachable and a reminder went to the 1,974 who had not yet answered, the cumulative response rate is still all unique completes over 2,220. The reminder did not enlarge or shrink the population being studied.
The per wave figure is calculated separately and against its own base. A reminder sent to 1,974 non-respondents that brings in 95 completes has a wave response rate of 4.8 percent, and the cumulative rate rises from 11.1 percent to 15.4 percent. Both numbers are worth keeping. The wave figure tells you whether reminders are still earning their place, because the yield on a third and fourth reminder usually collapses and the annoyance does not. The cumulative figure is what goes in the report.
Two related traps sit nearby. Adding people to the list between waves changes the population mid-study, so late additions should either be excluded or tracked as a separate cohort with its own denominator. And a survey left open indefinitely has no closing date, which means its rate keeps creeping up and can never be compared with a survey that closed after ten days. Fix a close date, state it, and calculate at that point.
Where the count goes wrong
Arithmetic is rarely the failure. Counting is.
Test submissions left in the data. Every survey collects a handful of internal test entries during setup. They inflate the numerator and distort open text analysis. Delete them before counting, and note how many were removed.
The same person answering twice. Common when a reminder goes out, when a link is shared, and when a form has no login. Deduplicating on email address is the practical fix, which is why it matters whether the tool holding the responses treats the same address as one contact across submissions rather than as two unrelated rows.
Staff and internal addresses in a customer list. A list assembled from a CRM export frequently contains colleagues, partners and the support alias. They belong in neither half of the fraction.
Screened-out respondents left in the denominator. If a survey about a specific product disqualifies 22 people who never bought it, those 22 were not eligible. Leaving them in the denominator understates the rate against the population that was actually being studied. Removing them requires saying so.
Soft bounces counted as delivered. A message deferred and never delivered is not delivered. Reconcile the delivery report before taking the figure, not a week later when the log has rotated.
Multiple channels, one person. A survey pushed by email, a website banner and a QR code on a receipt has three denominators and no combined one. Calculate each channel separately. A blended figure across channels cannot be interpreted and cannot be improved.
Calculating it when there is no mailing list
Plenty of surveys have no clean denominator at all, and the honest move is to change what is measured rather than invent one.
Website or in-product intercepts. The denominator is the number of sessions in which the prompt was actually shown, which the tool should report. If it cannot, the rate is not calculable and the count of responses per thousand sessions is the substitute.
QR codes on printed material. Nobody knows how many people saw the code. Use scans as the denominator if scan counts exist, and otherwise track responses per event or per hundred items distributed and compare like with like over time.
Paper forms handed out. Count what was handed out. That number is knowable if someone writes it down at the time, and unknowable an hour later.
Public links posted anywhere. There is no denominator. Report the response count and the traffic to the form page, and resist the urge to divide one by the other.
For these cases, a stable count per unit of exposure does the job a response rate would do, which is to show whether this run went better or worse than the last one.
Recording it so the next run is comparable
The calculation takes a minute. Making it comparable takes a habit.
- Snapshot the denominator when the survey is sent. List size and delivery counts both drift. A figure reconstructed later will not match.
- Write the definition next to the result. Which denominator, whether partials count, whether screened-out cases were removed, how duplicates were handled.
- Log the conditions. Channel, send day and time, subject line, question count, reminder or no reminder, incentive or none. A rate without conditions cannot be explained when it changes.
- Split by segment before looking at the total. An overall 11.1 percent that hides 28 percent among customers active in the last quarter and 3 percent among the rest is two separate findings.
- Keep the raw counts, not just the percentage. Every recalculation later depends on them, and percentages cannot be reverse engineered.
A tool that keeps each response as a record with its own owner, stage and history rather than as a row in an export makes step five automatic, and makes the duplicate problem visible instead of silent. The features that matter for this are contact matching on email address and an exportable log of what was sent and when, so the denominator and the numerator come from the same place. Pricing on unlimited responses also removes the quiet incentive to send to a smaller list than the survey needs.
What to change first
Pick one denominator, write it down, and recalculate the last two surveys the same way so there is a real comparison to work from. Then split the rate into starts over delivered and completes over starts, because that pair tells you whether the invitation or the questionnaire is costing the responses. Keeping the raw counts and the reply history in one place, as Halict does per response, is what makes the second survey comparable to the first.
Q1. Should bounced emails be removed from the denominator?
Yes for hard bounces, which are addresses that do not exist. Those people were never asked. Soft bounces need checking: if the message was eventually delivered it stays in the denominator, and if it was deferred and dropped it comes out. Whichever rule is chosen, apply it identically to every future send.
Q2. Do partial responses count in the response rate?
Either way is acceptable if it is consistent. The cleaner practice is to base the response rate on completes and report the partial count separately, since partials that stopped before the key question cannot be analysed anyway. Mixing the two definitions between runs is the only real error.
Q3. How is response rate different from completion rate?
Response rate divides responses by the people who were asked, so it measures the invitation and the form together. Completion rate divides finishers by starters, so it measures only the questionnaire. A survey can have a poor response rate and an excellent completion rate, which points the work at the invitation rather than the questions.
Q4. What if the same person responds more than once?
Count them once, and keep the duplicate record rather than deleting it so the count can be audited. Deduplicating on email address catches most cases. For anonymous surveys with no identifier, duplicates cannot be detected reliably, which is worth stating when the results are presented.
Q5. Is there a minimum response rate before results can be used?
There is no threshold that applies across survey types, and the absolute number of answers matters more than the percentage. Two hundred answers will show a clear pattern whatever the rate; eight answers will not, even at 70 percent. The more important check is whether the people who answered resemble the people who did not.