Two hundred and ten applications closed on Friday. Four people agreed to read them. Three weeks later there is a shortlist of twelve, and nobody can explain how a particular application at number thirteen differs from the one at number eleven. One reviewer read forty applications closely and gave nothing above a three. Another read ninety and gave out sevens. Two applications were read by nobody, which was discovered by accident. Half the applicants have not been told anything, and one has written twice to ask.
Almost every review process fails in the same place, and it is not the reading. It fails at the seam between reviewers, in the order the checks were done, and in the record left behind. Those three things can be fixed before a single application is opened, and they cost a meeting rather than a budget.
Set the order of the gates before anyone reads
A review is a sequence of gates, not one act of judgement. Getting the sequence wrong wastes most of the effort.
Eligibility comes first, and it is a different kind of question from merit. Is the application complete. Does the applicant meet the stated requirements. Is it inside the deadline, in the right category, for the right round. Every one of those is a yes or a no that a single person can answer, and in many processes a form can answer some of them at the point of submission. Scoring an application that turns out to be ineligible is work that produces nothing, and worse, it creates an argument later about whether a good but ineligible application should be let through.
Merit review comes second, and only on the applications that passed. This is where the reading time goes, and it should be the only stage that consumes a panel.
A third gate sits after the shortlist: verification. References, licences, documents, and where relevant a background check. Putting verification before the shortlist means collecting sensitive material from people who will not be selected, which is both wasted effort and a records liability nobody needs.
Write the eligibility criteria as a checklist with a yes or a no against each item, and publish it with the application form. Most disputes about fairness are really disputes about a criterion that existed in somebody's head. One consequence worth stating in advance: what happens to an application that is ineligible for a fixable reason, such as a missing attachment. Decide once whether that gets a chance to be corrected and by when, because deciding it case by case is how a process becomes arguable.
Who reads what, and how many times
The choice of reading model is a trade between reviewer hours and the errors each model catches.
| Model | Reviewer hours | What it catches | What it misses |
|---|---|---|---|
| One reader per application | Lowest | Obvious unsuitability | Anything a tired reader skims past |
| One reader, second read on borderline cases only | Low | Most errors near the cut line | Errors at the top and bottom |
| Two independent readers, all applications | Double | Individual severity and leniency | Shared assumptions across the panel |
| Two readers plus a third on disagreement | Double plus | The same, with a defined tie break | Nothing much, at a cost |
| Panel discussion of every application | Highest | Consistency of interpretation | Speed, and quiet dissent |
For most volumes the fourth row is the best value, and the first row is defensible only where the total is small or the decision is low stakes. The pattern to avoid is the one that happens by default: everybody reads everything, informally, in no fixed order, and the decision is made in a meeting by whoever remembers the most.
Two mechanics matter more than the model. Assign applications rather than leaving a shared pile, because a pile produces both double reading and applications nobody opened. And record who read what, since the question after any contested decision is not what the score was but who saw it.
Blind review is worth the effort where the criteria allow it. Removing names and contact details before reading changes the reading, and it is easier to do at the point of assignment than to retrofit onto a spreadsheet of complete applications.
Scoring that survives two reviewers
A score is useful only if two people using it on the same application land close to each other. Three rules get most of the way.
Three or four criteria, not ten. Ten criteria produce a number with a false precision, and reviewers quietly collapse them into an overall impression anyway.
An anchored scale rather than a bare one to five. Write a sentence describing what a two looks like and what a four looks like, in the language of this particular round. Without anchors, a five means different things to a generous reader and a severe one, and the difference is invisible in the total.
A short written note for every score, required rather than optional. The note is what makes a decision explicable weeks later, and it is the only thing that distinguishes a considered three from a default three.
Then there is the question of what to do with disagreement, which is where most processes lose their integrity. Averaging two very different scores produces a middling number that represents nobody's view and buries the most informative signal in the whole process. A gap of two or more points on the same application means the reviewers are reading different things, and the right response is a short conversation, not arithmetic. Resolve the difference first and rank afterwards.
Calibration takes half an hour and is skipped almost universally. Pick three applications, have every reviewer score them independently, then compare and discuss before the real reading starts. That half hour removes more inconsistency than any wording change to the form.
What the review has to leave behind
The record is the part nobody plans and everybody needs, and two separate obligations are involved.
The first is retention. The EEOC states that its regulations require employers to keep all personnel or employment records for one year, and that where an employee is involuntarily terminated, their personnel records must be retained for one year from the date of termination. It also states that once a charge has been filed, records relating to the issues under investigation must be kept until final disposition of the charge or any lawsuit based on it. Deleting rows to tidy a tracker is the wrong instinct. Archive instead.
The second is measurement. The Uniform Guidelines on Employee Selection Procedures set out how selection rates get judged. Under 29 CFR 1607.4D, a selection rate for any race, sex or ethnic group which is less than four fifths, or eighty percent, of the rate for the group with the highest rate will generally be regarded by the federal enforcement agencies as evidence of adverse impact, while a greater than four fifths rate generally will not be. The same paragraph adds the caveats that matter in practice: smaller differences may nevertheless constitute adverse impact where they are significant in both statistical and practical terms, and greater differences may not where they are based on small numbers and are not statistically significant. Section 1607.4C adds that where the total selection process shows adverse impact, the individual components of that process should then be evaluated.
Whether those guidelines bind a particular organisation is a question for its own advisers. The operational point is independent of the answer: selection rates cannot be calculated at all unless the process records how many applications entered each stage and how many left it. A tracker that holds only the shortlist cannot answer the question, and reconstructing it afterwards from an inbox is not realistic.
Where a decision is informed by a background check obtained from a screening company, a further sequence applies. The FTC states that before taking an adverse action based on information in a consumer report, the applicant must be given a copy of the report and a copy of A Summary of Your Rights Under the Fair Credit Reporting Act, and that after the adverse action a notice is required naming the consumer reporting company, stating that it did not make the decision and cannot give reasons for it, and setting out the right to dispute the information and to get an additional free report within 60 days. That is a sequence with steps in a fixed order, which means it belongs in the process rather than in somebody's memory.
What "under review" should mean to the applicant
A large share of the people searching for this phrase are applicants trying to work out what a status on a screen means. That is worth knowing, because it is a direct measure of how little most statuses communicate.
An internal status is for the team. A status shown to an applicant is a promise, and it should carry a date. Under review with no date tells the applicant nothing except that the system is still switched on. Under review, with decisions expected by the fourteenth, ends most of the follow up correspondence on its own.
Three messages cover the whole cycle. An acknowledgement on submission that repeats back what was received and states when decisions go out. An update if that date moves, which costs one message and buys more goodwill than anything else in the process. And the decision itself, sent to everyone on the same day rather than to the successful applicants first.
Do not let the internal status leak. Stages such as awaiting second reader or in dispute are useful inside the team and alarming outside it. Keeping an internal set and a published set is simple, and it is the difference between a process that looks considered and one that looks chaotic.
Where a spreadsheet and an inbox stop holding
| Situation | What holds | What breaks first |
|---|---|---|
| One reader, thirty applications | A sheet and an email folder | Nothing |
| Two readers, independent scoring | A sheet with one row per application | Scores overwritten, and no record of who read what |
| Reviewers who should see only their own assignments | Not a shared sheet | Confidentiality, immediately |
| Applicants who need a decision on a stated date | Not a sheet on its own | The replies, since they live in sent folders |
| A process that has to report selection rates by stage | Not a shortlist tab | The count, because the earlier stages were never recorded |
| An outside panel member reading twenty applications | Not a link to the whole sheet | Everything, at once |
The third and sixth rows are the ones that cause real damage. Sending an external reviewer a link to the spreadsheet hands them every applicant's contact details, every other reviewer's comments, and the outcomes for people they have no business reading about. Read only access to a defined subset is the feature that matters, and it is the one a grid does not have. Where each application needs its own owner, its own status and its own correspondence, the list belongs somewhere built for that, and a worked example settles the comparison faster than a feature list.
What to change first
Write the eligibility checklist and publish it with the form, then assign applications to named reviewers instead of leaving a shared pile. Spend half an hour calibrating three applications before the real reading starts, and put a date next to every status an applicant can see. If the scores, the notes and the replies should sit on the same screen as the application, that is what a tool like Halict is for.
Q1. What are the stages of an application review process?
Three gates in a fixed order. Eligibility, which is a yes or no checklist that one person can run. Merit review, which is the only stage that needs a panel and runs only on eligible applications. Verification of references, documents and any checks, which comes after the shortlist so that sensitive material is collected only from people still under consideration.
Q2. How many reviewers should read each application?
Two independent readers for anything consequential, with a third brought in only where the two disagree by a wide margin. One reader is defensible at low volume or low stakes. The pattern to avoid is everybody reading everything informally and deciding in a meeting, which produces both double reading and applications nobody opened.
Q3. What should be done when two reviewers disagree?
Talk, then rank. A gap of two or more points on the same application usually means the reviewers are reading different things, and averaging the scores hides the most useful signal in the process. Resolving the difference before ranking also produces a written reason, which is what makes the decision explicable later.
Q4. What records does an application review need to keep?
Enough to reconstruct who read what and how many applications entered and left each stage. The EEOC states that its regulations require all personnel or employment records to be kept for one year, with a further year from the date of termination where an employee is involuntarily terminated, and that records relating to a filed charge must be kept until final disposition. Archive rather than delete.
Q5. What does the four fifths rule mean?
Under 29 CFR 1607.4D, a selection rate for any race, sex or ethnic group that is less than four fifths, or eighty percent, of the rate for the group with the highest rate will generally be regarded by federal enforcement agencies as evidence of adverse impact. The same paragraph notes that smaller differences can still constitute adverse impact where statistically and practically significant, and that larger differences may not where the numbers are too small to be reliable.
Q6. What should an applicant be told while their application is under review?
The date a decision is expected, and an update if that date moves. A status with no date attached communicates nothing and generates follow up messages. Keep internal stages such as awaiting second reader out of what the applicant sees, and send decisions to everyone on the same day.