Screening tools do not degrade gracefully. They work, and then at some volume they stop working, and the failure is rarely the thing the sales conversation covered. Above roughly 500 applications the constraints that matter are batch ceilings, file handling, queue behaviour and the manual work either side of the automation. Feature lists tell you nothing about any of them.
This is a list of what actually breaks, and the questions that reveal a ceiling before you hit it.
Batch Ceilings Are Real and Rarely Advertised
Every system has a maximum number of documents it will process in one job, a maximum number of files per upload request, and a maximum file size. These are engineering realities rather than marketing choices, and they are the first thing to break.
Ours are 500 CVs per screening batch, 50 files per upload request, 10MB per file, and one concurrent batch per user. That is comfortable for the overwhelming majority of UK campaigns and a genuine constraint if you routinely run single batches above 500. We publish it because a ceiling you know about is a planning input, and a ceiling you discover at nine in the morning during a campaign is an incident.
Ask every vendor for those four numbers. A vendor who will not state them either does not know or does not want you to.
What Breaks First: Document Extraction
At twenty applications a badly formatted CV is an annoyance somebody notices. At six hundred it is invisible, and invisible extraction failure is the most damaging failure mode available because the system produces a confident score from incomplete text.
The recurring offenders are scanned PDFs with no text layer, two-column layouts where reading order does not match visual order, CVs built in tables, files submitted in unexpected formats, and documents where the candidate's experience sits in a text box the parser skips.
The question that matters is not whether a tool handles these. It is whether it tells you when it could not. Ask what happens to a file that fails extraction: is it flagged, silently scored, or silently dropped. A tool that silently drops files will quietly lose candidates and you will never know which.
Consistency Across the Batch
The point of automating a sift is that application 400 is assessed the same way as application 4. That is precisely what manual sifting fails at, and it is not automatic in software either.
Ask whether the same CV scored twice produces the same result, and ask what controls calibration across a large batch. Then test it: submit the same CV three times inside a real batch under different filenames and compare. If the three scores differ materially, the ranking is not a ranking.
The Work Either Side of the Automation
This is where the time actually goes at volume, and it never appears in a demonstration.
- Getting files in. If applications arrive in an ATS and the tool needs uploads, somebody is exporting and uploading. At 50 files per request and 600 applications that is twelve upload cycles.
- Getting results out. If output needs reformatting before a hiring manager will read it, that is manual work on every campaign.
- Handling the exceptions. Failed parses, duplicates, late applications, candidates who emailed a CV instead of applying.
Measure all three during a trial with a real campaign. The elapsed time that matters is from applications arriving to shortlist in a hiring manager's hands, not the processing time the vendor quotes.
Duplicates and Re-Applications
At volume, the same person applies twice, sometimes with a different CV, sometimes to two roles. Ask how duplicates are detected and what happens to them.
Two failure modes follow. The same candidate appearing twice in one shortlist looks careless to a hiring manager. Worse, a candidate whose second application was stronger being scored on the first is a defensibility problem, because you rejected a person on evidence they did not submit.
Queue Behaviour Under Pressure
Volume is rarely evenly distributed. A closing date produces a spike, and the spike is exactly when the system is most needed.
Ask what happens when your batch is queued behind other customers, whether there is prioritisation, what the behaviour is on timeout, and whether a partially completed batch can be resumed or has to be re-run from the start. Re-running from the start on a 500 document batch is an afternoon.
The Compliance Load Also Scales
Volume multiplies obligations, and this is the part organisations discover late.
Every rejected applicant is a person whose data you hold and who can make a subject access request. At 600 applications per role across 60 roles that is 36,000 records a year requiring a lawful basis, a retention schedule and the ability to produce a per-candidate rationale on request. The ICO's guidance on automated decision-making sets expectations about providing meaningful information on the logic involved, and that is considerably harder retrospectively than by design.
Ask whether rationales are exportable per candidate, whether retention can be automated on a schedule you define, and whether deletion actually removes the record rather than hiding it.
The Ten Questions That Find the Ceiling
- Maximum documents per batch?
- Maximum files and total size per upload request?
- Maximum individual file size, and which formats?
- Concurrent batch limit per user and per account?
- What happens to a file that fails extraction: flagged, scored anyway, or dropped?
- Does the same CV score consistently within a batch, and how is that controlled?
- How are duplicate applicants detected and resolved?
- What happens on timeout, and can a partial batch resume?
- Can we export a per-candidate rationale for a subject access request?
- Can retention and deletion run automatically on our schedule?
Ask for the answers in writing. Anything a vendor will not put in writing is not a commitment.
Frequently Asked Questions
Is 500 per Batch a Problem?
Rarely. Most UK roles attract well under that, and campaigns above it can be split by cohort or closing date. It matters if you genuinely need a single ranked list of several thousand in one pass, which is uncommon outside large graduate schemes.
How Do We Handle a Campaign Above the Ceiling?
Split deliberately rather than arbitrarily, for example by closing date or location, and keep the criteria identical across batches so rankings remain comparable. Record how you split it.
Does More Volume Mean Worse Quality?
For manual sifting, yes, because attention degrades across a pile. For automated scoring the risk is different: consistency is usually maintained and extraction failures become invisible. Different failure mode, so test for the right one.
Should We Cap Applications Instead?
Capping is blunt and disadvantages anyone who applies later, which frequently correlates with people balancing work or caring responsibilities. Tightening the advert and stating mandatory requirements reduces volume without that effect.
What Is the Most Overlooked Constraint?
Manual handling either side of the automation. It is the largest time cost at volume and the one no vendor demonstration includes.
The Short Version
Get the four ceiling numbers in writing. Establish what happens to files that fail extraction. Test consistency by submitting the same CV three times inside a real batch. Measure elapsed time end to end, including your own work. And confirm you can produce a per-candidate rationale, because at volume that obligation arrives whether you planned for it or not.
This article states our own documented limits and asks you to obtain competitors' limits directly from them.
Sources
Information Commissioner's Office. Rights Related to Automated Decision Making Including Profiling.
https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/rights-related-to-automated-decision-making-including-profiling/
Information Commissioner's Office. Right of Access.
https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/right-of-access/
Information Commissioner's Office. Employment Practices: Recruitment and Selection.
https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/recruitment-and-selection/
The Stationery Office. Equality Act 2010.
https://www.legislation.gov.uk/ukpga/2010/15/contents
InsightCircle
