Building a job recommendation engine

Recommending films is forgiving: the catalogue is stable, popular items stay popular, and a bad recommendation costs a user thirty seconds. None of that holds for jobs.

The inventory expires in weeks. The "item" a candidate wants is one they will get, which depends on the employer as much as on them. And success is not a click — it is an offer, months later, which you may never observe.

There is also a second party who has to agree. Recommending a role the candidate loves and will not be shortlisted for is a failure of a kind that has no equivalent in a media recommender, and it is invisible to any metric measured on the candidate side alone.

Separate candidate generation from ranking. Generation cuts millions of postings to a few hundred cheaply; ranking orders those few hundred carefully. Trying to score everything with your best model is the mistake that makes people conclude recommendation is too expensive.

Generation can be several retrievers running together — embedding similarity to the profile, recent searches, roles similar to past applications, postings from employers who responded before. Union them and let ranking sort it out.

Tune generation for recall and ranking for precision, and measure them separately. A role that never reached the shortlist cannot be recovered by a better ranker, so a recall failure and a ranking failure need different fixes and look identical in a final metric.

• Generate: cheap retrievers, high recall, a few hundred results
• Filter: hard constraints, applied as a gate rather than a score
• Rank: the expensive model, on the shortlist only
• Diversify: enforce spread before the list is shown

Right to work, location feasibility and a closed posting are not signals to weigh. Feeding them into a scoring function means a sufficiently strong match can overwhelm them, and the candidate is shown a role they cannot take.

Keep them as a filter that runs before ranking. It is simpler, it is faster, and it cannot be traded away by a confident model.

Freshness deserves the same treatment for closed postings specifically. A candidate sent to two dead links stops trusting the feed entirely, so re-checking the handful you are about to show is cheap insurance where re-checking the whole index is not.

In most recommenders cold start is an edge case. In a job marketplace it is the default: every posting is new, most expire before accumulating any interaction, and every candidate arrives with no history at the moment they most need good results.

That is the argument for content-based retrieval as the backbone rather than behavioural signals. A posting published an hour ago can be recommended on what it says, which is the only mechanism that works for the majority of the inventory.

The first session is also where preference data is cheapest to collect. Three dismissals with a one-tap reason, gathered while someone is looking at concrete roles, are worth more than a week of passive behaviour afterwards.

You only observe outcomes for what you showed. If the system rarely surfaces a category, you get no positive signal from it, so the model learns it is bad, so it surfaces it less. The loop closes quietly and the metrics look fine throughout.

The mitigations are known: reserve a fraction of slots for exploration, log what was shown as well as what was clicked, and evaluate on held-out data where you can reason about what was not shown. None is optional at scale.

Log position alongside the impression. A role at the top of a feed and the same role at the bottom get very different attention, so a click rate that ignores where something appeared is measuring the layout as much as the preference.

Keep a small unpersonalised holdout permanently. It is the only way to know whether six months of personalisation work improved anything, and it costs a fraction of the effort spent arguing about the alternative.

Optimising click-through gets you eye-catching titles and inflated salary bands. Optimising applications gets you low-effort postings. Both improve the number while making the product worse.

The honest target is a successful application — a response, an interview, an offer. That signal is sparse and slow, so use it to validate rather than to train, and treat the fast proxies as suspects that need checking against it.

Employer behaviour belongs in the ranking for the same reason. A role at a company that never responds produces applications and no outcomes, which flatters every fast metric while wasting the candidate’s effort entirely.

A job recommendation with no stated reason reads as spam, and candidates are already suspicious of automated feeds. "Because you worked with these tools at your last role" changes how the same result is received.

Explanation also disciplines the system. If you cannot articulate why something was recommended, that is usually a sign the pipeline matched on something you would not defend.

It only works if the intermediate result was kept. Matching claim to requirement leaves something to point at; comparing two document-level vectors leaves a number and nothing to say about it, which is a design decision made much earlier than the explanation feature.