6 Million Listings Had No Host ID, and Part of the Reason Was a Filter That Skips Missing Fields

I keep a census of short-term-rental listings in one Elasticsearch index: 8,812,665 documents as of 2026-11-05. I wanted a second dataset built around the hosts, and the obvious source for host IDs was that same census. Only 2,750,769 of those listings, 31.2%, carry one. The other 6,061,896 don't.

My first plan treated that as a capacity problem. Host IDs come from one place, a worker that opens each listing's detail page, and it had only reached part of the census. So the plan said: restart that fleet, give it more pods, and wait. Before asking for the pods I ran a few read-only aggregations to see what the missing 6 million looked like. The gap was not one thing.

TL;DR

• 39.2% of the missing host IDs (2,375,360 listings) belong to listings with no review_count field at all. None of them has a host ID. The enrichment filter, as the data shows it was run, could not select them.
• 60.3% (3,658,218) have reviews and were simply never reached. Coverage there is flat across countries and listing ages, so nothing explains it except that the passes were bounded.
• 0.5% (28,318) were attempted and still have no host ID.
• A cheaper fix exists. Each host's profile page embeds up to ten of its listings, and a worker already fetches those profiles. I sized it from 34 hosts, so treat the result as an order of magnitude.

Where host IDs come from

The search results that built the census carry no host ID. The listing detail page does. A separate enrichment worker visits those pages and fills in host_id on about 99% of the listings it processes. That means host coverage is exactly as large as the set of listings that worker has opened.

I counted, with read-only queries on 2026-11-05 and 2026-11-06:

Splitting the gap

The aggregation that mattered grouped the missing listings by whether they had any reviews.

The first row is the surprising one, so I went to the enrichment code. The scan takes a minimum review count. When it is above zero, the query adds a range clause: review_count greater than or equal to that number. The comment on the function says the point is to focus enrichment on listings that matter. The default is zero, which turns the clause off.

A range filter in Elasticsearch only matches documents that have a value for the field. A listing with no review_count isn't a listing with zero reviews. For this query it's invisible. In the data, every one of the 11 enrichment days I could look at has a minimum review count of 1 and no listing without the field. So those runs were launched with the minimum set above zero, and the 2.4 million listings with no field were outside what any of them could have returned.

I can't show you who set it. There is no enrichment manifest in the cluster repo and no reference in the orchestrator code, so the run configuration isn't recorded anywhere I can read. That the minimum was set is an inference from what the runs produced, and I'd rather say so than invent a launch command.

The filter was doing its documented job. It was also a good way to never enrich a listing that has no reviews yet, which is the kind of listing whose host is likely newer. For a dataset about hosts, that is the wrong side of the line.

The larger share was just never reached

The 3.66 million reviewed listings are the less interesting row, but I checked them for a pattern, because "never reached" can hide a bias.

Of 6,437,305 listings with at least one review, 2,750,769 have a host ID, which is 42.7%. Coverage was flat between 27% and 37% in every country. It was 31.8% for listings last seen in the previous 30 days and 30.9% for listings last seen 90 to 180 days ago. The scan orders listings by a hash of the listing ID, so any partial pass is a random sample on purpose. These numbers say it worked: no country, review count or freshness skew explains what is missing.

One caveat I can't remove. "Never reached" here means no enrichment timestamp. A failed page fetch writes nothing, so a listing that failed looks the same as one that was never selected. The one documented run failed 25 of 30,000 fetches, 0.08%, so the effect is probably small. I don't have the number for the other runs.

I also don't know why enrichment happened when it did. By last-enriched date, 1.54 million listings were done on 2026-07-03, 0.18 million on 07-04, 0.42 million on 07-06 and 0.10 million on 07-07. After that there are only 40,000 to 180,000 on each of the first and second days of August, September and October, and nothing on 2026-11-01 or 11-02. The monthly pattern suggests a scheduled trigger. I couldn't find one, and the scheduler that would have run it has been stalled since October. That is a guess, and I'm leaving it as one.

A cheaper way to close it

Re-running enrichment with the minimum set to zero would reach the 2.4 million unreviewed listings. It needs a fleet, a bandwidth estimate and a decision about cost, and I hadn't measured how large a detail page is next to a profile page.

There was a smaller option sitting in the pipeline I'd just built. A host's profile page embeds up to ten of that host's listings, and the host-hydration worker already fetches those pages. The IDs arrive in a request I'm already making. Linking them back to census documents costs no extra fetch.

I sampled 34 hosts across sizes, 33 of which parsed. Of 190 embedded listing IDs, 88% exist in the census. Of those, 42% already carried the right host ID and 57% would gain one. One of 167 pointed at a different host already, probably a co-host. Scaled to the mix of hosts I'd already fetched, that is about 2.8 million of the 6.06 million, from embedded listings alone. It's a range from a small sample, not a measured result.

Hosts with more than ten listings are a harder case. 19,388 of 133,272 hydrated hosts at sizing time (14.5%) have more than ten, and together they hold about a million listings beyond the first ten. Reaching those means the paginated listings API at 12 per call, about 94,659 extra calls. One host manages 8,589 listings. I haven't built that part.

The design rules mattered more than the idea. The link writes a host ID only where the field is empty. It never overwrites a different one and counts the conflict. It never touches the field that marks a listing as individually enriched, so the listing stays eligible for the real thing. And it records where the ID came from.

The third rule exists because of a mistake I made. On the first rollout, 2 of the first 930 links landed on listings that had already been enriched from their detail page. A host ID on those would have added a host to a distinct count in an aggregate we publish. The fix was to refuse to link anything that carries that marker, and the script re-checks it at write time.

Where it stands

The linking code is built and was not released when I last wrote this down. Everything above is dated measurement, not a result of the fix. The number I'll watch is 6,061,896, and I'll re-run the same aggregations after the release before I say anything about what the link changed.

If you run a crawler that fills a field from a second pass, the check is cheap. Group the documents that lack the field by whether they have the field your filter depends on. In my case that grouping explained two fifths of the gap in one query. I'd have spent a week sizing pods for it.