Indexing & Ranking (L2)

L2 uses a hybrid approach combining sparse filters, lightweight vector retrieval, and precomputed features.

Indexing

• Sparse: inverted index over keywords, entities, and tags
• Filters: project_id, type, freshness, pinned
• Vector (optional): small embedding over title+summary from L1
• Precompute features into CurationIndex for fast scoring

Ranking features (examples)

• entity_overlap (0..1)
• recency_days (lower is better)
• source_reliability (0..1)
• query_term_match (BM25-lite)
• pin_boost (pinned items get a fixed uplift)

Scoring outline

score = w1*entity_overlap + w2*query_term_match + w3*(1 - min(1, recency_days/30)) + w4*source_reliability + pin_boost

Weights are configured per project via ENV/provider config. Default top_k limited by L2_TOP_K_MAX.

Hot set policies

• Pin: maintain a small curated set of golden items
• Compress: store compact summaries for low-signal tail
• Evict: remove stale items by recency and zero-usage

Budgets and degradation

• Time budget: L2_TIME_MS_BUDGET (default 8 ms)
• Degradation path:
  1. Reduce top_k
  2. Drop vector stage, keep sparse+features
  3. Return metadata-only items

Evaluation hooks (L8)

• Track Recall@K, MRR@n per query class
• Emit evaluation events with request_id, project_id, t_ms, and chosen features