Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
docs=152 chars=1,476,544 chunks=2808
model loaded in 1.3s
embedded 2808 chunks in 156.9s

BM25 returned zero candidates for 0 of 60 queries (answerable: 0, unanswerable: 0)
BM25 (stopwords removed) zero candidates: answerable 0, unanswerable 5

Top-1 score distributions
cosine top-1                 answerable: median 0.619 [min 0.393, max 0.818]  near: median 0.385 [min 0.256, max 0.508]  far: median 0.218 [min 0.175, max 0.298]
BM25 top-1 (raw)             answerable: median 20.583 [min 14.025, max 34.632]  near: median 16.340 [min 11.336, max 20.492]  far: median 10.804 [min 8.019, max 14.926]
BM25 top-1 (no stopwords)    answerable: median 17.410 [min 10.924, max 29.798]  near: median 10.741 [min 6.063, max 16.934]  far: median 6.029 [min 0.000, max 12.935]

AUC (answerable vs unanswerable):
  cosine top-1           0.986
  BM25 top-1 raw         0.910
  BM25 no-stopword       0.927
  cosine top1-top2 gap   0.649

Cosine threshold sweep (refuse if top-1 cosine < thr)

    thr | answered-kept | near refused | far refused | all unanswerable refused
   0.00 |         1.000 |        0.000 |       0.000 | 0.000
   0.05 |         1.000 |        0.000 |       0.000 | 0.000
   0.10 |         1.000 |        0.000 |       0.000 | 0.000
   0.15 |         1.000 |        0.000 |       0.000 | 0.000
   0.20 |         1.000 |        0.000 |       0.200 | 0.100
   0.25 |         1.000 |        0.000 |       0.800 | 0.400
   0.30 |         1.000 |        0.133 |       1.000 | 0.567
   0.35 |         1.000 |        0.200 |       1.000 | 0.600
   0.40 |         0.967 |        0.600 |       1.000 | 0.800
   0.45 |         0.933 |        0.733 |       1.000 | 0.867
   0.50 |         0.900 |        0.933 |       1.000 | 0.967
   0.55 |         0.733 |        1.000 |       1.000 | 1.000
   0.60 |         0.567 |        1.000 |       1.000 | 1.000

Calibrate on half, test on the other half (cosine top-1)
  require >= 100% answerable kept on calibration -> thr 0.393 (cal: kept 1.00, refused 0.81); TEST: answerable kept 1.000 (15/15), unanswerable refused 0.786 (11/14)
  require >= 90% answerable kept on calibration -> thr 0.405 (cal: kept 0.93, refused 0.88); TEST: answerable kept 1.000 (15/15), unanswerable refused 0.786 (11/14)

Per-query cosine top-1 (sorted), for the unanswerable queries with highest scores:
  0.508 [near] How do I expose Prometheus metrics from my service?  -> advanced_opentelemetry.md

Lowest-scoring answerable queries:
  0.393 How can I require a logged-in user before an endpoint runs?
  0.405 How do I accept a persistent two-way connection from a browser?
  0.500 How do I connect my API to a relational database?
  0.508 How can I add logic that runs for every request, like a timing header?
  0.511 How can I make a numeric URL parameter require a minimum or maximum value?