- Sells as: the
dedup-index product (a product-scoped API key). This capability has no separate product-face contract — its public product route reuses the capability contract below; see the Edge route table's Product face row.
Near-duplicate detection capability — a durable, org-scoped, multi-worker-COHERENT 64-bit SimHash index. One check-and-store API + a per-org Hamming threshold, so products stop each re-rolling a per-worker in-memory dedup index (which split-brains across workers, letting duplicates through). The near-dup scan is serialized per org (advisory lock) against ONE shared store so decisions are coherent across every worker/replica. I4: own Postgres store, never a product Mongo.
- Group: Content & media
- Contract:
contracts/dedup-index/v1/openapi.yaml
- Console: Manage the
dedup-index product in the console →
- Runbook: operational checklist for
dedup-index — internal (Vagary Labs ops; not part of this public site): docs/runbooks/capability-operations.md#dedup-index
- Public base:
https://api.vagarylabs.com (the consolidated API gateway — one host, per-brand sibling api.<zone>)
- Auth: a product API key (
vgk_…) issued from the console — Authorization: Bearer vgk_…
- Product face (customer-keyed):
https://api.vagarylabs.com/product/v1/dedup/check
Edge route table
The routes this capability actually serves on the public edge — live-synced from GET https://api.vagarylabs.com/product/v1/_meta/catalog (snapshot v1). The metric is the request-granularity billing counter; the scope is the key permission required.
Product face (customer-keyed)
| Method | Path | Metric | Scope |
|---|
POST | /product/v1/dedup/check | dedup_checks | read |
Endpoints
| Method | Path | Summary |
|---|
POST | /v1/dedup/check | Compute a SimHash, find near-duplicates in the org's index, and (optionally) store it |
GET | /v1/dedup/stats | Fingerprint count + effective Hamming threshold for the caller's org |
PUT | /v1/dedup/threshold | Set the org's Hamming threshold (persisted to the shared per-org row — worker-coherent) |
GET | /v1/dedup/fingerprints/{article_id} | The stored SimHash fingerprint for one article (or 404) |
GET | /health | liveness |
GET | /metrics | Prometheus |
Schemas
CheckRequest
| Field | Type | Description |
|---|
organization_id | string | product-tier tenant (I6) |
article_id | string | identifier (platform canonical_url) |
text | string | the content to fingerprint |
threshold | integer | per-request Hamming override; omit to use the org threshold |
store | boolean | persist the fingerprint (false = check-only) |
CheckResult
| Field | Type | Description |
|---|
fingerprint | string | unsigned 64-bit SimHash as a decimal string (exceeds JS safe-int / signed BSON) |
is_duplicate | boolean | |
near_duplicates | array | |
Stats
| Field | Type | Description |
|---|
fingerprint_count | integer | |
hamming_threshold | integer | |
ThresholdRequest
| Field | Type | Description |
|---|
organization_id | string | product-tier tenant (I6) |
threshold | integer | |
ThresholdResult
| Field | Type | Description |
|---|
hamming_threshold | integer | |
Fingerprint
| Field | Type | Description |
|---|
article_id | string | |
fingerprint | string | |
Error
| Field | Type | Description |
|---|
error | string | |
Generated by scripts/gen-capability-docs.py from contracts/dedup-index/v1/openapi.yaml — the contract IS the source of truth; edit the contract, not this page. The Edge route table section is synced from the live edge catalog snapshot (v1) — re-sync with python3 scripts/sync-edge-catalog.py.