docs/catalog.mdtype: Reference

Name your bundles once, search them all.

OKF4net.Catalog names one or more local OKF bundles as sources in a hot-reloadable catalog.json manifest, then searches every enabled one. It's an OKF4net-specific manifest, not an OKF concept — it configures the catalog from the outside and isn't part of the OKF spec itself.

##

Install

two packages, one for hosting
$ dotnet add package OKF4net.Catalog

The catalog core — manifest, resolver, memory store. References only OKF4net.

$ dotnet add package OKF4net.Catalog.Hosting

Add this too for AddKnowledge/AddMemory — an IServiceCollection host. The sole project in the repo taking a Microsoft.Extensions.* dependency.

##

The catalog.json manifest

strict, never-throw parser
FieldDefault / rule
idrequired — a valid single concept-id segment, unique within the manifest
pathrequired — resolved relative to the manifest directory, must stay inside the catalog root
priority0 — higher-priority sources sort first within the grouped results
enabledtrue — a disabled source's path is never resolved or checked against the filesystem
role"knowledge" — or "memory"; any other string is rejected (IllegalRole)
tierrequired only when role is "memory": one of session, user, or tenant
{
  "version": 1,
  "sources": [
    { "id": "products", "path": "./bundles/products", "role": "knowledge" },
    { "id": "mem-user", "path": "./memory/user", "role": "memory", "tier": "user" }
  ]
}

A malformed manifest never throws mid-parse — every problem (wrong version, empty sources, a duplicate or invalid id, an illegal role/tier combination, an embedded NUL, an absolute or out-of-root path, a reparse point in the path, …) comes back as a structured diagnostic instead.

##

FileKnowledgeCatalog

fail-fast load, errors-as-data reload

Construction is fail-fast: an invalid initial manifest throws (CatalogException) rather than publishing a partial or empty catalog — a caller like a DI container at startup never gets a silently broken state. Every reload after that first successful load is errors-as-data: ReloadAsync() re-reads and re-validates a whole new snapshot before ever touching the live one, and only swaps it in if every enabled source's path is still valid — one bad source rejects the entire reload, not just that source, and the previous good snapshot keeps serving. A monotonic Generation counter increments only on a successful swap, so a caller can tell whether a search reflects the latest reload. A debounced (250ms default) file watcher on the manifest file itself triggers reloads automatically, but it's best-effort — ReloadAsync() is the only path with a delivery guarantee.

##

The multi-source resolver

three selectable ranking strategies

Every strategy searches all enabled Knowledge-role sources (sources with role: "memory" are never searched here — they feed the memory store instead), and a StalePolicy on the query (Use by default — admit everything) is applied across the combined result set. What differs is the order results come back in, which matters most to a caller that stops reading early — an agent spending a token budget top-down, say. Pick one per host, or override it per query:

  • GroupedBySource (the default) — each source's own ranked results concatenated, source by source, in priority then id order. No cross-source fusion or deduplication.
  • Merged — one ranking by descending score across every source, with source priority as a tie-break only.
  • PriorityWeighted — source priority first, score ordering only within a priority tier, so a higher-priority source never falls behind a lower-priority one however strong the latter's match.

The two merged strategies also collapse two manifest entries that resolve to the same directory, searching that bundle once rather than twice. Two different directories that happen to share a concept id are never merged — a concept id is relative to its own bundle root and is not a globally stable identity. Both accept an optional fairness quota that caps how many consecutive passages one source may contribute; it reorders and never drops, so it changes what a budget-truncated caller sees without changing what a caller reading the whole list gets.

##

Source visibility

restrict which sources a caller may search

KnowledgeQuery gains a Scope (KnowledgeAccessScope, default Local) and two mutually-exclusive ways to restrict which enabled Knowledge-role sources a given caller may see:

  • PermittedSourceIds — a host-precomputed set of source ids. The recommended default; there's no host-level default for it, since a static set can't represent "differs by tenant".
  • SourceVisibilityPolicy — a per-source function, with an optional KnowledgeOptions.DefaultSourceVisibilityPolicy host default; the function can still vary per call by reading the scope it's given.

Setting both on the same query is rejected. A query-level PermittedSourceIds always wins over a configured default when set. OkfContextProvider's scoped (V2) mode passes the same KnowledgeAccessScope it already resolves for memory into the knowledge query too, so visibility and memory scoping stay consistent for one caller.

##

Trust & staleness

§5 — TrustTier, Lifecycle, StalePolicy

Every KnowledgePassage carries the matching concept's TrustTier (default Unverified) and full Lifecycle (Status, StaleAfter), read straight off its frontmatter — a host can filter or render provenance without re-parsing anything. Staleness is a method, not a stored flag: Lifecycle.IsStale(today). StalePolicy has three modes: Use (admit everything, the default — never a silent drop), Strict (exclude anything stale), and Tolerate(graceDays) (admit until stale_after + graceDays).

##

Scoped memory (role: memory)

session → user → tenant

A role: "memory" source is written by capture (an agent's context provider), never searched by the resolver — it feeds an IMemoryStore instead, via FileMemoryStore. All three tiers — Session, User, Tenant — are backed by durable storage, read in most-specific-first order (session → user → tenant) so a host can layer per-session scratch memory over durable per-user and per-tenant memory. Each present scope segment is path-encoded as{lowercased}-{hash} (a truncated SHA-256 of the case-sensitive raw value), so case-variant tenant or user ids never collide on a case-insensitive filesystem. RGPD/audit needs are covered by DeleteScopeAsync and EnumerateAsync — both errors-as-data, never throwing on an expected filesystem condition.

##

Hosting (AddKnowledge / AddMemory)

the one Microsoft.Extensions.* dependency

OKF4net.Catalog.Hosting is the sole project in the dependency graph allowed a Microsoft.Extensions.* package — AddKnowledge registers IKnowledgeCatalog/IKnowledgeResolver lazily (no file I/O until the first resolve), and AddMemory registers IMemoryStore from whichever role:memory sources the manifest declares.

using OKF4net.Catalog;
using OKF4net.Catalog.Hosting;

services.AddKnowledge(o => o.AddCatalogFile("./catalog.json"));
services.AddMemory();

// Elsewhere, resolve and search:
IKnowledgeResolver resolver = provider.GetRequiredService<IKnowledgeResolver>();
KnowledgeContext result = await resolver.SearchAsync(new KnowledgeQuery("refund policy"));
ADDMEMORY IS FROZEN AT STARTUP

AddMemory resolves the set of role:memory sources once, at the first IMemoryStore resolution from the container. A later catalog ReloadAsync() does not pick up a memory source added, removed, or edited afterward — that requires rebuilding the DI container. The knowledge resolver has no such limit: it re-reads the live catalog on every search.

docs/agents.md — the context provider that reads and writes through this catalog