Turn file metadata into nodes and filter by it

Office documents and PDFs carry authoring metadata — who wrote the file, who saved it last, which company the template belonged to. Curiosity's extractor pulls that out and stores it on the _FileEntry node as a string dictionary, but a dictionary value is not something a user can filter by. This page walks through promoting those values to real graph nodes with a code index, so each one becomes a related facet in search.

The end result: a user searching for battery can narrow the results to documents authored by Alex Doe, or last saved by Sam Roe, from the normal search sidebar.

Before starting, you need a workspace with files already ingested and extracted, and admin access to Manage → Data and Manage → Indexes. The mechanics of the code-index editor itself are covered in Code Indexes — this page uses one, it doesn't explain one.

What the extractor gives you

Once a file has been extracted, its node looks like this (trimmed, with sample values):

{
  "UID": "3kQmXpT7vR2bN9wLcF4dHs",
  "Type": "_FileEntry",
  "OriginalName": "Shared Documents/product-catalog.docx",
  "ContentType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
  "Extension": "docx",
  "Indexed": true,
  "Metadata": {
    "LastSavedBy": "Sam Roe",
    "Author": "Alex Doe",
    "Company": "Contoso Ltd"
  }
}

Two fields matter here:

The keys the built-in extractor writes are Author, LastSavedBy, Company, Title and Description; connectors add their own (e.g. Subject on ingested emails). Which ones are worth promoting is a cardinality question — see Filters and facets. Author, LastSavedBy and Company repeat across many files and make good facets. Title and Description are near-unique per file; leave them as properties.

Why one node type per metadata key

Related facets are grouped by the node type of the neighbour, not by the edge type. The facet computation walks the edges of each search result and buckets every neighbour by its node type — the edge type only ever excludes an edge, it never splits a bucket.

So if Author and LastSavedBy both pointed at a shared Person type, they would collapse into one "Person" facet, and HasAuthor versus HasLastSavedBy would not separate them. One node type per metadata key is what gives you one facet per metadata key:

Step 1 — Create the node types

Go to Manage → Data → Nodes and click New node schema once per metadata key. Each type needs only a key field:

No extra fields are required. The key field carries the value, and because node identity for application-defined types is derived deterministically from (type, key), the same author name always resolves to the same node — which is exactly what makes the facet count meaningful. See Property graph model → Identity.

A new schema starts with its Label Field set to the key field and its display name derived from the type name (FileLastSavedBy → "File Last Saved By"). That label is what the facet shows, so it works out of the box; adjust the display name, icon, and colour under Manage → Data → Nodes → type → Styling if you want something friendlier.

Step 2 — Create the edge types

Under Manage → Data → Edges, click New edge schema for each direction:

The forward edge is the one the facet needs — facets are computed from the edges of the search results, and the search results are files. The reverse edge is what makes the metadata node useful to click on ("show me every document by this author"), so add both.

Step 3 — Write the code index

Create a Custom Code Index with Node Type set to _FileEntry (listed as File in the picker). The UI walkthrough, batch-size fields, and save/validation rules are in Creating a code index.

The extraction gate

A file node is committed as soon as the blob lands, long before its content has been extracted. At that point Indexed is false and Metadata is empty — there is nothing to promote yet, and the index run for that file has no useful work to do.

You do not need to retry it. When extraction finishes, the extractor writes Metadata and flips Indexed to true in the same commit, and committing a node re-queues it for every index targeting its type. So the file comes back to your code index on its own, with the metadata already in place:

Treat an unextracted file as done for this batch (don't add it to the failed list) — returning it as a failure just burns retries against a file that the extractor will re-queue anyway.

The body

var failed = new List<UID128>();

// One node type per metadata key: facets group by neighbour node type, not by edge type.
var mappings = new[]
{
    (MetadataKey: "Author",      NodeType: N.FileAuthor.Type,      KeyField: N.FileAuthor.Name,      Edge: E.HasAuthor,      Reverse: E.AuthorOf),
    (MetadataKey: "LastSavedBy", NodeType: N.FileLastSavedBy.Type, KeyField: N.FileLastSavedBy.Name, Edge: E.HasLastSavedBy, Reverse: E.LastSavedByOf),
    (MetadataKey: "Company",     NodeType: N.FileCompany.Type,     KeyField: N.FileCompany.Name,     Edge: E.HasCompany,     Reverse: E.CompanyOf),
};

foreach (var fileUid in ToIndex)
{
    if (CancellationToken.IsCancellationRequested) return ToIndex;

    if (!await TryPromoteMetadataAsync(fileUid))
    {
        failed.Add(fileUid);
    }
}

return failed;

async Task<bool> TryPromoteMetadataAsync(UID128 fileUid)
{
    // Read-only pass first: no lock, no commit, for the files that have nothing to do.
    if (!Graph.TryGet(fileUid, out var file)) return true;

    if (!file.GetBool(N._FileEntry.Indexed))
    {
        Logger.LogDebug("Skipping {UID}: not extracted yet", fileUid);
        return true;
    }

    var metadata = file.GetDictionary(N._FileEntry.Metadata);
    var wanted   = new List<(string NodeType, string KeyField, string Value, string Edge, string Reverse)>();

    foreach (var m in mappings)
    {
        if (!metadata.TryGetString(m.MetadataKey, out var value)) continue;

        value = (value ?? "").Trim();

        if (string.IsNullOrWhiteSpace(value)) continue;

        wanted.Add((m.NodeType, m.KeyField, value, m.Edge, m.Reverse));
    }

    if (wanted.Count == 0) return true;

    var locked = new List<LockedNode>();

    try
    {
        var fileNode = await Graph.TryGetLockedAsync(fileUid);

        if (fileNode is null) return true;

        locked.Add(fileNode);

        foreach (var w in wanted.OrderBy(w => w.NodeType).ThenBy(w => w.Value, StringComparer.Ordinal))
        {
            var metadataNode = await Graph.GetOrAddLockedAsync(w.NodeType, w.Value);
            locked.Add(metadataNode);

            // GetOrAddLockedAsync derives the UID from (type, key) but does not populate the key
            // field, so set it explicitly — it is what the facet renders as its label.
            metadataNode.SetString(w.KeyField, w.Value);

            fileNode.AddUniqueEdge(w.Edge, metadataNode);
            metadataNode.AddUniqueEdge(w.Reverse, fileNode);
        }

        await Graph.CommitAsync(locked.ToArray());
        return true;
    }
    catch (Exception ex)
    {
        Logger.LogError(ex, "Failed to promote metadata for {UID}", fileUid);
        Graph.AbandonChanges(locked.ToArray());
        return false;
    }
}

Points worth calling out:

• Every lock is released. Either CommitAsync or AbandonChanges runs for each locked node — the code-index scope fails the batch if a lock is left dangling.
• Locking order is deterministic (file first, then metadata nodes sorted by type and value), so two workers touching the same author node can't deadlock against each other.
• SetString on the key field is not optional. GetOrAddLockedAsync(type, key) uses the key only to derive the node's UID; the field itself stays empty until you write it, and an empty label field renders as a blank facet entry.
• An empty key throws. Trim and skip blank metadata values before calling GetOrAddLockedAsync.

Idempotency is what stops the loop

A Custom Code Index reacts to edge changes as well as content changes, so the commit above re-queues the file for this very index. That is fine, because the second pass is a no-op: AddUniqueEdge on an edge that already exists doesn't mark the node as changed, SetString with the same value doesn't either, and a commit with nothing changed doesn't re-queue anything. The file makes exactly one extra pass and then settles.

Step 4 — Enable the related facets

Creating the nodes is not enough: a node type only produces a facet once it's on the workspace's related-facet list. Two places do the same thing:

• Per type — Manage → Data → Nodes → type → Search → Other Settings → Related Facets. Flip the toggle for FileAuthor, FileLastSavedBy and FileCompany.
• All types at once — Manage → Search → Facets → Related Facets (#/manage/search/facets?show=related-facets), which lists every node type with the enabled ones first.

The setting is stored as DefaultRelatedFacets in the search pipeline configuration, so it travels with config/search.json in a definitions export.

Step 5 — Filter by it

From the UI there's nothing left to do: the facets appear in the search sidebar, and picking a value narrows the results to files carrying that edge.

From code, add the facet to the SearchRequest by the UID of the metadata node you want to filter on:

var authorUID = Node.GetUID(N.FileAuthor.Type, "Alex Doe");

var request = SearchRequest.For("battery")
   .SetBeforeTypesFacet(N._FileEntry.Type)
   .SetRelatedFacet(N.FileAuthor.Type, authorUID);

var query = await Graph.CreateSearchAsUserAsync(request, CurrentUser, CancellationToken);

return query.Emit();

SetRelatedFacet also takes an IEnumerable<UID128> for an OR across several values, plus two flags: invertedBehaviour: true excludes the matches instead of keeping them, and applyBefore: true applies the filter before facet counts are computed (so the remaining facets reflect the filtered set). See Filters and facets.

Backfilling files that were already extracted

Saving a new code index enqueues every existing node of the target type, so a freshly created index walks the whole file corpus on its own. Saving an edit to an existing index does not — from then on the new body only sees files that get committed again.

To force a full re-pass over files that were extracted before you changed the body, use Recreate this index in Manage → Indexes → Indexes, as described in Creating a code index → Forcing a re-run.

Troubleshooting

Related pages

• Creating a code index — the editor, batch sizes, validation rules, export/import.
• Code Index Scope — every identifier available in the body.
• Extract files to Markdown — the sibling recipe for the file's text rather than its metadata.
• Filters and facets — property, related, time, and value facets side by side.
• Schema design — the node-versus-property decision rule this page applies.
• Property graph model — node identity, keys, and edge uniqueness.