Curiosity

JSON recipe

Source: JsonSample/ · a top-level JSON array of skill documents.

Owns in the academic graph: the skill taxonomy — skills, skill categories, prerequisites, related skills, learning resources.

What it teaches

  • Two-pass ingestion to handle forward references — pass 1 emits all nodes, pass 2 wires inter-document edges by key.
  • Node.FromKey(nameof(Type), key) to reference a node whose key is built elsewhere in the same run.
  • Symmetric edges (e.g. RelatedTo in both directions).
  • AddOrUpdate for documents that may be re-emitted on re-run.

Two-pass ingest

public static void Ingest(Graph graph, IReadOnlyList<SkillDoc> skills)
{
    // Pass 1 — emit nodes and their direct edges.
    foreach (var s in skills)
    {
        var skill = graph.AddOrUpdate(new Nodes.Skill
        {
            Name           = s.Name,
            Description    = s.Description,
            Popularity     = s.Popularity,
            YearIntroduced = s.YearIntroduced,
        });

        if (!string.IsNullOrWhiteSpace(s.Category))
        {
            var category = graph.TryAdd(new Nodes.SkillCategory { Name = s.Category });
            graph.Link(skill, category, Edges.HasCategory, Edges.CategoryOf);
        }
    }

    // Pass 2 — skill→skill edges by key so document order doesn't matter.
    foreach (var s in skills)
    {
        var src = Node.FromKey(nameof(Nodes.Skill), s.Name);

        foreach (var prereq in s.Prerequisites)
            graph.Link(src, Node.FromKey(nameof(Nodes.Skill), prereq),
                       Edges.RequiresSkill, Edges.RequiredBy);

        foreach (var related in s.Related)
            graph.Link(src, Node.FromKey(nameof(Nodes.Skill), related),
                       Edges.RelatedToSkill, Edges.RelatedToSkill);
    }
}

Configuration

Variable Default
RECIPE_JSON_PATH data/skills.json
CURIOSITY_CONNECTOR_NAME JSON Sample (Skills)

Reuse notes

  • JsonSource.cs provides LoadArray<T> and LoadObject<T> for flexible JSON shapes.
  • The two-pass pattern is essential when documents forward-reference each other by name.