The Diagram Is a Projection

The first version of the DataParade scanner produced a file called dataflow.json and the app drew it. That is how most tools in this space work: scan, emit a diagram, done. It is also why most of those diagrams are wrong in ways nobody can fix, because the diagram is the only artifact and there is nothing underneath it to correct.

We changed the order. The scanner now writes Discoveries, in OCSF format, into a knowledge base. A Discovery is one sourced assertion about the system: this component exists, this data item appears at this line, this flow carries that item from here to there. Each one says where it came from, which today is a scan, a cloud provider, or an interview. The diagram is a projection of that knowledge base. dataflow.json still exists, with all four layers in it, but it is becoming an interchange format rather than the place where the truth lives.

The short version: once the diagram is a projection, you can ask a precise question. Given the diagram I want to draw, what does the knowledge base need to contain, and what is missing? The missing things are the interview. We built that, it works, and it asks some dumb questions.

Start from the diagram you want

I started with one specific target: the C4 system-context diagram. It is the highest-level picture of a system, with the system as one box, the people who use it, and the external systems it talks to. It is the first thing a privacy reviewer wants to see and the first thing a new engineer wants to see, and the C4 site is explicit about what it needs: actors, the system boundary, external dependencies, and the relationships between them.

So the question for the knowledge base is bounded:

• Do we know who the actors are?
• Do we know which components are inside the boundary and which are outside?
• For each flow across the boundary, do we know what it carries?

If the answers are there, draw the diagram. If they are not, each specific gap is a question.

That is a different thing from asking a model to "interview the user about their system". The gaps define the questions, which means the interview is as short as the knowledge base allows and no shorter.

The first question it asked

We ran it on our own codebase. The scanner had found both customer and user all over the place, and the knowledge base could not tell whether those were one actor or two. So the interviewer asked: are "customer" and "user" synonyms?

I said no, there are internal admin users who are not customers. It adjusted the OCSF records for those two actors and their relationships, re-ran the projection, and produced a system-context diagram with two actor boxes. It did not look very good. But it was the right structural answer, obtained with one question, and the knowledge base now holds a fact that no scanner could have found in the code.

That is the loop. The scan finds what the code can prove. The interview fills in what only a person knows. Both land in the same knowledge base with their source attached, and the diagram is drawn from both.

Every solution leads to new problems

A week later the loop runs end to end as a coding-agent skill: run the scanner, show the diagram, ask questions, update the knowledge base and the diagram from the answers. The new problem is that the interview is not great and it makes dumb decisions.

Two kinds of dumb:

1. Labeling. It will name a component something that is technically derived from the code and useless to a human, and nothing in the loop stops it. That one is fixable with writing guidelines, validation on the formats, and examples, and we are doing that.
2. Blast Radius. When the interview gets an answer, it has to decide how far that answer reaches. "No, admin users are not customers" should change two actor nodes. It should not rename every flow that touched the word user. Deciding the blast radius of an answer is a judgment call, and a model making that call with no evidence constraints will over-apply it about as often as it under-applies it.

The second phase of the interview

The system-context diagram is the start, not the goal. After the gaps are filled, the next thing a stakeholder says is something like "there are too many things on that diagram, it is supposed to be the highest-level view." At that point the interview stops filling knowledge gaps and starts iterating on the projection: what to collapse, what to hide, what this reviewer cares about and that one does not.

Those are different jobs and they need different rules. Filling a gap writes a Discovery with an interview source. Adjusting a projection should not touch the knowledge base at all. It should change the query. We have the first one working and the second one is sketched.

What we haven't tested

• The minimum number of questions. The design goal is to show the first useful diagram with as few questions as possible, or none if the scan is good enough. We have not measured how many questions it takes on a real partner codebase, or how often it asks one it did not need.
• Answers that survive a rescan. A person's correction should persist when the code changes and the scan runs again. Annotations are easy to preserve. Corrections that conflict with new evidence are not, and we have not decided the rule.
• Two repositories into one knowledge base. The scan runs across repositories now, so a frontend and a backend in separate repos can land in one knowledge base. The projection onto one canvas is behind that.

Lessons

1. Decide what you are drawing before you decide what to ask. A target diagram turns an open-ended interview into a checklist with a known end.
2. Separate filling gaps from adjusting views. One writes facts. The other writes queries. Mixing them is how a diagram tweak corrupts the record.
3. A fact from a person needs a source like any other. The answer to "are these synonyms?" is a Discovery with an interview source, not a special case.

The dogfood walkthrough, including the section on what a C4 system-context diagram actually needs, is in the knowledge-base wiki.