BACKGROUND
Data sovereignty requires knowing exactly where regulated data is stored, processed, and transmitted, and verifying that every destination complies with jurisdictional policy. Modern architectures make this straightforward requirement surprisingly difficult. Data moves through background queues, centralized logging pipelines, vendor SDKs, and external APIs, and many of these destinations never appear in the source repository. Endpoints and credentials are typically resolved at deployment time, while frameworks hide service clients behind dependency injection and logging systems capture payloads without explicit configuration. Reconstructing these flows manually across a portfolio of applications is slow and error-prone. Static scanners only find what they are explicitly programmed to track, so if a destination is injected at runtime or abstracted by a framework, the scanner simply misses it.
HYPOTHESIS
Could we reliably trace where sensitive data flows through modern applications, verify each step with actual code, and map those findings to jurisdictional policy constraints? If we could, we could automate sovereignty audits without relying on brittle, signature-based scanners. The challenge isn’t just finding endpoints. It’s following data across runtime configuration, dependency injection, framework wiring, and deployment-only settings, then proving each destination against policy rules with auditable evidence. Success would mean producing a structured, verifiable trail that compliance teams can trust, regardless of how the application is built or deployed.
APPROACH
We tested this by first building a traditional data-flow scanner. We built language-specific parsers for Python and TypeScript, integrated Joern to trace data across function boundaries, added a Terraform reader to pull deployment metadata, and maintained a registry of known providers alongside BC Government privacy rules as policy checks. The scanner worked outward from sensitive data, building a chain of facts until it hit a policy rule. On a synthetic benchmark, the results looked promising. Joern handled cross-function tracing effectively, and combining it with our native parsers yielded high precision and recall across our test cases. Benchmarks, however, only confirm coverage on known patterns. They do not reveal unknown architectural choices. When we ran the scanner against a production government application, it missed several obvious flows. The application used vendor-specific SMS SDKs, sent email through a standard mail library, wrote to PostgreSQL and Redis queues, and routed logs to a centralized pipeline, none of which matched our predefined destination categories. We also missed an object-storage flow because the client was instantiated in one module and passed to another via dependency injection, requiring us to follow the framework wiring across multiple files. Adding signatures for every SDK and framework would simply rebuild a static analyzer that never scales to real-world complexity.
Those blind spots pointed directly to the next phase. Instead of continuing to patch the scanner, we tested whether a coding agent could perform the full investigation. We replaced the signature-matching logic with an agent that reads the repository structure, maps modules, dependencies, configuration files, and infrastructure definitions, and asks targeted questions about data handling, storage, external services, and deployment-only destinations. By tracing the application structure rather than searching for hardcoded URLs, the agent found integrations, stores, and data movement without us needing to guess the exact library API in advance.
The current version of Sovereignty Trace is intentionally narrow. It launches a coding agent, initially Claude via the Agent SDK and later Swival, and runs a strict process around it. The agent follows a fixed instruction set, reading source files, searching dependencies, following configuration, inspecting infrastructure, mapping framework wiring, and updating earlier assumptions as it learns. It systematically looks for sensitive data handling, databases, external APIs, messaging, queues, logging destinations, object storage, AI providers, deployment configuration, and runtime-supplied destinations. Crucially, the agent never types quoted code. It points to a file and line range, and code reads the actual text directly from the repository. If a citation does not resolve to real lines, the finding is dropped, and a finding that loses all citations is discarded entirely.
The first version let the agent fill in the entire assessment, including risk scores and jurisdiction. Running it five times on the same commit produced 0, 4, 1, 5, and 4 high-risk findings, with four of those runs failing validation. Because large models struggle with consistent scoring, we narrowed the agent’s scope. Per finding, it now makes a few decisions: whether a finding exists, which provider matches our list, how the data is classified, whether the code path is live or alternate, which region the repository pins, and which category it falls under. It also writes plain-language notes. Everything derived from those decisions moved into plain code. Code reads the cited lines, resolves jurisdiction, scores risk, and rolls up coverage. The scoring rule is a test-covered function, meaning the same inputs always produce the same score.
We split provider and policy knowledge to control how the agent accesses it. We first tried keeping the prompt lean by letting the agent fetch details on demand, but across five runs it answered 2, then 4, then 3, then 5, then 4 of the same six policy questions. Judging policy compliance requires reading the full text, so we now inject the complete policy into every session upfront. Provider knowledge is split differently. The agent sees only enough to recognize a service, including its name, short description, domains, and package names. Where that service stores or processes data is held back, allowing code to read the residency facts after the agent names the provider. Giving the agent residency facts invites it to recite them from memory instead of checking the record, so the agent simply identifies the provider while code enforces the sovereignty rules. If a provider fact, location, or policy implication cannot be established from evidence, the result stays UNKNOWN. Both provider and policy records require human review. Subject matter experts who understand storage, processing, legal entities, and regional options curate provider records, while compliance experts curate policy records. We started with BC Government because it provides a concrete jurisdiction and policy context, but the structure supports other public bodies, provinces, or countries maintaining their own policy packs without changing the agent’s investigation method.
Because the agent no longer writes quoted evidence or derived fields, most of the earlier validation step disappeared. Code now checks that the draft matches the expected schema and that the assessment covered all ten component categories and six policy questions. A draft that does not parse is the only real error, and the system simply asks the agent to retry. Everything else is recorded as a gap in the final report, which still stands and explicitly notes where coverage is thin. Claude Code served as the initial runtime, but we later added Swival through an open agent protocol. The adapter test proved the method does not depend on Claude, as the same instructions and structured output drive both. Swival exists specifically to run fully offline, pointing at a model on organizational hardware so that source code and the model never leave the environment. For sovereignty work, that isolation is non-negotiable. We tested whether a local model can carry the investigation, and a local Qwen 3.6 model performs on par with Claude Sonnet 5 across the same repositories.
TAKEAWAYS
01
A strong benchmark score can still hide real gaps. Our controlled tests showed high precision and recall, but a production application exposed major blind spots. The benchmark only proved the scanner could find patterns we already knew how to track. It revealed nothing about runtime configuration, dependency injection, or framework wiring. A passing score only confirmed we had encoded our own assumptions.
02
Where data goes is usually an architectural relationship you trace through the application. We expected domains, URLs, and known destinations to be strong signals. In practice, services were hidden behind SDK abstractions, runtime config, and framework wiring. A repository can show an application uses a service without ever naming the endpoint. That shifts the task from extracting destinations to understanding how the application is structured.
03
An LLM placed after the scanner can only reason about what the scanner already found. Our first resolver improved provider identification, but it could only work with the scanner’s output. If the scanner missed a queue, database, or logging system, the LLM never saw it either. The fix was moving the agent upstream to run the repository investigation directly, instead of enriching incomplete scanner results.
04
Let the model judge; let plain code do the math. Our first version let the agent fill in the whole assessment, including risk scores. Five runs on the same commit produced 0, 4, 1, 5, and 4 high-risk findings. The model was good at reading repositories and spotting where data went, but shaky at applying scoring rules consistently. The agent now makes a few judgments per finding, while plain code handles jurisdiction, risk scoring, and validation. A test-covered function guarantees the same inputs always land on the same score.
05
Show the model what it needs to recognize something, and hold back the facts it would otherwise guess. The agent sees only enough to identify a service: its name, domains, and package names. It doesn’t see where that service stores data, because it would tend to recite that from memory instead of checking the record. Code reads the residency facts after the agent names the provider. Policy runs the other way. Judging compliance requires reading the full text, so we inject the complete policy into every session after learning that on-demand fetching produced inconsistent answers.
THE CODE
Sovereignty Trace is being developed as an open-source experiment.
The full source code is on GitHub.