This started as a straightforward idea.
I wanted to get Defender Threat and Vulnerability Management (TVM) data into Microsoft Sentinel for long-term retention and dashboarding. The data potentially has value, and Sentinel is designed to ingest large volumes of security data, so on the surface it felt like something that should already exist.
After building a working connector, testing it, and trying to scale it, I think I now understand why a 1st party data connector might not exist.
Understanding the Data
One of the first things that stood out is that TVM data behaves differently than most of the data we send to Sentinel.
TVM tables are state-based. They do not behave like traditional log tables where new events are appended continuously. Instead, they represent the current state of devices, software, and vulnerabilities. Records are updated or replaced over time, not appended.
From a descriptive standpoint, a reasonable working estimate is roughly 3,000 records per device when including the full TVM dataset. That number can vary based on how active a device is, how much software is installed, and how many vulnerabilities are present.
At first glance, that volume doesn’t seem extreme from a Sentinel perspective.
The issue isn’t just the row count. It’s the size of the data combined with the constraints of the APIs used to move it.
That means the data remains relatively stable in size, not extremely large, but still large enough to pose data migration challenges.
A key distinction is how vulnerability data is represented.
The table DeviceTvmSoftwareVulnerabilitiesKB is a global dataset, on the order of roughly 300,000 records, and it is the same for every environment. It does not scale per device.
By contrast, DeviceTvmSoftwareVulnerabilities contains the vulnerabilities observed on devices. Functionally, it represents the same vulnerability data, but scoped to the software present on each device. In my testing, this table did not provide enough additional value to justify the cost of moving it at scale. It can always be added back later if needed.
Including both tables makes the dataset appear significantly larger. Removing them changes the problem entirely.
The First Working Version
The initial Logic App followed a simple pattern.
It queried Advanced Hunting, retrieved a batch of data, sent that data to Log Analytics, and repeated the process using an incrementing offset. An Until loop made this easy to reason about. It was clear, predictable, and worked reliably.
In smaller environments, it worked well.
In earlier testing, I was able to move 200,000 records in about 13 minutes, which represented roughly 300 devices, or about one million records per hour.
That established the baseline.
Optimization Attempts
Once I had a working version, the next step was trying to make it faster and more scalable.
I started with the Until loop, then moved to a For each loop with parallel processing, and even experimented with a nested For each design. The nested approach seemed promising on paper, but in practice it added complexity without improving performance. I ultimately rolled back to a single For each loop with controlled parallelism.
The biggest improvement came from reducing the dataset.
Removing DeviceTvmSoftwareVulnerabilitiesKB and DeviceTvmSoftwareVulnerabilities cut the total volume dramatically. In my 300-device lab, the full dataset was about 1.48 GB, with roughly 1.11 GB coming from those two tables alone. After removing them, the dataset dropped to about 370 MB.
From there, I started working the remaining levers.
I tested increasing rows per request, carefully staying within API limits. I pushed parallelism as far as possible without triggering throttling. I also evaluated record size by table.
In the end, the most stable configuration I found was:
- ~250 records per write request
- ~40 parallel executions
With that configuration, I was able to process 112,542 records in under 5 minutes.
That translates to roughly:
- ~22,000 records per minute
- ~1.3 million records per hour
I was able to push to 300 records per request in testing, but it was not consistently reliable. At that point, payload size errors started to appear.
Scaling the Numbers
With the optimized dataset and configuration:
- 300 devices → ~110K records → ~5 minutes
- 3,000 devices → ~1.1M records → ~50 minutes
- 10,000 devices → ~3.5–4M records → ~3 hours
These are still estimates based on a 300-device lab, and larger environments would provide more accurate data.
That said, the trend is clear.
Even with aggressive optimization, runtime scales quickly. This approach is likely to become impractical somewhere beyond roughly 5,000 devices, where a single run begins to stretch into multi-hour execution.
Cost Considerations
From an ingestion cost perspective, the optimized dataset is still manageable, but it is no longer trivial.
In my 300-device lab, after removing the two largest vulnerability tables, the remaining dataset was approximately 370 MB per day. At an estimated $4.50 per GB, that works out to roughly $1.67 per day for 300 devices.
Scaling that out:
- 3,000 devices → about 3.7 GB per day (~$16.65/day)
- 10,000 devices → about 12.3 GB per day (~$55/day)
That translates to roughly:
- $500 per month at 3,000 devices
- $1,600+ per month at 10,000 devices
It is important to note that these estimates assume daily ingestion. In many scenarios, this data could be collected less frequently, such as weekly or on-demand, which would significantly reduce cost.
Even at daily frequency, cost is still not the primary blocker.
The larger concern remains runtime, API limits, and whether the workflow can reliably complete at scale.
API Constraints
There are several layers of constraints that shape the entire design.
The Advanced Hunting API has both row and size limits. You can retrieve up to 100,000 rows per query and up to 100 MB. The API also enforces throttling, with a practical guideline of around 45 calls per minute, along with execution limits that restrict sustained activity.
On the ingestion side, I used the modern Log Analytics ingestion API, which relies on the Data Collection Rule (DCR) pipeline. Each request is limited to about 1 MB, which translated to roughly 200–300 TVM records per request, depending on record size.
The legacy ingestion API is simpler, but it does not materially change this limitation.
There is also a layer of complexity around authentication and permissions.
Sentinel and Logic Apps rely on Azure RBAC and managed identities. The Advanced Hunting API does not. It requires explicit API permissions assigned to an application registration, which creates a disconnect and adds complexity to the solution.
Parallelism and Its Limits
Parallel processing helped, but only to a point.
I was able to run 40 parallel write operations without errors.
Batch size also required tuning. While 300 records per request was occasionally successful, 250 records per request proved to be the most consistent value.
For anyone attempting this, these settings will likely need adjustment. If errors occur, the first steps should be to:
- Reduce batch size (for example, from 250 to 200)
- Reduce parallelism to 35 or 30
Where This Leaves Us
At this point, I have a working Logic App.
It reliably extracts TVM data and sends it into Log Analytics. I will be sharing that Logic App and deployment instructions for anyone who wants to experiment with it:
https://github.com/AndrewBlumhardt/sentinel-defender-tvm-connector
The Logic App may still have value in more targeted scenarios. If you focus on a smaller subset of TVM tables instead of attempting to move the entire dataset, this approach becomes more practical.
The Logic App works for small to medium environments. It does not scale cleanly beyond that.
If a solution cannot be used by every Sentinel customer or organization, then it is not a complete solution.
If this were a MythBusters episode, this is where we would give it a very large stamp:
Busted
Current Workarounds
There are a few partial alternatives today.
Sentinel MCP can read TVM data and provide analysis, but it is not designed for structured extraction or dashboarding.
There is also emerging capability around Sentinel Data Lake and querying that data through Azure Monitor Workbooks and a Query API. However, data lake is not available in all environments, and TVM tables are not listed as supported Data Lake tables in official documentation. In my own testing, I was not able to confirm that TVM data is currently being replicated into Data Lake.
So while this may become viable in the future, it is not a complete solution today.
What a Better Solution Would Look Like
A more scalable approach would likely require:
- Controlled extraction from Advanced Hunting
- Temporary staging (like a Python dataframe)
- Chunking into ingestion-safe payloads
- Parallel ingestion with retry and backoff
In other words, a code-based solution, not a Logic App.
Even then, the value likely comes from focusing on a subset of tables rather than attempting full ingestion to reduce ingestion costs.
Final Thoughts
Going into this, I expected to build a connector.
What I ended up with was a better understanding of the problem.
The data itself is not extreme. The constraints around accessing and moving that data are.
At this point, I see this as validation of why a connector does not already exist.
For now, I consider this project complete. I may circle back to a more advanced solution in the future.
Epilogue: Logic Apps
This experience didn’t just expose the limits of this solution. It exposed the limits of Logic Apps.
The idea of low-code or no-code sounds great. In practice, it doesn’t really deliver here.
If you can code, or even vibe code, you are going to be faster and more effective building this in Python or PowerShell. You get control, visibility, and far fewer hidden constraints.
If you can’t code, Logic Apps don’t remove complexity. They just hide it. And when something breaks, it’s often harder to understand than it would be in code.
What’s more confusing is the idea of using vibe coding to build Logic Apps. If you’re describing a workflow in prompts, it makes more sense to generate code directly rather than target a constrained platform.
To me, the real value of vibe coding isn’t making Logic Apps easier. It’s making them unnecessary.
We’re not quite there yet.