Data collection rule

In the Ultra Transcenders books

AZ-305AZ-104AZ-700SC-500SC-200AZ-400AZ-802DP-800ALZ

Each book explains Data collection rule in context, with comparison tables and the common traps.

Terms in this definition

• Azure Monitor

  Observability platform for Azure that brings together metrics, logs and traces from both Azure and hybrid resources so they can be analysed and alerted on.

• Azure Monitor agent

  Agent now used to collect logs from a machine's guest OS, driven by data collection rules; it took over from the Log Analytics agent (MMA).

• Logs Ingestion API

  Lets you push your own data into Log Analytics: requests go to a data collection endpoint, and a data collection rule decides how it lands in the workspace.

• XPath

  Expressions such as Security!*[System[(EventID=4648)]], placed in a DCR's xPathQueries, that decide which Windows events AMA gathers. Each portal box takes 20 at most, and a DCR is limited to 100.

• Event

  Table in Log Analytics where entries from Windows event logs are kept.

• DCR transformation

  A KQL statement inside a data collection rule that drops or reshapes incoming records before storage. Selecting which Windows events the agent gathers is done with XPath instead.

Related terms

• CEF via AMA

  Sentinel data connector in which a Linux forwarder runs the Azure Monitor Agent with a DCR to receive CEF messages; it supersedes the older CEF connector built on the Log Analytics agent.

• Codeless Connector Framework

  Lets you create SaaS data connectors for Microsoft Sentinel without writing code, by defining four things in sequence: the custom output table, the DCR, the connector's UI and its connection rules.

• Custom log table

  Table in Log Analytics whose name ends in _CL and whose schema you define, populated through a DCR with data no standard table suits.

• Custom Logs via AMA

  Collects text-file logs into a _CL table, using the Azure Monitor Agent plus a data collection rule, from Windows or Linux machines or a log forwarder. It is a Microsoft Sentinel data connector still in preview.

• Custom table

  When no built-in schema fits your data, you create one of these in Log Analytics. Its name ends in _CL, and a data collection rule feeds data into it.

• Guest data

  Telemetry captured within a virtual machine's operating system: Windows events, Syslog, perf counters, IIS logs and text logs. Azure Monitor Agent has to be installed, with a data collection rule, before any of it is gathered.

• Ingestion-time transformation

  A KQL statement inside a data collection rule that drops or modifies incoming records before they are written to the workspace. Selecting which Windows events to gather is a separate job, done through XPath.

• Monitoring Metrics Publisher

  If an app or identity sends data via the Logs ingestion API, give it this Azure role on the data collection rule; it provides the Microsoft.Insights/Telemetry/Write permission.