“Grok” sounds like a standard. It reads like one too: %{IPV4:client} looks the same whether you paste it into Logstash, an Elasticsearch ingest pipeline, a Graylog extractor, or a Datadog log pipeline. So people write a grok pattern once and assume it ports. It mostly does — until the 10% where it quietly doesn’t, and you spend an afternoon working out why the exact same pattern that worked in Logstash drops fields in Graylog.
Here is what actually differs between the four.
Logstash grok — the reference implementation
Logstash’s grok filter is the one everyone means when they say “grok.” Under the hood it compiles to Oniguruma (via Joni, the JVM port), and the pattern names — IPV4, HTTPDATE, NUMBER, WORD — come from the logstash-patterns-core library. A pattern like:
%{IPV4:client} %{WORD:method} %{URIPATHPARAM:path} %{NUMBER:bytes:int}
expands each %{NAME} into its underlying regex and captures the :name part. Custom patterns go in a patterns_dir file. This is the baseline; the other three are variations on it.
Elasticsearch ingest grok — same patterns, different plumbing
The grok processor in an ingest pipeline uses the same Oniguruma engine and the same pattern library, so your Logstash pattern usually drops straight in. Two practical differences:
- Custom patterns don’t live in a file — they go inline in a
pattern_definitionsobject on the processor. - It runs inside the cluster, on the ingest node, as JSON. Which means your pattern is now a JSON string, and every backslash has to be doubled:
\[becomes\\[. Hand-editing this is where the bugs come from; it is worth generating.
{ "grok": { "field": "message",
"patterns": ["%{IPV4:client} \\[%{HTTPDATE:ts}\\]"],
"pattern_definitions": { "MYAPP_ID": "[A-Z]{3}-\\d+" } } }
Graylog grok — watch the named-capture default
Graylog uses java-grok, which is again Oniguruma-flavored, so the patterns match the same way. But Graylog has a setting that bites almost everyone the first time: only_named_captures defaults to false.
Why that matters: many stock patterns contain unnamed groups internally. %{IPV4}, %{HOSTNAME}, %{URIPATHPARAM} all have bare (...) groups inside their definitions. With only_named_captures off, Graylog promotes every one of those anonymous groups to a field with a numeric name. You asked for client and ts; you also get 0, 1, 2, 3 full of fragments. On a pipeline rule you set it explicitly:
let m = grok(pattern: "%{IPV4:client} %{HOSTNAME:host}", value: to_string($message.message), only_named_captures: true);
set_fields(m);
On an imported extractor, the equivalent key is named_captures_only in the extractor config, and it also defaults false. If your Graylog messages sprout mystery numeric fields, this is why.
Datadog “grok” — a different vocabulary entirely
Datadog’s Grok Parser is the odd one out. It uses the %{matcher:name} shape, so it looks like grok, but the matcher names are Datadog’s own small set — notSpace, word, integer, number, boolean, ipv4, ipv6, data, regex(...), date(...) — not the logstash-patterns-core names. %{IPV4:client} is not valid; you write %{ipv4:client}. And there is no HTTPDATE-style timestamp macro at all — you supply an explicit Joda-Time format:
my_rule %{ipv4:client} \[%{date("dd/MMM/yyyy:HH:mm:ss Z"):ts}\] "%{word:method} %{notSpace:path}"
Types without a native matcher (MAC address, UUID) fall back to notSpace or an explicit regex("…"). If you port a Logstash pattern to Datadog by find-and-replacing, you will get a rule that parses nothing, because every %{UPPERCASE} name is unknown to it.
The RE2 cousins
Two more engines are grok-adjacent and worth knowing about. Vector’s parse_grok uses an Oniguruma-based crate, so it supports the full pattern set including lookaround. But Vector’s parse_regex and Loki/promtail’s regex stage use RE2 (Rust and Go), which has no lookaround and uses (?P<name>) named groups — so a raw regex that leans on lookahead won’t port there at all. That distinction gets its own post.
So which “grok” is this pattern for?
The takeaway isn’t that one is better — it’s that “I have a grok pattern” is underspecified. Before you paste one, you need to know: which pattern library (IPV4 vs ipv4), how custom patterns are declared (file, inline pattern_definitions, or global registry), whether named-capture-only is on, and whether you’re about to lose an afternoon to JSON backslash-escaping.
LogForge sidesteps the whole thing: paste a log line and it emits the correct dialect for each target — Logstash Grok, an Elasticsearch ingest pipeline, a Graylog pipeline rule with only_named_captures set, and a Datadog rule with the right matcher names and a derived date() format. It runs entirely client-side; nothing is uploaded.
FAQ
Will my Logstash grok pattern work in Elasticsearch ingest? Usually yes — same engine, same pattern names. You mainly have to move custom patterns into pattern_definitions and double the backslashes for JSON.
Why does Graylog add fields I didn’t ask for? only_named_captures/named_captures_only defaults to false, and stock patterns contain unnamed groups. Set it true.
Can I copy a Logstash pattern straight into Datadog? No. Datadog uses its own matcher names (ipv4, word, integer) and explicit date() formats, not the logstash-patterns-core names.