log-timestamp-annotator

Parses mixed-format log streams and normalizes every timestamp (unix seconds, unix millis, RFC3339, and MySQL datetime) into a single canonical UTC ISO-8601 field, preserving the original line structure so downstream dashboards and alert rules keep working.

When to use

Use when a service emits timestamps in more than one format and a consumer (Grafana, Elasticsearch pipeline, or a hand-written script) needs one consistent field to sort, group, or bucket by. Also useful before loading old log dumps into a schema that expects a fixed datetime type.

Behavior

• Detects the format of each timestamp independently per line.
• Writes the normalized value into a new ts_utc field; never overwrites the original timestamp column.
• Leaves lines without a recognizable timestamp untouched.
• Handles up to 100k lines per invocation; larger inputs are processed in chunks with a progress counter on stderr.

Example

$ log-annotate --input app.log --output app.norm.log
  line 12:   1630000000            -> ts_utc=2021-08-26T21:46:40Z
  line 13:   1630000000123         -> ts_utc=2021-08-26T21:46:40.123Z
  line 14:   2021-08-26 21:46:40   -> ts_utc=2021-08-26T21:46:40Z

Exit codes

0 on success, 1 if any line failed to parse, 2 on invalid flags.

Limitations

Assumes timestamps are in UTC or carry an explicit offset. Naive local timestamps are reported to stderr instead of being guessed.