Parquet writer silently swallows JSON unmarshal errors — invisible data loss

[ngoyal16]

Problem

In main.go lines 283–288, the Parquet writer silently swallows JSON unmarshal errors:

var event DatabaseActivityEvent
if err := json.Unmarshal(eventBytes, &event); err == nil {
    if err := writer.Write(event); err != nil {
        return nil, fmt.Errorf("failed to write parquet row: %w", err)
    }
}
// If err != nil, the event is silently skipped — no logging, no metric

This means:

1. If a DAS event has an unexpected schema (e.g., new field types from an RDS engine update), events are silently dropped without any log or metric.
2. There is no way to distinguish between "no events matched the filter" and "events failed to unmarshal."
3. Data loss is invisible.

The same pattern exists at line 279 (json.Marshal error → continue with no log).

Proposed Solution

1. Log the error with context:

eventBytes, err := json.Marshal(eventMap)
if err != nil {
    slog.Warn("Failed to marshal filtered event", "error", err)
    continue
}

var event DatabaseActivityEvent
if err := json.Unmarshal(eventBytes, &event); err != nil {
    slog.Warn("Failed to unmarshal event into DAS struct", "error", err, "rawEvent", string(eventBytes))
    continue
}

2. Add a metric for dropped events (pairs with Add observability: metrics, tracing, and DLQ support #6 — observability).
3. Consider making strict unmarshal optional — unknown fields could be preserved or flagged.

Acceptance Criteria

• All silently-swallowed errors are logged with slog.Warn or slog.Error
• Log includes enough context to diagnose the root cause (event ID, field name if possible)
• A metric counts dropped events due to unmarshal failures