Audit logs miss real changes
Most network monitoring tools rely on audit logs to detect changes. The controller or device writes a log entry when an admin modifies something, and the monitoring tool picks it up. That works until a change arrives without a log entry, which happens more often than most teams assume. Bulk imports, API-driven changes, out-of-band modifications, firmware upgrades that reset defaults, scheduled automation that runs outside the audit window: all of these produce real changes that never appear in the audit trail.
SAMURAI doesn’t ask what a device says changed. It compares what the device actually looks like at two points in time. Every sync cycle fetches the full state of every collection (routes, ACLs, interfaces, policies, endpoints) and compares the result to what we stored last time.
Read what the device looks like, not what it says changed.
Strip the fields that always change
Raw device data is noisy. Uptime counters tick every second. Session IDs rotate. Timestamps on last-polled fields change even when nothing meaningful happened. Hash the raw output and every sync reports a “change”, which is as useless as reporting none.
SAMURAI maintains a volatileFields map per device type. Before hashing, the sync engine calls stripVolatileRecursive to walk the document tree and remove every field listed in the map. For vCenter, that means stripping guest_state, runtime uptime counters, connection state, and thin-disk drift metrics. For routers, it strips interface counters and last-input timestamps. Each device type has its own curated list, tuned over months of production use to eliminate false positives without hiding real changes.
Hash what’s left, deterministically
After volatile fields are stripped, the remaining document is serialized into a canonical form. Key ordering matters here: if two identical documents serialize with different key orders, the hashes diverge and you get a phantom diff. SAMURAI uses sortByMarshaledKey to pre-compute json.Marshal per element and sort deterministically. Getting this right mattered. An earlier approach that sorted by string comparison was roughly 300x slower on our largest collections and produced ordering drift that flapped hashes on every cycle.
The SHA-256 hash of the canonical form is compared against the stored hash. If they match, no new snapshot is written. The last_verified_at timestamp updates (proving the sync ran), but last_sync_at only updates when the data actually changed. That split lets operators see “data verified 2 minutes ago, last real change was 6 hours ago” at a glance.
How long snapshots are kept
When a hash does change, the new snapshot is stored alongside the previous one. SAMURAI retains up to SYNC_RETENTION_COUNT snapshots per collection (default: 168, which is one week of hourly syncs). Old snapshots are cleaned up by cleanupOldSnapshots, which scopes deletion to only the collections that were written in the current sync, never touching unrelated data.
What you end up with is a change history built from what the devices actually hold. You can diff any two snapshots side by side and see which fields changed, when, and on which device. Because the comparison runs against real device state rather than an audit log that may never have been written, it catches the changes log-based systems miss.
What this catches that audit logs miss
Configuration pushed over an API with auditing turned off, bulk imports that skip the change log, firmware upgrades that reset interface descriptions, neighbor tables that shift because someone moved a cable, route changes from an upstream provider, DHCP snooping entries for a device that just joined. None of these write an audit log entry, and all of them change device state, which is exactly what SAMURAI compares.