Cursor Fields
🔁 Incremental Syncs with Cursor Field – Minimizing Data Transfer in AudienceSync
🧠 Overview
AudienceSync supports an optimized Incremental Sync mode using a Cursor Field. This allows you to transfer only new or updated records by leveraging a field that tracks changes over time—resulting in faster syncs and less data duplication.
🔍 This feature is only available for sources that support cursor fields (currently only date fields are supported).
🧭 What Is a Cursor Field?
A Cursor Field acts as a timestamp marker to track what data has changed since the last sync.
Key Benefits:
- Ensures only modified or added records are synced.
- Enables the system to resume exactly where it left off.
- Optimizes performance and reduces API strain or data volume.
⚠️ Do not edit or remove a cursor field after setup—it may cause data loss or disrupt sync integrity.
🔄 Sync Run Examples
🏁 Sync Run 1
Cursor value: 2024-04-20 10:00:00
Source data:
Name | Plan | Updated At |
Charles Beaumont | free | 2024-04-20 10:00:00 |
Eleanor Villiers | free | 2024-04-20 11:00:00 |
Query:
sql
CopyEdit
SELECT * FROM source_table
WHERE updated_at >= '2024-04-20 10:00:00';
➡️ Both records are synced (all are considered new).
🔁 Sync Run 2
Cursor value: 2024-04-20 11:00:00
Source data:
Name | Plan | Updated At |
Charles Beaumont | free | 2024-04-20 10:00:00 |
Eleanor Villiers | paid | 2024-04-21 10:00:00 |
Query:
sql
CopyEdit
SELECT * FROM source_table
WHERE updated_at >= '2024-04-20 11:00:00';
➡️ Only Eleanor’s updated record is synced.
🔁 Sync Run 3
Cursor value: 2024-04-21 10:00:00
Source data:
Name | Plan | Updated At |
Charles Beaumont | paid | 2024-04-22 08:00:00 |
Eleanor Villiers | pro | 2024-04-22 09:00:00 |
Query:
sql
CopyEdit
SELECT * FROM source_table
WHERE updated_at >= '2024-04-21 10:00:00';
➡️ Both updated records are synced again.
⚖️ Handling Ambiguity & Inclusive Cursors
The Challenge:
If the cursor field is too coarse (e.g., tracks only dates), distinguishing between old and new records within the same day may be difficult.
AudienceSync’s Solution:
We implement "at least once delivery"—so even if there's ambiguity, data is safely re-sent. It avoids missing updates but may result in duplicates, which should be handled downstream.
🚫 Known Limitations
- If a record changes without updating the cursor field, it will not be picked up in incremental syncs.
- Editing or deleting a cursor field breaks the tracking system, leading to potential data loss or inconsistencies.
🧷 To maintain sync reliability: Never change or remove the cursor field after configuration.
Last updated on August 6, 2021