Process Modes

📄️ Process Modes - Overview

Data processing pipelines in Syntasa offer multiple process modes to efficiently manage data ingestion and updates. These modes are designed to handle different use cases, ensuring that data is either added, updated, or entirely replaced based on the required workflow. Choosing the right process mode depends on factors such as whether the dataset is partitioned, whether historical data should be retained, and how modifications should be managed.

📄️ Drop & Replace - Process Mode

The Drop & Replace mode is designed to completely overwrite an existing dataset with new incoming data during a job execution. This mode is useful when you want to ensure that the latest dataset reflects only the new data and does not retain any previous records.

📄️ Replace Date Range - Process Modes

In data science workflows, especially in time-partitioned datasets, managing data updates with precision is critical. Syntasa’s Replace Data Range process mode provides a reliable and controlled method to update output tables based on a specified execution date range.

📄️ Add New Only- Process Modes

The Add New Only process mode in Syntasa ensures that only new data partitions—those that do not already exist in the output table—are processed and added. Unlike other process modes, it does not overwrite or modify existing date partitions, making it an ideal choice when you only need to add new date partition without altering previously ingested records.

📄️ Add New & Replace Modified - Process Mode

The Add New & Replace Modified process mode in Syntasa is designed to efficiently handle data updates by incorporating new date partitions while ensuring that existing date partitions are updated if they have changed. This mode ensures that your dataset remains up to date by checking for both newly arriving data and modifications to previously processed records.