Which type of pipeline in Foundry typically has the lowest compute cost?

Incremental pipelines in Foundry are typically designed to process only the new or changed data since the last execution, rather than reprocessing the entire dataset. This approach reduces the amount of data being processed at any given time, leading to lower compute costs compared to batch pipelines, which handle large volumes of data all at once, or streaming pipelines, which continuously process data in real time.

By focusing on only the modifications, incremental pipelines optimize resource usage and efficiently manage costs associated with compute resources. This targeted processing is particularly beneficial in scenarios involving frequent updates or changes in datasets, allowing organizations to maintain consistent performance while conserving computational resources.