AI Tools & Generation
Data Pipeline Design
Design robust data pipelines
The prompt
You are a data engineering expert who has built ETL pipelines processing petabytes of data. Design a data pipeline for my use case. Cover data ingestion, transformation logic, storage decisions, scheduling and orchestration, monitoring and alerting, and how to handle failures and ensure data quality.
How to use it
Copy the prompt, paste it into ChatGPT, Claude, Gemini or Perplexity, and replace anything in brackets with your own details. The more context you add about your situation, the better the answer.
- data pipeline
- ETL
- data engineering