GPT-4o
⚙️ Technical
Advanced
Real-Time Data Pipeline Agent
Design an AI agent that processes and routes streaming data in real time — with transformation logic, AI enrichment, and an architecture diagram.
The Prompt
# Real-Time Data Pipeline Agent You are a data engineering architect designing an AI agent that processes, transforms, and routes streaming data in real time. ## Pipeline Spec - Data source: [SOURCE — e.g., webhook events, API polling, database change stream, IoT sensor feed] - Data volume: [APPROXIMATE EVENTS PER MINUTE OR HOUR] - Transformation required: [WHAT NEEDS TO CHANGE — e.g., enrich, normalize, filter, aggregate] - Downstream destinations: [WHERE PROCESSED DATA GOES — e.g., database, dashboard, alert system, another API] - Latency requirement: [ACCEPTABLE PROCESSING DELAY — e.g., under 5 seconds, near real-time, batch OK] ## Pipeline Design Design the full real-time agent pipeline: 1. Ingestion layer: how events are received and buffered 2. Transformation logic: the step-by-step operations applied to each event 3. AI enrichment step: where the agent applies LLM-based enrichment (e.g., classification, sentiment, entity extraction) 4. Routing rules: which events go where based on content 5. Error handling: what happens to malformed events — dead letter queue, retry, or discard 6. Monitoring: the metrics tracked to confirm the pipeline is healthy Produce an architecture diagram (ASCII) and a pseudocode implementation for the transformation and routing layer.
📝 Fill in the blanks
Replace these placeholders with your own content:
[SOURCE — e.g., webhook events, API polling, database change stream, IoT sensor feed]
[APPROXIMATE EVENTS PER MINUTE OR HOUR]
[WHAT NEEDS TO CHANGE — e.g., enrich, normalize, filter, aggregate]
[WHERE PROCESSED DATA GOES — e.g., database, dashboard, alert system, another API]
[ACCEPTABLE PROCESSING DELAY — e.g., under 5 seconds, near real-time, batch OK]
How to use this prompt
1
Hit "Run it in your AI"
We open ChatGPT, Claude, Gemini, Perplexity, Grok or Copilot with this prompt in the box, ready for you to press Enter — and copied to your clipboard as a backup. Pick your AI once and we remember it.
2
Replace the placeholders
Swap out anything in [BRACKETS] with your specific details.
3
Send it
This prompt is tuned for GPT-4o, but it works well in any of them. Prefer to do it by hand? Copy to clipboard is always there too.
Get prompts like this every Monday
Free weekly digest — hand-picked prompts, new tools, AI news.