Data Engineering Scenario Labs
Apply Apache Beam concepts to production-grade engineering problems. Review skeleton code scripts, analyze test datasets, inspect architectural DAG diagrams, and explore step-by-step solutions.
Employee Analytics
Learn basic transformations by filtering and aggregating employee records.
Student Records
Parse student grades and compute the average score per subject.
Product Catalog
Filter products by price range and list catalog summaries.
Sales Report
Summarize transactions to calculate total sales revenue.
Customer Orders
Analyze retail transactions and generate total customer spending aggregations.
Inventory Analytics
Calculate total stock valuation and identify low stock items.
Banking Transactions
Audit accounts by summing deposits and withdrawals.
HR Analytics
Compute average salary by department and identify the highest earner.
Telecom Monitoring
Process real-time streaming cellular events using Fixed Windows.
IoT Analytics
Aggregate device temperature readings using sliding windows.
Website Clickstream
Count real-time webpage pageviews using streaming windows.
Fraud Detection
Detect rapid successive banking transactions to flag potential fraud.
Session Analytics
Group website click logs into user session windows to measure engagement.
Kafka Pipeline
Consume from an Apache Kafka topic stream and output parsed JSON payloads.
Pub/Sub to BigQuery
Design a streaming ingestion pipeline from Google Cloud Pub/Sub to Google BigQuery.
Real-Time Dashboard Pipeline
Calculate continuous dashboard aggregations with dynamic triggers.