SoftwarePakistan2023
Streaming data platform for a B2B retail marketplace
Order, inventory, and credit events unified into a governed lakehouse feeding same-hour analytics.
- Client
- Bazaar Technologies
- Region
- Pakistan
- Sector
- B2B-commerce data platform
- Engagement
- 2023 · 22 weeks
- Team
- 3 data · 1 backend · 1 platform
- Status
- In production
The challenge
A fast-growing B2B commerce network ran analytics off nightly exports, so pricing, credit, and inventory decisions lagged a day behind the market. They needed streaming ingest from operational services into a governed lakehouse with same-hour freshness.
What we built
The full pipeline, end to end — not a blurb.
- 01Ingest
Kafka with Debezium change-data-capture off the Postgres services, governed by a schema registry.
- 02Transform
Spark structured streaming with dbt models and enforced data contracts.
- 03Storage
An Apache Iceberg lakehouse on S3 with Trino query and a semantic layer.
- 04Operations
EKS with Airflow orchestration, Terraform, and data-quality tests in CI.
Results
55 s
pipeline freshness for order + inventory events, from a 6-hour batch
99.95%
ingest reliability across 380M events/day
34%
lower warehouse cost vs. the prior nightly stack
1.9 s
p95 analyst query over 90-day windows
Pricing and credit teams now work from same-hour data.
Have a constraint like Bazaar Technologies’s? Bring us yours.
Next engagement
Sadeem
Environmental / flood IoT