Centerfield Events Processing Platform

5 million events a day from 100+ data sources. The hard part isn't throughput—it's keeping everyone sane about what's happening. Built the observability so complex systems don't become black boxes. Also: getting LLMs to actually work in production without breaking everything. It's weird and hard.

Highlights

• Event processing handling 2-5M events per day without catching fire
• Integrated 100+ data sources into a single pipeline
• Got LLMs actually working in production (the hard part)
• Built observability so people understand what the system is doing
• Infrastructure as Code (CloudFormation) for everything