Luca Bianchi

Luca Bianchi is CTO of Neosperience, he spent the last decade building next-generation customer analytics platform. Passionate about technology and software architectures, Luca had been one of the early adopters of Lambda in 2014 and CDK in 2019 (in production). He is also the co-organizer of the serverless meetup and ServerlessDays Milan, in Italy

Talk: Lessons Learned building a 150 million serverless data pipeline

Serverless is great for data pipelines, right? When the number of records per second becomes huge, the wrong architecture can lead to a lot of problems, even with serverless. In this talk, we’ll deep dive into common pitfalls and architecture choices, processing hundreds of events per second. In recent years the amount of data generated by brands increased dramatically, thanks to affordable storage costs and faster internet connections. In this article, we explore the advantages serverless technologies offer when dealing with a large amount of data and common pitfalls of these architectures. We are going to outline tips everyone should figure out before starting your next big data project At Neosperience, building our SaaS cloud on AWS, we managed to leverage a number of AWS services. This talk is a deep dive of the choices we made and the reason behind them that made us evolve a standard pipeline with API Gateway + Lambda + DynamoDB into an architecture, able to process hundreds of events per second. In this journey we’ll discover some unexpected behavior, tips and hidden gems of AWS services and how to use them in a real life use case. Basic knowledge of AWS services is required.

Back to the agenda