A Look At The Data Systems Behind The Gameplay For League Of Legends
Data Engineering Podcast
Shownotes Transcript
Summary
The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. In this episode Ian Schweer shares his experiences at Riot Games supporting player-focused features such as machine learning models and recommeder systems that are deployed as part of the game binary. He explains the constraints that he and his team are faced with and the various challenges that they have overcome to build useful data products on top of a legacy platform where they don’t control the end-to-end systems.
Announcements
Hello and welcome to the Data Engineering Podcast, the show about modern data management
When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode) today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show!
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Go to dataengineeringpodcast.com/atlan) today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos.
The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it. Select Star’s data discovery platform solves that out of the box, with an automated catalog that includes lineage from where the data originated, all the way to which dashboards rely on it and who is viewing them every day. Just connect it to your database/data warehouse/data lakehouse/whatever you’re using and let them do the rest. Go to dataengineeringpodcast.com/selectstar) today to double the length of your free trial and get a swag package when you convert to a paid plan.
Data engineers don’t enjoy writing, maintaining, and modifying ETL pipelines all day, every day. Especially once they realize 90% of all major data sources like Google Analytics, Salesforce, Adwords, Facebook, Spreadsheets, etc., are already available as plug-and-play connectors with reliable, intuitive SaaS solutions. Hevo Data is a highly reliable and intuitive data pipeline platform used by data engineers from 40+ countries to set up and run low-latency ELT pipelines with zero maintenance. Boasting more than 150 out-of-the-box connectors that can be set up in minutes, Hevo also allows you to monitor and control your pipelines. You get: real-time data flow visibility, fail-safe mechanisms, and alerts if anything breaks; preload transformations and auto-schema mapping precisely control how data lands in your destination; models and workflows to transform data for analytics; and reverse-ETL capability to move the transformed data back to your business software to inspire timely action. All of this, plus its transparent pricing and 24*7 live support, makes it consistently voted by users as the Leader in the Data Pipeline category on review platforms like G2. Go to dataengineeringpodcast.com/hevodata) and sign up for a free 14-day trial that also comes with 24×7 support.
Your host is Tobias Macey and today I’m interviewing Ian Schweer about building the data systems that power League of Legends
Interview
Introduction
How did you get involved in the area of data management?
Can you describe what League of Legends is and the role that data plays in the experience?
What are the characteristics of the data that you are working with? (e.g. volume/variety/velocity, structured vs. unstructured, real-time vs. batch, etc.)
What are the biggest data-related challenges that you face (technically or organizationally)?
Multiplayer games are very sensitive to latency. How does that influence your approach to instrumentation/data collection in the end-user experience?
Can you describe the current architecture of your data platform?
What are the notable evolutions that it has gone through over the life of the game/product?
What are the capabilities that you are optimizing for in your platform architecture?
Given the longevity of the League of Legends product, what are the practices and design elements that you rely on to help onboard new team members?
What are the seams that you intentionally build in to allow for evolution of components and use cases?
What are the most interesting, innovative, or unexpected ways that you have seen data and its derivatives used by Riot Games or your players?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on the data stack for League of Legends?
What are the most interesting or informative mistakes that you have made (personally or as a team)?
What do you have planned for the future of the data stack at Riot Games?
Contact Info
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
Thank you for listening! Don’t forget to check out our other shows. Podcast.init) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast) helps you go from idea to production with machine learning.
Visit the site) to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected])) with your story.
To help other people find the show please leave a review on Apple Podcasts) and tell your friends and co-workers
Links
The intro and outro music is from The Hug) by The Freak Fandango Orchestra) / CC BY-SA)
Sponsored By:
- Hevo): [
Are you sick of repetitive, time-consuming ELT work? Step off the hamster wheel and opt for an automated data pipeline like Hevo.
Hevo is a reliable and intuitive data pipeline platform that enables near real-time data movement from 150+ disparate sources to the destination of your choice. Hevo lets you set up pipelines in minutes, and its fault-tolerant architecture ensures no fire-fighting on your end. The pipelines are purpose-built to be ‘set and forget,’ ensuring zero coding or maintenance to keep data flowing 24×7. All it takes is 3 steps for your pipeline to be up and running. Moreover, transparent pricing and 24×7 live tech support ensure 24×7 peace of mind for you.
Don’t waste another minute on unreliable data pipelines or painstaking manual maintenance. Sprint your way towards near real-time data integration with a pipeline that is easy to set up and even easier to control. Head over to dataengineeringpodcast.com/hevo and sign up for a free 14-day trial that also comes with 24×7 support.
](https://www.dataengineeringpodcast.com/hevodata))
- Select Star): [
So now your modern data stack is set up. How is everyone going to find the data they need, and understand it? Select Star is a data discovery platform that automatically analyzes & documents your data.
From analyzing your metadata, query logs, and dashboard activities, Select Star will automatically document your datasets. For every table in Select Star, you can find out where the data originated from, which dashboards are built on top of it, who’s using it in the company, and how they’re using it, all the way down to the SQL queries. Best of all, it’s simple to set up, and easy for both engineering and operations teams to use.
With Select Star’s data catalog, a single source of truth in data is built in minutes, even across thousands of datasets.
Try it out for free at dataengineeringpodcast.com/selectstar If you’re a data engineering podcast subscriber, we’ll double the length of your free trial and send you a swag package when you continue on a paid plan.](https://dataengineeringpodcast.com/selectstar))