Here’s how Dream11 manages an uptime of 99.99 percent

Imagine your business has just the right product in the market and also has a huge consumer base. But when it comes to data analytics or giving the right recommendations to your users, your system suffers a downtime. And now think of the impact your users will have because of this downtime. Not a good sight to imagine, right?

As a fantasy sports platform that serves over 120 million users, Dream11 consumes more than 16+ terabytes of information every day and its teams perform multiple experiments in order to provide the best possible user experience. At such a scale, balancing the set of experiments with performance and rapidly incorporating the outcomes into the application for constant feedback, becomes critical.

This is especially relevant when the company prepares for big-ticket tournaments like the Indian Premier League (IPL), when the user concurrency reaches over 5.5 million at edge services and receives over 100 million requests per minute. One of the key challenges for Dream11 had been to manage this scale and prepare for the possibility of an even larger one.

To manage all these challenges, Dream11 relies on a data warehouse for transactional and interactions data. It serves as an Online Analytical Processing (OLAP) layer for business intelligence reporting.

In conversation with ETCIO, Abhishek Ravi, Chief Information Officer, Dream Sports, shared how investing in a data warehouse managed the company to scale and keep its downtime to the minimum.

“Stability and scalability are the two main things we have consistently achieved on our data warehouse, AWS Redshift. This resulted in giving us an average uptime of 99.99 per cent. This level of reliability is critical in the fantasy sports industry in order to keep customers engaged without any interruption, especially during IPL matches,’ he said.

As a sports technology company, Dream11 experimented with various technologies in the initial stages that gave the company a basic idea of what suits Dream11 the best. The company opted to build its infrastructure on Amazon Web Services (AWS) in 2014. Earlier, it used a mix of ad hoc and scheduled jobs on top of OLTP systems to power its analytics. This helped the team to learn what worked for them but was not scaling with their data-driven strategy. Switching to a cloud-based data warehouse shaped up its data platform roadmap for the massive scale the company handles today.

Ravi feels that manual interventions for repeatable decisions is not a scalable solution, especially when data-driven systems are far better equipped to take care of such tasks efficiently. In addition, data analytics platforms can be leveraged for experimentation platforms, switching to algorithms, conversions, in house data platform philosophy, and product features (referral programs, social, product design, winner templates, etc.)

Machine learning-driven recommendations benefit the users to give them options of which contests to join based on their past in-app behavior and other factors. At any given time, hundreds of contests are running simultaneously on the Dreami1 platform, so it makes the decision making process for the users easy.

“Across Dream Sports and its portfolio of brands that includes Dreaml1, we follow hyper- experimentation that is called HEAL – Hypothesis, Experiment, Analysis, and Learning. Everything that we do, has to run through this HEAL process, which allows us to fail fast, learn and build unique features for our users. We also help and encourage our teams to take every opportunity possible to upskill and learn the latest technology. This has allowed us to repeatedly adopt, and adapt to newer technologies seamlessly,” he concluded.