Data Migration & Modernization

Data Infrastructure Migration and Modernization

Driving Better Decision-making for a Global Commerce Company

Data Infrastructure Migration and Modernization

Driving Better Decision-making for a Global Ecommerce Company

At a glance

The client is one of the world’s largest commerce companies. The company’s 17 million users have earned $3.7B in cashback through its affiliate marketing programs. TVS Next partnered with them to modernize their data infrastructure, streamline data analytics, and accelerate their growth.

Services

Intelligence

Big Data Legacy Modernization, Data lake, Real Time Streaming, Data Virtualization, Data Replication, Machine Learning, Recommendation System, Market Basket Analysis, Feature Engineering, Offer Personalization

Commerce

Start a Conversation

Problem

The company caters to millions of customers each month, with various business divisions diligently monitoring these transactions for consumer behavior, pricing structures, commissions, and promotional offers. These insights are instrumental in delivering an optimal shopping experience and ensuring successful shopping cart conversions. To manage this monumental task, the company has historically relied on a disparate, localized, legacy system.

However, as the volume of data escalated, numerous business divisions began executing diverse workloads and spontaneous analytics on the platform. This increased activity started to strain the disk space and processing power, pushing them to their absolute limits. The applications were ill-equipped to handle such scale, resulting in frequent crashes that caused substantial disruption to their operations. The relentless need for maintenance and troubleshooting began to hamper productivity within the business units, thereby impacting their overall revenue.

The client recognized the urgent need to develop a scalable data infrastructure capable of processing the immense volume of data. This upgrade would ensure timely delivery of critical data insights to all business units, bolstering their ability to make informed decisions and drive growth.

Strategy

The team at TVS Next conducted intensive Discovery Workshops with the executive team of the commerce company in question. This process allowed us to gain a comprehensive understanding of the current performance level of their system and how it was hindering the company from achieving their business Key Performance Indicators (KPIs).

Subsequently, we developed a strategic roadmap aimed at modernizing their data infrastructure and transitioning their data warehouse to the Amazon Web Services (AWS) cloud. Our team executed a Proof of Concept (POC) over a span of four weeks, during which we successfully migrated 15 terabytes of on-premise data to the cloud, all while maintaining support for their existing environment.

Impressed by the agile, efficient, and seamless nature of our migration process, the company chose to partner with TVS Next for their entire migration and modernization journey.

Transformation Journey

Our team undertook a comprehensive modernization of the company’s data infrastructure and shifted the data warehouse workloads from an on-premise 270 TB Cloudera Hadoop cluster to the Amazon Web Services (AWS) cloud. Utilizing Snowflake, we established data marts on AWS and employed AtScale to optimize query performance and provide a virtualized data perspective.

We tapped into our proprietary Enterprise Data Platform Solution Accelerator to construct a job orchestration platform. This platform ingests, processes, and stores data within the data warehouse. The data ingestion layer pulls data from a variety of sources such as PostgreSQL, MySQL, and Apache Kafka, transferring it to Snowflake on AWS. The data processing layer retrieves data from Snowflake, processes it, and subsequently deposits it into a final data warehouse layer. We engineered the orchestration platform with a strong focus on ease-of-use and scalability.

Even a novice user can readily schedule and configure a job within this system. We successfully migrated approximately 600 jobs to the cloud using the job orchestration framework. The revamped data platform is capable of processing near real-time tasks and batch jobs on data, thereby empowering business units with actionable insights for more informed decision-making.

Business Outcomes

Complete Data Availability

Providing the groundwork for decision-making powered by advanced analytics.

Seamless Business Operations

Ensuring no disruptions, thanks to auto-scaling during periods of peak usage.

30% Reduction

In the load on the on-premise computing cluster, enhancing overall system performance.

68% faster

Real-time feedback and live monitoring, facilitating swift and informed decision-making.

Share it to your network