Leveraging Amazon’s AWS Cloud to Create an Integrated Analytics Platform
In 2018, the founders of Orby TV decided the traditional paid TV model, which was holding Americans captive, had become too expensive and complicated. As a team of TV industry veterans, they knew there had to be a better way— one that shifted control back to the customer. In that spirit, they launched Orby TV with the mission of bringing trust, transparency, and integrity back to the American TV market. Their goal is to provide simple, affordable, and flexible entertainment options without the stress, long term contracts, and gimmicks that have become commonplace in the industry.
In early 2019 Orby TV satellite services were launched and are now available in the 48 contiguous United States. Orby TV is privately held and is not affiliated with other TV providers.
The Challenge
Orby TV is utilizing many different applications from various software vendors to manage customers, installations, inventory, and marketing whilst also maintaining their own customer facing application. With the need for many different software platforms to manage their business, came the challenge to collect, store, process, analyze, and report on the data produced from each of those disparate systems. In order to consistently create the best experience for their customers, Orby TV felt they needed to have the ability to ask questions of all of their data in a timely, consistent, and cost-effective manner.
One unique challenge was getting access to the CRM (Customer Relationship Management) application database. Due to policy and security constraints, we were unable to use solutions such as data movement scripts directly against the database or API calls. The only viable method of receiving the data consisted of the CRM application performing a nightly full database backup and pushing the complete file.
Our Solution
The goal was to deliver a data warehouse, analytics environment, and reporting environment used for decision support. This needed to be done without incurring additional licensing or hardware. Due to the rapid customer growth Orby TV was experiencing, the solution needed to be scalable and cost effective.
Deploying the solution on Amazon Web Services (AWS) affords the opportunity to address concerns about the ability to scale out the computing and storage to accommodate additional sources as Orby TV’s customer base continues to grow. This approach also allowed the development team to use serverless components where appropriate and possible, which provides the needed scale and cost effectiveness.
Architectural Diagram
Not sure how to begin your journey adopting AWS? Let WCI be your guide.
Creating a Data Warehouse Using Fargate
Through the use of S3 buckets and Lambda functions, an automated solution was created to invoke a container within Fargate to restore the nightly CRM database backup to a Postgres database, running in EC2. Using Fargate allows Orby TV to easily, and programmatically, manage the lifecycle of the restore process. While in some cases this can be done with Lambda, variability in the size of the backup file and the time constraints that are inherent in the Lambda function would not consistently allow the restore to complete. The flexibility and ease with which we can manage Fargate and the container within makes this an optimal solution from a technical, time constraint, and cost management perspective.
Several of Orby TV’s key operational systems, including their E-Commerce platform, Supply Chain, and Field Services Partner Platform are integrated into the data warehouse in a similar fashion. A daily harvesting of data, whether pushed or pulled from each system, is moved into an S3 Data Lake via the use of SFTP, API calls, the S3 Transfer Acceleration service, and Amazon Simple Email Service (SES). Once the CRM database is restored, an event is triggered to kick off a series Lambda functions to move the files from the S3 data lake to a staging area on the Postgres database. Once all the data is staged, a final Lambda function is kicked off to run scripts to transform and load the data from the staging area into the data warehouse. From there, a set of automated reports are kicked off and delivered to the appropriate recipients each morning.
Within months of launch, the data has grown exponentially along with the Orby TV customer base. The “serverless first” approach gave Orby TV the flexibility to deal with an anticipated, but unknown, increase in resource requirements as customers quickly adopted the new service and internal information requests grew.
Results & Benefits
With the guidance and support of WCI Data Solutions, the Orby TV analytics platform was well received by all stakeholders. Given the importance placed on customer satisfaction, Orby TV now has the ability to marry their consumer data with operational data to create an exceptional experience for their customers and make data driven decisions for the business.
Orby TV now has a scalable solution, which makes simple work of the ever-increasing data volume and velocity from new sources and business ventures.
