Phase one was the creation of an AWS landing zone in order to launch the new web application to support their customers. EPLEXITY created an AWS landing zone and successfully launched the web application in the cloud using cloud native services including, AWS EKS and ElasticSearch with CloudFormation.
Once launched, phase two was to implement a Content Delivery Network (CDN), the goal of the CDN project was to provide global scale, high availability and performance of the content intensive services that Photobucket provides to its customers. This was critical to further expansion and infrastructure cost reduction and stability.
Upon completion of the CDN project, we launched into Phase three and four in parallel. Phase three was to design and create a Data Lake, replacing their home grown and extremely manual reporting tool. With the deployment of a Data Lake, the Photobucket team can now understand the metrics and data that drive their business. The Data Lake includes marketing data, subscriber data, and financial data. The data is available in real-time and enables timely and informed decision making.
Phase four was by far the largest and most challenging project. Photobucket stores all its customer data, images, videos, etc. on premise in an expensive NAS environment. The new web application currently connects to both AWS S3 cloud storage and the on-premise storage environment. The current on-premise storage has a volume of 3-5 petabytes, with 30,000 smaller segments of storage, 80 million customers, and billions of files.
EPLEXITY created a custom solution using ECS with an auto scaling EC2 Spot Fleet utilizing CloudWatch alarms to migrate the storage to the cloud. To move 15 billion images in flight, an automated process was created using AWS Lambda, Athena, and DynamoDB to trigger DataSync tasks to move the images without interruption to the service.