EPLEXITY designed an AWS cloud solution that focuses on the implementation of real time process control data streaming system. Each rig device complements its existing Supervisory Control and Data Acquisition (SCADA) system. SCADA is designed to work at a lower data frequency and in much large time batches (hours, days, weeks). The new AWS cloud system receives from 50 to 200 data elements every second from each rig. Rig data will be normalized across many different rig types, using transformation algorithms. The implementation is intended to be deployed to thousands of rigs that in the completion phase of operations (i.e. drilling).
The new IOT device connects over MQTT using TLS and x509 certificate for device security. The data stream is transformed when it reaches AWS into a high-volume FIFO queuing system, leveraging AWS SQS, S3, Dynamo DB, and Lambda functions. The SQS topics leverage the stream the data, eliminate sample duplicates, and maintain perfect time sequence without the need of a database store. The data process system allows each data sample to travel through various high capacity transformations, analyses, and augmentations.
The overall design objective of the system is to apply all data transformations in real time, without the need for the batching of samples. External to the real time stream, sample data is used for Machine Learning algorithms leveraging AWS Sagemaker and Sagemaker Studio for the analysis and training. Simulators have been built to replay historical data for analysis and testing, which utilize AWS ECS, Lambda, and MQTT.
Once the real time data has been augmented and cleaned, AWS Glue Batch jobs, AWS Sagemaker Machine Learning algorithms, and deep analysis are applied to provide instant comparison of the data stream to benchmarks or inferences. Within seconds of the data being ingested into AWS Cloud, the system will be able to detect important or critical changes to any or all oil and gas operations. The end to end assessment completes within a few seconds, for every data sample, without any batching or delays in the stream.