Enterprises are always looking for simpler, faster, more scalable, and cost-effective data processing solutions as the size and complexity of data grows. In addition, the productivity of data scientists and data engineers is a key metric for organizations looking to advance their data-centric applications. Platforms for basic analytics queries on 100GB - 1TB datasets may not work for complex machine learning and AI applications on 10TB+ datasets. Therefore, evaluating the efficacy of compute platforms can be challenging since there are several factors to consider.
One of our customers, a large Fortune 10 enterprise, recently evaluated Bodo for data engineering workloads in their new data infrastructure. We share the results here, which can be informative for anyone interested in large-scale data engineering. In summary, Bodo is much simpler to use and 10x faster than highly optimized Spark with the same cluster setup as measured by the customer.
The customer chose to derive findings from the industry-standard TPCxBB benchmark suite since it can represent data engineering workloads for various applications, including machine learning pipelines. It is designed to represent advanced analytics use-cases that utilize structured, semi-structured and unstructured data. In our experience, even TPCxBB is too simple to represent complex data pipelines in ML applications, and therefore cannot fully exercise Bodo’s optimization capabilities. But it can serve as a good starting point for evaluation.
The TPCxBB benchmarks represent workloads in a retail setting with various data sources (such as online and in-store transactions, logs, and reviews). Still, the benchmark patterns apply to all verticals. In addition, the data generator is scalable and can generate data at any scale factor, which is very useful for scalability tests. Note: Data in this Bodo evaluation is derived from the TPC-xBB Q26 benchmark, and as such is not comparable to published TPC-xBB Q26 results.
Query 26 of the TPCxBB benchmark suite, clusters retail customers into groups based on their purchase history. This particular benchmark is interesting since it represents data preparation and feature engineering stages of an ML pipeline well. Further, focusing on a single benchmark allows examining many factors (memory consumption, energy cost, etc.) in more depth. Above are both Spark and Bodo versions of the code. The Bodo version uses native Python/Pandas APIs and is a straightforward translation of the original SQL code. The Spark version uses SQL for faster speed but for simplicity could also be written in the equivalent PySpark DataFrames APIs which are wrappers around the SparkSQL engine.
The customer environment is a custom AWS infrastructure setup that meets their specific security and governance requirements. Setting up Bodo in this environment proved to be straightforward since Bodo uses a “clean” engine design that has minimal infrastructure requirements. As the alternative baseline for comparison, the customer chose the Elastic Map Reduce (EMR) service of AWS, which provides an optimized version of Apache Spark. Furthermore, their data engineering experts examined Spark’s dozens of configuration options and tuned them manually if necessary. On the other hand, Bodo requires no tuning and can be run efficiently out of the box by anyone experienced with Python.
Both the Bodo and the Spark executions use the same cluster resources. The cluster has 125 instances of c5n.18xlarge type, with a total of 4,500 physical CPU cores. All nodes are in a single AWS Placement Group so that they are physically close to each other in the data center to reduce networking overhead. The input dataset is scale factor 40,000, which has 52 billion rows and is 2.5TB in Parquet format on disk. In this setup, Bodo takes advantage of the Elastic Fabric Network (EFA) feature of AWS, which provides HPC networking (allows a high level of inter-node communications without TCP protocol overheads) at no extra cost.
As the graph below illustrates, Bodo is about 10x faster than EMR Spark for this use-case. Since the exact same compute resources are used, this translates to about 90% AWS infrastructure savings. Furthermore, the team found Bodo to be much easier to use by data scientists and data engineers since it provides standard Python APIs while being easier to set up for the IT team with minimal configuration setup.
In addition, the benchmarking team gathered a lot of other data points. Notably, Bodo scales linearly with the data and the number of nodes while Spark struggles to handle datasets > 1TB and requires additional tuning for efficiency > 1,000 cores. In addition, using EFA is significant for performance, but Bodo is still much faster than Spark even with EFA turned off (Bodo without EFA is 5x faster than Spark). Furthermore, as measured by the customer, Bodo typically uses much less memory than Spark. We encourage the readers to replicate this benchmark and test Bodo for your own applications and let us know your findings here!
Data processing platforms are becoming a critical part of modern data-driven enterprises, and their efficiency translates directly into business value and cost savings. Apache Spark enabled a generation of data platforms and use-cases by improving the simplicity and performance of data processing by 10x or more over Hadoop. With Bodo’s new technology, we aim to advance the data analytics area by providing 10x or more simplicity and performance improvements over Spark. We hope that you also share our excitement of the future to come with Bodo and welcome you to see Bodo in action.
