Qubical, Inc announces the beta release of QubicEngine, a columnar execution engine for Apache Spark that would speed up data queries up to 10x. The company is opening up their sign-up page to the public for those who are interested in trying out the cutting edge technology.

Apache Spark is currently the most commonly used technology for analytics, data science, and AI. The nature of horizontal scaling allows users to add more machines to process data faster. This means data processing speed is limited by the number of machines and the amount of money spent. The acceleration of processing speed implies reduced wait time and cost.

Here are some remarks from the founder and CEO of the company, Tiong Lee:
“We are extremely excited to get the product to the beta release stage – after working on it for more than a year. Our main mission is to accelerate the data science and AI innovation by speeding up the bottleneck in ETL pipelines and queries.”

“Whether people realize it or not, we are coming to the intersection where the increase in data collection and the defunct of Moore’s law would make BigData processing more expensive. The consequences of this are stifled data innovation due to the cost. Qubiqe would allow us to bring data science and AI revolution back on track – without breaking the bank”. Tiong said.

To explain how the technology works at the high-level:

“Qubiqe works by planning a totally different execution plan in Apache Spark and operates on columnar data representation. Columnar-approach for data processing is not new. What is groundbreaking about Qubiqe is the algorithms that make the processing of data much more efficient than traditional approaches. Currently for 1TB distributed TPC-DS we are seeing 2x, 3x, 5x and even 10x performance improvement against Apache Spark. We are still at the dawn of exploring the implications of Qubiqe for BigData processing.”

Tiong is a software veteran and entrepreneur with more than 20 years of experience. He has previously worked as a Principal Staff and Lead in BigData team in Ooyala. He also worked in Oracle, Inc on performance monitoring product.

While Apache Spark is open-source, Qubiqe is not open-source and adopts a commercial licensing model.
“One of the main reasons that Qubiqe is not open-source is that performance engineering is hard to achieve at crowd-sourced environment without compromising the integrity of the product. Regardless, we will contribute back to the Apache Spark community as much as we can when we see generic changes that could improve the overall performance of Apache Spark.”

The company currently is self-funded and according to Tiong, this is mainly to avoid scaling up too fast too soon.

“We love talking to VC and we always kept them in the loop so that we can raise funding when time is right.”

The beta sign-up is located at http://qubical.io.

Apache®, Apache Spark are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.