What are the differences between SingleStoreDB and Spark SQL?

  • Spark SQL treats datasets (RDDs) as immutable - there is currently no concept of an INSERT, UPDATE, or DELETE. You could express these concepts as a transformation, but this operation returns a new RDD rather than updating the dataset in place. In contrast, SingleStoreDB is an operational database with full transactional semantics.

  • SingleStoreDB supports updatable relational database indexes. The closest analogue in Spark is IndexRDD, which is currently under development, and provides updatable key/value indexes.

You can connect SingleStoreDB to Spark with the SingleStore Spark Connector. The SingleStore Spark Connector 3.0 is the latest GA version.