Data Type Conversions

Data Type Conversion Remarks

• When using the onDuplicateKeySQL option, the connector will error when writing a null-terminated StringType (i.e., \0).

• DECIMAL in SingleStore and DecimalType in Spark have different maximum scales and precisions. An error will occur if you perform a read or write from a table or DataFrame with unsupported precision or scale. SingleStore’s maximum scale for the DECIMAL data type is 30, while Spark’s maximum scale for DecimalType is 38. Similarly, SingleStore’s maximum precision is 65, while Spark’s maximum precision is 38.

• TIMESTAMP in SingleStore supports values from 1000 to 2147483647999. SingleStore treats a null value in the TIMESTAMP column as the current time.

• The Avro format does not support writing TIMESTAMP and DATE types. As a result, the SingleStore Spark Connector currently does not support these types with Avro serialization.