How do I save a Java bean using the Spark Cassandra Connector?

Question

I've been through the Spark docs but unsure as to how to to save a Java bean into a table using the Spark Cassandra Connector?

public class NewImageMetadataRow implements Serializable {

    private final String merchant;
    private final String productId;
    private final String url;
    private final int width;
    private final int height;

    public NewImageMetadataRow(NewImageMetadataRow row) {
        this.merchant = row.getMerchant();
        this.productId = row.getProductId();
        this.url = row.getUrl();
        this.width = row.getWidth();
        this.height = row.getHeight();
    }

    public String getMerchant() {
        return merchant;
    }

    public String getProductId() {
        return productId;
    }

    public String getUrl() {
        return url;
    }

    public int getWidth() {
        return width;
    }

    public int getHeight() {
        return height;
    }
}

I have an RDD RDD[NewImageMetadataRow] that I'm trying to save like so:

myRDD.saveToCassandra(keyspace, "imagemetadatav3", SomeColumns("merchant", "productid", "url"))

This leads to this error:

java.lang.IllegalArgumentException: requirement failed: Columns not found in com.mridang.image.NewImageMetadataRow: [merchant, productid, url]
    at scala.Predef$.require(Predef.scala:281)
    at com.datastax.spark.connector.mapper.DefaultColumnMapper.columnMapForWriting(DefaultColumnMapper.scala:106)
    at com.datastax.spark.connector.mapper.MappedToGettableDataConverter$$anon$1.<init>(MappedToGettableDataConverter.scala:35)
    at com.datastax.spark.connector.mapper.MappedToGettableDataConverter$.apply(MappedToGettableDataConverter.scala:26)
    at com.datastax.spark.connector.writer.DefaultRowWriter.<init>(DefaultRowWriter.scala:16)
    at com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1.rowWriter(DefaultRowWriter.scala:30)
    at com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1.rowWriter(DefaultRowWriter.scala:28)
    at com.datastax.spark.connector.writer.TableWriter$.apply(TableWriter.scala:423)
    at com.datastax.spark.connector.RDDFunctions.saveToCassandra(RDDFunctions.scala:35)

From my understanding (and bad Scala foo) it seems to be failing on deducing the property names from the Java bean.

The other issue is that the column names in my table are all lowercased with spaces and hyphens removed ie the corresponding Cassandra column for the getter getProductId is productid .

(If I were using Jackson, I could simply add JsonProperty annotation. I'm wondering I can do the same with the Cassadra Mapper. )

Answer 1

This took a bit of pokningg around but turned out to be like so:

val columns: RowWriterFactory[NewImageMetadataRow] =
  CassandraJavaUtil.mapToRow(classOf[NewImageMetadataRow])

myRDD.saveToCassandra(keyspace, "imagemetadatav3")(CassandraConnector(sc), columns)

The fields in the bean needed to be public and annotated with the @CqlName annotation.

@CqlName("merchant")
public final String merchant;