Package org.apache.cassandra.spark.bulkwriter

Class BulkWriterConfig

java.lang.Object

org.apache.cassandra.spark.bulkwriter.BulkWriterConfig

All Implemented Interfaces:

java.io.Serializable

public final class BulkWriterConfig
extends java.lang.Object
implements java.io.Serializable

Immutable configuration data class for BulkWriter jobs that is safe to broadcast to Spark executors. This class contains pre-computed, serializable values that were computed on the driver.

Serialization Architecture: This class is the ONLY object that gets broadcast to Spark executors (via Spark's broadcast mechanism). It contains broadcastable wrapper implementations with ZERO transient fields and NO Logger references:

• BroadcastableClusterInfo or BroadcastableClusterInfoGroup - cluster metadata
• BroadcastableJobInfo - job configuration with BroadcastableTokenPartitioner
• BroadcastableSchemaInfo - schema metadata

On the driver, BulkWriterContext instances use driver-only implementations: CassandraClusterInfo, CassandraJobInfo, CassandraSchemaInfo. Before broadcasting, these are converted to broadcastable wrappers to avoid Logger references and minimize Spark SizeEstimator overhead.

On executors, BulkWriterContext instances are reconstructed from this config using BulkWriterContext.from(BulkWriterConfig), which detects the broadcastable wrappers and reconstructs the full implementations with fresh data from Cassandra Sidecar.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
BulkWriterConfig(BulkSparkConf conf, int sparkDefaultParallelism, BroadcastableJobInfo jobInfo, IBroadcastableClusterInfo clusterInfo, BroadcastableSchemaInfo schemaInfo, java.lang.String lowestCassandraVersion)	Creates a new immutable BulkWriterConfig with pre-computed values

Method Summary

Modifier and Type	Method	Description
IBroadcastableClusterInfo	getBroadcastableClusterInfo()
BroadcastableJobInfo	getBroadcastableJobInfo()
BroadcastableSchemaInfo	getBroadcastableSchemaInfo()
BulkSparkConf	getConf()
java.lang.String	getLowestCassandraVersion()
int	getSparkDefaultParallelism()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BulkWriterConfig

public BulkWriterConfig(@NotNull
                        BulkSparkConf conf,
                        int sparkDefaultParallelism,
                        @NotNull
                        BroadcastableJobInfo jobInfo,
                        @NotNull
                        IBroadcastableClusterInfo clusterInfo,
                        @NotNull
                        BroadcastableSchemaInfo schemaInfo,
                        @NotNull
                        java.lang.String lowestCassandraVersion)

Creates a new immutable BulkWriterConfig with pre-computed values

Parameters:

conf - Bulk writer Spark configuration

sparkDefaultParallelism - Spark default parallelism setting

jobInfo - Broadcastable job information

clusterInfo - Broadcastable cluster information (BroadcastableCluster or BroadcastableClusterInfoGroup)

schemaInfo - Broadcastable schema information

lowestCassandraVersion - Lowest Cassandra version in the cluster

Method Detail

getConf

public BulkSparkConf getConf()

getSparkDefaultParallelism

public int getSparkDefaultParallelism()

getBroadcastableJobInfo

public BroadcastableJobInfo getBroadcastableJobInfo()

getBroadcastableClusterInfo

public IBroadcastableClusterInfo getBroadcastableClusterInfo()

getBroadcastableSchemaInfo

public BroadcastableSchemaInfo getBroadcastableSchemaInfo()

getLowestCassandraVersion

public java.lang.String getLowestCassandraVersion()