High performing connector to object storage for Apache Spark. Comes with a complete driver for OpenStack Swift
@SparkTC / (1)
Stocator is high performing connector to object storage for Spark, achieving performance by leveraging object store semantics. Stocator comes with a complete driver for OpenStack Swift and can easily be extended to support other object storage interfaces.
Include this package in your Spark Applications using:
spark-shell, pyspark, or spark-submit
> $SPARK_HOME/bin/spark-shell --packages com.ibm.stocator:stocator:1.0.8
In your sbt build file, add:
libraryDependencies += "com.ibm.stocator" % "stocator" % "1.0.8"
MavenIn your pom.xml, add:
<dependencies> <!-- list of dependencies --> <dependency> <groupId>com.ibm.stocator</groupId> <artifactId>stocator</artifactId> <version>1.0.8</version> </dependency> </dependencies>