spark-acid (homepage)

Hive ACID datasource for Apache Spark

@qubole / (0)

A Datasource on top of Spark Datasource V1 APIs, that provides Spark support for Hive ACID transactions.

This provides the capability to work with Hive ACID V2 tables, both Full ACID tables as well as Insert-Only tables.

Tags

  • 1|spark
  • 1|DataSource
  • 1|data source
  • 1|hive
  • 1|acid
  • 1|hive-acid
  • 1|transactions

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages qubole:spark-acid:0.4.0-s_2.11

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "qubole/spark-acid:0.4.0-s_2.11"

Otherwise,

resolvers += "Spark Packages Repo" at "https://repos.spark-packages.org/"

libraryDependencies += "qubole" % "spark-acid" % "0.4.0-s_2.11"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>qubole</groupId>
    <artifactId>spark-acid</artifactId>
    <version>0.4.0-s_2.11</version>
  </dependency>
</dependencies>
<repositories>
  <!-- list of other repositories -->
  <repository>
    <id>SparkPackagesRepo</id>
    <url>https://repos.spark-packages.org/</url>
  </repository>
</repositories>

Releases

Version: 0.4.0-s_2.11 ( f445ee | zip | jar ) / Date: 2019-07-26 / License: Apache-2.0 / Scala version: 2.11

Version: 0.3.0-s_2.11 ( 2621dc | zip | jar ) / Date: 2019-07-26 / License: Apache-2.0 / Scala version: 2.11

Version: 0.2.0 ( 63d1ed | zip | jar ) / Date: 2019-07-25 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1.0 ( d6c3ba | zip | jar ) / Date: 2019-07-25 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1-0 ( 96a6fb | zip | jar ) / Date: 2019-07-25 / License: Apache-2.0 / Scala version: 2.11