spark-bigquery (homepage)

Google BigQuery support for Spark, SQL, and DataFrames

@spotify / (3)

Add support to read BigQuery tables and SELECT query results as DataFrames and write DataFrames to BigQuery tables.

Tags

  • 1|sql
  • 1|input
  • 1|data source
  • 1|Google Cloud

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages com.spotify:spark-bigquery_2.10:0.2.2

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "spotify/spark-bigquery:0.2.2"

Otherwise,

libraryDependencies += "com.spotify" % "spark-bigquery_2.10" % "0.2.2"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>com.spotify</groupId>
    <artifactId>spark-bigquery_2.10</artifactId>
    <version>0.2.2</version>
  </dependency>
</dependencies>

Releases

Version: 0.2.2-s_2.10 ( 0d1aee | zip | jar ) / Date: 2017-11-29 / License: Apache-2.0 / Scala version: 2.10

Version: 0.2.2-s_2.11 ( 0d1aee | zip | jar ) / Date: 2017-11-29 / License: Apache-2.0 / Scala version: 2.11

Version: 0.2.1-s_2.11 ( 6fbb83 | zip | jar ) / Date: 2017-08-16 / License: Apache-2.0 / Scala version: 2.11

Version: 0.2.0 ( 9d5042 | zip ) / Date: 2017-01-18 / License: Apache-2.0

Version: 0.1.2-s_2.11 ( 396e43 | zip | jar ) / Date: 2016-08-09 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1.2-s_2.10 ( 396e43 | zip | jar ) / Date: 2016-08-09 / License: Apache-2.0 / Scala version: 2.10

Version: 0.1.1-s_2.11 ( 04be4a | zip | jar ) / Date: 2016-07-13 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1.1-s_2.10 ( 04be4a | zip | jar ) / Date: 2016-07-13 / License: Apache-2.0 / Scala version: 2.10

Version: 0.1.0-s_2.11 ( a2455c | zip | jar ) / Date: 2016-05-25 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1.0-s_2.10 ( a2455c | zip | jar ) / Date: 2016-05-25 / License: Apache-2.0 / Scala version: 2.10