spark-tensorflow-connector (homepage)

Spark Tensorflow Connector

This package contains a library for loading and storing TensorFlow records with Apache Spark. The library implements data import from the standard TensorFlow record format (TFRecords) into Spark SQL DataFrames, and data export from DataFrames to TensorFlow records.

Tags

  • 2|data source
  • 2|tensorflow
  • 1|library

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages tapanalyticstoolkit:spark-tensorflow-connector:1.0.0-s_2.11

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "tapanalyticstoolkit/spark-tensorflow-connector:1.0.0-s_2.11"

Otherwise,

resolvers += "Spark Packages Repo" at "https://repos.spark-packages.org/"

libraryDependencies += "tapanalyticstoolkit" % "spark-tensorflow-connector" % "1.0.0-s_2.11"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>tapanalyticstoolkit</groupId>
    <artifactId>spark-tensorflow-connector</artifactId>
    <version>1.0.0-s_2.11</version>
  </dependency>
</dependencies>
<repositories>
  <!-- list of other repositories -->
  <repository>
    <id>SparkPackagesRepo</id>
    <url>https://repos.spark-packages.org/</url>
  </repository>
</repositories>

Releases

Version: 1.0.0-s_2.11 ( b561f8 | zip | jar ) / Date: 2017-02-21 / License: Apache-2.0 / Scala version: 2.11