spark-deep-learning (homepage)

Deep Learning Pipelines for Apache Spark

@databricks / (3)

Deep Learning Pipelines aims at enabling everyone to easily integrate scalable deep learning into their workflows, from machine learning practitioners to business analysts. It builds on Apache Spark's ML Pipelines for training, and on Spark DataFrames and SQL for deploying models. It includes high-level APIs for common aspects of deep learning so they can be efficiently done in a few lines of code.

Tags

  • 1|machine learning
  • 1|deep learning
  • 1|GPU

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages databricks:spark-deep-learning:1.5.0-spark2.4-s_2.11

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "databricks/spark-deep-learning:1.5.0-spark2.4-s_2.11"

Otherwise,

resolvers += "Spark Packages Repo" at "https://repos.spark-packages.org/"

libraryDependencies += "databricks" % "spark-deep-learning" % "1.5.0-spark2.4-s_2.11"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>databricks</groupId>
    <artifactId>spark-deep-learning</artifactId>
    <version>1.5.0-spark2.4-s_2.11</version>
  </dependency>
</dependencies>
<repositories>
  <!-- list of other repositories -->
  <repository>
    <id>SparkPackagesRepo</id>
    <url>https://repos.spark-packages.org/</url>
  </repository>
</repositories>

Releases

Version: 1.5.0-spark2.4-s_2.11 ( 154e7a | zip | jar ) / Date: 2019-01-25 / License: Apache-2.0 / Scala version: 2.11

Version: 1.4.0-spark2.4-s_2.11 ( a4792f | zip | jar ) / Date: 2018-11-18 / License: Apache-2.0 / Scala version: 2.11

Version: 1.3.0-spark2.4-s_2.11 ( 7e5238 | zip | jar ) / Date: 2018-11-12 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.0-spark2.3-s_2.11 ( bc7ed2 | zip | jar ) / Date: 2018-08-28 / License: Apache-2.0 / Scala version: 2.11

Version: 1.1.0-spark2.3-s_2.11 ( ca31f2 | zip | jar ) / Date: 2018-06-18 / License: Apache-2.0 / Scala version: 2.11

Version: 1.0.0-spark2.3-s_2.11 ( b09974 | zip | jar ) / Date: 2018-05-01 / License: Apache-2.0 / Scala version: 2.11

Version: 0.3.0-spark2.2-s_2.11 ( d6ef20 | zip | jar ) / Date: 2018-02-08 / License: Apache-2.0 / Scala version: 2.11

Version: 0.3.0-spark2.1-s_2.11 ( d6ef20 | zip | jar ) / Date: 2018-01-30 / License: Apache-2.0 / Scala version: 2.11

Version: 0.2.0-spark2.1-s_2.11 ( 4cd6d4 | zip | jar ) / Date: 2017-10-26 / License: Apache-2.0 / Scala version: 2.11

Version: 0.1.0-spark2.1-s_2.11 ( 2bf96c | zip | jar ) / Date: 2017-06-06 / License: Apache-2.0 / Scala version: 2.11