elasticsearch-hadoop (homepage)

Official integration between Apache Spark and Elasticsearch real-time search and analytics

@elastic / (3)

Native Java/Scala API for Elasticsearch in Spark. Read/write RDDs and DataFrames from/to Elasticsearch.
Reference documentation available at https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html 
Note that artifacts for Scala 2.11 are also available - simply use the _2.11 suffix instead in the artifact id.

Tags

  • 1|sql
  • 1|elasticsearch
  • 1|search
  • 1|analytics
  • 1|realtime
  • 1|core
  • 1|data source

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages org.elasticsearch:elasticsearch-spark-20_2.11:5.3.1

sbt

In your sbt build file, add:

libraryDependencies += "org.elasticsearch" % "elasticsearch-spark-20_2.11" % "5.3.1"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch-spark-20_2.11</artifactId>
    <version>5.3.1</version>
  </dependency>
</dependencies>

Releases

Version: 5.3.1 ( 3c2b92 | zip | jar ) / Date: 2017-04-21 / License: Apache-2.0 / Scala version: 2.11

Version: 5.3.0 ( 484aae | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.11

Version: 5.2.2 ( fc99b9 | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.11

Version: 5.2.1 ( ed8e0d | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.11

Version: 5.2.0 ( d85a25 | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.4 ( bb73bd | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.10

Version: 5.1.2 ( 9056be | zip | jar ) / Date: 2017-03-28 / License: Apache-2.0 / Scala version: 2.11

Version: 5.1.1 ( c94f08 | zip | jar ) / Date: 2016-12-15 / License: Apache-2.0 / Scala version: 2.11

Version: 5.0.2 ( 86bae4 | zip | jar ) / Date: 2016-12-15 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.3 ( 6e0a38 | zip | jar ) / Date: 2016-12-15 / License: Apache-2.0 / Scala version: 2.10

Version: 2.4.2 ( 6fa4d4 | zip | jar ) / Date: 2016-12-15 / License: Apache-2.0 / Scala version: 2.10

Version: 2.4.1 ( 37795a | zip | jar ) / Date: 2016-12-15 / License: Apache-2.0 / Scala version: 2.10

Version: 5.0.1 ( 571224 | zip | jar ) / Date: 2016-11-15 / License: Apache-2.0 / Scala version: 2.11

Version: 5.0.0 ( b3b703 | zip | jar ) / Date: 2016-11-03 / License: Apache-2.0 / Scala version: 2.11

Version: 5.0.0-rc1 ( 9731ed | zip | jar ) / Date: 2016-11-03 / License: Apache-2.0 / Scala version: 2.11

Version: 5.0.0-beta1 ( 508575 | zip | jar ) / Date: 2016-11-03 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.0 ( 4ab7a3 | zip | jar ) / Date: 2016-08-31 / License: Apache-2.0 / Scala version: 2.10

Version: 5.0.0-alpha5 ( c51336 | zip | jar ) / Date: 2016-08-10 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.4 ( 054bf2 | zip | jar ) / Date: 2016-08-04 / License: Apache-2.0 / Scala version: 2.10

Version: 2.3.3 ( d28498 | zip | jar ) / Date: 2016-07-07 / License: Apache-2.0 / Scala version: 2.10

Version: 5.0.0-alpha4 ( 705f66 | zip | jar ) / Date: 2016-06-30 / License: Apache-2.0 / Scala version: 2.10

Version: 5.0.0-alpha3 ( 5e5a7d | zip | jar ) / Date: 2016-06-01 / License: Apache-2.0 / Scala version: 2.10

Version: 2.3.2 ( cae8b7 | zip | jar ) / Date: 2016-05-22 / License: Apache-2.0 / Scala version: 2.10

Version: 5.0.0-alpha2 ( e5fb0f | zip | jar ) / Date: 2016-05-03 / License: Apache-2.0 / Scala version: 2.10

Version: 2.3.1 ( cf38a7 | zip | jar ) / Date: 2016-05-03 / License: Apache-2.0 / Scala version: 2.10

Version: 2.3.0 ( a2b025 | zip | jar ) / Date: 2016-04-08 / License: Apache-2.0 / Scala version: 2.10

Version: 2.2.1 ( 10957b | zip | jar ) / Date: 2016-04-08 / License: Apache-2.0 / Scala version: 2.10

Version: 2.2.0 ( c11e37 | zip | jar ) / Date: 2016-02-02 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 33% , - 70% , - 44% , - 72% , - 24% , - 63%

Version: 2.1.3 ( f46e95 | zip | jar ) / Date: 2016-02-02 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 45% , - 100% , - 24% , - 80% , - 34% , - 92%

Version: 2.2.0-rc1 ( ab852c | zip | jar ) / Date: 2016-01-09 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 24% , - 63% , - 33% , - 70% , - 44% , - 72%

Version: 2.2.0-beta1 ( 66539b | zip | jar ) / Date: 2015-10-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 80% , - 34% , - 92% , - 45% , - 100% , - 24%

Version: 2.1.2 ( 4decc4 | zip | jar ) / Date: 2015-10-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 100% , - 24% , - 80% , - 34% , - 92% , - 45%

Version: 2.2.0-m1 ( adfde0 | zip | jar ) / Date: 2015-08-28 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 25% , - 85% , - 35% , - 100% , - 47%

Version: 2.1.1 ( a1fc48 | zip | jar ) / Date: 2015-08-28 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 47% , - 25% , - 85% , - 35% , - 100%

Version: 2.1.0 ( 76e511 | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 25% , - 85% , - 35% , - 100% , - 47%

Version: 2.1.0.rc1 ( 5cc3f5 | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 47% , - 25% , - 85% , - 35% , - 100%

Version: 2.1.0.Beta4 ( 2c62e2 | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 96% , - 50% , - 27% , - 100% , - 37%

Version: 2.1.0.Beta3 ( 157763 | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 100% , - 56% , - 93% , - 70% , - 57%

Version: 2.1.0.Beta2 ( 18188d | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 100% , - 56% , - 93% , - 70% , - 57%

Version: 2.1.0.Beta1 ( cbe367 | zip | jar ) / Date: 2015-06-29 / License: Apache-2.0 / Scala version: 2.10

Spark Scala/Java API compatibility: - 95% , - 97% , - 91% , - 95% , - 100%