spark-select (homepage)

Spark select enables retrieving only required data from an object

@minio / (1)

S3 Select is supported with CSV and JSON files using s3selectCSV and s3selectJSON values to specify the data format.

Tags

  • 2|library
  • 2|sql
  • 2|input
  • 2|scala
  • 2|data source
  • 2|s3select
  • 1|tutorial

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages io.minio:spark-select_2.11:2.1

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "minio/spark-select:2.1"

Otherwise,

libraryDependencies += "io.minio" % "spark-select_2.11" % "2.1"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>io.minio</groupId>
    <artifactId>spark-select_2.11</artifactId>
    <version>2.1</version>
  </dependency>
</dependencies>

Releases

Version: 2.1-s_2.11 ( fcfc72 | zip | jar ) / Date: 2019-04-04 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0-s_2.11 ( c35e46 | zip | jar ) / Date: 2019-02-11 / License: Apache-2.0 / Scala version: 2.11

Version: 1.1-s_2.11 ( 1752a8 | zip | jar ) / Date: 2019-01-11 / License: Apache-2.0 / Scala version: 2.11

Version: 1.0.1-s_2.11 ( 564624 | zip | jar ) / Date: 2018-12-18 / License: Apache-2.0 / Scala version: 2.11

Version: 1.0.0-s_2.11 ( 356d74 | zip | jar ) / Date: 2018-12-05 / License: Apache-2.0 / Scala version: 2.11

Version: 0.0.1 ( 73695c | zip ) / Date: 2018-12-04 / License: Apache-2.0