riff (homepage)

Spark SQL row-oriented indexed file format

@sadikovi / (1)

Riff is a Spark SQL row-oriented file format designed for faster writes and point queries, and reasonable range queries compare to Parquet. It is built to work natively with Spark SQL eliminating row values conversion, performing predicate pushdown and indexing.

Tags

  • 1|library
  • 1|sql
  • 1|input
  • 1|data source
  • 1|index
  • 1|riff

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages sadikovi:riff:0.2.0-s_2.11

sbt

If you use the sbt-spark-package plugin, in your sbt build file, add:

spDependencies += "sadikovi/riff:0.2.0-s_2.11"

Otherwise,

resolvers += "Spark Packages Repo" at "https://repos.spark-packages.org/"

libraryDependencies += "sadikovi" % "riff" % "0.2.0-s_2.11"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>sadikovi</groupId>
    <artifactId>riff</artifactId>
    <version>0.2.0-s_2.11</version>
  </dependency>
</dependencies>
<repositories>
  <!-- list of other repositories -->
  <repository>
    <id>SparkPackagesRepo</id>
    <url>https://repos.spark-packages.org/</url>
  </repository>
</repositories>

Releases

Version: 0.2.0-s_2.11 ( 5908c3 | zip | jar ) / Date: 2017-09-08 / License: MIT / Scala version: 2.11

Version: 0.2.0-s_2.10 ( 5908c3 | zip | jar ) / Date: 2017-09-08 / License: MIT / Scala version: 2.10

Version: 0.1.0-s_2.11 ( c720c5 | zip | jar ) / Date: 2017-05-28 / License: MIT / Scala version: 2.11