A community index of third-party packages for Apache Spark.

Showing packages 1 - 50 out of 517

Next >

spark-als

Another, hopefully better, implementation of ALS on Spark (already merged into MLlib)

@mengxr / Latest release: 0.1.0 (2014-11-27) / BSD 3-Clause / (1)

  • 3|ml
  • 2|mllib
  • 2|recommendation

mllib-grid-search

An example project for doing grid search in MLlib

@spark-ml / Latest release: 0.0.1 (2014-11-27) / BSD 3-Clause / (2)

  • 1|ml
  • 1|example
  • 1|examples

spark-avro

Integration utilities for using Spark with Apache Avro data

@databricks / Latest release: 4.0.0-s_2.11 (2017-10-30) / Apache-2.0 / (13)

  • 6|sql
  • 4|input
  • 4|avro

spark-redshift

Redshift Data Source for Apache Spark

@databricks / Latest release: 3.0.0-preview1 (2016-11-01) / Apache-2.0 / (3)

  • 2|sql
  • 2|data source
  • 2|redshift

kafka-spark-consumer

High Performance Kafka Consumer for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker

@dibbhatt / Latest release: 2.1.0 (2019-08-28) / Apache-2.0 / (7)

  • 4|streaming
  • 3|kafka

thunder

Large-scale neural data analysis with Spark

@freeman-lab / Latest release: 0.4.1 (2014-11-27) / BSD 3-Clause / (6)

  • 3|neuroscience
  • 2|python
  • 2|machine learning

spark_azure

Spark launch script for Microsoft Azure

@sigmoidanalytics / No release yet / (10)

  • 1|Azure
  • 1|spark
  • 1|Microsoft

spork

Pig on Apache Spark

@sigmoidanalytics / No release yet / (9)

  • 1|streaming
  • 1|spark
  • 1|pig

spark_gce

Spark launch script for Google Compute Engine

@sigmoidanalytics / No release yet / (8)

  • 1|Google Compute Engine
  • 1|spark
  • 1|GCE

spark-jobserver

REST job server for Spark

@spark-jobserver / No release yet / (3)

  • 1|application
  • 1|REST
  • 1|Mesos

GMM

Gaussian Mixture Model Implementation in Pyspark

@FlytxtRnD / Latest release: 0.1 (2015-04-07) / EPL-1.0 / (5)

  • 1|python
  • 1|mllib

spark-csv

Spark SQL CSV data source

@databricks / Latest release: 1.5.0-s_2.11 (2016-09-07) / Apache-2.0 / (10)

  • 4|csv
  • 3|sql
  • 2|DataSource

spark-indexedrdd

An efficient updatable key-value store for Apache Spark

@amplab / Latest release: 0.4.0 (2017-01-11) / Apache-2.0 / (1)

  • 2|core
  • 2|kv
  • 1|anothertag

spark-perf

Performance tests for Spark

@databricks / No release yet / (1)

killrweather

KillrWeather is a reference application (in progress) showing how to easily leverage and integrate Apache Spark, Apache Cassandra, and Apache Kafka for fast, streaming computations on time series data in asynchronous Akka event-driven environments.

@killrweather / No release yet / (1)

  • 1|streaming

spark-hash

Locality Sensitive Hashing for Apache Spark

@mrsqueeze / No release yet / (0)

  • 1|mllib
  • 1|lsh

zeppelin

Zeppelin, a web-based notebook that enables interactive data analytics.

@NFLabs / No release yet / (3)

  • 1|Applications
  • 1|notebook
  • 1|interactive

spark-hbase

Integration utilities for using Spark with Apache HBase data

@haosdent / No release yet / (1)

  • 1|hbase

spark-dbf

Spark SQL DBF Library

@mraad / No release yet / (0)

  • 1|sql

spark_hbase

The example in Scala of reading data saved in hbase by Spark and the example of converter for python

@GenTang / No release yet / (3)

  • 1|python
  • 1|hbase

sparkling

A Clojure library for Apache Spark: fast, fully-features, and developer friendly

@gorillalabs / Latest release: 1.0.0 (2014-12-31) / EPL-1.0 / (3)

  • 2|clojure
  • 1|API

spark-kernel

A kernel that enables applications to interact with Apache Spark.

@ibm-et / No release yet / (0)

  • 1|ipython
  • 1|foundation
  • 1|interactive

pyspark-pictures

Learn the pySpark API through pictures and simple examples

@jkthompson / No release yet / (0)

  • 2|tutorials
  • 2|examples

deep-spark

Connecting Apache Spark with different data stores

@Stratio / Latest release: 0.7.0-RC1 (2015-01-14) / Apache-2.0 / (20)

  • 6|database
  • 6|mongo
  • 6|cassandra

streaming-cep-engine

Streaming CEP Engine Powered by Spark Streaming & Siddhi

@Stratio / Latest release: 0.6.2 (2015-01-14) / Apache-2.0 / (19)

  • 5|spark streaming
  • 5|cep
  • 4|complex event processing

generalized-kmeans-clustering

This project generalizes the Spark MLLIB K-Means clusterer to support arbitrary distance functions

@derrickburns / No release yet / (3)

  • 1|clustering
  • 1|mllib
  • 1|machine learning

sparkling-water

Sparkling Water provides H2O algorithms inside Spark cluster

@h2oai / Latest release: 1.4.3 (2015-07-06) / Apache-2.0 / (2)

  • 1|h2o
  • 1|algorithms
  • 1|machine learning

spark-ml-streaming

Visualize streaming machine learning in Spark

@freeman-lab / No release yet / (1)

  • 1|streaming
  • 1|machine learning
  • 1|visualization

spark

Apache Camel Streaming Consumer

@synsys / Latest release: 1.0.0 (2015-01-26) / Apache-2.0 / (0)

  • 1|streaming
  • 1|consumer
  • 1|camel

spark-hbase-connector

Connect Spark to HBase for reading and writing data with ease

@nerdammer / Latest release: 1.0.3 (2016-04-20) / Apache-2.0 / (3)

  • 1|streaming
  • 1|hbase
  • 1|library

spark-testing-base

Base classes to use when writing tests with Spark

@holdenk / Latest release: 2.2.2_0.11.0 (2018-12-23) / Apache-2.0 / (10)

  • 3|testing
  • 1|streaming
  • 1|tools

pyspark-csv

An external PySpark module that works like R's read.csv or Panda's read_csv, with automatic type inference and null value handling. Parses csv data into SchemaRDD. No installation required, simply include pyspark_csv.py via SparkContext.

@seahboonsiew / No release yet / (1)

  • 2|python
  • 2|csv
  • 1|sql

spark-mongodb

MongoDB data source for Spark SQL

@Stratio / Latest release: 0.12.0 (2016-08-31) / Apache-2.0 / (14)

  • 5|MongoDB
  • 5|Spark SQL
  • 2|sql

pyspark-cassandra

PySpark Cassandra brings back the fun in working with Cassandra data in PySpark.

@TargetHolding / Latest release: 0.3.5 (2016-03-30) / Apache-2.0 / (1)

  • 1|python
  • 1|spark
  • 1|sql

spark_dbscan

DBSCAN clustering algorithm on top of Apache Spark

@alitouka / No release yet / (0)

  • 1|clustering
  • 1|ml
  • 1|dbscan

demo-scala-python

A Spark Package Template

@brkyvz / Latest release: 1.2-s_2.10 (2016-05-25) / Apache-2.0 / (1)

  • 1|python
  • 1|demo
  • 1|template

sbt-spark-package

Sbt plugin for Spark packages

@databricks / Latest release: 0.2.4 (2016-07-15) / Apache-2.0 / (3)

  • 1|tools
  • 1|sbt

spark-scalding

Use Cascading Taps and Scalding DSL with Spark — Edit

@tresata / Latest release: 0.5.0-s_2.10 (2015-11-13) / Apache-2.0 / (0)

spark-sorted

Secondary sort and streaming reduce for Spark

@tresata / Latest release: 0.4.0-s_2.11 (2015-11-03) / Apache-2.0 / (0)

  • 1|core

spark-kafka

Low level integration of Spark and Kafka

@tresata / Latest release: 0.6.0-s_2.10 (2015-11-13) / Apache-2.0 / (0)

  • 1|streaming

spark-cassandra-connector

Connects Spark to Cassandra

@datastax / Latest release: 2.4.0-s_2.11 (2018-11-29) / Apache-2.0 / (14)

  • 3|spark
  • 3|cassandra
  • 2|nosql

spark-package-cmd-tool

A command line tool for Spark packages

@databricks / Latest release: 0.3.0 (2015-03-17) / Apache-2.0 / (1)

  • 1|tools

spawncamping-dds

Data-Driven Spark allows quick data exploration based on Apache Spark

@FRosner / No release yet / (0)

spark-power-bi

Power BI API adapter for Apache Spark

@granturing / Latest release: 1.5.0_0.0.7 (2015-09-13) / Apache-2.0 / (0)

  • 2|streaming
  • 1|sql
  • 1|realtime

meetup-stream

Spark Streaming, Machine Learning and meetup.com streaming API.

@actions / No release yet / (1)

  • 1|ml
  • 1|example
  • 1|streaming

demo-python-sp

Pure python package used for testing Spark Packages

@brkyvz / Latest release: 0.4.2 (2016-02-14) / Apache-2.0 / (0)

spark-mrmr-feature-selection

Feature selection based on information gain: maximum relevancy minimum redundancy

@wxhC3SC6OPm8M1HXboMy / No release yet / (0)

  • 1|mllib

spark-csv2sql

Hand routine to import csv files as tables in spark sql

@wxhC3SC6OPm8M1HXboMy / No release yet / (0)

spark-sequoiadb

Spark connector for SequoiaDB

@SequoiaDB / Latest release: 1.12-s_2.11 (2015-03-30) / Apache-2.0 / (2)

  • 2|sequoiadb
  • 2|nosql
  • 2|sql

spark-notebook

Use Apache Spark straight from the Browser

@andypetrella / Latest release: v0.4.0 (2015-03-29) / Apache-2.0 / (2)

  • 1|notebook
  • 1|charts
  • 1|interactive

Next >