ParallelTool (homepage)

Tool design to speed up spark applications

Tool design to speed up Spark applications and increase the intensity of use of the cluster.
The target of this tool is optimizing the use of the cluster, increasing the parallelization that spark does.

Tags

  • 1|scala
  • 1|tool
  • 1|parallel
  • 1|speed
  • 1|futures
  • 1|promises
  • 1|multithreading
  • 1|reflection

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages com.github.marino-serna:parallel-tool:1.0.1-00

sbt

In your sbt build file, add:

libraryDependencies += "com.github.marino-serna" % "parallel-tool" % "1.0.1-00"

Maven

In your pom.xml, add: 
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>com.github.marino-serna</groupId>
    <artifactId>parallel-tool</artifactId>
    <version>1.0.1-00</version>
  </dependency>
</dependencies>

Releases

Version: 1.0.1-00 ( 656db0 | zip | jar ) / Date: 2018-07-22 / License: Apache-2.0 / Scala version: 2.11

Version: 1.0.0-02 ( efcc03 | zip | jar ) / Date: 2018-07-17 / License: Apache-2.0 / Scala version: 2.11