sparkhpc (homepage)

launching and controlling spark on hpc clusters made easy

@rokroskar / (0)

The sparkhpc package greatly simplifies deploying and managing Apache Spark clusters on large computing clusters. It provides a command-line tool for submitting and monitoring standalone spark clusters running on an HPC resource, as well as a python package to dynamically deploy spark from within python applications.

Tags

  • 1|pyspark
  • 1|hpc
  • 1|high-performance computing

How to

This package doesn't have any releases published in the Spark Packages repo, or with maven coordinates supplied. You may have to build this package from source, or it may simply be a script. To use this Spark Package, please follow the instructions in the README.

Releases

No releases yet.