spark-nlp (homepage)

Natural Language Processing Library for Apache Spark.

Spark NLP is a Natural Language Processing library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that scale easily in a distributed environment. Spark NLP comes with 1100+ pretrained pipelines and models in more than 192+ languages. It supports state-of-the-art transformers such as BERT, XLNet, ELMO, ALBERT, and Universal Sentence Encoder that can be used seamlessly in a cluster. It also offers Tokenization, Word Segmentation, Part-of-Speech Tagging, Named Entity Recognition, Dependency Parsing, Spell Checking, Multi-class Text Classification, Multi-class Sentiment Analysis, Machine Translation (+180 languages), Summarization and Question Answering (Google T5), and many more NLP tasks.

NOTE: We no longer publish Spark NLP in spark-packages, please check the main repo (https://github.com/JohnSnowLabs/spark-nlp) or Maven repository (https://mvnrepository.com/artifact/com.johnsnowlabs.nlp).


Tags

  • 2|pyspark
  • 2|NLP
  • 2|machine-learning
  • 2|nlu
  • 2|natural-language-processing
  • 2|natural-language-understanding
  • 2|spell-checker
  • 2|part-of-speech-tagger
  • 2|named-entity-recognition
  • 2|lemmatizer
  • 2|stemmer
  • 2|sentiment-analysis
  • 1|word-embeddings
  • 1|bert-embeddings
  • 1|sentence-embeddings
  • 1|chunk-embeddings

How to

Include this package in your Spark Applications using:

spark-shell, pyspark, or spark-submit

> $SPARK_HOME/bin/spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:3.0.1

sbt

In your sbt build file, add:

libraryDependencies += "com.johnsnowlabs.nlp" % "spark-nlp-gpu_2.12" % "3.0.1"

Maven

In your pom.xml, add:
<dependencies>
  <!-- list of dependencies -->
  <dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-gpu_2.12</artifactId>
    <version>3.0.1</version>
  </dependency>
</dependencies>

Releases

Version: 3.0.1 ( bec772 | zip | jar ) / Date: 2021-04-02 / License: Apache-2.0 / Scala version: 2.12

Version: 3.0.0-rc1 ( 7bd3c7 | zip | jar ) / Date: 2021-03-12 / License: Apache-2.0 / Scala version: 2.12

Version: 2.7.5 ( 30789f | zip | jar ) / Date: 2021-03-05 / License: Apache-2.0 / Scala version: 2.11

Version: 2.7.4 ( 3928a6 | zip | jar ) / Date: 2021-02-19 / License: Apache-2.0 / Scala version: 2.11

Version: 2.7.3 ( 1f4be7 | zip | jar ) / Date: 2021-02-05 / License: Apache-2.0 / Scala version: 2.11

Version: 2.7.2 ( 491d07 | zip | jar ) / Date: 2021-01-25 / License: Apache-2.0 / Scala version: 2.11

Version: 2.7.1 ( eb83f3 | zip | jar ) / Date: 2021-01-08 / License: Apache-2.0 / Scala version: 2.11

Version: 2.7.0 ( de80e2 | zip | jar ) / Date: 2021-01-04 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.5 ( ebd2d8 | zip | jar ) / Date: 2020-12-15 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.4 ( 7f8a96 | zip | jar ) / Date: 2020-11-24 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.3 ( 835d22 | zip | jar ) / Date: 2020-10-28 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.2 ( b0f048 | zip | jar ) / Date: 2020-10-01 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.1 ( fea85d | zip | jar ) / Date: 2020-09-11 / License: Apache-2.0 / Scala version: 2.11

Version: 2.6.0 ( 79914c | zip | jar ) / Date: 2020-09-02 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.5 ( 1afe6b | zip | jar ) / Date: 2020-08-04 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.4 ( c65388 | zip | jar ) / Date: 2020-07-20 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.3 ( 9f389c | zip | jar ) / Date: 2020-07-03 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.2 ( 917e68 | zip | jar ) / Date: 2020-06-11 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.1 ( 04c889 | zip | jar ) / Date: 2020-05-26 / License: Apache-2.0 / Scala version: 2.11

Version: 2.5.0 ( d9f5ad | zip | jar ) / Date: 2020-05-10 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.5 ( dc8b65 | zip | jar ) / Date: 2020-04-03 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.4 ( e7b220 | zip | jar ) / Date: 2020-04-03 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.3 ( b54bd6 | zip | jar ) / Date: 2020-03-21 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.2 ( 84c5cf | zip | jar ) / Date: 2020-03-21 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.1 ( ab7473 | zip | jar ) / Date: 2020-03-18 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.6 ( 6f20d2 | zip | jar ) / Date: 2020-02-08 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.0 ( db91af | zip | jar ) / Date: 2020-02-05 / License: Apache-2.0 / Scala version: 2.11

Version: 2.4.0-rc1 ( 45d3b7 | zip | jar ) / Date: 2020-02-05 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.5 ( 319786 | zip ) / Date: 2020-02-05 / License: Apache-2.0

Version: 2.3.4 ( f7d14f | zip | jar ) / Date: 2019-11-27 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.3 ( 290bd3 | zip | jar ) / Date: 2019-11-21 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.2 ( b089d5 | zip | jar ) / Date: 2019-11-08 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.1 ( 70e704 | zip | jar ) / Date: 2019-10-30 / License: Apache-2.0 / Scala version: 2.11

Version: 2.3.0 ( e4197c | zip | jar ) / Date: 2019-10-26 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.2 ( 75987c | zip | jar ) / Date: 2019-09-26 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.1 ( e1e84d | zip | jar ) / Date: 2019-08-28 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.0 ( 65f323 | zip | jar ) / Date: 2019-08-23 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.0-rc3 ( 5957dc | zip | jar ) / Date: 2019-08-21 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.0-rc2 ( 1b51e0 | zip | jar ) / Date: 2019-08-18 / License: Apache-2.0 / Scala version: 2.11

Version: 2.1.1 ( 3c9559 | zip | jar ) / Date: 2019-08-18 / License: Apache-2.0 / Scala version: 2.11

Version: 2.2.0-rc1 ( db5762 | zip | jar ) / Date: 2019-08-16 / License: Apache-2.0 / Scala version: 2.11

Version: 2.1.0 ( 6b841b | zip | jar ) / Date: 2019-07-14 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.9 ( 310ebd | zip | jar ) / Date: 2019-07-02 / License: Apache-2.0 / Scala version: 2.11

Version: 2.1.0-rc2 ( 594782 | zip | jar ) / Date: 2019-06-30 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.8 ( 624035 | zip | jar ) / Date: 2019-06-05 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.7 ( 7e14d2 | zip | jar ) / Date: 2019-06-02 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.6 ( 631748 | zip | jar ) / Date: 2019-05-30 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.5 ( 4e1c94 | zip | jar ) / Date: 2019-05-29 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.4 ( 10a228 | zip | jar ) / Date: 2019-05-16 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.3 ( cf6c7a | zip | jar ) / Date: 2019-04-30 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.2 ( b44156 | zip | jar ) / Date: 2019-04-29 / License: Apache-2.0 / Scala version: 2.11

Version: 1.8.4 ( 54bcee | zip | jar ) / Date: 2019-03-31 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.1 ( 9834a8 | zip | jar ) / Date: 2019-03-24 / License: Apache-2.0 / Scala version: 2.11

Version: 2.0.0 ( d78f04 | zip | jar ) / Date: 2019-03-20 / License: Apache-2.0 / Scala version: 2.11

Version: 1.8.3 ( fa7349 | zip | jar ) / Date: 2019-02-24 / License: Apache-2.0 / Scala version: 2.11

Version: 1.8.2 ( e09de9 | zip | jar ) / Date: 2019-02-08 / License: Apache-2.0 / Scala version: 2.11

Version: 1.8.1 ( a37919 | zip | jar ) / Date: 2019-01-28 / License: Apache-2.0 / Scala version: 2.11

Version: 1.8.0 ( 044474 | zip | jar ) / Date: 2018-12-23 / License: Apache-2.0 / Scala version: 2.11

Version: 1.7.3 ( 531909 | zip | jar ) / Date: 2018-11-12 / License: Apache-2.0 / Scala version: 2.11

Version: 1.7.2 ( 6c2bef | zip | jar ) / Date: 2018-10-20 / License: Apache-2.0 / Scala version: 2.11

Version: 1.7.1 ( 1fd316 | zip | jar ) / Date: 2018-10-19 / License: Apache-2.0 / Scala version: 2.11

Version: 1.7.0 ( 07eb28 | zip | jar ) / Date: 2018-10-16 / License: Apache-2.0 / Scala version: 2.11

Version: 1.6.3 ( 48556b | zip | jar ) / Date: 2018-09-17 / License: Apache-2.0 / Scala version: 2.11

Version: 1.6.2 ( f7b290 | zip | jar ) / Date: 2018-08-20 / License: Apache-2.0 / Scala version: 2.11

Version: 1.6.1 ( dfdfd8 | zip | jar ) / Date: 2018-08-10 / License: Apache-2.0 / Scala version: 2.11

Version: 1.6.0 ( a85111 | zip | jar ) / Date: 2018-07-07 / License: Apache-2.0 / Scala version: 2.11

Version: 1.5.4 ( 458e8c | zip | jar ) / Date: 2018-05-19 / License: Apache-2.0 / Scala version: 2.11

Version: 1.5.3 ( 750b4a | zip | jar ) / Date: 2018-05-03 / License: Apache-2.0 / Scala version: 2.11

Version: 1.5.2 ( 52e243 | zip | jar ) / Date: 2018-04-30 / License: Apache-2.0 / Scala version: 2.11

Version: 1.5.1 ( 3666db | zip | jar ) / Date: 2018-04-16 / License: Apache-2.0 / Scala version: 2.11

Version: 1.5.0 ( 5b2ecb | zip | jar ) / Date: 2018-03-30 / License: Apache-2.0 / Scala version: 2.11

Version: 1.4.2 ( 7ffd0a | zip | jar ) / Date: 2018-03-12 / License: Apache-2.0 / Scala version: 2.11

Version: 1.4.1 ( 1244e9 | zip | jar ) / Date: 2018-02-26 / License: Apache-2.0 / Scala version: 2.11

Version: 1.4.0 ( f1933e | zip | jar ) / Date: 2018-02-08 / License: Apache-2.0 / Scala version: 2.11

Version: 1.3.0 ( fcd5cc | zip | jar ) / Date: 2018-01-27 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.6 ( 59e9ff | zip | jar ) / Date: 2018-01-12 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.5 ( a6a417 | zip | jar ) / Date: 2018-01-10 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.4 ( fce846 | zip | jar ) / Date: 2017-12-28 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.3 ( 3f9ca3 | zip | jar ) / Date: 2017-10-31 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.2 ( 7746a5 | zip | jar ) / Date: 2017-10-21 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.1 ( e02a45 | zip | jar ) / Date: 2017-10-19 / License: Apache-2.0 / Scala version: 2.11

Version: 1.2.0 ( e00c3f | zip | jar ) / Date: 2017-10-17 / License: Apache-2.0 / Scala version: 2.11

Version: 1.1.0 ( 05992d | zip | jar ) / Date: 2017-10-11 / License: Apache-2.0 / Scala version: 2.11