Update dl4j versions to stable, remove old repositories used for testing

Author: agibsonccc

README.md

@@ -19,7 +19,7 @@
 The **Eclipse Deeplearning4J** (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application. This means starting with the raw data, loading and preprocessing it from wherever and whatever format it is in to building and tuning a wide variety of simple and complex deep learning networks.
 
 The DL4J stack comprises of:
-- **DL4J**: High level API to build MultiLayerNetworks and ComputationGraphs with a variety of layers, including custom ones. Supports importing Keras models from h5, including tf.keras models (as of 1.0.0-beta7) and also supports distributed training on Apache Spark
+- **DL4J**: High level API to build MultiLayerNetworks and ComputationGraphs with a variety of layers, including custom ones. Supports importing Keras models from h5, including tf.keras models (as of 1.0.0-M1) and also supports distributed training on Apache Spark
 - **ND4J**: General purpose linear algebra library with over 500 mathematical, linear algebra and deep learning operations. ND4J is based on the highly-optimized C++ codebase LibND4J that provides CPU (AVX2/512) and GPU (CUDA) support and acceleration by libraries such as OpenBLAS, OneDNN (MKL-DNN), cuDNN, cuBLAS, etc
 - **SameDiff** : Part of the ND4J library, SameDiff is our automatic differentiation / deep learning framework. SameDiff uses a graph-based (define then run) approach, similar to TensorFlow graph mode. Eager graph (TensorFlow 2.x eager/PyTorch) graph execution is planned. SameDiff supports importing TensorFlow frozen model format .pb (protobuf) models. Import for ONNX, TensorFlow SavedModel and Keras models are planned. Deeplearning4j also has full SameDiff support for easily writing custom layers and loss functions.
 - **DataVec**: ETL for machine learning data in a wide variety of formats and files (HDFS, Spark, Images, Video, Audio, CSV, Excel etc)
@@ -35,6 +35,17 @@ Users can also refer to the [simple sample project provided](./mvn-project-templ
 
 Build tools are considered standard software engineering best practice. Besides this the complexities posed by the projects in the DL4J ecosystem make dependencies too difficult to manage manually. All the projects in the DL4J ecosystem can be used with other build tools like Gradle, SBT etc. More information on that can be found [here](https://deeplearning4j.konduit.ai/config/buildtools).
 
+## Support
+
+For help with the examples, please go to our [support forum](https://community.konduit.ai/)
+
+Note for users of 1.0.0-beta7 and prior, some examples and modules have been removed to reflect
+changes in the framework's direction. Please see and comment on our post [here](https://community.konduit.ai/t/upcoming-removal-of-modules-and-roadmap-changes/1240)
+
+If you would like a workaround for something you may be missing,
+please feel free to post on the forums, and we will do what we can to help you.
+
+
 ## Example Content
 Projects are based on what functionality the included examples demonstrate to the user and not necessarily which library in the DL4J stack the functionality lives in.
 
@@ -64,9 +75,6 @@ This project contains a set of examples that demonstrate how raw data in various
 - [nd4j-ndarray-examples](nd4j-ndarray-examples/README.md)
 This project contains a set of examples that demonstrate how to manipulate NDArrays. The functionality of ND4J demonstrated here can be likened to NumPy.
 
-- [arbiter-examples](arbiter-examples/README.md)
-This project contains a set of examples that demonstrate usage of the Arbiter library for hyperparameter tuning of Deeplearning4J neural networks.
-
 - [rl4j-examples](rl4j-examples/README.md)
 This project contains examples of using RL4J, the reinforcement learning library in DL4J.
 

android-examples/README.md

@@ -8,14 +8,6 @@ The build configuration for Android can be found in [app/build.gradle](./app/bui
 
 [Go back](../README.md) to the main repository page to explore other features/functionality of the **Eclipse Deeplearning4J** ecosystem. File an issue [here](https://github.com/eclipse/deeplearning4j-examples/issues) to request new features.
 
-# Known Issues
-Due to an unfortunately timed change, the 1.0.0-beta7 release doesn't work well with most Android devices. For this reason, this example project uses the SNAPSHOT version.
-
-Again unfortunately, gradle has issues retrieving SNAPSHOT versions when combined with classifiers, which are used here to retrieve only the android specific backend dependencies.
-
-If you want to run this example locally, you will therefore need to have Maven installed in order to download those specific dependencies.
-
-Once you have installed Maven, you will need to run the following commands from the command line to download the correct android backend files:
 
 ```
 mvn dependency:get -DremoteRepositories=snapshots::::https://oss.sonatype.org/content/repositories/snapshots -Dartifact=org.nd4j:nd4j-native:1.0.0-SNAPSHOT:jar:android-arm

android-examples/app/build.gradle

@@ -27,7 +27,7 @@ android {
 }
 
 dependencies {
-    def dl4jVersion = '1.0.0-SNAPSHOT'
+    def dl4jVersion = '1.0.0-M1'
     def openblasVersion = '0.3.10-1.5.5'
     def opencvVersion = '4.5.1-1.5.5'
     def leptonicaVersion = '1.80.0-1.5.5'

data-pipeline-examples/pom.xml

@@ -23,7 +23,7 @@
 
     <groupId>org.deeplearning4j</groupId>
     <artifactId>data-pipeline-examples</artifactId>
-    <version>1.0.0-beta7</version>
+    <version>1.0.0-M1</version>
     <name>Building a data pipeline prior to modeling</name>
     <description>Loading raw data and processing it before training</description>
 
@@ -42,6 +42,22 @@
         <spark.version>2.4.3</spark.version>
     </properties>
 
+    <repositories>
+        <repository>
+            <id>sonatype-nexus-snapshots</id>
+            <name>Sonatype Nexus Snapshots</name>
+            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
+            <releases>
+                <enabled>false</enabled>
+            </releases>
+            <snapshots>
+                <enabled>true</enabled>
+                <updatePolicy>daily</updatePolicy>  <!-- Optional, update daily -->
+            </snapshots>
+        </repository>
+    </repositories>
+
+
     <dependencies>
         <dependency>
             <groupId>org.nd4j</groupId>
@@ -124,6 +140,20 @@
     <!-- Maven Enforcer: Ensures user has an up to date version of Maven before building -->
     <build>
         <plugins>
+
+            <plugin>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <version>3.0.0-M5</version>
+                <inherited>true</inherited>
+                <dependencies>
+                    <dependency>
+                        <groupId>org.apache.maven.surefire</groupId>
+                        <artifactId>surefire-junit-platform</artifactId>
+                        <version>3.0.0-M5</version>
+                    </dependency>
+                </dependencies>
+            </plugin>
+
             <plugin>
                 <artifactId>maven-enforcer-plugin</artifactId>
                 <version>1.0.1</version>

dl4j-distributed-training-examples/pom.xml

@@ -23,7 +23,7 @@
 
     <groupId>org.deeplearning4j</groupId>
     <artifactId>dl4j-distributed-training-examples</artifactId>
-    <version>1.0.0-beta7</version>
+    <version>1.0.0-M1</version>
     <name>Introduction to Distributed Training with DL4J</name>
     <description>A set of examples introducing distributed training with the DL4J framework</description>
 
@@ -52,14 +52,6 @@
     </properties>
 
     <repositories>
-        <repository>
-            <id>orgdeeplearning4j-1165</id>
-            <url>https://oss.sonatype.org/content/repositories/orgdeeplearning4j-1165</url>
-            <releases>
-                <enabled>true</enabled>
-            </releases>
-        </repository>
-
         <repository>
             <id>sonatype-nexus-snapshots</id>
             <name>Sonatype Nexus Snapshots</name>
@@ -75,6 +67,7 @@
     </repositories>
 
 
+
     <dependencies>
         <dependency>
             <groupId>org.apache.spark</groupId>
@@ -150,6 +143,19 @@
 
     <build>
         <plugins>
+            <plugin>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <version>3.0.0-M5</version>
+                <inherited>true</inherited>
+                <dependencies>
+                    <dependency>
+                        <groupId>org.apache.maven.surefire</groupId>
+                        <artifactId>surefire-junit-platform</artifactId>
+                        <version>3.0.0-M5</version>
+                    </dependency>
+                </dependencies>
+            </plugin>
+
             <plugin>
                 <groupId>org.codehaus.mojo</groupId>
                 <artifactId>exec-maven-plugin</artifactId>

dl4j-distributed-training-examples/scripts/patentExamplePreproc.sh

@@ -32,7 +32,7 @@ MINIBATCH=32                                    #Minibatch size for preprocessed
 
 #Other variables. Don't modify these
 SCRIPTDIR=$(dirname "$0")
-JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-beta7-bin.jar
+JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-M1-bin.jar
 AZURE_ACCT=fs.azure.account.key.${AZURE_STORAGE_ACCT}.blob.core.windows.net
 
 

dl4j-distributed-training-examples/scripts/patentExampleTrain.sh

@@ -40,7 +40,7 @@ AERON_BUFFER=33554432
 
 #Other variables. Don't modify these
 SCRIPTDIR=$(dirname "$0")
-JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-beta7-bin.jar
+JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-M1-bin.jar
 AZURE_ACCT=fs.azure.account.key.${AZURE_STORAGE_ACCT}.blob.core.windows.net
 
 

dl4j-distributed-training-examples/scripts/tinyImagenetTrain.sh

@@ -32,7 +32,7 @@ EXECUTOR_MEMORY=12G
 MASTER_PORT=7077                                # Port for the spark master. Default is 7077
 MINIBATCH=32                                    # Minibatch size for preprocessed datasets
 SCRIPTDIR=$(dirname "$0")
-JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-beta7-bin.jar
+JARFILE=${SCRIPTDIR}/../target/dl4j-spark-1.0.0-M1-bin.jar
 
 #Memory
 OFFHEAP_MEM_JAVACPP=20G         # Maximum amount of off-heap memory

dl4j-examples/README.md

@@ -6,6 +6,8 @@ The pom file in this project can be used as a template for a user's own project.
 
 [Go back](../README.md) to the main repository page to explore other features/functionality of the **Eclipse Deeplearning4J** ecosystem. File an issue [here](https://github.com/eclipse/deeplearning4j-examples/issues) to request new features.
 
+
+
 ### QUICKSTART
 
 #### Modeling Examples

dl4j-examples/pom.xml

@@ -31,7 +31,7 @@
         <dl4j-master.version>1.0.0-M1</dl4j-master.version>
         <!-- Change the nd4j.backend property to nd4j-cuda-X-platform to use CUDA GPUs -->
         <!-- <nd4j.backend>nd4j-cuda-10.2-platform</nd4j.backend> -->
-        <nd4j.backend>nd4j-native</nd4j.backend>
+        <nd4j.backend>nd4j-cuda-11.2</nd4j.backend>
         <java.version>1.8</java.version>
         <maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
         <maven.minimum.version>3.3.1</maven.minimum.version>
@@ -48,15 +48,6 @@
 
 
     <repositories>
-        <repository>
-            <id>orgdeeplearning4j-1165</id>
-            <url>https://oss.sonatype.org/content/repositories/orgdeeplearning4j-1165</url>
-            <releases>
-                <enabled>true</enabled>
-            </releases>
-        </repository>
-
-
         <repository>
             <id>sonatype-nexus-snapshots</id>
             <name>Sonatype Nexus Snapshots</name>
@@ -77,6 +68,8 @@
             <artifactId>${nd4j.backend}</artifactId>
             <version>${dl4j-master.version}</version>
         </dependency>
+
+
         <dependency>
             <groupId>org.datavec</groupId>
             <artifactId>datavec-api</artifactId>
@@ -162,17 +155,26 @@
             <scope>test</scope>
         </dependency>
 
-        <dependency>
-            <groupId>org.bytedeco</groupId>
-            <artifactId>cuda-platform-redist</artifactId>
-            <version>11.2-8.1-1.5.5</version>
-        </dependency>
 
 
     </dependencies>
     <!-- Maven Enforcer: Ensures user has an up to date version of Maven before building -->
     <build>
         <plugins>
+
+            <plugin>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <version>3.0.0-M5</version>
+                <inherited>true</inherited>
+                <dependencies>
+                    <dependency>
+                        <groupId>org.apache.maven.surefire</groupId>
+                        <artifactId>surefire-junit-platform</artifactId>
+                        <version>3.0.0-M5</version>
+                    </dependency>
+                </dependencies>
+            </plugin>
+
             <plugin>
                 <artifactId>maven-enforcer-plugin</artifactId>
                 <version>1.0.1</version>