Prep 1.1 release

README.md

@@ -39,7 +39,7 @@
     <tbody align="center">
         <tr>
             <td >2.3</td>
-            <td rowspan=3><a href="https://github.com/dotnet/spark/releases/tag/v1.0.0">v1.0.0</a></td>
+            <td rowspan=3><a href="https://github.com/dotnet/spark/releases/tag/v1.1.0">v1.1.0</a></td>
         </tr>
         <tr>
             <td>2.4*</td>

benchmark/scala/pom.xml

@@ -3,7 +3,7 @@
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.microsoft.spark</groupId>
   <artifactId>microsoft-spark-benchmark</artifactId>
-  <version>1.0.0</version>
+  <version>1.1.0</version>
   <inceptionYear>2019</inceptionYear>
   <properties>
     <encoding>UTF-8</encoding>

docs/release-notes/1.1/release-1.1.md

@@ -0,0 +1,208 @@
+# .NET for Apache Spark 1.1 Release Notes
+
+### New Features/Improvements
+
+* Support for Arrow 2.0 and GroupedMapUdf in Spark 3.0.0 ([#711](https://github.com/dotnet/spark/pull/711))
+* Use pattern matching in arrow test utils to improve readability ([#725](https://github.com/dotnet/spark/pull/725))
+* Check whether file is found before trying to dereference it ([#759](https://github.com/dotnet/spark/pull/759))
+* Ml/feature hasher has only internal contructors ([#761](https://github.com/dotnet/spark/pull/761))
+* Support for stop words removers ([#726](https://github.com/dotnet/spark/pull/726))
+* Support for adding NGram functionality ([#734](https://github.com/dotnet/spark/pull/734))
+* Add support for SQLTransformer ML feature ([#781](https://github.com/dotnet/spark/pull/781))
+* Add skeletal support for FileSystem extension ([#787](https://github.com/dotnet/spark/pull/787))
+* Using (processId, threadId) as key to mantain threadpool executor instead of only threadId ([#793](https://github.com/dotnet/spark/pull/793))
+* Support for Hyperspace 0.4.0 ([#815](https://github.com/dotnet/spark/pull/815))
+* Support for Delta Lake 0.8.0 ([#823](https://github.com/dotnet/spark/pull/823))
+* Add support for Spark 3.0.2 ([#833](https://github.com/dotnet/spark/pull/833))
+* Migrating master to main branch ([#847](https://github.com/dotnet/spark/pull/847), [#849](https://github.com/dotnet/spark/pull/849))
+* Add DOTNET_WORKER_<ver>_DIR environment variable ([#861](https://github.com/dotnet/spark/pull/861))
+* Add spark.dotnet.ignoreSparkPatchVersionCheck conf to ignore patch version in DotnetRunner ([#862](https://github.com/dotnet/spark/pull/862))
+
+### Bug Fixes
+
+* Fix signer information mismatch issue ([#752](https://github.com/dotnet/spark/pull/752))
+* Fix package-worker.ps1 to handle output path with ":" ([#742](https://github.com/dotnet/spark/pull/742))
+* Fixes for TimestampType and DateType conversion ([#765](https://github.com/dotnet/spark/pull/765))
+* Fix for using Broadcast variables in Databricks ([#766](https://github.com/dotnet/spark/pull/766))
+* Fix macOS Catalina Permissions ([#784](https://github.com/dotnet/spark/pull/784))
+* Fix for memory leak in JVMObjectTracker ([#801](https://github.com/dotnet/spark/pull/801))
+
+### Infrastructure / Documentation / Etc.
+
+* Fix flaky CallbackTests.TestCallbackHandlers Test ([#745](https://github.com/dotnet/spark/pull/745))
+* Run E2E tests on Linux in build pipeline and add Backward/Forward E2E tests ([#737](https://github.com/dotnet/spark/pull/737))
+* Add comments and cleanup azure pipeline ([#764](https://github.com/dotnet/spark/pull/764))
+* Update dotnet-interactive deprecated feed ([#807](https://github.com/dotnet/spark/pull/807), [#808](https://github.com/dotnet/spark/pull/808))
+* Remove unnecessary RestoreSources ([#812](https://github.com/dotnet/spark/pull/812))
+
+### Breaking Changes
+
+* None
+
+### Known Issues
+
+* Broadcast variables do not work with [dotnet-interactive](https://github.com/dotnet/interactive) ([#561](https://github.com/dotnet/spark/pull/561))
+* UDFs defined using class objects with closures does not work with [dotnet-interactive](https://github.com/dotnet/interactive) ([#619](https://github.com/dotnet/spark/pull/619))
+* In [dotnet-interactive](https://github.com/dotnet/interactive) blocking Spark methods that require external threads to unblock them does not work. ie `StreamingQuery.AwaitTermination` requires `StreamingQuery.Stop` to unblock ([#736](https://github.com/dotnet/spark/pull/736))
+* UDFs don't work in Linux with Spark 2.3.0 ([#753](https://github.com/dotnet/spark/issues/753))
+
+### Compatibility
+
+#### Backward compatibility
+
+The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker.
+
+<table>
+    <thead>
+        <tr>
+            <th>Oldest compatible Microsoft.Spark.Worker version</th>
+            <th>Incompatible features</th>
+        </tr>
+    </thead>
+    <tbody align="center">
+        <tr>
+            <td>v1.0.0</td>
+            <td>GroupedMap in Spark 3.0 is not compatible with Worker 1.0 <a href="https://github.com/dotnet/spark/pull/654">(#654)</a></td>
+        </tr>
+    </tbody>
+</table>
+
+#### Forward compatibility
+
+The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with.
+
+<table>
+    <thead>
+        <tr>
+            <th>Oldest compatible .NET for Apache Spark release version</th>
+        </tr>
+    </thead>
+    <tbody align="center">
+        <tr>
+            <td>v1.0.0</td>
+        </tr>
+    </tbody>
+</table>
+
+### Supported Spark Versions
+
+The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with:
+
+<table>
+    <thead>
+        <tr>
+            <th>Spark Version</th>
+            <th>microsoft-spark JAR</th>
+        </tr>
+    </thead>
+    <tbody align="center">
+        <tr>
+            <td>2.3.*</td>
+            <td>microsoft-spark-2-3_2.11-1.1.0.jar</td>
+        </tr>
+        <tr>
+            <td>2.4.0</td>
+            <td rowspan=7>microsoft-spark-2-4_2.11-1.1.0.jar</td>
+        </tr>
+        <tr>
+            <td>2.4.1</td>
+        </tr>
+        <tr>
+            <td>2.4.3</td>
+        </tr>
+        <tr>
+            <td>2.4.4</td>
+        </tr>
+        <tr>
+            <td>2.4.5</td>
+        </tr>
+        <tr>
+            <td>2.4.6</td>
+        </tr>
+        <tr>
+            <td>2.4.7</td>
+        </tr>
+        <tr>
+            <td>2.4.2</td>
+            <td><a href="https://github.com/dotnet/spark/issues/60">Not supported</a></td>
+        </tr>
+        <tr>
+            <td>3.0.0</td>
+            <td rowspan=3>microsoft-spark-3-0_2.12-1.1.0.jar</td>
+        </tr>
+        <tr>
+            <td>3.0.1</td>
+        </tr>
+		<tr>
+            <td>3.0.2</td>
+        </tr>
+    </tbody>
+</table>
+
+### Supported Delta Versions
+
+The following table outlines the supported Delta versions along with the Microsoft.Spark.Extensions version to use with:
+
+<table>
+    <thead>
+        <tr>
+            <th>Delta Version</th>
+            <th>Microsoft.Spark.Extensions.Delta</th>
+        </tr>
+    </thead>
+    <tbody align="center">
+        <tr>
+            <td>0.1.0</td>
+            <td rowspan=9>1.1.0</td>
+        </tr>
+        <tr>
+            <td>0.2.0</td>
+        </tr>
+        <tr>
+            <td>0.3.0</td>
+        </tr>
+        <tr>
+            <td>0.4.0</td>
+        </tr>
+        <tr>
+            <td>0.5.0</td>
+        </tr>
+        <tr>
+            <td>0.6.0</td>
+        </tr>
+        <tr>
+            <td>0.6.1</td>
+        </tr>
+        <tr>
+            <td>0.7.0</td>
+        </tr>
+		<tr>
+            <td>0.8.0</td>
+        </tr>
+    </tbody>
+</table>
+
+### Supported Hyperspace Versions
+
+The following table outlines the supported Hyperspace versions along with the Microsoft.Spark.Extensions version to use with:
+
+<table>
+    <thead>
+        <tr>
+            <th>Hyperspace Version</th>
+            <th>Microsoft.Spark.Extensions.Hyperspace</th>
+        </tr>
+    </thead>
+    <tbody align="center">
+        <tr>
+            <td>0.1.0</td>
+            <td rowspan=3>1.1.0</td>
+        </tr>
+        <tr>
+            <td>0.2.0</td>
+        </tr>
+		<tr>
+            <td>0.4.0</td>
+        </tr>
+    </tbody>
+</table>

eng/Versions.props

@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
   <PropertyGroup>
-    <VersionPrefix>1.0.0</VersionPrefix>
+    <VersionPrefix>1.1.0</VersionPrefix>
     <PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
   </PropertyGroup>
 </Project>

src/scala/pom.xml

@@ -7,7 +7,7 @@
   <version>${microsoft-spark.version}</version>
   <properties>
     <encoding>UTF-8</encoding>
-    <microsoft-spark.version>1.0.0</microsoft-spark.version>
+    <microsoft-spark.version>1.1.0</microsoft-spark.version>
   </properties>
 
   <modules>