Skip to content

Added Pivot method to RelationalGroupedDataset #598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 22, 2020
Merged

Added Pivot method to RelationalGroupedDataset #598

merged 6 commits into from
Aug 22, 2020

Conversation

moredatapls
Copy link
Contributor

@moredatapls moredatapls commented Jul 22, 2020

Added support for RelationalGroupedDataset.Pivot (excluding the overloads with values).

Fixes #591

So we can do the best job, please check:

  • There's a descriptive title that will make sense to other developers some time from now.
  • There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
  • Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
  • You have included any necessary tests in the same PR.

@dnfadmin
Copy link

dnfadmin commented Jul 22, 2020

CLA assistant check
All CLA requirements met.

@imback82
Copy link
Contributor

Thanks @moredatapls! Is this ready for PR? I see the PR as "Draft".

@moredatapls
Copy link
Contributor Author

@imback82 It's still a draft because I wanted to run the tests locally but I'm getting a NullReferenceException when I try to run them, see the strack trace below. I probably didn't configure my environment correctly, is there any guide that describes the local setup? Built the jars and the worker and configured the DOTNET_WORKER_DIR to point to the publish directory.

System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Spark.E2ETest.SparkFixture.<>c__DisplayClass10_0.<.ctor>b__0(Object sender, DataReceivedEventArgs arguments) in /Users/helge.bruegner/Projects/spark/src/csharp/Microsoft.Spark.E2ETest/SparkFixture.cs:line 78
   at System.Diagnostics.Process.OutputReadNotifyUser(String data)
   at System.Diagnostics.AsyncStreamReader.FlushMessageQueue(Boolean rethrowInNewThread)
--- End of stack trace from previous location where exception was thrown ---
   at System.Diagnostics.AsyncStreamReader.<>c.<FlushMessageQueue>b__18_0(Object edi)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

@moredatapls moredatapls changed the title Added Pivot method to RelationalGroupedDataset [WIP] Added Pivot method to RelationalGroupedDataset Jul 23, 2020
suhsteve
suhsteve previously approved these changes Aug 21, 2020
Copy link
Member

@suhsteve suhsteve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, but LGTM. Thanks @moredatapls !

@suhsteve suhsteve changed the title [WIP] Added Pivot method to RelationalGroupedDataset Added Pivot method to RelationalGroupedDataset Aug 21, 2020
@moredatapls moredatapls marked this pull request as ready for review August 21, 2020 17:16
Niharikadutta
Niharikadutta previously approved these changes Aug 21, 2020
Copy link
Collaborator

@Niharikadutta Niharikadutta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well, thanks @moredatapls !

Copy link
Contributor

@imback82 imback82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for nit comments

Copy link
Contributor

@imback82 imback82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @moredatapls!

@imback82 imback82 merged commit 6bdc9be into dotnet:master Aug 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE REQUEST]: Support for RelationalGroupedDataset.Pivot
5 participants