Skip to content

ENH: Parallelization support for pairwise correlation #61737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

gangula-karthik
Copy link

@gangula-karthik gangula-karthik commented Jun 29, 2025

Used cython.parallel to parallelize the nancorr function.

@gangula-karthik
Copy link
Author

gangula-karthik commented Jul 6, 2025

Looks like webAssembly/emscripten environments don't have OpenMP support. To get around this Im thinking of modifying the menson.build and setup.py files to conditionally include OpenMP compiler and linker flags only when not building for WebAssembly/Emscripten. (WIP)

@rhshadrach
Copy link
Member

I'm positive on adding parallelization as is being done here, but negative without the proper framework for use across pandas. See #43313.

@rhshadrach rhshadrach added Multithreading Parallelism in pandas cov/corr labels Jul 19, 2025
@mroeschke
Copy link
Member

Thanks for the PR, but as mentioned the project needs to coordinate & decide a common pattern for parallelism before implementing it in any specific method yet, so closing as that discussion needs resolution first

@mroeschke mroeschke closed this Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: parallelize DataFrame.corr
3 participants