Update to v0.5.0

Author: hochunlin

Project.toml

@@ -2,7 +2,7 @@ name = "MESTI"
 uuid = "8d7f31fa-9986-4e1d-8c81-72752476e54d"
 license = "GPL-3.0"
 authors = ["Ho-Chun Lin<hochunlin0508@gmail.com>, Shiyu Li<lishiyu@usc.edu>, Zeyu Wang<wangzeyu@usc.edu>, and Chia Wei Hsu<cwhsu@usc.edu>"]
-version = "0.4.4"
+version = "0.5.0"
 
 [deps]
 GeometryPrimitives = "17051e67-205e-509e-8301-03b320b998e6"

README.md

@@ -66,9 +66,9 @@ export PATH=".../julia-1.9.3/bin/"
 
 where  <code>...</code> is the path to your Julia.
 
-Before installing MESTI.jl, the user first needs to install the parallel version of the sparse linear solver [MUMPS](https://mumps-solver.org/index.php). Without MUMPS, MESTI.jl can still run but cannot use the APF method and will only use a conventional method with the built-in linear solver, which can be orders of magnitude slower and uses much more memory (especially in 3D and for large 2D systems). See this [MUMPS installation](./mumps) page for steps to install MUMPS. For this MESTI (v0.4.4) version, it is compatible with MUMPS versions between v5.3.3 and v5.6.2. It is not compatible with MUMPS versions v5.7.0 or v5.7.1. 
+Before installing MESTI.jl, the user first needs to install the parallel version of the sparse linear solver [MUMPS](https://mumps-solver.org/index.php). Without MUMPS, MESTI.jl can still run but cannot use the APF method and will only use a conventional method with the built-in linear solver, which can be orders of magnitude slower and uses much more memory (especially in 3D and for large 2D systems). See this [MUMPS installation](./mumps) page for steps to install MUMPS. For this MESTI (v0.5.0), it is compatible with MUMPS versions 5.7.0 and 5.7.1, but not with versions earlier than 5.7.0. We highly recommend using the latest version of MUMPS. 
 
-After the MUMPS installation, if you have a clean Julia environment (*i.e.* have not installed any Julia package before or have not installed new version of [Makie.jl](https://github.com/MakieOrg/Makie.jl)(v0.20 and v0.21) and [GeometryPrimitives.jl](https://github.com/stevengj/GeometryPrimitives.jl)(v0.5.0), you can install MESTI.jl (v0.4.4) by opening the command-line interface of Julia and typing:  
+After the MUMPS installation, if you have a clean Julia environment (*i.e.* have not installed any Julia package before or have not installed new version of [Makie.jl](https://github.com/MakieOrg/Makie.jl)(v0.20 and v0.21) and [GeometryPrimitives.jl](https://github.com/stevengj/GeometryPrimitives.jl)(v0.5.0), you can install MESTI.jl (v0.5.0) by opening the command-line interface of Julia and typing:  
 
 ```julia
 import Pkg; Pkg.add("MESTI")
@@ -80,7 +80,7 @@ On the other hand, if you have installed these two Julia packages: the new versi
 import Pkg; Pkg.add(Pkg.PackageSpec(;name="Makie", version="0.19.12")); Pkg.add("GeometryPrimitives"); Pkg.add("MESTI")
 ```
 
-to downgrade them to the old version compatible with our MESTI.jl and then install MESTI.jl(v0.4.4).
+to downgrade them to the old version compatible with our MESTI.jl and then install MESTI.jl(v0.5.0).
 
 After installing MESTI.jl, if the user happens to run the command to upgrade every Julia package to the latest version, such as *Pkg.update()*, it would install the latest version of Makie.jl and make MESTI.jl not compatible with it. So please rerun the line above to downgrade to the compatible version and reinstall MESTI.jl.
 
@@ -93,7 +93,7 @@ import Pkg; Pkg.status("MESTI")
 If you want our latest MESTI.jl, it should show
 
 ```julia
-[8d7f31fa] MESTI v0.4.4
+[8d7f31fa] MESTI v0.5.0
 ```
 
 ## Tests

mumps/README.md

@@ -37,10 +37,10 @@ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LMETISDIR
 After installing METIS, if you set <code>opts.use_METIS = true</code> in <code>mesti()</code> or <code>mesti2s()</code>, MUMPS will use METIS for matrix ordering. From our experience, in 2D, AMD is usually faster when using the APF method, but METIS can sometimes reduce memory usage. In 3D, METIS is strongly recommended, which is much faster than AMD. By default, 2D systems use AMD, while 3D systems use METIS (if it is available).
 
 ## Compile MUMPS
-Suppose you downloaded the 5.6.2 version of MUMPS to your ~/Downloads/ folder. Then, go to the folder where you want to compile MUMPS, and enter
+Suppose you downloaded the 5.7.1 version of MUMPS to your ~/Downloads/ folder. Then, go to the folder where you want to compile MUMPS, and enter
 ```shell
-tar zxvf ~/Downloads/MUMPS_5.6.2.tar.gz
-cd MUMPS_5.6.2
+tar zxvf ~/Downloads/MUMPS_5.7.1.tar.gz
+cd MUMPS_5.7.1
 ```
 in terminal.
 
@@ -51,8 +51,8 @@ Read the file <code>INSTALL</code>, copy the closest <code>Makefile.inc</code> f
  - <code>LAPACK</code>: how the Fortran compiler can link to the LAPACK library
  - <code>SCALAP</code>: how the Fortran compiler can link to the ScaLAPACK library
  - <code>LIBBLAS</code>: how the Fortran compiler can link to the BLAS library
- - <code>RPATH_OPT</code>: the path to shared libraries that will be built up, such as <code>.../MUMPS_5.6.2/lib/</code>
-where <code>...</code> is the path to MUMPS_5.6.2 folder. 
+ - <code>RPATH_OPT</code>: the path to shared libraries that will be built up, such as <code>.../MUMPS_5.7.1/lib/</code>
+where <code>...</code> is the path to MUMPS_5.7.1 folder. 
 
 Note that from our experience, <code>RPATH_OPT</code> must be specified to successfully install the parallel version of MUMPS on Linux and Windows.
 
@@ -86,10 +86,10 @@ After compiling the parallel version of MUMPS, in <code>startup.jl</code> we sho
 
 ```shell
 mkdir ~/.julia/config
-echo 'ENV["MUMPS_PREFIX"] = ".../MUMPS_5.6.2/lib"' >> ~/.julia/config/startup.jl
+echo 'ENV["MUMPS_PREFIX"] = ".../MUMPS_5.7.1/lib"' >> ~/.julia/config/startup.jl
 ```
 
-where <code>...</code> is the path to MUMPS_5.6.2 folder.
+where <code>...</code> is the path to MUMPS_5.7.1 folder.
 
 
 When we run Julia interface for MUMPS, the machine may not find the libraries by itself. To solve those issues, please follow the steps depending on your OS:
@@ -112,16 +112,19 @@ export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_scalapack_lp64.so
 
 or
 
-- macOS
+- <a name="mpi-config"></a> macOS
 
-We can configure [MPI.jl](https://juliaparallel.org/MPI.jl/stable/configuration/) before running MUMPS in Julia. The steps are straightforward using MPIPreferences.jl. First, install MPIPreferences.jl by entering
+To use MUMPS.jl, we need to install and [configure](https://juliaparallel.org/MPI.jl/stable/configuration/) MPI.jl. Begin by installing MPI.jl and MPIPreferences.jl with the following command in Julia:
 ```
-julia --project -e 'using Pkg; Pkg.add("MPIPreferences")'
+using Pkg; Pkg.add(["MPI", "MPIPreferences"])
 ```
-in terminal. Then run <code>MPIPreferences.use_system_binary()</code> in Julia or through the command line:
+Once the installation is complete, run the following command in Julia to configure MPI.jl:
 ```
-julia --project -e 'using MPIPreferences; MPIPreferences.use_system_binary()'
+using MPIPreferences; MPIPreferences.use_system_binary()
 ```
-This should automatically find the OpenMPI installed in macOS.
+This will automatically find the OpenMPI library installed by Homebrew. If MPIPreferences cannot locate the library, you need to specify the library path manually:
 
+```
+MPIPreferences.use_system_binary(; library_names=["/path/to/open-mpi/5.x.x/lib/libmpi"])
+```
 Now, you are ready to install MESTI.jl. Please go back to [install MESTI.jl](../#installation).

mumps/linux/Makefile.inc

@@ -1,6 +1,6 @@
 #
-#  This file is part of MUMPS 5.6.2, released
-#  on Wed Oct 11 09:36:25 UTC 2023
+#  This file is part of MUMPS 5.7.1, released
+#  on Thu May  2 10:15:09 UTC 2024
 #
 ################################################################################
 #
@@ -55,7 +55,7 @@
 
 LPORDDIR = $(topdir)/PORD/lib/
 IPORD    = -I$(topdir)/PORD/include/
-LPORD    = -L$(LPORDDIR) -lpord
+LPORD    = -L$(LPORDDIR) -lpord$(PLAT)
 
 # The following is the path of the METIS on Hsu's group in USC Discovery cluster.
 # Please modify the path based on your machine.
@@ -104,7 +104,7 @@ SONAME = -soname
 # Adapt/uncomment RPATH_OPT to avoid modifying
 # LD_LIBRARY_PATH in case of shared libraries
 # The following is the path of the MUMPS shared libraries on Hsu's group in USC Discovery cluster:
-RPATH_OPT = -Wl,-rpath,/project/cwhsu_38/shared/software/mumps-5.6.2-par/lib/
+RPATH_OPT = -Wl,-rpath,/project/cwhsu_38/shared/software/mumps-5.7.1-par/lib/
 OUTC    = -o 
 OUTF    = -o 
 # RM : remove files
@@ -152,7 +152,7 @@ SCALAP = -L$(MKLROOT)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
 LIBPAR = $(SCALAP) $(LAPACK)
 
 INCSEQ = -I$(topdir)/libseq
-LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq
+LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq$(PLAT)
 
 # DEFINE HERE YOUR BLAS LIBRARY
 # BLAS library is under the folder of $(MKLROOT)/lib/intel64

mumps/macOS/Makefile.inc

@@ -1,6 +1,6 @@
 #
-#  This file is part of MUMPS 5.6.2, released
-#  on Wed Oct 11 09:36:25 UTC 2023
+#  This file is part of MUMPS 5.7.1, released
+#  on Thu May 2 10:15:09 UTC 2024
 #
 ################################################################################
 #
@@ -9,7 +9,7 @@
 #   This defines some parameters dependent on your platform; you should
 #   look for the approriate file in the directory ./Make.inc/ and copy it
 #   into a file called Makefile.inc. For example, from the MUMPS root
-#   directory, use
+#   directory, use 
 #   "cp Make.inc/Makefile.inc.generic ./Makefile.inc"
 #   (see the main README file for details)
 #

mumps/macOS/README.md

@@ -41,9 +41,27 @@ MUMPS requires the ScaLAPACK, MPI and OpenMP libraries, which can be installed v
 brew install scalapack open-mpi llvm
 ```
 
- The <code>clang</code> compiler from Apple does not support OpenMP by default and we need to use the <code>clang</code> compiler from LLVM instead. After installing LLVM, enter
+ The <code>clang</code> compiler from Apple does not support OpenMP and we need to use the <code>clang</code> compiler from LLVM instead. After installing LLVM, enter
 
 ```
 export PATH=/opt/homebrew/opt/llvm/bin:$PATH
 ```
-to override the <code>clang</code> compiler from Apple.
+to override the <code>clang</code> compiler from Apple.
+
+## Troubleshooting
+
+### Issue 1: "clang: error: unsupported option '-fopenmp'"
+This error occurs if you compile with the default <code>clang</code> compiler from Apple. Make sure you are using the <code>clang</code> compiler from LLVM by running the following command in terminal:
+```
+which clang
+```
+The output should be <code>/path/to/llvm/bin/clang</code>. If this is not the case, you need to override the <code>clang</code> compiler from Apple by running the following command in terminal:
+```
+export PATH=/opt/homebrew/opt/llvm/bin:$PATH
+```
+
+### Issue 2: The tests from MESTI.jl take a very long time to run
+The tests in <code>MESTI.jl/test</code> and <code>MESTI.jl/examples</code> should complete in a few seconds. If the test keeps running without outputting any errors, ensure that MPI.jl is configured properly by following those [steps](../README.md#Running-MUMPS-in-Julia).
+
+### Issue 3: "shmem: mmap: an error occurred while determining whether or not /var/folders/.../sm_segment.xxx could be created."
+If you are using OpenMPI v5.0.3 or later, you may encounter this error while using MESTI.jl. The code can still complete but segmentation fault or deadlock may occur in some [cases](https://github.com/open-mpi/ompi/issues/12307). If those issues occur, you can downgrade OpenMPI to v5.0.2

mumps/windows/Makefile.inc

@@ -1,6 +1,6 @@
 #
-#  This file is part of MUMPS 5.6.2, released
-#  on Wed Oct 11 09:36:25 UTC 2023
+#  This file is part of MUMPS 5.7.1, released
+#  on Thu May  2 10:15:09 UTC 2024
 #
 ################################################################################
 #
@@ -55,7 +55,7 @@
 
 LPORDDIR = $(topdir)/PORD/lib/
 IPORD    = -I$(topdir)/PORD/include/
-LPORD    = -L$(LPORDDIR) -lpord
+LPORD    = -L$(LPORDDIR) -lpord$(PLAT)
 
 # The following is the path of the METIS by default installation.
 # You may need to modify the path based on your machine.
@@ -103,7 +103,7 @@ FPIC_OPT = -fPIC
 SONAME = -soname
 # Adapt/uncomment RPATH_OPT to avoid modifying
 # LD_LIBRARY_PATH in case of shared libraries
-RPATH_OPT = -Wl,-rpath,/home/usc/Downloads/MUMPS_5.6.2/lib/
+RPATH_OPT = -Wl,-rpath,/home/usc/Downloads/MUMPS_5.7.1/lib/
 OUTC    = -o 
 OUTF    = -o 
 # RM : remove files
@@ -151,7 +151,7 @@ SCALAP = -L$(MKLROOT)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
 LIBPAR = $(SCALAP) $(LAPACK)
 
 INCSEQ = -I$(topdir)/libseq
-LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq
+LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq$(PLAT)
 
 # DEFINE HERE YOUR BLAS LIBRARY
 # BLAS library is under the folder of $(MKLROOT)/lib/intel64

src/mesti_matrix_solver.jl

@@ -1030,7 +1030,9 @@ function MUMPS_analyze_and_factorize(A::Union{SparseMatrixCSC{Int64, Int64},Spar
     if opts.use_L0_threads
         # Utilize L0-threads feature.
         # This typically improves the time performance, but marginally increases the memory usage in full multithread.
-        set_keep!(id,401,1)
+        set_icntl!(id,48,1;displaylevel=0)
+    else
+        set_icntl!(id,48,0;displaylevel=0)
     end
     set_job!(id,1) # what to do: analysis  
     if opts.use_given_ordering

src/mumps3_struc.jl

@@ -1,9 +1,9 @@
-# this file mirros the relevant content of the "[sdcz]mumps_c.h" file of MUMPS 5.3.3
+# this file mirros the relevant content of the "[sdcz]mumps_c.h" file of MUMPS 5.7.1
 # `gc_haven`, contains Julia references to protect the pointers passed to C
 # from gargage collection.
 export Mumps
 
-const MUMPS_VERSION = "5.6.2"
+const MUMPS_VERSION = "5.7.1"
 const MUMPS_VERSION_MAX_LEN = 30
 
 # mirror of structre in [sdcz]mumps_c.h
@@ -53,16 +53,26 @@ mutable struct Mumps{TC,TR}
     colsca_from_mumps::MUMPS_INT
     rowsca_from_mumps::MUMPS_INT
 
+    colsca_loc::Ptr{TR}
+    rowsca_loc::Ptr{TR}
+
+    rowind::Ptr{MUMPS_INT}
+    colind::Ptr{MUMPS_INT}
+    pivots::Ptr{TC}
+
     rhs::Ptr{TC}
     redrhs::Ptr{TC}
     rhs_sparse::Ptr{TC}
     sol_loc::Ptr{TC}
     rhs_loc::Ptr{TC}
+    rhsintr::Ptr{TC}
 
     irhs_sparse::Ptr{MUMPS_INT}
     irhs_ptr::Ptr{MUMPS_INT}
     isol_loc::Ptr{MUMPS_INT}
     irhs_loc::Ptr{MUMPS_INT}
+    glob2loc_rhs::Ptr{MUMPS_INT}
+    glob2loc_sol::Ptr{MUMPS_INT}
 
     nrhs::MUMPS_INT
     lrhs::MUMPS_INT
@@ -71,6 +81,7 @@ mutable struct Mumps{TC,TR}
     lsol_loc::MUMPS_INT
     nloc_rhs::MUMPS_INT
     lrhs_loc::MUMPS_INT
+    nsol_loc::MUMPS_INT
 
     schur_mloc::MUMPS_INT
     schur_nloc::MUMPS_INT
@@ -80,6 +91,7 @@ mutable struct Mumps{TC,TR}
     nblock::MUMPS_INT
     nprow::MUMPS_INT
     npcol::MUMPS_INT
+    ld_rhsintr::MUMPS_INT
 
     info::NTuple{80,MUMPS_INT}
     infog::NTuple{80,MUMPS_INT}
@@ -90,23 +102,26 @@ mutable struct Mumps{TC,TR}
     deficiency::MUMPS_INT
     pivnul_list::Ptr{MUMPS_INT}
     mapping::Ptr{MUMPS_INT}
+    singular_values::Ptr{TR}
+
 
     size_schur::MUMPS_INT
     listvar_schur::Ptr{MUMPS_INT}
     schur::Ptr{TC}
 
-    instance_number ::MUMPS_INT
     wk_user     ::Ptr{TC}
 
     version_number ::NTuple{MUMPS_VERSION_MAX_LEN+1+1,Cchar}
-    ooc_tmpdir     ::NTuple{256,Cchar}
-    ooc_prefix      ::NTuple{64,Cchar}
-    write_problem  ::NTuple{256,Cchar}
+    ooc_tmpdir     ::NTuple{1024,Cchar}
+    ooc_prefix      ::NTuple{256,Cchar}
+    write_problem  ::NTuple{1024,Cchar}
     lwk_user    ::MUMPS_INT
-    save_dir    ::NTuple{256,Cchar}
+    save_dir    ::NTuple{1024,Cchar}
     save_prefix ::NTuple{256,Cchar}
 
     metis_options::NTuple{40,MUMPS_INT}
+    
+    instance_number::MUMPS_INT
 
     _gc_haven::Array{Ref,1}
     _finalized::Bool

src/mumps3_types.jl

@@ -1,4 +1,4 @@
-# this file mirrors the "mumps_c_types.h" header file of MUMPS 5.2.2
+# this file mirrors the "mumps_c_types.h" header file of MUMPS 5.7.1
 
 const MUMPS_INT = Cint
 const MUMPS_INT8 = Int64
@@ -28,3 +28,5 @@ const MUMPS_ARITH_REAL = ( MUMPS_ARITH_s | MUMPS_ARITH_d )
 const MUMPS_ARITH_CMPLX = ( MUMPS_ARITH_c | MUMPS_ARITH_z )
 const MUMPS_ARITH_SINGLE = ( MUMPS_ARITH_s | MUMPS_ARITH_c )
 const MUMPS_ARITH_DBL = ( MUMPS_ARITH_d | MUMPS_ARITH_z )
+
+const MUMPS_OFF_T = MUMPS_INT8