JSO Compliance : LM #200

MaxenceGollier · 2025-08-07T13:17:27Z

We should merge after #199.

src/LM_alg.jl

codecov · 2025-08-07T13:23:25Z

Codecov Report

❌ Patch coverage is 98.08917% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.05%. Comparing base (e0f214d) to head (7df41cc).
⚠️ Report is 140 commits behind head on master.

Files with missing lines	Patch %	Lines
src/LM_alg.jl	97.81%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #200       +/-   ##
===========================================
+ Coverage   61.53%   86.05%   +24.51%     
===========================================
  Files          11       13        +2     
  Lines        1292     1563      +271     
===========================================
+ Hits          795     1345      +550     
+ Misses        497      218      -279

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

src/LMModel.jl

src/LM_alg.jl

dpo · 2025-09-11T19:25:26Z

src/LMModel.jl

+  σ::T
+  meta::NLPModelMeta{T, V}
+  counters::Counters
+end


Here too, would it be possible to reuse LLSModels?

I am not sure how, LLSModels does $\frac{1}{2} ∥Ax-b∥^2_2$, not $\frac{1}{2} ∥Ax-b∥^2_2 + \frac{\sigma}{2}∥x∥^2_2$,
sure we could write
$$\frac{1}{2} \|Ax-b\|^2_2 + \frac{\sigma}{2}\|x\|^2_2 = \frac{1}{2} \| \begin{bmatrix} A \\ \sqrt{\sigma} I \end{bmatrix} x - \begin{bmatrix} b \\ 0 \end{bmatrix} \|_2^2$$
but this will be impractical in my opinion.

src/LM_alg.jl

MaxenceGollier · 2025-09-16T17:14:51Z

LM Test

Without bounds

subsolver : R2

On my branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "LM-JSO")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 1)

LM(bpdn, h, options)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        σ      ‖x‖      ‖s‖     ‖J‖² LM 
[ Info:      0     20  1.8e+00  0.0e+00  9.3e-01  1.0e+00  7.4e-04  0.0e+00  3.1e+00  1.0e+00 ↘
[ Info:      1     23  1.1e-02  4.6e-01  2.8e-03  1.0e+00  2.5e-04  3.1e+00  9.0e-03  1.0e+00 ↘
[ Info:      2  10001  1.1e-02  4.6e-01  4.1e-06  1.0e+00  8.2e-05  3.1e+00  1.7e-05  1.0e+00 ↘
[ Info:      3      0  1.1e-02  4.6e-01  5.1e-08  1.0e+00  2.7e-05  3.1e+00  1.9e-08  1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 5.055168491119672e-8
"Execution stats: first-order stationary"

On master,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "master")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 10)

LM(bpdn, h, options)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       σ     ‖x‖     ‖s‖   ‖Jₖ‖² reg
[ Info:      1       20  1.8e+00  0.0e+00 9.3e-01 1.1e+00  1.0e+00 7.4e-04 0.0e+00 3.1e+00 1.0e+00 ↘
[ Info:      2       23  1.1e-02  4.6e-01 2.8e-03 3.5e-03  1.0e+00 2.5e-04 3.1e+00 9.0e-03 1.0e+00 ↘
[ Info:      3    10001  1.1e-02  4.6e-01 4.1e-06 5.8e-06  1.0e+00 8.2e-05 3.1e+00 1.7e-05 1.0e+00 ↘
[ Info:      4        1  1.1e-02  4.6e-01 5.1e-08 5.1e-08          2.7e-05 3.1e+00 1.9e-08 1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 5.055168491119672e-8
"Execution stats: first-order stationary"

subsolver : R2DH

On my branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "LM-JSO")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 1)

LM(bpdn, h, options, subsolver = R2DHSolver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        σ      ‖x‖      ‖s‖     ‖J‖² LM 
[ Info:      0      6  1.8e+00  0.0e+00  9.3e-01  1.0e+00  7.4e-04  0.0e+00  3.1e+00  1.0e+00 ↘
[ Info:      1      6  1.1e-02  4.6e-01  2.4e-03  1.0e+00  2.5e-04  3.1e+00  7.3e-03  1.0e+00 ↘
[ Info:      2  10001  1.1e-02  4.6e-01  4.1e-06  1.0e+00  8.2e-05  3.1e+00  1.7e-05  1.0e+00 ↘
[ Info:      3      0  1.1e-02  4.6e-01  4.7e-08  1.0e+00  2.7e-05  3.1e+00  1.4e-09  1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 4.713968924434026e-8
"Execution stats: first-order stationary"

On master,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "master")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 10)

LM(bpdn, h, options, subsolver = R2DH)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       σ     ‖x‖     ‖s‖   ‖Jₖ‖² reg
[ Info:      1        6  1.8e+00  0.0e+00 9.3e-01 1.1e+00  1.0e+00 7.4e-04 0.0e+00 3.1e+00 1.0e+00 ↘
[ Info:      2        6  1.1e-02  4.6e-01 2.4e-03 2.9e-03  1.0e+00 2.5e-04 3.1e+00 7.3e-03 1.0e+00 ↘
[ Info:      3    10001  1.1e-02  4.6e-01 4.1e-06 5.9e-06  1.0e+00 8.2e-05 3.1e+00 1.7e-05 1.0e+00 ↘
[ Info:      4        1  1.1e-02  4.6e-01 4.7e-08 4.7e-08          2.7e-05 3.1e+00 1.4e-09 1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 4.713968924434026e-8
"Execution stats: first-order stationary"

With bounds

subsolver : R2

On my branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1, bounds = true)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "LM-JSO")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 1)

LM(bpdn, h, options)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        σ      ‖x‖      ‖s‖     ‖J‖² LM 
[ Info:      0     13  2.2e+00  0.0e+00  1.2e+00  1.0e+00  7.4e-04  0.0e+00  2.8e+00  1.0e+00 ↘
[ Info:      1     15  3.1e-01  4.8e-01  2.5e-03  1.0e+00  2.5e-04  2.8e+00  5.9e-03  1.0e+00 ↘
[ Info:      2  10001  3.1e-01  4.8e-01  3.3e-06  1.0e+00  8.2e-05  2.8e+00  1.4e+00  1.0e+00 ↘
[ Info:      3     22  8.1e-03  6.0e-01  1.2e-04  1.0e+00  2.7e-05  3.2e+00  4.2e-04  1.0e+00 ↘
[ Info:      4      0  8.1e-03  6.0e-01  1.1e-07  1.0e+00  9.1e-06  3.2e+00  1.0e-07  1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 1.1401465572257465e-7
"Execution stats: first-order stationary"

On master,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1, bounds = true)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "master")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 10)

LM(bpdn, h, options)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       σ     ‖x‖     ‖s‖   ‖Jₖ‖² reg
[ Info:      1       13  2.2e+00  0.0e+00 1.2e+00 1.2e+00  1.0e+00 7.4e-04 0.0e+00 2.8e+00 1.0e+00 ↘
[ Info:      2       15  3.1e-01  4.8e-01 2.5e-03 2.7e-03  1.0e+00 2.5e-04 2.8e+00 5.9e-03 1.0e+00 ↘
[ Info:      3    10001  3.1e-01  4.8e-01 3.3e-06 4.3e-01  1.0e+00 8.2e-05 2.8e+00 1.4e+00 1.0e+00 ↘
[ Info:      4       22  8.1e-03  6.0e-01 1.2e-04 1.6e-04  1.0e+00 2.7e-05 3.2e+00 4.2e-04 1.0e+00 ↘
[ Info:      5        1  8.1e-03  6.0e-01 1.1e-07 1.1e-07          9.1e-06 3.2e+00 1.0e-07 1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 1.1401465572257465e-7
"Execution stats: first-order stationary"

subsolver : R2DH

On my branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1, bounds = true)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "LM-JSO")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 1)

LM(bpdn, h, options, subsolver = R2DHSolver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        σ      ‖x‖      ‖s‖     ‖J‖² LM 
[ Info:      0      6  2.2e+00  0.0e+00  1.2e+00  1.0e+00  7.4e-04  0.0e+00  3.2e+00  1.0e+00 ↘
[ Info:      1      6  8.2e-03  6.0e-01  2.1e-03  1.0e+00  2.5e-04  3.2e+00  5.7e-03  1.0e+00 ↘
[ Info:      2  10001  8.1e-03  6.0e-01  2.6e-06  1.0e+00  8.2e-05  3.2e+00  9.6e-06  1.0e+00 ↘
[ Info:      3      0  8.1e-03  6.0e-01  4.7e-08  1.0e+00  2.7e-05  3.2e+00  7.9e-10  1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 4.713968924434026e-8
"Execution stats: first-order stationary"

On master,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = bpdn_model(1, bounds = true)[2]
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL0(λ)

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "master")
using RegularizedOptimization
options = ROSolverOptions(β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 10)

LM(bpdn, h, options, subsolver = R2DHSolver)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       σ     ‖x‖     ‖s‖   ‖Jₖ‖² reg
[ Info:      1        6  2.2e+00  0.0e+00 1.2e+00 1.3e+00  1.0e+00 7.4e-04 0.0e+00 3.2e+00 1.0e+00 ↘
[ Info:      2        6  8.2e-03  6.0e-01 2.1e-03 2.4e-03  1.0e+00 2.5e-04 3.2e+00 5.7e-03 1.0e+00 ↘
[ Info:      3    10001  8.1e-03  6.0e-01 2.6e-06 3.5e-06  1.0e+00 8.2e-05 3.2e+00 9.6e-06 1.0e+00 ↘
[ Info:      4        1  8.1e-03  6.0e-01 4.7e-08 4.7e-08          2.7e-05 3.2e+00 7.9e-10 1.0e+00
[ Info: LM: terminating with √(ξ1/ν) = 4.713968924434026e-8
"Execution stats: first-order stationary"

MaxenceGollier · 2025-09-16T17:16:48Z

I think we are ready for this one @dpo

MaxenceGollier added 8 commits August 4, 2025 08:53

start LM update

58f26a1

JSO-compliant LM draft

3ffc73b

make LMModel type stable

02b6cce

improve type stability in LM

458aae8

add LM doc

5101a10

add LM function signatures

144f9a4

update tests

576bb88

solve function signature bugs

f85deac

MaxenceGollier commented Aug 7, 2025

View reviewed changes

src/LM_alg.jl Outdated Show resolved Hide resolved

remove dead code

53c40b8

MaxenceGollier commented Aug 7, 2025

View reviewed changes

src/LMModel.jl Outdated Show resolved Hide resolved

MaxenceGollier mentioned this pull request Aug 8, 2025

JSO Compliance : LMTR #201

Open

apply julia formatter

3c8e92e

dpo reviewed Sep 7, 2025

View reviewed changes

src/LMModel.jl Outdated Show resolved Hide resolved

src/LM_alg.jl Outdated Show resolved Hide resolved

MaxenceGollier and others added 3 commits September 8, 2025 10:05

implicit jacobian update

de6a62b

Merge branch 'JuliaSmoothOptimizers:master' into LM-JSO

63b9e3a

remove temporary struct

b4cb5b7

dpo reviewed Sep 11, 2025

View reviewed changes

MaxenceGollier and others added 4 commits September 13, 2025 11:27

Merge branch 'JuliaSmoothOptimizers:master' into LM-JSO

25fc1da

update callback docstring

77ddd78

Merge branch 'JuliaSmoothOptimizers:master' into LM-JSO

69e8409

fix arrow prints

7df41cc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

JSO Compliance : LM #200

JSO Compliance : LM #200

MaxenceGollier commented Aug 7, 2025

Uh oh!

Uh oh!

codecov bot commented Aug 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dpo Sep 11, 2025

Uh oh!

MaxenceGollier Sep 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

MaxenceGollier commented Sep 16, 2025

Uh oh!

MaxenceGollier commented Sep 16, 2025

Uh oh!

Uh oh!

JSO Compliance : LM #200

Are you sure you want to change the base?

JSO Compliance : LM #200

Conversation

MaxenceGollier commented Aug 7, 2025

Uh oh!

Uh oh!

codecov bot commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dpo Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

MaxenceGollier Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MaxenceGollier commented Sep 16, 2025

LM Test

Without bounds

subsolver : R2

subsolver : R2DH

With bounds

subsolver : R2

subsolver : R2DH

Uh oh!

MaxenceGollier commented Sep 16, 2025

Uh oh!

Uh oh!

codecov bot commented Aug 7, 2025 •

edited

Loading

MaxenceGollier Sep 13, 2025 •

edited

Loading