many updates

cleaning up the code, adding some type hints, simplyfing the example

Author: coursekevin

README.md

@@ -1,21 +1,52 @@
-# gradoptorch
-Gradient based optimizers for python with a PyTorch backend. This package intends to allow more fine-grain control of optimizers.
+<div align="center">
 
-Note that this toolbox is meant for prototyping and built-in PyTorch optimizers will almost certainly have better performance.
+# GradOpTorch
+
+#### Classical gradient based optimization in PyTorch.
+
+</div>
+
+- [Installation](#installation)
+- [Usage](#usage)
+
+## What is GradOpTorch?
+
+GradOpTorch is a suite of classical gradient-based optimization tools for
+PyTorch. The toolkit includes conjugate gradients, BFGS, and some 
+methods for line-search.
+
+## Why not [torch.optim](https://pytorch.org/docs/stable/optim.html)?
+
+Not every problem is high-dimensional with noisy gradients.  
+For such problems, classical optimization techniques 
+can be more appropriate.
+
+## Installation
+
+GradOpTorch can be installed from PyPI:
+
+```bash
+pip install gradoptorch
+```
+
+## Usage
+
+### Included optimizers:
 
-## Included optimizers:
     'grad_exact' : exact gradient optimization
     'conj_grad_fr' : conjugate gradient descent using Fletcher-Reeves search direction
     'conj_grad_pr' : conjugate gradient descent using Polak-Ribiere search direction
     'newton_exact' : exact newton optimization
     'bfgs' : approximate newton optimization using bfgs
 
-## Included line-search methods:
+### Included line-search methods:
+
     'back_tracking' : backing tracking based line-search
     'quad_search' : quadratic line-search
     'constant' : no line search, constant step size used
 
 ## Setup
+
 ```bash
 pip install git+https://github.com/coursekevin/gradoptorch.git
 ```

examples/functional_example.py

@@ -0,0 +1,53 @@
+import torch
+from gradoptorch import optimizer
+
+import matplotlib.pyplot as plt  # type: ignore
+
+dim = 2
+
+torch.set_default_dtype(torch.float64)
+torch.manual_seed(42)
+
+
+# Define test objective function
+def f(x):
+    a = 1.0
+    b = 100.0
+    return (a - x[0]).pow(2) + b * (x[1] - x[0].pow(2)).pow(2)
+
+
+def main():
+    # ---------------------------------------------------------------------------------
+    # optimizing using all of the different methods
+    opt_method = ["conj_grad_pr", "conj_grad_fr", "grad_exact", "newton_exact", "bfgs"]
+    histories = []
+    for method in opt_method:
+        x = torch.tensor([-0.5, 3.0])
+        # using quadratic line search
+        x_opt, hist = optimizer(f, x, opt_method=method, ls_method="quad_search")
+        histories.append(hist)
+
+    X, Y = torch.meshgrid(
+        torch.linspace(-2.5, 2.5, 100), torch.linspace(-1.5, 3.5, 100), indexing="ij"
+    )
+
+    # ---------------------------------------------------------------------------------
+    # making some plots
+    _, axs = plt.subplots(1, 2)
+
+    ax1 = axs[0]
+    ax2 = axs[1]
+
+    for hist in histories:
+        ax1.plot(torch.tensor(hist.f_hist).log10())
+        x_hist = torch.stack(hist.x_hist, dim=0)
+        ax2.plot(x_hist[:, 0], x_hist[:, 1], "x-")
+    ax1.legend(opt_method)
+
+    ax2.contourf(X, Y, f(torch.stack([X, Y], dim=0)), 50)
+
+    plt.show()
+
+
+if __name__ == "__main__":
+    main()

gradoptorch/__init__.py

@@ -1 +1 @@
-from .gradoptorch import gradoptorch
+from .gradoptorch import optimizer, default_opt_settings, default_ls_settings