How to implement a plug-in system with multibind?

[eirikur-grid]

I've used Guice in the past for registering and instantiating plug-ins using the multibind extension. I found that straight-forward to do based on the Guice documentation. However, I'm struggling to do the same using python injector. All the tests and documentation I've found use multibind in conjunction with lists or dictionaries containing concrete values (strings or ints). I need the plug-ins to be instantiated by the DI framework.

If I bind to a list of classes like this

binder.multibind(List[Plugin], [GreenPlugin, RedPlugin])

then injector.get(List[Plugin]) just returns the list of classes, rather than a list of instantiated objects.

If I do this:

binder.multibind(List[Plugin], GreenPlugin)
binder.multibind(List[Plugin], RedPlugin)

then I get the following error when calling injector.get(List[Plugin])

>   return [i for provider in self._providers for i in provider.get(injector)]
E   TypeError: 'GreenPlugin' object is not iterable

I've been able to get this approach to work by having my base Plugin class pretend to be iterable like so:

class Plugin:
    # Dependency Injection hack for making multibind work
    def __getitem__(self, item):
        if item == 0:
            return self
        raise IndexError

but it's not very nice.

I could do this:

def configure(binder):
  binder.bind(GreenPlugin)
  binder.bind(RedPlugin)

@provider
def get_plugins(green: GreenPlugin, red: RedPlugin) -> List[Plugin]:
  return [green, red]

but that results in a lot of repetition. Adding a new plugin would mean binding it, adding it to the argument list of the provider function and the list it returns.

I feel like I must be missing something. How should a plug-in system be implemented using python injector?

[alecthomas]

Being able to multibind to types was definitely the original intent, as you can see from the documentation. I'll take a look.

[jstasiak]

@alecthomas This is the relevant part of multibind signature right now:

    @overload
    def multibind(
        self,
        interface: Type[List[T]],
        to: Union[List[T], Callable[..., List[T]], Provider[List[T]]],
        scope: Union[Type['Scope'], 'ScopeDecorator'] = None,
    ) -> None:
        pass

I think extending it to be

    @overload
    def multibind(
        self,
        interface: Type[List[T]],
        # Line below changed
        to: Union[Callable[..., List[T]], Provider[List[T]], List[Union[T, Provider[T], Type[T], Callable[..., T]]]],
        scope: Union[Type['Scope'], 'ScopeDecorator'] = None,
    ) -> None:
        pass

or something similar and handling it would to the job.

@eirikur-grid You could also try

@provider
def get_plugins(injector: Injector) -> List[Plugin]:
  return [injector.get(plugin_class) for plugin_class in [X, Y, Z]]

right now, which should be slightly less annoying.

[JosXa]

On that regard, wouldn't it be sensible to support things from the collections module and some typing ones aswell?
Iterable, Collection, Counter, Sequence, ...

[jstasiak]

I have no reasonable idea how those would even work and fit the system so I can't really say.

[darkoob12]

jstasiak's solution worked for me, but got an error that I should use @multiprovirder instead of @provider:

@injector.multiprovider
def get_all_plugins(di: injector.Injector) -> Dict[str, PluginInterface]:
     return {
         "red": di.get(RedPlugin),
         "green": di.get(GreenPlugin)
     }

library version is 0.18.4

[bveldhoen]

Hello,

I could really use the branch created by @eirikur-nc (#277). Would it be possible to create a new version for this?

[davidparsson]

Thanks for the reminder. I'll take a look.