pylsp_completion_item_resolve crashes sometimes on rope auto-import completion

[igmacs]

When using python-lsp-server with the rope autoimport plugin enabled, selecting an auto-import completion item with a fresh state causes completionItem/resolve to fail. However if you do a normal Jedi completion before, or the completion list have items both from Jedi and Rope, then it works. By "fresh state" I mean a new pylsp session, but also more situations (for example this also happens if I discard changes from git)

Environment

I'm using Emacs's Eglot as LSP client, and pysl version v1.14.0. But from the code I've read, I think I know the cause and the client and versions are irrelevant.

Reproduction

Since I use eglot and I'm new to pylsp I'm not sure how to provide self-contained reproduction steps. But the description I've given above should be enough.

Expected behavior

Auto-import completion items should resolve cleanly and insert the corresponding import, even if no prior Jedi completion has been performed.

Actual behavior

There is an exception in jedi_completion.py (backtrace below), and completionItem/resolve returns an empty list. In my client, completion at point is finished, but there is an error and the import is not added.

Backtrace

This is the relevant events shown with eglot-events-buffer, which includes the Python's backtrace

[jsonrpc] --> completionItem/resolve[348] {"jsonrpc":"2.0","id":348,"method":"completionItem/resolve","params":{"label":"bisect_left","kind":3,"sortText":"[z00100","data":{"doc_uri":"file:///home/ignacio/repos/my-repos/heathen-ledger/src/heathen_ledger/parser.py"},"detail":"# Auto-Import\nfrom bisect import bisect_left","additionalTextEdits":[{"range":{"start":{"line":2,"character":0},"end":{"line":2,"character":0}},"newText":"from bisect import bisect_left\n"}]}}
[stderr]  WARNING - pylsp.config.config - Failed to load hook pylsp_completion_item_resolve: 'LAST_JEDI_COMPLETIONS'
[stderr]  Traceback (most recent call last):
[stderr]    File ".../pylsp/config/config.py", line 40, in _hookexec
[stderr]      return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
[stderr]    File ".../pluggy/_manager.py", line 475, in traced_hookexec
[stderr]      return outcome.get_result()
[stderr]    File ".../pluggy/_result.py", line 103, in get_result
[stderr]      raise exc.with_traceback(tb)
[stderr]    File ".../pluggy/_result.py", line 62, in from_call
[stderr]      result = func()
[stderr]    File ".../pluggy/_manager.py", line 472, in <lambda>
[stderr]      lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
[stderr]    File ".../pluggy/_callers.py", line 167, in _multicall
[stderr]      raise exception
[stderr]    File ".../pluggy/_callers.py", line 121, in _multicall
[stderr]      res = hook_impl.function(*args)
[stderr]    File ".../pylsp/plugins/jedi_completion.py", line 150, in pylsp_completion_item_resolve
[stderr]      shared_data = document.shared_data["LAST_JEDI_COMPLETIONS"].get(
[stderr]                    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
[stderr]  KeyError: 'LAST_JEDI_COMPLETIONS'
[jsonrpc] <-- completionItem/resolve[348] {"jsonrpc":"2.0","id":348,"result":[]}

Suspected cause

jedi_completion.py::pylsp_completion_item_resolve fails if document.shared_data["LAST_JEDI_COMPLETIONS"] does not exist, which should never happen since it's populated by jedi_completion.py::pylsp_completions if there are any actual completions suggested.

But if we are in the first completion request and there are only completion suggestions from rope_completion.py::pylsp_completions, jedi_completion.py::pylsp_completion_item_resolve is called anyway and fails because document.shared_data["LAST_JEDI_COMPLETIONS"] hasn't been previously populated.

If there had been completion requests before, or the completion list contains suggestions from both providers, it works because jedi_completion.py::pylsp_completion_item_resolve doesn't need to use that data anyway (the selected completion is already given as a parameter)

So the bug is probably one of these two, probably the first:

• If the user has selected a completion from Rope, rope_completion.py::pylsp_completion_item_resolve is the one that should be used
• jedi_completion.py::pylsp_completions should initialize document.shared_data["LAST_JEDI_COMPLETIONS"] even if it doesn't have any completion suggetion (this one would be more of a workaround)

How are these hooks called anyway? I gess pylsp_completions is called for all plugins which implement it and the results are aggregated. But what about pylsp_completion_item_resolve? In a hookspecs.py I read

@hookspec(firstresult=True)
def pylsp_completion_item_resolve(config, workspace, document, completion_item) -> None:
    pass

so I guess the idea is to call them in order until one doesn't return None. In that case, I think the fix would be to make jedi_completion.py::pylsp_completion_item_resolve return None if document.shared_data["LAST_JEDI_COMPLETIONS"] doesn't exist or doesn't contain the completion given as parameter

Finally, there are both rope_completion.py, which implements both hooks, and rope_autoimport.py, which implements pylsp_completions but not pylsp_completion_item_resolve. I've only talked so far about the first, but no idea which one is the one that actually manages autoimports.

What do you think? If you agree with my diagnosis, I can try to submit a PR