Does anyone want an LLM-based autosuggester?

[lstein]

I've written an AutoSuggest class which suggests prompt completions using a locally-installed or remotely-hosted large language model. It can be customized to produce different types of completions depending on the writing task (coding, fiction, documentation) and is (optionally) aware of the context in which the prompt is being written.

Is there any interest in my contributing this to the repo as a pull request?

For further information, the class will introduce a number of package dependencies:

• langchain
• langchain_core
• PyEnchant

I haven't done this yet, but I'm planning to try an AutoCompleter as well which will provide a pop-up menu of next tokens, ordered by their probabilities. I'm not sure how useful this will be, however.

[lin-calvin]

Cool idea! But what about litellm? It's much lighter than langchain, so putting this in contrib would be nice. But wouldn't adding it cause the pip install to become slower?

[asmeurer]

Related #1913

[lstein]

Cool idea! But what about litellm? It's much lighter than langchain, so putting this in contrib would be nice. But wouldn't adding it cause the pip install to become slower?

The code doesn't use any of langchain's fancier features, so swapping out langchain for lightllm is easily doable.

I hadn't heard about lightllm until now. It seems to have an order of magnitude fewer github stars and forks than langchain, but is it up and coming?

Regarding pip install, I don't like adding a bunch of dependencies to a project just in order to support one of its lesser-used features. I could make this an optional feature in prompt_toolkit's pyproject.toml:

pip install .[aisuggest]

So wouldn't slow the default pip install at all.

[lin-calvin]

Cool idea! But what about litellm? It's much lighter than langchain, so putting this in contrib would be nice. But wouldn't adding it cause the pip install to become slower?

The code doesn't use any of langchain's fancier features, so swapping out langchain for lightllm is easily doable.

I hadn't heard about lightllm until now. It seems to have an order of magnitude fewer github stars and forks than langchain, but is it up and coming?

Regarding pip install, I don't like adding a bunch of dependencies to a project just in order to support one of its lesser-used features. I could make this an optional feature in prompt_toolkit's pyproject.toml:

pip install .[aisuggest]

So wouldn't slow the default pip install at all.

I means litellm not lightllm

[lstein]

I just did a pull request: #1995 . I did have a look at litellm but it is a pretty low-level API that doesn't have the many features brought by langchain, such as support for tools, agents, and chat memory.

The size differences are not all that significant either. litellm with all its dependencies consumes 34Mb of disk. langchain and all its dependencies use 54M.

Porting the autosuggester to litellm would not be particularly difficult, and I am happy to do so if there is demand for it.

[lin-calvin]

I just did a pull request: #1995 . I did have a look at litellm but it is a pretty low-level API that doesn't have the many features brought by langchain, such as support for tools, agents, and chat memory.

The size differences are not all that significant either. litellm with all its dependencies consumes 34Mb of disk. langchain and all its dependencies use 54M.

Porting the autosuggester to litellm would not be particularly difficult, and I am happy to do so if there is demand for it.

What about plain openai sdk?

[lstein]

I just did a pull request: #1995 . I did have a look at litellm but it is a pretty low-level API that doesn't have the many features brought by langchain, such as support for tools, agents, and chat memory.
The size differences are not all that significant either. litellm with all its dependencies consumes 34Mb of disk. langchain and all its dependencies use 54M.
Porting the autosuggester to litellm would not be particularly difficult, and I am happy to do so if there is demand for it.

What about plain openai sdk?

Correct me if I'm wrong, but wouldn't using the openai SDK lock people into using the OpenAI API? I want people to be able to swap in anthropic, local llama, gemini, and all the other alternatives out there.

[lin-calvin]

I just did a pull request: #1995 . I did have a look at litellm but it is a pretty low-level API that doesn't have the many features brought by langchain, such as support for tools, agents, and chat memory.
The size differences are not all that significant either. litellm with all its dependencies consumes 34Mb of disk. langchain and all its dependencies use 54M.
Porting the autosuggester to litellm would not be particularly difficult, and I am happy to do so if there is demand for it.

What about plain openai sdk?

Correct me if I'm wrong, but wouldn't using the openai SDK lock people into using the OpenAI API? I want people to be able to swap in anthropic, local llama, gemini, and all the other alternatives out there.

It seems that all of the world support the openai-like api

[robgale]

I would really like to use an LLM based feature. Was that ever implemented? I don't see it in the readme.

[lstein]

Hi, No, the prompt-toolkit team deliberated a bit and decided that the LLM autocomplete feature was out of scope for incorporation into the base package. They suggested that I make it an external project that people could download and install. Unfortunately I haven't done the work to make this happen yet. You should be able to download and install the pull request <#1995> to get the feature. It looks like there are no conflicts with the base branch. If you need help doing this, I'm happy to lend a hand. Lincoln

…

On Tue, Jan 27, 2026 at 4:40 PM robgale ***@***.***> wrote: *robgale* left a comment (prompt-toolkit/python-prompt-toolkit#1993) <#1993 (comment)> I would really like to use an LLM based feature. Was that ever implemented? I don't see it in the readme. — Reply to this email directly, view it on GitHub <#1993 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3EVLBUCZZAFZEQ7QKFFD4I7LNRAVCNFSM6AAAAACTC7SBPKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTQMBXGY3TGNBWGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Lincoln Stein (he/him/his) Scientific Director (Acting), and Head, Adaptive Oncology Ontario Institute for Cancer Research Faculty Affiliate, Vector Institute Professor, Molecular Genetics, University of Toronto ***@***.*** Tel: 416-673-8514 | Cell: 416-817-8240 Executive Assistant Michelle Xin ***@***.*** Tel: 647-260-7927 Ontario Institute for Cancer Research MaRS Centre, 661 University Avenue, Suite 510, Toronto, Ontario, Canada M5G 0A3 www.oicr.on.ca Collaborate. Translate. Change lives. This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.