mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-19 05:13:46 +00:00
community[patch]: optimize xinference llm import (#25809)
Thank you for contributing to LangChain! - [ ] **PR title**: "community: optimize xinference llm import" - [ ] **PR message**: - **Description:** from xinferece_client import RESTfulClient when there is no importing xinference. - **Dependencies:** xinferece_client - **Why do so:** the total xinference(pip install xinference[all]) is too heavy for installing, let alone it is useless for langchain user except RESTfulClient. The modification has maintained consistency with the xinference embeddings [embeddings/xinference](../blob/master/libs/community/langchain_community/embeddings/xinference.py#L89).
This commit is contained in:
parent
9decd0b243
commit
3555882a0d
@ -17,6 +17,12 @@ class Xinference(LLM):
|
||||
|
||||
pip install "xinference[all]"
|
||||
|
||||
If you're simply using the services provided by Xinference, you can utilize the xinference_client package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install xinference_client
|
||||
|
||||
Check out: https://github.com/xorbitsai/inference
|
||||
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers
|
||||
|
||||
@ -91,10 +97,13 @@ class Xinference(LLM):
|
||||
):
|
||||
try:
|
||||
from xinference.client import RESTfulClient
|
||||
except ImportError:
|
||||
try:
|
||||
from xinference_client import RESTfulClient
|
||||
except ImportError as e:
|
||||
raise ImportError(
|
||||
"Could not import RESTfulClient from xinference. Please install it"
|
||||
" with `pip install xinference`."
|
||||
" with `pip install xinference` or `pip install xinference_client`."
|
||||
) from e
|
||||
|
||||
model_kwargs = model_kwargs or {}
|
||||
|
Loading…
Reference in New Issue
Block a user