community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

Moved the following modules to new package langchain-community in a backwards compatible fashion: ``` mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community ``` Moved the following to core ``` mv langchain/langchain/utils/json_schema.py core/langchain_core/utils mv langchain/langchain/utils/html.py core/langchain_core/utils mv langchain/langchain/utils/strings.py core/langchain_core/utils cat langchain/langchain/utils/env.py >> core/langchain_core/utils/env.py rm langchain/langchain/utils/env.py ``` See .scripts/community_split/script_integrations.sh for all changes
2025-09-23 19:39:58 +00:00 · 2023-12-11 13:53:30 -08:00
parent c0f4b95aa9
commit ed58eeb9c5
2446 changed files with 171805 additions and 137118 deletions
--- a/libs/community/langchain_community/retrievers/you.py
+++ b/libs/community/langchain_community/retrievers/you.py
@@ -0,0 +1,64 @@
+from typing import Any, Dict, List, Optional
+
+from langchain_core.callbacks import CallbackManagerForRetrieverRun
+from langchain_core.documents import Document
+from langchain_core.pydantic_v1 import root_validator
+from langchain_core.retrievers import BaseRetriever
+from langchain_core.utils import get_from_dict_or_env
+
+
+class YouRetriever(BaseRetriever):
+    """`You` retriever that uses You.com's search API.
+
+    To connect to the You.com api requires an API key which
+    you can get by emailing api@you.com.
+    You can check out our docs at https://documentation.you.com.
+
+    You need to set the environment variable `YDC_API_KEY` for retriever to operate.
+    """
+
+    ydc_api_key: str
+    k: Optional[int] = None
+    n_hits: Optional[int] = None
+    n_snippets_per_hit: Optional[int] = None
+    endpoint_type: str = "web"
+
+    @root_validator(pre=True)
+    def validate_client(
+        cls,
+        values: Dict[str, Any],
+    ) -> Dict[str, Any]:
+        values["ydc_api_key"] = get_from_dict_or_env(
+            values, "ydc_api_key", "YDC_API_KEY"
+        )
+        return values
+
+    def _get_relevant_documents(
+        self, query: str, *, run_manager: CallbackManagerForRetrieverRun
+    ) -> List[Document]:
+        import requests
+
+        headers = {"X-API-Key": self.ydc_api_key}
+        if self.endpoint_type == "web":
+            results = requests.get(
+                f"https://api.ydc-index.io/search?query={query}",
+                headers=headers,
+            ).json()
+
+            docs = []
+            n_hits = self.n_hits or len(results["hits"])
+            for hit in results["hits"][:n_hits]:
+                n_snippets_per_hit = self.n_snippets_per_hit or len(hit["snippets"])
+                for snippet in hit["snippets"][:n_snippets_per_hit]:
+                    docs.append(Document(page_content=snippet))
+                    if self.k is not None and len(docs) >= self.k:
+                        return docs
+            return docs
+        elif self.endpoint_type == "snippet":
+            results = requests.get(
+                f"https://api.ydc-index.io/snippet_search?query={query}",
+                headers=headers,
+            ).json()
+            return [Document(page_content=snippet) for snippet in results]
+        else:
+            raise RuntimeError(f"Invalid endpoint type provided {self.endpoint_type}")