community[minor]: add AskNews retriever and AskNews tool (#21581)

We add a tool and retriever for the [AskNews](https://asknews.app)
platform with example notebooks.

The retriever can be invoked with:

```py
from langchain_community.retrievers import AskNewsRetriever

retriever = AskNewsRetriever(k=3)

retriever.invoke("impact of fed policy on the tech sector")
```

To retrieve 3 documents in then news related to fed policy impacts on
the tech sector. The included notebook also includes deeper details
about controlling filters such as category and time, as well as
including the retriever in a chain.

The tool is quite interesting, as it allows the agent to decide how to
obtain the news by forming a query and deciding how far back in time to
look for the news:

```py
from langchain_community.tools.asknews import AskNewsSearch
from langchain import hub
from langchain.agents import AgentExecutor, create_openai_functions_agent
from langchain_openai import ChatOpenAI

tool = AskNewsSearch()

instructions = """You are an assistant."""
base_prompt = hub.pull("langchain-ai/openai-functions-template")
prompt = base_prompt.partial(instructions=instructions)
llm = ChatOpenAI(temperature=0)
asknews_tool = AskNewsSearch()
tools = [asknews_tool]
agent = create_openai_functions_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    verbose=True,
)

agent_executor.invoke({"input": "How is the tech sector being affected by fed policy?"})
```

---------

Co-authored-by: Emre <e@emre.pm>
This commit is contained in:
Robert Caulk 2024-05-21 03:23:06 +02:00 committed by GitHub
parent fc79b372cb
commit 54adcd9e82
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
13 changed files with 762 additions and 0 deletions

View File

@ -0,0 +1,221 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# AskNews\n",
"\n",
"> [AskNews](https://asknews.app) infuses any LLM with the latest global news (or historical news), using a single natural language query. Specifically, AskNews is enriching over 300k articles per day by translating, summarizing, extracting entities, and indexing them into hot and cold vector databases. AskNews puts these vector databases on a low-latency endpoint for you. When you query AskNews, you get back a prompt-optimized string that contains all the most pertinent enrichments (e.g. entities, classifications, translation, summarization). This means that you do not need to manage your own news RAG, and you do not need to worry about how to properly convey news information in a condensed way to your LLM.\n",
"> AskNews is also committed to transparency, which is why our coverage is monitored and diversified across hundreds of countries, 13 languages, and 50 thousand sources. If you'd like to track our source coverage, you can visit our [transparency dashboard](https://asknews.app/en/transparency).\n",
"\n",
"## Setup\n",
"\n",
"The integration lives in the `langchain-community` package. We also need to install the `asknews` package itself.\n",
"\n",
"```bash\n",
"pip install -U langchain-community asknews\n",
"```\n",
"\n",
"We also need to set our AskNews API credentials, which can be generated at the [AskNews console](https://my.asknews.app)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"os.environ[\"ASKNEWS_CLIENT_ID\"] = getpass.getpass()\n",
"os.environ[\"ASKNEWS_CLIENT_SECRET\"] = getpass.getpass()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It's also helpful (but not needed) to set up [LangSmith](https://smith.langchain.com/) for best-in-class observability"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
"# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Usage"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='[1]:\\ntitle: US Stock Market Declines Amid High Interest Rates\\nsummary: The US stock market has experienced a significant decline in recent days, with the S&P 500 index falling 9.94 points or 0.2% to 4320.06. The decline was attributed to interest rates, which are expected to remain high for a longer period. The yield on 10-year Treasury notes rose to 4.44%, the highest level since 2007, which has a negative impact on stock prices. The high interest rates have also affected the technology sector, with companies such as Intel and Microsoft experiencing declines. The auto sector is also experiencing fluctuations, with General Motors and Ford experiencing declines. The labor market is also facing challenges, with workers demanding higher wages and benefits, which could lead to increased inflation. The Federal Reserve is expected to keep interest rates high for a longer period, which could have a negative impact on the stock market. Some economists expect the Fed to raise interest rates again this year, which could lead to further declines in the stock market.\\nsource: ALYAUM Holding Group for Press\\npublished: May 12 2024 13:12\\nLocation: US\\nTechnology: S&P 500\\nQuantity: 9.94 points\\nNumber: 4320.06\\nProduct: 10-year Treasury notes\\nDate: 2007, this year\\nOrganization: General Motors, Fed, Intel, Ford, Microsoft, Federal Reserve\\nclassification: Finance\\nsentiment: -1', metadata={'title': 'الأسهم الأمريكية تتطلع لتعويض خسائرها بعد موجة تراجع كبيرة', 'source': 'https://www.alyaum.com/articles/6529353/%D8%A7%D9%84%D8%A7%D9%82%D8%AA%D8%B5%D8%A7%D8%AF/%D8%A3%D8%B3%D9%88%D8%A7%D9%82-%D8%A7%D9%84%D8%A3%D8%B3%D9%87%D9%85/%D8%A7%D9%84%D8%A3%D8%B3%D9%87%D9%85-%D8%A7%D9%84%D8%A3%D9%85%D8%B1%D9%8A%D9%83%D9%8A%D8%A9-%D8%AA%D8%AA%D8%B7%D9%84%D8%B9-%D9%84%D8%AA%D8%B9%D9%88%D9%8A%D8%B6-%D8%AE%D8%B3%D8%A7%D8%A6%D8%B1%D9%87%D8%A7-%D8%A8%D8%B9%D8%AF-%D9%85%D9%88%D8%AC%D8%A9-%D8%AA%D8%B1%D8%A7%D8%AC%D8%B9-%D9%83%D8%A8%D9%8A%D8%B1%D8%A9', 'images': 'https://www.alyaum.com/uploads/images/2024/05/12/2312237.jpg'}),\n",
" Document(page_content=\"[2]:\\ntitle: US Federal Reserve's Decision to Limit Stock Market Correction\\nsummary: The Federal Reserve of the United States, led by Jerome Powell, has achieved its goal of limiting the correction of the stock market by reducing the balance of the central bank and maintaining massive liquidity injections into the markets to combat various crises that have occurred since the pandemic. Despite April's contraction of around 5%, the stock market has behaved well this week, with most indices showing increases. The Dow Jones and S&P 500 have risen almost 2% after declines of around 5% in April, while the Nasdaq has increased by 1.4% after a decline of over 4% in April. The correction is taking place in an orderly manner, and the market is trying to find a new equilibrium in asset valuations, adapted to a normalized cost of money and a moderate but positive growth framework.\\nsource: okdiario.com\\npublished: May 12 2024 04:45\\nOrganization: Federal Reserve of the United States, Dow Jones, S&P 500, Nasdaq\\nPerson: Jerome Powell\\nEvent: pandemic\\nDate: April\\nclassification: Business\\nsentiment: 1\", metadata={'title': 'Las Bolsas afrontan una corrección ordenada apoyas por la Fed de EEUU', 'source': 'https://okdiario.com/economia/reserva-federal-mantiene-liquidez-asegura-correccion-limitada-bolsas-12798172', 'images': 'https://okdiario.com/img/2023/08/25/bild-powell-subida-de-tipos-interior.jpg'}),\n",
" Document(page_content=\"[3]:\\ntitle: How the Fed's quest for transparency made markets more volatile\\nsummary: The Federal Reserve's increased transparency and communication with the public may be contributing to market volatility, according to some experts. The Fed's forecasting strategy and frequent communication may be causing \\nsource: NBC4 Washington\\npublished: May 11 2024 12:00\\nOrganization: Fed, Federal Reserve\\nclassification: Business\\nsentiment: 0\", metadata={'title': \"How the Fed's quest for transparency made markets more volatile\", 'source': 'https://www.nbcwashington.com/news/business/money-report/how-the-feds-quest-for-transparency-made-markets-more-volatile/3613897', 'images': 'https://media.nbcwashington.com/2024/05/107409380-1714652843711-gettyimages-2151006318-_s2_5018_hwe7dfbl.jpeg?quality=85&strip=all&resize=1200%2C675'})]"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_community.retrievers import AskNewsRetriever\n",
"\n",
"retriever = AskNewsRetriever(k=3)\n",
"\n",
"retriever.invoke(\"impact of fed policy on the tech sector\")"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content=\"[1]:\\ntitle: US Stocks End Week with Modest Gains Ahead of Inflation Data\\nsummary: US stocks ended the week with modest gains, marking another weekly advance for the three main indices, as investors evaluated comments from Federal Reserve officials and awaited crucial inflation data next week. The S&P 500 and Dow Jones experienced slight increases, while the Nasdaq closed almost unchanged. The Dow Jones recorded its largest weekly percentage gain since mid-December. Federal Reserve officials' comments generated expectations, with market participants awaiting inflation data to be released next week. The data includes the Consumer Price Index (CPI) and Producer Price Index (PPI) from the Department of Labor, which is expected to provide insight into progress towards the 2% inflation target.\\nsource: El Cronista\\npublished: May 10 2024 23:35\\nLocation: US\\nOrganization: Dow Jones, Department of Labor, Nasdaq, Federal Reserve\\nDate: next week, mid-December\\nclassification: Business\\nsentiment: 0\", metadata={'title': 'Modesta suba en los mercados a la espera de los datos de inflación', 'source': 'http://www.cronista.com/usa/wall-street-dolar/modesta-suba-en-los-mercados-a-la-espera-de-los-datos-de-inflacion', 'images': 'https://www.cronista.com/files/image/141/141554/5ff7985549d06_600_315!.jpg?s=99126c63cc44ed5c15ed2177cb022f55&d=1712540173'}),\n",
" Document(page_content=\"[2]:\\ntitle: US Stocks End Week on a High Note\\nsummary: The US stock market ended the week on a positive note, with the Dow Jones Industrial Average closing 0.32% higher at 39,512.84, the S&P 500 rising 0.16% to 5,222.68, and the Nasdaq 100 gaining 0.26% to 18,161.18. The three indices all recorded gains for the week, with the Dow leading the way with a 2.2% increase. The Federal Reserve's stance on interest rates was a key factor, with several Fed members expressing caution about cutting rates in the near future. The University of Michigan's consumer sentiment survey showed a decline in May, and consumer inflation expectations also increased, which tempered expectations for rate cuts. Despite this, the market remained resilient and ended the week on a strong note.\\nsource: Investing.com\\npublished: May 10 2024 20:19\\nLocation: US\\nQuantity: 39,512.84\\nOrganization: University of Michigan, Fed, Federal Reserve\\nDate: May\\nclassification: Business\\nsentiment: 1\", metadata={'title': 'Aktien New York Schluss: Erneut höher zum Ende einer starken Börsenwoche', 'source': 'https://de.investing.com/news/economy/aktien-new-york-schluss-erneut-hoher-zum-ende-einer-starken-borsenwoche-2618612', 'images': 'https://i-invdn-com.investing.com/news/LYNXMPED2T082_L.jpg'}),\n",
" Document(page_content=\"[3]:\\ntitle: US Stocks End Week with Slight Gain Despite Inflation Concerns\\nsummary: The US stock market ended the week with a slight gain, despite inflation expectations and skepticism from Federal Reserve members about potential interest rate cuts. The Dow Jones Industrial Average rose 0.24% to 39,480.38, while the S&P 500 gained 0.07% to 5,217.67. The Nasdaq 100 also rose 0.15% to 18,140.02. The week's performance was driven by a number of factors, including the release of inflation expectations and the comments of Federal Reserve members. The University of Michigan's consumer sentiment survey also showed a decline in May, with consumer inflation expectations rising. The stock of 3M, the best-performing Dow stock, rose 2% after analysts at HSBC upgraded it to \\nsource: Yahoo\\npublished: May 10 2024 18:11\\nLocation: US\\nOrganization: University of Michigan, 3M, HSBC, Federal Reserve\\nQuantity: 39,480.38, 5,217.67\\nDate: May\\nclassification: Business\\nsentiment: 0\", metadata={'title': 'Aktien New York: Knapp im Plus gegen Ende einer starken Börsenwoche', 'source': 'https://de.finance.yahoo.com/nachrichten/aktien-new-york-knapp-plus-181143398.html', 'images': 'https://s.yimg.com/cv/apiv2/social/images/yahoo_default_logo-1200x1200.png'})]"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# you have full control on filtering by category, time, pagination, and even the search method you use.\n",
"from datetime import datetime, timedelta\n",
"\n",
"start = (datetime.now() - timedelta(days=7)).timestamp()\n",
"end = datetime.now().timestamp()\n",
"\n",
"retriever = AskNewsRetriever(\n",
" k=3,\n",
" categories=[\"Business\", \"Technology\"],\n",
" start_timestamp=int(start), # defaults to 48 hours ago\n",
" end_timestamp=int(end), # defaults to now\n",
" method=\"kw\", # defaults to \"nl\", natural language, can also be \"kw\" for keyword search\n",
" offset=10, # allows you to paginate results\n",
")\n",
"\n",
"retriever.invoke(\"federal reserve S&P500\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Chaining\n",
"\n",
"We can easily combine this retriever in to a chain."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.output_parsers import StrOutputParser\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.runnables import RunnablePassthrough\n",
"from langchain_openai import ChatOpenAI\n",
"\n",
"prompt = ChatPromptTemplate.from_template(\n",
" \"\"\"The following news articles may come in handy for answering the question:\n",
"\n",
"{context}\n",
"\n",
"Question:\n",
"\n",
"{question}\"\"\"\n",
")\n",
"chain = (\n",
" RunnablePassthrough.assign(context=(lambda x: x[\"question\"]) | retriever)\n",
" | prompt\n",
" | ChatOpenAI(model=\"gpt-4-1106-preview\")\n",
" | StrOutputParser()\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"According to the information provided in the second news article, the impact of Federal Reserve policy on the technology sector has been negative. The article mentions that due to expectations that interest rates will remain high for a longer period, the US stock market has experienced a significant decline. This rise in interest rates has particularly affected the technology sector, with companies such as Intel and Microsoft experiencing declines. High interest rates can lead to increased borrowing costs for businesses, which can dampen investment and spending. In the technology sector, where companies often rely on borrowing to fund research and development and other growth initiatives, higher rates can be especially challenging.\\n\\nTherefore, the Federal Reserve's policy of maintaining high interest rates has had a detrimental effect on tech stocks, contributing to a decrease in their market valuations.\""
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"chain.invoke({\"question\": \"What is the impact of fed policy on the tech sector?\"})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@ -0,0 +1,172 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "a6f91f20",
"metadata": {},
"source": [
"# AskNews"
]
},
{
"cell_type": "markdown",
"id": "5e24a889",
"metadata": {},
"source": [
"> [AskNews](https://asknews.app) infuses any LLM with the latest global news (or historical news), using a single natural language query. Specifically, AskNews is enriching over 300k articles per day by translating, summarizing, extracting entities, and indexing them into hot and cold vector databases. AskNews puts these vector databases on a low-latency endpoint for you. When you query AskNews, you get back a prompt-optimized string that contains all the most pertinent enrichments (e.g. entities, classifications, translation, summarization). This means that you do not need to manage your own news RAG, and you do not need to worry about how to properly convey news information in a condensed way to your LLM.\n",
"> AskNews is also committed to transparency, which is why our coverage is monitored and diversified across hundreds of countries, 13 languages, and 50 thousand sources. If you'd like to track our source coverage, you can visit our [transparency dashboard](https://asknews.app/en/transparency).\n",
"\n",
"## Setup\n",
"\n",
"The integration lives in the `langchain-community` package. We also need to install the `asknews` package itself.\n",
"\n",
"```bash\n",
"pip install -U langchain-community asknews\n",
"```\n",
"\n",
"We also need to set our AskNews API credentials, which can be obtained at the [AskNews console](https://my.asknews.app)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "e0b178a2-8816-40ca-b57c-ccdd86dde9c9",
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"os.environ[\"ASKNEWS_CLIENT_ID\"] = getpass.getpass()\n",
"os.environ[\"ASKNEWS_CLIENT_SECRET\"] = getpass.getpass()"
]
},
{
"cell_type": "markdown",
"id": "1c97218f-f366-479d-8bf7-fe9f2f6df73f",
"metadata": {},
"source": [
"## Usage\n",
"\n",
"Here we show how to use the tool individually."
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "8b3ddfe9-ca79-494c-a7ab-1f56d9407a64",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"\\n<doc>\\n[1]:\\ntitle: Market Awaits Comments From Key Fed Official\\nsummary: The market is awaiting comments from Fed Governor Christopher Waller, but it's not expected to move markets significantly. The recent Consumer Price Index (CPI) report showed slimming inflation figures, leading to a conclusion that inflation is being curbed. This has led to a 'soft landing' narrative, causing bullish sentiment in the stock market, with the Dow, S&P 500, Nasdaq, and Russell 2000 indices near all-time highs. NVIDIA is set to report earnings next week, and despite its 95% year-to-date growth, it remains a Zacks Rank #1 (Strong Buy) stock. The article also mentions upcoming economic data releases, including New and Existing Home Sales, S&P flash PMI Services and Manufacturing, Durable Goods, and Weekly Jobless Claims.\\nsource: Yahoo\\npublished: May 17 2024 14:53\\nOrganization: Nasdaq, Fed, NVIDIA, Zacks\\nPerson: Christopher Waller\\nEvent: Weekly Jobless Claims\\nclassification: Business\\nsentiment: 0\\n</doc>\\n\\n<doc>\\n[2]:\\ntitle: US futures flat as Fed comments cool rate cut optimism\\nsummary: US stock index futures remained flat on Thursday evening, following a weak finish on Wall Street, as Federal Reserve officials warned that bets on interest rate cuts were potentially premature. The Fed officials, including Atlanta Fed President Raphael Bostic, New York Fed President John Williams, and Cleveland Fed President Loretta Mester, stated that the central bank still needed more confidence to cut interest rates, and that the timing of the move remained uncertain. As a result, investors slightly trimmed their expectations for a September rate cut, and the S&P 500 and Nasdaq 100 indexes fell 0.2% and 0.3%, respectively. Meanwhile, Reddit surged 11% after announcing a partnership with OpenAI, while Take-Two Interactive and DXC Technology fell after issuing disappointing earnings guidance.\\nsource: Yahoo\\npublished: May 16 2024 20:08\\nLocation: US, Wall Street\\nDate: September, Thursday\\nOrganization: Atlanta Fed, Cleveland Fed, New York Fed, Fed, Reddit, Take-Two Interactive, DXC Technology, OpenAI, Federal Reserve\\nTitle: President\\nPerson: Loretta Mester, Raphael Bostic, John Williams\\nclassification: Business\\nsentiment: 0\\n</doc>\\n\""
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_community.tools.asknews import AskNewsSearch\n",
"\n",
"tool = AskNewsSearch(max_results=2)\n",
"tool.invoke({\"query\": \"Effect of fed policy on tech sector\"})"
]
},
{
"cell_type": "markdown",
"id": "21c5b56f-0da0-485f-b6f5-38950bae4fd0",
"metadata": {},
"source": [
"## Chaining\n",
"We show here how to use it as part of an agent. We use the OpenAI Functions Agent, so we will need to setup and install the required dependencies for that. We will also use [LangSmith Hub](https://smith.langchain.com/hub) to pull the prompt from, so we will need to install that.\n",
"\n",
"```bash\n",
"pip install -U langchain-openai langchainhub\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "a1c8ea19-7100-407d-8e8c-f037f9317255",
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "520767b8-9e61-4485-840a-d16f1da5eb3a",
"metadata": {
"ExecuteTime": {
"end_time": "2023-10-21T13:15:37.974229Z",
"start_time": "2023-10-21T13:15:10.007898Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'input': 'How is the tech sector being affected by fed policy?',\n",
" 'output': 'The tech sector is being affected by federal policy in various ways, particularly in relation to artificial intelligence (AI) regulation and investment. Here are some recent news articles related to the tech sector and federal policy:\\n\\n1. The US Senate has released a bipartisan AI policy roadmap, addressing areas of consensus and disagreement on AI use and development. The roadmap includes recommendations for intellectual property reforms, funding for AI research, sector-specific rules, and transparency requirements. It also emphasizes the need for increased funding for AI innovation and investments in national defense. [Source: The National Law Review]\\n\\n2. A bipartisan group of US senators, led by Senate Majority Leader Chuck Schumer, has proposed allocating at least $32 billion over the next three years to develop AI and establish safeguards around it. The proposal aims to regulate and promote AI development to maintain US competitiveness and improve quality of life. [Source: Cointelegraph]\\n\\n3. The US administration is planning to restrict the export of advanced AI models to prevent China and Russia from accessing the technology. This move is part of efforts to protect national security and prevent the misuse of AI by foreign powers. [Source: O Cafezinho]\\n\\n4. The US and China have discussed the risks of AI technologies, with the US taking the lead in the AI arms race. The US has proposed a $32 billion increase in federal spending on AI to maintain its lead, despite concerns about stifling innovation. [Source: AOL]\\n\\nThese articles highlight the ongoing discussions and actions related to AI regulation, investment, and export restrictions that are impacting the tech sector in response to federal policy decisions.'}"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain import hub\n",
"from langchain.agents import AgentExecutor, create_openai_functions_agent\n",
"from langchain_community.tools.asknews import AskNewsSearch\n",
"from langchain_openai import ChatOpenAI\n",
"\n",
"prompt = hub.pull(\"hwchase17/openai-functions-agent\")\n",
"llm = ChatOpenAI(temperature=0)\n",
"asknews_tool = AskNewsSearch()\n",
"tools = [asknews_tool]\n",
"agent = create_openai_functions_agent(llm, tools, prompt)\n",
"agent_executor = AgentExecutor(agent=agent, tools=tools)\n",
"agent_executor.invoke({\"input\": \"How is the tech sector being affected by fed policy?\"})"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "917f2045",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

View File

@ -28,6 +28,9 @@ if TYPE_CHECKING:
from langchain_community.retrievers.arxiv import (
ArxivRetriever,
)
from langchain_community.retrievers.asknews import (
AskNewsRetriever,
)
from langchain_community.retrievers.azure_ai_search import (
AzureAISearchRetriever,
AzureCognitiveSearchRetriever,
@ -143,6 +146,7 @@ _module_lookup = {
"AmazonKnowledgeBasesRetriever": "langchain_community.retrievers.bedrock",
"ArceeRetriever": "langchain_community.retrievers.arcee",
"ArxivRetriever": "langchain_community.retrievers.arxiv",
"AskNewsRetriever": "langchain_community.retrievers.asknews",
"AzureAISearchRetriever": "langchain_community.retrievers.azure_ai_search",
"AzureCognitiveSearchRetriever": "langchain_community.retrievers.azure_ai_search",
"BM25Retriever": "langchain_community.retrievers.bm25",
@ -196,6 +200,7 @@ __all__ = [
"AmazonKnowledgeBasesRetriever",
"ArceeRetriever",
"ArxivRetriever",
"AskNewsRetriever",
"AzureAISearchRetriever",
"AzureCognitiveSearchRetriever",
"BM25Retriever",

View File

@ -0,0 +1,146 @@
import os
import re
from typing import Any, Dict, List, Literal, Optional
from langchain_core.callbacks import (
AsyncCallbackManagerForRetrieverRun,
CallbackManagerForRetrieverRun,
)
from langchain_core.documents import Document
from langchain_core.retrievers import BaseRetriever
class AskNewsRetriever(BaseRetriever):
"""AskNews retriever."""
k: int = 10
offset: int = 0
start_timestamp: Optional[int] = None
end_timestamp: Optional[int] = None
method: Literal["nl", "kw"] = "nl"
categories: List[
Literal[
"All",
"Business",
"Crime",
"Politics",
"Science",
"Sports",
"Technology",
"Military",
"Health",
"Entertainment",
"Finance",
"Culture",
"Climate",
"Environment",
"World",
]
] = ["All"]
historical: bool = False
similarity_score_threshold: float = 0.5
kwargs: Optional[Dict[str, Any]] = {}
client_id: Optional[str] = None
client_secret: Optional[str] = None
def _get_relevant_documents(
self, query: str, *, run_manager: CallbackManagerForRetrieverRun
) -> List[Document]:
"""Get documents relevant to a query.
Args:
query: String to find relevant documents for
run_manager: The callbacks handler to use
Returns:
List of relevant documents
"""
try:
from asknews_sdk import AskNewsSDK
except ImportError:
raise ImportError(
"AskNews python package not found. "
"Please install it with `pip install asknews`."
)
an_client = AskNewsSDK(
client_id=self.client_id or os.environ["ASKNEWS_CLIENT_ID"],
client_secret=self.client_secret or os.environ["ASKNEWS_CLIENT_SECRET"],
scopes=["news"],
)
response = an_client.news.search_news(
query=query,
n_articles=self.k,
start_timestamp=self.start_timestamp,
end_timestamp=self.end_timestamp,
method=self.method,
categories=self.categories,
historical=self.historical,
similarity_score_threshold=self.similarity_score_threshold,
offset=self.offset,
doc_start_delimiter="<doc>",
doc_end_delimiter="</doc>",
return_type="both",
**self.kwargs,
)
return self._extract_documents(response)
async def _aget_relevant_documents(
self, query: str, *, run_manager: AsyncCallbackManagerForRetrieverRun
) -> List[Document]:
"""Asynchronously get documents relevant to a query.
Args:
query: String to find relevant documents for
run_manager: The callbacks handler to use
Returns:
List of relevant documents
"""
try:
from asknews_sdk import AsyncAskNewsSDK
except ImportError:
raise ImportError(
"AskNews python package not found. "
"Please install it with `pip install asknews`."
)
an_client = AsyncAskNewsSDK(
client_id=self.client_id or os.environ["ASKNEWS_CLIENT_ID"],
client_secret=self.client_secret or os.environ["ASKNEWS_CLIENT_SECRET"],
scopes=["news"],
)
response = await an_client.news.search_news(
query=query,
n_articles=self.k,
start_timestamp=self.start_timestamp,
end_timestamp=self.end_timestamp,
method=self.method,
categories=self.categories,
historical=self.historical,
similarity_score_threshold=self.similarity_score_threshold,
offset=self.offset,
return_type="both",
doc_start_delimiter="<doc>",
doc_end_delimiter="</doc>",
**self.kwargs,
)
return self._extract_documents(response)
def _extract_documents(self, response: Any) -> List[Document]:
"""Extract documents from an api response."""
from asknews_sdk.dto.news import SearchResponse
sr: SearchResponse = response
matches = re.findall(r"<doc>(.*?)</doc>", sr.as_string, re.DOTALL)
docs = [
Document(
page_content=matches[i].strip(),
metadata={
"title": sr.as_dicts[i].title,
"source": str(sr.as_dicts[i].article_url)
if sr.as_dicts[i].article_url
else None,
"images": sr.as_dicts[i].image_url,
},
)
for i in range(len(matches))
]
return docs

View File

@ -46,6 +46,9 @@ if TYPE_CHECKING:
from langchain_community.tools.arxiv.tool import (
ArxivQueryRun,
)
from langchain_community.tools.asknews.tool import (
AskNewsSearch,
)
from langchain_community.tools.azure_ai_services import (
AzureAiServicesDocumentIntelligenceTool,
AzureAiServicesImageAnalysisTool,
@ -324,6 +327,7 @@ __all__ = [
"AIPluginTool",
"APIOperation",
"ArxivQueryRun",
"AskNewsSearch",
"AzureAiServicesDocumentIntelligenceTool",
"AzureAiServicesImageAnalysisTool",
"AzureAiServicesSpeechToTextTool",
@ -467,6 +471,7 @@ _module_lookup = {
"AIPluginTool": "langchain_community.tools.plugin",
"APIOperation": "langchain_community.tools.openapi.utils.api_models",
"ArxivQueryRun": "langchain_community.tools.arxiv.tool",
"AskNewsSearch": "langchain_community.tools.asknews.tool",
"AzureAiServicesDocumentIntelligenceTool": "langchain_community.tools.azure_ai_services", # noqa: E501
"AzureAiServicesImageAnalysisTool": "langchain_community.tools.azure_ai_services",
"AzureAiServicesSpeechToTextTool": "langchain_community.tools.azure_ai_services",

View File

@ -0,0 +1,7 @@
"""AskNews API toolkit."""
from langchain_community.tools.asknews.tool import (
AskNewsSearch,
)
__all__ = ["AskNewsSearch"]

View File

@ -0,0 +1,80 @@
"""
Tool for the AskNews API.
To use this tool, you must first set your credentials as environment variables:
ASKNEWS_CLIENT_ID
ASKNEWS_CLIENT_SECRET
"""
from typing import Optional, Type
from langchain_core.callbacks import (
AsyncCallbackManagerForToolRun,
CallbackManagerForToolRun,
)
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.tools import BaseTool
from langchain_community.utilities.asknews import AskNewsAPIWrapper
class SearchInput(BaseModel):
"""Input for the AskNews Search tool."""
query: str = Field(
description="Search query to be used for finding real-time or historical news "
"information."
)
hours_back: Optional[int] = Field(
0,
description="If the Assistant deems that the event may have occurred more "
"than 48 hours ago, it estimates the number of hours back to search. For "
"example, if the event was one month ago, the Assistant may set this to 720. "
"One week would be 168. The Assistant can estimate up to on year back (8760).",
)
class AskNewsSearch(BaseTool):
"""Tool that searches the AskNews API."""
name: str = "asknews_search"
description: str = (
"This tool allows you to perform a search on up-to-date news and historical "
"news. If you needs news from more than 48 hours ago, you can estimate the "
"number of hours back to search."
)
api_wrapper: AskNewsAPIWrapper = Field(default_factory=AskNewsAPIWrapper) # type: ignore[arg-type]
max_results: int = 10
args_schema: Type[BaseModel] = SearchInput
def _run(
self,
query: str,
hours_back: int = 0,
run_manager: Optional[CallbackManagerForToolRun] = None,
) -> str:
"""Use the tool."""
try:
return self.api_wrapper.search_news(
query,
hours_back=hours_back,
max_results=self.max_results,
)
except Exception as e:
return repr(e)
async def _arun(
self,
query: str,
hours_back: int = 0,
run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
) -> str:
"""Use the tool asynchronously."""
try:
return await self.api_wrapper.asearch_news(
query,
hours_back=hours_back,
max_results=self.max_results,
)
except Exception as e:
return repr(e)

View File

@ -3,6 +3,7 @@
Other LangChain classes use **Utilities** to interact with third-part systems
and packages.
"""
import importlib
from typing import TYPE_CHECKING, Any
@ -19,6 +20,9 @@ if TYPE_CHECKING:
from langchain_community.utilities.arxiv import (
ArxivAPIWrapper,
)
from langchain_community.utilities.asknews import (
AskNewsAPIWrapper,
)
from langchain_community.utilities.awslambda import (
LambdaWrapper,
)
@ -173,6 +177,7 @@ __all__ = [
"ApifyWrapper",
"ArceeWrapper",
"ArxivAPIWrapper",
"AskNewsAPIWrapper",
"AudioStream",
"BibtexparserWrapper",
"BingSearchAPIWrapper",
@ -234,6 +239,7 @@ _module_lookup = {
"ApifyWrapper": "langchain_community.utilities.apify",
"ArceeWrapper": "langchain_community.utilities.arcee",
"ArxivAPIWrapper": "langchain_community.utilities.arxiv",
"AskNewsAPIWrapper": "langchain_community.utilities.asknews",
"AudioStream": "langchain_community.utilities.nvidia_riva",
"BibtexparserWrapper": "langchain_community.utilities.bibtex",
"BingSearchAPIWrapper": "langchain_community.utilities.bing_search",

View File

@ -0,0 +1,115 @@
"""Util that calls AskNews api."""
from __future__ import annotations
from datetime import datetime, timedelta
from typing import Any, Dict, Optional
from langchain_core.pydantic_v1 import BaseModel, Extra, root_validator
from langchain_core.utils import get_from_dict_or_env
class AskNewsAPIWrapper(BaseModel):
"""Wrapper for AskNews API."""
asknews_sync: Any #: :meta private:
asknews_async: Any #: :meta private:
asknews_client_id: Optional[str] = None
"""Client ID for the AskNews API."""
asknews_client_secret: Optional[str] = None
"""Client Secret for the AskNews API."""
class Config:
"""Configuration for this pydantic object."""
extra = Extra.forbid
@root_validator()
def validate_environment(cls, values: Dict) -> Dict:
"""Validate that api credentials and python package exists in environment."""
asknews_client_id = get_from_dict_or_env(
values, "asknews_client_id", "ASKNEWS_CLIENT_ID"
)
asknews_client_secret = get_from_dict_or_env(
values, "asknews_client_secret", "ASKNEWS_CLIENT_SECRET"
)
try:
import asknews_sdk
except ImportError:
raise ImportError(
"AskNews python package not found. "
"Please install it with `pip install asknews`."
)
an_sync = asknews_sdk.AskNewsSDK(
client_id=asknews_client_id,
client_secret=asknews_client_secret,
scopes=["news"],
)
an_async = asknews_sdk.AsyncAskNewsSDK(
client_id=asknews_client_id,
client_secret=asknews_client_secret,
scopes=["news"],
)
values["asknews_sync"] = an_sync
values["asknews_async"] = an_async
values["asknews_client_id"] = asknews_client_id
values["asknews_client_secret"] = asknews_client_secret
return values
def search_news(
self, query: str, max_results: int = 10, hours_back: int = 0
) -> str:
"""Search news in AskNews API synchronously."""
if hours_back > 48:
method = "kw"
historical = True
start = int((datetime.now() - timedelta(hours=hours_back)).timestamp())
stop = int(datetime.now().timestamp())
else:
historical = False
method = "nl"
start = None
stop = None
response = self.asknews_sync.news.search_news(
query=query,
n_articles=max_results,
method=method,
historical=historical,
start_timestamp=start,
end_timestamp=stop,
return_type="string",
)
return response.as_string
async def asearch_news(
self, query: str, max_results: int = 10, hours_back: int = 0
) -> str:
"""Search news in AskNews API asynchronously."""
if hours_back > 48:
method = "kw"
historical = True
start = int((datetime.now() - timedelta(hours=hours_back)).timestamp())
stop = int(datetime.now().timestamp())
else:
historical = False
method = "nl"
start = None
stop = None
response = await self.asknews_async.news.search_news(
query=query,
n_articles=max_results,
method=method,
historical=historical,
start_timestamp=start,
end_timestamp=stop,
return_type="string",
)
return response.as_string

View File

@ -5,6 +5,7 @@ EXPECTED_ALL = [
"AmazonKnowledgeBasesRetriever",
"ArceeRetriever",
"ArxivRetriever",
"AskNewsRetriever",
"AzureAISearchRetriever",
"AzureCognitiveSearchRetriever",
"BreebsRetriever",

View File

@ -9,6 +9,7 @@ EXPECTED_ALL = [
"AIPluginTool",
"APIOperation",
"ArxivQueryRun",
"AskNewsSearch",
"AzureAiServicesDocumentIntelligenceTool",
"AzureAiServicesImageAnalysisTool",
"AzureAiServicesSpeechToTextTool",

View File

@ -1,4 +1,5 @@
"""Test the public API of the tools package."""
from langchain_community.tools import __all__ as public_api
_EXPECTED = [
@ -10,6 +11,7 @@ _EXPECTED = [
"AIPluginTool",
"APIOperation",
"ArxivQueryRun",
"AskNewsSearch",
"AzureAiServicesDocumentIntelligenceTool",
"AzureAiServicesImageAnalysisTool",
"AzureAiServicesSpeechToTextTool",

View File

@ -5,6 +5,7 @@ EXPECTED_ALL = [
"ApifyWrapper",
"ArceeWrapper",
"ArxivAPIWrapper",
"AskNewsAPIWrapper",
"BibtexparserWrapper",
"BingSearchAPIWrapper",
"BraveSearchWrapper",