docs: run how-to guides in CI (#27615)

Add how-to guides to [Run notebooks
job](https://github.com/langchain-ai/langchain/actions/workflows/run_notebooks.yml)
and fix existing notebooks.

- As with tutorials, cassettes must be updated when HTTP calls in guides
change (by running existing
[script](https://github.com/langchain-ai/langchain/blob/master/docs/scripts/update_cassettes.sh)).
- Cassettes now total ~62mb over 474 files.
- `docs/scripts/prepare_notebooks_for_ci.py` lists a number of notebooks
that do not run (e.g., due to requiring additional infra, slowness,
requiring `input()`, etc.).
This commit is contained in:
ccurme
2024-10-30 12:35:38 -04:00
committed by GitHub
parent 88bfd60b03
commit 595dc592c9
420 changed files with 2333 additions and 321 deletions

View File

@@ -1,5 +1,24 @@
#!/bin/bash
# Use this script to update cassettes for a notebook. The script does the following:
#
# 1. Delete existing cassettes for the specified notebook
# 2. Pre-download and cache nltk and tiktoken files
# 3. Modify the notebook to generate cassettes for each cell.
# 4. Execute the notebook.
#
# Important: make sure the notebook is in a clean state, with any desired changes
# staged or committed. The script will modify the notebook in place, and these
# modifications should be discarded after the cassettes are generated.
#
# Usage:
# In monorepo env, `poetry install --with dev,test`
# `./docs/scripts/update_cassettes.sh path/to/notebook`
# e.g., `./docs/scripts/update_cassettes.sh docs/docs/how_to/tool_choice.ipynb`
#
# Make sure to set any env vars required by the notebook.
# Get the working directory from the input argument, default to 'all' if not provided
WORKING_DIRECTORY=${1:-all}
@@ -21,8 +40,8 @@ delete_cassettes() {
delete_cassettes "$WORKING_DIRECTORY"
# Pre-download tiktoken files
echo "Pre-downloading tiktoken files..."
poetry run python docs/scripts/download_tiktoken.py
echo "Pre-downloading nltk and tiktoken files..."
poetry run python docs/scripts/cache_data.py
# Prepare notebooks
echo "Preparing notebooks for CI..."