mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-15 19:38:57 +00:00
148 lines
4.1 KiB
Plaintext
148 lines
4.1 KiB
Plaintext
---
|
|
sidebar_position: 6
|
|
---
|
|
|
|
# Testing
|
|
|
|
All of our packages have unit tests and integration tests, and we favor unit tests over integration tests.
|
|
|
|
Unit tests run on every pull request, so they should be fast and reliable.
|
|
|
|
Integration tests run once a day, and they require more setup, so they should be reserved for confirming interface points with external services.
|
|
|
|
## Unit Tests
|
|
|
|
Unit tests cover modular logic that does not require calls to outside APIs.
|
|
If you add new logic, please add a unit test.
|
|
|
|
To install dependencies for unit tests:
|
|
|
|
```bash
|
|
poetry install --with test
|
|
```
|
|
|
|
To run unit tests:
|
|
|
|
```bash
|
|
make test
|
|
```
|
|
|
|
To run unit tests in Docker:
|
|
|
|
```bash
|
|
make docker_tests
|
|
```
|
|
|
|
To run a specific test:
|
|
|
|
```bash
|
|
TEST_FILE=tests/unit_tests/test_imports.py make test
|
|
```
|
|
|
|
## Integration Tests
|
|
|
|
Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
|
|
If you add support for a new external API, please add a new integration test.
|
|
|
|
**Warning:** Almost no tests should be integration tests.
|
|
|
|
Tests that require making network connections make it difficult for other
|
|
developers to test the code.
|
|
|
|
Instead favor relying on `responses` library and/or mock.patch to mock
|
|
requests using small fixtures.
|
|
|
|
To install dependencies for integration tests:
|
|
|
|
```bash
|
|
poetry install --with test,test_integration
|
|
```
|
|
|
|
To run integration tests:
|
|
|
|
```bash
|
|
make integration_tests
|
|
```
|
|
|
|
### Prepare
|
|
|
|
The integration tests use several search engines and databases. The tests
|
|
aim to verify the correct behavior of the engines and databases according to
|
|
their specifications and requirements.
|
|
|
|
To run some integration tests, such as tests located in
|
|
`tests/integration_tests/vectorstores/`, you will need to install the following
|
|
software:
|
|
|
|
- Docker
|
|
- Python 3.8.1 or later
|
|
|
|
Any new dependencies should be added by running:
|
|
|
|
```bash
|
|
# add package and install it after adding:
|
|
poetry add tiktoken@latest --group "test_integration" && poetry install --with test_integration
|
|
```
|
|
|
|
Before running any tests, you should start a specific Docker container that has all the
|
|
necessary dependencies installed. For instance, we use the `elasticsearch.yml` container
|
|
for `test_elasticsearch.py`:
|
|
|
|
```bash
|
|
cd tests/integration_tests/vectorstores/docker-compose
|
|
docker-compose -f elasticsearch.yml up
|
|
```
|
|
|
|
For environments that requires more involving preparation, look for `*.sh`. For instance,
|
|
`opensearch.sh` builds a required docker image and then launch opensearch.
|
|
|
|
|
|
### Prepare environment variables for local testing:
|
|
|
|
- copy `tests/integration_tests/.env.example` to `tests/integration_tests/.env`
|
|
- set variables in `tests/integration_tests/.env` file, e.g `OPENAI_API_KEY`
|
|
|
|
Additionally, it's important to note that some integration tests may require certain
|
|
environment variables to be set, such as `OPENAI_API_KEY`. Be sure to set any required
|
|
environment variables before running the tests to ensure they run correctly.
|
|
|
|
### Recording HTTP interactions with pytest-vcr
|
|
|
|
Some of the integration tests in this repository involve making HTTP requests to
|
|
external services. To prevent these requests from being made every time the tests are
|
|
run, we use pytest-vcr to record and replay HTTP interactions.
|
|
|
|
When running tests in a CI/CD pipeline, you may not want to modify the existing
|
|
cassettes. You can use the --vcr-record=none command-line option to disable recording
|
|
new cassettes. Here's an example:
|
|
|
|
```bash
|
|
pytest --log-cli-level=10 tests/integration_tests/vectorstores/test_pinecone.py --vcr-record=none
|
|
pytest tests/integration_tests/vectorstores/test_elasticsearch.py --vcr-record=none
|
|
|
|
```
|
|
|
|
### Run some tests with coverage:
|
|
|
|
```bash
|
|
pytest tests/integration_tests/vectorstores/test_elasticsearch.py --cov=langchain --cov-report=html
|
|
start "" htmlcov/index.html || open htmlcov/index.html
|
|
|
|
```
|
|
|
|
## Coverage
|
|
|
|
Code coverage (i.e. the amount of code that is covered by unit tests) helps identify areas of the code that are potentially more or less brittle.
|
|
|
|
Coverage requires the dependencies for integration tests:
|
|
|
|
```bash
|
|
poetry install --with test_integration
|
|
```
|
|
|
|
To get a report of current coverage, run the following:
|
|
|
|
```bash
|
|
make coverage
|
|
```
|