diff --git a/docs/docs/concepts/evaluation.mdx b/docs/docs/concepts/evaluation.mdx new file mode 100644 index 00000000000..274ef98367c --- /dev/null +++ b/docs/docs/concepts/evaluation.mdx @@ -0,0 +1,17 @@ +# Evaluation + + +Evaluation is the process of assessing the performance and effectiveness of your LLM-powered applications. +It involves testing the model's responses against a set of predefined criteria or benchmarks to ensure it meets the desired quality standards and fulfills the intended purpose. +This process is vital for building reliable applications. + +![](/img/langsmith_evaluate.png) + +[LangSmith](https://docs.smith.langchain.com/) helps with this process in a few ways: + +- It makes it easier to create and curate datasets via its tracing and annotation features +- It provides an evaluation framework that helps you define metrics and run your app against your dataset +- It allows you to track results over time and automatically run your evaluators on a schedule or as part of CI/Code + +To learn more, check out [this LangSmith guide](https://docs.smith.langchain.com/concepts/evaluation). + diff --git a/docs/docs/concepts/index.mdx b/docs/docs/concepts/index.mdx index 6b143ef4147..06f12671627 100644 --- a/docs/docs/concepts/index.mdx +++ b/docs/docs/concepts/index.mdx @@ -41,6 +41,8 @@ The conceptual guide will not cover step-by-step instructions or specific implem - **[Output Parsers](/docs/concepts/output_parsers)**: Output parsers are responsible for taking the output of a model and transforming it into a more suitable format for downstream tasks. Output parsers were primarily useful prior to the general availability of [chat models](/docs/concepts/chat_models) that natively support [tool calling](/docs/concepts/tool_calling) and [structured outputs](/docs/concepts/structured_outputs). - **[Few shot prompting](/docs/concepts/few_shot_prompting)**: Few-shot prompting is a technique used improve the performance of language models by providing them with a few examples of the task they are expected to perform. - **[Example Selectors](/docs/concepts/example_selectors)**: Example selectors are used to select examples from a dataset based on a given input. They can be used to select examples randomly, by semantic similarity, or based on some other constraints. Example selectors are used in few-shot prompting to select examples for a prompt. +- **[Tracing](/docs/concepts/tracing)**: Tracing is the process of recording the steps that an application takes to go from input to output. Tracing is essential for debugging and diagnosing issues in complex applications. For more information on tracing in LangChain, see the [LangSmith documentation](https://docs.smith.langchain.com/concepts/tracing). +- **[Evaluation](/docs/concepts/evaluation)**: Evaluation is the process of assessing the performance and effectiveness of your LLM-powered applications. It involves testing the model's responses against a set of predefined criteria or benchmarks to ensure it meets the desired quality standards and fulfills the intended purpose. This process is vital for building reliable applications. For more information on evaluation in LangChain, see the [LangSmith documentation](https://docs.smith.langchain.com/concepts/evaluation). ## Glossary diff --git a/docs/docs/concepts/tracing.mdx b/docs/docs/concepts/tracing.mdx new file mode 100644 index 00000000000..659992eeb95 --- /dev/null +++ b/docs/docs/concepts/tracing.mdx @@ -0,0 +1,10 @@ +# Tracing + + + +A trace is essentially a series of steps that your application takes to go from input to output. +Traces contain individual steps called `runs`. These can be individual calls from a model, retriever, +tool, or sub-chains. +Tracing gives you observability inside your chains and agents, and is vital in diagnosing issues. + +For a deeper dive, check out [this LangSmith conceptual guide](https://docs.smith.langchain.com/concepts/tracing).