Description

LangCheck is a simple, Pythonic toolkit for evaluating LLM applications. It provides metrics for text quality, factual consistency, query relevance, and text structure, along with interactive visualizations and data augmentation tools.

The project is available as a Python package and includes examples, quickstart guidance, and language support spanning English, Japanese, Chinese, and German for many of its metrics.

How we innovate

LangCheck stands out for combining a broad set of LLM evaluation metrics, built-in interactive plotting, and augmentation utilities in a lightweight, Pythonic package.

Use Case / Scenario

1. Evaluate Generated Text

Use LangCheck metrics to assess LLM outputs for qualities such as fluency, toxicity, sentiment, semantic similarity, factual consistency, and answer relevance.

2. Visualize Metric Results

Create interactive scatter plots and histograms to inspect metric values and compare multiple measures across generated outputs.

3. Test and Augment Prompts

Turn metric checks into assertions for automated testing, and use text augmentations such as gender-based rewrites to probe model robustness.

Visit Website