vLLM

An open-source Python library by Hugging Face providing pre-trained state-of-the-art transformer models.

A lightweight C/C++ library for running and deploying large language models locally across a range of hardware.

A tool for running and managing large language models locally, providing a simple interface for downloading, serving, and interacting with models.

A library that provides a unified interface for interacting with multiple large language model providers and APIs.

A software service that hosts one or more AI models and processes inference requests by receiving inputs, executing model predictions, and returning outputs through APIs while managing resources, batching, scaling, and hardware acceleration.