inference

📚 Introduction to Modern Statistics - A college-level open-source textbook with a modern approach highlighting multivariable relationships and simulation-based inference.

Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.

Provider-pluggable orchestration runtime for multi-model AI inference. ( Sakana Fugu style )

Python package for the creation, manipulation, and learning of Causal DAGs

Evidence-aware on-prem LLM inference sizing and TCO calculator

Full in-browser Semantic Search with Huggingface Transformers.js and ElectricSQL's PGlite!

Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

TensorFlow as a Service, a general purpose framework to serve TF models.

A Tool to Build NodeJS and Web Based Expert Systems

A web-based memory usage and performance calculator for Huggingface GGUF models

A GPU defined in software. Runs Llama 3.2 1B at 3.6 tok/sec. Zero dependencies.

AI in A Box

🔀 Bedrock Proxy Endpoint ⇢ Spin up your own custom OpenAI API server endpoint for easy AWS Bedrock inference (using standard baseUrl, and apiKey params)

AIME API Server - Scalable AI Model Inference API Server

Scalable AI Inference Server for CPU and GPU with Node.js | Utilizes llama.cpp and parts of llamafile C/C++ core under the hood.

Util for casting any object to specified model with typescript type inference

Object detection inference with Roboflow Train models on NVIDIA Jetson devices.

Interactive visualization and exploration of large language model (LLM) systems, infrastructure, and inference workflows

A machine learning compiler for GPUs, CPUs, and ML accelerators