Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready and real time inference.
-
Updated
Feb 10, 2023 - Python
#
batch-inference
Here are 30 public repositories matching this topic...
Production-ready batch inference for Cohere’s Arabic/English ASR model, with optimized Silero VAD, multi-file GPU batching, subtitles, and optional word-level timestamps.
-
Updated
Jul 15, 2026 - Python
Optimize and run consistent AI Functions.
s3 prompt data-engineering data-pipelines observability evaluation-framework synthetic-data unstructured-data mlops batch-inference prompt-engineering llm-inference evals distributed-inference evals-engineering
-
Updated
Oct 2, 2026 - Python
RWKV7 推理服务 | 高并发批量推理，动态扩缩容，兼容OpenAI API，CUDA优化，低显存部署大模型推理 RWKV linear attention batch inference server — auto-queue, dynamic capacity, OpenAI compatible.
high-throughput fastapi linear-attention batch-inference dynamic-batching rwkv llm-serving llm-inference rwkv7 openai-compatible
-
Updated
Jun 20, 2026 - Python
Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.
-
Updated
Sep 25, 2026 - Python
Serve pytorch inference requests using batching with redis for faster performance.
-
Updated
Apr 11, 2021 - Python
Self-hosted batch LLM pipeline for analyzing customer feedback from Excel. Upload xlsx, describe the task, configure output fields — get structured results. Works with any OpenAI-compatible API and Ollama.
sentiment-analysis text-analytics customer-feedback fraud-detection no-code human-in-the-loop unstructured-data structured-output excel-automation russian-nlp batch-inference prompt-engineering llm-pipeline ai-data-analysis
-
Updated
Apr 24, 2026 - Python
Fake Anthropic / Bedrock (InvokeModel, Converse, batch inference with a bundled S3 emulation) / OpenAI server for zero-cost human/AI-in-the-loop LLM debugging, a scripted test harness (scenario rules, latency and rate-limit simulation, pytest fixture) and a cross-provider relay bridge
testing debugging proxy mock-server s3 pytest bedrock openai gpt boto3 test-harness grok human-in-the-loop claude fastapi batch-inference llm anthropic cross-provider
-
Updated
Sep 30, 2026 - Python
Tutorial: mass-parallelized RAI compliance checks with vLLM batch inference on Cloud TPU v5e, using Gemma 4 as an LLM-as-a-Judge
-
Updated
Aug 4, 2026 - Python
Queue-driven, scale-from-zero GPU inference for any Kubernetes — bursts to cross-region VMs when GPUs run dry
python kubernetes machine-learning gpu helm inference self-hosted gpu-computing autoscaling inference-server model-serving ml-infrastructure mlops scale-to-zero serverless-gpu batch-inference serverless-inference gpu-inference ai-infrastructure
-
Updated
Jun 20, 2026 - Python
Fine-tune anime-seg (ISNetDIS) bằng LoRA trên 1306 cặp ảnh-mask để tách nền anime; có split cố định, evaluation trên mask thô, hậu xử lý alpha_gain cho RGBA cutout, merge adapter vào checkpoint, và CLI rmbg cho train/eval/predict/report/make-split với manifest tái lập đầy đủ.
computer-vision deep-learning pytorch transfer-learning lora image-segmentation rgba model-evaluation fine-tuning segmentation-model alpha-matting background-removal checkpointing pytorch-lightning batch-inference anime-segmentation
-
Updated
Aug 4, 2026 - Python
AWS-oriented ML platform for fraud detection, churn prediction, anomaly detection, experimentation, monitoring, and risk inference.
python aws data-science machine-learning risk-analysis scikit-learn cloudwatch experimentation ab-testing churn-prediction fraud-detection anomaly-detection sagemaker mlops root-cause-analysis drift-detection model-monitoring batch-inference real-time-inference champion-challenger
-
Updated
Jun 2, 2026 - Python
End-to-end AWS SageMaker model training, real-time deployment, and batch inference workflow
-
Updated
Aug 18, 2026 - Python
RankShift Serving — batch engagement prediction with per-row fault isolation
python machine-learning fault-tolerance rest-api model-serving mlops fastapi streamlit engagement-prediction batch-inference
-
Updated
Sep 14, 2026 - Python
Production-style batch ML pipeline for manufacturing claims anomaly detection with drift checks (synthetic data).
-
Updated
Jan 29, 2026 - Python
Production-ready ML system for credit default prediction on transactional data (458k clients). Features end-to-end pipeline: 1,158 engineered features, ablation & Top-500 pruning, LightGBM HPO (AMEX 0.791, Gini 0.923), multi-seed stability, CLI batch inference (20.1s/458k), and 267/267 automated tests.
python machine-learning feature-selection fintech xgboost hyperparameter-optimization lightgbm feature-engineering credit-scoring risk-management catboost optuna time-series-aggregation experiment-tracking ablation-study default-prediction batch-inference ml-system-design reproducible-ml amex-default-prediction
-
Updated
Aug 9, 2026 - Python
Summarise payment backlogs and render an audit-ready PDF digest through one Infrai credential.
-
Updated
Sep 29, 2026 - Python
B2B Fintech SaaS Churn Intelligence & Customer Lifetime Value (CLV) Engine with Kimball Star Schema, LightGBM, Tweedie Regressor, and TreeSHAP.
python machine-learning data-engineering fintech lightgbm churn-prediction customer-lifetime-value cohort-analysis explainable-ai b2b-saas feature-store shap github-actions azure-sql streamlit duckdb batch-inference saas-metrics
-
Updated
Sep 27, 2026 - Python
A lightweight, production-style Kubernetes setup that deploys, mounts volumes to, and executes a containerized Python ML inference job on a local cluster using KinD (Kubernetes in Docker).
-
Updated
Oct 2, 2026 - Python
Offline trip-safety GUI and batch inference with a 19-feature decision-tree pipeline, 63 tests, and a reproducible ~78.8 MB Windows executable smoke test.
desktop-app python data-science machine-learning csv scikit-learn pytest classification feature-engineering decision-tree student-project driver-behavior singapore-polytechnic model-card practical-ai batch-inference customtkinter trip-safety sp-daaa
-
Updated
Sep 7, 2026 - Python
Add this topic to your repo
To associate your repository with the batch-inference topic, visit your repo's landing page and select "manage topics."