CoolPrompt is a framework for automatic prompt creation and optimization.

Join our telegram channel to be in touch.

Practical cases

• Automatic prompt engineering for solving tasks using LLM
• (Semi-)automatic generation of markup for fine-tuning
• Formalization of response quality assessment using LLM
• Prompt for AI Agentic Pipelines
• Etc.

Core features

• Optimize prompts with our APO methods:
  • HyPER / HyPER Light
  • RE-GPS
  • RIDER
  • BRAVE
  • PromptCompressor
  • (legacy/deprecated): ReflectivePrompt, DistillPrompt
• LLM-Agnostic Choice: work with your custom llm (from open-sourced to proprietary) using supported Langchain LLMs
• Develop own custom APO method in one library
• Generate synthetic evaluation data when no input dataset is provided
• Evaluate a quality of prompts incorporating multiple metrics for both classification and generation tasks
• Evaluate costs of optimization processes by a number of tokens/calls and a price.
• Automatic task detecting for scenarios without explicit user-defined task specifications

APO methods comparison

CoolPrompt provides several automatic prompt optimization methods with different trade-offs in data requirements, runtime, expected quality, and API cost. The levels below are qualitative and task-dependent: they are intended as a quick guide for choosing a method before running a benchmark.

Compared metrics:

• Data - whether dataset is required for the method to run.
• Runtime - relative wall-clock time of one optimization run.
• Performance - expected ability to improve task quality compared with the initial prompt.
• Cost - relative compute/API cost: LLM calls, evaluation calls, token usage, and extra scoring overhead.

Method	Data	Runtime ↓	Performance ↑	Cost ↓
hyper_light	None	Low	Medium	Low
hyper	Required	Medium	High	Medium
regps	Required	High	Very High	High
rider	Required	Very High	Very High	Very High
brave	Required	High	Very High	Budget-controlled
compress	None	Low	Medium	Low
reflective	Required	High	High	High
distill	Required	High	High	High

Quick install

• Install with pip:

pip install coolprompt

• Install with git:

git clone https://github.com/CTLab-ITMO/CoolPrompt.git
cd CoolPrompt

pip install -e .

Quick start

Set your OpenAI API key before running. The default model is gpt-4o-mini via the OpenAI API (OPENAI_API_KEY environment variable)

from coolprompt.assistant import PromptTuner

prompt_tuner = PromptTuner()

prompt_tuner.run('Write an essay about autumn')

print(prompt_tuner.final_prompt)

# You are an expert writer and seasonal observer tasked with composing a rich,
# well-structured, and vividly descriptive essay on the theme of autumn...

Run the data-driven BRAVE optimizer by selecting it as the method:

final_prompt = prompt_tuner.run(
    "Classify the sentiment of the text: {text}",
    task="classification",
    dataset=["Great product", "Very disappointing"],
    target=["positive", "negative"],
    method="brave",
    problem_description="Classify product-review sentiment.",
    max_steps=20,
    initial_budget_tokens=50_000,
)

Automatic APO method selection

Use method="auto" to select an optimizer from experiment metadata. The selector profiles the task with the system model, retrieves up to three similar benchmark datasets, and ranks configurations by quality, cost, and runtime. The selected method is available in prompt_tuner.meta_selection and is added to telemetry exports.

final_prompt = prompt_tuner.run(
    "Classify the sentiment of the text: {text}",
    task="classification",
    dataset=["Great product", "Very disappointing"],
    target=["positive", "negative"],
    method="auto",
    problem_description="Classify product-review sentiment.",
    meta_dataset_name="my_product_reviews",  # optional
)

The bundled metadata contains SAPO, RIDER, and HyPER results. CoolPrompt does not yet implement SAPO, so an SAPO recommendation is transparently executed with hyper; the fallback reason is recorded in meta_selection. Pass meta_classifier_path="/path/to/metadata.csv" to use a custom CSV. Without dataset and target, method="auto" uses hyper_light.

Examples

See more examples in notebooks to familiarize yourself with our framework

About project

• The framework is developed by Computer Technologies Lab (CT-Lab) of ITMO University.
• API Reference

Contributing

• We welcome and value any contributions and collaborations, so please contact us. For new code check out CONTRIBUTING.md.

Reference

For technical details and full experimental results, please check our papers + citations inside.

RIDER

@inproceedings{dragomirov2026rider,
  author = {Dragomirov, Daglar and Kulin, Nikita and Muravyov, Sergey and Makarov, Ilya and Sukhorukov, Daniil and Mozikov, Mikhail},
  title = {RIDER: Evolutionary Prompt Optimization with Adaptive Operator Selection for Software Engineering},
  booktitle = {Companion Proceedings of the 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  series = {FSE Companion '26},
  year = {2026},
  doi = {10.1145/3803437.3807393}
}

RE-GPS

@inproceedings{kulin2026re,
  title={RE-GPS: Reflective Evolutionary Gradient Prompting System for Large Language Models},
  author={Kulin, Nikita and Zhuravlev, Viktor and Khairullin, Artur and Muravyov, Sergey},
  booktitle={2026 39th Conference of Open Innovations Association (FRUCT)},
  pages={157--163},
  year={2026},
  organization={IEEE}
}

CoolPrompt

@INPROCEEDINGS{11239071,
  author={Kulin, Nikita and Zhuravlev, Viktor and Khairullin, Artur and Sitkina, Alena and Muravyov, Sergey},
  booktitle={2025 38th Conference of Open Innovations Association (FRUCT)}, 
  title={CoolPrompt: Automatic Prompt Optimization Framework for Large Language Models}, 
  year={2025},
  volume={},
  number={},
  pages={158-166},
  keywords={Technological innovation;Systematics;Large language models;Pipelines;Manuals;Prediction algorithms;Libraries;Prompt engineering;Optimization;Synthetic data},
  doi={10.23919/FRUCT67853.2025.11239071}
}

ReflectivePrompt

@misc{zhuravlev2025reflectivepromptreflectiveevolutionautoprompting,
      title={ReflectivePrompt: Reflective evolution in autoprompting algorithms}, 
      author={Viktor N. Zhuravlev and Artur R. Khairullin and Ernest A. Dyagin and Alena N. Sitkina and Nikita I. Kulin},
      year={2025},
      eprint={2508.18870},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.18870}, 
}

DistillPrompt

@misc{dyagin2025automaticpromptoptimizationprompt,
      title={Automatic Prompt Optimization with Prompt Distillation},
      author={Ernest A. Dyagin and Nikita I. Kulin and Artur R. Khairullin and Viktor N. Zhuravlev and Alena N. Sitkina},
      year={2025},
      eprint={2508.18992},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.18992}, 
}