ragleap-optimizer

version — loading live from PyPI…

Status: alpha. Not independently audited. The library does not promise savings: every step reports what it changed and whether it can lose information, and you measure quality on your own cases. No token-saving or speed number has been measured.

pip install ragleap-optimizer

Python 3.10 or newer. It depends on ragleap-security (for the cache scope) and otherwise only on the standard library.

Highlights

• Measure first: Optimizer runs steps over a request (a dict with a list of messages and optionally tools, system and other fields) and returns the new request plus a Report with tokens before and after, per step. A step whose result is longer than its input is discarded, and the input is never changed
• Lossless steps: dedupe_context() removes a long message that is an exact copy of a later one (system and short messages are never touched), minify_whitespace() changes only spacing, and stable_prefix() puts system messages first and tool JSON in sorted key order so a provider prompt cache can recognise the start of the request
• One lossy step, labelled: trim_tool_output(max_tokens) keeps the start and end of an oversized tool message with an explicit marker. It loses the middle, so the report says lossy=True whenever it cuts
• ExactCache: in memory, a Scope from ragleap-security is required, the whole request is the key (model, messages, tools and every other field), entries expire, the least recently used are evicted, and entries can be erased per scope
• retry and with_fallback: bounded attempts with doubling waits and a deadline, and trying several callables in order; the error names the failures without their text
• evaluate: runs your cases with and without the optimizer, scores both answers with your judge, and reports input tokens saved and the quality change with a 95% interval (seeded bootstrap), and lists every case that got worse. With fewer than 30 cases it gives no savings figure

Try it (copied from the README at the 0.1.0 tag)

from ragleap_optimizer import Optimizer, ExactCache, evaluate, steps
from ragleap_security import Scope

opt = Optimizer([steps.dedupe_context(), steps.minify_whitespace(), steps.trim_tool_output(max_tokens=800)])
out = opt.run(request)
print(out.report.tokens_before, out.report.tokens_after, out.report.lossy)

cache = ExactCache(ttl_seconds=3600)
scope = Scope("acme", subject="alice")
hit, answer = cache.get(scope, out.request)
if not hit:
    answer = my_llm_call(out.request)
    cache.put(scope, out.request, answer)

table = evaluate(cases, run=my_llm_call, optimizer=opt, judge=my_judge)
print(table.summary())

Measurements and limits

Our Tests page lists 6 test files and 86 test functions on the main branch (counted from the repository, not a pass count). The changelog records, as separate measures: 156 tests pass and all 84 mutations are killed in the author’s sandbox (Python 3.12) and on the VPS (Python 3.10.12); CI on pull request #1183 passed all 52 checks, including the optimizer tests on Python 3.10, 3.11 and 3.12; and a lossless pipeline never made a request longer on 200 random requests. Not verified, as the changelog says: the package as published on PyPI (at the time of writing), any speed or token-saving number on real traffic, token counts against a real tokenizer, message shapes other than the OpenAI-style messages list, several processes sharing one cache, and no independent security review has been done.

What it does not do

Token counts are estimates (one token per 3 UTF-8 bytes, which over-counts English); pass counter= to use a real tokenizer, and evaluate counts input tokens only. The cache lives in one process and stores whatever you put in it: a wrong answer is served until it expires, so use a short ttl_seconds and never cache answers that depend on the current time or on private context that is not in the request. evaluate is only as good as your judge and your cases. Not in 0.1.0 (planned, design issue #1164): model routing, prompt compression, a semantic cache, secret redaction and framework adapters.