RagLeap
ragleap-agents · release notes

ragleap-agents v0.2.0

Second release of ragleap-agents. Install: pip install ragleap-agents==0.2.0.

Behaviour change from 0.1.0

A model reply that is not a single JSON object (prose, truncated JSON, an empty reply) now stops the run with status="stopped", stop_reason="unparseable_reply". 0.1.0 reported these as done, so a model failure looked like success. Nothing executed in either case.

What is new

  • openai_compatible(...): a standard-library adapter for OpenAI-compatible chat APIs, in json and native (tool-calling) mode. https required, no redirects, size cap, bounded retries; errors carry only a status code or an exception type. error_detail=True (off by default) adds the start of an error body for diagnosis, with the key scrubbed. It sends its own User-Agent, because Groq's gateway rejects Python's default with HTTP 403.
  • Policy(repair_attempts=...) (default 0, capped at 2): one bounded re-prompt after an unusable reply. A repaired proposal goes through the same validation, taint rule and approval gate.
  • scripts/live_runner.py (repository only): the runner behind the results below.
  • A rejected run's summary now counts only the calls that ran.

Verified

  • 112 tests pass on Python 3.10, 3.11 and 3.12. Mutation-checked: 19 of 19 adapter and 19 of 20 runner mutations caught; the survivor cannot matter.
  • The published wheel installs fresh on 3.10, 3.11 and 3.12 with pip check clean and reproduces the behaviour change above. A live run of the published wheel against Groq (openai/gpt-oss-20b), 5 tasks in each mode, finished all 10 runs without repair.
  • Live sweeps against Groq on 2026-10-11, one run per task and configuration, 20 tasks: tasks done were json 15, native 17, json+repair 16, native+repair 20, each of 20. These are single runs; the differences between configurations are within the run-to-run variation seen.

Observed with a real model (Groq; three models; not a benchmark)

  • JSON mode: HTTP 400 tool_use_failed on 3 of 20 tasks, the same three in both sweeps, when the model answered in its own function-call format. Prefer native mode with models that have one.
  • Native mode: 2 of 46 saved native runs failed with HTTP 400 because the model called a tool named json that was not offered.
  • Empty replies (4 runs) and an invented refusal action to decline (5 runs). Repair fired in 5 of 40 runs: 4 ended with a valid action. It cannot help when the provider rejects the request.

Not verified

  • The taint rule against a real model: 16 runs on three models read the injected note and none proposed the injected send, so the rule was never triggered. It is tested with scripted models only.
  • Any provider other than Groq (Gemini has not been tried), long runs, and concurrent resumes across processes.
  • Whether repair improves pass rates: five firings on one model do not show that.
  • Native mode's instruction still asks for a JSON object while real tools are offered; two models answered by calling a tool named json. It has not been changed.

Provenance

  • Code: PR #680, merge commit a03e917 (tested head b2a8de7). CI on the merge commit: run 38168208680, success.
  • Publish: tag run 38207845859, trusted publishing with attestations; no "already exists" skip, so this was the first upload.
  • Digests (sha256), identical on PyPI and in the run log:
    • ragleap_agents-0.2.0-py3-none-any.whl: ae9e30969602212fd884f19382af59840df6299ef79c4a1849b6239d37c8e3cc
    • ragleap_agents-0.2.0.tar.gz: 18b9cfbfd598392f81c179fec3c2e848b299a8bb6dee1bd7f111068f6a6d3426
View on GitHub Package page All releases
← v0.1.0

Release notes mirrored from GitHub Releases.