This page lists every automated test and benchmark in the project: its name, what it checks, what it uses and its real latest result. It is generated from the repository (main, commit 86a17cb, 2026-11-20) and its CI logs, never typed by hand. Results below come from the latest successful CI run on main: run 44686139027, 2026-11-20, commit 86a17cb.
Test files: 490. Test functions: 5492. Suites: 23. Functions described by their authors: 239. For the rest the page shows the test name in words and says so. Open a suite to see each test.
Test suites
| Suite | Files / tests | Uses (from ci.yml) | Latest CI result |
|---|---|---|---|
| ragleap-core (the app) | 136 / 1581 | pytest, psql, service container pgvector/pgvector:pg16 | passed 2136 passed, 13 skipped, 0 failed |
| ragleap-agents | 11 / 176 | pytest | passed 251 passed, 0 skipped, 0 failed |
| ragleap-ansible | 1 / 1 | not stated in ci.yml | no CI job |
| ragleap-app-chart | 1 / 1 | not stated in ci.yml | no CI job |
| ragleap-graph | 11 / 190 | pytest, service container neo4j:5 | passed 240 passed, 14 skipped, 0 failed |
| ragleap-integrations | 9 / 303 | pytest | passed 626 passed, 0 skipped, 0 failed |
| ragleap-memory | 18 / 286 | pytest | passed 596 passed, 0 skipped, 0 failed |
| ragleap-observability | 1 / 1 | not stated in ci.yml | no CI job |
| ragleap-ops | 1 / 1 | not stated in ci.yml | no CI job |
| ragleap-optimizer | 7 / 120 | pytest | passed 217 passed, 0 skipped, 0 failed |
| ragleap-phone | 8 / 103 | pytest | passed 244 passed, 3 skipped, 0 failed |
| ragleap-physical | 40 / 410 | pytest | passed 787 passed, 16 skipped, 0 failed |
| ragleap-rag | 36 / 375 | pytest, service container pgvector/pgvector:pg16 | passed 486 passed, 18 skipped, 0 failed |
| ragleap-security | 10 / 169 | pytest | passed 679 passed, 1 skipped, 0 failed |
| ragleap-terraform | 1 / 1 | not stated in ci.yml | no CI job |
| ragleap-tools | 21 / 337 | pytest | passed 526 passed, 0 skipped, 0 failed |
| ragleap-trace | 2 / 47 | not stated in ci.yml | no CI job |
| ragleap-vectorstores | 10 / 109 | pytest, service container redis/redis-stack-server:latest, service container redis:7-alpine | passed 81 passed, 32 skipped, 0 failed |
| ragleap-voice | 33 / 365 | pytest | passed 535 passed, 3 skipped, 0 failed |
| ragleap-rag (Java port) | 62 / 534 | Maven, service container pgvector/pgvector:pg16 | passed 470 passed, 64 skipped, 0 failed |
| ragleap-graph (Java port) | 43 / 196 | Maven | passed 144 passed, 52 skipped, 0 failed |
| ragleap-vectorstores (Java port) | 8 / 28 | Maven | passed 18 passed, 10 skipped, 0 failed |
| @ragleap/rag (npm, TypeScript) | 20 / 158 | not stated in ci.yml | no CI job |
End-to-end smoke tests (Docker)
The full stack is built and started with Docker Compose, then exercised. Result of each step in the latest run: CI job.
- Run actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 success
- Create .env for CI success
- Build and start stack success
- Wait for services to be healthy success
- Wait for Neo4j to actually accept connections success
- Smoke test - health check success
- Smoke test - upload and chat (skipped if no CI API key) success
- Smoke test - knowledge graph (skipped if no CI API key) success
- Dump logs on failure skipped
- Tear down success
Other CI workflows
These workflows are separate from the main test run above. Each description is copied from the workflow file’s own header comment, and the last column is the latest successful run on any branch. A green run means that workflow’s own steps finished, not that the whole project passes.
| Workflow file | What its header says | Latest successful run |
|---|---|---|
ansible-container-test.ymlansible-container-test | Runs the ragleap-ansible roles twice inside a throwaway ubuntu:22.04 container with the package's own tests/container_test.sh: idempotence, the deploy user, opt-in SSH hardening (and the refusal without a key, and the rollback of a bad sshd config) and the opt-in firewall. It never touches the runner itself. Manual (workflow_dispatch) or when a PR changes this file or the Ansible package. Not a required check. | 2026-11-09, pull_request on test/fw-break-1, run |
ansible-vm-firewall-test.ymlansible-vm-firewall-test | Runs the ragleap-ansible FIREWALL role on a REAL host (a GitHub-hosted runner, a disposable VM) and tests enforcement from a container on the Docker bridge: an opened port must stay reachable, a port that was not opened must be blocked. The script refuses to run anywhere except a GitHub-hosted runner. Manual (workflow_dispatch) or when a PR changes this file, the script or the Ansible package. Not a required check. | 2026-11-09, workflow_dispatch on main, run |
ansible-vm-test.ymlansible-vm-test | Runs the ragleap-ansible roles on a REAL host: a GitHub-hosted runner, which is a disposable VM. Base role twice, opt-in SSH hardening twice (real reload handler, real systemd), then a real key login against the hardened sshd. The script refuses to run anywhere except a GitHub-hosted runner. Manual (workflow_dispatch) or when a PR changes this file, the script or the Ansible package. Not a required check. | 2026-11-09, pull_request on test/fw-break-1, run |
chart-install-test.ymlchart-install-test | Install, upgrade and uninstall test for the three RagLeap Helm charts on a throwaway kind cluster. Manual (workflow_dispatch) or when a PR changes this file or the test script. Not a required check. | 2026-11-19, workflow_dispatch on test-rel-observability-0.8.6, run |
chart-upgrade-test.ymlchart-upgrade-test | Upgrade test for the ragleap-ops chart: install a PUBLISHED version from PyPI on a throwaway kind cluster, write data, helm upgrade to the chart in this checkout, and check that the databases were not restarted and the data survived. Manual (workflow_dispatch, with a from_version input) or when a PR changes this file or the script. Not a required check. | 2026-10-26, pull_request on test/upgrade-k8s-matrix, run |
java-ansible-test.ymlJava Ansible Resource Parity | no description in the file header | 2026-11-09, pull_request on test/fw-break-1, run |
java-app-chart-test.ymlJava App Chart Parity | no description in the file header | 2026-10-28, pull_request on gpt-java-app-chart-parity-1028, run |
java-observability-test.ymlJava Observability Resource Parity | no description in the file header | 2026-11-19, pull_request on obs-0.8.6-images, run |
java-ops-test.ymlJava Ops Resource Parity | no description in the file header | 2026-11-11, pull_request on test/trivy-break-1, run |
java-terraform-test.ymljava-terraform-test | Builds and tests the Java jar of ragleap-terraform and checks that it holds the same files as the PyPI wheel. Never publishes. Manual (workflow_dispatch) or when a PR touches the Java folder, the Python package or this check. Not a required check. | 2026-11-03, pull_request on docs/java-jars-terraform-ansible, run |
obs-k8s-test.ymlobs-k8s-test | Kubernetes test of the ragleap-observability SLO alerts and AlertManager wiring on a throwaway kind cluster. Manual (workflow_dispatch) or when a PR changes this file or the test folder. Not a required check. | 2026-11-19, workflow_dispatch on test-rel-observability-0.8.6, run |
ops-app-pods-test.ymlops-app-pods-test | DIAGNOSTIC EXPERIMENT: can the ragleap-ops chart's app and voice pods pull the private ghcr.io/ragleap/ragleap-app image with the workflow token and become Ready on a throwaway kind cluster? The job is green whatever the outcome: read the VERDICT line in the log. Manual (workflow_dispatch) or when a PR changes this file or the script. Not a required check. | 2026-10-24, workflow_dispatch on main, run |
ragleap-trace-test.ymlragleap-trace-test | no description in the file header | 2026-11-06, pull_request on docs/ragleap-trace-0.1.0-release-prep, run |
velero-test.ymlvelero-test | Backup and restore test: Velero with a RustFS S3 stand-in on a throwaway kind cluster, using the ragleap-ops chart. Manual (workflow_dispatch, with version inputs) or when a PR changes this file or the test script. Not a required check. The velero-drill job runs all three cluster and storage combinations every Wednesday 03:30 UTC (schedule), or on demand with drill=true. | 2026-11-18, schedule on main, run |
Benchmarks and evaluations
Java port: cross-backend retrieval benchmark and failure-mode matrix
Answers issue #565: do the Java vector backends return comparable results, and how do they behave on edge cases? Run on 2026-10-03. Every number below is generated by generate_report.py from the raw files in results/.
Result: Summary Retrieval is identical on FAISS, pgvector, Qdrant and Weaviate: the same top-10 documents for all 60 queries (overlap@10 = 1.000 against an exact float64 scan), and each finds the source paragraph exactly as often as the exact scan.…
Python ragleap-rag benchmark
All numbers below come from a real, timed run against real infrastructure — never projected or estimated. See run_benchmark.py for the exact methodology; re-run it yourself to reproduce.
Result: see the report
AI Employee role evaluations
Manual, on-demand eval harness for sensitive-domain AI Employee roles
(legal_intake, healthcare_intake, insurance_agent,
compliance_officer as of this first batch). Makes real /chat calls
against a running instance to spot-check that a role's guardrails
(never giving legal/medical/coverage/compliance rulings, always
escalating) actually hold in practice -- not just that its personality
prompt says it will.
Result: see the report
Other benchmark data
The benchmarks/ folder contains: THROUGHPUT.md, http_throughput.py, stress_office, stt-neutral-10s, voice. Open the folder.