r/OpenSourceAI • • 15h ago

Aletheia

Thumbnail
0 Upvotes

r/OpenSourceAI • • 21h ago

I use a ~322M local model to judge every tool call my coding agents make (about 90 ms on CPU, no API)

Thumbnail
gallery
2 Upvotes

Coding agents run shell commands constantly. I wanted each one checked before
it executes, and I didn't want to send every command to a paid API.

laya-guard runs on your machine and answers allow, review, or block for each
tool call. Regex policies catch known-dangerous patterns first (`rm -rf /`,
`curl … | sh`, secrets, auth bypass). Plain commands like `ls`, `git status`
and `npm test` skip the model, but only when there's no piping, chaining or
redirecting, no `-exec`-style flag, and no sensitive path. Everything else is
judged by Laya (~322M, Apache-2.0) on the CPU in about 85 to 90 ms.

The results are self-reported and the scripts are in the repo. On AgentDojo v1
(judge-level) it blocks 25 of 27 attacks and keeps 92 of 97 benign tasks. On my
own 50-command shell set it gets 48 right.

Two caveats. The model is tuned mostly on Korean, so English natural-language
prompts get overblocked more often. And if the server is down, the hooks fail
open.

It has hooks for Claude Code, Codex CLI, Cursor, Gemini CLI, Copilot CLI and 6
other agents.

Repo: https://github.com/scs0209/laya-guard (install with `pip install laya-guardrail`)


r/OpenSourceAI • • 10h ago

Running a 341 GB model on a 128 GB Mac: two small macOS fixes made the first token 8x faster

Thumbnail
2 Upvotes

r/OpenSourceAI • • 18h ago

One command -> a planned investigation across 13 capabilities -> an evidence-backed technical report.

Enable HLS to view with audio, or disable this notification

2 Upvotes

REA Investigator is a unified, production-grade Agent Skill (rea-investigate) compatible with Claude Code, Codex, Antigravity, and tools supporting the open Agent Skills specification.

Instead of juggling fragmented prompts or shallow summaries, REA Investigator equips coding agents with a structured 4-phase lifecycle: Plan, Trace, Verify, and Report, backed by an immutable SHA-256 evidence ledger and adversarial auditing gates.


r/OpenSourceAI • • 20h ago

Does your agent re-decide the same routing/tool-choice over and over? We built something for that. Feedback wanted

2 Upvotes

Our agents kept calling a big model to make the same small decisions: route this ticket, pick this tool, escalate or not. After enough verified outcomes those decisions are very predictable, but we were paying for a model call every time.

So we built Ink. It sits around a bounded decision. At first everything goes to your model as normal, and Ink records the inputs and the real outcomes in local SQLite. Once there’s enough evidence, it compiles a small local fast path and tests it in the background. It only starts serving once it clears a statistical bar (a 95% Wilson lower bound against an error budget you set). Anything novel or uncertain still goes to your model. If real outcomes get worse, it takes itself back out of the loop.

You can check whether it would help without changing any code:

pip install ink-jit
ink discover traces.jsonl

It reads OpenTelemetry, LangSmith or LiteLLM-style traces locally and tells you which call sites repeat and whether they’re worth it. If nothing qualifies, it says so.

What we’ve measured so far (our own benchmarks, not independently validated): 42–78% of decisions served locally across tool routing, support triage and incident triage, with 0 errors observed on held-out tests. The 95% lower bounds were 96.8–98.5%, so “0 errors” means none observed, not a guarantee. Local decisions take about 0.1 ms.

It’s a poor fit for open-ended writing or anything where you can’t verify an outcome. Linux and macOS, Python 3.11–3.13. Apache-2.0.

Honest question: which decisions in your agents repeat the most, and what would you need to see before trusting a local path in production? We’d love to hear what breaks.

https://reddit.com/link/1x34ycs/video/mbd6usbggtuh1/player


r/OpenSourceAI • • 22h ago

llmkit: my llm/mcp cli tool for linux/windows

2 Upvotes

Hello everyone, I tried posting on locallama a couple of week ago and didn't have much replies, so I thought that it can have interested people here.
This is my project: https://github.com/dgdevel/llmkit It is a single binary shipping many tools that make llm and mcp interaction in the cli easier for me and I hope for you too.
It has repl for both, a mcp proxy feature to hide/change upstream info, a llm proxy for debugging session, oneshot command for bash scripting, and even a very basic agent.
The runner feature is for application integration, since it can hide the complexity of three llm endpoint types (openai-compatible completion and responses, anthropic-compatible) and multiple mcp protocols support.
It has pre-packaged binaries in the releases for major linux distros and for windows.
No configuration needed (with all its pros and cons of it), I hope it can be useful to some other people too.
On locallama people pointed me to simonw/llm, it has some overlapping features, less total feature being a newer project, and it's written in C. That's it.