# Jorge Roberts: personal applied AI portfolio (full content) Personal portfolio of some of the projects I have built, to show my skills in applied AI. These are personal projects, built and run on my own, not client work. Each one is documented from its own repository: the problem, the scope, a technical overview (tools, providers, integrations), the architecture, the design decisions and the results. Web: https://deliveryit.com.ar/ · Spanish: https://deliveryit.com.ar/es/ Author: Jorge Roberts · LinkedIn: https://www.linkedin.com/in/jorgeroberts-delivery/ · Email: jorge@deliveryit.com.ar - CV (PDF): https://deliveryit.com.ar/cv/Jorge-Roberts-CV-1p.pdf - CV for AI (Markdown): https://deliveryit.com.ar/cv/Jorge-Roberts-CV-AI-ready.md ## LLM Gateway - Status: production · Type: infra · Started: 2026-06 · Selected initiative - Summary: Self-hosted gateway that routes requests from several apps to local and free models by capability tier, with failover, per-app guardrails and end-to-end tracing in Jaeger and Langfuse. - Scale: 8 model groups across 5 providers - LLM usage, last 30 days: 428 requests, 1,964,349 tokens - Stack and AI capabilities: model-routing, failover, guardrails, prompt-injection-defense, observability, python, litellm, postgres, redis, docker, opentelemetry ### Highlights - 8 model groups across 5 providers, with no per-call cost. - Four model retirements absorbed by changing configuration only, without touching any app. - Prompt-injection guardrails, with each app's policy declared and versioned in its own repository. - End-to-end traces (app, gateway, provider) with cost, latency and errors per call in Jaeger and Langfuse. ### Technical overview - **Platform:** LiteLLM Proxy (OpenAI-compatible API) on Docker, with a custom Python callback (`async_pre_call_hook`) that applies each key's policy before routing. - **Providers and models by tier:** local Ollama (`qwen2.5:0.5b`, `llama3.2:1b`) for cheap classification; Groq (`gpt-oss-20b`, `gpt-oss-120b`, `qwen3.6-27b` for vision); NVIDIA NIM (`llama-3.1-8b`, `nemotron-3-nano-30b-a3b` for function calling, `nemotron-3-ultra-550b-a55b` for reasoning, `llama-3.2-11b-vision`, `nemotron-3-embed-1b` for embeddings); Google Gemini (`3.6-flash`); OpenRouter (`llama-3.3-70b:free`). - **Data:** PostgreSQL 16 for virtual keys, policy metadata and spend logs; Redis as response cache (1 h TTL). - **Governance:** one virtual key per app; `tools/policy-sync` reads the `llm-policy.yaml` versioned in each repo and applies it to its key (timeout, sanitize, tools, purpose, RPM/TPM). Fallbacks filtered per key; public-only groups return 403 unless explicitly enabled. - **Observability:** OpenTelemetry (OTLP) into a custom collector with a persistent queue, fanning out to Jaeger (local, Badger, 30 days) and Langfuse Cloud. - **Clients:** shared `@homelab/llm-client` package (Node.js) used by the homelab apps. - **Quality:** guardrail unit tests inside the image and an end-to-end suite with an isolated LiteLLM, Postgres and a custom echo provider, with no quota usage. - **Deploy:** own image built from the repo, a service in the homelab `docker-compose`; changes are tested on a separate instance and rollback means reverting the config to the previous commit. ### Problem Several personal bots and apps need LLMs with different requirements (tool use, reasoning, vision, embeddings, sensitive or public data) without paying per call. Talking to each provider directly scatters API keys, makes it impossible to measure usage or quality, and every time a free provider retires a model a different app breaks. ### Scope - One OpenAI-compatible API for every app. - Capability tiers, each with several providers in order and automatic failover. - One virtual key per app, with allowed models, limits and its own guardrail policy, declared in the app's repository. - Generic prompt-injection guardrails applied at the proxy. - Caching, OpenTelemetry traces and usage logging. ### Results - 8 model groups across 5 providers, all free or local. - Four model retirements (NVIDIA, Groq, OpenRouter) absorbed by changing configuration only, without touching the apps. - A single end-to-end trace (app → gateway → provider) in Jaeger and Langfuse, with cost and latency per call. - Automatic failover across providers, filtered by what each key is allowed to use. ### Design decisions - **One gateway instead of one SDK per provider:** a model change is solved in a single file and the apps do not change. - **Tiers by capability, not by provider:** the app asks for "reasoning", not for "NVIDIA". - **Each app decides its input length:** every project knows the legitimate size of its prompts, so the limit lives in its policy, not in the gateway. - **The gateway enforces the maximum duration per key:** an app cannot ask for more time than its policy allows. - **Tag escaping by allowlist:** every `<` that could open a tag is escaped, while `->`, `>99` and `<100` pass through. - **A separate group for public data:** allows using providers that log requests without exposing personal data. - **Guardrails in two layers** (ADR 0001): a versioned policy in each app, plus specific controls in each app's code. ### Architecture (architecture diagram available on the web page) Evidence: Real trace of a daily report: the app opens the trace, the gateway joins it and the router recovers on its own from a provider error (red mark) without the app noticing. The 1.8 minutes belong to the reasoning model that writes the report in the background. Evidence: The fallback chain is filtered per request: failover never lands on a model the key is not allowed to use. ## HealthBot - Status: production · Type: bot · Started: 2026-07 · Selected initiative - Summary: Personal health coach on Telegram that logs food, training and sleep from natural language or photos, and produces daily, weekly and monthly plans using LLMs, RAG over cited evidence and deterministic guards on every number. - Scale: 589 videos and 17,588 indexed chunks; bot, API and an Android app - LLM usage, last 30 days: 120 requests, 350,507 tokens - Stack and AI capabilities: rag, vision, structured-extraction, llm-reports, deterministic-guards, tier-escalation, observability, node, sqlite, sqlite-vec, express, telegram, python, docker, opentelemetry ### Highlights - Cascading monthly, weekly and daily reports that adjust nutrition, sleep, training and supplementation targets on their own; 67 reports generated and stored. - RAG over 589 videos and 17,588 chunks, with one vector index per source and principles distilled across sources. - 70 deterministic guard tests that validate every number the model produces. - Food vision where the model identifies and weighs and the code does the math; the same meal gives the same number by photo and by text. - End-to-end OpenTelemetry traces (bot, gateway, provider) of every run and every report, in Jaeger and Langfuse. - Retrieval quality measured with a frozen eval dataset; hit@5 raised from 0.64 to 0.74. ### Technical overview - **Platform:** Node.js on Docker; Telegram bot over long polling (`node-telegram-bot-api`), an Express HTTP API (fail-closed Bearer auth with attempt limits) for the Android app and the Health Connect receiver, and a cron scheduler in local time. - **Data:** SQLite with `better-sqlite3` (38 tables) and `sqlite-vec` for three vector indexes, one per corpus source; migrations at startup. - **LLM:** text goes through the self-hosted gateway with its own key and policy (`llm-policy.yaml`), via `@homelab/llm-client`. Tiers: `tier-2` for chat, `tier3-tools` for the daily briefing, `tier3-reasoning` for weekly and monthly reports, `tier-vision` for food photos. Embeddings with NVIDIA NIM and backup vision with Google Gemini. - **RAG:** producer-consumer ingestion of YouTube transcripts (Python + Node), embeddings, per-source retrieval and distilled principles; eval with a frozen dataset (hit@5). - **Integrations:** Telegram, Health Connect (through the Android app), YouTube (transcripts), LLM Gateway. - **Observability:** end-to-end OpenTelemetry (bot, gateway, provider) into Jaeger and Langfuse. - **Quality:** 70 deterministic guard tests plus custom calibration tools for photos, macros and RAG. - **Deploy:** a service in the homelab `docker-compose`, script-only deploys, with a tagged rollback image. ### Problem Following a nutrition and training plan requires logging a lot (meals, water, loads, weight, sleep) and, above all, cross-checking that data to decide what to adjust. Commercial apps log but do not reason, and a generic LLM chat reasons but does not remember, makes up numbers and has no access to the watch data. HealthBot is a personal health coach: you talk to it in natural language on Telegram (or send a photo of the plate), it logs what applies, and every day it builds a plan against the real history, with evidence cited from a corpus of science communication. The design premise is that the LLM contributes judgement, but every number that can be calculated or verified is calculated and verified by code. ### Scope - Logging by natural language, commands and photos: food, water, weight and measurements, workouts (sets, reps, load, RIR, drop sets and supersets), supplements, subjective check-in, lab results. - Watch data ingestion through Health Connect (sleep, HRV, heart rate, steps, calories, exercise sessions). - Macro estimation from text and from photos (plate or nutrition label). - Cascading monthly → weekly → daily reports that adjust targets. - Dynamic routine with deterministic load progression and mesocycles. - RAG over transcripts from Huberman Lab, Dr. La Rosa and a backup source, with distilled principles. - Read and write HTTP API for the app. - End-to-end traces of every run. ### Results - In daily use since July 2026, with 67 cascade reports generated and stored (49 daily, 15 weekly, 3 monthly) and target adjustments applied automatically. - Corpus of 589 videos and 17,588 chunks indexed in three vector indexes, one per source. - 70 automated guard tests that validate every number before it reaches the user. - The same meal gives the same number by photo and by text: vision identifies and weighs, and calories come from the same function as text. - Food logging by text from the app in under a second. ### Design decisions - **The prompt asks, the guard guarantees.** Everything that can be checked against the user's text or a plausible range is checked in code; the LLM is left with judgement. Every guard has its test. - **The model identifies and weighs; the code calculates.** A photo says what is there, not how much it weighs: vision returns items and grams, and calories come from the same function the text path uses. - **Machine-readable directives.** Reports emit `AJUSTE`, `RUTINA` and `MESOCICLO` as structured blocks, with a narrow extraction as a safety net. - **A cascade with a single writer per field.** Monthly and weekly write the adjustment; the daily report only reads it with `COALESCE`. - **Real rotation instead of a calendar.** What to train today comes from the workout history, with detection of breaks. - **One vector index per source.** Distances across indexes are not comparable; kept apart, each source can change embedding model without affecting the others. - **Thin client.** The app has no logic and no LLM keys: every number comes from a single function on the server. - **Only validated output goes out.** A report that does not pass validation is regenerated before it is sent. - **Dates in a single module.** The container runs in UTC and the day is always calculated in local time, in one place. ### Architecture (architecture diagram available on the web page) Evidence: The model identifies and estimates weight; the code does the math. If the person stated the amount, the weight comes from multiplying, not from estimating. ## AI Digest - Status: development · Type: service · Started: 2026-10 · Selected initiative - Summary: Weekly AI newsletter produced by a six-stage agentic pipeline, from reading more than 25 sources to an email with up to 20 validated news items. - Scale: 25+ sources and up to 20 news items per edition - LLM usage, last 30 days: 220 requests, 1,600,463 tokens - Stack and AI capabilities: agentic-pipeline, curation, summarization, source-discovery, output-validation, node, nunjucks, readability, docker ### Highlights - Six-stage pipeline where code validates and the model only decides what requires judgement. - From about 260 items a week to 20 at most, with no URL or figure that is not in the source. - A model chosen per stage by measuring against a fixed fixture, not by intuition. - Every LLM call traced through the gateway in Langfuse and Jaeger, with cost, tokens and latency. - Page fetching with SSRF protection, and new sources added only with human approval. - 28 network-free tests, including an end-to-end run with a fake LLM. ### Technical overview - **Platform:** Node.js in a short-lived container per run (own image), triggered by cron on Mondays; CLI with a lock, `--dry-run`, `--since` and `--month`. - **Sources:** more than 25 across RSS/Atom feeds, Hugging Face, arXiv and GitHub, declared in a versioned registry (`sources.yaml`). - **Processing:** Mozilla Readability for full text, downloads with custom SSRF protection (ports 80/443 only, no private IPs, redirects revalidated), date and duplicate filters. - **LLM:** everything through the self-hosted gateway with the `app-ai-digest` key; `tier-2` and `tier3-public` (a group reserved for public data). Streaming, JSON extraction and schema validation, automatic retries and a 45,000-character cap per request. - **Prompts:** a versioned skill per stage (triage, enrich, compose, discover), with external content always inside tags and never in the system prompt. - **Deterministic validation:** URLs and figures checked against the input; per-section limits. - **Observability:** every LLM call goes through the gateway and is traced in Langfuse and Jaeger (cost, tokens and latency per call). - **Delivery:** HTML rendered with Nunjucks, sent over SMTP to fixed recipients, an archive of every edition and alerts through the homelab notifier (Telegram). - **Quality:** 28 tests with `node --test`, network-free, including an end-to-end run against a fixture with a fake LLM. ### Problem Keeping up with AI means reading many scattered sources, and most of what gets published is noise or repetition. The goal is that **every Monday morning a newsletter-style email arrives** ("Weekly report") with **up to 20 news items** from the week, each one linking to the original source with a short take on why it matters. **Editorial focus:** news with practical value, not "AI news". It prioritizes new models (capabilities and pricing), papers and benchmarks, practices and methods for working with agents, skills, agent security and repositories. It discards funding rounds, people moves, controversies and corporate announcements with no technical substance. Secondary goal: make the project a practical example of a closed agentic pipeline, with a versioned skill, tier routing in LiteLLM, output validation and observability. ### Scope **Includes:** the weekly and monthly jobs, a `--dry-run` mode in both, manual runs for a week (`--since`) or a month (`--month`), and an archive of every edition. **Does not include** - **LinkedIn and X.** They offer no feed and scraping them goes against their terms. - Paid sources or sources behind a login. - Republishing full texts. - Adding sources without human approval. ### Design decisions - **A closed six-stage agentic pipeline**, with code wherever things can be verified (fetching, filters, validation, rendering) and the model only where judgement is needed (selection, reading, writing). - **Validation decides, not the model:** no URL or figure reaches the email unless it is in the input. - **Discovery through citations between registry sources**, with no web search and with human approval: the registry cannot be poisoned. - **A model per stage chosen by measuring against a fixed fixture:** each stage uses the one that performed best (the reasoning model for prefiltering and writing). - **Resilient delivery:** if a source or a stage is unavailable, the edition still goes out and flags it. ### Architecture (architecture diagram available on the web page) Evidence: Real edition produced by the pipeline: every item with its source, and no URL or figure that is not in the input. It is No. 35 because numbering continues from the reference edition used as a model. Evidence: Figure validation: a sentence with a number that does not appear in the sources is dropped before sending. ## Papers ### De Scrum a Spec-Driven Development - Documento metodológico, version 1.0, octubre de 2026, 19 pages, language ES - PDF: https://deliveryit.com.ar/papers/de-scrum-a-spec-driven-development.pdf - Cite as: Roberts, J. (2026). De Scrum a Spec-Driven Development (version 1.0). Cuando un agente de IA genera la mayor parte del código, el costo de construir cae en órdenes de magnitud, mientras que decidir qué construir y validar lo construido sigue dependiendo de personas. Scrum se diseñó sobre la estructura de costos inversa. El documento presenta Spec-Driven Development en términos operativos: el ciclo de trabajo con sus artefactos y gates, cómo se reconvierte cada ceremonia de Scrum, las métricas que reemplazan a story points y velocity, por qué no es un regreso a la cascada y un modelo de madurez de tres niveles (spec-first, spec-anchored y spec-as-source). Incluye un ejemplo completo en una app móvil, un plan de transición de ocho semanas y un catálogo de modos de falla. ### From Scrum to Spec-Driven Development - Methodology paper, version 1.0, October 2026, 22 pages, language EN - PDF: https://deliveryit.com.ar/papers/from-scrum-to-spec-driven-development.pdf - Cite as: Roberts, J. (2026). From Scrum to Spec-Driven Development (version 1.0). When an AI agent generates most of the code, the cost of building drops by orders of magnitude, while deciding what to build and validating what was built still depends on people. Scrum was designed on the opposite cost structure. The paper presents Spec-Driven Development in operational terms: the work cycle with its artifacts and gates, how each Scrum ceremony is converted, the metrics that replace story points and velocity, why it is not a return to waterfall, and a three-level maturity model (spec-first, spec-anchored and spec-as-source). It includes a complete example in a mobile app, an eight-week transition plan and a catalog of failure modes. ## Posts ### Story points measure volume. Measure validation instead. (2026-11-19) Original: https://www.linkedin.com/posts/jorgeroberts-delivery_softwaredelivery-aiagents-engineeringmanagement-share-7529212107825946624-fT1k/ Most teams still plan sprints in story points. Once an AI agent writes most of the code, I think that number stops telling you much. Generating code got cheap. Checking that it's right didn't. For 15 years I ran delivery where velocity was a decent proxy. Building was the slow, expensive part, so counting what got built made sense. Now an agent can produce a feature in an afternoon. A team can double its points and, in the same sprint, pile up rework, drift between spec and code, and review debt that never shows up on the board. So the question I ask a team has changed. How much of what you built was accepted the first time someone reviewed it? Across the 70+ AI projects I've worked with this year, these are the numbers I look at, with the bar I set: - first-pass acceptance: 80% or more of agent output approved without rework - rework: under 20% of effort going back into work already marked done - validation latency: finished work waits less than 1 day for a person to check it None of them reward volume. A team that generates faster than it can verify sees them slip within 2 or 3 sprints, long before velocity says anything is wrong. When building is nearly free, judgement is the scarce resource. That's what the plan should measure. If you run teams with agents in the loop: what replaced story points in your planning, if anything did? #SoftwareDelivery #AIAgents #EngineeringManagement ### Human review should follow risk, not volume. (2026-11-17) Original: https://www.linkedin.com/feed/update/urn:li:activity:7528495117540843522/ 65% to 78%. That's the number Anthropic chose to report about how its team reviews AI-written code. Not lines reviewed. Not velocity. The share of pull requests that pass after a single review round. This week the Claude Code team explained how they review code now that agents write a growing share of it. From a delivery point of view, two decisions stood out to me. First, human review follows risk, not volume. Low-risk changes can merge without human approval. Medium-risk changes need one person. High-risk changes need two, and sensitive areas like CI always get a human. Second, review loops are capped. After three rounds, the author is asked to rethink the implementation instead of patching it again. PRs that needed five or more rounds dropped from 8.9% to 1.5%. That second rule is the one I see missing most often. Endless review rounds are where AI speed quietly turns into rework, and nobody's board shows it. The takeaway for delivery leaders: decide in advance where human judgement is required, and measure whether work passes the first time. The person directing the agent is still accountable for the code. Governance just decides where that person's attention goes. How does your team decide which AI-generated changes still need two human approvals? #SoftwareDelivery #AIAgents #EngineeringManagement