checking API…
Living AI PortfolioGrounded AI chat · typed API · auto-deploy · live telemetry · cost-metered
Samuel Hill
Full-Stack Agentic AI Engineer

I build production LLM applications.

Self-taught and project-driven. I design, build, and ship real software end to end — authentication, payments, grounded LLM features — and I document the decisions, trade-offs, and failures behind every build.

Samuel Hill
SH
This site is the portfolio

It doesn't list my skills — it runs them.

Most portfolios describe what someone can do. This one is the proof: everything below is either live in production right now, or on a public roadmap — each item labeled honestly, never blurred together.

The system it runs on · six services, real connections

One push rebuilds two independent clouds in parallel. Step through a flow, or click any service to follow it.

Running now · live in production
Live

RAG assistant with citations

The "Ask my AI" chat retrieves from Samuel's real documents, answers only from them, shows the sources it used, refuses what they don't cover, and never exposes the API key.

Live

Typed FastAPI backend

A Python API with Pydantic models that validate every request and response against a schema.

Live

Continuous deployment

Push to GitHub and the backend redeploys on Render while the site redeploys on Vercel — two clouds, automatically.

Live

CORS-hardened API

The API accepts browser requests only from approved origins — locked down, and configurable per environment.

Live

Automated tests

A pytest suite exercises the API and guards the assistant's honesty — the tests fail if the bot ever overclaims.

Live

Auto-generated API docs

Interactive OpenAPI documentation at /docs, generated straight from the typed schema.

Live

Request tracing & metrics

Every request gets an ID and a structured JSON log line; each answer carries a timing trace, and /metrics reports live latency, request counts, and running cost.

On the roadmap · built in public, shipping in order
01RAG with citationsanswers grounded in real documents, with sourcesShipped
02Evalsquality scores on a 108-case golden dataset — see the numbersShipped
03Observabilityper-request cost, latency, and tracing — live /metricsShipped
04Agentsa tool-using agent over the portfolio data — try itShipped
05MCP serverthe portfolio tools over the Model Context Protocol — the serverShipped
06Streamingtoken-by-token responses — ask the assistant and watch it typeShipped
07Dockercontainerized, reproducible builds — the image builds & runs healthyShipped
08Terraformthe whole stack as infrastructure-as-code — imported, and terraform plan round-trips clean against liveShipped
Architecture X-ray

Follow one question through the stack.

This is the real path a message takes when you ask the assistant something. Every layer below is pulled from the code that is actually running — not a diagram of how it might work. Click through them.

The architecture, end to end

The whole system, written out.

The X-ray above follows one request; this is the entire build in prose — plain-English enough for a newcomer, exact enough to defend to an engineer. Every number, file name, and constant in it was read straight from the source code.

Architecture guide

How the Living Portfolio works

The request lifecycle, the RAG pipeline, evals, observability, the agent and MCP server, streaming, containerization, and the infrastructure-as-code underneath — one document, end to end.

Open the architecture document ↗
source-grounded · nothing claimed that the code doesn't back up
Evals · Module 2

I don't claim it works — I measure it.

The assistant is graded on a fixed set of 108 questions with known-correct behaviour. A deterministic harness scores the objective parts; a stronger model grades answer quality as a judge. Every number below is from one real run — the full results live in the repo.

Objective scores · deterministic, no judgment call
0.992
Retrieval recall@5
65 of 66 answerable questions retrieved every source a correct answer needs.
92 / 92
Citation validity
Every citation pointed to a source that was actually retrieved. Zero fabricated.
108
Golden-set cases
66 should-answer, 42 should-refuse, six categories — each label verified against the corpus.
$0.41
Cost per full eval run
Generate 106 answers ($0.21) plus judge every reply ($0.20) — real token counts.
What the evals caught
Found → fixed → deployed

Early on, asked whether Samuel had shipped response streaming and Terraform — both still in progress at the time — the live bot answered yes, he's shipped them. The judge flagged the overclaim. The grounding rule was tightened, the fix re-measured on the failing cases, then deployed and verified on production — the bot went back to calling that work in progress. (Both have since shipped for real, the honest way.) Measure → find a real bug → fix → re-measure → deploy → verify. Closing that loop is the point of the module.

Judge scores Calibration pending
0.917
Overall pass
answer cases 59/66 · refuse cases 40/42.
0.944
Grounded
the answer stayed within the sources it cited.
0.981
Behaviour correct
answered when it should, refused when it should.
0.963
Captures the answer
included the facts a correct answer needs.

A stronger model (Claude Sonnet 5) grades what the runtime model (Haiku 4.5) wrote. These are uncalibrated: a judge is only trustworthy once it's shown to agree with a human on the same answers. A 24-item blind hand-labelling slice is built and waiting; until it's scored, these numbers carry this label instead of a headline. Publishing a judge score as fact before that check would be the exact overconfidence this module exists to catch.

One number that lies, and why
A naive "did it refuse correctly?" score on the similarity gate reads 0.611 — but it blames the wrong component. In the code, every grounded question is routed to the model; the 0.35 gate only screens out wildly off-topic ones, so the gate was never the thing that refuses. The real refusal happens in the model's own words, reading its sources and saying "these don't cover that." That's true from the code alone, before any judge enters the picture — the (still-uncalibrated) judge, scoring the actual replies, only corroborates it, at 0.981 behaviour-correct. The lesson: an easy-to-compute metric can measure the wrong decision boundary — judge the system's real output, not its cheapest proxy.
Live metrics · Module 4

The system, reporting on itself.

Read live from the API's /metrics endpoint as this page loads — request counts, latency, and running cost since the current server process started. They reset when a new version deploys, and the line below says exactly when that was.

reading /metrics…
Requests served
total this process has handled
Latency p50
median round-trip, all routes
Latency p95
slow tail — cold retrieval shows here
Answers
grounded chat replies
Refusals
out-of-corpus, answered at $0
Chat cost
running total, real token counts

Process-local and honest about it: a single free-tier instance with no metrics database, so these are the numbers since this deploy — not lifetime totals, the same discipline as the rest of the site. If the panel says the API is waking, the backend is a free instance that sleeps when idle and is spinning back up.

Agent · Module 5

Watch an agent use real tools.

Give it a task — a job description, work you need built, a question about fit. It decides which tools to call over Samuel's real data, loops until it can answer, and shows every step. This is the real draw of agents: autonomous, multi-step tool use grounded in real information — not a single canned reply.

Tools it can call: search_portfolio · list_skills · list_projects · list_services

The agent calls the live API and a real model, so each run costs a fraction of a cent and takes a few seconds. Tools are read-only — it looks things up, it never takes an action.

Work with me

Hire me to build it.

Productized packages with transparent "starting at" pricing and two revision rounds on every build. Bigger scopes we shape together on a short call.

Starter Support Bot

from $1,500

A grounded customer-support chatbot on Claude or OpenAI — honest refusal and a clean human handoff, like the live demo.

  • Embedded on one site (React + Vercel serverless)
  • Basic branding, single language
  • Grounded answers, no hallucination
~1 week · 2 revisions
Start a project →

Business Bot + Integration

from $3,500

Everything in Starter, plus real integration into your existing app and the plumbing behind it.

  • LLM integration into an existing site or app
  • Firebase auth + one external integration (CRM / email / Stripe)
  • Automated test suite, deploy to Vercel / Render / Cloud Run
2–3 weeks · 2 revisions
Start a project →

Full-Stack AI Feature / App

from $7,500

A production feature or app, built end to end — typed, tested, and deployed.

  • Typed FastAPI backend (Pydantic, pytest, CI/CD)
  • React / Next.js or React Native (Expo) frontend
  • Stripe payments, up to five-language i18n, streaming UX
  • Deployment + documentation
scoped per project · 2 revisions
Start a project →

Landing Page / Marketing Site

from $1,200

A fast, framework-free responsive site built for conversion.

  • Conversion-focused, fully responsive
  • Analytics + form / CRM integration
~1 week
Start a project →

Care & Improve retainer

$300–$1,200/mo

Ongoing care after your build ships — so it keeps working and keeps getting better.

  • Monitoring, uptime, prompt tuning, minor updates
  • Starter $300 · Growth $600 · Pro $1,200 per month
  • Higher tiers add new-feature work + monthly strategy calls
offered after a build
Ask about retainers →
Now unlocked · each shipped as a module of this site, and open for work

RAG Knowledge Assistant

Available

RAG with citations (pgvector), source attribution, and an eval pipeline. From $6,000.

shipped — it's the "Ask my AI" chat running live on this site
Start a project →

AI Agent / Automation

Available

Agentic tool use with integrations, error handling, observability, and containerized deployment. From $4,000.

shipped — try the live agent in the Agent section above
Start a project →

Custom MCP Server

Available

Wrap your APIs and data as an MCP server — auth, rate limiting, logging, Claude/OpenAI clients. From $6,000.

shipped — the open-source MCP server, ready for any MCP client
Start a project →
Start a project

Tell me what you're building.

There's a real typed API behind this form: every field is validated server-side by Pydantic, then stored.