Best LLM/AI ecosystem stack

The AI tooling & LLM ecosystem moves quickly. As always see x.com/the_mewc for latest commentary.

General

• Cursor with MCP + background agents for long-running tasks
• Cursor CLI and claudecode cli are good contenders for CI, oai are catching up
• Replit, lovable, v0 and the like are early toys but aren't ready for serious
• builder.io is promising for marketing sites

Tooling

• Firecrawl for structured web extraction and crawl orchestration
• AI SDK / OpenAI / OpenRouter clients with retries, timeouts, backoff
• Unify observability: request/response traces, redaction, prompt versioning

Retrieval & data pipelines

• ETL: Postgres/ClickHouse as the ground truth; CDC for freshness
• Embedding stores with predictable chunking and TTLs; hybrid search where needed
• Permissions in retrieval layer, not in prompts

Orchestration & agents

• Task graphs > monolithic agents; isolate tools; sandbox risky actions
• Background workers with idempotency keys; retries with dead-letter queues
• Human-in-the-loop for risky or user-facing actions

Testing, evals & observability

• Golden sets; regression gates in CI; online vs offline evals split
• Diff prompts and models; measure cost, error rate, and outcome quality
• Capture traces; scrub PII; keep reproduction kits for incidents

Deployment & ops

• Multi-region routing and failover; rate limits and circuit breakers
• Secrets via KMS; key rotation; request signing and audit trails
• Model routing by use case and budget; fallbacks for degraded modes

Safety, governance & compliance

• Content filtering; jailbreak tests; red-teaming as a practice
• Tenant isolation; retention windows; GDPR/region pinning
• Provenance for prompts and retrieved data

I’ll keep this post fresh with concrete vendor picks and example configs.

[!note] Snapshot vs narrative
The homepage section is a quick reference. This post carries the fuller reasoning and tradeoffs.

[!tip] Defaults
Start simple. Add complexity only when the bottleneck is proven by traces and cost reports.

[!warning] Risky actions
Sandbox tools that can change state, and require human review where appropriate.

Example code block

type Vendor = "OpenAI" | "OpenRouter" | "Anthropic";
const chooseVendor = (useCase: string): Vendor => {
  if (useCase === "evaluation") return "OpenAI";
  if (useCase === "cost-sensitive") return "OpenRouter";
  return "Anthropic";
};

Line breaks demo:
First line.
Second line after a single newline (should break due to remark-breaks).

Paragraph demo:

This is a new paragraph because there is a blank line above.