Buy tokens, rent a GPU, or own one? The solopreneur inference bill

There are four different purchases

A direct model API buys a specific provider’s model, rate limits, data terms, and billing. OpenRouter buys one API, consolidated credits, routing, fallbacks, and a large catalog. A rented GPU buys machine time. An owned GPU buys depreciating capacity, power draw, heat, and an operations queue.

OpenRouter currently passes through provider list price and charges a 5.5% credit-purchase fee with an $0.80 minimum. BYOK has a separate allowance and fee policy. Free models have strict daily limits and are not a production capacity plan.^[1][2]

Direct providers expose different caching, batch, context, and rate-tier economics. Anthropic, for example, charges a higher long-context rate after a documented input threshold and offers a 50% Batch API discount. The cheapest headline model can become expensive when output or uncached context dominates.^[5]

The first optimization is accounting, not hardware

Track cost per product action: one support answer, one document extraction, one generated image, one completed agent task. Token totals without user outcome make routing look efficient while the product remains uneconomic.

OpenRouter exposes usage by provider, model, and key, plus live credit information. Direct OpenAI and Anthropic APIs return usage fields that should be stored beside request latency, errors, retries, and the product operation ID.^[2][3][4][5]

The GPU break-even is mostly an utilisation chart

Runpod lists on-demand GPU-hour prices from smaller 24 GB cards through 96 GB and datacenter GPUs. An RTX 5090 instance can be rented by the hour, while serverless workers charge a higher active rate in exchange for elasticity. Persistent storage and idle pod time still need explicit teardown.^[7]

A local RTX 5090 has 32 GB of memory, 575 W total graphics power, and a 1000 W recommended system power. A professional RTX PRO 6000 has 96 GB ECC memory and a 600 W maximum. VRAM determines which model and batch size fit before nominal compute speed matters.^[8][9]

At $0.20/kWh, a 700 W system running continuously costs about $102/month before cooling. Add three-year depreciation, downtime, networking, and the founder’s time. At 10% useful utilisation, most of that fixed cost produces heat rather than tokens.

A staged buying strategy

Start direct when one provider and one model can validate the product. Add OpenRouter when routing, fallback, unified accounting, or fast model experiments are worth the fee. Keep provider selection explicit for privacy-sensitive requests.

Rent a GPU when an open model passes a real evaluation and hourly demand is measurable. Buy hardware only after the rented workload is consistently busy, the model fits the target VRAM, and downtime has an acceptable fallback.

• Prototype: one direct API, prepaid budget where available, per-feature cost telemetry.
• Multi-model product: gateway or OpenRouter with per-key limits and privacy-aware routing.
• Stable open-model demand: rented GPU with autoscaling and a retained-volume budget.
• Sustained private inference: owned hardware plus a cloud fallback and tested recovery path.