Want to run local LLMs to escape corporate AI APIs? Check out CanIRun.ai first to see if your rig can handle it, or if it'll just melt your GPU.
Lately, everywhere you look, tech bros and senior devs are preaching the gospel of "Local AI." They want you to download LLMs to run offline for privacy, censorship resistance, and sheer unadulterated geekiness. It sounds incredibly badass—until you realize how aggressively these models gobble up your RAM and VRAM.
Long story short, the site canirun.ai (which recently hit nearly 1,000 upvotes on Hacker News) is basically "Can You Run It" but for the AI ecosystem.
If you grew up pirating PC games, you probably remember checking those system requirement sites to see if your potato could handle GTA V or Crysis. Now, replace games with Llama 3, Mistral, or Phi-3. You punch in your CPU, RAM, and GPU specs, and it gives you the brutal truth: Will it run smoothly, will it stutter like dial-up, or will it literally turn your hardware into a space heater? It calculates this based on the VRAM needed to load the model weights and the estimated inference speed (tokens per second).
Scrolling through the HN comment section, the community is deeply divided. The holy wars are real, and they mostly fall into these camps:
1. The Apple Silicon Flexers: Ever since Apple dropped the M-series chips with Unified Memory (sharing RAM and VRAM), Mac users have become the unexpected kings of Local AI. "Just spinning up a 70B model on my 128GB Mac Studio, runs like butter." It's insane because getting 128GB of VRAM in the PC world requires selling a kidney to buy enterprise Nvidia GPUs.
2. The PC Master Race Crying Over "The Nvidia Tax": PC builders are cursing Jensen Huang's leather jacket. Consumer gaming GPUs are severely starved for VRAM (maxing out at 24GB for the RTX 4090). You can run a small 8B model fine, but try loading a 70B model and your system throws an OutOfMemory exception and dies on the spot.
3. The Pragmatists: "Just rent it, you fools": Some veterans are just shaking their heads. Why drop $5,000 on a rig just to chat with a localized bot? Just spin up a VPS or rent a cloud GPU instance. Or better yet, just use standard API endpoints or an existing ai generator to actually get shit done instead of reinventing the wheel.
At the end of the day, canirun.ai is a massive reality check for devs with delusions of grandeur about their personal computers.
The Bottom Line: If you want to dive into the self-hosted AI world to learn architectures, tinker, and break things? Go for it! Download Ollama, have fun, fry a GPU. But if you are trying to ship a product or build a startup, drop the "Self-hosted" ego. The electricity bill and hardware depreciation will cost you 100x more than just paying OpenAI or Anthropic for their APIs. Be a smart dev, not a masochistic one!
Source:
