I burned through 220,000 AI tokens just because I could
· pcworld.com · popularity 20/100 · 0 similar stories today · AI & Data Science (Artificial Intelligence)
Read the original at pcworld.com →
Summary
I made my laptop generate 220,202 AI tokens in about 13 minutes because I wanted to answer a question. Then I started wondering what would happen if I did it again. And again. And again. That’s when local AI started getting really interesting. I’ve been testing the $7,000 Asus ProArt P16 RTX Spark by throwing an unusually large real-world research project at it. Rather than asking a chatbot questions, I’m using OpenAI’s massive 120-billion-parameter gpt-oss model to chew through hundreds of potential data points, evaluate them in different ways, challenge its own conclusions, and spit out enormous piles of analysis. One early run examined 403 candidates and generated 220,202 tokens in just under 13 minutes. That’s a stupid amount of AI. And because the model was running entirely on my laptop, I didn’t care. The meter changes everything Those tokens weren’t free. A $7,000 laptop sure as hell isn’t free. Electricity isn’t free. My time isn’t free (sorry not sorry boss). If you’re buying hardware specifically to avoid cloud AI bills, you need to account for all of that before declaring victory. But once the laptop was sitting on my desk, generating another 200,000 tokens didn’t produce another API charge. That changes how you use it. With cloud AI, every additional experiment has a marginal cost. Should I analyze every candidate or narrow the field first? Should I run another challenger against these results? Should I repeat the entire process later when the underlying information changes? Is answering this weird little question worth another gigantic pile of tokens? Running massive local AI factories can really hammer your system and battery life, but unlocks completely new workload possibilities if you can figure out a way to minimize electricity costs. Brad Chacos/Foundry Those are sensible questions when every token runs through a meter. Running locally, I stopped asking them nearly as often. I ran broader searches. Added independent challengers. Repeated analyses. Tested ideas that might lead nowhere. Eventually I started contemplating running some of these enormous sweeps far more frequently simply because the cost of another run had effectively disappeared into the cost of owning the machine. That’s the part I hadn’t really appreciated before testing RTX Spark. The interesting economic argument for local AI isn’t necessarily replacing work you’re already paying to run in the cloud. It can make entirely new amounts of computation reasonable! My dudes, I did not think my brain would go down this path when I pulled this $7000 laptop out of the box. But it’s not a new idea. Computers are supposed to let you screw around Imagine if every command you ran on your first home computer had been metered. Every game of Doom added a few cents to your bill. Every weird batch file you wrote cost money to execute. Every night spent screwing around with Linux came with an itemized invoice the next morning. People would’ve used their PCs differently. One of the revolutionary things about the personal computer wasn’t merely that you could own a computer. Once you’d bought the damn thing, you could waste its compute however the heck you wanted! You could experiment, tinker, spend six hours doing something profoundly stupid, and occasionally stumble into something brilliant. Some of my funniest PC memories come from simply doing weird fun stuff with my computer. Cloud computing gave us access to astonishing amounts of computational power, but it also brought the meter back. That’s usually a perfectly reasonable tradeoff. Renting a gigantic pile of compute when you need it beats buying a gigantic pile of compute that sits idle when you don’t for a lot of people (though I suspect regular Full Nerd viewers, like me, strenuously disagree with the idea no matter how many “BRAD SAID GEFORCE NOW TAKE A SHOT” jokes happened over the pandemic). Local AI tilts that equation in a funky new (old) direction. The laptop on my desk has already been paid for, so when I wonder whether running another massive analysis might reveal something useful, the incremental cost of finding out is tiny. Sure, the laptop itself costs a fortune, but once you have it, you can mine it endlessly and do fun weird stuff with your computer. That doesn’t make computational efficiency obsolete. A faster model still saves power and time. Smaller models can be more practical. Generating mountains of AI garbage just because you can is still generating AI garbage. But after weeks of testing RTX Spark, I’ve stopped thinking about local AI purely as a cheaper place to run the same work. Instead, I’m starting to think its real advantage might be all the work you’d never bother attempting if you had to pay for every token.
Found in: PCWorld. Rating basis: single outlet; outlet tier 2 (unlisted); 1 feeds; feed position 5.