Moving from idea to output with local power rather than cloud

AI and chatbots might be solving the world’s problems and allowing everyone to become creative in ways they could only dream of — but they’re not without their problems. At the moment, most AI services run in the cloud. When you ask them to do something, they make use of remote computing hardware to complete the task, whether it’s answering a simple ChatGPT question or creating a complex video.

This is great because the hardware you’re using is largely irrelevant: you get the same results on an ancient smartphone as you would on a high-end PC. However, cloud-based computing is pushing data centres to their limits, and subscription prices are creeping upwards as we become increasingly reliant on AI to help with everything from life advice to making movies.

You might have reasons to not use cloud-based applications at all. If latency or internet speed/access is an issue, being tied to an online-only service can be risky. Equally, depending on what you’re working on, you might not want or even be able to share information on files online. It doesn’t take long to find numerous cases of sensitive information being put in places that it shouldn’t have been.

The answer is to run AI locally, on your desktop or laptop. This bypasses the need to send data to the cloud, and if you’ve got a decent machine it just makes sense to use its powerful hardware for your AI needs. If you don’t already have an AI-optimised machine, it might be time to look for an upgrade, especially with Amazon Prime Day now upon us.

Going Local

With a local Large Language Model (LLM), you can download open-source models and run them on your own hardware. They also completely close the loop on privacy—you may have noticed that most public LLMs use your data to train their services, which is how they get better, but it can also raise issues of who gets to see what you share.

A local NVIDIA RTX GPU will only pull power for the exact seconds it takes to process a query, dropping back down to an efficient idle the moment it finishes. It does so without any latency or peak-hour server lag, and it’ll even work when your internet goes down or in some apocalyptic scenarios.

Understandably, running an LLM locally sounds like a formidable task, with an extensive setup process. But software such as LM Studio and AnythingLLM are simple programs that you can run with a single click. You don’t need to do any coding or complicated setup, and you don’t even need to log in to use the systems; just download the app and select your LLM model from among some of the top choices around.

LM Studio lets users download and host LLMs with an easy-to-use interface, while also giving customisation options to fine-tune your new local AI assistant. Similarly, AnythingLLM also allows users to run local LLMs, and has a dedicated community hub with ways to extend their agentic AI capabilities. Best of all, both programmes feature accelerations for NVIDIA RTX GPUs that significantly cut down on time waiting for responses.

The chats

Once they’re installed, they work just like an online LLM such as ChatGPT or Google Gemini, and you can ask them for everything from advice on how to email your client to how to plan your day.

Of course, there are some things local models can’t do versus their larger online cousins. Local models have progressed drastically and are able to access the internet now through features in programs like AnythingLLM if the need arises, but they are limited by the size available on your device, holding fewer context parameters than models provided by huge data centres.

An online LLM has to receive the file, analyse it on the cloud, and then present an answer, local LLMs can take the files on your computer, such as 1000-page PDFs or 200-MP images, and tell you what you need to know about them in seconds. As you’re hosting the model yourself, you also avoid being tied down to a costly subscription with any services.

Make yourself ComfyUI

While local LLMs can take care of the day-to-day rigmarole of chatting, they’re very limited when it comes to image and video generation. This is where ComfyUI comes in — it’s a tool that allows you to integrate text to image generation models in a modular fashion, inviting you to connect nodes to dictate how your image is generated, upscaled, and stylised, offering a level of control that chatbot-based image generators can’t match.

If you’re prototyping an image as part of a project, for instance, you’ll often need to make quick small changes while maintaining the rest of the design or image. Rather than praying the online AI models take your “Don’t change anything but this” prompt seriously, ComfyUI can help precisely change what you request, like keeping a character consistent through generations, or specifically changing lighting effects.

Pushing pixels locally, however, requires high-speed Video RAM (VRAM)—the physical memory built onto your graphics card. Think of VRAM as your AI's desk space. To generate images or run models at maximum speed, they must fit entirely into your GPU's memory.

This is where dedicated RTX hardware becomes critical. A device with an entry-level card with 8GB of VRAM (such as a GeForce RTX 5060) is fantastic for running efficient, compact language models, serious creators making use of ComfyUI pipelines should look toward the creative sweet spot of 12GB to 16GB found in cards like the RTX 5070, or go all-in with the 24GB flagship, the RTX 5090.

Keeping things local

Cloud-based versus local AI is a bit like the difference between going to a gym and having a premium workout setup at home. As demand increases during peak hours at the gym, you frequently find that you have to wait in line for the best equipment, giving you less efficient time to get your work done, all while paying a recurring monthly fee.

Similarly, using local AI means no waiting for your prompt’s turn in the queue, and no subscription fees. If you invest in your own high-end hardware at home, it’s yours outright after an initial investment — no recurring charges for credits or tokens, or limits on requests. You can use it the exact second you want in total privacy, and it’s tailored perfectly to your specific needs.

If you are ready to break free of the metaphorical peak-hour gym crowds and experience the total freedom of your own private digital studio, Amazon’s Prime Day sales are the perfect highway to secure a deal on a GeForce RTX 50-series desktop, laptop, or GPU upgrade – and bring your AI home.