alpaca

Inference at the Edge: Running a Large Language Model Chatbot on Consumer Hardware

Mike ConnellyArtificial Intelligence, Technology

Large language models can now run entirely on consumer laptops, with no cloud connection required. We explore how hobbyists and open-source developers are deploying instruction-tuned, locally optimized LLMs on standard hardware and what that means for privacy, cost, and accessibility. The practical tradeoffs between model size and response quality are what make this shift more consequential than it first appears.
Continue reading Inference at the Edge: Running a Large Language Model Chatbot on Consumer Hardware

A Primer on Conversational Artificial Intelligence Agents & Large Language Models

Mike ConnellyArtificial Intelligence, Technology

Conversational agents and the large language models (LLMs) have become increasingly proficient at mimicking human language and behavior so that they can respond to a wide variety of instructions. At their mathematical core, they are statistical distributions of token adjacency probabilities. This article covers how LLMs work, how they evolved from first-generation transformers to instruction-tuned systems, and what the temperature parameter actually controls.
Continue reading A Primer on Conversational Artificial Intelligence Agents & Large Language Models