The Case for Open Models
Why Openness Matters in AI
A critical debate is emerging about the future of large language models (LLMs): closed versus open. While flashy, proprietary AI solutions dominate headlines, there's a compelling case to be made for open models that offer flexibility, transparency, and long-term sustainability.
The AI Gold Rush
We're witnessing an unprecedented surge of AI-related products and services. Every startup seems desperate to slap an AI label on their offering, often prioritizing the technological novelty over genuine problem-solving. However, you need to deliver real value to achieve product stickiness. Using the latest O-x model wouldn’t achieve this alone.
Successful AI products will use machine learning as one ingredient in a broader solution, not as the entire recipe. When you build a product on a closed, proprietary model, there’s a compelling argument that you're essentially constructing your entire business on shifting sands.
The Risk of Closed Models
There are many risks when using a closed model as the backbone of your application. Below are a few examples.
Model Deprecation
Model providers can deprecate their models, including the one you’ve decided to build your service upon a few months ago. This often happens when they release new models. While new models are generally more capable, there are applications where the older model is notably better. This could be due to the way the prompts have been optimized specifically for the older model, or the fact that the newer model is simpler worse at this task.
An example of this is gpt-4o-2024-05-13 being much worse at generating JSON output compared to gpt-4 and gpt-4 turbo which caused workflows for generating structured data to break.
Dramatic Price Changes
OpenAI’s financials shows they’re still in the red. While a big portion of their costs is R&D, it’s been widely reported that model providers are subsidizing their services and offering it at lower-than-cost prices. Additionally, OpenAI has raised ~$20B at a valuation of ~$157B and is planning on transitioning to a for-profit organization. While we haven’t seen a price hike for LLM services so far, the investors will want returns on their investments and this might mean significantly raising the prices. It happened with ride-sharing (Uber & Lyft) companies as well as streaming services (do you remember when Nextflix monthly subscription was a single digit?).
UPDATE:
As I was finalizing the draft of this post, Sam Altman just shared OpenAI is losing money even on its newly-introduced $200 Pro subscriptions.
Denial of Service
When you’re using a service like ChatGPT or OpenAI’s API, you’re governed by their terms and conditions. They can decide to stop providing their service if they “determine you’ve breached these terms”. A notable incident is OpenAI banning Bytedance
No Guaranteed Long Term Stability
Even using third-party hosting services, like AWS Bedrock and GCP’s VertexAI to host Claude, doesn't guarantee long-term stability. A model you depend on today might vanish tomorrow, potentially crippling your entire operation. Alternatively, the service might be killed altogether. Google is especially know for killing services that have existed for years with millions of users.
Insufficient or Unstable Capacity
Limited provider choices and regional constraints pose real challenges with closed AI models. GPT is tied to OpenAI/Azure, Claude to AWS/GCP/Anthropic, and Gemini to Google's ecosystem.
Regional capacity caps add another layer of complexity. If you're operating in the UK and need GDPR compliance, you're restricted to UK-south's allocation, competing with countless other users. Need more throughput? You'll have to wait for capacity upgrades or request provisioned throughput, which still depends on regional limits. With no easy path to self-hosting or switching providers, you're essentially locked in and vulnerable to service disruptions.
Furthermore, when outages occur, there's little you can do but wait it out. You can’t simply switch to another cloud provider and deploy the model or self host on your infrastructure easily.
Even if you’re content with the latency, you’re still relying on a single (or two) point of failure and there’s not much you can about service outages.
The Open Model Advantage
Transparency
Open models provide complete access to their internals: architecture, weights, inference code, and (sometimes) the training data. This enables proper security auditing, bias detection, and precise understanding of model behavior independently by third parties. Organizations can validate vendor claims and understand exactly what they're deploying.
Tailor Models to Specific Needs
Technical approaches for adaptation include fine-tuning for domain specialization, quantization for memory optimization, speculative decoding for faster inference, and pruning for size reduction. Organizations can add or modify content filters (or disable them altogether) to match their specific use cases.
Deploy Anywhere, Anytime
Self-hosting models enable air-gapped deployments and complete network isolation. Small models can run on edge devices for offline operation or in hybrid setups combining local and cloud resources. This flexibility is crucial for organizations with strict security requirements, or for personal chats with sensitive data: many people love chatting with their LLMs offline on 12-hour cross-continent flights.
Collective Progress
The open-source community continuously improves these models through shared development. Examples include the implementation of Classifier-Free Guidance (CFG) in llama.cpp for constrained outputs (way before OpenAI rolled structured outputs), specialized training recipes, and optimization techniques (e.g. min_p sampling) that benefit the entire ecosystem rather than remaining proprietary.
Beyond Technology: A Strategic Imperative
This is particularly crucial for organizations with complex, sensitive requirements like government agencies. Open models like Llama 3, DeepSeek, and Mistral aren't just technological alternatives, they're strategic assets that offer:
Enhanced security controls
Compliance flexibility
Independent operation
Reduced vendor lock-in
Llama 3 models have been adopted by organizations in the defence, legal and healthcare industries where there is definite need for complete data privacy
The Tradeoffs: Nothing Is Free
Let’s be clear: open models aren't a magic solution. They require higher initial technical and capital investment. They also still lag behind frontier models, especially in mulitmodal and multilingual capabilities.
Not every organization will find them immediately suitable. But for those willing to invest, the long-term benefits can be substantial. Additionally, you can start simple: platforms such as Hyperbolic.xyz and together.ai offer secure LLM hosting services where you get charged for the number of allocated GPUs.
Shameless self-promotion: Self-hosting LLMs is one of the most requested tasks I do as an ML consultant. Feel free to reach out if you wanna discuss a tailored solution for your needs.
Looking Forward
As AI continues to transform industries, the most successful organizations will be those that maintain strategic flexibility. Open models represent more than a technological choice, they're a philosophy of ownsership, innovation, control, and collaborative progress.
The future of AI isn't about who has the most impressive model today, as these change rapidly, but who can adapt, customize, and evolve their solutions tomorrow.