Are Open Source LLMs Energy Efficient?

Overview: I use Ollama to run open source LLM models with various data sources to ask questions and include cited sources. If you would like to try this out on your machine (or try out any of the other examples) you can clone the following github repository: https://github.com/jbsoftware-io/gen-ai-starter

Model Used: Mistral:7b

Data Source: Web (Brave Search API)

Question: Are small open source LLMs more energy efficient than proprietary large models?  If so, why and how can that help small business?

Answer:

Small open source LLMs are generally more energy-efficient than proprietary large models for several reasons:

1. Model Size: Open source LLMs, especially those that are smaller, require less computational power to run compared to large proprietary models. This is because they have fewer parameters and use less memory.
2. Training Efficiency: Training smaller open source models often requires fewer resources than training large proprietary models. For instance, XGen uses only 7B parameters for training, which is significantly less than most powerful open-source LLMs like LLaMA 2 or Falcon.
3. Economies of Scale: Smaller businesses can leverage cloud services to run their AI applications with relatively low costs compared to running large proprietary models locally. This is because they only pay for the computational resources they use, and these resources are often shared among many users, leading to economies of scale.
4. Reduced Carbon Footprint: By consuming less energy during training and inference, smaller open source LLMs contribute to a reduced carbon footprint compared to large proprietary models. This is particularly important for businesses that prioritize sustainability.

By using small open source LLMs, small businesses can save on operational costs related to energy consumption, making their AI applications more cost-effective. Additionally, the democratized access to cutting-edge AI technology provided by open-source models can help these businesses stay competitive without incurring the high costs associated with proprietary models.

Cited Sources:

• 8 Top Open-Source LLMs for 2024 and Their Uses | DataCamp
• How to Beat Proprietary LLMs With Smaller Open Source Models
• Open-Source LLMs vs Closed: Unbiased Guide for Innovative Companies [2025]