The AI Energy Score initiative, co-led by Hugging Face and Salesforce in partnership with Cohere and Carnegie Mellon University, aims to establish a standardized framework for evaluating the energy efficiency of AI model inference. This project addresses the urgent need to assess and mitigate the environmental impact of AI systems, which are projected to consume significant amounts of energy in the coming years.

With the increasing ubiquity of large AI models, particularly large language models (LLMs), the environmental impact of AI has become a pressing concern. Despite these growing concerns, there is no consensus on what constitutes “AI” or how to comprehensively account for its environmental effects whatsoever. Existing studies primarily focus on energy consumption and emissions, with some adopting a life-cycle approach. However, there is a critical gap in standardized methods to assess the environmental footprint of individual AI systems.

The AI Energy Score initiative aims to fill this gap by introducing a standardized framework for reporting AI models’ energy efficiency. This framework includes:

• Specific Tasks and Hardware: Evaluating models across 10 common tasks spanning multiple modalities, using controlled and comparable metrics.
• Relative Scoring System: Providing a clear and recognizable system for users to evaluate model efficiency.
• Public Leaderboard: Publishing results on the AI Energy Score Leaderboard to track progress over time.

The initiative aims to empower various stakeholders, including researchers, developers, organizations, and policymakers, by providing actionable recommendations. These recommendations will help integrate energy efficiency considerations into decision-making processes, from developing and deploying AI models to selecting the most energy-efficient options.

By providing a standardized framework for evaluating energy efficiency, driving transparency and encouraging the development of lower-impact models, the project aims to promote AI sustainability and contribute to a more responsible AI future.

“If you’re a customer of large AI models, you hold the power. Your voice, especially in enterprise procurement, can drive transparency, and thus AI sustainability.” Boris Gamazaychikov, Head of AI Sustainability at Salesforce :