ProtoStar puts AI’s climate forecasts up against simpler methods

Before an AI model can make a useful climate prediction, someone has to make the weather records usable. ProtoStar, a new collaboration between Hugging Face and research institutions, is putting those two tasks into the same test: ask teams to work with historical city data, then document what helped or obstructed them.

The initiative was announced on October 29, 2026. Its first challenge, hosted by Carnegie Mellon University and scheduled for November 2–13, concerns climate risk in 210 U.S. cities. The question is practical: can AI extract useful predictive information from shared research, and can another team understand and build on the work?

The past supplies a test

The current challenge rules ask teams to act as if they were forecasting from the end of 2018, predicting annual conditions in 2019–2025. Inputs that reveal later weather are prohibited. Targets include annual average temperature, precipitation, the hottest day and cooling degree days, an indicator of air-conditioning demand.

Comparison methods are deliberately simple: repeat the last year, use a 30-year average, or extend a linear trend. Predictions are scored against recorded conditions. This makes the exercise a test against observations rather than a demonstration illustrated with hypothetical future outcomes.

There is a documentation discrepancy worth noticing before participating. Hugging Face’s launch post describes a 2019–2026 prediction period and final forecasting results in early 2027. The current challenge page instead specifies 2019–2025 and judging after the deadline. This article follows the current participant rules; teams should check them again before submitting.

Open files still need an operating environment

Carnegie Mellon frames ProtoStar as an infrastructure effort. Data may be available yet fragmented across systems, difficult to combine or insufficiently documented. Its University Libraries has funded access to Hugging Face’s Academia Hub as a shared environment for working with datasets and models, initially supporting the challenges.

The wider project joins universities including Purdue, Georgia Tech, Arizona State and George Washington through the Open Forum for AI, alongside Hugging Face, the Astera Institute and the Open Research Community Accelerator. The launch plan calls for successive challenges at participating institutions, rather than treating one forecasting competition as proof that the infrastructure problem is solved.

George Washington’s announcement makes the institutional work concrete. Its Open Source Program Office will help researchers share data, code and models through a university presence on Hugging Face and bring the challenges to its community. The university also plans hands-on experience for students and classroom use by instructors.

Hugging Face’s earlier public-data preparation guide shows why that assistance matters. Its Massachusetts education-data example describes spreadsheets with inconsistent headers and column names, numbers stored as text, and definitions that changed across years. Converting a file is only part of the job; interpreting its contents correctly requires documented decisions.

A failed reuse attempt can still teach something

The challenge’s judging criteria explicitly recognize documented negative results and feedback about barriers to reuse or reproduction. A polished forecast is therefore only one possible contribution.

That design makes the exercise more informative than an accuracy leaderboard alone. A model can perform well while leaving its inputs or methods difficult to inspect. Conversely, a careful account of missing metadata or an unreproducible environment can identify work the next researcher needs someone to fix. Those are implications of the challenge design, not results it has already delivered.

For a city or research team considering these tools, the eventual evidence to watch is twofold: whether predictions improve on the reference methods, and whether the released materials let others repeat the work. ProtoStar has set up the test. It has not yet supplied the answer.