Datasets
Public time series, cleaned, joined, and validated before you open the file. All five datasets are free on Kaggle and Hugging Face under CC BY 4.0, with Arimancy LLC as licensor.
Power markets, one shared schema
ERCOT, CAISO, and PJM hourly demand joined to population weighted ERA5 weather. The three tables share one schema, so features and models written against one grid run on the other two without rewriting anything. All three baselines share one evaluation frame: trained through 2024-12-31, tested from 2025-01-01 onward, roughly 18 months. Demand history and calendar features close most of the gap to the seasonal naive on their own; the weather join adds roughly 0.4 to 0.8 percentage points of MAPE reduction across the three grids. Solar features are included for solar generation and price modeling, not because they drive the demand baselines. What the join buys you is documented provenance, a verified population weighting, and a fresh tail that carries preliminary recent months straight from the EIA API.
ERCOT Hourly Demand + Population-Weighted Weather
- rows
- over 96,000
- 2015 to 2026
- 11 yrs
- columns
- 29
- baseline MAPE
- 4.84%
Texas grid demand joined to population weighted metro weather, with calendar and degree hour features. A LightGBM baseline reaches 4.84% MAPE against an 8.72% seasonal naive on the shared evaluation frame. Hourly, July 2015 to July 2026, updated quarterly.
CAISO Hourly Demand + Population-Weighted Weather
- rows
- over 96,000
- 2015 to 2026
- 11 yrs
- columns
- 35
- baseline MAPE
- 2.71%
California grid demand on the same schema as ERCOT and PJM, with solar and cloud cover features for behind the meter solar work. A LightGBM baseline reaches 2.71% MAPE against a 6.04% seasonal naive on the shared evaluation frame. Hourly, July 2015 to July 2026, updated quarterly.
PJM Hourly Demand + Population-Weighted Weather
- rows
- over 96,000
- 2015 to 2026
- 11 yrs
- columns
- 44
- baseline MAPE
- 3.42%
Demand for the PJM Interconnection on the same schema as ERCOT and CAISO, weighted across the region's major metros. A LightGBM baseline reaches 3.42% MAPE against an 8.78% seasonal naive on the shared evaluation frame. Hourly, July 2015 to July 2026, updated quarterly.
Other public time series
Observed Weather for Every US Flight, 2019 to 2025
- recent 24 mo rows
- 14M+
- recent 24 mo columns
- 211
- airport weather rows
- 22.2M+
- airport weather columns
- 28
Every US domestic flight from January 2019 to July 2025 joined to observed weather at both the origin and the destination airport. The flagship flights table carries 42.8M+ flight rows; a recent 24 month table and an airport hourly weather table ship alongside it. A complete historical panel with a fixed end date: NOAA stopped ingesting global hourly data on August 27, 2025.
CFTC Commitments of Traders Weekly Panel
- panel rows
- 3.99M+
- panel columns
- 17
- features rows
- 1.02M+
- features columns
- 78
Every CFTC Commitments of Traders report family in one tidy weekly panel, 1986 to the most recent release, with a wide features table of net positions, concentration ratios, and COT indices. Updated weekly.
Sources and attribution
Arimancy LLC is the licensor of the derived datasets above, all published under CC BY 4.0. The upstream sources are credited below.
Weather: Weather data by Open-Meteo.com (CC BY 4.0). ERA5 reanalysis: Hersbach et al. 2023, ECMWF, doi:10.24381/cds.adbb2d47.
Demand:U.S. Energy Information Administration, Form EIA-930, cleaned and gap filled via Catalyst Cooperative's PUDL project (CC BY 4.0).
Futures positioning: Commodity Futures Trading Commission, Commitments of Traders reports.
Flights and airport weather: Bureau of Transportation Statistics On-Time Performance data and NOAA Integrated Surface Database observations. Airport coordinates from OurAirports.
Data from EIA, CFTC, BTS, NOAA, and the US Census Bureau are US Government works in the public domain. None of the agencies named here endorse Arimancy or these datasets, and any errors introduced in cleaning or joining are Arimancy's alone.