The data was public. The investigation was still difficult.
U.S. electricity information is distributed across operational feeds, market datasets, weather services, and interconnection records. Each source answers a narrow question, while analysts often need to compare several signals before deciding what deserves attention.
The design goal was therefore not another collection of charts. It was one workspace where a user could move from an overview to a balancing authority, route, anomaly, or transition signal without losing analytical context.
My contribution
I implemented the data acquisition, transformation, analytical logic, cloud storage, and interface. The product includes a batch ETL path, validated processing functions, a BigQuery serving layer, and a Streamlit application.
I also reframed the interface around decision questions: where forecasts look unusual, which interchange routes warrant inspection, how transition conditions differ, and where signals contradict one another.
Multiple sources were organized around one analytical context.
Acquisition and transformation run separately from the interface. This makes slow, paginated collection repeatable while keeping user interactions responsive.
Three choices made the product more useful.
Organize around investigations, not source tables
Executive briefing, anomaly detection, interchange signals, transition scoring, and reporting each begin with a user question.
Preserve context across views
Shared state keeps the selected balancing authority or market location in focus as the analyst moves between modules.
Keep interpretation transparent
Peer comparisons and contradiction flags use visible rules and component measures. They are prompts for investigation, not claims of trading or policy certainty.
Validation matched the project’s actual risk.
The code separates fetching, transformation, and presentation so processing behavior can be checked without launching the interface. Source schemas validate expected fields and types, while tests cover key calculations and missing-data behavior.
Methodological limits are also visible in the product. Physical interchange is not presented as proof of a tradable price spread, and the transition score is a transparent composite rather than a prediction of project success.
Outcome and limits
The project produced a complete public-data pipeline and an interactive analytical prototype. It demonstrates how data engineering, analysis, and interface design can turn fragmented sources into a coherent workflow.
- The intended user groups informed the design, but the project did not include formal user research or adoption measurement.
- Price coverage varies across regions, limiting some cross-market comparisons.
- Composite scores and interpretation cards remain heuristic and require analyst judgment.