Sitelet https://github.com/quintel/etsource/pull/3465
Skip to content

Direct emissions import of Dutch regional emissions.csv from regional data pipeline - #3465

Open
kndehaan wants to merge 1 commit into
masterfrom
emissions-dutch-regional
Open

kndehaan wants to merge 1 commit into
masterfrom
emissions-dutch-regional

Conversation

@kndehaan

@kndehaan kndehaan commented Jul 23, 2026 •

Copy link
Copy Markdown
Member

Context

This PR updates the emissions.csv files to Dutch regional dataset, imported from ETDataset as result of the Dutch regional data pipeline.

Implemented changes

Data pipeline related:

  • New data pipeline added for Dutch regional datasets. Emissieregistratie is used as source applies and conversion steps are applied to generate the required CSV format.
  • Automatic validation is done in the data pipeline.
  • Intermediate emissions.csv are exported for manual checks.
  • README added
  • Restructering of folders in the tools/emissions folder

Related

Goes with:

Checklist

  • I have tested these changes
  • I have updated documentation as needed
  • I have tagged the relevant people for review

@jort-wolda jort-wolda left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. To be able to run the model locally I had to (locally) rebase with master. I didn't push this but since there were no conflicts it should be fine to merge.

@kndehaan
kndehaan force-pushed the emissions-dutch-regional branch from 98c8067 to b7ea7cf Compare September 16, 2026 10:14
@kndehaan

Copy link
Copy Markdown
Member Author

Ready for final review by @mabijkerk

@mabijkerk

Copy link
Copy Markdown
Member

I haven't had the time to look into these PRs. One thing that I did not already is the role and data pipeline for 1.A.3.e (Other transportation). It requires further investigation. I had AI summarise some of the discussion:

Flag for review: possible double counting of 1.A.3.e (Other transportation).
In ETSource, transport_non_specified is mapped to CRT code 1.A.3 (all of transport), while other_other_transportation is a separate 1.A.3.e subsector. Both are in the direct_emissions group and both are summed. In ETDataset-public, the ETLocal key energetic_emissions_other_ghg_transport takes the full 1.A.3 total (greenhouse_gas_emissions.xlsx, Sectors!O16) and does not subtract 1.A.3.e. The NL municipal mapping (all_ghg_mapping.csv) also assigns gas transport N₂O (1.A.3.e.i) to that transport key. As a result, any regional dataset that also gets a value for other_other_transportation will count 1.A.3.e twice. For the country emissions.csv files, 1.A.3.e matches the EEA data exactly, but the transport value matches neither full 1.A.3 nor 1.A.3 minus 1.A.3.e. Someone needs to check in the pipeline whether 1.A.3.e is subtracted from transport.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants