Documenting a Data Pipeline with Ardoq
Hafslund uses Ardoq to visually explore their data pipeline documentation, helping them connect key metrics to the data sources they come from.
Employees and managers at Hafslund are helped to make informed decisions by a number of dashboards and BI-tools displaying key metrics from every aspect of the business. However, these key metrics are generally approximations or aggregations of raw data pulled from an even greater number of sources.
Hafslund is Norway’s largest power grid operator, a leading supplier of electricity and a major producer of renewable energy from hydropower and district heating.
Hafslund has largely automated the import of raw data into the data platform where these calculations are made, and also the export to the various dashboards and BI-tools. Because of the complexity of these systems, there is a need for documenting data input and output. Thanks to Ardoq, it is possible to navigate through a visual representation of the data pipeline rather than having to dig through code or unstructured documentation.
“Ardoq is an integral part of our data platform.”
– Ståle Heitmann, Senior Developer at Hafslund
By using Ardoq, Hafslund benefits from:
- An automated data input to Ardoq through our API, which saves time spent on manual data entry while ensuring the documentation remains up-to-date at all times.
- A range of automatically generated visualizations of their data, all created and updated in real-time, thereby avoiding time spent on drawing and redrawing diagrams and graphs.
- All the data in Ardoq being structured and organized – this allows readers to effectively search, sort and filter to find what they’re looking for.
Now, if someone at Hafslund wants to know how a figure on a dashboard was derived (for instance, average electric meter readings), they are able to identify where in the data platform this figure has been pulled from, and from which source it originated (all the way down to which column in which database) by navigating through the visualizations in Ardoq. If this figure is an aggregation of data pulled from several sources, they can easily identify the relevant aggregation report to pinpoint what raw data has been aggregated as well as how often this report runs, when it last ran and when it will run next. This capability empowers everyone in the organization to better understand the data they rely on.