The Data Engineer Job Description
...
The Data Engineer/Techncial Lead is an expert in SQL development support to the data and analytics IT experts in database design, data flow and data analysis activities. The Data Engineer plays a key role in the development and deployment of innovative big data platforms for advanced analytics and data processing. The Data Engineer defines and builds the data pipelines that will enable faster, better, data-informed decision-making within the business. Deep experience in developing enterprise data management strategies including operational data stores, data warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. Accountabilities:This resource will be part of the Aspen Unified Pilot for Pernis to migrate existing integrations for on-premises application (Aspen Planning & Scheduling for refinery) to a cloud version of the same, Will act as Technical Lead for this body for work and needs to own and lead workstream to conduct discovery, documentation, developing existing ETL with deeply embedded business logic into APIs. Needs to be capable of high level of business interface and explain complex topics to a non-IT technical audience.• Work together with the Solution- and Data Architect to design the next generation big data platform to support the digital marketing platform.
• Design and implement modern data pipelines to extract, clean and process data in batch and real-time from different data sources.
• Use the latest development, testing and deployment techniques to quickly deploy new releases to e.g. deploy new data pipelines and add data sources.
• Help establish best practices around data governance, data security- and privacy (GDPR).
• Work together with the solution- and data architect to the define the target data strategy.
Drives and applies Agile Framework within his/her operating environment. Will take direct reports, when applicable, through Agile Framework and its respective tools.
show more
The Data Engineer Job Description
The Data Engineer/Techncial Lead is an expert in SQL development support to the data and analytics IT experts in database design, data flow and data analysis activities. The Data Engineer plays a key role in the development and deployment of innovative big data platforms for advanced analytics and data processing. The Data Engineer defines and builds the data pipelines that will enable faster, better, data-informed decision-making within the business. Deep experience in developing enterprise data management strategies including operational data stores, data warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. Accountabilities:This resource will be part of the Aspen Unified Pilot for Pernis to migrate existing integrations for on-premises application (Aspen Planning & Scheduling for refinery) to a cloud version of the same, Will act as Technical Lead for this body for work and needs to own and lead workstream to conduct discovery, documentation, developing existing ETL with deeply embedded business logic into APIs. Needs to be capable of high level of business interface and explain complex topics to a non-IT technical audience.• Work together with the Solution- and Data Architect to design the next generation big data platform to support the digital marketing platform.
...
• Design and implement modern data pipelines to extract, clean and process data in batch and real-time from different data sources.
• Use the latest development, testing and deployment techniques to quickly deploy new releases to e.g. deploy new data pipelines and add data sources.
• Help establish best practices around data governance, data security- and privacy (GDPR).
• Work together with the solution- and data architect to the define the target data strategy.
Drives and applies Agile Framework within his/her operating environment. Will take direct reports, when applicable, through Agile Framework and its respective tools.
show more