Senior Data Engineer
Job Overview:
We are seeking a senior data engineer to join our Data and Analytics IT Team. This candidate will work in an agile development environment and build a Data Lake and Data Warehouse solution to meet the needs of Data Science and Business Management teams. The solution will integrate data from external sources, IOT sensors from client database and business data from CRM, Billing and other IT systems. This integrated data will be used by Data science teams to create and test solutions for energy optimisation, energy trading and predictive fault detection. Moreover, the solution will be used to develop Power BI reports for internal and external stake holders.
You will have an opportunity to contribute to the company platform developing through analysis, design, implementation and delivery of industry leading big data and data warehousing solutions. The solution will be developed using Azure's Cloud services and the candidate should have hands on experience of using Azure Data Platform. Ideally the candidate would have gone through the whole development life cycle from design to implementation to production on Azure during previous projects.
Main Responsibilities:
- Leading the design and delivery of innovative data solutions using latest services from Azure eco-system.
- Exploiting industry best practices and your experience to create a data platform to enable business growth.
- Liaise with internal and external stakeholders in order to collect and document technical requirements.
- Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance and innovate solutions and improve process efficiency.
Required Experience and Skills:
- A minimum of 5 year's professional experience is required.
- Azure/AWS Experience (3+ years). Azure Data Certification a plus
- Experience with Data Warehouse design and Kimball methodology
- ETL experience using SSIS and Azure Data Factory
- Experience working with SQL Server, SQL Data Warehouse and Azure Data Lake
- SQL server schema design and optimization to meet performance requirements
- Python programming for Data engineering and clean up. Spark/Data bricks experience will be a plus
- Experience of working with Power BI and DAX programming.
- Good understanding and experience of working with data security options on Azure.
- Proven ability to translate complex business requirements into design of end-to-end data systems including scalability, HA and DR.
Technologies:
- Technologies: SQL Server, data integration, ETL Design, Azure Data Lake, Azure Data Factory, Azure SQL Server and Azure Synapse.
- Core Programming languages: T-SQL, Power Shell, Azure API, DAX.
Desired Qualifications and Other Experience:
- A minimum of 5 years of experience, a third level degree in Maths, Physics, Engineering, Computer Sciences or a related discipline.
- Commercial experience in Data Warehousing and business intelligence projects.
- Experience in handling large and complex datasets using multiple programming languages (T-SQL, Python, R etc.) is essential.
- Experience analysing multiple business domains and translating them to dimensional designs for analytics and reporting.
- Basic knowledge of or exposure to machine learning, data visualisation, and artificial intelligence techniques. Experience working with Data Scientists and Machine Learning teams.
- Data governance experience desirable.
The following experience would be advantageous but not essential:
- Experience working in an energy related industry combined with a passion for energy sustainability and innovation.
Benefits:
- Competitive salary.
- Pension.
- Life cover benefit.
The candidate must have a valid visa to work in Ireland.
