story
My name is Sophie and I joined Kubrick in May 2022 as a Data Engineering consultant. Once I completed my training, I was placed within one of the world's leading energy companies.
I am a Data Engineer within the digital and data team - which is made up of several smaller squads that operate using agile principles. Workflow is managed using Azure DevOps, where the code repositories and deployment pipelines are. Kubrick's DevOps training was a great starting point for daily use of these tools.
I have contributed to many projects during my time with the client - including bringing data from new sources into a centralised platform that can be used across different products. Using Azure Data Factory, I have built end-to-end pipelines that extract, transform and load data into Hive tables using scripts written in Databricks. I have also been able to obtain similar results using Azure Functions as an alternative to Databricks, working closely with the data scientists. The functions are written in Python and so my training has been useful here. The new data sets have then been used (for example in PowerBI dashboards) to provide valuable insights into the uptake of digital products within the company. Some data has also been displayed within products by the front end engineers, with whom I work closely to ensure data requirements are met.
I have also been involved in the development of best practice and compliance tools for use across the team. This has included creating a script for removing any data from the Datalake and blob storage that is no longer required or no longer meets GDPR standards.
Kubrick's training in SQL, Python and Azure have been the most relevant for me while on client site and have been crucial to my daily work. I have also had the opportunity to gain new skills over the past year, using tools like Terraform, PowerBI, and Jira.
Latest insights
