Job Title: Data Engineering Lead
Reporting To: Head of Data Engineering
Business: Technology and Innovation
Team: Technology – Data Engineering
Location: India
Lloyd’s List Intelligence has proven success across the globe with offices in APAC (Singapore, China, Hong Kong and Japan), EMEA (UK, UAE) and USA. We now look to further expand our global footprint and add a location that gives us the capability to build teams of highly talented colleagues and deliver growth in our Technology, Data and Product functions.
Role Profile
Reporting into the Head of Data Engineering, we are seeking a Data Engineer Lead to join our team of existing Data Engineers to assist with the ongoing maintenance and innovation of our data processing and ingestion pipelines and systems.
The position presents an exciting opportunity for an experienced Data Engineer to manage and technically lead a team of Data Engineers responsible for delivering project deliverables involving back-end development, maintenance of our data ingestion, streaming, and analytical systems. The team is tasked with ensuring our various data ingestion systems achieve a high level of reliability and work closely with our product, data science and software engineering teams to support existing and new data workflows.
This is a full-time permanent role reporting to the Head of Data Engineering. This role is balanced working in our Chennai office.
Key Responsibilities
... - Management of the global Data Engineering team this includes training, coaching, mentoring and recruitment
- Driving the team and individuals’ objectives according to the KPIs
- Hands-on implementation and delivery of data solutions
- Collaborate with Product teams ensuring that raw data is cleansed and transformed and useable by downstream consumers (Data Scientists, ML Engineers, BI analytics)
- Assist and advise on the re-development and modernisation of end-to-end ETL pipelines and introduce new technologies where appropriate in a real-time streaming environment dealing with very large data volumes
- Working with AWS cloud infrastructure (specifically SQS, SNS, Redshift, OpenSearch, Athena, DynamoDB, Kinesis, step functions, lambda) to develop, innovate and maintain data flowing through various queues and data warehouses
- Working with a variety of data repository platforms (including SQL stores such as Oracle), as well as implementing data visualisation and network analysis (e.g GraphDB)
- Maintain and ‘productionise’ machine learning and AI models
- Assist in the creation of next generation data ingestion platforms – sourcing data using Webscrapes, APIs, Email, and flat file (FTP) methods
- Understanding conflict resolution methods and assist subject matter experts in the debugging on data ingestion and managing overall feed uptimes across a large set of data collectors.
- Create and maintain detailed documentation and functional design specifications including data flows and data conversion
- Provide technical information to assist in the development of client facing product documentation
- Adhere to change management protocols and version control
- Present advanced technical designs with non-technical stakeholders
What we are looking for:
Essential
- At least 5 years’ experience as a Senior or Lead Data Engineer
- Proven and demonstratable experience of Python, AWS, and SQL/ Oracle
- Bachelor’s honours degree preferably in computer science, or engineering field, or demonstrable technical experience in a related engineering discipline
- At least 3 years of ETL experience using flat files to transposed Data Models
- Experience in troubleshooting and debugging Python, SQL (Oracle & other SQL platforms), SNS, SQS, Kinesis, C++, AWS Lambda
- Strong Data Mining and Analytical skills required to encode sophisticated business transformation logic in a pipeline and serve predictive models
- Familiarity with the design, development and maintenance of best-in-class BI capabilities, including data warehouse data structures and data pipelines (Redshift, Athena etc)
- Familiarity of architectural design patterns for micro-services leveraging relational and big data technologies
- Familiarity with Agile development process
- Exceptional analytical thinking ability, ease with quantitative analysis, and excellent problem-solving skills
- Self–discipline and willingness to learn
- Ability to make connections with technical and non-technical stakeholders
- Excellent verbal and written communication skills
Desirable
- Previous experience of managing a team
- Domain knowledge or experience working within the maritime or an adjacent industry
- Previous experience using C#, C++, and JAVA
- Experience of streaming or ‘real-time’ processing at high data volumes
- Familiarity of architectural design patterns for micro-services leveraging relational and big data technologies
- Demonstrable experience working with Big Data environments
- Experience of the SAS Analytics platform of particular interest
We know that sometimes the 'perfect candidate' doesn't exist, and people can be put off from applying for a job if they don't fit all the requirements. If you're excited about working for us and have most, but not all, of the skills or experience we're looking for, please go ahead and apply. You could be just what we need!
We believe strongly in the value of diversity and in creating supportive, inclusive environments where our colleagues can succeed. As such, we are proud to be an Equal Opportunity Employer. We do not discriminate based on race, color, ancestry, national origin, religion, mental or physical disability, medical condition, genetic information, sex (including pregnancy, childbirth, and related medical conditions), sexual orientation, gender identity, gender expression, age, marital status, military or veteran status, citizenship, or other protected characteristics under federal, state, or local law.
Please let us know of any adjustments we need to make for you to show us your best self.
About us
We are the industry experts delivering actionable maritime insight, data, technology, and analytics trusted by 60,000 professionals across the globe to drive commercial advantage, evaluate risk, and support the efficient and lawful movement of seaborne trade.
Our data is collected only from the most reputable partners and accurate sources, many of which are exclusive to us and our customers.
Our advanced analytics, artificial intelligence, and industry expertise transform our unparalleled data into powerful insight delivered through data and analytics platforms and services, news and commentary, and publications that help professionals in maritime operations, risk, and compliance stay well informed.
Leading seaborne trade through change with data transparency, innovative technology, and human ingenuity.
Our Values and Behaviours
For LLI, these aren’t just the wallpaper. These are the behavioural expectations led and role modelled by our CEO Michael Dell, and they are also the design principles of an change or initiative we put in place.
Build trust & Transparency
Transparency and trust needs to be at the heart, with commitment to honest communication about
individual progress, as well as company news, regular townhall meetings, clear messaging about the
company's direction, and transparency in addressing internal challenges such as gender and race
inequality and recognition/reward.
Embrace collective brilliance
Collaboration and teamwork are the foundation of our collective success. Our data highlights the importance of
working together cross-company and communicating with agility to solve problems. There is a strong emphasis
on friendly, authentic relationships and courageous collaboration.
Work with respect
Respect for one another's well-being, fairness in recognition of effort, inclusivity in approach, and
celebration of cultural differences is critical. There is emphasis placed on social awareness and
positive impact.
Take responsibility
The importance of being accountable to each other cannot be underestimated, recognizing each
other's work, and maintaining a challenge culture to ensure credibility, dependability, and
trustworthiness. There is focus on stepping up to take on more if it helps the collective organization.
Simplify the complex
There is a focus in our data on delivering excellence, recognizing and learning from failures, and
a commitment to delivering work that puts customers first and provides market-leading
expertise.
Move fast & be bold
The data sources emphasize the need to embrace change in an agile way, with a positive approach to
challenges and transformation. It includes a desire to evolve and remain industry leaders, embracing
challenges and promoting a culture of curiosity and continuous learning.
experience
8
