Roles & Responsibilities:
- Expected to be an SME, collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams. ...
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5+ years of overall IT experience, which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design, build and deployment of Python-based applications.
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR, API Gateway, RDS instance, and Lambda.
- Understand complex data sets and ETL processes, and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
- Experience with cloud-based data platforms and distributed computing frameworks.
- Ability to optimize data workflows for performance and scalability.
- Familiarity with database management systems and data warehousing concepts.
- Competence in scripting and automation to streamline data processing tasks.
experience
8show more Roles & Responsibilities:
- Expected to be an SME, collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5+ years of overall IT experience, which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design, build and deployment of Python-based applications. ...
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR, API Gateway, RDS instance, and Lambda.
- Understand complex data sets and ETL processes, and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
- Experience with cloud-based data platforms and distributed computing frameworks.
- Ability to optimize data workflows for performance and scalability.
- Familiarity with database management systems and data warehousing concepts.
- Competence in scripting and automation to streamline data processing tasks.
experience
8show more