Associate Data Engineer

Position Responsibilities: 

• Support: With Solution Architects / Data Architecture support solution design for the service layer for APIs and Data Services for Analytics
• Lead:  API Development, Data migration using ETL/ADF or other means.
• Lead: Develop business logic as needed for developing and consuming API services
• Lead: Stakeholder requirements / design for taking the project from scratch to completion.
• Lead: SQL query creation for complex business functionalities
• Support / Lead: With Domain Data Architect for data pipeline methods / service in support of additional data types from the Data Strategy

Basic Qualifications (Required Skills/Experience): 

• Experience with data engineering using Apache Spark [2.4+, 3.1+] & Scala [2.11+] (or Java 11+), Spark SQL, Tuning Spark ETL jobs etc.
• Bachelors’ Degree or higher is required as a basic qualification
• Should be proficient with Databases like SQLServer: MS-SQL/T-SQL; Teradata[16.10+];
• General awareness of other Databases like  ANSI SQL with advanced SQL skills
• Experience working with Data Warehouses AND/OR Data Lakes
• Working with *nix VMs OR working with containers [docker/podman/lxc etc.], container orchestration platforms [CF/Kubernetes/GKE/AKS]
• Shell scripting [sh/bash], knowlege of *nix VM operation and management [Fedora 28+/RHEL 8+/OL 8+]
• Source control systems to manage code and repositories [Git/Mercurial]
• Experience using continuous integration/deployment tools to continually tweak and deploy code [GitLab-CI other distributed, stateless CI systems]

Preferred Qualifications (Desired Skills/Experience): 

• Expertise in using advanced Excel features and functionality, VBA programming with macros
• Data modelling – ER mapping, should be able to represent business problems using optimized data structures & schemas
• Data intensive, cloud native application development using the Java/Spring [Boot/Cloud/Data/Security] stack
• Experience with task & operation sequencing [manual/using a DAG runner like Airflow]

Typical Education & Experience: 

• Typically, 4-8 years related work experience

Relocation:

• This position offers relocation within India based on candidate’s eligibility

Export Control Requirements: Not an export control position