Data Engineer Position

ClifyX is seeking a Data Engineer responsible for building and maintaining data pipelines and products to process large volumes of structured and unstructured data. The role involves analyzing data needs, migrating data into an enterprise data lake, and developing real-time and batch-based ETL pipelines.

Responsibilities

• Design and develop ETL workflows to migrate data from varied data sources including SQL Server, Netezza, Kafka etc. in batch and real-time
• Develop checks and balances to ensure integrity of the ingested data
• Design and Develop Spark jobs as per requirements for data processing needs
• Work with Analysts and Data Scientists to assist them in building scalable data products
• Designs systems, alerts, and dashboards to monitor data products in production

Company

• Digitalization, cloud transformation, and Salesforce/ServiceNow implementation services
• South Plainfield, New Jersey, USA
• https://clifyx.com/
• Founded in 1998
• 501-1000 employees

Qualifications

• Expertise working with large scale distributed systems (Hadoop, Spark)
• Strong understanding of the big data cluster, and its architecture
• Experience building and optimizing big data ETL pipelines
• Advanced programming skills with Python, Java, Scala
• Good knowledge of spark internals and performance tuning of spark jobs
• Strong SQL skills and is comfortable operating with relational data models and structure
• Capable of accessing data via a variety of API/RESTful services
• Experience with messaging systems like Kafka
• Experience with No SQL databases. Neo4j, mongo, etc
• Expertise with Continuous Integration/Continuous Delivery workflows and supporting applications
• Exposure to cloud environments and architectures. (preferably Azure)
• Ability to work collaboratively with other teams
• Experience with containerization using tools such as Docker
• Strong knowledge of Linux and Bash. Can interact with the OS at the command line and create shell scripts to automate workflows
• Advanced understanding of software development and collaboration, including experience with tools such as Git
• Excellent written and verbal communication skills, comfortable presenting in front of non-technical audiences
• ClifyX provides innovative business solutions which satisfy requirements for mission-critical reliability, scalability, interoperations.
• South Plainfield, New Jersey, USA
• https://clifyx.com
• Late Stage
• Leadership Team
• Anil Thakur
• CEO
• Glenn Gilmore
• Senior Managing Partner and Executive Coach
• Company data provided by
• AI Tools
• Customize Your Resume
• Maximize your interview chances
• Build Cover Letter
• Make your application stand out
• Analyze How Well You Fit
• Understand your strength & weakness

Required skills

• API/RESTful services, Azure, Bash, Continuous Integration/Continuous Delivery, Docker, ETL pipelines, Git, Hadoop, Java, Kafka, Linux, MongoDB, Neo4j, NoSQL databases, Python, SQL, Scala, Spark

Experience

• Mid level