Title: Senior Data Engineer
Location: O’Fallon, MO (Onsite)
Type: Contract
Need Independent Visa Candidate USC, GC, EAD
Only qualified Senior Data Engineer candidates located near the O’Fallon, MO area to be considered due to the position requiring an onsite presence***
Qualifications:
• Excellent problem-solving skills with good understanding of data engineering concepts
• Proficient in Apache Spark with Python and related technologies
• Strong knowledge of SQL and performance tuning
• Experience in Big Data technologies like Hadoop and Oracle Exadata
• Solid knowledge of Linux environments and proficiency with bash scripting
• Effective verbal and written communication skills
Nice to have knowledge or prior experience with any of the following:
• Apache Kafka, Apache Spark with Scala
• Orchestration with Apache Nifi, Apache Airflow
• Java development and microservices architecture
• Build tools like Jenkins
• Log analysis and monitoring using Splunk
• Databricks, AWS
• Working with large data sets with terabytes of data
Seeking an enthusiastic and adaptable Data Engineer who will be responsible for developing and managing Data Pipelines and assisting various data-driven requests across the company. The position involves a wide variety of technical skills, from developing high throughput Spark jobs to tuning performance of Big Data solutions. The ideal candidate will have experience working with large-scale data and building automated data ETL processes. Data warehousing experience would be ideal.
Key responsibilities include:
• Build and maintain big data technologies, environments, and applications, seeking opportunities for improvements and efficiencies
• Performing ETL (Extract, Transform, Load) processes based on business requirements using Apache Spark and data ingestion from Apache Kafka
• Work with various data platforms including Apache Hadoop, Apache Ozone, AWS S3, Delta Lake, Apache Iceberg
• Utilizing orchestration tools like Apache NiFi for managing and scheduling data flows efficiently
• Writes performant SQL statement to analyze data with Hive/Impala/Oracle
• Full application development lifecycle (SDLC) from design to deployment
• Working with multiple stakeholders across teams to fulfill ad-hoc investigations, including large-scale data extraction, transformation, and analyses