Big Data Engineer in Atlanta, GA at HUNTER Technical Resources

Date Posted: 2/28/2021

Job Snapshot

  • Employee Type:
    Contractor
  • Location:
    Atlanta, GA
  • Job Type:
  • Experience:
    Not Specified
  • Date Posted:
    2/28/2021
  • Job ID:
    5673530

Job Description


Position: Big Data Engineer

Location: Atlanta, GA

Duration: 2 + year contract

 

Skills and Experience:

              Required:
  • 6+ years of overall IT experience
  • 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
    • Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
    • Deep knowledge of troubleshooting and tuning Spark applications
  • 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
  • 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
  • 3+ years of experience building, testing, and optimizing ‘ Big Data’ data ingestion pipelines, architectures and data sets
  • 2+ years of experience with Python (and/or Scala) and PySpark
  • 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
  • Knowledge of Unix/Linux platform and shell scripting is a must
  • Strong analytical and problem solving skills

 

              Preferred (Not Required):  
  • Experience with Cloudera/Hortonworks HDP and HDF platforms
  • Experience with NIFI, Schema Registry, NIFI Registry
  • Strong SQL skills with ability to write intermediate complexity queries
  • Strong understanding of Relational & Dimensional modeling
  • Experience with GIT code versioning software
  • Experience with REST API and Web Services
  • Good business analyst and requirements gathering/writing skills