Big Data Engineer

Company: Universal Business Solutions
Location: Atlanta
Posted on: March 19, 2023

Job Description:

Job Description

Big Data Engineer
Atlanta, GA (Midtown Area)
Contract - 1 Year Plus

Work with State-of-the-art Big Data Platforms (Cloudera). In this position the candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions, and perform data ingestion development and support duties.
Skills and Experience Required:
--- 6+ years of overall IT experience
--- 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
o Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
o Deep knowledge of troubleshooting and tuning Spark applications
--- 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
--- 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
--- 3+ years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets
--- 2+ years of experience with Kudu and Impala
--- 2+ years of experience with Scala (and/or Python) and PySpark/Scala-Spark
--- 2+ years of experience with NoSQL databases, including HBASE and/o
2+ years of experience with Scala (and/or Python) and PySpark/Scala-Spark
--- 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
--- Knowledge of Unix/Linux platform and shell scripting is a must
--- Strong analytical and problem-solving skills
Preferred (Not Required):
--- Experience with Cloudera/Hortonworks HDP and HDF platforms
--- Experience with NIFI, Schema Registry, NIFI Registry
--- Strong SQL skills with ability to write intermediate complexity queries
--- Strong understanding of Relational & Dimensional modeling
--- Experience with GIT code versioning software
--- Experience with REST API and Web Services
--- Good business analyst and requirements gathering/writing skills
Education:
Bachelor's Degree required. Preferably in Information Systems, Computer Science, Computer Information Systems, or related field

Keywords: Universal Business Solutions, Atlanta , Big Data Engineer, Engineering , Atlanta, Georgia