Search Jobs

Big Data Engineer/manager - Hadoop/java/python

6 - 11 yrs
Apply without registration
Not Disclosed by Recruiter Openings: 1 Posted 30+ days ago Job Applicants: 87 Job Views: 345

Job Description

Send me Jobs like this
    Position title : Big Data Engineer Manager

    Job Purpose :

    - Manager in the IT Advisory Services team to lead the delivery of projects for our customers across the globe.

    Your Technical Responsibilities :

    - Hands-on Engineering of Data Processing applications

    - Experience working with NoSQL databases and search engines such as MongoDB, Cassandra, Elasticsearch

    - Experience with Big Data technologies, assisting clients in building software solutions that are distributed, highly scalable and across multiple data centers

    - Hands on experience in architecting Big Data applications using Hadoop technologies such as Spark, MapReduce, YARN, HDFS, Hive, Impala, Pig, Sqoop, Oozie, HBase, Elasticsearch, Cassandra.

    - Database setup/ configuration/ troubleshooting

    - Configuring/ Optimizing database clusters and load balancing setups

    - Experience using Neo4J or understanding of graph databases

    - Strong Experience with event stream processing technologies such as Spark streaming, Storm, Akka, Kafka

    - Experience with at least one programming language (Java, Scala, Python)

    - Extensive experience with at least one major Hadoop platform (Cloudera, Hortonworks, MapR)

    - Experience working with Business Intelligence teams, Data Integration developers, Data Scientists, Analysts and DBA's to deliver well-architected and scalable Big Data & Analytics eco-system

    - Proven track record of architecting Distributed Solutions dealing with real high volume of data(petabytes)

    - Experience in Artificial Intelligence and Machine Learning technologies (H20, TensorFlow, IBM Watson etc)

    - Strong troubleshooting and performance tuning skills.

    - Experience with SQL and scripting languages (such as Python, R)

    - Deep understanding of cloud computing infrastructure and platforms.

    - Good understanding of Big data design patterns

    - Ability to analyze business requirement user stories and model it to domain based services

    - Experience working under agile delivery methodology

    Required Candidate profile

    Requirements (Qualifications) :

    We are looking for the candidates with the following :

    - BE/BTech in computer science Or MCA with a sound industry experience (10-14 years) in various roles.

    - A minimum of 5 years experience in hands-on development

    - 5+ years experience deploying data engineering solutions in a production setting

    - Experience in Java, Python, R or Scala. (Production level coding)

    - Capability to architect highly scalable distributed data pipelines using open source tools and big data technologies such as Hadoop, HBase, Spark, Storm, ELK, etc.

    - Familiar with at least one Machine Learning library/framework (scikit-learn, Mahoot, MLib, H2o, etc.). Proficiency in a deep learning framework is plus (Tensorflow, Keras, Theano, MXNet, etc.)

    - Experience of designing scalable solutions with proficiency in use of data structures and algorithms

    - Experience in cloud-based environment with PaaS & IaaS

    - Work iteratively in a team with continuous collaboration

    Mandatory Skills :

    - No SQL, MongoDB, Cassandra, Neo4J

    - HDFS, MapReduce, YARN, Pig, Hive, HBase, Zookeeper, Oozie

    - Oracle, MySQL, SQL, Stored Procedures

    - Python, Scala, Java

    - Strong ETL Experience

    - Spark, Spark Streaming

    - Storm, Kafka, Flume

    - Machine Learning Frameworks

    - Data Modeling (Basic Level) experience

    - Data Standardization/Canonical Data Format definition

    - Data Integration Patterns

    - Data Quality/Lineage Solutions

    - Performance Benchmarking and Tuning

    - Data Conversion and Migration between different databases

    Preferred :

    - Elastic Search

    - Tableau, QlickView

    - RabbitMQ

    - Data Standardization

    - Data Integration Patterns

    - Data Quality/Lineage Solutions

Salary: Not Disclosed by Recruiter

Industry: IT-Software, Software Services

Functional Area: IT Software - Application Programming , Maintenance

Role Category:Programming & Design

Role:Database Architect/Designer

Employment Type: Full Time, Permanent



UG: B.Tech/B.E. - Computers

PG:MCA - Computers

Company Profile:


Client of Catalyst.
View Contact Details
Apply without registration

IEIL has taken all reasonable steps to ensure that information on this site is authentic. Applicants are advised to research bonafides of advertisers independently. IEIL shall not have any responsibility in this regard. We also recommend that you visit Security Guidelines and Terms and Conditions for more comprehensive information on this aspect.

Job Posted by

Sudeep V Talent Enabler Catalyst Solutions Bengaluru / Bangalore


Get Better Jobs & Higher Salaries. Suggested courses

I agree to the Terms and Conditions