Job Description:
OverviewCome join the Data Solutions EngineeringTeam as a Staff Software Developer. We are leveraging big data technologies to gain new insights into our customer experiences; using data frameworks, create ingestion pipelines and tools. Some of the technologies we are leveraging include Hadoop, Vertica, Hive and AWS Big Data Systems (EMR, Redshift, S3).What you'll bringDeep software development experience with work experience in developing DB schemas, creating ETLs, and familiar with MPP/Hadoop systems.Working Knowledge of XML, JavaScript, JSON, YML and Linux Advanced experience with scripting language - Python or Shell is a must haveBig Data: Building and maintaining highly scalable, robust & fault-tolerant complex data processing pipelinesStrong knowledge of software development methodologies and practices Experience working in Agile development teams; working knowledge of Agile (Scrum) development methodologiesExperience with Amazon web services: EC2, S3, and EMR (Elastic Map Reduce) , Redshift or equivalent cloud computing approachesStrong expertise in writing analytical SQLs, Data Marts, Data Warehousing and analytic architectureExperience working with large data volumesHands on experience with Hadoop stack of technologies - mainly Hive, Hive on sparkExperience creating and consuming JSON/REST web services and communicating with systems.Skilled in developing Software for Java (Spring & Springboot), Scala for spark streaming & spark applications, or other JVM based languages.How you will lead70-85% hands-on development in all phases of the software life cycle.Designing/developing ETL jobs across multiple big data platforms and tools including S3, EMR, Hive, VerticaDesigning end to end data pipeline given business and ops requirements (ingestion, processing and storage). Conduct design and code reviews Defect remediationEstimates and sequence of individual activities as inputs to project plansAnalyzes and synthesizes a variety of inputs to create software and services.Identify dependencies as inputs to project plansCollaborates effectively with peer engineers and architects to solve complex problems spanning their respective areas to deliver end-to-end quality in our technology and customer experience. Influences and communicates effectively with non-technical audiences including senior product and business management.Designing/developing ETL jobs across multiple big data platforms and tools including S3, EMR, Hive, VerticaDesigning end to end data pipeline given business and ops requirements (ingestion, processing and storage).
