Search Jobs
Find your next opportunity here.
Data Engineer
Bethlehem, PA 18017 US
Job Description
Summary:
This role will be responsible for end-to-end development and operationalization of data science and analytics pipelines. The ideal candidate MUST have a good amount of hands-on experience working with Databricks to design and build Data pipelines.
Responsibilities:
• Architect, build, and maintain scalable data & analytics pipelines for machine learning models, reports, dashboard and other analytics solutions using Databricks platform.
• Re-design and migrate existing EMR data pipelines into Databricks.
• Design and implement low-latency, high-availability, and performant data applications and recommend & implement innovative engineering solutions.
• Participate in developing MLOps and DevOps pipelines to automate analytics and machine learning solutions.
• Work on development of new products and enhancements to existing systems iteratively by building quick POCs and converting ideas into real products.
• Apply and provide guidance on software engineering techniques like design patterns, code refactoring, framework design, code reusability, code versioning, performance optimization, and continuous build and Integration (CI/CD) to make the data analytics team robust and efficient.
• Identify opportunities for further enhancements and refinements to standards and processes.
Qualifications:
• 5+ years of data engineering experience.
• 3+ years of hands-on experience in programming languages Python, Spark (PySpark) & SQL.
• 1+ years of experience working within Databricks platform.
• 3+ years of experience, building end to end production data pipelines.
• 3+ years of experience working within cloud ecosystem (AWS, Azure, GCP or similar)
• 1+ year experience building microservices architecture using containerization technologies like docker or Kubernetes.
• Knowledge of data warehousing concepts.
• Proficient in understanding and incorporating software engineering principles in design & development process.
• Efficiency in handling data - tracking data lineage & ensuring data quality, and improving reliability of data.
• Excellent verbal and written communication; Proven interpersonal skills and ability to convey key insights from complex analysis in summarized terms; Ability to effectively communicate with technical teams.
Location: Hybrid – Bethlehem, PA
Meet Your Recruiter
Wendy Crowell
Wendy is a Client Relationship Manager for Panther’s Strategic Accounts division. Wendy focuses on the delivery of top talent to Panther clients to ensure their recruiting needs are met. She is passionate about helping candidates take the next steps in their careers while partnering with new and existing clients to attract candidates.
Fun fact: She loves to flip furniture in her spare time.
