Databricks Engineer

Location: Seattle, WA Duration: 6+ months Rate: $90/hr

Microsoft Azure | Databricks | PySpark

Skills

68 years of experience in building scalable data pipelines and data solutions. Hands-on experience with: o Azure o Databricks o PySpark Strong experience designing, developing, and optimizing big data processing systems on cloud platforms. Expertise in Azure data services and advanced Spark-based data engineering. Experience with ETL workflows and distributed data processing. Knowledge of CI/CD practices for data pipelines. Strong understanding of data quality, monitoring, and governance practices. Ability to work with structured and unstructured data sources.

Responsibilities:

Design, develop, and maintain scalable data pipelines and ETL workflows using Azure Databricks and PySpark. Build and optimize data models, data transformations, and distributed data processing solutions. Work closely with data architects, analysts, and business stakeholders to understand requirements and deliver robust data solutions. Develop and maintain data ingestion frameworks for structured and unstructured data sources. Implement data quality checks, validation rules, and automated monitoring for pipeline reliability. Optimize Spark jobs for performance and cost efficiency in Azure cloud environments. Develop reusable components, frameworks, and best practices for data engineering processes. Collaborate with DevOps teams for CI/CD deployment of data pipelines. Ensure security, compliance, and governance across all data processes and environments. Maintain detailed technical documentation and support production systems.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.