Devops - Senior Engineer

Job Description

Kubernetes & Helm Management
•    Design, implement, and maintain Kubernetes clusters across multiple environments (development, staging, production)
•    Develop, customize, and deploy Helm charts for Dremio and other products and services
•    Troubleshoot and optimize Kubernetes deployments, including resource allocation, scaling, and performance tuning
•    Implement and maintain security best practices for Kubernetes workloads and cluster configurations
•    Monitor cluster health, resource utilization, and application performance
Dremio Platform Management
•    Deploy, configure, and manage Dremio clusters on Kubernetes using Helm charts
•    Optimize Dremio cluster performance through proper resource allocation, scaling strategies, and cluster tuning
•    Monitor Dremio cluster health, query performance, and resource utilization
•    Troubleshoot Dremio-specific issues including connectivity, performance bottlenecks, and data source integrations
CI/CD Pipeline Development
•    Design and implement comprehensive CI/CD pipelines using GitHub Actions for automated testing, building, and deployment
•    Integrate Helios workflows into deployment automation processes, particularly for Dremio deployments
•    Establish automated testing frameworks and quality gates within deployment pipelines
•    Implement infrastructure as code practices using tools like Terraform or similar
•    Create and maintain deployment strategies including blue-green, canary, and rolling deployments for Dremio clusters
Automation & Process Improvement
•    Develop automation scripts and tools to streamline Dremio deployment processes and reduce manual intervention
•    Implement monitoring and alerting solutions for Dremio clusters, deployment pipelines, and application health
•    Create comprehensive documentation for Dremio deployment procedures, upgrade processes, and troubleshooting guides
•    Collaborate with data engineering teams to establish best practices for Dremio configuration and optimization
Required Qualifications
Technical Skills
•    3+ years of hands-on experience with Kubernetes in production environments
•    Strong proficiency in creating, customizing, and managing Helm charts
•    Extensive experience with GitHub Actions for CI/CD pipeline development
•    Working knowledge of Helios and its integration with deployment workflows
•    Experience with Dremio or similar data lake engines (Trino, Presto) is highly preferred
•    Strong scripting skills in Bash, Python
•    Experience with monitoring tools (Prometheus, Grafana)
Soft Skills
•    Strong problem-solving and troubleshooting abilities, particularly with distributed systems
•    Excellent communication skills and ability to work collaboratively with data engineering and analytics teams
•    Experience in agile development environments
•    Ability to work independently and manage multiple priorities effectively
•    Experience with capacity planning and performance optimization

Mandatory Competencies