OpenSearch Engineer (#472)

We are looking for a skilled OpenSearch Engineer to design, deploy, and manage our high-performance, distributed search and analytics platform. This role is critical for ensuring our application search, log analytics, and observability pipelines are secure, scalable, and highly available to support our growing user base and data volume. You will be a key contributor to the architecture and operational excellence of our core data systems.

Key Responsibilities

Design & Development:

• Design, implement, and maintain large-scale OpenSearch/Elasticsearch clusters for various use cases (e.g., application search, logging, observability).

• Develop, optimize, and tune complex search queries, indexing strategies, and data models to ensure fast and accurate results.

• Develop custom OpenSearch plugins or integrations as needed to extend platform functionality.

• Collaborate with application development teams to integrate OpenSearch into new and existing applications.

Platform & Operations:

• Manage cluster health, capacity planning, and resource allocation to ensure high availability, performance, and scalability.

• Implement and manage data pipelines using tools like Logstash, Fluentd, or custom solutions for efficient data ingestion.

• Define and enforce security policies, including fine-grained access control, encryption, and network segmentation.

• Automate deployment, configuration, and monitoring using Infrastructure-as-Code (IaC) tools like Terraform or CloudFormation.

• Participate in on-call rotations to diagnose and resolve production incidents related to the search platform.

Optimization & Strategy:

• Proactively identify and resolve performance bottlenecks, conducting regular cluster performance testing and optimization, and enhance the search functionality using AI.

• Stay current with the OpenSearch/Elasticsearch ecosystem, new features, and best practices to drive platform innovation.