Location: Remote
Contract Type: B2B
Experience: 6+ years total, 3+ years with Elasticsearch in production
English Level: C1 (Advanced)
Compensation: Gross salary (to be discussed)
Holidays: 10 public holidays per year (vacation and sick leave unpaid)

Role Overview

We are looking for a highly experienced Senior Data Engineer with deep Elasticsearch expertise to take full ownership of a high-throughput, mission-critical indexing platform.

This role is focused on performance optimization, scalability, reliability, and observability of Elasticsearch indexing systems supporting multiple applications.

This is a hands-on role requiring strong technical leadership, operational maturity, and independent execution.

Key Responsibilities

Elasticsearch Indexing Ownership

• Own the end-to-end Elasticsearch indexing function and associated application layer.

• Design, implement, and optimize high-throughput bulk indexing workflows.

• Improve indexing strategies to support increasing data volumes.

• Optimize index structures, mappings, shard allocation, and lifecycle strategies.

• Lead zero-downtime reindexing processes.

Performance, Scaling & Optimization

• Diagnose and resolve indexing bottlenecks and latency issues.

• Optimize:

  • Bulk ingestion workflows

  • Shard design and distribution

  • Refresh intervals

  • Replication strategies

• Design parallel ingestion pipelines.

• Improve scalability and system resilience under high load.

Reliability, Failure Handling & Observability

• Implement robust retry mechanisms and dead-letter queues.

• Design comprehensive monitoring and alerting strategies.

• Improve system observability using metrics, structured logging, and tracing.

• Reduce operational overhead and manual interventions.

Data Ingestion & Pipeline Architecture

• Design ingestion pipelines from multiple structured and unstructured sources.

• Work with CDC (Change Data Capture) and event-driven architectures.

• Ensure schema consistency, normalization, and mapping correctness.

• Support high-volume batch processing systems.

Application-Level Optimization

• Optimize concurrency, batching strategies, and connection pooling.

• Eliminate sequential processing bottlenecks.

• Improve application-level performance interacting with Elasticsearch.

Production & Deployment

• Implement safe deployment strategies.

• Ensure zero-downtime index migrations and reindexing.

• Improve CI/CD processes related to indexing infrastructure.

Required Qualifications

• 6+ years of experience in data engineering or backend engineering.

• 3+ years of hands-on Elasticsearch experience in production.

• Proven ownership of indexing pipelines and search infrastructure.

• Strong Python and SQL.

• Experience solving high-scale performance and scalability challenges.

• Strong debugging, monitoring, and production troubleshooting skills.

• Advanced English (C1) for cross-functional collaboration.

Strongly Preferred

• Experience with CDC systems and event-driven architectures.

• High-volume batch processing systems.

• Queue-based architectures (e.g., Kafka, RabbitMQ).

• Index lifecycle management and reindexing strategies.

• Zero-downtime deployment experience.

Nice to Have

• Experience with vector search or hybrid retrieval systems.

• Cloud-managed Elasticsearch (AWS, GCP, Azure).

• Real-time search workloads.