Skip to content
Nikhil Gautam
  • Software Engineer
  • Backend Engineering
  • Distributed Systems

Building reliable software systems at scale.

I’m a software engineer focused on backend architecture, distributed systems, event-driven platforms, and data-intensive applications.

I enjoy solving problems involving scalability, consistency, performance, reliability, and the engineering trade-offs behind complex systems.

Work

Selected engineering work

Production systems I’ve worked on, described without confidential detail. Full case studies are being written.

  1. Search indexing driven by change data capture: database changes captured with Debezium, delivered through Kafka, and processed by consumers that keep OpenSearch indices, including vector indices, in step with the source of truth.

    Themes
    • Change data capture
    • Event-driven architecture
    • Idempotent consumers
    • Index consistency
    • Vector search
    Stack
    • Debezium
    • Kafka
    • OpenSearch
    Professional workCase study in preparation
  2. High-volume batch processing

    Large-volume data processing on AWS built with Spring Batch: orchestrating jobs, working with data in S3 and Athena, and designing retry and failure handling so that partial failures can be recovered without reprocessing everything.

    Themes
    • Data pipelines
    • Job orchestration
    • Retry and failure handling
    • Fault recovery
    Stack
    • Java
    • Spring Batch
    • AWS Batch
    • S3
    • Athena
    Professional workCase study in preparation
  3. Microservices and API performance

    Decomposing a monolith into services and improving backend performance: drawing service boundaries, moving work onto asynchronous paths, and reasoning about throughput, latency and the architectural trade-offs between them.

    Themes
    • Monolith decomposition
    • Service boundaries
    • Asynchronous processing
    • Throughput and latency
    Professional workCase study in preparation

All work

Principles

How I approach systems

The positions I start from when designing or reviewing a system.

  1. Correctness before optimization

    Define what “correct” means first: invariants, ordering guarantees, what happens on a retry. An optimization that weakens those properties without saying so isn’t an improvement. It’s a deferred incident.

  2. Reliability is an architectural concern

    Retries, timeouts, back-pressure and recovery paths shape a system’s structure. They belong in the design review, not in the post-incident action items.

  3. Distributed systems trade-offs, made explicit

    Every boundary between processes forces a choice about consistency, availability and latency. I prefer to name that choice, write down why, and decide what the system does when the assumption breaks.

  4. Observability is part of the design

    If a system can’t explain its own behavior in production, it isn’t finished. Useful signals come from asking which questions an on-call engineer will need answered, before the system ships.

  5. Performance grounded in measurement

    Profile, then form a hypothesis, then change one thing. Throughput and latency claims without a baseline and a method are opinions.

Writing

Technical writing

Notes I’m preparing on the problems behind the work above. None are published yet.

  • In preparation

    Designing reliable CDC-driven indexing pipelines

    Keeping a search index faithful to its source database when the only link between them is a change stream.

  • In preparation

    Handling duplicate and out-of-order events in Kafka

    What delivery guarantees actually promise, and what the consumer still has to do itself.

  • In preparation

    Designing idempotent event consumers

    Patterns for making reprocessing safe, and the cases where idempotency is harder than it looks.

  • In preparation

    Hybrid search: lexical and semantic retrieval

    Combining keyword relevance with vector similarity, and the ranking problems that appear where they meet.

All writing

About

Background

I’m a backend-focused software engineer with experience across full-stack and cloud engineering. My work has centred on search, data processing and service architecture: change-data-capture pipelines feeding search indices, large-volume batch processing on AWS, and moving a monolith towards services.

I’m most interested in the parts of a system that are easy to get subtly wrong, such as consistency between data stores, recovery after partial failure, and performance under real load.