Detecting OpenAI & AI Provider Issues in Under 10 Minutes with a New Observability System (opens in a new tab)
How the team monitors AI provider health and surfaces incidents through a shared observability system.
Senior AI Infrastructure Engineer
Shubham Nimbalkar
I build infrastructure that makes generative AI systems observable, reliable, and secure. At Salesforce, I work on the infrastructure behind Agentforce, from cross-provider metrics and inference routing to Kubernetes operations and cloud authentication.
Previously at Tesla, I built manufacturing data infrastructure connecting streaming pipelines, analytics, and Spark on Kubernetes.
CURRENT FOCUS · SALESFORCE
Making AI infrastructure observable
Pulse
An internal platform bringing Agentforce metrics and dashboards together across AI providers, services, and finance.
From data to insight
ETL with Trino, Airflow, and Iceberg connects usage and operational data. Pulse hosts dashboards, cross-database queries, complex operations, and scheduled jobs.
Visibility across the stack
OpenTelemetry metrics surface capacity, throughput, errors, and latency. Customer dashboards show call volume, agent and model mix, and success rates.
Trino · Airflow · Iceberg · OpenTelemetry · GCP · Vertex AI
Experience
Senior AI Infra Engineer (SMTS) @ Salesforce
September 2024 - Present
- Built Pulse, an internal platform for real-time Agentforce AI metrics, with Trino, Airflow, and Iceberg ETL spanning providers, services, and finance, supporting cross-database queries, dashboards, complex operations, and scheduled jobs.
- Designed GCP OpenTelemetry metrics infrastructure to manage Vertex AI capacity, latency, token and invocation throughput, errors, Apigee and ALB metrics.
- Operate multi-region EKS infrastructure, including cluster upgrades, node groups, IAM roles for service accounts (IRSA), OIDC integration, and on-call incident response. Mentored and handed off this responsibility to the team in India.
- Drove Amazon Bedrock inference multi-regional routing across models and providers, incorporating regional preferences and resilience to support low-latency inference.
- Migrated Terraform pipeline authentication to Workload Identity Federation across GCP.
- Set up OpenAI monitoring alerts and initiated a proof of concept to onboard to mTLS.
- Co-led cross-team response to a Dubai outage, coordinating Agentforce S3 and relational database snapshots and replication to protect customer data.
Published engineering stories
Salesforce Engineering coverage of our team's work.
Building AWS Bedrock Model Availability: Slashing AI Routing Discovery From Days to Minutes (opens in a new tab)
How automated model discovery and regional fallback routing support reliable Agentforce inference.
Software Engineer @ Tesla, Inc.
February 2023 - August 2024
- Built Spark on Kubernetes on-prem infrastructure to horizontally scale Cell Manufacturing data jobs by 10x
- Maintained real-time analytics for 3 factories on the in-house dashboard that provides yields at different granularities
- Used Kafka, Spark, Hudi and Clickhouse to optimize latency and compute resources
- Designed data pipeline for genealogy mapping from a cell to raw materials across different shops in manufacturing
- Built long running streaming jobs with E2E tests and easy deployment that can consume millions of messages per hour
Software Engineering Intern @ Apple, Inc.
June 2022 - September 2022
- Early adoption of internal multi-device frameworks to automate and migrate jobs in current test pipeline
- ShieldSense - A dashboard for teams to visualise xcode project coverage metrics to improve software quality
Software Engineer @ Rakuten
September 2019 - July 2021
- Developed an end-to-end campaign intelligence platform that improved ad targeting and increased campaign sales by 5%.
- Designed Spark, Elasticsearch, and SQL pipelines and cloud infrastructure processing 20 GB of data daily.
- Built a marketing frontend using React, supported by Kubernetes and Docker, and an LSTM sales predictor with a two-week forecasting horizon.
Research Trainee @ Rakuten Institute of Technology
January 2019 - August 2019
- Developed face-liveness detection for Rakuten Pay and collected infrared depth data from 300 participants for model development.
- Built image-tagging research applications and video-processing pipelines for metadata extraction and scene-classified movie posters, targeting a 90% reduction in human annotation effort.
Intern @ Rakuten
2018
- Built a TF-IDF item-bundle recommendation engine and a server health monitoring and alerting system using dependency graphs.
About Me
I'm a California-based engineer focused on AI infrastructure and distributed data systems. My work spans observability, inference reliability, cloud identity, and production operations, with experience at Salesforce, Tesla, Apple, and Rakuten.
I hold a Master's degree in Computer Science from UC Irvine and a Bachelor of Engineering degree from Pune Institute of Computer Technology.
My preferred technologies include:
Scala Python Swift C++
AWS/GCP/Linux Kubernetes Docker Git
Trino, Iceberg, ClickHouse, Hudi, Postgres, Airflow Elastic, Splunk Superset, Kibana OpenTelemetry, Vertex AI, Bedrock, Terraform
Selected Projects & Research
In-house Search Engine | 2022
An in-house search engine project exploring natural language processing and information retrieval. Watch the demo or explore the source.
Tag extraction by human saliency | 2019
To describe an image with tags ranked by visual importance
Plant Disease Detection | 2018
Published paper at IEEE for augmentation with GANs
