Scheduled Cron Job Skipped Silently With No Error Logged
A scheduled job never fired and nothing showed up in logs. Fix it by going UTC-only, checking startingDeadlineSeconds, adding a heartbeat, and alerting on missed runs.
#Backend
Articles tagged with #Backend
Postgres Migration Stuck on ALTER TABLE in Production
An ALTER TABLE migration hangs forever on prod. Find the blocker with pg_blocking_pids, terminate it, and re-run with a lock_timeout so it fails fast instead of stalling.
Docker Container Restarts With Exit Code 137 (OOM Killed): Fix It
Container exits 137 with no stack trace. That's SIGKILL from the OOM killer hitting your --memory cap. Confirm it, find the leak with a heap dump, set a sane limit, and add a guardrail.
Fix gRPC DEADLINE_EXCEEDED Errors Under Load
gRPC clients throw DEADLINE_EXCEEDED when traffic rises. Propagate deadlines, set per-RPC timeouts by SLO, add the built-in retry policy with throttling, and trip a circuit breaker so one slow backend stops cascading.
JWT 'jwt expired' on Fresh Tokens: Fix Clock Skew
JWT verify fails with 'jwt expired' or 'token not yet valid' on tokens issued seconds ago. Fix server clock drift with NTP and add 5s verifier leeway.
Kafka Consumer Lag Keeps Growing After Scaling Consumers
You added more consumer pods and lag is still climbing. The bottleneck is almost never too few consumers. It is partition count, a poison message, a slow downstream write, or rebalance thrash.
MongoDB Aggregation With $lookup + $group Runs for 30 Seconds
A MongoDB pipeline with $lookup + $group crawls in production. Read explain('executionStats'), index the join field, push $match first, and read the new EQ_LOOKUP strategy.
RabbitMQ Consumers Connected But Not Processing — Queue Depth Growing
RabbitMQ shows healthy consumers but the queue keeps growing. Fix prefetch, unacked messages, consumer_timeout, and dead-letter routing on RabbitMQ 4.x.
Postgres Autovacuum Stalled by a Long-Running Transaction
Dead tuples climb, the table bloats, and VACUUM logs say dead but not yet removable. One forgotten transaction is pinning the xmin horizon so vacuum reclaims nothing. Find and kill it.
S3 Presigned URL Returns 403 Mid-Upload on Large Files
AWS S3 presigned URL works for small files but 403s mid-upload on large ones. Fix the TTL, the credential lifetime, and switch big files to multipart uploads.
GraphQL Rate Limit Cascade: Stop One Slow Resolver Taking Down the Gateway
One slow resolver hits an upstream 429 and the whole GraphQL gateway stalls. Fix it with per-query complexity costs, DataLoader batching, fail-fast circuit breakers, and per-upstream connection pools.
Message Queue Dead-Letter Queue Building Up
DLQ growing in SQS, RabbitMQ, or Kafka without bound. Sample and classify the failures, fix the poison-message root cause, cap the retry budget, then redrive safe messages.
Postgres Connection Pool Exhausted Under Load
Postgres throws 'remaining connection slots reserved' under traffic. Fix it by sizing the pool, putting PgBouncer in transaction mode in front, and killing idle-in-transaction connections.
Redis Cluster Failover Stuck: No Replica Gets Promoted
A master node died but no replica took over and the cluster sits in fail state. Fix it by checking master quorum, the FAIL flag, replica eligibility, and forcing a manual takeover.
Auth Redirect Returns to the Wrong URL (redirect_uri_mismatch)
Sign-in redirects to localhost or the wrong domain, or the provider errors with redirect_uri_mismatch. Add the exact callback URI to the allowed list and drive it from env.
CORS Error Calling Your Own API: The Fix
Browser blocks your fetch with a CORS error while curl works. Diagnose the real cause and fix it server-side, with verification.
Edge Function Timeout: Vercel / Cloudflare / Netlify Fix
FUNCTION_INVOCATION_TIMEOUT or 504 on edge? Move heavy work off edge, stream responses, and timeout every upstream fetch. Verified limits, June 2026.
Fix Firebase Deploy "Permission Denied" (403)
firebase deploy fails with 403 or "caller does not have permission"? Diagnose the identity chain — account, project, IAM roles, CI auth — and fix it fast.
Firebase Function Not Found: Fix the Region/Name/Deploy Mismatch (2026)
A callable returns "not found" even though the console shows it. Diagnose region, name, and deploy state in minutes.
Firebase Hosting Rewrites Not Firing (2026)
Your firebase.json rewrites for an SPA or Cloud Function don't trigger. Fix the order, region, and deploy target in minutes.
Works Locally, Fails in Production: The Fix Checklist
Works on localhost, breaks on Vercel / Render / Cloudflare Pages. Reproduce the prod build locally, diff env vars, pin the Node version, and fix Linux case-sensitivity — the systematic checklist.
Fix External API 429 Rate Limit Errors
Stripe, Twilio, and SendGrid all return 429 once traffic ramps. Read the rate-limit headers, add exponential back-off with jitter, cache idempotent GETs, dedupe concurrent calls, and batch writes — in that order.
Object Storage Upload Denied (S3 / Firebase / Supabase 403)
Fix S3, Firebase Storage, and Supabase Storage upload 403s — IAM permission, signed-URL expiry, bucket policy, or disabled-ACL gotcha.
Supabase Env Vars Undefined in Production: The Fix
Supabase URL or anon/publishable key is undefined after deploy — fix the host env config, framework prefix, stale build, and the 2026 key migration.
Supabase RLS Returns an Empty Array (No Error)
Your Supabase table has rows but the frontend gets [] with no error. Diagnose which RLS bucket you're in and add the right SELECT policy in minutes.
Webhook Not Firing: Provider Says 200, Your Endpoint Is Silent
Stripe, GitHub, or Shopify says the webhook was delivered but your server never sees it. Read the delivery log, find the status code, and fix the real cause.