When a Service Can't Reach Redis: VNet Integration Changed the Outbound IP

A push service in a test environment couldn’t connect to Azure Redis. It was a Node.js service on Azure App Service, with BullMQ as the job queue and the queue stored in Azure Cache for Redis. Queue initialisation failed, and the pushes that depended on the queue stopped going out.

The report I wrote afterwards is what this post is built from. I’m not going to reconstruct the order in which I checked things from memory. What I can say is what the evidence showed.

Which layer?

“Can’t connect” can mean several quite different things:

• The host or port is misconfigured.
• The password or TLS settings are wrong: the connection opens but the handshake or authentication fails.
• The network doesn’t get through at all and the connection is refused.

The report recorded the host, the port (6379) and TLS off, and it described the failure as the network blocking the connection, a refusal rather than an authentication error. That points at the network: Redis wasn’t accepting the request.

Azure Redis has a firewall that only admits allow-listed IPs. So the question became: which IP is this Web App actually using to reach Redis?

The outbound IP isn’t the one you think

App Service lists a set of “outbound IP addresses” in the portal. Those were what the Redis firewall allowed. On paper, fine.

But this Web App had VNet integration enabled, with “route all” turned on (vnetRouteAllEnabled: true), and its subnet had a NAT gateway attached. According to the report, that meant the connection to Redis left through the NAT gateway. From Redis’s side, the request came from the NAT gateway’s public IP, not from any of the addresses listed on the App Service page.

I’m only claiming this for the path to Redis. I didn’t check every destination the app talks to, or any private routes.

The outbound IPs in the portal weren’t wrong. With this network setup, they just weren’t the exit this connection used any more.

Put three pieces of information side by side and the cause is clear:

1. Web App: VNet integration on, route all, bound to a subnet with a NAT gateway.
2. NAT gateway: one fixed public IP.
3. Redis firewall: only App Service’s default outbound IPs, not the NAT one.

The fix

The network fix is a single step: add the NAT gateway’s public IP to the Redis firewall allow list.

az redis firewall-rules create \
  --resource-group <resource-group> --name <redis-name> \
  --rule-name AllowNatGateway \
  --start-ip <nat-public-ip> --end-ip <nat-public-ip>

Afterwards the service’s health check showed Redis, the queue and the application itself all passing, with Redis responding in a few milliseconds. At that point the environment was still on the non-TLS port, so TLS had nothing to do with the fix.

While I was there: getting ready for TLS

The test environment was on the non-TLS port 6379, so I made the queue’s connection config able to switch TLS on with an environment variable. In our setup we use the TLS port 6380 and set servername to the Redis host name (SNI). I haven’t tested what happens without it, so I won’t call it a requirement.

const connection = {
  host: process.env.REDIS_HOST ?? 'localhost',
  port: Number(process.env.REDIS_PORT ?? 6379),
  password: process.env.REDIS_PASSWORD,
  ...(process.env.REDIS_TLS_ENABLED === 'true'
    ? { tls: { servername: process.env.REDIS_HOST } } // SNI: how we connect to Azure Redis
    : {}),
};

The plan is that switching to TLS is two App Settings (the flag and the port) and a restart, with no code change, and switching them back is the rollback. That plan is in the report; I haven’t verified the switch itself.

Things left to tidy up

The report listed what should follow. These are suggestions, not things I’ve confirmed as done:

• Clean up the old firewall rules. If all of this service’s traffic to Redis leaves through the NAT gateway, the rules for App Service’s default outbound IPs no longer do anything. The report says “if”, and I haven’t checked for other clients, deployment slots or rollback paths that might still use them, so I wouldn’t delete them on this evidence alone.
• Close the non-TLS port once TLS is stable.
• Add an alert. Alert on Redis connection failures in Application Insights instead of waiting for QA to notice pushes aren’t going out.
• Document the NAT exit. The best prevention for “the network changed in one place and an allow list elsewhere didn’t follow” is to have it on the change checklist.

Looking back

The lesson here isn’t about Redis, it’s about the idea of an “outbound IP”. Whenever network routing changes, for example turning on VNet integration, adding a NAT gateway or changing the exit, a downstream service that relies on an IP allow list (a database, Redis, a third-party API) can lock you out. And the outbound IPs shown in the portal won’t tell you they’ve stopped being the exit a given connection uses.

The question to ask is the one above: when this connection fails, which IP is it actually coming from, and is that IP on the allow list? Look at the error type to decide the layer, then compare the real exit against the firewall rules.