Spring boot fails to connect to managed Redis cluster; "Connection refused"

[pulsar-gupta]

Type: Bug

Component: ElastiCache

Describe the bug
I am using Spring boot 3 with aws dependencies - spring-boot-starter-data-redis. The managed redis is created on my public subnet, and I can connect to it using redis-cli from my local as well as from EC2 server in same VPC. The problem lies when I deploy my code on EKS and try to access the logic through API, I get "Connection refused". To check the networking from the pod, I went inside the pod and tried to connect to cache using redis-cli and I was successful. The connectivity is not the issue, I suspect Spring boot is not able to resolve the host.

Sample

1. kubectl exec -it service-9b55879cf-892nb -n dev -- /bin/sh
   redis-cli -h public-001-001.public.1ysfdy.cache.amazonaws.com -p 6379
   public-001-001.public.1ysfdy.cache.amazonaws.com:6379 "I am able to perform all operations"

2. Logs of the same pod
   org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/controlling].[dispatcherServlet].log(175) : Servlet.service() for servlet [dispatcherServlet] in context with path [/controlling] threw exception [Request processing failed: org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis] with root cause
   java.net.ConnectException: Connection refused
   at java.base@17.0.7/sun.nio.ch.Net.pollConnect(Native Method)
   at java.base@17.0.7/sun.nio.ch.Net.pollConnectNow(Net.java:672)
   at java.base@17.0.7/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
   at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337)
   at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
   at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
   at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
   at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
   at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:838)
   at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)

[maciejwalkowiak]

Why do you suspect it has something to do with Spring Cloud AWS? Looks like a networking/permissions issue

[pulsar-gupta]

I have tested the connections by executing redis-cli inside pod and it connects to redis. However, when I try to call my api, which actually uses @Cacheable, I get connection refused error. I suspect either aws dependency is not able to acquire proper details or it is not able to resolve the name. Since, I am in a project phase I have kept the cache on public net with no password.