APNS connection issues

[orhan-abdula]

Hello,

We are using pushy for about 10 months now and since month or so ago we are seeing an uptick in connection issues - Connection reset ,
io.netty.channel.ConnectTimeoutException ,
io.netty.resolver.dns.DnsNameResolverTimeoutException query via UDP timed out after 5000 milliseconds (no stack trace available) i suspect there is something wrong with the DNS resolving, maybe netty caches an old IP address?
Has anyone seen these kind of issues, i found some old questions about DNS but i am not sure if they are applicable for the latest versions.
Our version is 0.14.2 and here is how we create our client

 NioEventLoopGroup eventLoopGroup = new NioEventLoopGroup(1, executor);
      ApnsClient apnsClient = new ApnsClientBuilder().setApnsServer(getApnsHost(devMode))
          .setClientCredentials(new ByteArrayInputStream(certValue), p12Password)
          .setConcurrentConnections(concurrentConnections)
          .setEventLoopGroup(eventLoopGroup)
          .build();

[jchambers]

It sounds like the problem is happening when you're trying to resolve (for example) api.push.apple.com to a list of IP addresses. That sounds like a local network issue to me, and is unlikely to be a problem within Pushy itself (especially if it started happening spontaneously after many months without any significant changes).

Can you say more about your network setup? Are you behind a proxy?

[orhan-abdula]

We have a k8s deployment with istio sidecar and no custom config for apns requests. After some more testing and researching i suspect that we reach a limitation of total concurrent connections but i cannot reproduce it in a dev setup.
We use .setConcurrentConnections(100)

[orhan-abdula]

Now we suspect that APNS is somehow limiting our connections by resetting some of them so i wanted to ask if you know about any limitations from APNS about the number of connections per IP address or single client/certificate or anything else because we cannot get an answer from their support.

[jchambers]

I don't have any more concrete knowledge than you do, but in your previous post, you wrote:

We use .setConcurrentConnections(100)

Even absent knowledge of any specific upstream limits, that's a lot of concurrent connections! I don't think you're going to see any performance benefit from using more connections than there are APNs servers.

At the time of writing, I count 8 servers (you can check yourself with nslookup api.push.apple.com or dig api.push.apple.com). I don't have any specific reason to believe it will solve your problem, but I think you might see better results generally if you limit your concurrent connection count to somewhere around 8 (and I'd say no more than 16). That's especially true if you're limiting you're event loop group to a single thread, as you pointed out in your initial comment.