GRPC service tap not working and not appearing in dashboard services overview

[janwytze]

I'm trying to locally (Using KIND) run my GRPC service with Linkerd, the pod and deployment are listed in the dashboard and they have a success-rate of 100%. But I cannot see any requests in tap, there are no services in the services overview and the traffic also does not get load balanced.

This is my yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: grpc-app
  template:
    metadata:
      labels:
        app: grpc-app
    spec:
      containers:
      - name: grpc-app
        image: ...
        ports:
        - containerPort: 50051
---
apiVersion: v1
kind: Service
metadata:
  name: grpc-service
spec:
  selector:
    app: grpc-app
  ports:
    - protocol: TCP
      port: 50051
      targetPort: 50051
  type: ClusterIP

I've followed every step in the Getting started page. All checks seem to pass and I can reach the service just fine. Only the Linkerd part is not working.

The emojivoto project runs fine on my KIND cluster and the services are visible in the dashboard. I've actually edited the emojivoto yaml with my service, but that didn't work.

[alpeb]

Any chance to provide a complete minimal manifest for a repro?

[janwytze]

@alpeb The problem was that I was trying to reach the grpc-service with port-forwarding. Is there a way to get this working? Here is my example where load balancing fails:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: grpc-server
  template:
    metadata:
      labels:
        app: grpc-server
    spec:
      containers:
      - name: grpc-server
        image: ghcr.io/janwytze/grpc-test-server:main
        imagePullPolicy: Always
        ports:
        - containerPort: 50051
---
apiVersion: v1
kind: Service
metadata:
  name: grpc-service
spec:
  selector:
    app: grpc-server
  type: LoadBalancer
  ports:
    - port: 50051
      targetPort: 50051
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-client
spec:
  selector:
    matchLabels:
      app: grpc-client
  template:
    metadata:
      labels:
        app: grpc-client
    spec:
      containers:
      - name: grpc-client
        image: ghcr.io/janwytze/grpc-test-client:main
        imagePullPolicy: Always
        env:
        - name: GRPC_SERVER
          value: "grpc-service.default.svc.cluster.local:50051"

The client does a status request to the server every second. Each server chooses a random status at start, so when load balancing works the client should report different statuses. When I check out the client logs load balancing seems to work!! But when I setup port forwarding with:

kubectl port-forward svc/grpc-service 50051:50051

And them do a request to localhost:50051 I always get the same status, so only one pod is getting the requests.

Is it possible to get load balancing with port-forwarding?

[alpeb]

Hey, sorry for the late reply. When used against a service, I think kubectl port-forward will just select one target pod from the service selector, and stick to it. There's no going through kube-proxy nor any load-balancer, and the connection is established directly to the pod.