Unable to communicate to a machine in the network(outside of the k8s cluster) with cidr 10.0.x.0/24 from within the pod

Question

I have two machines within my netwrok which I want communicate from the pod.

Ips are as follows :

10.0.1.23 - Lets call it X

13.0.1.12  - Lets call it Y

When I ssh into the master node or agent node and then do a ping to X or Y, the ping is successful. Therefore the machines are reachable.

Now I create a deployment, I log into the shell of the pod using ( kubectl exec -it POD_NAME — /bin/sh ).

Ping to Y is successful. But ping to X fails.

CIDR details :

Master Node : 14.1.255.0/24
Agent Node: 14.2.0.0/16
Pod CIDR: 
   Agent : 10.244.1.0/24
   Master: 10.244.0.0/24

My understanding on what could be the issue :

acs-engine has kube-proxy setup the service network with 10.0.0.0/16 If this is the problem how do i change the kube-proxy cidr?

Additional Info:

I am using acs-engine for my deployment of cluster.

Output for ip route

default via 10.244.1.1 dev eth0 10.244.1.0/24 dev eth0 src 10.244.1.13

Another suspect: On running iptables-save I see

-A POSTROUTING ! -d 10.0.0.0/8 -m comment --comment "kubenet: SNAT for outbound traffic from cluster" -m addrtype ! --dst-type LOCAL -j MASQUERADE

Answer 1

Based on your question, it sounds like you've added another subnet to the k8 Virtual Network that gets deployed with the ACS Kubernetes cluster.

As it turns out, I ran into this exact same problem in our project. Azure Container Services uses very specific routing rules for the agent nodes. When the k8 cluster is deployed, they create a Route Table resource in the same resource group as all your cluster entities. So, if you...

1. Open the k8 Route Table in the Azure Portal
2. Go to the Subnets section
3. +Associate with the subnet that your other VMs/PaaS services are in

...this will create the routes that the k8 agents are looking for when routing the outbound Pod container traffic.

Answer 2

I have the exact same problem, after googling so much I found a posible solution:

Use ip-masq-agent to masq the target CIDR in order to MASQUERADE that destination

https://kubernetes.io/docs/tasks/administer-cluster/ip-masq-agent/

Some similar example:

http://madorn.com/kubernetes-non-masquerade-cidr.html#.XMDGI-H0nb0

Answer 3

You cannot ping a kubernetes service. More information here: https://github.com/kubernetes/kubernetes/issues/7996#issuecomment-100413276 . To test connectivity, you can expose a simple web server on a port, and confirm using curl from inside or outside the container.