k8s nginx-ingress 浏览器可以实现轮询,但是curl命令无法轮询

请教一下,通过k8s 的ingress实现的7层负载均衡,为啥浏览器可以实现轮询,但是curl命令无法轮询,希望大佬能提供下思路

前提:

1、在k8s中配置了一个ingress,配置如下

[root@node-1 ~]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx <none> nginx.zerchin.xyz 172.16.0.211     80 31m


[root@node-1 ~]# kubectl get ingress nginx -o yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
name: nginx
namespace: default
spec:
rules:
- host: nginx.zerchin.xyz
http:
paths:
- backend:
serviceName: nginx-test
servicePort: http-nginx-test
path: /

2、后端的nginx-test是一个基于nginx镜像启动的web服务

[root@node-1 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-test-96b89c957-77npv 1/1 Running 0 35m
nginx-test-96b89c957-bjtbs 1/1 Running 0 35m
nginx-test-96b89c957-r7g89 1/1 Running 0 35m [root@node-1 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 20d
nginx-test ClusterIP 10.43.114.167 <none> 80/TCP 35m
nginx-test-nodeport NodePort 10.43.23.216 <none> 80:32370/TCP 35m


问题现象:

1、如果我通过curl对应的service,可以实现轮询(因为后端是基于ipvs转发的)

[root@node-1 ~]# for i in `seq 1 12`;do curl 10.43.114.167;done
111
3
22
111
3
22
111
3
22
111
3 22

2、但是如果curl ingress对应的地址,反而不能实现轮询

[root@node-1 ~]# for i in `seq 1 12`;do curl nginx.zerchin.xyz ;done
3
22
111
22
111
111
3
3
3
3
22
22

3、如果是通过浏览器访问,通过Ctrl+R刷新浏览器,这个却是可以实现轮询


这个结果跟之前通过curl对应的service是一样的


因为k8s的ingress-nginx是基于lua实现的,我不太懂lua语言,所以要请教一下大佬们,为啥curl和浏览器访问的结果不一样。  


排查:

通过Postman工具,如果勾选了connection:keep-alive,就可以实现轮询,取消了这个header,就无法轮询,想知道nginx的什么语法可以控制基于connection:keep-alive控制负载的轮询


k8s ingress-controller相关的nginx.conf配置

        server {
server_name nginx.zerchin.xyz ;

listen 80 ;
listen 443 ssl http2 ;

set $proxy_upstream_name "-";

ssl_certificate_by_lua_block {
certificate.call()
}

location / {

set $namespace "default";
set $ingress_name "nginx";
set $service_name "nginx-test";
set $service_port "http-nginx-test";
set $location_path "/";

rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}

# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}

header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}

body_filter_by_lua_block {
}

log_by_lua_block {
balancer.log()

monitor.call()

plugins.run()
}

port_in_redirect off;

set $balancer_ewma_score -1;
set $proxy_upstream_name "default-nginx-test-http-nginx-test";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;

set $pass_server_port $server_port;

set $best_http_host $http_host;
set $pass_port $pass_server_port;

set $proxy_alternative_upstream_name "";

client_max_body_size 1m;

proxy_set_header Host $best_http_host;

# Pass the extracted client certificate to the backend

# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection $connection_upgrade;

proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $remote_addr;

proxy_set_header X-Forwarded-Proto $pass_access_scheme;

proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;

proxy_set_header X-Scheme $pass_access_scheme;

# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";

# Custom headers to proxied server

proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;

proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;

proxy_max_temp_file_size 1024m;

proxy_request_buffering on;
proxy_http_version 1.1;

proxy_cookie_domain off;
proxy_cookie_path off;

# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 1;
proxy_next_upstream_tries 3;

proxy_pass http://upstream_balancer;

proxy_redirect off;

}

}


ingress相关参数文档:https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md





邀请回答
提问于 2020-10-30 17:06
376 次浏览
共0个回答

发表评论
提问者

zerchin

暂无个人介绍

  • 0

    文章

  • 0

    粉丝

  • 0

    被赞

zerchin
按Enter键发送
您已邀请位用户
版权所有©F5 Networks,Inc.保留所有权利。京ICP备16013763号-5