proxy_buffering默认打开会影响k8s apiserver watch场景

后端是k8s原生api:

/api/v1/namespaces/test/pods?watch=true

当pod没有任何变化的时候,如果直接curl k8s apiserver api,展示如下:

curl -v -k -H 'Authorization: Bearer XXX' https://10.209.31.201:6443/api/v1/namespaces/test/pods?watch=true
* About to connect() to 10.209.31.201 port 6443 (#0)
* Trying 10.209.31.201...
* Connected to 10.209.31.201 (10.209.31.201) port 6443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=kube-apiserver,O=kubernetes
* start date: Mar 30 08:09:26 2022 GMT
* expire date: Mar 06 08:09:26 2122 GMT
* common name: kube-apiserver
* issuer: CN=kubernetes
> GET /api/v1/namespaces/test/pods?watch=true HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.209.31.201:6443
> Accept: */*
> Authorization: Bearer XXX
>
< HTTP/1.1 200 OK
< Audit-Id: 6c514cca-6601-4ded-8a3b-e0b03250b109
< Cache-Control: no-cache, private
< Content-Type: application/json
< X-Kubernetes-Pf-Flowschema-Uid: 0fc4472b-b418-4da2-8aaf-6de6b1aff3c7
< X-Kubernetes-Pf-Prioritylevel-Uid: e6055748-c935-45b2-b700-700e78616a2c
< Date: Mon, 11 Jul 2022 02:42:59 GMT
< Transfer-Encoding: chunked
<
^C

注意到这里返回是有一个

HTTP/1.1 200 OK


如果curl nginx,upstream配置为apiserver地址,基本配置:

location /api/v1/* {                                                
proxy_pass https://10.209.31.201:6443;
}

curl的结果是:

curl -v -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPMGw1YjFrN3plVDNrR1RrZlZJcXNQQ0FIOEtuOWtMSGRJNFJVYW5UUEVNIn0.eyJleHAiOjE2NTkyNTkzMDIsImlhdCI6MTY1NjY2NzMwMiwianRpIjoiODViZTIxZDItNWI4OS00MTgzLWFjMzUtZjJhMGM2ZWYxMGZjIiwiaXNzIjoiaHR0cHM6Ly9wYWFzLWxpbmdoYW5nLmtleWNsb2FrLm5pbmdiby1wb2QzLTIwOS0xNTguNGEuY21pdC5jbG91ZDoyMDA0Mi9hdXRoL3JlYWxtcy9rZW0iLCJhdWQiOlsiaGFyYm9yIiwia3ViZXJuZXRlcyIsImtlbSIsImFjY291bnQiXSwic3ViIjoiZDc4ODg1YmUtOWEyYi00NjRmLThjZTItOGFkMGIwYzdlMWJiIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoia2VtIiwic2Vzc2lvbl9zdGF0ZSI6ImU2YmQyMGUwLTVmNDUtNDYzOC1iNjg5LWY0MGMwYmM0N2ExMyIsImFjciI6IjEiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiIsImRlZmF1bHQtcm9sZXMta2VtIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsia2VtIjp7InJvbGVzIjpbInVtYV9wcm90ZWN0aW9uIiwiQWRtaW4iXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoia3ViZXJuZXRlcyBlbWFpbCBwcm9maWxlIGhhcmJvciBrZW0iLCJzaWQiOiJlNmJkMjBlMC01ZjQ1LTQ2MzgtYjY4OS1mNDBjMGJjNDdhMTMiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsIm5hbWUiOiJ0ZW5hbnRfYWRtaW4iLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJlY2FkbWluIiwiZ2l2ZW5fbmFtZSI6InRlbmFudF9hZG1pbiIsImVtYWlsIjoiZWNhZG1pbkBlYy5jb20ifQ.fjH7f_N86N-5gs8mHiFszmyzqZZ_0KH2bTGWkc97mYiBliTSnEReR1QD-iMgO7iB9MDPiKlhmeq1cl2lswYP0KjgJdH8w5dFw7mwaGLW8Sc7pRdLwJrt4s11VcIyPs_aRE8J5XJ1x1Vp6psUtQbzeYDPNrVwhkYw_h3TKROcWXObKwSFrTB5Mqbx28IV3vdAkkOzVSYPYPwUcGH0g8-CxaJykGhpNN7wqEehlbt5oH_DwXZwJK7bluvICdhZSpsJce3pe3Ijs-AQjT-2SJ4pxqvL0iWCTy7EsbxlJpfCAXnFxAdt5fVXjFYuFfH0A_DS71VKpSw4NvfGG9XuuVoHZQ' http://127.0.0.1:80/api/v1/namespaces/test/pods?watch=true
* About to connect() to 127.0.0.1 port 80 (#0)
* Trying 10.32.75.58...
* Connected to 127.0.0.1 (10.32.75.58) port 80 (#0)
> GET /api/v1/namespaces/test/pods?watch=true HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:80
> Accept: */*
> Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPMGw1YjFrN3plVDNrR1RrZlZJcXNQQ0FIOEtuOWtMSGRJNFJVYW5UUEVNIn0.eyJleHAiOjE2NTkyNTkzMDIsImlhdCI6MTY1NjY2NzMwMiwianRpIjoiODViZTIxZDItNWI4OS00MTgzLWFjMzUtZjJhMGM2ZWYxMGZjIiwiaXNzIjoiaHR0cHM6Ly9wYWFzLWxpbmdoYW5nLmtleWNsb2FrLm5pbmdiby1wb2QzLTIwOS0xNTguNGEuY21pdC5jbG91ZDoyMDA0Mi9hdXRoL3JlYWxtcy9rZW0iLCJhdWQiOlsiaGFyYm9yIiwia3ViZXJuZXRlcyIsImtlbSIsImFjY291bnQiXSwic3ViIjoiZDc4ODg1YmUtOWEyYi00NjRmLThjZTItOGFkMGIwYzdlMWJiIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoia2VtIiwic2Vzc2lvbl9zdGF0ZSI6ImU2YmQyMGUwLTVmNDUtNDYzOC1iNjg5LWY0MGMwYmM0N2ExMyIsImFjciI6IjEiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiIsImRlZmF1bHQtcm9sZXMta2VtIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsia2VtIjp7InJvbGVzIjpbInVtYV9wcm90ZWN0aW9uIiwiQWRtaW4iXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoia3ViZXJuZXRlcyBlbWFpbCBwcm9maWxlIGhhcmJvciBrZW0iLCJzaWQiOiJlNmJkMjBlMC01ZjQ1LTQ2MzgtYjY4OS1mNDBjMGJjNDdhMTMiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsIm5hbWUiOiJ0ZW5hbnRfYWRtaW4iLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJlY2FkbWluIiwiZ2l2ZW5fbmFtZSI6InRlbmFudF9hZG1pbiIsImVtYWlsIjoiZWNhZG1pbkBlYy5jb20ifQ.fjH7f_N86N-5gs8mHiFszmyzqZZ_0KH2bTGWkc97mYiBliTSnEReR1QD-iMgO7iB9MDPiKlhmeq1cl2lswYP0KjgJdH8w5dFw7mwaGLW8Sc7pRdLwJrt4s11VcIyPs_aRE8J5XJ1x1Vp6psUtQbzeYDPNrVwhkYw_h3TKROcWXObKwSFrTB5Mqbx28IV3vdAkkOzVSYPYPwUcGH0g8-CxaJykGhpNN7wqEehlbt5oH_DwXZwJK7bluvICdhZSpsJce3pe3Ijs-AQjT-2SJ4pxqvL0iWCTy7EsbxlJpfCAXnFxAdt5fVXjFYuFfH0A_DS71VKpSw4NvfGG9XuuVoHZQ
>
^C

 并不会打印

HTTP/1.1 200 OK


我将proxy_buffering默认配置从on改成off了后,就一致了

proxy_buffering off;


另外,如果是pod有数据,走不走nginx都是符合预期的,会先打印一个http 200的response.

apiserver的watch如果没有返回值先返回一个200,是apiserver业务层的设计。


不知道这块算是nginx的问题还是使用方法不正确导致的


邀请回答
提问于 2022-07-13 09:41
82 次浏览
共1个回答

发表评论
提问者

sl1836

暂无个人介绍

  • 0

    文章

  • 0

    粉丝

  • 0

    被赞

sl1836
按Enter键发送
您已邀请位用户
Copyright 公安部网络安全保卫局 All Rights Reserved
京公网安备 11010502047880号    京ICP备05070602号