1.问题来源

Rancher2.3上给用户授权项目权限的时候，查看UI界面上的命名空间的时候显示此项目未添加命名空间，更新项目中的容器时命名空间为null，然而其它的项目是没有问题的，起初怀疑是授权的问题，然后给授予项目的管理员的权限，没鸟用，还是一样，然后尝试给授予全局管理员权限，问题是解决了，可是这个权限也忒大了，不行，的考虑其它方案

2.排查问题思路

1. 添加一个本地用户进行授予成员的身份
2. 授权多个项目给此用户
3. 用添加的用户进行登录验证（有的项是正常的，有的项目是有问题的）

3.切回自己的账号登录

查看有问题的项目及正常的项目，发现命令有问题的项目里命名空间中用Terminating状态，而正常的项目中没有次问题

4.删除Terminating状态的namespace

1.找出Terminating状态的ns

[root@cnvs-kubm-101-103 ~]# kubectl get ns |grep etcd
etcd                        Terminating   37d
etcd-operator-6ssxh         Active        38d

2.进行删除使用--force参数依然无法删除，报错:

[root@cnvs-kubm-101-103 ~]# kubectl delete ns etcd --force
warning: --force is ignored because --grace-period is not 0.
Error from server (Conflict): Operation cannot be fulfilled on namespaces "etcd": The system is ensuring all content is removed from this namespace.  Upon completion, this namespace will automatically be purged by the system.

 3.然后导出json文件进行删除

[root@cnvs-kubm-101-103 ~]# kubectl get namespace etcd -o json > etcd.json

##内容如下
[root@cnvs-kubm-101-103 ~]# cat etcd.json
{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "annotations": {
            "cattle.io/status": "{\"Conditions\":[{\"Type\":\"InitialRolesPopulated\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2020-02-06T05:02:31Z\"},{\"Type\":\"ResourceQuotaInit\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2020-02-06T05:02:30Z\"}]}",
            "field.cattle.io/creatorId": "u-nlwptzyggc",
            "field.cattle.io/projectId": "local:p-rnm7t",
            "lifecycle.cattle.io/create.namespace-auth": "true"
        },
        "creationTimestamp": "2020-02-06T05:02:29Z",
        "deletionTimestamp": "2020-02-10T07:26:27Z",
        "labels": {
            "cattle.io/creator": "norman",
            "field.cattle.io/projectId": "p-rnm7t"
        },
        "name": "etcd",
        "resourceVersion": "27716037",
        "selfLink": "/api/v1/namespaces/etcd",
        "uid": "fd829b33-5f2b-47f5-a1e6-036f886de249"
    },
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2020-02-10T07:26:34Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2020-02-10T07:26:34Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2020-02-10T07:26:34Z",
                "message": "All content successfully deleted",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            }
        ],
        "phase": "Terminating"
    }
}

##删除其中的spec字段，因为k8s集群是携带认证的
 "spec": {
 },

因为这边的K8s集群是带认证的，所以又新开了窗口运行kubectl proxy跑一个API代理在本地的8081端口

[root@cnvs-kubm-101-103 ~]# kubectl proxy
Starting to serve on 127.0.0.1:8001

执行以下curl命令，使用kube-apiserver的8080端口，执行删除操作

[root@cnvs-kubm-101-103 ~]#  curl -k -H "Content-Type: application/json" -X PUT --data-binary @etcd.json http://127.0.0.1:8001/api/v1/namespaces/etcd/finalize

执行的结果内容如下：

{
  "kind": "Namespace",
  "apiVersion": "v1",
  "metadata": {
    "name": "etcd",
    "selfLink": "/api/v1/namespaces/etcd/finalize",
    "uid": "fd829b33-5f2b-47f5-a1e6-036f886de249",
    "resourceVersion": "27716037",
    "creationTimestamp": "2020-02-06T05:02:29Z",
    "deletionTimestamp": "2020-02-10T07:26:27Z",
    "labels": {
      "cattle.io/creator": "norman",
      "field.cattle.io/projectId": "p-rnm7t"
    },
    "annotations": {
      "cattle.io/status": "{\"Conditions\":[{\"Type\":\"InitialRolesPopulated\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2020-02-06T05:02:31Z\"},{\"Type\":\"ResourceQuotaInit\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2020-02-06T05:02:30Z\"}]}",
      "field.cattle.io/creatorId": "u-nlwptzyggc",
      "field.cattle.io/projectId": "local:p-rnm7t",
      "lifecycle.cattle.io/create.namespace-auth": "true"
    }
  },
  "spec": {

  },
  "status": {
    "phase": "Terminating",
    "conditions": [
      {
        "type": "NamespaceDeletionDiscoveryFailure",
        "status": "False",
        "lastTransitionTime": "2020-02-10T07:26:34Z",
        "reason": "ResourcesDiscovered",
        "message": "All resources successfully discovered"
      },
      {
        "type": "NamespaceDeletionGroupVersionParsingFailure",
        "status": "False",
        "lastTransitionTime": "2020-02-10T07:26:34Z",
        "reason": "ParsedGroupVersions",
        "message": "All legacy kube types successfully parsed"
      },
      {
        "type": "NamespaceDeletionContentFailure",
        "status": "False",
        "lastTransitionTime": "2020-02-10T07:26:34Z",
        "reason": "ContentDeleted",
        "message": "All content successfully deleted"
      }
    ]
  }
}

4.验证结果

##Terminating状态的namespace以删除
[root@cnvs-kubm-101-103 ~]# kubectl get ns |grep etcd
etcd-operator-6ssxh         Active   38d

参考地址：
https://github.com/kubernetes/kubernetes/issues/19317
https://my.oschina.net/u/4222205/blog/3113044
https://www.ichenfu.com/2019/02/20/kubernetes-namespaces-stuck-in-terminating-state/
https://www.jianshu.com/p/00351f9eb37b

AD：《熊孩子博客》熊孩子QQ:443060965