Cannot delete deployment

jennn

I've got an infra-manager deployment which was created via a one-click "solution" app (https://console.cloud.google.com/products/solutions/details/generative-ai-rag) that can't be deleted. It can't because the Terraform destroy plan cannot be applied. The plan cannot be applied because some infrastructure managed by tf no longer exists in GCP. I can't seem to get tf to run to create the absent resources so I can then delete all the resources it knows about either.

ERROR: build step 0 "us-central1-docker.pkg.dev/cloud-config-sdk/images/config-sdk-tf:public-image-v0.0.150_1.5.7" failed: step exited with non-zero status: 1
ERROR
Finished Step #0 - "Destroy blueprint"
Step #0 - "Destroy blueprint": Error: running tf destroy: terraform destroy failed: running terraform failed: exit status 1
Step #0 - "Destroy blueprint": 
Step #0 - "Destroy blueprint": on .terraform/modules/kuberay-operator/modules/kuberay-operator/kuberay.tf line 15
Step #0 - "Destroy blueprint": 
Step #0 - "Destroy blueprint": error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint": Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint": running tf destroy: terraform destroy failed: running terraform failed: exit status 1
Step #0 - "Destroy blueprint": Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint": time="2024-05-02T09:22:08Z" level=error msg="Error (exit code 1) running \"terraform destroy -auto-approve -json\". Stderr:\n"
Step #0 - "Destroy blueprint": Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint": 
Step #0 - "Destroy blueprint": on .terraform/modules/namespace/modules/kubernetes-namespace/main.tf line 16
Step #0 - "Destroy blueprint": 
Step #0 - "Destroy blueprint": error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint": Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Step #0 - "Destroy blueprint":

This terraform is being run by a cloud build job that was created when the app was created; I don't know where to find the tf code or if it's possible to do statefile surgery (e.g. pseudocode: gcloud infra-manager deployment export-statefile - | jq 'del(.resources[]|select('resources to delete'))' | gcloud infra-manager import-statefile -). I've gone and deleted everything I could find so far to try and keep from getting charged as this is on a personal account. Help?

AlfredB

Hi jenn,

Welcome to Google Cloud Community.

As per checking on the error, it seems that the Kubernetes cluster is unreachable, as stated you may try to attempt first to Manually set the KUBERNETES_MASTER environment variable. You may try to attempt defining this variable with the address of your Kubernetes API endpoint, and try to destroy it.

Also you may try checking the terraform state files, as it will conflict with the destroy command if some resources no longer exist. You can try to manually edit the state file to remove the non-existent resources if possible.

Lastly you can try forcing the destroy command. terraform destroy -force. However please be wary and use this with caution as it might lead to unexpected outcomes to your other resources in GCP. Please do try to have a backup when executing the -force flag in destroy command.

More info about errors in deployment in this document.

I hope this information is helpful.

If you need further assistance, you can always file a ticket on our support team.