Unable to connect ADO with google-dataform

[amirkhan]

I am getting below mentioned error while connecting my ADO repository with google dataform using. Even after providing public and private SSH correctly I am getting the same error. I am using some other service account where Dataform Admin role is provided to it. Also the same principal has Secret Manager Secret Accessor role. Still I am unable to connect. Could someone please help?

 

[ms4446]

Connecting Azure DevOps (ADO) with Dataform using SSH can sometimes present challenges. It's crucial to approach this systematically to identify and resolve the underlying issue. Below, we've outlined steps to help troubleshoot and resolve the connection error you're experiencing.

SSH Key Configuration

1. Key Validity: Ensure your SSH key pair is correctly generated (RSA, minimum of 2048 bits). For guidance on generating a valid SSH key pair, refer to GitHub's documentation.
2. Correct Key Upload: Confirm that the public key is added to your ADO profile under "SSH Public Keys" and the private key is correctly stored in Google Secret Manager.

Service Account Permissions

• Secret Manager Secret Accessor Role: You've correctly assigned this role, but also verify that the service account is authorized to access secrets within Dataform.
• Secret Access Management: Make sure you're referencing the latest version of the secret containing the private key. Labeling secrets clearly can help avoid confusion.

Network and Firewall Settings

• Network Accessibility: Check for any firewall rules that might be blocking Google Cloud Dataform's access to Azure DevOps. This includes verifying open ports and ensuring no network policies are interfering with connectivity.

Repository URL Format

• SSH URL Format: The repository URL should follow the format: ssh://git@ssh.dev.azure.com/v3/<organization>/<project>/<repository>. Ensure this is correctly set in Dataform.

Additional Troubleshooting Steps

• Verbose Logging: Enable verbose logging within Dataform to capture more detailed error messages, which can provide insights into what might be going wrong.
• Manual SSH Connection Test: Attempt to manually connect to your ADO repository using SSH from an environment similar to Dataform's to isolate the issue.

Considerations

• Cloud Build Service Agent Role: Although not initially mentioned, granting the service account the Cloud Build Service Agent role temporarily can help determine if additional permissions are required for your operations.

Support

• Google Cloud Support: If the issue persists after following these steps, reaching out to Google Cloud Support can provide more in-depth troubleshooting assistance.

[amirkhan · Accepted solution]

@ms4446 I've resolved this issue. Thanks for responding. I've done 2 things in order to resolve this, 

1. Reformatted the SSH URL in this format: ssh://git@ssh.dev.azure.com/v3/<organization>/<project>/<repository>.
2. Took the SSH public key from the known_hosts file

[tushensu]

Hi @amirkhan I am currently struggling with this, where did you get the know_hosts file ? Should it be in the machine where I generate my keys or is it somewhere is ADO ?

[tushensu]

I found the key, I had to ssh using bash then ADO was added to the known hosts files locally and then I was able to copy this.

[VictorC]

Same question i´m using windows, can you help me?. tks!.

[VictorC]

Hi!, Where did you get the know_hosts file in windows?.

[VictorC]

I found it, is in the folder C:\Users\user\.ssh.