Could not access services with Access Context Manager/VPC Service Control setting

[Kassy300]

Hi, I'm tring to create IP address fillter using Access Context Manager and VPC Service Control. However I could not access the resorces with filtered IP address.

I am struggling several days from what I should check.

May I have your suggestion where I should check?

[pfilourenco]

Hi,

Your public IP match the IP filter?
To debug you can also go to the log's to see what is the error or get the VPC-SC unique identifier (the identifier can be found in the 403 error message), and use the VPC Service Controls Troubleshooter.

BR,

Pedro Lourenço 

[RyoYoneda]

HI. I'm having the same problem.

Even after setting the IP filter to the same IP as the public IP and giving the VPC-SC access level as per the official documentation below, I still get the log "NO_MATCHING_ACCESS_LEVEL" and cannot connect from the public IP.
https://cloud.google.com/vpc-service-controls/docs/use-access-levels

The official document stated that it takes 30 minutes to reflect VPC-SC, but we believe it is not a waiting time issue as the IP restriction does not work even after waiting for over few hour.

If I allow all sources in the VPC-SC ingress policy without giving access level, I can connect from any IP.

Therefore, I believe that the access level is not working properly.

[pfilourenco]

This always worked for me, if you removed the location match you have the same problem? 

[RyoYoneda]

I have the same problem even after removing the location

I am using the web console to set it up.
Do I need to use the cloud shell for it to work properly?

[sambit_h]

I Have the same problem I am struggling to find the issue since past few days.

[Bryce_June]

Hi Kassy,

Hope so you're doing well, if you are experiencing issues with accessing resources using the IP address filter in Access Context Manager and VPC Service Control, there are a few areas you can check to troubleshoot the problem.

1. Verify IP address filter configuration: Double-check your IP address filter configuration in Access Context Manager. Ensure that you have correctly set the desired IP address ranges and that there are no typos or mistakes in the configuration.

2. Confirm resource association: Ensure that the resources you are trying to access are correctly associated with the IP address filter. Check that the resources are added to the corresponding access level that includes the IP address filter.

3. Validate VPC Service Control configuration: If you are using VPC Service Control, review your VPC Service Control configuration to confirm that it is correctly set up. Check that the services and service perimeters are properly configured and that the IP address filter is associated with the appropriate service perimeter.

4. Review firewall rules: If you have firewall rules in place, check if they are blocking the incoming traffic from the filtered IP addresses. Adjust the firewall rules as necessary to allow access from the filtered IP addresses.

5. Check for network connectivity issues: Ensure that there are no network connectivity issues between the client requesting access and the resources. Verify that the network configurations, routing, and firewalls within your network environment are properly set up to allow communication.

If you have checked these areas and are still experiencing difficulties, it may be helpful to consult the documentation or support resources specific to Access Context Manager and VPC Service Control for further guidance.

Hope so this will work for you!

Regards,

Bryce June

[harinsan]

@Kassy300 @sambit_h  @RyoYoneda were you guys able to resolve this? I am having same issue. If I add access level  (which is my location and IP address range) as condition to ingress, I am unable to access the protected APIs even though my IP and location match the "true" condition for access level.

Even after setting the IP filter to the same IP as the public IP and giving the VPC-SC access level as per the official documentation below, I still get the log "NO_MATCHING_ACCESS_LEVEL" and cannot connect from the public IP.

If I remove the access level from Ingress rule and make it all sources then  it works. I mean other condition i.e Identity is verified as expected.

[RyoYoneda]

@harinsan 

Not resolved yet.

[pfilourenco]

Hi,

If you remove the location, you have the same behavior? 

BR,

Pedro Lourenço

[harinsan]

yes same behavior, be it any matching condition in access level, it doesn't validate. Always get  "NO_MATCHING_ACCESS_LEVEL"  in logs

[Kassy300]

Thank you guys,
Actually the problem was solved by using "default" allow policy in IAM.

Maybe something was wrong in allow policy. We could not estimate detail but I hope it helps.

Best, 
Kassy 

[harinsan]

Thanks @Kassy300 . Can you please elaborate a bit more on the steps you did to resolve and if possible post a snapshot of the change. Will be of great Help!

thanks,

[AlexKhaerov]

I'm currently having a problem with my ingress policy. Every time I set the access level using specific IP subnets or geographical location, I consistently receive a 403 error with a message "NO_MATCHING_ACCESS_LEVEL".

Interestingly, when I alter the access level to allow 'any source', the system functions as expected. This leads me to believe the issue is rooted in the specifics of defining access levels.

Any insights or assistance would be greatly appreciated.

[AlexKhaerov]

I've submitted a support ticket to GCP support, but I'm still struggling with their first-line team who seem to rely heavily on templated answers.

[sambit_h]

The problem seems like access policy as @Kassy300 pointed out. It works with both scoped and un-scoped access policy, however to keep it simple try with un-scoped default org access policy (do not include project while creating access policy in the first step) and create the perimeter at an org level instead of project level ( the top dropdown to select project/org).

[RyoYoneda]

It works with both scoped and un-scoped access policy, however to keep it simple try with un-scoped default org access policy (do not include project while creating access policy in the first step) and create the perimeter at an org level instead of project level ( the top dropdown to select project/org).

VPC-SC now works!

thanks

[harinsan]

for un-scoped accesspolicy it works seamlessly. Thanks all for your replies.

FYI you are  interested please check my blogs:

https://itnext.io/implement-gcp-vpc-service-controls-using-terraform-terragrunt-72c6c046b66b

https://harinderjitss.medium.com/testing-gcp-vpc-service-controls-with-shared-vpc-network-55b4a76bde...

[AlexKhaerov]

Symptoms:

Error: Error when reading or editing BigQuery service account not found: googleapi: Error 403: VPC Service Controls:

Request is prohibited by organizations policy. vpoServiceControlsUniqueldentifier: XXX policyViolation

Environment and fix:

If you encounter a "NO_MATCHING_ACCESS_LEVEL" /Error 403: VPC Service Controls: Request is  prohibited by organizations policy 

error when using your VPC Service Control Perimeter at the project or folder level (also known as scoped level) and you're utilizing an access level (IP Subnet matсhing / etc)with your ingress and egress SC policies, follow these steps to address the issue:

1. Go to org level (! important)

2. Proceed to the VPC Service Controls and click "manage policies"

3. Manually create a new access policy at the organization level, label this as the "default policy" and leave the scope empty, click "create access policy"

4. Once this is done, your existing scoped policies should operate without further errors.

Detials and documentation:

You can infer this particular nuance indirectly from the information provided here: https://cloud.google.com/access-context-manager/docs/create-access-policy.

Under the "Warning" section, it states: "If there is no organization-level access policy in existence for your organization, any scoped policies you create at the folder or project-level will fail to function."

Moreover, when working with the Google Cloud Platform's console (UI), a "default policy" is automatically created for you.

As noted in the resource, "When you create an access level, a default access policy is automatically generated. No further manual intervention is needed."

I find it disappointing that the description and default behavior of the VPC Service Controls feature in GCP are lacking clarity and exhibit unconventional characteristics. Such observations may suggest potential areas for improvement in terms of the maturity, development quality, and overall user experience of VPC Service Controls. Moreover, I have encountered a reliance on templated responses from the official support, which further contributes to the sense of dissatisfaction.

Hit the "Like" button on this post if it saved you a day's worth of work! (It took me 4 hours to figure it out!)