As the number of versions of a secret in Secrets Manager increases, the API calls to request the secret value become increasingly slower. When the number of versions is >75, the API requires >10 seconds, which can result in timeout errors for other parts of an application.
It seems like a similar situation to logs for Batch: the service does not scale.