René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Error in unit test of ci-build section","articleBody":"Hi I've followed the guide and I've gone over it twice now and I don't know where I messed up. I can run the \"Create a Build IaC Pipeline\" just fine and I get no errors. But when I run the \"azdo-ci-build-train.yml\" script it fails on the unit test and subsequent \"Publish linting and unit test\".\r\nWhat do I need to do? I really want to complete this tutorial.\r\n\r\nI'm on the Free Trial Sub for the next 33 days, but I'm not sure if I have to sign up for the full services for this to work.\r\nIn the guide it states the following below (Note). Does that mean I need to sign up for the Azure AD P1/P2 service?\r\n\r\nWhen I go to Sign-ins in the Azure portal I see: \r\nAccess denied\r\nYou do not have access\r\nTo see sign-in data, upgrade your organization's subscription to include Azure AD P1 or P2. Your current license status: Azure AD Free\r\nStart a free Premium Trial\r\n\r\n\u003e Note: You **must have sufficient permissions to register an application with your Azure AD tenant**, and assign the application to a role in your Azure subscription. Contact your subscription administrator if you don't have the permissions. Normally a subscription admin can create a Service principal and can provide you the details.\r\n\r\n\r\n`##[section]Starting: Run unit tests\r\n==============================================================================\r\nTask         : Command line\r\nDescription  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows\r\nVersion      : 2.151.2\r\nAuthor       : Microsoft Corporation\r\nHelp         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line\r\n==============================================================================\r\nGenerating script.\r\nScript contents:\r\npytest --junitxml=/__w/1/b/unit-testresults.xml /__w/1/s/tests/unit\r\n========================== Starting Command Output ===========================\r\n[command]/bin/bash --noprofile --norc /__w/_temp/50f20f82-d423-44ee-9ba0-8d5181b2cc57.sh\r\n============================= test session starts ==============================\r\nplatform linux -- Python 3.7.4, pytest-4.3.0, py-1.8.0, pluggy-0.13.0\r\nrootdir: /__w/1/s, inifile:\r\ncollected 5 items\r\n\r\ntests/unit/code_test.py F                                                [ 20%]\r\ntests/unit/data_test.py ....                                             [100%]\r\n\r\n=================================== FAILURES ===================================\r\n______________________________ test_get_workspace ______________________________\r\n\r\nself = \u003cmsrestazure.azure_active_directory.AdalAuthentication object at 0x7f4d536acb50\u003e\r\nsession = \u003crequests.sessions.Session object at 0x7f4d536acb10\u003e\r\n\r\n    def signed_session(self, session=None):\r\n        \"\"\"Create requests session with any required auth headers applied.\r\n    \r\n        If a session object is provided, configure it directly. Otherwise,\r\n        create a new session and return it.\r\n    \r\n        :param session: The session to configure for authentication\r\n        :type session: requests.Session\r\n        :rtype: requests.Session\r\n        \"\"\"\r\n        session = super(AdalAuthentication, self).signed_session(session)\r\n    \r\n        try:\r\n\u003e           raw_token = self._adal_method(*self._args, **self._kwargs)\r\n\r\n/usr/local/lib/python3.7/site-packages/msrestazure/azure_active_directory.py:448: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.authentication_context.AuthenticationContext object at 0x7f4d536ac950\u003e\r\nresource = 'https://management.core.windows.net/', client_id = ''\r\nclient_secret = ''\r\n\r\n    def acquire_token_with_client_credentials(self, resource, client_id, client_secret):\r\n        '''Gets a token for a given resource via client credentials.\r\n    \r\n        :param str resource: A URI that identifies the resource for which the\r\n            token is valid.\r\n        :param str client_id: The OAuth client id of the calling application.\r\n        :param str client_secret: The OAuth client secret of the calling application.\r\n        :returns: dict with several keys, include \"accessToken\".\r\n        '''\r\n        def token_func(self):\r\n            token_request = TokenRequest(self._call_context, self, client_id, resource)\r\n            return token_request.get_token_with_client_credentials(client_secret)\r\n    \r\n\u003e       return self._acquire_token(token_func)\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/authentication_context.py:179: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.authentication_context.AuthenticationContext object at 0x7f4d536ac950\u003e\r\ntoken_func = \u003cfunction AuthenticationContext.acquire_token_with_client_credentials.\u003clocals\u003e.token_func at 0x7f4d536a9680\u003e\r\ncorrelation_id = None\r\n\r\n    def _acquire_token(self, token_func, correlation_id=None):\r\n        self._call_context['log_context'] = log.create_log_context(\r\n            correlation_id or self.correlation_id, self._call_context.get('enable_pii', False))\r\n        self.authority.validate(self._call_context)\r\n\u003e       return token_func(self)\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/authentication_context.py:128: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.authentication_context.AuthenticationContext object at 0x7f4d536ac950\u003e\r\n\r\n    def token_func(self):\r\n        token_request = TokenRequest(self._call_context, self, client_id, resource)\r\n\u003e       return token_request.get_token_with_client_credentials(client_secret)\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/authentication_context.py:177: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.token_request.TokenRequest object at 0x7f4d536aed90\u003e\r\nclient_secret = ''\r\n\r\n    def get_token_with_client_credentials(self, client_secret):\r\n        self._log.debug(\"Getting token with client credentials.\")\r\n        try:\r\n            token = self._find_token_from_cache()\r\n            if token:\r\n                return token\r\n        except AdalError:\r\n            self._log.exception('Attempt to look for token in cache resulted in Error')\r\n    \r\n        oauth_parameters = self._create_oauth_parameters(OAUTH2_GRANT_TYPE.CLIENT_CREDENTIALS)\r\n        oauth_parameters[OAUTH2_PARAMETERS.CLIENT_SECRET] = client_secret\r\n    \r\n\u003e       token = self._oauth_get_token(oauth_parameters)\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/token_request.py:310: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.token_request.TokenRequest object at 0x7f4d536aed90\u003e\r\noauth_parameters = {'client_secret': '', 'grant_type': 'client_credentials', 'resource': 'https://management.core.windows.net/'}\r\n\r\n    def _oauth_get_token(self, oauth_parameters):\r\n        client = self._create_oauth2_client()\r\n\u003e       return client.get_token(oauth_parameters)\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/token_request.py:112: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cadal.oauth2_client.OAuth2Client object at 0x7f4d536aedd0\u003e\r\noauth_parameters = {'client_secret': '', 'grant_type': 'client_credentials', 'resource': 'https://management.core.windows.net/'}\r\n\r\n    def get_token(self, oauth_parameters):\r\n        token_url = self._create_token_url()\r\n        url_encoded_token_request = urlencode(oauth_parameters)\r\n        post_options = util.create_request_options(self, _REQ_OPTION)\r\n    \r\n        operation = \"Get Token\"\r\n    \r\n        try:\r\n            resp = requests.post(token_url.geturl(),\r\n                                 data=url_encoded_token_request,\r\n                                 headers=post_options['headers'],\r\n                                 verify=self._call_context.get('verify_ssl', None),\r\n                                 proxies=self._call_context.get('proxies', None),\r\n                                 timeout=self._call_context.get('timeout', None))\r\n    \r\n            util.log_return_correlation_id(self._log, operation, resp)\r\n        except Exception:\r\n            self._log.exception(\"%(operation)s request failed\", {\"operation\": operation})\r\n            raise\r\n    \r\n        if util.is_http_success(resp.status_code):\r\n            return self._handle_get_token_response(resp.text)\r\n        else:\r\n            if resp.status_code == 429:\r\n                resp.raise_for_status()  # Will raise requests.exceptions.HTTPError\r\n            return_error_string = _ERROR_TEMPLATE.format(operation, resp.status_code)\r\n            error_response = \"\"\r\n            if resp.text:\r\n                return_error_string = u\"{} and server response: {}\".format(return_error_string,\r\n                                                                           resp.text)\r\n                try:\r\n                    error_response = resp.json()\r\n                except ValueError:\r\n                    pass\r\n\u003e           raise AdalError(return_error_string, error_response)\r\nE           adal.adal_error.AdalError: Get Token request returned http error: 404\r\n\r\n/usr/local/lib/python3.7/site-packages/adal/oauth2_client.py:289: AdalError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nname = 'RHML-AML-WS', resource_group = 'RHML-AML-RG', subscription_id = ''\r\ntenant_id = '', app_id = '', app_secret = ''\r\n\r\n    def get_workspace(\r\n            name: str,\r\n            resource_group: str,\r\n            subscription_id: str,\r\n            tenant_id: str,\r\n            app_id: str,\r\n            app_secret: str):\r\n        service_principal = ServicePrincipalAuthentication(\r\n            tenant_id=tenant_id,\r\n            service_principal_id=app_id,\r\n            service_principal_password=app_secret)\r\n    \r\n        try:\r\n            aml_workspace = Workspace.get(\r\n                name=name,\r\n                subscription_id=subscription_id,\r\n                resource_group=resource_group,\r\n\u003e               auth=service_principal)\r\n\r\nml_service/util/workspace.py:23: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nname = 'RHML-AML-WS'\r\nauth = \u003cazureml.core.authentication.ServicePrincipalAuthentication object at 0x7f4d536ace50\u003e\r\nsubscription_id = '', resource_group = 'RHML-AML-RG'\r\n\r\n    @staticmethod\r\n    def get(name, auth=None, subscription_id=None, resource_group=None):\r\n        \"\"\"Return a workspace object for an existing Azure Machine Learning Workspace.\r\n    \r\n        Throws an exception if the workspace does not exist or the required fields\r\n        do not uniquely identify a workspace.\r\n    \r\n        :param name: The name of the workspace to get.\r\n        :type name: str\r\n        :param auth: The authentication object. For more details refer to https://aka.ms/aml-notebook-auth.\r\n            If None, the default Azure CLI credentials will be used or the API will prompt for credentials.\r\n        :type auth: azureml.core.authentication.ServicePrincipalAuthentication or\r\n            azureml.core.authentication.InteractiveLoginAuthentication\r\n        :param subscription_id: The subscription ID to use.\r\n            The parameter is required if the user has access to more than one subscription.\r\n        :type subscription_id: str\r\n        :param resource_group: The resource group to use.\r\n            If None, the method will search all resource groups in the subscription.\r\n        :type resource_group: str\r\n        :return: The workspace object.\r\n        :rtype: azureml.core.Workspace\r\n        \"\"\"\r\n        if not auth:\r\n            auth = InteractiveLoginAuthentication()\r\n    \r\n        # If everything is specified then we use the get operation, which is faster than the list\r\n        # operation.\r\n        if subscription_id and resource_group:\r\n            return Workspace(subscription_id, resource_group, name, auth=auth)\r\n    \r\n        if not subscription_id:\r\n\u003e           subscription_id = Workspace._fetch_subscription(auth)\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/core/workspace.py:372: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nauth = \u003cazureml.core.authentication.ServicePrincipalAuthentication object at 0x7f4d536ace50\u003e\r\n\r\n    @staticmethod\r\n    def _fetch_subscription(auth):\r\n        \"\"\"Get all subscriptions a user has access to.\r\n    \r\n        If a user has access to only one subscription than that is  returned, otherwise an exception is thrown.\r\n    \r\n        :param auth: The authentication object. For more details refer to https://aka.ms/aml-notebook-auth.\r\n        :type auth: azureml.core.authentication.AbstractAuthentication\r\n        :return: A subscription id\r\n        :rtype: str\r\n        \"\"\"\r\n\u003e       all_subscriptions = auth._get_all_subscription_ids()\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/core/workspace.py:760: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cazureml.core.authentication.ServicePrincipalAuthentication object at 0x7f4d536ace50\u003e\r\n\r\n    def _get_all_subscription_ids(self):\r\n        \"\"\"Return a list of subscriptions that are accessible through this authentication.\r\n    \r\n        :return: Returns a list of SubscriptionInfo named tuples.\r\n        :rtype: list, str\r\n        \"\"\"\r\n\u003e       arm_token = self._get_arm_token()\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/core/authentication.py:842: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cazureml.core.authentication.ServicePrincipalAuthentication object at 0x7f4d536ace50\u003e\r\nargs = (), kwargs = {}, field_name = '_cached_arm_token'\r\n\r\n    def wrapper(self, *args, **kwargs):\r\n        \"\"\"Wrapper.\r\n    \r\n        :param args:\r\n        :type args: list\r\n        :param kwargs:\r\n        :type kwargs: dict\r\n        :return: Returns the test function.\r\n        :rtype: object\r\n        \"\"\"\r\n        field_name = ServicePrincipalAuthentication._token_type_to_field_dict[token_type]\r\n        if self._enable_caching:\r\n            cached_token = getattr(self, field_name)\r\n            if not cached_token or self._is_token_expired(cached_token):\r\n                with ServicePrincipalAuthentication._sp_auth_lock:\r\n                    # Getting it again after acquiring the lock in case some other thread might have updated it.\r\n                    cached_token = getattr(self, field_name)\r\n                    if not cached_token or self._is_token_expired(cached_token):\r\n                        s = time.time()\r\n                        module_logger.debug(\"Calling {} in ServicePrincipalAuthentication \"\r\n                                            \"to get token.\".format(actual_function))\r\n                        new_token = actual_function(self, *args, **kwargs)\r\n                        module_logger.debug(\"{} call completed in {} s\".format(\r\n                            actual_function, (time.time()-s)))\r\n                        setattr(self, field_name, new_token)\r\n                        return new_token\r\n                    else:\r\n                        return cached_token\r\n            else:\r\n                return cached_token\r\n        else:\r\n\u003e           return actual_function(self, *args, **kwargs)\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/core/authentication.py:755: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cazureml.core.authentication.ServicePrincipalAuthentication object at 0x7f4d536ace50\u003e\r\n\r\n    @_sp_auth_caching_decorator(\"ARM_TOKEN\")\r\n    def _get_arm_token(self):\r\n        \"\"\"Return arm access token.\r\n    \r\n        :return: Returns arm token from sp credential object\r\n        :rtype: str\r\n        \"\"\"\r\n\u003e       header = self._get_sp_credential_object().signed_session().headers['Authorization']\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/core/authentication.py:797: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = \u003cmsrestazure.azure_active_directory.AdalAuthentication object at 0x7f4d536acb50\u003e\r\nsession = \u003crequests.sessions.Session object at 0x7f4d536acb10\u003e\r\n\r\n    def signed_session(self, session=None):\r\n        \"\"\"Create requests session with any required auth headers applied.\r\n    \r\n        If a session object is provided, configure it directly. Otherwise,\r\n        create a new session and return it.\r\n    \r\n        :param session: The session to configure for authentication\r\n        :type session: requests.Session\r\n        :rtype: requests.Session\r\n        \"\"\"\r\n        session = super(AdalAuthentication, self).signed_session(session)\r\n    \r\n        try:\r\n            raw_token = self._adal_method(*self._args, **self._kwargs)\r\n        except adal.AdalError as err:\r\n            # pylint: disable=no-member\r\n            if 'AADSTS70008:' in ((getattr(err, 'error_response', None) or {}).get('error_description') or ''):\r\n                raise Expired(\"Credentials have expired due to inactivity.\")\r\n            else:\r\n\u003e               raise AuthenticationError(err)\r\nE               msrest.exceptions.AuthenticationError: Get Token request returned http error: 404\r\n\r\n/usr/local/lib/python3.7/site-packages/msrestazure/azure_active_directory.py:454: AuthenticationError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\n    def test_get_workspace():\r\n        workspace_name = os.environ.get(\"BASE_NAME\")+\"-AML-WS\"\r\n        resource_group = os.environ.get(\"BASE_NAME\")+\"-AML-RG\"\r\n        subscription_id = os.environ.get(\"SUBSCRIPTION_ID\")\r\n        tenant_id = os.environ.get(\"TENANT_ID\")\r\n        app_id = os.environ.get(\"SP_APP_ID\")\r\n        app_secret = os.environ.get(\"SP_APP_SECRET\")\r\n    \r\n        aml_workspace = get_workspace(\r\n            workspace_name,\r\n            resource_group,\r\n            subscription_id,\r\n            tenant_id,\r\n            app_id,\r\n\u003e           app_secret)\r\n\r\ntests/unit/code_test.py:23: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nname = 'RHML-AML-WS', resource_group = 'RHML-AML-RG', subscription_id = ''\r\ntenant_id = '', app_id = '', app_secret = ''\r\n\r\n    def get_workspace(\r\n            name: str,\r\n            resource_group: str,\r\n            subscription_id: str,\r\n            tenant_id: str,\r\n            app_id: str,\r\n            app_secret: str):\r\n        service_principal = ServicePrincipalAuthentication(\r\n            tenant_id=tenant_id,\r\n            service_principal_id=app_id,\r\n            service_principal_password=app_secret)\r\n    \r\n        try:\r\n            aml_workspace = Workspace.get(\r\n                name=name,\r\n                subscription_id=subscription_id,\r\n                resource_group=resource_group,\r\n                auth=service_principal)\r\n    \r\n            return aml_workspace\r\n        except Exception as caught_exception:\r\n            print(\"Error while retrieving Workspace...\")\r\n            print(str(caught_exception))\r\n\u003e           sys.exit(1)\r\nE           SystemExit: 1\r\n\r\nml_service/util/workspace.py:29: SystemExit\r\n----------------------------- Captured stdout call -----------------------------\r\nError while retrieving Workspace...\r\nGet Token request returned http error: 404\r\n=============================== warnings summary ===============================\r\n/usr/local/lib/python3.7/site-packages/socks.py:58\r\n  /usr/local/lib/python3.7/site-packages/socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\r\n    from collections import Callable\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/data/dataset_type_definitions.py:69\r\n  /usr/local/lib/python3.7/site-packages/azureml/data/dataset_type_definitions.py:69: DeprecationWarning: FileType Enum is Deprecated in \u003e 1.0.39. Use strings instead.\r\n    category=DeprecationWarning)\r\n\r\n/usr/local/lib/python3.7/site-packages/azureml/_base_sdk_common/utils.py:11\r\n  /usr/local/lib/python3.7/site-packages/azureml/_base_sdk_common/utils.py:11: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\r\n    from collections import Iterable\r\n\r\n-- Docs: https://docs.pytest.org/en/latest/warnings.html\r\n-------------- generated xml file: /__w/1/b/unit-testresults.xml ---------------\r\n================ 1 failed, 4 passed, 3 warnings in 1.68 seconds ================\r\n##[error]Bash exited with code '1'.\r\n##[section]Finishing: Run unit tests\r\n`\r\n\r\n`##[section]Starting: Publish linting and unit test results\r\n==============================================================================\r\nTask         : Publish Test Results\r\nDescription  : Publish test results to Azure Pipelines\r\nVersion      : 2.158.0\r\nAuthor       : Microsoft Corporation\r\nHelp         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results\r\n==============================================================================\r\nTimestamp is not available for one or more testsuites. Total run duration is being calculated as the sum of time durations of detected testsuites\r\nTimestamp is not available for one or more testsuites. Total run duration is being calculated as the sum of time durations of detected testsuites\r\n##[section]Async Command Start: Publish test results\r\nPublishing test results to test run '22'\r\nTest results remaining: 17. Test run id: 22\r\nPublished Test Run : https://tfsprodcus4.visualstudio.com/Aabd9a587-291d-437a-9967-95c61ab46f08/RH-Project/_TestManagement/Runs?runId=22\u0026_a=runCharts\r\nPublishing test results to test run '24'\r\nTest results remaining: 5. Test run id: 24\r\nPublished Test Run : https://tfsprodcus4.visualstudio.com/Aabd9a587-291d-437a-9967-95c61ab46f08/RH-Project/_TestManagement/Runs?runId=24\u0026_a=runCharts\r\n##[error]There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.\r\n##[section]Async Command End: Publish test results\r\n##[section]Finishing: Publish linting and unit test results\r\n`","author":{"url":"https://github.com/reuben-hilliard","@type":"Person","name":"reuben-hilliard"},"datePublished":"2019-10-29T03:41:06.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/85/MLOpsPython/issues/85"}