René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"'NoneType' object is not subscriptable","articleBody":"## Error\n\nFrom 2025-10-24, I've noticed I was getting the following error:\n\n```shell\n[i] Logging into Substack...\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): substack.com:443\nDEBUG:urllib3.connectionpool:https://substack.com:443 \"GET /api/v1/user/profile/self HTTP/1.1\" 200 None\n[-] An unexpected error occurred: 'NoneType' object is not subscriptable\n```\n\n## Code\n\nThis is my code which is in use:\n\n```python\nprint(\"[i] Logging into Substack...\")\ntry:\n    api = Api(\n        debug=logging.getLogger().isEnabledFor(logging.DEBUG),\n        publication_url=os.getenv(\"PUBLICATION_URL\"),\n        cookies_path=cookie,\n        email=email,\n        password=password\n    )\n    print(\"[+] Logged into Substack!\")\nexcept Exception as e:\n    print(f\"[-] An unexpected error occurred: {e}\")\n    sys.exit(1)\n```\n\n## API (Today)\n\nDigging into the API response as of today (2025-12-18):\n\n```console\n$ curl \\\n    --silent \\\n    --header 'Cookie: substack.sid=[REMOVED]' \\\n   'https://substack.com/api/v1/user/profile/self' \\\n  | jq '.. | objects | keys'\n[\n  \"bio\",\n  \"can_dm\",\n  \"facebookAccount\",\n  \"followerCount\",\n  \"followsViewer\",\n  \"handle\",\n  \"hasActivity\",\n  \"hasGuestPost\",\n  \"hasHiddenPublicationUsers\",\n  \"hasHiddenSubscriptions\",\n  \"hasLikes\",\n  \"id\",\n  \"isFollowing\",\n  \"isSubscribed\",\n  \"lists\",\n  \"max_pub_tier\",\n  \"name\",\n  \"photo_url\",\n  \"previousSlug\",\n  \"previous_name\",\n  \"primaryPublicationSubscriptionState\",\n  \"profile_disabled\",\n  \"profile_set_up_at\",\n  \"publicationUsers\",\n  \"reader_installed_at\",\n  \"slug\",\n  \"status\",\n  \"subscriberCount\",\n  \"subscriberCountNumber\",\n  \"subscriberCountString\",\n  \"subscribesToViewerSubdomain\",\n  \"subscriptions\",\n  \"subscriptionsTruncated\",\n  \"theme\",\n  \"tos_accepted_at\",\n  \"twitterAccount\",\n  \"userLinks\",\n  \"visibleSubscriptionsCount\"\n]\n[\n  \"id\",\n  \"label\",\n  \"type\",\n  \"url\",\n  \"value\"\n]\n[\n  \"id\",\n  \"label\",\n  \"type\",\n  \"url\",\n  \"value\"\n]\n[\n  \"id\",\n  \"label\",\n  \"type\",\n  \"url\",\n  \"value\"\n]\n[]\n[\n  \"badge\",\n  \"bestsellerTier\",\n  \"leaderboard\",\n  \"paidPublicationIds\",\n  \"subscriber\",\n  \"subscriberTier\",\n  \"vip\"\n]\n$\n```\n\n## API (Previously)\n\nLooking at [NHagar/substack_ap](https://github.com/NHagar/substack_api/blob/master/examples/usage_walkthrough.ipynb) ([WayBackMachine](https://web.archive.org/web/20251218122222/https://github.com/NHagar/substack_api/blob/master/examples/usage_walkthrough.ipynb)), was able to see a able to see the historical version of the API response:\n\n```console\n$ cat /tmp/usage_walkthrough.json | jq 'keys'\n[\n  \"bestseller_badge_disabled\",\n  \"bestseller_tier\",\n  \"bio\",\n  \"followsViewer\",\n  \"handle\",\n  \"hasActivity\",\n  \"hasGuestPost\",\n  \"hasHiddenPublicationUsers\",\n  \"hasLikes\",\n  \"id\",\n  \"isFollowing\",\n  \"isPersonalEligible\",\n  \"isSubscribed\",\n  \"lists\",\n  \"max_pub_tier\",\n  \"name\",\n  \"photo_url\",\n  \"previousSlug\",\n  \"previous_name\",\n  \"primaryPublication\",\n  \"profile_disabled\",\n  \"profile_set_up_at\",\n  \"publicationUsers\",\n  \"rough_num_free_subscribers\",\n  \"rough_num_free_subscribers_int\",\n  \"slug\",\n  \"subdomainUrl\",\n  \"subscriberCount\",\n  \"subscriberCountNumber\",\n  \"subscriberCountString\",\n  \"subscriptions\",\n  \"subscriptionsTruncated\",\n  \"tos_accepted_at\",\n  \"userLinks\",\n  \"visibleSubscriptionsCount\"\n]\n$\n```\n\nConverted JSON: [usage_walkthrough.json](https://github.com/user-attachments/files/24237139/usage_walkthrough.json)\n\n## Compare APIs\n\n```console\n$ cat /tmp/usage_walkthrough.json| jq 'keys' \u003e usage_walkthrough-keys\n$ \n$ curl \\\n    --silent \\\n    --header 'Cookie: substack.sid=[REMOVED]' \\\n   'https://substack.com/api/v1/user/profile/self' | jq 'key \u003e self-keys\n$ \n$ diff self-keys usage_walkthrough-keys\n1a2,3\n\u003e   \"bestseller_badge_disabled\",\n\u003e   \"bestseller_tier\",\n3,5d4\n\u003c   \"can_dm\",\n\u003c   \"facebookAccount\",\n\u003c   \"followerCount\",\n11d9\n\u003c   \"hasHiddenSubscriptions\",\n14a13\n\u003e   \"isPersonalEligible\",\n22c21\n\u003c   \"primaryPublicationSubscriptionState\",\n---\n\u003e   \"primaryPublication\",\n26c25,26\n\u003c   \"reader_installed_at\",\n---\n\u003e   \"rough_num_free_subscribers\",\n\u003e   \"rough_num_free_subscribers_int\",\n28c28\n\u003c   \"status\",\n---\n\u003e   \"subdomainUrl\",\n32d31\n\u003c   \"subscribesToViewerSubdomain\",\n35d33\n\u003c   \"theme\",\n37d34\n\u003c   \"twitterAccount\",\n$\n```\n\n## Following the error\n\nLooking for `/api/v1/user/profile/self`:\n\n```console\n$ git clone https://github.com/ma2za/python-substack\n[...]\n$\n$ cd ./python-substack/\n$\n$ grep -Rn '/api/v1/user/profile/self'\n$\n$ grep -Rn '/self'\n./substack/api.py:234:        response = self._session.get(f\"{self.base_url}/user/profile/self\")\n$\n$ grep -Rn -C 5 '/self'\n./substack/api.py-229-\n./substack/api.py-230-    def get_user_profile(self):\n./substack/api.py-231-        \"\"\"\n./substack/api.py-232-        Gets the users profile\n./substack/api.py-233-        \"\"\"\n./substack/api.py:234:        response = self._session.get(f\"{self.base_url}/user/profile/self\")\n./substack/api.py-235-\n./substack/api.py-236-        return Api._handle_response(response=response)\n./substack/api.py-237-\n./substack/api.py-238-    def get_user_settings(self):\n./substack/api.py-239-        \"\"\"\n```\n\nCan see that `./substack/api.py` has `get_user_profile()`.\n\n- - - \n\n```console\n$ grep -n get_user_profile ./substack/api.py | wc -l\n       4\n$\n$ grep -n get_user_profile ./substack/api.py\n179:        profile = self.get_user_profile()\n205:        profile = self.get_user_profile()\n218:        profile = self.get_user_profile()\n230:    def get_user_profile(self):\n$\n```\n\nAnd its only used called in 3 places.\n\n```console\n$ grep -nB 6 self.get_user_profile ./substack/api.py\n173-    def get_user_id(self):\n174-        \"\"\"\n175-\n176-        Returns:\n177-\n178-        \"\"\"\n179:        profile = self.get_user_profile()\n--\n199-\n200-    def get_user_primary_publication(self):\n201-        \"\"\"\n202-        Gets the users primary publication\n203-        \"\"\"\n204-\n205:        profile = self.get_user_profile()\n--\n212-\n213-    def get_user_publications(self):\n214-        \"\"\"\n215-        Gets the users publications\n216-        \"\"\"\n217-\n218:        profile = self.get_user_profile()\n$ \n```\n\n- `get_user_id()`\n- `get_user_primary_publication()`\n- `get_user_publications()`\n\n### get_user_id()\n\n```console\n$ grep get_user_id ./substack/api.py\n    def get_user_id(self):\n$\n```\n\n...As our code doesn't use `get_user_id`, its not going to be this\n\n### get_user_primary_publication() / get_user_publications()\n\n```console\n$ grep self.get_user_primary_publication ./substack/api.py\n            user_publication = self.get_user_primary_publication()\n$ \n$ grep self.get_user_publications ./substack/api.py\n            user_publications = self.get_user_publications()\n$ \n```\n\nSo let's see where these are being called:\n\n```python\nclass Api:\n[...]\n    def __init__(\n[...]\n        # if the user provided a publication url, then use that\n        if publication_url:\n[...]\n            user_publications = self.get_user_publications()\n[...]\n        else:\n            # get the users primary publication\n            user_publication = self.get_user_primary_publication()\n```\n\nBingo! This starts to match up whats in the code (right at the top)\nSo using `Api()`, if `publication_url` is set, `get_user_publications()` is called, Otherwise its `get_user_primary_publication()`\n_For what its worth, `publication_url` is set for me_\n\n### get_user_publications()\n\nLooking at the function:\n\n```python\n    def get_user_publications(self):\n        \"\"\"\n        Gets the users publications\n        \"\"\"\n\n        profile = self.get_user_profile()\n\n        # Loop through users \"publicationUsers\" list, and return a list\n        # of dictionaries of \"name\", and \"subdomain\", and \"id\"\n        user_publications = []\n        for publication in profile[\"publicationUsers\"]:\n[...]\n```\n\nIts expecting `publicationUsers` in the API response back....\n**It USED to be there, but its not anymore!**\n\n### get_user_primary_publication()\n\nLooking at the function:\n\n```python\n    def get_user_primary_publication(self):\n        \"\"\"\n        Gets the users primary publication\n        \"\"\"\n\n        profile = self.get_user_profile()\n        primary_publication = profile[\"primaryPublication\"]\n[...]\n```\n\nIts expecting `primaryPublication` in the API response back....\n**Again It USED to be there, but its not anymore!**\n\nThis may start to explain why its failing....\n","author":{"url":"https://github.com/g0tmi1k","@type":"Person","name":"g0tmi1k"},"datePublished":"2025-12-18T14:28:42.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/33/python-substack/issues/33"}