René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"How do I write code that talks to two different datasets with two different sets of credentials?","articleBody":"Let's say I have:\n- `/creds1.json`\n- `/creds2.json`\n- `dataset1`\n- `dataset2`\n- `gcloud.datastore`\n\nHow do I pull down `Person:1` from `dataset1`, retrieve the `'name'` property, and write it back to `Log:2` in `dataset2`?\n\nHere's my best guess so far:\n\n``` python\nfrom gcloud import credentials\nfrom gcloud import datastore\n\ncreds1 = credentials.get_for_service_account_json('/creds1.json')\ncreds2 = credentials.get_for_service_account_json('/creds2.json')\n\nconnection1 = datastore.Connection(credentials=creds1)\nconnection2 = datastore.Connection(credentials=creds1)\n\nperson1_key = datastore.Key('Person', 1, dataset_id='dataset1')\nlog2_key = datastore.Key('Log', 2, dataset_id='dataset2')\n\nperson1 = datastore.get(datastore.Key(person1_key), connection=connection1, dataset_id='dataset1')\nlog2 = datastore.get(datastore.Key(log2_key), connection=connection2, dataset_id='dataset2')\n\nlog2['data'] = person1['name']\ndatastore.put(log2, connection=connection2, dataset_id='dataset2')\n```\n\nI only got that by digging through tons of code. It made me sad.\n\nWhat I want to write:\n\n``` python\nfrom gcloud.datastore import get_connection\n\ndataset1 = get_connection(credentials_json='/creds1.json').get_dataset('dataset1')\ndataset2 = get_connection(credentials_json='/creds2.json').get_dataset('dataset2')\n\nperson1 = dataset1.get('Person', 1)\nlog2 = dataset2.get('Log', 2)\n\nlog2['data'] = person1['name']\nlog2.put()\n```\n\nIt seems that somewhere along the way, we lost the \"hierarchy\" of high-level concepts (Datastore -\u003e Connection -\u003e Dataset -\u003e Entity) so that things don't seem to know who their \"parent\" is in the tree on the way up.\n\nThis means things like `dataset.get()` because it needs to be provided it's connection and credentials. It seems we've tried to overcome this by storing defaults, but that blows up when you have more than one set of credentials...\n\nMaybe I'm totally misunderstanding?\n\n---\n\nI'm thinking that it'd be cool if we could allow three things:\n1. datastore.\u003cmethod\u003e that accepts _all_ the parameters to be absurdly specific (here is the connection, here is the dataset_it, etc)\n2. datastore.\u003cmethod\u003e that has lots of `None`s as default parameters, and we \"go get the default\" if you left it as `None` (ie, `connection=None -\u003e get_default_connection()`)\n3. Datastore -\u003e Connection -\u003e Dataset -\u003e Entity drill down that \"pre-fills\" these things going up the chain. That is, I can say:\n\n``` python\nconnection = datastore.get_connection(...)\ndataset = connection.get_dataset(...)\nentity = dataset.get(...)\n```\n\nThis means that the `datastore` module, `Connection`, and `Dataset` all would likely have the same methods, just with fewer things you can specify because some of those fieldsare specified when you \"ask for\" the next level down (ie, a Dataset knows it's `dataset_id`, so you don't have an option to provide that) .\n\n---\n### Example\n#### datastore.py:\n\n``` python\ndef get(key, connection=None, dataset_id=None):\n  connection = connection or get_default_connection()\n  dataset_id = dataset_id or get_default_dataset_id()\n\n# Means I can do: datastore.get(Key('Person', 1), dataset_id='dataset1')\n```\n#### dataset.py:\n\n``` python\nclass Dataset(object):\n\n  def __init__(self, dataset_id, connection=None):\n    self.dataset_id = dataset_id\n    self.connection = connection\n\n  def get(self, key):\n    return datastore.get(key, dataset_id=self.dataset_id, connection=self.connection)\n```\n\n/cc @pcostell @dhermes @tseaver \n","author":{"url":"https://github.com/jgeewax","@type":"Person","name":"jgeewax"},"datePublished":"2015-02-17T16:52:04.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":6},"url":"https://github.com/659/google-cloud-python/issues/659"}