René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Queries currently fail on large datasets","articleBody":"Queries currently fail on large datasets (\u003e6000 entities). [1] We\nhave many cases where we have well over 30K entities of a given Model that\nwe'd like the ability to iterate over.\n\n[1] Code snippet \u0026 traceback:\n\n``` python\n\u003e\u003e\u003e query = dataset.query().kind('Account')\n\u003e\u003e\u003e print datetime.datetime.utcnow(); all_accts = query.fetch(6000); print\ndatetime.datetime.utcnow()\n\n2014-10-02 09:27:55.235381\n2014-10-02 09:28:54.029017\n\n\u003e\u003e\u003e print datetime.datetime.utcnow(); all_accts = query.fetch(7000); print\ndatetime.datetime.utcnow()\n\n2014-10-02 09:29:01.298517\nTraceback (most recent call last):\nFile \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\nFile\n\"/Users/ehiggins/src/datastore/.env/lib/python2.7/site-packages/gcloud-0.02.2-py2.7.egg/gcloud/datastore/query.py\",\nline 307, in fetch\nquery_pb=clone.to_protobuf(), dataset_id=self.dataset().id())\nFile\n\"/Users/ehiggins/src/datastore/.env/lib/python2.7/site-packages/gcloud-0.02.2-py2.7.egg/gcloud/datastore/connection.py\",\nline 207, in run_query\ndatastore_pb.RunQueryResponse)\nFile\n\"/Users/ehiggins/src/datastore/.env/lib/python2.7/site-packages/gcloud-0.02.2-py2.7.egg/gcloud/datastore/connection.py\",\nline 64, in _rpc\ndata=request_pb.SerializeToString())\nFile\n\"/Users/ehiggins/src/datastore/.env/lib/python2.7/site-packages/gcloud-0.02.2-py2.7.egg/gcloud/datastore/connection.py\",\nline 58, in _request\nraise Exception('Request failed. Error was: %s' % content)\nException: Request failed. Error was: Internal Error\n```\n","author":{"url":"https://github.com/feczo","@type":"Person","name":"feczo"},"datePublished":"2014-10-14T00:10:40.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":21},"url":"https://github.com/242/google-cloud-python/issues/242"}