René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"data integrity checks for uploads/downloads","articleBody":"For uploads/downloads, check in order of priority:\n- md5 computed for local file against md5 contained in Object’s metadata (more expensive, more reliable)\n- crc32c computed for local file against crc32c contained in Object’s metadata (less expensive, less reliable)\n\n[eaxmple gslib code for the checks chain](https://github.com/GoogleCloudPlatform/gsutil/blob/ccca18e64f3f7d87c989675f8673af7bdecae0d1/gslib/copy_helper.py#L805)\n\nFor copies of objects already in the storage, compare the checksums in the Object’s metadata with the same priority (md5 could not be available in some cases)\n[example gslib_code](https://github.com/GoogleCloudPlatform/gsutil/blob/ccca18e64f3f7d87c989675f8673af7bdecae0d1/gslib/copy_helper.py#L756)\n\nIf the checksum check fails, the created file should be deleted by the client, and an exception raised.\n[example gslib_code](https://github.com/GoogleCloudPlatform/gsutil/blob/ccca18e64f3f7d87c989675f8673af7bdecae0d1/gslib/copy_helper.py#L1694)\n\nWe should allow to override the configuration for the checks (e.g. use only crc32c, do not perform checks), but it should be strongly discouraged.\n","author":{"url":"https://github.com/silvolu","@type":"Person","name":"silvolu"},"datePublished":"2015-01-14T19:33:45.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/547/google-cloud-python/issues/547"}