René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"dashes in word_count.txt cause errors with WordCount.py","articleBody":"### Issue:\r\nThe`ndash` characters in `word_count.txt` cause an error when following the \"Run your first Spark Job\" tutorial. There are only two occurences of this character here: \"`from 1913–74.`\" and here: \"`near–bankruptcy`\".\r\n\r\n#### To Recreate:\r\nusing `spark-2.3.2-bin-hadoop2.7` on Ubuntu18, pyspark/python 2.7, Installed following instructions from lecture 5, go to directory where you cloned `python-spark-tutorial` and run the following from lecture 6:\r\n\r\n`spark-submit ./rdd/WordCount.py`\r\n\r\nThe execution halts about halfway through the frequency counter with the following error:\r\n\r\n```\r\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\u2013' in position4: ordinal not in range(128)\r\n```\r\nSpoiler, it's the dash. I'm not sure whether or not the utf16 dash was intentional, so I'm posting. \r\n\r\n#### Work-Around:\r\n\r\nI changed the two `ndash` characters to  \"`from 1913-74.`\" and \"`near-bankruptcy`\", which solved the issue for me. Related [stackoverflow thread](https://stackoverflow.com/questions/20329896/python-2-7-character-u2013) where someone else ran into a similar problem with python2.7 and used the same solution.\r\n","author":{"url":"https://github.com/HarryCaveMan","@type":"Person","name":"HarryCaveMan"},"datePublished":"2018-11-04T05:16:14.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/12/python-spark-tutorial/issues/12"}