Code Challenge 07 - Twitter Sentiment Analysis - Review

PyBites, Sun 26 February 2017, Challenges

code review, codechallenges, learning, sentiment, TextBlob, Twitter

It's end of the week again so we review the code challenge of this week. It's never late to join, just fork our challenges repo and start coding.

Possible solution / learning

Getting the data

First we let the data gathering script we provided run for 5 days and 10 hours. As promised we would do the analysis on 50 shades of darker:

$ nohup python getting_data.py 50 shades darker &

It ran Mon Feb 20 09:13 - Sat Feb 25 19:25 and collected > 10K tweets, storing them in data_1487581986.json

We kept it simple, question to be answered:

Is there overly positive or negative talk about 50 shades of darker on Twitter?

Enter TextBlob

TextBlob is a Python (2 and 3) library for processing textual data. It provides a simple API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more.

It makes sentiment analysis very easy.

Code

Our script is here. What we did:

The verdict

People talk mostly positive about the movie:

$ time python sentiment.py data_1487581986.json
Analyzed 10053 tweets
Positive: 33.85%
Negative: 13.86%
Neutral: 52.29%

Would be nice to break this down further, maybe by region. We wanted to look at gender, but this data was not provided by the Twitter API.

Some anonymous examples

We peaked at some tweets for validation. As you see it's not 100% correct (doubts marked with '?'), but it does give you a good indication:

neutral
-- 50 shades darker though.. oouuu. πŸ™†πŸ½
-- ❀🀴🏼❀ β€” watching 50 shades darker...
-- still don't know what to think of 50 shades darkerπŸ€”πŸ€”
---

positive
-- can't get over how amazing 50 shades darker is 😍
-- i enjoyed 50 shades darker very much
?-- that 50 shades darker was kind of sorry
---

negative
?-- still haven't seen 50 shades darker :(  
-- 50 shades darker might be the worst movie i've ever seen ...
-- 50 shades darker was terrible god i wish i could get those 2 hours back
---

Further reading

This great article by Real Python shows another Twitter Sentiment Analysis example adding Docker, Elasticsearch, Kibana to the mix.

Feedback

What was your solution? Feel free to share in the comments below.

We hope you enjoy these challenges. Please provide us feedback if we can improve anything ...

If you have an interesting challenge you want us to feature, don't hesitate to reach out to us.

PyBites Python Tips

Do you want to get 250+ concise and applicable Python tips in an ebook that will cost you less than 10 bucks (future updates included), check it out here.

Get our Python Tips Book

"The discussions are succinct yet thorough enough to give you a solid grasp of the particular problem. I just wish I would have had this book when I started learning Python." - Daniel H

"Bob and Julian are the masters at aggregating these small snippets of code that can really make certain aspects of coding easier." - Jesse B

"This is now my favourite first Python go-to reference." - Anthony L

"Do you ever go on one of those cooking websites for a recipe and have to scroll for what feels like an eternity to get to the ingredients and the 4 steps the recipe actually takes? This is the opposite of that." - Sergio S

Get the book