Is my MEP coherent?

Alessandro Piscopo, Luis-Daniel Ibáñez, Pavlos Vougiouklis and Nikolaos Liappas

The European Parliament (EP) is the institution in charge of the legislative function of the EU. To help citizens understand whether institutional acts of the EP members are consistent with their public discourse and whether they protect the interests of the constituency in which they were elected, we propose to:

  1. Link MEP’s speeches in the EP to tweets and Facebook posts published on their social accounts to concepts, using techniques applied in previous creative camps;
  2. Compute values of the sentiment speeches express towards a concept, using state of the art sentiment analysis techniques;
  3. Link EP speeches to the geographical places mentioned in them, i.e. to IRIs in GeoNames, using state of the art entity recognition techniques.

These linkages would enable an assessment of the coherency between political action and public discourse of MEPs, by connecting their institutional behaviour and their social media declarations based on both the concepts covered and the places mentioned, according to the related sentiment

Data gathered

We wanted to be enable the connection of contemporaneous MEPs’ speeches and social media actions: since the LinkedEP dataset includes all plenary debates in the EP between July 1999 and January 2014, we excluded the parliamentary term beginning from May 2014, date of the last EP elections and decided to focus on the previous term (July 2009 – May 2014).

Nevertheless, whereas the list of current MEPs and their social media profiles are easily accessible, the first one from the EP website, the latter ones by scraping related portals or websites (EP Twitter and Facebook official pages, the EPnewshub website), obtaining these with regard to the previous term was not so straightforward. Therefore, given the limited time available, we decided to include only MEPs from the UK, whose Twitter and Facebook IDs we collected manually. This choice allowed us to deal only with English texts, avoiding issues derived from processing less common languages, while one of the countries with the highest number of seats (72).

In total, the 2009 – 2014 term British MEPs’ IDs collected were 56 for Twitter and 29 for Facebook. As regards the missing IDs, either the MEPs had no social media profile, or we were unable to find them.

Our available dataset consist of 48,614 tweets in total and 21,082 Facebook posts. Due to the short time available, we were able to process only tweets.

Concept Linking

Our first goal was to connect MEPs’ speeches to their tweets and Facebook posts, through the concepts expressed in them. For this purpose, we took advantage of the already existing annotation of the plenary debates with the EuroVoc thesaurus. This includes several terms, connected either by hierarchical (i.e. broader/narrower) or associative (i.e. related to) relationships.

The first step was to clean the texts in our corpus, by removing web addresses and special characters, and the words in a stopword list – the most common English words, such as articles or prepositions –, to reduce clutter. We used the “Porter et al.” stopword list from the NLTK python package[1].

The first approach considered was a supervised classification, consisting in training a machine learning algorithm on the already annotated debates’ texts and subsequently applying it to the texts in our corpus (the Tweets). However, we discarded it because of the different structure of debates and tweets – the first ones being much longer than the 140 characters limit of tweets.

Therefore, we tried an unsupervised approach based on latent Dirichlet allocation (LDA), which provides as an output a user-defined number of groups of words, each characterising a group of similar observations. The idea underlying the application of this approach was to generate a number of groups and verify afterwards if these could be matched to the higher level concepts in EuroVoc. Nevertheless, hardly any match was found and many of the groups identified could not be referred to any meaningful classification, due to several factors: our corpus was too small; tweets were often shorter than 140 characters and could not be correctly classified; tweets were often related to local events or topics and therefore were hardly connectable to EuroVoc topics.

We decided then to explore a simple approach, based on matching words within tweets with lower level terms in the EuroVoc thesaurus. Every time that a match was found, we assigned a tweet to the domain of the matching term. To make an example, a tweet like “Oil-based policies are not sustainable, go green!” would be assigned to the Energy domain, since the term “oil” belonged to that.

The domains used were: Politics; International relations; European Union; Law; Economics; Trade; Finance; Social questions; Education and communications; Science; Business and competition; Employment and working conditions; Transport; Environment; Agriculture, forestry and fisheries; Agri-foodstuffs; Production, technology and research; Energy; Industry; Geography; International organisations.

In this case, we were able to connect a large number of tweets and domains. However, these matches were often either wrong or missing, being useless for our goal.

What kind of approach could be used then, to cope with the characteristics of tweets?

A possible solution would be to crowdsource the process, giving user a choice of concepts from which to choose, e.g. the EuroVoc domains, and asking them to annotate each tweet with one of them. These annotations could be used in the future to learn a supervised classification of new tweets.

Tweet sentiment analysis

Our second goal was to annotate each tweet/Facebook post with a sentiment score, in order to understand its position with regard to a topic or concept. We could not reliably match tweets to concepts, as we have previously seen. Nevertheless we investigated the overall tone of the social media discourse of MEPs by performing a sentiment analysis on their tweets.

We had to choose the most suitable algorithm for this, considering their characteristics. Due to their shortness and their different purposes – a tweet can express an opinion, but also report a fact, a link or be a reminder for an event – tweets’ sentiments are often unclear. The algorithm chosen was Sentiment 140[2], which is specifically built to evaluate the sentiment of brands, topics or products on Twitter. It uses machine learning classifiers and provides a sentiment score for each tweet, facilitating the evaluation of its accuracy. The score returned is a number, whose values can be 0, 2 or 4: we translated these respectively into negative, neutral and positive.

We performed a sentiment analysis on all the tweets in our dataset, generating for each of them the related sentiment score. Figure 1 shows the average value for all the tweets analysed: by far, the most common sentiment was neutral. This could be explained by either the characteristics of the tweets in our dataset – which might, for example, just report links to news articles or act as a reminder for some event – or by the low sensitivity of the algorithm.

Fig 1

Country Linking and sentiment analysis

We generated links to Geonames by matching each country’s official names to the text of the speech. Using these links, already some interesting queries can be asked:

What are the top 10 mentioned countries in the European Parliament? translates to the following SPARQL query:

SELECT DISTINCT ?countryname (COUNT(?speech) AS ?no)

WHERE

{ ?speech purl:references ?country .

   ?country geonames:name ?countryname .

}

GROUP BY ?countryname

order by DESC(?no)

Producing the following results:

Fig 2

Top-10 countries mentioned in speeches in the EP

What are the countries that are most mentioned together in a speech? Translates into the following SPARQL query:

SELECT DISTINCT ?countryname (COUNT(?speech) AS ?no)

WHERE

{ ?speech purl:references ?country .

   ?country geonames:name ?countryname .

}

GROUP BY ?countryname

order by DESC(?no)

Producing the following results:

 Fig 3

Top-10 pairs of countries most mentioned together in the EP speeches.

The combination with current Eurovoc annotations on the agenda can also point social scientists to interesting paths of analysis. For example, the number of speeches mentioning “Venezuela” is 369, while the number of speeches part of an agenda item that included “Venezuela” is 269. This can have two explanations, Venezuela is mentioned outside of the agenda several times (maybe as a comparison?), or the Eurovoc annotations might be incomplete.

For the sentiment analysis we used the Stanford Open NLP library. As the library analyze the sentiment of a sentence, we splitted the speech in the sentences containing the mention, and input them to the analyzer. The analyzer assigns of the following 5 values: Positive, negative, neutral, very positive and very negative. Following the strategy used for the Eurovoc vocabulary, we created a dummy entity “Sentiment annotation” linked to the speech, containing the sentence and its sentiment value.

We analyzed the case of Venezuela and the Netherlands:

Fig 4

Fig 5

The general sentiment for both countries shows a remarkable shift towards negativity. Setting aside the (real) possibility that the EP is a place with really bad karma, we hypothesize two explanations for these results:

  1. The training set of the model used by the analyzer uses movie reviews, this might not be accurate enough for political discourse.
  2. In some cases, what is happening is that the speaker is talking about a bad event that happen in the country in question (i.e., Floods in Venezuela in december 1999)

This opens the door to improvements in sentiment analysis applied to political discourse.

Conclusions

The Creative Camp in The Hague was an intense, challenging week. We participated with an ambitious project – perhaps too much – with the aim of enabling citizens and researchers to easily compare how their representatives at the European Parliament behave and talk on social media and in institutional debates, and what are their declarations and sentiment about the geographical places they mention. At the end of it, we had some concrete results, but also some valuable experience on the topics we covered.

Unfortunately, we were unable to provide reliable linkings based on the concepts mentioned in their tweets and in their speeches and, as we discussed earlier, the sentiment analysis of the tweets did not give any decisive results.

Nevertheless, we were able to create links between MEPs, their Twitter accounts and the tweets they wrote. These are a valuable contribution to facilitate the understanding of how MEPs behave on their social media profiles. Furthermore, each tweet has a timestamp, which enables several possible applications, such as the connection of tweets and EP debates through their dates and discovery of tweets contemporaneous to events.

Finally, we provided linkings between the speeches and the countries they mentioned and the sentiment that characterised each mention. In spite of the limitations highlighted, such linkings could be a useful base to analyse the aptitudes of MEPs towards different EU countries and understand the polarisations of EP political groups.

[1] http://www.nltk.org/book/ch02.html

[2] http://www.sentiment140.com/