Analysis of Twitter messages using big data tools to evaluate and locate the activity in the city of Valencia (Spain)

Abstract

This paper presents the big data architecture and work flow used to download georeferenced tweets, store them in a NoSQL database, analyse them using the Apache Spark framework, and visualize the results. The study covers a complete year (from December 10, 2016 to December 10, 2017) in the city of Valencia (Eastern Spain), which is considered to be the third most important in Spain, having a population of nearly 800,000 inhabitants and a size of 135 km2. The concepts of a specific event map and a specific event map with positive or negative sentiment are developed to highlight the location of an event. This approach is undertaken by subtracting the heat map of a specific day from the mean daily heat map, which is obtained by taking into account the 365 days of the studied period. This paper demonstrates how the proposed analysis from tweets can be used to depict city events and discover their spatiotemporal characteristics. Finally, the combination of all daily specific events maps in a single map, leads to the conclusion that the city of Valencia city has appropriate urban infrastructures to support these events. © 2018 Elsevier Ltd