A framework to simplify pre-processing location-based social media big data for sustainable urban planning and management. (February 2021)

• Record Type: Journal Article
• Authors: Abdul-Rahman, Mohammed; Chan, Edwin H.W.; Wong, Man Sing; Irekponor, Victor E.; Abdul-Rahman, Maryam O.
• Is Part Of: Cities. Volume 109(2021)
• Publication Date: 2021-02
• Subjects: Big data -- Community resilience -- Internet of things -- Natural language processing -- Social media -- Urban planning and management; City planning -- Periodicals; Urban policy -- Periodicals; 711.4
• DOI: 10.1016/j.cities.2020.102986
• Languages: English
• ISSNs: 0264-2751

Abstract

Abstract: Over the last decade, 90% of Big Data has been generated by people living in urban areas. With the advent of Internet of Things (IoT) and the increased use of the internet, Social Media has become an integral part of people's daily lives. Millions of unstructured data are being sent to the cloud every second, providing opinions practically on any discourse. This makes microblogs such as Twitter, Instagram, WeChat, and Facebook smart instruments for urban planners to harvest 'big data' on socioeconomics, urban dynamics, transportation, land uses, resilience, etc. This study proposed a framework for social media big data mining and data analytics using Twitter. It demonstrated the functionalities of the framework on a case study using Natural Language Processing and Machine Learning techniques like Latent Dirichlet Allocation and VADER Sentiment Analysis to mine, clean, process, and validate the data. The validated results from the case study showed high accuracy that Social Media Big Data can be used to study the spatiotemporal dynamism of community challenges. Highlights: A framework was developed to pre-process location-based social media big data for sustainable urban planning and management The methodologies used for the proposed framework were demonstrated using a case study Data from the case study was validated and found have high accuracy, which justified the usability of the framework All other steps involved are fully justified for research Contributions All codes and procedures are provided in the appendix and links to the GitHub Libraries are also provided