Michael Achmann-Denkler 
The analysis of texutal data has a long tradition under the term Natural Language Processing (NLP). As noted by Bengfort, Bilbro, and Ojeda (2018), “Language is unstructured data that has been produced by people to be understood by other people”. This characterization of language as unstructured data highlights its contrast with structured or semi-structured data. Unlike structured data, which is organized in a way that computers can easily parse and analyze, unstructured data like language requires more complex methods to be processed and understood. In the context of e.g. Instagram, CrowdTangle exports contain structured data columns such as ‘User Name’, ‘Like Count’, or ‘Comment Count’. These pieces of data are quantifiable and can be easily sorted, filtered, or counted, e.g. using tools like Excel or Python’s pandas library. For instance, we can quickly determine the most active users by counting the number of rows associated with each username. In contrast, unstructured data is not organized in a predefined manner and is typically more challenging to process and analyze. The ‘Description’ column in our dataset, which contains the captions of Instagram posts, is a prime example of unstructured data. These captions, composed of paragraphs or sentences, require different analytical approaches to extract meaningful insights. Unlike structured data, we cannot simply count or sort these texts in a straightforward manner. In our context, we often refer to the collection of texts we analyze as a “Corpus”. Each individual piece of text is called a “Document”. Each document can be broken down into smaller units known as “features”. Features can be words, phrases, or even patterns of words, which we then use to quantify and analyze the text (compare p. 230 Haim 2023). For the goal of our research seminar, we can follow the three technical perspectives inspired by Haim (2023): 1. Frequency Analysis, 2. Contextual Analysis, and 3. Content Analysis.
We are working with Python and pandas, our data is structured in tables, also known as DataFrames. Each DataFrame (df) consists of rows and columns. We can store and structure data differently using these two dimensions, one concept for storing research data using tables is Tidy Data (Wickham 2014). According to this standard
- Each variable forms a column.
- Each observation forms a row.
- Each type of observational unit forms a table.
What, in context of social media data, is an obersavtion? Is it a post? I suggest to start by seeing posts as observations, i.e. rows. Thus, we have one table for our corpus, consisting of one row per post with multiple columns for different variables, including an ID, possibly a link, a referrence to the image / video, and one or more text variables for each post. When dealing with Instagram or TikTok posts, we might have three text columns: caption / description, OCR, and transcription. When dealing with stories two: OCR and transcription.
When dealing with more complex data, e.g. Instagram albums that may contain multiple images per post, we will have to reconsider this choice. In this case we might consider each observation to be one image / video, which has variables like OCR and transcription. Keeping the ID column for images and videos, we have a fixed reference to the original post, thus we may re-merge the data later on with the post metadata or combine variables across media for one post.
All data exported from CrowdTangle, 4CAT, and Zeeschuimer-F are saved as CSV files. Throughout the semester, we keep using this file format to save our progress. We work with multiple Jupyter notebooks, generally one notebook per task. This helps to keep a good structure of our projects. Each time we modified the df, we save the CSV file to our Google Drive / Harddrive. In the two examples below we add an OCR and a Transcription column to our DataFrame, for each task we use one notebook. After completing each task, we store the results in a file. While Google Drive provides file versioning to mitigate data loss in certain scenarios, I recommend to save your results to a new file during the experimental phase. This practice ensures data safety until you have fully verified the functionality of your code. Additionally, I recommend naming your files in a YYYY-MM-DD-descriptive-name.csv fashion. When working with colab notebooks I recommend to keep track of notebooks using notes / lists, e.g. using the Dataloom plugin for Obsidian.
The CSV files contain only metadata, the actual media files (images / videos) are saved to different locations. The OCR and Transcription notebooks below contain code to import media files from 4CAT and Zeeschuimer-F. I suggest to save the files to media/videos or media/images. Both notebooks introduce a column image_file or video_file where the relative location of the media files is written to. Creating a new ZIP file using the new folder structure and saving the file to Google Drive allows us to use the media files in future notebooks (e.g. for image classification) without modifying the image_file or video_file columns again.
This page and all referenced notebooks deal with 4CAT and Zeeschuimer-F metadata and media files. Generally all information applies to instaloader as well. Its advisable to use the --filename-pattern command line parameter to control the filename of the media files. Mapping JSON metadata to actual media objects becomes easier this way. Once all posts / stories have been loaded using instaloader, I recommend to read all JSON files in a loop and create a DataFrame (see Data Collection / Posts / Instaloader for more information and code examples).
Key Take-Aways
CSV files, either on our harddrive or Google Drivemedia/videos, and media/images, which we compress to ZIP and keep on our Google Drive (or central HDD location)Computational approaches for text analyses are established as part of computational sociales science research (Baden et al. 2022), which we may utilize when dealing with visual and multimodal social media. Instagram posts often contain embedded text, TikTok posts often contain an audio layer, both of which we can transform to computer readable text. For the first, we are going to use OCR, for the second we apply Whisper. The following subchapters demonstrate the application of these technique in order to extract textual content from images and videos. In the thirs subchapter, I demonstrate a simple application of corpus analytics for a first analysis of the social media content based on word frequencies.
OpenAI offers Whisper transcriptions as a service, see their documentation. The notebook below takes you step-by-step through using the Whisper model on your own computer / colab.
Among a variety of possibilities, we can, for example, look at the frequencies of the words contained in the corpus or examine the corpus for recurring themes it contains.
First we need to import all the required libraries once again. The Natural Language Toolkit (NLTK) gives us access to a variety of natural language processing functions (e.g. tokenisation, stop word removal, part-of-speech tagging, …).
import nltk
nltk.download('punkt')
from nltk.tokenize import RegexpTokenizer
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
import requests
import pandas as pdWhen analysing word frequencies, we can use stop word lists to ignore words that occur frequently but are not relevant to us. We can easily download such a list. However, this can also be individually adapted to the purpose.
# Retrieve Stopwords from Github
sw_json = requests.get('https://github.com/stopwords-iso/stopwords-de/raw/master/stopwords-de.json')Now we can tokenise the existing text, remove the stop words or punctuation marks they contain, convert the words to lower case, or use bi-grams in addition to single-word tokens.
We then sum up the occurrences of the individual words and make the results available in a DataFrame.
def word_freq(text, punctuation=False, stop_words = False, lowercasing = False, bigrams = False):
if punctuation:
# Tokenizing, removing punctuation
tokens = RegexpTokenizer(r'\w+').tokenize(text) # https://regexr.com/
else:
# Tokenizing, w/o removing punctuation
# tokens = text.split()
tokens = word_tokenize(text)
if stop_words:
# Removing Stopwords
tokens = [w for w in tokens if not w.lower() in stop_words]
if lowercasing:
# Lower-Casing
tokens = [w.lower() for w in tokens]
if bigrams:
# Converting text tokens into bigrams
tokens = nltk.bigrams(tokens)
# Creating Data Frame
freq = nltk.FreqDist(tokens) # display(freq)
df = pd.DataFrame.from_dict(freq, orient='index')
df.columns = ['Frequency']
df.index.name = 'Term'
# Here we calculate the total number of tokens in our Frequency List
total_tokens = sum(freq.values()) # sum([2,3,4,5,6])
# Here we add a new column `Relative` (*100 for percentage)
df['Relative'] = (df['Frequency'] / total_tokens) * 100
return dffrom pathlib import Path
import os
#@markdown Do you want bigrams included?
bigrams = True #@param {type:"boolean"}
#@markdown Should all words get lower cased before counting the occurances?
lowercasing = True #@param {type:"boolean"}
#@markdown Do you want to exclude stopwords in your result list?
stopwords = True #@param {type:"boolean"}
#@markdown Do you want to remove punctuation before counting the occurances?
punctuation = True #@param {type:"boolean"}# Load stopwords file if necessary
if stopwords:
stopwords = sw_json.json()
# Read source file and concat all texts
text = ' '.join(list(df[text_column]))
# Call word_freq() with specified parameters
df_freq = word_freq(text, punctuation = punctuation, stop_words = stopwords, lowercasing = lowercasing, bigrams = bigrams)
# Sort results for descending values
df_freq = df_freq.sort_values("Relative", ascending = False)
display(df_freq[0:10])| Frequency | Relative | |
|---|---|---|
| Term | ||
| (jüdisches, leben) | 5 | 1.259446 |
| (allerheiligen, allerseelen) | 4 | 1.007557 |
| (ilse, aigner) | 3 | 0.755668 |
| (bayerischer, landtag) | 3 | 0.755668 |
| (klare, haltung) | 2 | 0.503778 |
| (wünschen, einfach) | 2 | 0.503778 |
| (vaters, freundschaftliche) | 2 | 0.503778 |
| (tod, vaters) | 2 | 0.503778 |
| (günter, tod) | 2 | 0.503778 |
| (schwiegervater, günter) | 2 | 0.503778 |
Wordcloud
One way to visualise word frequencies and recurring themes of texts are word clouds. These basically show the most frequently occurring words in the text (similar to the table created earlier), but more frequently occurring words are depicted larger than less frequently occurring words.
First, we have to install the necessary library wordcloud.
!pip install -q wordcloudThe actual implementation of this approach is relatively simple. We need to combine all the texts into a single text, as we did in the previous step with the frequency analysis, and pass it to the imported library.
from wordcloud import WordCloud, STOPWORDS
def generate_wordcloud(text, path):
text = ' '.join(list(text))
# Generate a word cloud image
wordcloud = WordCloud(background_color="white",width=1920, height=1080).generate(text)
# Dazugehörige Grafik erstellen
plt.imshow(wordcloud, interpolation="bilinear") # Auflösung/Interpolation der Grafik
plt.axis("off")
plt.figtext(0.5, 0.1, wordcloud_subcaption, wrap=True, horizontalalignment='center', fontsize=12)
plt.savefig(path, dpi=300)
plt.show()Once again, we have the option of adjusting various parameters. Remember to specify the right file path, file name and column of your text data!
#@markdown Input for additional stopwords; whitespace separated
stopwords_extension_wc = '' #@param {type: "string"}
#@markdown Subcaption for the wordcloud, leave blank to ignore
wordcloud_subcaption = 'Markus S\xF6der' #@param {type: "string"}Now all we have to do is load the stop word file, add our own additions and then trigger the creation of the word cloud using the function we created at the beginning.
The result image is saved in the defined data_path.
import matplotlib.pyplot as plt
import requests
# Retrieve Stopwords from Github
r = requests.get('https://github.com/stopwords-iso/stopwords-de/raw/master/stopwords-de.json')
stop_words = r.json()
# Convert input into list
stopwords_extension_wc_list = stopwords_extension_wc.split(' ')
stop_words.extend(stopwords_extension_wc_list)
# Stopwörter in die WordCloud laden
STOPWORDS.update(stop_words)
generate_wordcloud(df[text_column], 'wordcloud.png')
In summary, this session provides us with the practical skills to use Python, pandas, and Jupyter notebooks for the computational analysis of multimodal social media data. Our adherence to Tidy Data principles and the integration of technologies like OCR and Whisper are integral to extract and analyze textual content from multimedia sources. In the next session we will keep exploring the content through a textual lens. Further, we will use prompting as a technique to classify texts as part of a computational content analysis.
Python & Computational Social Sciences
Python & NLP