For research on written texts, the machine learning turn marks a paradigm shift away from the hermeneutic tradition of understanding and towards a functional approach to texts as machine-readable and thus quantifiable encodings of meaning. Confronted with the fact that such forms of computer-based text analysis can be effective even in the absence of any expert knowledge about the make-up of texts, researchers in humanities need to be open not only to the adoption of new technologies in their research, but also to self-reflection (e.g., Dewey, 1964). Given its position at the intersection between technological and humanistic research traditions, Digital Humanities in particular has a responsibility and an interest to critically assess its own practices.
In this vein, our research project addresses distinct research methodologies as well as the epistemic dimension of textual Digital Humanities. In two aligned case studies that combine machine learning applications of sentiment detection with more traditional manual and semi-automatic text analysis, such as close reading and annotation, we study evaluative language use in web 2.0 discourses as an example of text production in the context of digitization. Our practical aim in these case studies is to examine how available dictionaries, algorithms, and machine learning procedures work on two sets of German textual data extracted from the social web, travel blogs (travelblog.org) and literature reviews (lovelybooks.de). Addressing the question of how users convey their evaluation of journeys and books, we on the one hand find and generate sets of answers to the question itself using traditional humanities and machine learning approaches (level 0), and on the other hand shed light on the role of the distinct research epistemologies as well as our own research practices and attitudes which frame or even shape our findings (level 1).
One of the central aims for our multidisciplinary mixed-method research design is thus a thorough reflection of the premises underlying our own research practice as different types of actors in the field of textual humanities. Scrutiny will be directed at built-in biases of training corpora used for machine-learning, but also at researchers’ guiding schemata for scientific/scholarly action (Fleck’s ‘thought styles’, see Andresen et al., 2018, and the epistemic cultures described by Knorr-Cetina, 1999): these are mutually determined by and determine reality and humanity, showing in research practice – as types of research questions, argumentation, interpretation, and methodology.
Our methods for level 1 consist in a range of tools for reflective thinking, including meta-annotation of the (semi-)automatic and manual tags (for literary annotation, see Gius & Jacke, 2017) and memos for documenting observations (Glaser, 2008). Level 1 will address the following questions:
- Theoretical frameworks: What understanding of language and communication motivate the different approaches? What assumptions are made about the world, reality and humanity?
- Operationalization: What, if any, linguistic and literary categories and theories are being operationalized? What covert and overt decisions inform existing analytical practices?
- Results and interpretation: What rationale is used to discuss and interpret results? How do interpretations of results tie in with the respective theoretical frameworks? What open questions are foregrounded? What role and importance are given to the interpretation of data within the overall research projects? How is subjectivity discussed?
- Research practice: What explicit and implicit rationale motivates the employment of particular research processes?
Using the empirical research on web 2.0 discourses throughout its stages for our enquiry, we examine the underlying epistemologies that our team brings to the table, extrapolating from here to our field of textual DH generally. Our goal is not to find the best method, but to reveal underlying worldviews and epistemologies that guide the analysis – through principled reflective thinking: “it enables us to know what we are about when we act. It converts action that is merely appetitive, blind, and impulsive into intelligent action” (Dewey, 1964, p. 211).