ELIZA reloaded
The "ELIZA reloaded" project investigated the transformation of knowledge work through the use of generative language-based AI systems. The focus was on three areas: Programming, science and consulting.
Background
Since the release of ChatGPT, there has been a lively debate about the impact of generative AI systems: Does the software replace jobs, or is it a useful tool for working better? Is ChatGPT brilliant and human-like, or misleading and error-prone? Does the technology facilitate better use of knowledge, or does it lead to misinformation and deception, such as in exams at school and university?
So far, public discussion has involved various speculations but little solid knowledge.
The “ELIZA Reloaded“ project examined how knowledge work is changing due to the use of generative language-based AI systems. Our exploratory study focused on three representative areas of knowledge work: programming, science, and consulting.
Methods
The investigation into potential changes in these professions was guided by three research questions:
- What expectations and concerns do professionals in different knowledge work fields have about ChatGPT, and what potentials and risks of the technology are being discussed?
- How does ChatGPT, as a new work tool, alter the work processes and professional roles of people in these fields?
- What concrete benefits do professionals gain from using ChatGPT for performing knowledge-intensive tasks?
The focus was primarily on how members of these professions used the new technological tools, how they interacted with them, and what risks and potentials of ChatGPT and generative AI systems have been discussed in knowledge work.
The project was structured into three work packages: Started with an initial analysis of policy documents, recommendations, and position papers on the risks and potentials of ChatGPT, we conducted further exploratory expert interviews with relevant professional associations and stakeholders. The focus here was on possible changes to work and the factors that structured the use of these technologies, such as facilitating or, for instance, complicating factors like data protection regulations. Builded on these insights, we designed a controlled experimental setup to simulate various knowledge-intensive tasks at the workplace. This setup has been conducted both with and without generative AI systems, testing the specific potentials of the technology.
Objective
The project thus provided an initial empirical contribution to the use of generative language-based AI systems in knowledge work and how work, learning, and knowledge practices are changing in the context of digitalization.
By the way: The project title "ELIZA Reloaded" refers to the first chatbot developed by computer pioneer Joseph Weizenbaum, named "ELIZA." Weizenbaum observed with interest but also concern how laypeople attributed intelligence to this chatbot and revealed their most private secrets to it. He cautioned that AI should be properly understood and discussed in terms of how it can be used for the benefit of society. In this spirit, we aim to approach the topic of "ChatGPT."
