What will the future with intelligent machines be like?

They learn and decide for themselves: So-called autonomous systems, such as self-driving automobiles, carebots, networked home appliances and autonomous weapons systems, are already being used in daily life or are just about ready for market. Ingenious sensors, complex and self-learning algorithms as well as comprehensive possibilities for connectivity are allowing such systems to react to their environment rapidly by synchronizing multifaceted data and to act largely independently from human interventions. From this a range of ethical, legal and social questions arises. The public should have its say and be involved in shaping the future use of the considerable potential of these new developments, the Chair of the Ethics Council, Peter Dabrock, emphasised right at the start of the conference: "In the ocean created by our data streams, can we in a self-determined manner remain ourselves, or are we stumbling, out of sheer joy at miniaturized improvements and more intoxicated than conscious, into a trap of immaturity?"

During the first presentation, Henning Kagermann from the National Academy of Science and Engineering (acatech) gave an account of what highly automated systems can already do now and how, by means of artificial intelligence methods, a new generation of increasingly autonomous systems is emerging. Whether in industrial production, mobility, smart homes or in (rescue) measures dangerous for humans – for all fields of application it holds true that humans are to be supported by machines and their capabilities supplemented, but that they should not be replaced. Therefore, according to Kagermann, "an early and long-term societal dialogue [is] necessary, in which opportunities and risks are made transparent and weighed against each other".

The use of autonomous systems will "change markets in terms of a 'creative destruction' just as quickly [...] as social and administrative institutions", Christoph M. Schmidt, Chair of the German Council of Economic Experts, predicted. He called for new regulatory approaches that place an emphasis on enabling individual participation and security rather than protection through the state.

Katharina A. Zweig from the University of Kaiserslautern lamented the dubious quality of algorithm-based, decision-supporting systems. She strongly urged "the development of quality-assuring processes for their design, implementation, maintenance and continual improvement. Additionally, a public discussion is required about which societal processes are generally appropriate for algorithmic decision systems and according to what criteria they should be optimized", Zweig said.

From a philosophical perspective, Julian Nida-Rümelin from the Ludwig Maximilian University of Munich set out why autonomous systems could not assume any responsibility. The concept of responsibility is linked to intentionality and personality – capabilities that belong to humans alone, according to Nida-Rümelin. The development and the increasing use of autonomous systems may indeed by desirable, "but are ethically tenable only on the condition that autonomous systems [would be] ascribed no mental and specifically personal qualities". A strong artificial intelligence would, moreover, be actually hostile to technology, for one would consequently have to concede truly intelligent machines rights and dignity, which would set narrow limits to their instrumentalization.

The legal expert Christiane Wendehorst from the University of Vienna added that in current law, machines are not entitled to any legal identity even when they are endowed with advanced artificial intelligence. "They are hence not addressees of legal regulations and can neither 'be allowed' nor 'not be allowed'", according to Wendehorst. Rather, addressees of regulation are the humans or legal persons who produce, sell and use machines. Like Nida-Rümelin, she also viewed a development towards the "e-person" to be undesirable – an assessment that was in part objected to by the audience.

In four forums taking place in parallel in the afternoon, the participants discussed autonomous systems in various fields of application. At issue were self-driving automobiles; medical machines and carebots; the networked home; and autonomous weapons systems – in the last forum, the commander of the German Air Force, Lieutenant General Karl Müllner, spoke, among others. In all the forums, it became clear that, contrary to a tendency to make humans into the objects of autonomous systems, humans and their autonomy must remain central.

In her closing speech, the writer Thea Dorn judged the application to machines of concepts such as intelligence, autonomy and learning to be extremely problematic, since these belong among the "kingly virtues of mankind". Machine automation, in contrast, is first and foremost about adjusting these into well-functioning helpers. In view of a tendency towards mental lethargy, where all too many competencies are ceded to machines, Dorn called for "character-forming exercise bands" in order to build individual and societal resiliencies.

The programme of the event, the talks and discussion contributions of the participants, including audio and video recordings as well as a transcript, are available (in German) here.