I, Probot

If you are a professor, be afraid! Be very afraid, for the “Probot” – or “Professor-Robot” – may be about to replace you! I am not exaggerating: robots are now capable of truly remarkable feats in many domains including higher education. CMM readers can find a recent example of what these machines can do in Project Debater, an Artificial Intelligence programme from IBM[1]. Last month, in San Francisco, California, and earlier this month, in Givataim, Israel, the company demonstrated Project Debater successfully debating with humans. The matters deliberated were as diverse as subsidies for space exploration, use of telemedicine and the pros and cons of mass surveillance and genetic engineering. 

According to IBM, Project Debater did not know the debate topics in advance and had not received pre-training on any specific issue. For each debate, Project Debater and its human opponent had to prepare a four-minute opening statement, followed by a four-minute “rebuttal” and a two-minute closing summary. To compose a coherent speech, Project Debater had to be able to piece together relevant points and evidence from hundreds of millions of newspaper and magazine articles. It then had to deliver the speech in a persuasive manner, even occasionally trying humour to enliven the discussion. In addition, it must listen to and understand the arguments presented by its opponent, identifying key claims hidden within long continuous spoken language and classifying the stance of each claim to construct a strong rebuttal.  

Overall, Project Debater held its own well against its expert human opponents. Admittedly, it had a somewhat monotonous voice and suffered a few slip-ups, such as repeating itself, mentioning points that were not entirely pertinent and telling jokes that did not come out right. However, Project Debater beat the human debater in San Francisco on the subject of telemedicine by changing the mind of nine audience members.[2] In Givataim, although the results were not as clear-cut, with the robot and the human debater tying in one debate and the human debater being more persuasive in another, the robot did score higher on knowledge and speech clarity.

Project Debater represents the state of the art in Artificial General Intelligence, the Holy Grail of computer science. However, there are much more established instances of knowledge-based technology used to assist or replace teachers. So-called intelligent tutoring systems that can both detect incorrect answers and diagnose why they were wrong have been around for many years. Examples include commercially available math courseware MATHia for schools and Mika for higher education. These “cognitive tutors” incorporate a model of what the student has to learn and create a dynamic model of the learner based on his or her evolving knowledge. By comparing the two models, they can adapt the instructional strategy and customise the teaching to suit the student’s learning style as he or she progresses through the content.

Research has shown that using intelligent tutoring systems enables students to achieve more compared to their peers who learned from textbooks or underwent teacher-led, large-group instruction and ordinary computer-based instruction. Furthermore, intelligent tutoring systems and individualised human tutoring or small-group instruction produced similar learning outcomes while “significant, positive effect sizes were found at all levels of education, in almost all subject domains evaluated.” [3] Thus, it would seem that intelligent tutoring systems are poised to take over teaching jobs from humans. However, that has so far not happened and intelligent tutoring systems have yet to flourish. The main reason is that authoring and maintaining intelligent tutoring systems require a great deal of programming expertise and this has been the main bottleneck preventing their wider adoption. 

Over the years, there have been a plethora of authoring tools designed to deskill the process of developing intelligent tutoring systems, but the quest for the perfect robot to automate the creation and multiplication of Probots goes on. Given how far the area of Artificial General Intelligence has progressed, as the example of Project Debater has demonstrated, it will be a matter of time before that goal is reached. For now, it looks like Asimov’s Zeroth Law holds: a robot may not harm humanity, or through inaction allow humanity to come to harm. Although powerful Probots do exist and can technically replace teachers in limited areas, they are still a rare and relatively innocuous breed and professors’ jobs are still safe – pro tem.

References

[1] https://www.research.ibm.com/artificial-intelligence/project-debater/research.html

[2] https://www.timesofisrael.com/ibm-showcases-debating-robot-in-israel-where-it-was-sired-by-haifa-team/

[3] W Ma, O O Adesope, J C Nesbit and Q Liu, J of Educational Psychology, 2014, 106 (4), 901–918

About the Author

Duc Pham is Chance Professor of Engineering at The University of Birmingham

Featured in CMM