Be it resolved, the quest for true AI is one of the great existential risks of our time.

A novel written by artificial intelligence is shortlisted for a literary prize. Google software beats a human opponent at Go, one of the most complex board games in the world. Self-driving cars recognize images and then make decisions.

These are just some of the extraordinary accomplishments based on artificial intelligence that we have witnessed in the past few years. But there are many scientists who are pushing for a more cautious approach to how we move forward on machine intelligence. They say that we are not far off from developing superintelligent machines whose IQ far surpasses that of humans and who don’t come with an off switch—with seriously negative consequences for humanity. These scientists argue that we can prevent this loss of control but we need to act now by making sure algorithms ensure that benevolence and human mastery are foundational pillars.

Critics say that this view of superintelligence highly overrates the abilities of machines today and in the future, and deeply underestimates the incredible powers of human thinking. They say that AI is nowhere close to matching the human talent for understanding and generalization, and may never come close. Unsubstantiated fears of a superintelligent future are getting in the way of resolving one of the riddles of human existence–human intelligence–which could unlock untold creativity and progress.