Oleg Zabluda's blog
Saturday, May 05, 2018
 
AI researchers allege that machine learning is alchemy
AI researchers allege that machine learning is alchemy
"""
examples of what they see as the alchemy problem and offer prescriptions for bolstering AI's rigor. The issue is distinct from AI's reproducibility problem [...] or "interpretability" problem
[...]
when other researchers stripped most of the complexity from a state-of-the-art language translation algorithm, it actually translated from English to German or French better and more efficiently, showing that its creators didn't fully grasp what those extra parts were good for. Conversely, sometimes the bells and whistles tacked onto an algorithm are the only good parts [...] In some cases, he says, the core of an algorithm is technically flawed, implying that its good results are "attributable entirely to other tricks applied on top."
[...]
researchers should conduct "ablation studies" like those done with the translation algorithm: deleting parts of an algorithm one at a time to see the function of each component. [...] AI needs to borrow from physics, where researchers often shrink a problem down to a smaller "toy problem."
[...]
the field also needs to reduce its emphasis on competitive testing. At present, a paper is more likely to be published if the reported algorithm beats some benchmark than if the paper sheds light on the software's inner workings [...] That's how the fancy translation algorithm made it through peer review.
[...]
Yann LeCun says "It's not alchemy, it's engineering. Engineering is messy."
"""
http://www.sciencemag.org/news/2018/05/ai-researchers-allege-machine-learning-alchemy
http://www.sciencemag.org/news/2018/05/ai-researchers-allege-machine-learning-alchemy

Labels:


- posted by Oleg Zabluda @ 8:06 PM | permalink | | | technorati | bloglines | feedster
| |

Home

Powered by Blogger