Oleg Zabluda's blog
Friday, September 01, 2017
 
Inside an Epic Hotel Room Hacking Spree | WIRED
Inside an Epic Hotel Room Hacking Spree | WIRED
https://www.wired.com/2017/08/the-hotel-hacker/
https://www.wired.com/2017/08/the-hotel-hacker/

Labels:


 
Transformer: A Novel Neural Network Architecture for Language Understanding
Transformer: A Novel Neural Network Architecture for Language Understanding
"""
deciding on the most likely meaning and appropriate representation of the word “bank” in the sentence “I arrived at the bank after crossing the…” requires knowing if the sentence ends in “... road.” or “... river.”

RNNs have in recent years become the typical network architecture for translation, processing language sequentially in a left-to-right or right-to-left fashion.
[...]
In contrast, the Transformer only performs a small, constant number of steps (chosen empirically). In each step, it applies a self-attention mechanism which directly models relationships between all words in a sentence, regardless of their respective position. [...] Transformer can learn to immediately attend to the word “river” and make this decision in a single step.
"""
https://research.googleblog.com/2017/08/transformer-novel-neural-network.html

Labels:



Powered by Blogger