Trjtdtk

group theory by geometry

Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?

Can a one-dimensional blade cut everything ? (chainsaw) (Sword)

In a future war, an old lady is trying to raise a boy but one of the weapons has made everyone deaf

Credit cards used everywhere in Singapore or Malaysia?

Why one should not leave fingerprints on bulbs and plugs?

How to deal with taxi scam when on vacation?

Do I need to be arrogant to get ahead?

Min function accepting varying number of arguments in C++17

Why won't this compile? Argument of h has an extra {

I am confused as to how the inverse of a certain function is found.

How do I change two letters closest to a string and one letter immediately after a string using Notepad++?

Why Choose Less Effective Armour Types?

How well should I expect Adam to work?

Life insurance that covers only simultaneous/dual deaths

A Cautionary Suggestion

Tikz diagrams and node placements

Is it normal that my co-workers at a fitness company criticize my food choices?

How do I write a formula in words in LaTeX?

How difficult is it to simply disable/disengage the MCAS on Boeing 737 Max 8 & 9 Aircraft?

A formula for delta function in quantum mechanics

Sailing the cryptic seas

Brexit - No Deal Rejection

How do I hide Chekhov's Gun?

How Transformer is Bidirectional - Machine Learning

Machine Learning vs Deep LearningIntro to Machine LearningLearning AI, Machine Learning, Deep LearningMachine Learning Design DocumentHow to learn Machine LearningMachine Learning & Image Recognition: How to start?How to chose a Machine Learning algorithm?How can cognitive neuroscience enhance machine learning?Machine learning over machine learning resultWhat is the reason for the speedup of transformer-xl?

1

$begingroup$

Asking question in datascience forum, as this forum seems well suited for data science related questions: https://stackoverflow.com/questions/55158554/how-transformer-is-bidirectional-machine-learning/55158766?noredirect=1#comment97066160_55158766

I am coming from Google BERT context (Bidirectional Encoder representations from Transformers). I have gone through architecture and codes. People say this is bidirectional by nature. To make it unidirectional attention some mask is to be applied.

Basically a transformer takes key, values and queries as input; uses encoder decoder architecture; and applies attention to these keys, queries and values. What I understood is we need to pass tokens explicitly rather than transformer understanding this by nature.

Can someone please explain what makes transformer bidirectional by nature

Answer received so far:

1. People confirmed that Transformer has Bidirectional nature, rather than an external code making it bidirectional.

2.
My doubt: We are passing Q K V embeddings to transformer, to which it applies N layers of self attention using ScaledDotMatrix attention. Same thing can be done by unidirection approach as well. May I know what part I am missing in my understanding. If someone can point to code where it is getting bidirectional, it would be a great help.

machine-learning transformer bert

share|improve this question

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

$endgroup$

add a comment | 

1

$begingroup$

Asking question in datascience forum, as this forum seems well suited for data science related questions: https://stackoverflow.com/questions/55158554/how-transformer-is-bidirectional-machine-learning/55158766?noredirect=1#comment97066160_55158766

I am coming from Google BERT context (Bidirectional Encoder representations from Transformers). I have gone through architecture and codes. People say this is bidirectional by nature. To make it unidirectional attention some mask is to be applied.

Basically a transformer takes key, values and queries as input; uses encoder decoder architecture; and applies attention to these keys, queries and values. What I understood is we need to pass tokens explicitly rather than transformer understanding this by nature.

Can someone please explain what makes transformer bidirectional by nature

Answer received so far:

1. People confirmed that Transformer has Bidirectional nature, rather than an external code making it bidirectional.

2.
My doubt: We are passing Q K V embeddings to transformer, to which it applies N layers of self attention using ScaledDotMatrix attention. Same thing can be done by unidirection approach as well. May I know what part I am missing in my understanding. If someone can point to code where it is getting bidirectional, it would be a great help.

machine-learning transformer bert

share|improve this question

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

$endgroup$

add a comment | 

$begingroup$

Asking question in datascience forum, as this forum seems well suited for data science related questions: https://stackoverflow.com/questions/55158554/how-transformer-is-bidirectional-machine-learning/55158766?noredirect=1#comment97066160_55158766

I am coming from Google BERT context (Bidirectional Encoder representations from Transformers). I have gone through architecture and codes. People say this is bidirectional by nature. To make it unidirectional attention some mask is to be applied.

Basically a transformer takes key, values and queries as input; uses encoder decoder architecture; and applies attention to these keys, queries and values. What I understood is we need to pass tokens explicitly rather than transformer understanding this by nature.

Can someone please explain what makes transformer bidirectional by nature

Answer received so far:

1. People confirmed that Transformer has Bidirectional nature, rather than an external code making it bidirectional.

2.
My doubt: We are passing Q K V embeddings to transformer, to which it applies N layers of self attention using ScaledDotMatrix attention. Same thing can be done by unidirection approach as well. May I know what part I am missing in my understanding. If someone can point to code where it is getting bidirectional, it would be a great help.

machine-learning transformer bert

share|improve this question

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

$endgroup$

share|improve this question

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

share|improve this question

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

edited 7 hours ago

I_Play_With_Data

1,212531

edited 7 hours ago

I_Play_With_Data

1,212531

asked 10 hours ago

user10557045user10557045

165

asked 10 hours ago

user10557045user10557045

165