Which is better: GPT or RelGAN for text generation?2019 Community Moderator ElectionWhat is the difference between TextGAN and LM for text generation?GANs (generative adversarial networks) possible for text as well?Generator loss not decreasing- text to image synthesisChoosing a right algorithm for template-based text generationHow should I format input and output for text generation with LSTMsGumbel Softmax vs Vanilla Softmax for GAN trainingWhich neural network to choose for classification from text/speech?NLP text autoencoder that generates text in poetic meterWhat is the interpretation of the expectation notation in the GAN formulation?What is the difference between TextGAN and LM for text generation?How to prepare the data for text generation task
Neighboring nodes in the network
Is it canonical bit space?
Why do I get two different answers for this counting problem?
Python: return float 1.0 as int 1 but float 1.5 as float 1.5
Drawing a Grid/Tikz image
Why is it a bad idea to hire a hitman to eliminate most corrupt politicians?
Why is this clock signal connected to a capacitor to gnd?
Blender 2.8 I can't see vertices, edges or faces in edit mode
How to say in German "enjoying home comforts"
Is there an expression that means doing something right before you will need it rather than doing it in case you might need it?
Can a virus destroy the BIOS of a modern computer?
Saudi Arabia Transit Visa
Why "Having chlorophyll without photosynthesis is actually very dangerous" and "like living with a bomb"?
What is the word for reserving something for yourself before others do?
Why do bosons tend to occupy the same state?
Is the Joker left-handed?
What method can I use to design a dungeon difficult enough that the PCs can't make it through without killing them?
Is it possible to create light that imparts a greater proportion of its energy as momentum rather than heat?
Why is consensus so controversial in Britain?
numexpr behavior in math mode and/or TikZ
Facing a paradox: Ehrenfest theorem in one dimension
90's TV series where a boy goes to another dimension through portal near power lines
Why doesn't using multiple commands with a || or && conditional work?
I would say: "You are another teacher", but she is a woman and I am a man
Which is better: GPT or RelGAN for text generation?
2019 Community Moderator ElectionWhat is the difference between TextGAN and LM for text generation?GANs (generative adversarial networks) possible for text as well?Generator loss not decreasing- text to image synthesisChoosing a right algorithm for template-based text generationHow should I format input and output for text generation with LSTMsGumbel Softmax vs Vanilla Softmax for GAN trainingWhich neural network to choose for classification from text/speech?NLP text autoencoder that generates text in poetic meterWhat is the interpretation of the expectation notation in the GAN formulation?What is the difference between TextGAN and LM for text generation?How to prepare the data for text generation task
$begingroup$
Based on my understanding, gpt or gpt-2 are using language model loss to train and generate text, which do not contains GAN.
So which is better: GPT vs RelGAN/LeakGAN/SeqGAN/TextGAN
I am so confused about this question.
deep-learning gan natural-language-process text-generation transformer
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
$endgroup$
add a comment |
$begingroup$
Based on my understanding, gpt or gpt-2 are using language model loss to train and generate text, which do not contains GAN.
So which is better: GPT vs RelGAN/LeakGAN/SeqGAN/TextGAN
I am so confused about this question.
deep-learning gan natural-language-process text-generation transformer
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
$endgroup$
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14
add a comment |
$begingroup$
Based on my understanding, gpt or gpt-2 are using language model loss to train and generate text, which do not contains GAN.
So which is better: GPT vs RelGAN/LeakGAN/SeqGAN/TextGAN
I am so confused about this question.
deep-learning gan natural-language-process text-generation transformer
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
$endgroup$
Based on my understanding, gpt or gpt-2 are using language model loss to train and generate text, which do not contains GAN.
So which is better: GPT vs RelGAN/LeakGAN/SeqGAN/TextGAN
I am so confused about this question.
deep-learning gan natural-language-process text-generation transformer
deep-learning gan natural-language-process text-generation transformer
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
asked Mar 26 at 8:39
不是phd的phd不是phd的phd
2049
2049
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14
add a comment |
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper:
This research demonstrates that well-adjusted language models are a remarkably strong baseline
and that temperature sweeping can provide a very clear characterization of model performance. A
well-adjusted language model outperforms the considered GAN variants as evaluated on both local,
and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework
shares characteristics with a Receiver Operating Curve. Analogously, if one needed a single scalar to
compare NLG models, one could compute area under the curve and seek the model with the smallest
value (lower is better for our considered metrics).
GAN-based generative models have been proven effective on real-valued data, however, but there
exist many difficult pernicious issues of moving to discrete data. These issues must be overcome
before they will improve over the strong MLE baselines. On the datasets and tasks considered,
potential issues caused by exposure bias were less than the issues of training GANs in discrete data.
GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must
first surpass.
This way, OpenAI's GPT and GPT-2 may be considered superior in text generation quality to current textual GANs.
answered Mar 26 at 9:10
ncasasncasas
3,7381131
$endgroup$
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48005%2fwhich-is-better-gpt-or-relgan-for-text-generation%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper:
This research demonstrates that well-adjusted language models are a remarkably strong baseline
and that temperature sweeping can provide a very clear characterization of model performance. A
well-adjusted language model outperforms the considered GAN variants as evaluated on both local,
and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework
shares characteristics with a Receiver Operating Curve. Analogously, if one needed a single scalar to
compare NLG models, one could compute area under the curve and seek the model with the smallest
value (lower is better for our considered metrics).
GAN-based generative models have been proven effective on real-valued data, however, but there
exist many difficult pernicious issues of moving to discrete data. These issues must be overcome
before they will improve over the strong MLE baselines. On the datasets and tasks considered,
potential issues caused by exposure bias were less than the issues of training GANs in discrete data.
GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must
first surpass.
This way, OpenAI's GPT and GPT-2 may be considered superior in text generation quality to current textual GANs.
answered Mar 26 at 9:10
ncasasncasas
3,7381131
$endgroup$
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
add a comment |
$begingroup$
According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper:
This research demonstrates that well-adjusted language models are a remarkably strong baseline
and that temperature sweeping can provide a very clear characterization of model performance. A
well-adjusted language model outperforms the considered GAN variants as evaluated on both local,
and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework
shares characteristics with a Receiver Operating Curve. Analogously, if one needed a single scalar to
compare NLG models, one could compute area under the curve and seek the model with the smallest
value (lower is better for our considered metrics).
GAN-based generative models have been proven effective on real-valued data, however, but there
exist many difficult pernicious issues of moving to discrete data. These issues must be overcome
before they will improve over the strong MLE baselines. On the datasets and tasks considered,
potential issues caused by exposure bias were less than the issues of training GANs in discrete data.
GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must
first surpass.
This way, OpenAI's GPT and GPT-2 may be considered superior in text generation quality to current textual GANs.
answered Mar 26 at 9:10
ncasasncasas
3,7381131
$endgroup$
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
add a comment |
$begingroup$
According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper:
This research demonstrates that well-adjusted language models are a remarkably strong baseline
and that temperature sweeping can provide a very clear characterization of model performance. A
well-adjusted language model outperforms the considered GAN variants as evaluated on both local,
and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework
shares characteristics with a Receiver Operating Curve. Analogously, if one needed a single scalar to
compare NLG models, one could compute area under the curve and seek the model with the smallest
value (lower is better for our considered metrics).
GAN-based generative models have been proven effective on real-valued data, however, but there
exist many difficult pernicious issues of moving to discrete data. These issues must be overcome
before they will improve over the strong MLE baselines. On the datasets and tasks considered,
potential issues caused by exposure bias were less than the issues of training GANs in discrete data.
GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must
first surpass.
This way, OpenAI's GPT and GPT-2 may be considered superior in text generation quality to current textual GANs.
answered Mar 26 at 9:10
ncasasncasas
3,7381131
$endgroup$
According to [Caccia et al., 2018], in general textual GANs are no rival for LMs regarding several quality measures. These are the conclusions of the paper:
This research demonstrates that well-adjusted language models are a remarkably strong baseline
and that temperature sweeping can provide a very clear characterization of model performance. A
well-adjusted language model outperforms the considered GAN variants as evaluated on both local,
and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework
shares characteristics with a Receiver Operating Curve. Analogously, if one needed a single scalar to
compare NLG models, one could compute area under the curve and seek the model with the smallest
value (lower is better for our considered metrics).
GAN-based generative models have been proven effective on real-valued data, however, but there
exist many difficult pernicious issues of moving to discrete data. These issues must be overcome
before they will improve over the strong MLE baselines. On the datasets and tasks considered,
potential issues caused by exposure bias were less than the issues of training GANs in discrete data.
GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must
first surpass.
This way, OpenAI's GPT and GPT-2 may be considered superior in text generation quality to current textual GANs.
answered Mar 26 at 9:10
ncasasncasas
3,7381131
answered Mar 26 at 9:10
ncasasncasas
3,7381131
answered Mar 26 at 9:10
ncasasncasas
3,7381131
answered Mar 26 at 9:10
ncasasncasas
3,7381131
3,7381131
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
add a comment |
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
$begingroup$
Thank you very much.
$endgroup$
– 不是phd的phd
Mar 27 at 1:07
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48005%2fwhich-is-better-gpt-or-relgan-for-text-generation%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
arxiv.org/abs/1810.12686
$endgroup$
– 不是phd的phd
Mar 28 at 2:14