convLSTM : how to structure input dataCNN for classification giving extreme result probabilitiesClassification of obfuscated text dataHow to import image data into python for keras?How to describe the process of using existing weights for training a deep learning model?Is it possible to pass multiple data generators for multiple inputs to model.fit_generator functionKeras CNN image input and outputData Augmentation recommended pipelineFeed-forward neural network not training with Keras function generators deep_learning data_science machine_learning pythonHow to optimally train deep learning model using output as new inputDefining Input Shape for Time Series using LSTM in Keras

Force user to remove USB token

Giving EXEC (@Variable) a Column name and Concatenation

Is it possible to upcast ritual spells?

What are some nice/clever ways to introduce the tonic's dominant seventh chord?

Backup with Hanoi Strategy

Equation Array Exceed Right Margin

Problems with making formula look great

Python: Check if string and its substring are existing in the same list

When do we add an hyphen (-) to a complex adjective word?

How to explain that I do not want to visit a country due to personal safety concern?

Instead of Universal Basic Income, why not Universal Basic NEEDS?

Why did it take so long to abandon sail after steamships were demonstrated?

Happy pi day, everyone!

Create shipment and invoice in mass action

Increase thickness of graph lines larger than ultra thick

If I can solve Sudoku can I solve TSP? If yes, how?

Is it normal that my co-workers at a fitness company criticize my food choices?

Official degrees of earth’s rotation per day

Why would a flight no longer considered airworthy be redirected like this?

Why doesn't the EU now just force the UK to choose between referendum and no-deal?

Be in awe of my brilliance!

Science-fiction short story where space navy wanted hospital ships and settlers had guns mounted everywhere

What has been your most complicated TikZ drawing?

Is it true that good novels will automatically sell themselves on Amazon (and so on) and there is no need for one to waste time promoting?



convLSTM : how to structure input data


CNN for classification giving extreme result probabilitiesClassification of obfuscated text dataHow to import image data into python for keras?How to describe the process of using existing weights for training a deep learning model?Is it possible to pass multiple data generators for multiple inputs to model.fit_generator functionKeras CNN image input and outputData Augmentation recommended pipelineFeed-forward neural network not training with Keras function generators deep_learning data_science machine_learning pythonHow to optimally train deep learning model using output as new inputDefining Input Shape for Time Series using LSTM in Keras













1












$begingroup$


I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC :



 fileName var_t+15m var_t+30m var_t+45m var_t+60m var_t+90m var_t+120m var_t+180m var_t+240
id
2016-10-15 15:00:00 201610151500.jpg 211.00 197.80 170.80 66.90 34.2000 10.120000 0.000867 0.001267
2016-10-15 15:15:00 201610151515.jpg 197.80 170.80 66.90 71.75 20.1600 2.120000 0.001534 0.000534
2016-10-15 15:30:00 201610151530.jpg 170.80 66.90 71.75 34.20 10.1200 0.206200 0.001000 0.001067
2016-10-15 15:45:00 201610151545.jpg 66.90 71.75 34.20 20.16 2.1200 0.012270 0.000400 0.000733
2016-10-15 16:00:00 201610151600.jpg 71.75 34.20 20.16 10.12 0.2062 0.000867 0.001267 0.000934


The task consists in predicting a certain variable at t+X where X goes from 15 minutes up to 240 minutes. So this is a regression task where my training input consists in timestamped picture.



In order to work with these data, I was until now using the .flow_from_dataframe method from Keras in order to perform data augmentation/pre-processing easily and to avoid loading the entire training set consisting of pictures inside the memory.



Up until now I did not leverage the time information and to do so I would like to try the convLSTM model available in Keras. Howevever I am very unfamilar with working with time series.



Has someone used the Keras convLSTM layer combined with the .flow_from_dataframe function ? I am unsure how to structure my data for this setup (convLST + .flow_from_dataframe) and I could not find an example on the internet.










share|improve this question









$endgroup$
















    1












    $begingroup$


    I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC :



     fileName var_t+15m var_t+30m var_t+45m var_t+60m var_t+90m var_t+120m var_t+180m var_t+240
    id
    2016-10-15 15:00:00 201610151500.jpg 211.00 197.80 170.80 66.90 34.2000 10.120000 0.000867 0.001267
    2016-10-15 15:15:00 201610151515.jpg 197.80 170.80 66.90 71.75 20.1600 2.120000 0.001534 0.000534
    2016-10-15 15:30:00 201610151530.jpg 170.80 66.90 71.75 34.20 10.1200 0.206200 0.001000 0.001067
    2016-10-15 15:45:00 201610151545.jpg 66.90 71.75 34.20 20.16 2.1200 0.012270 0.000400 0.000733
    2016-10-15 16:00:00 201610151600.jpg 71.75 34.20 20.16 10.12 0.2062 0.000867 0.001267 0.000934


    The task consists in predicting a certain variable at t+X where X goes from 15 minutes up to 240 minutes. So this is a regression task where my training input consists in timestamped picture.



    In order to work with these data, I was until now using the .flow_from_dataframe method from Keras in order to perform data augmentation/pre-processing easily and to avoid loading the entire training set consisting of pictures inside the memory.



    Up until now I did not leverage the time information and to do so I would like to try the convLSTM model available in Keras. Howevever I am very unfamilar with working with time series.



    Has someone used the Keras convLSTM layer combined with the .flow_from_dataframe function ? I am unsure how to structure my data for this setup (convLST + .flow_from_dataframe) and I could not find an example on the internet.










    share|improve this question









    $endgroup$














      1












      1








      1





      $begingroup$


      I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC :



       fileName var_t+15m var_t+30m var_t+45m var_t+60m var_t+90m var_t+120m var_t+180m var_t+240
      id
      2016-10-15 15:00:00 201610151500.jpg 211.00 197.80 170.80 66.90 34.2000 10.120000 0.000867 0.001267
      2016-10-15 15:15:00 201610151515.jpg 197.80 170.80 66.90 71.75 20.1600 2.120000 0.001534 0.000534
      2016-10-15 15:30:00 201610151530.jpg 170.80 66.90 71.75 34.20 10.1200 0.206200 0.001000 0.001067
      2016-10-15 15:45:00 201610151545.jpg 66.90 71.75 34.20 20.16 2.1200 0.012270 0.000400 0.000733
      2016-10-15 16:00:00 201610151600.jpg 71.75 34.20 20.16 10.12 0.2062 0.000867 0.001267 0.000934


      The task consists in predicting a certain variable at t+X where X goes from 15 minutes up to 240 minutes. So this is a regression task where my training input consists in timestamped picture.



      In order to work with these data, I was until now using the .flow_from_dataframe method from Keras in order to perform data augmentation/pre-processing easily and to avoid loading the entire training set consisting of pictures inside the memory.



      Up until now I did not leverage the time information and to do so I would like to try the convLSTM model available in Keras. Howevever I am very unfamilar with working with time series.



      Has someone used the Keras convLSTM layer combined with the .flow_from_dataframe function ? I am unsure how to structure my data for this setup (convLST + .flow_from_dataframe) and I could not find an example on the internet.










      share|improve this question









      $endgroup$




      I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC :



       fileName var_t+15m var_t+30m var_t+45m var_t+60m var_t+90m var_t+120m var_t+180m var_t+240
      id
      2016-10-15 15:00:00 201610151500.jpg 211.00 197.80 170.80 66.90 34.2000 10.120000 0.000867 0.001267
      2016-10-15 15:15:00 201610151515.jpg 197.80 170.80 66.90 71.75 20.1600 2.120000 0.001534 0.000534
      2016-10-15 15:30:00 201610151530.jpg 170.80 66.90 71.75 34.20 10.1200 0.206200 0.001000 0.001067
      2016-10-15 15:45:00 201610151545.jpg 66.90 71.75 34.20 20.16 2.1200 0.012270 0.000400 0.000733
      2016-10-15 16:00:00 201610151600.jpg 71.75 34.20 20.16 10.12 0.2062 0.000867 0.001267 0.000934


      The task consists in predicting a certain variable at t+X where X goes from 15 minutes up to 240 minutes. So this is a regression task where my training input consists in timestamped picture.



      In order to work with these data, I was until now using the .flow_from_dataframe method from Keras in order to perform data augmentation/pre-processing easily and to avoid loading the entire training set consisting of pictures inside the memory.



      Up until now I did not leverage the time information and to do so I would like to try the convLSTM model available in Keras. Howevever I am very unfamilar with working with time series.



      Has someone used the Keras convLSTM layer combined with the .flow_from_dataframe function ? I am unsure how to structure my data for this setup (convLST + .flow_from_dataframe) and I could not find an example on the internet.







      deep-learning keras lstm cnn






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 17 hours ago









      FenryrMKIIIFenryrMKIII

      1163




      1163




















          0






          active

          oldest

          votes











          Your Answer





          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "557"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47313%2fconvlstm-how-to-structure-input-data%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Data Science Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47313%2fconvlstm-how-to-structure-input-data%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Adding axes to figuresAdding axes labels to LaTeX figuresLaTeX equivalent of ConTeXt buffersRotate a node but not its content: the case of the ellipse decorationHow to define the default vertical distance between nodes?TikZ scaling graphic and adjust node position and keep font sizeNumerical conditional within tikz keys?adding axes to shapesAlign axes across subfiguresAdding figures with a certain orderLine up nested tikz enviroments or how to get rid of themAdding axes labels to LaTeX figures

          Tähtien Talli Jäsenet | Lähteet | NavigointivalikkoSuomen Hippos – Tähtien Talli

          Do these cracks on my tires look bad? The Next CEO of Stack OverflowDry rot tire should I replace?Having to replace tiresFishtailed so easily? Bad tires? ABS?Filling the tires with something other than air, to avoid puncture hassles?Used Michelin tires safe to install?Do these tyre cracks necessitate replacement?Rumbling noise: tires or mechanicalIs it possible to fix noisy feathered tires?Are bad winter tires still better than summer tires in winter?Torque converter failure - Related to replacing only 2 tires?Why use snow tires on all 4 wheels on 2-wheel-drive cars?