Trjtdtk

Examples of non trivial equivalence relations , I mean equivalence relations without the expression " same ... as" in their definition?

How much cash can I safely carry into the USA and avoid civil forfeiture?

Why do games have consumables?

What is the most expensive material in the world that could be used to create Pun-Pun's lute?

How to make a pipeline wait for end-of-file or stop after an error?

Why isn't the definition of absolute value applied when squaring a radical containing a variable?

How did Captain America manage to do this?

Minor Revision with suggestion of an alternative proof by reviewer

How to creep the reader out with what seems like a normal person?

Was there a shared-world project before "Thieves World"?

Was there a Viking Exchange as well as a Columbian one?

Fizzy, soft, pop and still drinks

a sore throat vs a strep throat vs strep throat

What language was spoken in East Asia before Proto-Turkic?

Examples of subgroups where it's nontrivial to show closure under multiplication?

A Strange Latex Symbol

How do I reattach a shelf to the wall when it ripped out of the wall?

Why other Westeros houses don't use wildfire?

Why do Computer Science majors learn Calculus?

Can someone publish a story that happened to you?

How would one muzzle a full grown polar bear in the 13th century?

Is the claim "Employers won't employ people with no 'social media presence'" realistic?

How to have a sharp product image?

Which big number is bigger?

Keras NN - Learning a simple formula

Reshaping of data for deep learning using KerasConsistently inconsistent cross-validation results that are wildly different from original model accuracyIndex error in simple keras modelKeras LSTM: use weights from Keras model to replicate predictions using numpySimple neural network implementation in kerasLearning a simple sequence with RNN (Keras)Simple prediction with KerasValueError: Error when checking target: expected dense_2 to have shape (1,) but got array with shape (0,)Keras Conv1D for simple data target predictionproblem of entry format for a simple model in Keras

0

$begingroup$

I'm struggling with a seemingly simple problem and could really use your help! I'm starting to learn about neural networks. I thought I would create a simple example to get started, using one dimensional inputs X and one dimensional outputs Y. When I first started with Y = 2X, the neural network was able to learn it very well over 1000 epochs. However, when I increased the complexity of the formula to below, it completely breaks apart (the MSE is very high).

X = random numbers between 0 and 100 (one-dimensional)

Y = (8(x^3)) + 5

I have tried adding more hidden layers and epochs without much success. Could anyone provide me with some guidance on where I'm going wrong in my method?

Much appreciated!

Code:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
# fix random seed for reproducibility
np.random.seed(7)

def generate_Y(X):
 return np.add(np.power(np.multiply(X, 2.0), 3.0), 5.0).T[0]

X = np.random.rand(100,1)
X = np.multiply(X, 100.0)
Y = generate_Y(X)

# create model
model = Sequential()
model.add(Dense(12, input_dim=1))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(1))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
# Fit the model
model.fit(X, Y, epochs=10000, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)
print("n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

test_X = np.random.rand(10,1)
test_X = np.multiply(test_X, 500.0)
test_Y = generate_Y(test_X)
preds = model.predict(test_X)

print preds
diffs = np.subtract(preds.T[0], test_Y)
print diffs

MSE: mean_squared_error: 73360074604544.00%

Prediction vs actual diffs:
[ 4.97601229e+05 -1.60322447e+06 -7.85835700e+08 -7.00977203e+08
-7.52094155e+08 -1.88780991e+08 -1.64643340e+08 -9.38229901e+08
-2.02173897e+08 4.45327360e+05]

python neural-network keras

share|improve this question

asked Apr 7 at 9:03

RB25RB25

1

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You are not using any of the activation functions like sigmoid or ReLU.
  $endgroup$
  – Shubham Panchal
  Apr 7 at 10:58
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I thought the sigmoid function was better used for classification rather than regression problems? I tried adding those activation functions and it didn't improve the result.
  $endgroup$
  – RB25
  Apr 7 at 11:10
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  The formula you are trying to learn is clearly non linear, therefore you have to use some non linearity activation as @shubam Panchal suggested. Please show the code after this update.
  $endgroup$
  – Francesco Pegoraro
  Apr 7 at 14:23
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  and generate_Yreturns just a value... are you shure about the usage of .T[0]?
  $endgroup$
  – Francesco Pegoraro
  Apr 7 at 14:38
  

  
  
  
  

add a comment | 

0

$begingroup$

I'm struggling with a seemingly simple problem and could really use your help! I'm starting to learn about neural networks. I thought I would create a simple example to get started, using one dimensional inputs X and one dimensional outputs Y. When I first started with Y = 2X, the neural network was able to learn it very well over 1000 epochs. However, when I increased the complexity of the formula to below, it completely breaks apart (the MSE is very high).

X = random numbers between 0 and 100 (one-dimensional)

Y = (8(x^3)) + 5

I have tried adding more hidden layers and epochs without much success. Could anyone provide me with some guidance on where I'm going wrong in my method?

Much appreciated!

Code:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
# fix random seed for reproducibility
np.random.seed(7)

def generate_Y(X):
 return np.add(np.power(np.multiply(X, 2.0), 3.0), 5.0).T[0]

X = np.random.rand(100,1)
X = np.multiply(X, 100.0)
Y = generate_Y(X)

# create model
model = Sequential()
model.add(Dense(12, input_dim=1))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(1))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
# Fit the model
model.fit(X, Y, epochs=10000, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)
print("n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

test_X = np.random.rand(10,1)
test_X = np.multiply(test_X, 500.0)
test_Y = generate_Y(test_X)
preds = model.predict(test_X)

print preds
diffs = np.subtract(preds.T[0], test_Y)
print diffs

MSE: mean_squared_error: 73360074604544.00%

Prediction vs actual diffs:
[ 4.97601229e+05 -1.60322447e+06 -7.85835700e+08 -7.00977203e+08
-7.52094155e+08 -1.88780991e+08 -1.64643340e+08 -9.38229901e+08
-2.02173897e+08 4.45327360e+05]

python neural-network keras

share|improve this question

asked Apr 7 at 9:03

RB25RB25

1

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You are not using any of the activation functions like sigmoid or ReLU.
  $endgroup$
  – Shubham Panchal
  Apr 7 at 10:58
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I thought the sigmoid function was better used for classification rather than regression problems? I tried adding those activation functions and it didn't improve the result.
  $endgroup$
  – RB25
  Apr 7 at 11:10
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  The formula you are trying to learn is clearly non linear, therefore you have to use some non linearity activation as @shubam Panchal suggested. Please show the code after this update.
  $endgroup$
  – Francesco Pegoraro
  Apr 7 at 14:23
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  and generate_Yreturns just a value... are you shure about the usage of .T[0]?
  $endgroup$
  – Francesco Pegoraro
  Apr 7 at 14:38
  

  
  
  
  

add a comment | 

$begingroup$

I'm struggling with a seemingly simple problem and could really use your help! I'm starting to learn about neural networks. I thought I would create a simple example to get started, using one dimensional inputs X and one dimensional outputs Y. When I first started with Y = 2X, the neural network was able to learn it very well over 1000 epochs. However, when I increased the complexity of the formula to below, it completely breaks apart (the MSE is very high).

X = random numbers between 0 and 100 (one-dimensional)

Y = (8(x^3)) + 5

I have tried adding more hidden layers and epochs without much success. Could anyone provide me with some guidance on where I'm going wrong in my method?

Much appreciated!

Code:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
# fix random seed for reproducibility
np.random.seed(7)

def generate_Y(X):
 return np.add(np.power(np.multiply(X, 2.0), 3.0), 5.0).T[0]

X = np.random.rand(100,1)
X = np.multiply(X, 100.0)
Y = generate_Y(X)

# create model
model = Sequential()
model.add(Dense(12, input_dim=1))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(1))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
# Fit the model
model.fit(X, Y, epochs=10000, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)
print("n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

test_X = np.random.rand(10,1)
test_X = np.multiply(test_X, 500.0)
test_Y = generate_Y(test_X)
preds = model.predict(test_X)

print preds
diffs = np.subtract(preds.T[0], test_Y)
print diffs

MSE: mean_squared_error: 73360074604544.00%

Prediction vs actual diffs:
[ 4.97601229e+05 -1.60322447e+06 -7.85835700e+08 -7.00977203e+08
-7.52094155e+08 -1.88780991e+08 -1.64643340e+08 -9.38229901e+08
-2.02173897e+08 4.45327360e+05]

python neural-network keras

share|improve this question

asked Apr 7 at 9:03

RB25RB25

1

$endgroup$

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
# fix random seed for reproducibility
np.random.seed(7)

def generate_Y(X):
 return np.add(np.power(np.multiply(X, 2.0), 3.0), 5.0).T[0]

X = np.random.rand(100,1)
X = np.multiply(X, 100.0)
Y = generate_Y(X)

# create model
model = Sequential()
model.add(Dense(12, input_dim=1))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(12))
model.add(Dense(1))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
# Fit the model
model.fit(X, Y, epochs=10000, batch_size=10)
# evaluate the model
scores = model.evaluate(X, Y)
print("n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

test_X = np.random.rand(10,1)
test_X = np.multiply(test_X, 500.0)
test_Y = generate_Y(test_X)
preds = model.predict(test_X)

print preds
diffs = np.subtract(preds.T[0], test_Y)
print diffs

share|improve this question

asked Apr 7 at 9:03

RB25RB25

1

share|improve this question

asked Apr 7 at 9:03

RB25RB25

1

asked Apr 7 at 9:03

RB25RB25

1

asked Apr 7 at 9:03

RB25RB25

1