keras mnist.load_data() is superslow and throw an error after some times
Date : March 29 2020, 07:55 AM
wish of those help I wrote how I fixed the problem in a comment but I want to make it more clear.
The problem happened only when the program was lauched from idle AND if it was the first time you would import mnist data from keras.
|
In which folder on PC (Windows 10) does load_data() save a dataset in Keras?
Tag : python , By : SachinJadhav
Date : March 29 2020, 07:55 AM
I wish this helpful for you By default the download folder is C:\Users\ \.keras\datasets
After I run the following code, I can see the following files in that folder:from keras.datasets import cifar100
(x_train, y_train), (x_test, y_test) = cifar100.load_data(label_mode='fine')
|
What's the difference between keras.datasets.mnist and tensorflow.examples.tutorials.mnist?
Date : March 29 2020, 07:55 AM
will be helpful for those in need It is very likely that the images in tensorflow.examples.tutorials.mnist have been normalized to the range [0, 1] and therefore you obtain better results. Whereas, the values in MNIST dataset in Keras are in the range [0, 255] and you are expected to normalize them (if needed, of course). Try this: (xtr, ytr), (xte, yte) = mnist.load_data()
xtr = xtr.astype('float32') / 255.0
xte = xte.astype('float32') / 255.0
|
How does Keras load_data() know what part of the data is the train and test set?
Date : March 29 2020, 07:55 AM
Any of those help The best way to find out is looking at Kera's code: def load_data(path='mnist.npz'):
path = get_file(path, origin='https://s3.amazonaws.com/img-datasets/mnist.npz', file_hash='8a61469f7ea1b51cbae51d4f78837e45')
with np.load(path, allow_pickle=True) as f:
x_train, y_train = f['x_train'], f['y_train']
x_test, y_test = f['x_test'], f['y_test']
return (x_train, y_train), (x_test, y_test)
|
Import own data like MNIST or CIFAR10 load_data()
Date : March 29 2020, 07:55 AM
fixed the issue. Will look into that further The object np.array was shape l,b,c at this point np_array = np.asarray(img) you then reshaped it with np_array = np_array.reshape(l*b*c,) which is what you didn't want. Just remove those 2 lines Also since you label is always 0 no need to have it append in the loop, just return it. def get_data(path):
all_images_as_array=[]
for filename in os.listdir(path):
img=Image.open(path + filename)
np_array = np.asarray(img)
all_images_as_array.append(np_array)
all_images = np.array(all_images_as_array)
return all_images, np.zeros_like(all_images)
|
Boards Message :
Related QUESTIONS :