logo
down
shadow

How can one use TensorFlow.js tf.data.generator for remote data sources since generators can't use callbacks


How can one use TensorFlow.js tf.data.generator for remote data sources since generators can't use callbacks

Content Index :

How can one use TensorFlow.js tf.data.generator for remote data sources since generators can't use callbacks
Tag : javascript , By : Kirks
Date : December 02 2020, 10:46 PM

To fix this issue You can pass an async function (or a function returning a Promise) to the generator. It is then okay to use await inside the function (even inside a loop) to handle any asynchronous tasks.
Code Sample
const dataset = tf.data.generator(async function* () {
    const dataToDownload = await fetch(/* ... */);
    while (/* ... */) {
        const moreData = await fetch(/* ... */);
        yield otherData;
    }
});
async function* foo(index) {
  while (true) {
    yield index++;
  }
}

(async () => {
  const iterator = foo(0);
  console.log((await iterator.next()).value); // 0
  console.log((await iterator.next()).value); // 1
})();

Comments
No Comments Right Now !

Boards Message :
You Must Login Or Sign Up to Add Your Comments .

Share : facebook icon twitter icon

Hadoop remote data sources connectivity


Tag : hadoop , By : Munir
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , What are the available hadoop remote data sources connectivity options? , These are the few I am aware of :
Oracle

Load up multiple remote data sources for knockout.js data binding


Tag : javascript , By : user118656
Date : March 29 2020, 07:55 AM
hope this fix your issue You can use lazy loading observable to get data into your viewModel observables, and computed to subscribe on load of the parent level observables.
function ViewModel() {
   this.workout = ko.onDemandObservable(ViewModel.prototype.getWorkout, this);
   this.diet = ko.onDemandObservable(ViewModel.prototype.getDiet, this);
   this.feedback= ko.onDemandObservable(ViewModel.prototype.getFeedback, this);
   this.suggestedWorkouts = ko.observable();

   ko.computed(ViewModel.prototype.listsLoaded, this);
}

ViewModel.prototype.listsLoaded= function () {    
   if (this.workout.loaded() && this.diet.loaded() && this.feedback.loaded()) {
        this.loadSuggestedWorkouts();
   }
}

ViewModel.prototype.getWorkout = function () {
   ...
}

ViewModel.prototype.getDiet = function () {
   ...
}

ViewModel.prototype.getFeedback = function () {
   ...
}

ViewModel.prototype.loadSuggestedWorkouts = function () {
  ...
}

Understanding tensorflow behavior when fetching Data from two different data sources with dataset api


Tag : python , By : Dave M
Date : March 29 2020, 07:55 AM
seems to work fine Here is how to fetch data from multiple dataset separately. Yet I would like know the answer for my other questions regarding the behavior of tensorflow and why should data2 = iter2.get_next() should be defined within a method.
import tensorflow as tf
import numpy as np

d1 = tf.data.Dataset.range(1, 1000)
iter1 = d1.make_initializable_iterator()

d2 = tf.data.Dataset.range(1000, 2000)
iter2 = d2.make_initializable_iterator()

d3 = tf.data.Dataset.range(2000, 3000)
iter3 = d3.make_initializable_iterator()

d4 = tf.data.Dataset.range(3000, 4000)
iter4 = d4.make_initializable_iterator()

def return_data1_2():
    data1 = iter1.get_next()
    data2 = iter2.get_next()
    return data1, data2

def return_data2_3():
    data2 = iter2.get_next()
    data3 = iter3.get_next()
    return data2, data3

def return_data3_4():
    data3 = iter3.get_next()
    data4 = iter4.get_next()
    return data3, data4

def return_data4_1():
    data4 = iter4.get_next()
    data1 = iter1.get_next()
    return data4, data1

index1 = tf.placeholder(dtype=tf.int32)
index2 = tf.placeholder(dtype=tf.int32)

data = tf.case(pred_fn_pairs=[
    (tf.logical_and(tf.equal(index1, 1), tf.equal(index2, 2)), lambda: return_data1_2()), 
    (tf.logical_and(tf.equal(index1, 2), tf.equal(index2, 3)), lambda: return_data2_3()),
    (tf.logical_and(tf.equal(index1, 3), tf.equal(index2, 4)), lambda: return_data3_4()),
    (tf.logical_and(tf.equal(index1, 4), tf.equal(index2, 1)), lambda: return_data4_1())], exclusive=False)

init_op = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init_op)
    sess.run([iter1.initializer, iter2.initializer, iter3.initializer, iter4.initializer])


    for i in range(2000):
        try:
            if i < 500:
                print(sess.run(data, feed_dict={index1: 1, index2: 2}), "1-2")
            elif i < 1000:
                print(sess.run(data, feed_dict={index1: 2, index2: 3}), "2-3")
            elif i < 1500:
                print(sess.run(data, feed_dict={index1: 3, index2: 4}), "3-4")
            elif i < 2000:
                print(sess.run(data, feed_dict={index1: 4, index2: 1}), "4-1")
        except tf.errors.OutOfRangeError as error:
            print("error")

Could validation data be a generator in tensorflow.keras 2.0?


Tag : python , By : tlync
Date : March 29 2020, 07:55 AM
it should still fix some issue Yes it can, that's strange that it is not in the doc but is it working exactly like the x argument, you can also use a keras.Sequence or a generator. In my project I often use keras.Sequence that acts like a generator
Minimum working example that shows that it works :
import numpy as np
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Flatten

def generator(batch_size): # Create empty arrays to contain batch of features and labels
    batch_features = np.zeros((batch_size, 1000))
    batch_labels = np.zeros((batch_size,1))
    while True:
        for i in range(batch_size):
            yield batch_features, batch_labels

model = Sequential()
model.add(Dense(125, input_shape=(1000,), activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

train_generator = generator(64)
validation_generator = generator(64)

model.fit(train_generator, validation_data=validation_generator, validation_steps=100, epochs=100, steps_per_epoch=100)

Hibernate can't instantiate id generator in Spring project with multiple data sources


Tag : java , By : Salikh
Date : March 29 2020, 07:55 AM
Related Posts Related QUESTIONS :
  • Python GEE to extract featurecollection timeseries from Landsat imageries
  • How to write negative binary number?
  • Unable to set property of individual object
  • Why Named Function Expression itself cannot assign Name to another Value?
  • How to reset scroll position on single page pagination
  • How to scroll to bottom of page, when a new message is sent - Socket.io
  • How to reconstruct audio blob from a base64 encoded String?
  • How to calculate numbers within an array of objects
  • Using JavaScript to change CSS properties of SVG circles for all circles that was not clicked
  • Is there value in propTypeing repeat required properties at every level of nested components?
  • Call function if variable does not exist in a filter
  • localStorage value doesn't get updated automatically
  • React not updating state with setState correctly inside promises
  • Direct native JavaScript or jQuery method to get containing block of an element
  • Dynamic JSON object to Html Table
  • How to send Id and file to same controller?
  • How to Make a jQuery Table Cell(td) clickable to Run a Function?
  • Prevent the duplicate function onchange event jQuery
  • Reactjs Input Upload doesn't return local URL
  • An unexpected 'StartObject' node was found when reading from the JSON reader. A 'PrimitiveValue' node was expected
  • How to filter a JSON Array, with an eventListener input value at keyup
  • How to separate the different properties of filter in css?
  • Why dont work change event of bootstrap input spinner?
  • How to hide overflow text content in react
  • Is there another way to write line 6
  • Cypress - how to find by text content?
  • How to setstate for the dynamic checkboxes in a loop in react?
  • Is display:none safe?
  • How to create a form to change part of a href url of a button?
  • Put many function inside function in Javascript
  • how to exit from loop after a ajax call
  • Do Angular Reactive Form Validator.min/max check Perform Type Conversions before checking the value?
  • How to Replace Text Within Script Tags Using JQuery
  • add variable to object if not undefined using short-hand
  • How can I effectively curve my items inside a list?
  • Add new record Keyboard Shortcut on Kendo Grid Mvc
  • Time and Date running on page
  • Looping through page links puppeteer doesn't return values from the newly loaded page
  • Problem with Navigator.geoLocation.getCurrentPosition not working
  • Trying to change the id of an element with Javascript
  • Embedded Schema Inserting Blank in Meteor using Collection2
  • avoid useEffect to render at first load?
  • Why is this CSS transition only working correctly on the some clicks? How to solve it?
  • Interpolated values does not reflect the latest values in an Angular app
  • Javascript classes define attribute in constructor or outside the class
  • How to block scroll event passing through to iframe until iframe is active?
  • What is the purpose of "constructor" method in javascript?
  • Stencil JS not working with non-default imports
  • Are frameworks misleadingly named for marketing purposes?
  • React <Redirect> after transition not working
  • How to send multiple parameters in a function using EJS
  • React conditional style on custom click
  • Push a block of information to an array X times based on an external variable
  • How do I populate dropdown with names, and values (numeric ids)?
  • How to apply styling to elements created with .insertAdjacentHtml method
  • Positioning key:value of an object
  • Referencing DOM Elements with React Hooks
  • TypeError: Method "getRange" of null when using multiple scripts (Libraries) in Google scripts
  • Why can't I check if an environment variable is undefined?
  • Is there a way to find what parts are the same between two strings?
  • shadow
    Privacy Policy - Terms - Contact Us © scrbit.com