logo
down
shadow

When making network requests, when should I use Threads vs Processes?


When making network requests, when should I use Threads vs Processes?

Content Index :

When making network requests, when should I use Threads vs Processes?
Tag : ruby , By : Giles
Date : November 28 2020, 08:01 AM


Comments
No Comments Right Now !

Boards Message :
You Must Login Or Sign Up to Add Your Comments .

Share : facebook icon twitter icon

Qt application crashes when making 2 network requests from 2 threads


Tag : qt , By : Ernie
Date : March 29 2020, 07:55 AM
Does that help First of all you're doing it wrong! Fix your threading first
// EDIT From my own experience with this pattern i know that it may lead to many unclear crashes. I would start from clearing this thing out, as it may straighten some things and make finding problem clear. Also I don't know how do you invoke makeRequest. Also about QNetworkRequest. It is only a data structure so you don't need to make it on heap. Stack construction would be enough. Also you should remember (or protect somehow) from overwriting m_reply pointer. Do you call makeRequest more than once? If you do, then it may lead to deleting currently processed request after previous request finished.

Python requests - threads/processes vs. IO


Tag : python , By : Hugo Hernan Buitrago
Date : March 29 2020, 07:55 AM
I wish this helpful for you Looking at your multiprocessing code at the top of the question. It seems that a HttpConnectionPool() is being called each time ReqOsrm is called. Thus a new pool is created for each url. Instead, use the initializer and args parameter to create a single pool for each process.
conn_pool = None

def makePool(host, port):
    global conn_pool
    pool = conn_pool = HTTPConnectionPool(host=host, port=port, maxsize=1)

def ReqOsrm(url_input):
    ul, qid = url_input

    try:
        response = conn_pool.request('GET', ul)
        json_geocode = json.loads(response.data.decode('utf-8'))
        status = int(json_geocode['status'])
        if status == 200:
            tot_time_s = json_geocode['route_summary']['total_time']
            tot_dist_m = json_geocode['route_summary']['total_distance']
            used_from, used_to = json_geocode['via_points']
            out = [qid, status, tot_time_s, tot_dist_m, used_from[0], used_from[1], used_to[0], used_to[1]]
            return out

        else:
            print("Done but no route: %d %s" % (qid, req_url))
            return [qid, 999, 0, 0, 0, 0, 0, 0]

    except Exception as err:
        print("%s: %d %s" % (err, qid, req_url))
        return [qid, 999, 0, 0, 0, 0, 0, 0]

if __name__ == "__main__":
    # run:
    pool = Pool(initializer=makePool, initargs=('127.0.0.1', 5005))
    calc_routes = pool.map(ReqOsrm, url_routes)
    pool.close()
    pool.join()
calc_routes = []
futures = {}
with FuturesSession(executor=ThreadPoolExecutor(max_workers=1000)) as session:
    # Submit all the requests and process in background
    for i in range(len(url_routes)):
        url_in, qid = url_routes[i]  # url |query-id
        future = session.get(url_in, background_callback=lambda sess, resp: ReqOsrm(sess, resp))
        futures[future] = qid

    # this was indented under the code in section B of the question
    # process the futures as they become copmlete
    for future in as_completed(futures):
        r = future.result()
        try:
            row = [futures[future]] + r.data

        except Exception as err:
            print('No route')
            row = [futures[future], 999, 0, 0, 0, 0, 0, 0]
        print(row)
        calc_routes.append(row)

Does Django use processes or threads to handle user requests in view?


Tag : django , By : user183842
Date : March 29 2020, 07:55 AM
I hope this helps . Django is run as a WSGI application. How that happens is determined by your WSGI server (e.g. uWSGI, Gunicorn, mod_wsgi).
Django's request handler is thread-safe. You can configure your WSGI server to use any number of processes (sometimes called workers) and threads per process.

Python - should I use threads or processes for network activity?


Tag : python , By : lietkynes
Date : March 29 2020, 07:55 AM
I hope this helps . So for real parallel work you would need multiprocessing, since threads only gives an advantage in a few cases (like IO).
As already pointed out, for your IO work threads should be fine, although you should look into multiprocessing, since from my experience it isn't hard to implement either.

Batch file to monitor a processes RAM, CPU%, Network data, threads


Tag : batch-file , By : user182548
Date : March 29 2020, 07:55 AM
Related Posts Related QUESTIONS :
  • How do I create a Class using the Singleton Design Pattern in Ruby?
  • How do I update Ruby Gems from behind a Proxy (ISA-NTLM)
  • How do I run Rake tasks within a Ruby script?
  • Mapping values from two array in Ruby
  • Reverse DNS in Ruby?
  • Why is aws-sdk-ruby connecting to '169.254.169.254' on initialization?
  • Rspec: Using Let with context correctly?
  • Algorithm for ‘spreading` sum in Hash
  • Capybara 'visit' is not considering '#' character in URL?
  • Call a Rake task repeatedly
  • ruby selenium execute_script Net::ReadTimeout
  • Right usage of ruby global variable $*
  • ruby-debug19: Can't get working with Ruby 1.9.1p376
  • How to retrieve caller context object in Ruby?
  • Conditionally assign return value of functions in one line
  • I want to convert translation curl command to ruby
  • Create inner-class object after executing outer-class's constructor in Ruby?
  • Implement a ruby method to find the next largest number with the same digits of the input number in ruby
  • Is there a way to create dynamic variable names in Ruby?
  • Detect <,>, >=,<= operators in Ruby and use in .where method (no active record)
  • How do we talk about classes that do not inherit from the `Class` class in Ruby?
  • Sum of first nth term of series
  • Space before first word after .join array to string
  • How to make alias_method in Ruby use the sub-class' custom method?
  • Can I specify a duck type in method signatures?
  • Ruby BCrypt salting/hashing seems ... wrong?
  • Not getting expected answer
  • Sorbet asking a `sig` for `attr_reader`
  • key: value vs key :value in ruby?
  • Ruby dig set - Assign values using Hash#dig
  • How to determine big O Time complexity of array difference in Ruby
  • How to resolve Rubocop error 'Favor `format` over `String#%`'
  • Blocks in Ruby, Error:Wrong Number of arguments
  • Reassigned hash changes the original hash
  • rspec hooks(before and after) is not working
  • Next-ing from child each loop in Ruby
  • convert byte to string in rails
  • What does the ruby ? method do?
  • ruby code error: '+'no implicit conversion of Integer into String
  • How to run the simplest unit test
  • How do I update controller param to accept nested attributes with rails?
  • How does the method reference operator .: work?
  • How to "magically" add code to all public class methods in ruby?
  • Trying to AND all elements in a list of lists
  • How can I assign a variable the largest 32 bit signed value in ruby without explicitly defining it
  • I'm trying to convert a value to an integer what am I doing wrong here?
  • Why do I get syntax errors when I deploy capistrano?
  • When is it a good idea to define a static ruby class?
  • How to find the number of unique occurrences for an Array in Ruby
  • In Sinatra, how to render json?
  • why do I have to reference the method instead of the @variable?
  • How to get table name for a simple Sequel Dataset object?
  • ruby exact number log (logarithm) function
  • Hexdecimal to String Convert - Ruby
  • Spreadsheet::Workbook.new creates .xls instead of xlsx
  • How to split a long number into pairs of digits
  • Ruby/Minitest Spec: How to test that a method returns a value within a given array?
  • Ruby Enumerable#count performance issue in algorithm
  • Ruby library to parse strings into the appropriate data type Google Sheets-style
  • Chaining method blocks (Ruby)
  • shadow
    Privacy Policy - Terms - Contact Us © scrbit.com