logo
down
shadow

Vectorizing string formatting across NumPy array


Vectorizing string formatting across NumPy array

Content Index :

Vectorizing string formatting across NumPy array
Tag : python , By : Robert MacGregor
Date : November 29 2020, 04:01 AM

Any of those help I have two integer arrays that I want to combine, per element, into a single array of strings of the form 'a[i]_b[i]'. That is, I have
fv = np.frompyfunc("{}_{}".format, 2, 1)
result = fv(a, b)  # array(['1_4', '2_5', '3_6'], dtype=object)
In [2]: a = np.arange(100000)

In [3]: b = np.arange(100000) + a.size

In [4]: fv = np.frompyfunc("{}_{}".format, 2, 1)

In [5]: def f(a, b): return np.array(["{}_{}".format(a,b) for a,b in zip(a,b)], dtype=object)

In [6]: %timeit f(a,b)
370 ms ± 12.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [7]: %timeit fv(a,b)
137 ms ± 1.48 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Comments
No Comments Right Now !

Boards Message :
You Must Login Or Sign Up to Add Your Comments .

Share : facebook icon twitter icon

Vectorizing operation on numpy array


Tag : python , By : Jesenko Mehmedbasic
Date : March 29 2020, 07:55 AM
will help you The problem is that just by using partial it doesn't make the existence of the other arguments go away for the sake of vectorize. The function underlying the partial object will be vectorizedWarpAffine.pyfunc, which will keep track of whatever pre-bound arguments you'd like it to use when calling vectorizedWarpAffine.pyfunc.func (which is still a multi-argumented function).
You can see it like this (after you import inspect):
In [19]: inspect.getargspec(vectorizedWarpAffine.pyfunc.func)
Out[19]: ArgSpec(args=['M', 'size', 'img'], varargs=None, keywords=None, defaults=None)
vectorizedWarpAffine = np.vectorize(partialWarpAffine, 
                                    excluded=set((0, 1)))
In [21]: vectorizedWarpAffine(data[:, 0])
OpenCV Error: Assertion failed (cn <= 4 && ssize.area() > 0) in remapBilinear, file -------src-dir-------/opencv-2.4.6.1/modules/imgproc/src/imgwarp.cpp, line 2296
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-21-3fb586393b75> in <module>()
----> 1 vectorizedWarpAffine(data[:, 0])

/home/ely/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.pyc in __call__(self, *args, **kwargs)
   1570             vargs.extend([kwargs[_n] for _n in names])
   1571 
-> 1572         return self._vectorize_call(func=func, args=vargs)
   1573 
   1574     def _get_ufunc_and_otypes(self, func, args):

/home/ely/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.pyc in _vectorize_call(self, func, args)
   1628         """Vectorized call to `func` over positional `args`."""
   1629         if not args:
-> 1630             _res = func()
   1631         else:
   1632             ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)

/home/ely/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.pyc in func(*vargs)
   1565                     the_args[_i] = vargs[_n]
   1566                 kwargs.update(zip(names, vargs[len(inds):]))
-> 1567                 return self.pyfunc(*the_args, **kwargs)
   1568 
   1569             vargs = [args[_i] for _i in inds]

/home/ely/programming/np_vect.py in <lambda>(M, size, img)
     10 size = (96, 96) # output image size
     11 
---> 12 warpAffine = lambda M, size, img : cv2.warpAffine(img, M, size) # re-order function parameters
     13 partialWarpAffine = partial(warpAffine, M, size)
     14 

error: -------src-dir-------/opencv-2.4.6.1/modules/imgproc/src/imgwarp.cpp:2296: error: (-215) cn <= 4 && ssize.area() > 0 in function remapBilinear
vectorizedWarpAffine = np.vectorize(warpAffine, 
                                    excluded=(0, 1), 
                                    otypes=[np.ndarray])
In [29]: vectorizedWarpAffine(M, size, data[:, 0])
Out[29]: 
array([[[ array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       ..., 
       ...

Nested For Loops Numpy Array: Is vectorizing possible?


Tag : python , By : Epora75
Date : March 29 2020, 07:55 AM
With these it helps Vectorizing won't help you much here, but avoiding repeated work will:
patterns = [re.compile('\\b'+name[idx]+'\\b') for idx in index]
for i, row in enumerate(data):
    for j, patt in enumerate(patterns):
        x[i, j] = len(patt.findall(row[1]))
        y[i, j] = len(patt.findall(row[2]))

Vectorizing NumPy covariance for 3D array


Tag : python , By : suresh
Date : March 29 2020, 07:55 AM
wish helps you Hacked into numpy.cov source code and tried using the default parameters. As it turns out, np.cov(x[i,:,:]) would be simply :
N = x.shape[2]
m = x[i,:,:]
m -= np.sum(m, axis=1, keepdims=True) / N
cov = np.dot(m, m.T)  /(N - 1)
N = x.shape[2]
m1 = x - x.sum(2,keepdims=1)/N
y_out = np.einsum('ijk,ilk->ijl',m1,m1) /(N - 1)
In [155]: def original_app(x):
     ...:     n = x.shape[0]
     ...:     y = np.zeros((n,2,2))
     ...:     for i in np.arange(x.shape[0]):
     ...:         y[i]=np.cov(x[i,:,:])
     ...:     return y
     ...: 
     ...: def proposed_app(x):
     ...:     N = x.shape[2]
     ...:     m1 = x - x.sum(2,keepdims=1)/N
     ...:     out = np.einsum('ijk,ilk->ijl',m1,m1)  / (N - 1)
     ...:     return out
     ...: 

In [156]: # Setup inputs
     ...: n = 10000
     ...: x = np.random.rand(n,2,4)
     ...: 

In [157]: np.allclose(original_app(x),proposed_app(x))
Out[157]: True  # Results verified

In [158]: %timeit original_app(x)
1 loops, best of 3: 610 ms per loop

In [159]: %timeit proposed_app(x)
100 loops, best of 3: 6.32 ms per loop

Vectorizing code of looping through NumPy 3D array


Tag : python , By : davidg
Date : March 29 2020, 07:55 AM
I wish did fix the issue. If img and truth_table are arrays where truth_table is of broadcastable shape:
img[truth_table == 12] = 0

Vectorizing Numpy 3D and 2D array operation


Tag : python , By : Aki Björklund
Date : March 29 2020, 07:55 AM
Does that help I'm trying to create K MxN matrices in Python, stored in a (M,N,K) numpy array, C, from two matrices, A and B, with shapes (K, M) and (K,N) respectively. The first matrix is computed as C0 = a0.T x b0, where a0 is the first row of A and b1 is the first row of B, the second matrix as C1 = a1.T x b0 and so on.
In [235]: A = np.random.random((10,800)) 
     ...: B = np.random.random((10,500)) 
     ...: C = np.zeros((800,500,10)) 
     ...: for k in range(10): 
     ...:     C[:,:,k] = A[k,:][:,None] @ B[k,:][None,:] 
     ...:                                                                            
In [236]: C.shape                                                                    
Out[236]: (800, 500, 10)
In [237]: np.allclose((A[:,:,None]@B[:,None,:]).transpose(1,2,0), C)                 
Out[237]: True
In [238]: np.allclose((A[:,:,None]*B[:,None,:]).transpose(1,2,0), C)                 
Out[238]: True
Related Posts Related QUESTIONS :
  • Pandas giving IndexError on one dataframe but not on another similar dataframe
  • Django Rest Framework - Testing client.login doesn't login user, ret anonymous user
  • Running dag without dag file in airflow
  • Filling across a specified dimension of a numpy array
  • Python populating dataframe in pandas from text files
  • How to interpolate a single ("non-piecewise") cubic spline from a set of data points?
  • Divide 2 integers (leetcode 29) - recursion issue
  • Can someone explain why do I get this output in Python?
  • How do I scrape pdf and html from search results without obvious url
  • Is there a way to automatically make a "collage" of plots with matplotlib?
  • How to combine multiple rows in pandas with shared column values
  • How do I get LOAD_CLASSDEREF instruction after dis.dis?
  • Django - How to add items to Bootstrap dropdown?
  • Linear Regression - Does the below implementation of ridge regression finding coefficient term using gradient method is
  • How to drop all rows in pandas dataframe with negative values?
  • Most Efficient Way to Find Closest Date Between 2 Dataframes
  • Execution error when Passing arguments to a python script using os.system. The script takes sys.argv arguments
  • Looping through a function
  • Create a plot for each unique ID
  • a thread python with 'while' got another thread never start
  • Solution from SciPy solve_ivp contains oscillations for a system of first-order ODEs
  • trigger python events driven by selenium controlled browser
  • Passing line-edits to a contextmanager to set validators
  • Python: globals().items() iterations try to change a dict
  • Is it possible to specify starting values for each parameter (instead of bounds) for scipy's differential evolution?
  • why datetime.now() and constructed datetime using all fields(like year,month...) of now has big timedelta?
  • MySQL multiple table UPDATE query using sqlalchemy core?
  • find if a semantic version is superset of of another version python
  • Type checking against dynamically created objects
  • Struggling with simple reverse function
  • Is there a function for finding the midpoint of n points on sklearn.neighbors.NearestNeighbors?
  • How to set max number of tweets to fetch
  • PYTHON 3.7.4 NOT USING SQLITE 3.29.0
  • How to replace Nan value with zeros in a numpy array?
  • How to speed up calculating variance among sparse matrix
  • cupy code is not fast enough compared with numpy
  • How to count frequency of select values in Python pandas dataframe
  • Scrape Span Text from Google
  • Python watchdog, watch a directory and rename file on event.modification
  • Filtering rows in DataFrame with dependent conditions
  • How to check if a character is a not a part of number or URL in string?
  • Compare corresponding elements of a list
  • Python misinterprets 3 character string as UTF-8 continuation byte
  • Merge two columns in Pandas
  • Side Effect error in Python in an online compiler
  • How to convert a navigation list with depth levels to a parent-child flat list?
  • Retrieving values from a paired key dictionary in Python
  • How to test the current text of a Tkinter text box widget before inserting new text after user clicks on a button?
  • computing the Cumulative Sum, where Sum can be reset by a condition
  • Where do you specify your API key when making a request with the Google API python library?
  • Pandas DataFrame, computing the Time Difference between one row and other row which satisfies a condition
  • Discord.py bot not reading other bot's messages
  • How do I write a Django query that finds words in a Postgres column?
  • Download S3 File Using Boto3
  • Retrieve status of model training after closing notebook
  • If Current Date is in Column Then Show Row
  • How do I stay in a for loop's element until a condition is met?
  • QPainter point size is 1 pixel and can not be changed
  • How to download the excel file which has no link in Beautifulsoup?
  • Can't read the content of a certain page of a pdf file available online
  • shadow
    Privacy Policy - Terms - Contact Us © scrbit.com