C RUBY-ON-RAILS MYSQL ASP.NET DEVELOPMENT RUBY .NET LINUX SQL-SERVER REGEX WINDOWS ALGORITHM ECLIPSE VISUAL-STUDIO STRING SVN PERFORMANCE APACHE-FLEX UNIT-TESTING SECURITY LINQ UNIX MATH EMAIL OOP LANGUAGE-AGNOSTIC VB6 MSBUILD

# applying window function to big data set (how to optimize?)

## applying window function to big data set (how to optimize?) Tag : development , By : user180941 Date : November 25 2020, 07:06 PM

wish help you to fix your issue I have to do some data analysis on a table with 400+ million rows. I got this to work on a small sample but I'm sure it will run out of memory in production. , You can use lag to do this.
``````select *
from (select t.*
,lag(status_2) over(partition by serial_no order by date) as prev_status_2
,lag(date) over(partition by serial_no order by date) as prev_date
from tbl t
) t
where status_1 = 'in_transit' and prev_status_2 = 'x'
``````

Share :

## Applying a function to subsets of data in a data frame using a moving window in R

Tag : r , By : Michael T.
Date : March 29 2020, 07:55 AM
Hope this helps As you can see, there are lots of ways to go. I think you could do this with a series of loops like @Codoremifa showed you or with some handy add-on packages such as data.table that @RInatM walked you through. I made an example working with the sapply function to loop through the data.
First, I calculated the distance between each pair of points in sequence for the whole dataset based on your code. I used with to avoid having to use dollar sign notation or the extract function [. You can see the vector output pairdist is 1 unit shorter than the number of rows in the dataset.
``````pairdist = sapply(2:nrow(bird), function(x) with(bird, trackDistance(Longitude[x-1], Latitude[x-1],
Longitude[x], Latitude[x], longlat=TRUE) ))
``````
``````totdist= sapply(seq(1,length(pairdist)-3, by = 4), function(x) sum(pairdist[x:(x+3)]))
``````
``````straight = sapply(seq(1, nrow(bird)-4, by = 4), function(x) with(bird,trackDistance(Longitude[x],
Latitude[x],
Longitude[x+4], Latitude[x+4], longlat=TRUE) ))
``````
``````bird\$Sinuosity = c(NA, rep(totdist/straight, each = 4),
rep(NA, length(pairdist)-4*floor(length(pairdist)/4)))
``````

## Applying function to data.frame generates NAs while applying it to columns works

Tag : r , By : Jason Haar
Date : March 29 2020, 07:55 AM
this will help The operation will work as intended if you use sapply instead instead of apply:
``````sapply(frame, weight.fun)
#         x  y
#  [1,] 290 10
#  [2,] 790 10
#  [3,] 410 10
#  [4,] 860 10
#  [5,] 910 10
#  [6,]  50 20
#  [7,] 500 20
#  [8,] 830 20
#  [9,] 510 20
# [10,] 420 20
``````
``````as.matrix(frame)
#        x    y
#  [1,] "29" "1"
#  [2,] "79" "1"
#  [3,] "41" "1"
#  [4,] "86" "1"
#  [5,] "91" "1"
#  [6,] " 5" "2"
#  [7,] "50" "2"
#  [8,] "83" "2"
#  [9,] "51" "2"
# [10,] "42" "2"
``````

## Applying window function in one query without CTE

Tag : sql-server , By : chintown
Date : March 29 2020, 07:55 AM
it helps some times Demo with some test data on how this works
`````` select top (1) with ties
p1.Gid  Gid,
p1.id id,
p1.prod prod,
p1.orderdate orderdate,
p2.shipdate shipdate
from shpro p1 inner join shpro p2
on p1.id=p2.id
where cast(p1.orderdate as DATE)>GETDATE() and cast(p1.shipdate as DATE)<GETDATE()-1
order by
Rank() over (partition by p1.prod order by p1.id desc)
``````
``````  CREATE TABLE #TEST
(
Id INT,
Name VARCHAR(10)
)

Insert Into #Test
select 1,'A'
Union All
Select 1,'B'
union all
Select 1,'C'
union all
Select 2,'D'

select top 1 with ties id,name
from #test
order by row_number() over (partition by id order by id)
``````

## Fitting data to Faddeeva function using python's optimize.leastsq() and optimize.curve_fit

Tag : python , By : adrianmooreuk
Date : March 29 2020, 07:55 AM
Hope this helps Hello Stackoverflow community, , The third comment by PRMoureu on my question fixed the problem.

## FFT - Applying window on PCM data

Tag : c , By : ganok_tor
Date : March 29 2020, 07:55 AM