logo
down
shadow

Which is faster in memory, ints or chars? And file-mapping or chunk reading?


Which is faster in memory, ints or chars? And file-mapping or chunk reading?

Content Index :

Which is faster in memory, ints or chars? And file-mapping or chunk reading?
Tag : cpp , By : bikefixxer
Date : January 02 2021, 06:48 AM


Comments
No Comments Right Now !

Boards Message :
You Must Login Or Sign Up to Add Your Comments .

Share : facebook icon twitter icon

Large file (100GB) opening and reading chunk by chunk using memory mapping


Tag : c , By : Fernando
Date : March 29 2020, 07:55 AM
wish of those help Certainly a variable of type "int", which is 32 bits on Linux, is not large enough to contain the size in bytes of a 100 GB file. For file sizes/offsets you need to use the type "off_t" instead (which, when enabling LFS support as you have done, is an alias for off64_t, a signed 64-bit integer).
Similarly, the "length" argument to mmap is of type size_t, not int.

Trouble reading ints (well chars and then converting) from text file one line at a time in C++


Tag : cpp , By : Webtoad
Date : March 29 2020, 07:55 AM
may help you . I have read different posts/questions on here as well as cplusplus.com and various sources and am still having trouble with my code as far as reading a text file. The text file has one int (single digit) on each line. So for example: 5 1 2 5 . . . etc Each number is supposed to represent a specific variable. I am having trouble I believe it's with the '\n' character? Because it will read the first int but then the second one and a few others will be -38!! I've tried the ignore() thing (although maybe I did that wrong?) and that didn't work. I'm sure it's a quick fix and probably staring me in the face but I can't see it! Here is the part of my code that I'm having issues with: , Are you reading from an ifstream?
#include <fstream>
std::ifstream infile("inData.txt");
int c;
infile >> c;

out of memory error when reading csv file in chunk


Tag : python , By : Ernie
Date : March 29 2020, 07:55 AM
it should still fix some issue Based on your snippet, when reading line-by-line.
I assume that kb_2 is the error indicator,
groups={}
with open("data/petaJoined.csv", "r") as large_file:
    for line in large_file:
        arr=line.split('\t')
        #assuming this structure: ka,kb_1,kb_2,timeofEvent,timeInterval
        k=arr[0]+','+arr[1]
        if not (k in groups.keys())
            groups[k]={'record_count':0, 'error_sum': 0}
        groups[k]['record_count']=groups[k]['record_count']+1
        groups[k]['error_sum']=groups[k]['error_sum']+float(arr[2])
for k,v in groups.items:
    print ('{group}: {error_rate}'.format(group=k,error_rate=v['error_sum']/v['record_count']))

Mapping chunk of shared memory for reading/writing in Ada


Tag : development , By : Andrew
Date : March 29 2020, 07:55 AM
wish helps you I have a chunk (1024 bytes) of shared memory between two processes for which I have an address pointing to. I want to copy some data to this shared memory, and read it on the other process. Coming from a C background, it seems easiest to map a record to this address, and then write to the record, but it does not seem to be copying correctly. , I would try to do it this way:
procedure Payload is 

   type Payload_Array_Type is array (1..255) of Integer_32;

   type Common_Buffer_Type is record
      Size : Integer_32;
      Payload : Payload_Array_Type;
   end record;
   for Common_Buffer_Type use record -- representation clause should be common to both processes
      Size    at 0 range 0 .. 31;
      Payload at 0 range 32 .. 1023;
   end record;
   for Common_Buffer_Type'Size use 1024; -- check this is also used in the other process.

   type Common_Buffer_Ptr_Type is access Common_Buffer_Type;

   Common_Memory_Ptr : System.Address; -- assuming this is where the shared object resides with a real address, possibly make it constant

   procedure Copy_To_Common_Buffer (Size    : in Integer_32;
                    Payload : in Payload_Array_Type) is
      Common_Buffer : Common_Buffer_Type;
      for Common_Buffer'Address use Common_Memory_Ptr; -- address overlay
   begin
      Common_Buffer := (Size => Size,
            Payload => Payload);
   end Copy_To_Common_Buffer;

begin

   Copy_To_Common_Buffer (9,(others => 876));

end Payload;

Reading from file inserting values into ints and chars C++


Tag : cpp , By : user105769
Date : March 29 2020, 07:55 AM
will help you Open your file, read the file data line by line and extract what you need from the line.
  std::string line;
  ifstream read;
  //open data files     
  read.open(file_name);
  if(read.is_open())
    cout << "File ./" << file_name << " is open.\n";
  else {
    cout << "Error opening " << file_name << ".\n";
    exit(0);
    }

    while (std::getline(read, line))
    {
    // line =3 A 0100 3 E 0101 3 G 0110 3 ...     
     std::istringstream iss (std::move(line));
     std::string val_str, al, bin; 
        while(! iss.str().empty())
        {
            try{
                iss>>val_str;               
                int val= std::stoi(val_str);     //val = 3 in the first run of the while loop
                iss >> al;               //al = A  in the first run of the while loop
                iss >> bin
                // you can use val, al ,bin
            }catch(..){
                break;
            }
        }
    }
Related Posts Related QUESTIONS :
  • The conditional operator is not allowing the program to terminate
  • Define a c++ string as "\"
  • memcpy on __declspec naked returns unexpected bytes
  • What is the proper way to link enums with CMake?
  • is it safe to use the same mutex with lock_gard and without it in other parts of code
  • How to decode MAP Invoke messages using asn1c generated code
  • How do you write multiple lines in a .txt with recursion?
  • Member function with strange type causing callback function mismatch
  • Visual Studio optimisations break SDL graphical output
  • How to use less memory in Sieve_of_Eratosthenes
  • Covariance in Callback Parameters C++
  • switch may fall through (no it may not)
  • Compilation fails calling Cocoa function from C++
  • How to handle classes with differently named member functions in algorithms?
  • Convert QString to QJsonArray
  • Data exchange finished in CPropertyPage::OnOK?
  • Template member specialization in template class
  • Is it not possible to assign a struct to an index of a vector?
  • Why is empty unordered_map.find not returning end()?
  • Template argument deduction for inheriting specializations
  • dlopen undefined reference
  • Member function of class with template arguments and default arguments outside class
  • Is it possible to implement a non-owning "slightly smart" pointer on top of standard weak pointers?
  • how to configure the AcquireCredentialsHandleA correctly
  • Using private versions of global extern variables with OpenMP
  • Eigen Block wrong amount of columns and rows
  • Memory alignment rules in inheritance
  • Is nullptr falsy?
  • tm_wday returns a large integer outside 0-6 range
  • Scope a using declaration, inside a header
  • How to specify constructor's template arguments inside a new expression?
  • Sort an array via x86 Assembly (embedded in C++)?? Possible?
  • How to Replace only Part of the Variable using #define
  • How do you compare the performace of valarrays vs built-in arrays?
  • Is it normal for C++ static initialization to appear twice in the same backtrace?
  • c++ generate a good random seed for psudo random number generators
  • Why isn't my operator overloading working properly?
  • Getting meaningful error messages from fstream's in C++
  • C++: Converting Julian dates to Gregorian
  • Could someone explain this interesting behaviour with Sleep(1)?
  • Is it possible to roll a significantly faster version of modf
  • Updating pointer using signals and slots
  • How are classes more secure than structures?
  • finding "distance" between two pixel's colors
  • C++ Greatest Number Verification
  • Why does my token return NULL and how can I fix it?(c++)
  • C++ enforce conditions on inherited classes
  • what happened if an exception is not captured?
  • Redundant naming in C/C++ typedefs/structs
  • question about STL thread-safe and STL debugging
  • killing a separate thread having a socket
  • Returning the size of available virtual memory at run-time in C++
  • Parallel computing for integrals
  • How do I force my std::map to deallocate memory used?
  • C++ Templates: implicit conversion, no matching function for call to ctor
  • Adding python script to c++ project
  • C++ private pointer "leaking"?
  • Initializing Primitive Array to One Value
  • how is push_back implemented in STL vector?
  • C++ Profiling: KiFastSystemCallRet
  • shadow
    Privacy Policy - Terms - Contact Us © scrbit.com