C++ :: Data Buffer With Memory Mapped Files?
Jan 27, 2014
I have a data buffer project in Windows 7 x64 embedded OS (using Visual Studio 2008), that would work simply like that:
One writer application will send data (no network protocols, just procedure call on the same machine) to my app like 20 packages per second, each data packages will be approximately 3 MB size and comes with a timestamp tag.
My application will store each data item for 100 minutes and then remove. (So I can calculate the total size from beginnig no need for dynamic allocation etc...)
Meanwhile there will be up to 5 reader applications which will query data from my app via Timestamp tag and retreive data (no updates or deletitions on data by reader apps).
So since the data in my buffer app can grow over 50GB I don't think that shared memory is going to work for my case.
I'm thinking about using Boost Memory Mapped Files or Windows API Memory Mapped Files.
So theoratically I will crate a fixed size File on harddisk (lets say 50GB) and then map some portion to RAM write the data to this portion, if the reader applications wants to use the data which is mapped currently on memory, then they will use directly, otherwise they will map some other potion of the file to their address spaces and use from there...
My problem is I haven't use Map File concept at all and I'm not a C++ developer so I'm not very familiar with the language fundementals. I've searched tutorials of boost and msdn for windows as well but there is not too much code example.
So I don't know how to map some portion of the data to memory and manage it, also how to search data in file or memory according to the timestamp tag. Yes there are codes for creating files and mapping the whole file to memory but none for dividing it into regions, aligning or padding the data which I need for my case.
View 2 Replies
ADVERTISEMENT
Aug 21, 2014
i've created dll and implemented shared memory that every connected process can use. My problem is that i can't change anything in object, which is stored in the memory.
my class :
class MyClass
{
public:
MyClass();
void test();
int counter;
};
void MyClass::test() {
MessageBoxA(NULL, "test", "test", 0x0000000L);
counter++;
}
in stdafx.h i have :
static offset_ptr<MyClass> offset_mt;
static managed_shared_memory *memSegment;
I initialize shared memory and pointer :
memSegment = new managed_shared_memory(create_only, SHARED_MEMORY_NAME, 4096);
offset_mt = memSegment->construct<MyClass>("MyClass myClass")();
And then in an exported function i call
offset_mt.get()->test();
Im calling this from Java using JNA and result is a memory error (Invalid memory access). However, if I delete 'counter++' from test method, everything works fine - message box appears. Is there a limitation that I cant modify objects inside mapped memory or is this done the other way?
View 3 Replies
View Related
Jan 20, 2014
I have a C program. Within it, I would like to embed either a file or a pre-assigned variable. I would prefer a method that is platform independent.
I am using the data type "RSA" from <openssl/rsa.h>. I have a key file in PEM format, but I would like to embed an RSA object in the program instead. I tried creating a char array and then casting the pointer, but that caused some sort of illegal casting issue. I tried using memcpy but I haven't been able to get it working that way. What is the best way of going about this? Is it possible to read a file directly from a memory buffer?
View 1 Replies
View Related
Mar 1, 2014
I'm writing a program that communicates with another program over TCP/IP. I need to assemble contiguous buffers of mixed data to send to the server. For example, one of the messages must contain multiple 32-bit integers and 64-bit floats, each in its own appropriate field. I also need to receive and decode similar buffers.
The Windows TCP function "send(socket s,char *buf, int len, flags)" takes a pointer to a character array to send. Likewise, the TCP function "recv(socket s,char *buf,int len,flags)" takes a pointer to a character array to fill. I've tried creating structures describing the send and receive messages, with the fields appropriately laid out. But the CODE::BLOCKS compiler complains when I try to hand send and recv the pointers to the structure variables. Am I on the right track, or is there a better way to do this?
View 2 Replies
View Related
Dec 24, 2014
I am trying to add some functionality to this project: [URL] ...
Under this [URL] ...
This latest comment based on;
host.data = (u_char *) "112.168.231.19:80";
for (i = 0; i < peer->name.len; i++) {
peer->name.data[i] = host.data[i];
}
or more simpler:
peer->name.data = host.data;
States: "Because strings always require an extra memory buffer (and the burden of memory management) while primitive values like booleans and numbers don't."
Which makes sense since a boolean change (down/up) does actually changes a runtime value, however the string replacement shows via '/upstreams' that the value has changed but at runtime it is still using the old value.
So in short it looks like I do need a buffer to change the string value.
My question is why? if 'peer->down' works on a memory fragment where its value resides then why doesn't a 'peer->name.data = host.data' do the same thing? can this be done without a buffer? once you allocate memory and store host.data in to this where do I copy it to?
View 5 Replies
View Related
Nov 19, 2012
Got buffer with raw data. How to display it in MFC dialog and refresh it?
View 3 Replies
View Related
Apr 5, 2013
I am writing a program to hide files behind other files using Alternate Data Streams in Windows NTFS file systems.
The program is as follows:
Code:
#include <stdio.h>
#include <stdlib.h>
int main(void){
char hostfile[75], hiddenfile[75], hiddenFileName[15] ;
printf("Enter the name(with extension) and path of the file whose behind you want to hide another file: ");
scanf("%75s", hostfile);
[Code]...
The complier is showing error as "Extra Perimeter in call to system" but I am not getting where?
View 4 Replies
View Related
Jan 4, 2014
I am parsing a binary data file by casting a buffer to a struct. It seems to work really well apart from this one double which is always being accessed two bytes off, despite being in the correct place.
Code:
typedef struct InvoiceRow {
uint INVOICE_NUMBER;
...
double GROSS;
...
char VAT_NUMBER[31];
} InvoiceRow;
If I attempt to print GROSS using printf("%f", row->GROSS) I get 0.0000. However, if I change the type of GROSS to char[8] and then use the following code, I am presented with the correct number...
Code:
typedef struct example {
double d;
}
example;
example *blah = (example*)row->GROSS;
printf("%f", blah->d);
View 7 Replies
View Related
Dec 13, 2013
I want to send data from a laptop (windows 7, processor 2.60GHz) to a desktop (windows xp, processor 3.10GHz) using serial communication (using a USB to RS232 convertor). The WriteFile function is able to send the data from the laptop (NumberOfBytesWritten is correct). But on the desktop side, ClearCommError detects no data in the read buffer.
This is the relevant code in my desktop:
while(1) {
ClearCommError(hPort,&dwErrors,&commStatus);
if (commStatus.cbInQue != 0) ReadFile(hPort,&data,1,&dwBytesRead,NULL);
Sleep(10);
}
The if condition is never satisfied. The baudrate and other parameters in the DCB struct are the same on both sides.
The same code works when I write and read in the same system by shorting the RX and TX pins in the RS232 connector.
View 2 Replies
View Related
Mar 26, 2013
Modify program 3 by saving the header file on your memory stick and remove it from the Header section of the VC++ in the Solutions window.
Code:
#include<stdio.h>
#define pi 3.141592654
#define A (num) * (pi) * (Radius) /* a macro defines a math equation */
float spherevolume (float);
main() {
[Code] ....
View 3 Replies
View Related
May 30, 2012
I am trying to understand what techiques can be used to sort really huge files (larger than available memory). I did some googling and came across one technique.
1. Are there any better ways to get this done?
2. Is there some tweaking that can be done to make this itself better?
Large enough so that you get a lot of records, but small enough such that it will comfortably fit into memory
3. How do you decide on this value? Consider, memory is 4 GB and currently about 2GB is consumed, and file to sort is 10GB in size. (Consumed memory could of course change dynamically during execution - consumed more/less by other apps.)
View 5 Replies
View Related
Jul 10, 2013
I have written the following code to add data to text files which are required to store 3D scan data (I have to calculate the y-coordinate). My code seems to work except that it stops working when I want to create more than ten text files i.e. the directory I am trying to store them in will not hold any more than ten text files. Code is shown below.
#include <string>
#include <locale>
#include <iomanip>
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;
[Code] ....
View 1 Replies
View Related
Aug 7, 2012
Worked with genetic algorithms. im solving travelling salesman problem with genetic algorithm and unfortunately stuck at crossover im using partially mapped cross over. i have tried coding it but it some time does not gives valid tour. Looking for building its logic.
View 1 Replies
View Related
Jan 14, 2014
A C++ container type called std::map is used to store elements that are formed by a combination of key value and mapped value. How do you iterate through this container and find the value mapped to a specific key?
View 3 Replies
View Related
Sep 16, 2014
Currently I'm trying to do something extremely basic: delete or move a file
I have a mapped network drive (Y) where I have a file I need to move from one folder to another
First was doing the obvious:
System.IO.File.Move("Y:TempFile.txt", "Y:Temp2File.txt");
Not much to go wrong here, right? As long as I have the correct permission to write in the folders and the folders exists, this would normally go smooth. But for some reason it only copies the file. The source file still exists, but it has successfully copied the file to the destination folder. No exceptions has been thrown or anything... but the file still exists in the source directory
I then changed the code to the following:
System.IO.File.Copy("Y:TempFile.txt", "Y:Temp2File.txt");
System.IO.File.Delete("Y:TempFile.txt");
Console.WriteLine("See mommy, no Exceptions");
As expected the file copies correctly, but the delete command is completely ignored. The file is not deleted, but the Console.WriteLine command is called
I'm running the application as the logged in user. The user has permission to delete the file. If I try from Windows Explorer, the file is correctly deleted. Unfortunately my application refuses to delete the file and even worse, it will not even throw an Exception.
View 10 Replies
View Related
Jun 27, 2014
I'm reading strings one after another and trying to split it using strtok_r. This doesnot seem to be working
I have a .gz file from which I'm reading the data into the buffer in chunks.
The below code works fine only for the first chunk on data. Later it just breaks out.
while(1) {
char buffer[SIZE];
int bytes_read = gzread (f, buffer, SIZE - 1);
[Code] .....
View 3 Replies
View Related
Jan 31, 2015
So, it seems I have a bug in my code:
Code:
char *mem = malloc(9), *bottom = malloc(3), *in = "abc";
int position = 2, i;
mem = "onetwo";
printf("OLD mem = %s
", mem);
[Code]....
What I'm trying to do is insert "abc" in the 3rd position of "onetwo". Results would be "oneabctwo", but all I get is crashing.
View 9 Replies
View Related
Mar 27, 2013
Is it possible to store data to CPU memory, if so how much memory can I store and how? Say I wanted to store an array, about 6MB in size?
View 1 Replies
View Related
Nov 11, 2014
When I return mem add I cannot access the data in that add: in bin search tree
Curr=findMin(root);
cout<<"val: "<<Curr->barcode<<endl;
bst::hwareItem* bst:: findMin(hwareItem*& root){
while(root!=NULL){
root=root->left;
}
return root;
}
View 1 Replies
View Related
Jul 11, 2014
Using the C # programming language LED controller through which controls the array is connected to the controller via the RJ45 port. I already know the basics, I mean specifics, ie
Using the instructions and DLL included with the hardware, I want to read data from it, but I can not deal with it.
Model controller is "C-Power5200" and dll is "CP5200.dll".
I would like to:-pick the data-display-edit and - save in the controller.
To read the data I am using the function
CP5200_Net_Read (BYTE * pBuf, int nSize);
and to save
CP5200_Net_Write (const BYTE * pBuf, int nLength);
View 2 Replies
View Related
Jul 26, 2014
I have a csv file (comma delimited text file really) and I have data ,about 6400 lines, and I need to read and display data from 1 line. At the bottom is sample data from my csv file.
I want to read onto the screen the 3rd item .. ie Test User 1 and the date and time at the end (the last 2 items).
I will enter in a number .. from which I will want to retrieve the appropriate record .. in this case I would enter in 514, 563 or 874 and get back the info for that record.
This is a DOS 16 bit application .. and I know that some of the data is in memory already but I am not sure how to tell .. or how to retrieve it.
I do have the ability to save the data to the file at any time .. so I can use that to ensure that the data is current in the text file before I retrieve the data from this 1 row.
I think that it may just be easier to get from the text file ? but I am not sure.
0000,7,0,"Test User 1",53c145cb,db,5,514,514,2014/07/12,10:27:23
0001,7,0,"Test User 2",53d7f,1903,10,563,563,2014/07/24,11:04:31
0002,7,0,"Test User 3",529c,5ffb03,0,874,874,2013/12/02,09:07:58
I am a database guy .. and I am just a bit lost when it comes to C programming as such.
View 9 Replies
View Related
Mar 21, 2013
So, if I'm right, computer store their data as binary values. So if I write int x = 5; , my computer converts the value of x from decimal (5) into binary (101) and stores it in memory as a a binary number. If I print that value on the screen that value is converted(by default) back into a decimal number before being printed on the screen.
Now, my question is if there is any way to print the value of x directly into binary(as it's stored in memory) without it being converted back into a decimal value?
View 10 Replies
View Related
Oct 23, 2014
I serialize a XML file, and i have this values here:
public class ServiceConfig {
public List<DatabaseDescriptor> Databases { get; set; }
} public class DatabaseDescriptor {
[XmlElement("Name")]
[Code] ....
I have a form1 which is a datagrid view and get's updated like this:
dataGridView1.DataSource = xmlData.Databases;
Now i have a button: get tables, and it opens up form 2, there it's supposed to appear all the tables of the selected Database, when i click on the row i get the database name so i get all the list of Tables from
DatabaseDescriptor
where
DatabaseDescriptor.Name == name
(I get this when i select the row)
But then on the 2nd form, i also have 2 textbox so i add a new table, i add the tablename and id. the new value should appear in the datagridview of the tables, and when i close it and reopen it, it should be there. Maybe i should concatenate first the values or something, how do i do?
View 3 Replies
View Related
Mar 21, 2014
I have an assignment that wants me to better understand how certain calls like malloc() and free() work. I'm supposed to have one function that allocates a pool of memory for the program to use (this is the only place where I'm allowed to use the malloc function. Anyplace else is off limits, except when allocating space for data structures.) Another function needs to be defined that allocates memory from this large pool of memory that we have already allocated. A third function needs to free a block of memory when a call from function main is made and a fourth does something else (haven't gotten that far yet.) I'm not sure if it's ok or not, but I'll attach the pdf form that describes all the requirements. If that's not ok, I'll delete it.)
Anyways, when thinking of data structures to use for this particular problem, one doesn't jump out at me as being THE data structure that would be most useful. How I would imagine using it is to store the addresses currently being used as well as the block length of those addresses (so basically, each node of the linkedlist would have a pointer that points to an allocated address and another field that told the size of the allocated block of memory.) How to communicate to the large pool of memory that a certain block of memory has been allocated and shouldn't be used until a free is made on that specific block of memory.
Attached File(s)
pa3.pdf (61.07K)
View 2 Replies
View Related
Apr 24, 2013
When you have constant arrays, they are stored in the programme memory space. Upon execution, they are copied into the data space for easy access. To save space in the data space, it is possible to stop it from being copied. Some compilers will deal with this by just specifying the array as const, g++ has a type modifier called PROGMEM.
In any case, the problem is that I'd like to generate some data, preferably without using an external programme to do it. Hence, my question using the template system. Now I could possibly use macro metaprogramming via boost instead of template metaprogramming, and I may have to go that route. However there are limitations, especially since its numbering system is only equivalent to an unsigned byte, which I possibly could use, but could become more messy than if I use the template system that has long long types as well as a whole slue of type safety mechanisms.
View 14 Replies
View Related
Aug 30, 2013
The problem is I have a function which sets some variables and I want to access the variables, but myfunc() is nested too deeply for me to pass a data structure all the way down and to return all the way up. The functions reside in different files. I'm not allowed to use extern structs (i.e. a global variable).
How to use a class and instantiate it in myfunc(). My solutions are:
- using the singleton class method
- static variables and function residing in the class (but I'm suspicious of this way. seems like it's just class variables masquerading as global variables.
View 2 Replies
View Related
