C++ :: Printing To Multiple Files Because Output Is Too Large?
Apr 18, 2013
I have managed to make a program that permutates a string with repetition.
I ran it to permutate "abcdefghijklmnopqrstuvwxyz1234567890" with a limit of 5 characters.
This took a little over 5 hours for my pc to process this and I ended up with a .txt 403MB in size. Needless to say I am unable to open this .txt in notepad without Notepad.exe not responding and me having to end the process.
So what I want to do is modify my code to break up the output in to several files rather than one. Possibly all permutations starting with a in one file, b in another, etc.
Here is my current code:
#include <iostream>
#include <string>
#include <sstream>
[Code]....
As you can see it currently appends permutation.txt with all output. I would like it to make files like this permut_5char_a.txt, permut_5char_b.txt, etc.
View 12 Replies
ADVERTISEMENT
Apr 14, 2014
Windows 7, 64 bits, Visual Studio 10.
I have a problem to read a large number of binary files, process them and store them under a new name. The program and routines go very well for 505 files. After reading 506 files, the program now refuses to read the next file. I have 16 Gb of memory and tried to close all other programs and restart the PC. it always stops after 506 files (512 files would be more understanding in a way...).
Here is my code. I have tried many things without success. This is only part of the loop that stops. The if test if (myfile.is_open() returns false by some reason. I can start the process again starting with the file that does not open and then it stops again after 506 files.
char * tfiBlock;
ifstream myfile (OrigFilename, ios::in|ios::binary|ios::ate);
if (myfile.is_open()) {
int lengde = myfile.tellg();
tfiBlock = new char [lengde];
//static char memblock [size];
[Code] .....
Clean up procedure:
delete[] tfiBlock;
Are there any limits to how many files that can be opened, or is it maybe someting to be set in the compiler?
View 5 Replies
View Related
Jul 5, 2012
I am reading in some large files to process. The files need to be parsed into multi-line sub units for processing (by a different physical process). My current setup is crude and parses the entire file into memory before beginning to process the sub units. This is fine as long as the file fits, so I get into trouble > 2GB or so when my machine flat runs out of memory. The simple thing to do is to only read in some of the file, process what was read in, and then read in more. I more or less know how to stop reading at some point in the file, but I'm not sure how to resume from that point later when I need more data. Is there a way to count lines and then getline() starting from a specific point in the file? From reading the getline() doc, it doesn't look like there is.
This is my current function,
Code:
void ParseFile(char *path, char type) {
// open input file
ifstream input_file(path);
if( !input_file.good() ) { cerr << "Failed to open " << path << endl; exit(ERRCODE_ERROR); }
int pos = 1;
[Code] ....
I need to remove the code to open the file from this function and open it elsewhere. I would like to call this every time I run out of data, if( tasks.empty() ), and get more data from the input file, but I don't know how to resume reading the input file from where I left off.
If this is not possible, I guess I would have to place a function call in while( input_file.good() ) to call out and process the data I have. When the current tasks list is finished, the list could be cleared and control returned into the while loop to continue reading input and start re-populating the list.
View 2 Replies
View Related
Jan 16, 2014
I am writing a piece of code that requires me to display the last 1000 lines from a multiple text files (log files). FYI, I am running on Linux and using g++.
I have a log file from which - if it contains more than 1000 lines, I need to display the last 1000 lines. However, the log file could get rotated. So, in case where the current log file contains less than 1000 lines, I have to go to older log file and display the remaining. For e.g., if log got rotated and new log file contains 20 lines, I have to display the 980 lines from old log file + 20 from current log files.
What is the best way to do this? Even an outline algorithm will work.
View 6 Replies
View Related
May 21, 2014
I want to read a folder that contains large number of tif files. (say 1000 tif images) requirement is to read all files and store in a array/array pointer.
View 1 Replies
View Related
Mar 31, 2014
I am creating a simple log parser (loads a text file and filters out unnecessary information, but has the option to show the full log) and I'm running into an issue with fairly large log sizes (50+mgs). I have seen a few recommendations from a stream to memory manged files and even alternate 3rd party controls.
I foresee a few issues with any of the non-third party solutions (which I would prefer to avoid third-party add-ins) such as the scroll bar not correctly reporting the relative length or position of the complete text in the box (when displaying only a portion of the file at a time) and in the stream solution where you read on scroll (as necessary) have not only the same issues, but how do you resume reading in the middle of the file? This also all assumes I would be periodically clearing the RichTextBox to keep the memory usage down to avoid an OutOfMemoryException (which I have been running into.)
View 14 Replies
View Related
Jan 14, 2014
We have a program that sequentially processes a large number of files (currently about 700 expected to increase to about 1500). The program performs the same processing on each file (and doesn't involve any other file) which is io-bound and not cpu-bound. This process takes several hours and it is normally performed overnight.
I've refactored the program so that the processing for each file is done within its own thread (ie one thread created for the processing of one file). This gives rise to many hundreds of io-bound threads. This refactored program is working with no errors reported and has reduced the total processing time down to about 10 minutes.
Any problems that might arise having this number of threads (700 to 1500) created/running?
View 14 Replies
View Related
Feb 9, 2015
I'm trying to make sure my code is written in smaller modules, so my first step is to create my initialization process in and external file to load the necessary data from external sources and set up things like the content of drop down list boxes.
My first attempt failed to give me access to the combobox items add function so I moved that code back into the form1.h file:
Code:
public:
Form1(void) {
InitializeComponent();
//
//TODO: Add the constructor code here
//
}
void AddDate(char *date, int ID)
{
this->comboBox1->Items->Add("line 1");
}
It compiles fine, but the call to it in my Initialize.cpp file
Code:
MarketView::Form1::AddDate("abs",1);
Gives error C2352: 'MarketView::Form1::AddDate' : illegal call of non-static member function
OK, so I change "void AddDate" to "static void AddDate" and now get the error that "static member functions do not have 'this' pointers" so I go back to the "MarketView::Form1::comboBox1" situation where there is no legal syntax after "Box1 to get me to Items->Add
I've been an old fashion programmer for over 47 years. It seems as is the concept of programming computers has changed from the concepts of logic to memorization of complex syntax.
There has to be a simple answer to do this other than to write thousands of lines of code in one Form1.h file. I refuse to believe that the new programming concepts will not allow you to write code in smaller more manageable modules.
What is the proper syntax for breaking up the larger file into more manageable chucks?
View 12 Replies
View Related
Jul 15, 2013
I'm using multiple C++ files in one project for the first time. Both have need to include a protected (#ifndef) header file. However, when I do that, I get a multiple definition error.
From what I found from research, adding the word inline before the function fixes the error. Is this the right way to do this, and why does it work? Should I make a habbit of just declaring any function that might be used in two .cpp files as inline?
View 5 Replies
View Related
Jan 15, 2013
Opening large files in c++. In my application, i am trying to save video as long as users have space in harddisk. What I am trying to do is when user is recording video i am trying to append the video data in to the file. The problem is that every time file size reach over 2GB my software crashes.
View 5 Replies
View Related
Apr 12, 2014
This code i made is a cent converter from 5 to 95 cents. The problem i'm receiving is when the 'cents' function is sent back to the 'main' function it only prints one line. It seems to just print the first if construct that complies with the statement. Is there anyway i can have this function print multiple cent values? For example if 60 cents was entered it would only print '50c', and i want it to print '50c' and '10c' instead.
Code:
#include <stdio.h>
int x;
void check(int x)
{
if( x < 5)
printf("Less then 5 cannot be calculated
");
else if(x > 95)
[code]....
View 3 Replies
View Related
Mar 7, 2014
I need to create a sinewave. I have the values in an Excel file. The code reads the excel file and prints the values in the cmd but I need it to print back into excel as a sine wave. Pretty sure its currently values for a triangle wave but whatever I want a wave before I care about the type.
Description: Creates an array of values of a triangle wave, prints the result to the screen and creates a CSV (comma separated variable) file of data points. Allows user to enter number of amplitude and number of cycles
#include <stdio.h>
#include <stdlib.h>
#include <conio.h>
#define N_SAMPLES 1000 // crashes if array size > 1000?
#define UP 1
#define DOWN 0
[Code] ....
View 11 Replies
View Related
Feb 16, 2013
I'm having an issue with output, luckily everything else works!! I'm working with Mixed Numbers and operations on them. So, here's the code I'm testing with:
int main() {
Mixed m1(5), m2(1,1,1), m5(2,2,3);
cout << "m1+m2= " << m1+m2 << endl;
cout << "m1 + 10=" << m1+10 << endl;
cout << "m1 - 10=" << m1-10 << endl;
return 0;
}
And here is the output for that code:
File name: fract.h
#ifndef fract_H
#define fract_H
#include <iostream>
using namespace std;
const int DEFAULT_VAL = 0;
[Code] ....
View 2 Replies
View Related
Oct 11, 2013
I have to convert a binary value from an input file (the name of which is given by the user) and then convert the binary to decimal value and print that in an output file.
Right now it is compiling just fine with no error messages, but when I run the program, it doesn't end or print to the output file.
Here is my code at the moment:
#include <cstdlib>
#include <iostream>
#include <iomanip>
#include <fstream>
[Code].....
View 2 Replies
View Related
Feb 23, 2015
I'm reading in a string from the command line into a char array followed by a series of ints which are read into an int array.
The command line
Code: lab3 word word word 0 0 2 3 results in the following output:
Code: Word: .N=▒
Number of Words: 0
Word: .N=▒word word word
Number of Words: 3
Num: 0
Num: 0
Num: 2
Num: 3
Here is the source code:
int main(int argc, char *argv[]){
if(argc < 2){
//command line must have at least three arguments (one
//char, one integer)
printf("Error: invalid number of arguments
[Code] .....
Where is the .N=▒ coming from?
View 3 Replies
View Related
Nov 13, 2014
I'm trying to make a simple temperature converter with a menu that lets users pick which conversion to perform. But, it won't seem to print the right conversion. It just prints the same temperature that I inputted.
for example:
#include <iostream>
using namespace std;
//prints the menu
void userMenu(){
cout << "Temperature Calculator" << endl << endl;
[Code] .....
View 2 Replies
View Related
Mar 22, 2013
So I have a rather large (for me) project, requiring me to have two .cpp files and a header. Anyway, both of the .cpp files #include the header file, but I recieve linker errors because the variables and functions in the header are declared and defined twice (once in each .cpp file). How am I supposed to do this?
View 8 Replies
View Related
Sep 24, 2014
We typically don't bother with massive, monolithic code files that get processed from top to bottom. In the Object Oriented world, code files don't mean much. In fact, in C#, I could have multiple classes defined in one file, or have one class split across several files.
View 7 Replies
View Related
Mar 23, 2014
I am struggling with the concept of having different ccp's and header files. I made a really bad example project for representation, but basically my question is are any of the #includes unnecessary that I have? Technically it functions, but if I am doing it wrong I want to prevent myself from starting bad habits in the future. My code just basically uses strings and sets a name and prints it. My code is really bad, but I wanted to just use includes in such a way for a quick example.
//MAIN.CCP
#include "functions.h"
using namespace std;
int main()
[Code]......
View 1 Replies
View Related
May 16, 2012
I am trying to run multiple source files but right after the first one finishes running the program closes and doesn't move on ...
View 2 Replies
View Related
Mar 7, 2013
How I can compile multiple source file in visual studio 2012 ???
View 1 Replies
View Related
Sep 7, 2013
I was trying out programs based on extern and as i understand, this is useful when accessing variables across multiple files having only one definition. But i tried a simple program as below without "extern" and thing seem to work when i expected it would fail during linking process
Code:
file5.c
1 #include <stdio.h>
2 #include "var.h"
3
4 int main() {
5
6 printf("
File5.c a = %d", a);
[Code] .....
As i have included "var.h" in all header files without extern, "int a" would be included in both the .c file and during linking, compiler should have thrown a warning or error message but it compiles file without any issue. Shouldn't var.h have the following "extern int a"?
View 8 Replies
View Related
Apr 9, 2013
I have a big un-editable program, A, which I need to run for like a 1000 different input files. It takes about 15 minutes for each file, so a little parallelisation wouldn't hurt.
I have installed openmpi and it works fine. I have made a small program, B, which selects an input file, moves it to another directory, calls program A with the path to the selected input file and then - when A is done - selects a new input file etc. It should loop until there are no more files in the initial directory.
The problem is this: When I have several processors they might pick the same file and that leads to errors. I have a working program, but it is not pretty.
Code:
#include <stdio.h>
#include <mpi.h>
#include <dirent.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char *argv[]) {
int num_procs, procs_id, i, exit;
struct dirent *ent;
[Code]...
Every time a processor tries to move a file that another processor has just moved, the output shows an error message before looping to the next file and trying again. It works, but it is a bit annoying. So my questions are:
1) Can I switch off the error message somehow?
2) Is there a better way to do this?
View 3 Replies
View Related
Aug 2, 2014
So say I create a header file which contains a list of structs, and I want to use these structs through out my source and some of my classes... how would I accomplish this?
When I try to do it via #include, I get re-definition errors, due to the nature of #pragma once. If I switch to #ifndef then I lack defenitions in files other than the source.
Is there a way to define things such as structs across multiple files, which doesn't lead to re-definition errors, and doesn't involve manually re-created all the structs for each file?
View 2 Replies
View Related
Apr 28, 2014
I am currently working on a C++ program for school. I am actually not finding too much difficulty in constructing the functions, enum-types, arrays and structs, however, I am finding great difficulty in using on ifstream variable to open multiple files.
I have posted the entire code that I have so far (even though I have pinpointed the issue to not properly opening the second file in ifstream).
I spent a couple of hours getting rid of certain functions/procedures, loops and variables and I get the same output (if what I removed doesnt crash it). I also get the same output whether I "open" the second file or not (meaning I removed all of the code for it and got the same output).
Here is the code (it's not finished because I am stuck on this file issue). It's a bit messy since I am now in debug mode versus program mode:
View 8 Replies
View Related
Oct 6, 2014
I have been working on code for quite some time and am able to successfully read in a text document and take certain words and information that I need. The issue is that I need to read in close to 100 plus documents and was wondering how I could read in the multiple documents. I thought about creating a structure of arrays and have each text document be an element and walk through taking each document but I am not sure how this works.
View 8 Replies
View Related
