File IO Within a Loop – Lessons Learned

Consider the following piece of code:

//Reads a NWS HLRDHM output file containing time series data - highly specialized
void readOutputTimeSeries(int numFiles, string *filenames, double **timeSeries)
{
    string test, junk;

    //Loop through output files
    for (int file=0; file<numFiles; file++)
    {
        ifstream in;
        //Open the file
        in.open(filenames[file].c_str(), ios::in);

        //Test file stream
        if (!in)
        {
            cerr << "Error opening model output file! Filename: " << filenames[file] << endl;
            exit(1);
        }

        //Find the line beginning with RDHM OUTPUTS
        while (1)
        {
            in >> test;
            if (test == "RDHM")
            {
                in >> test;
                if (test == "OUTPUTS")
                {
                    //We're there - ingore the rest of this line and the next
                    in.ignore(1000,'\n');
                    in.ignore(1000,'\n');
                    break;
                }
            }
        }

        //Now, we're at the discharge data
        int i=0;
        while (!in.eof())
        {
            //TODO: This will need to get updated to handle multiple time series outputs
            in >> junk >> junk >> junk >> timeSeries[file][i];
            in.ignore(1000,'\n');
            i++;
        }
        in.close();
    }
    return;
}

Description:

This function loops through a set of output files in different folders, looks for the beginning of the needed data, and collects the data in a 2-dimensional array.

The Problem:

Does anybody see anything wrong with this…?

While this seemed to work fine on some HPC clusters, today I ended up spending a couple hours dealing with “Error opening model output file!” messages. Thinking that it might have something to do with the fact that I was running in an MPI environment, I spent a lot of time ensuring that the path and file it was using was correct.  As it turns out, there was nothing wrong with the path at all.  Apparently, (and this doesn’t appear to be universal across systems), when the eof (end-of-file) flag got set after reading through the first file in the loop, this flag doesn’t get reset the second time through the loop, even though we are closing the file stream.  Hence, the “if (!in)” conditional will always test true, even if the file was opened successfully.

The solution:

Place an “in.clear()” statement immediately before the “in.close() in order to reset any error flags that get set on the file stream from the previous run through the loop.

Advertisements

One thought on “File IO Within a Loop – Lessons Learned

  1. Yeah, you always should be careful when reusing standard classes and make sure that the “close” or whatever ‘clear’ / ‘reset’ commands you’re giving it indeed reset the class to its initial (empty) state. Similarly, make sure that when you design your own classes for reuse that they have the ability to reset themselves.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s