File IO Within a Loop – Lessons Learned

Consider the following piece of code:

//Reads a NWS HLRDHM output file containing time series data - highly specialized
void readOutputTimeSeries(int numFiles, string *filenames, double **timeSeries)
{
    string test, junk;

    //Loop through output files
    for (int file=0; file<numFiles; file++)
    {
        ifstream in;
        //Open the file
        in.open(filenames[file].c_str(), ios::in);

        //Test file stream
        if (!in)
        {
            cerr << "Error opening model output file! Filename: " << filenames[file] << endl;
            exit(1);
        }

        //Find the line beginning with RDHM OUTPUTS
        while (1)
        {
            in >> test;
            if (test == "RDHM")
            {
                in >> test;
                if (test == "OUTPUTS")
                {
                    //We're there - ingore the rest of this line and the next
                    in.ignore(1000,'\n');
                    in.ignore(1000,'\n');
                    break;
                }
            }
        }

        //Now, we're at the discharge data
        int i=0;
        while (!in.eof())
        {
            //TODO: This will need to get updated to handle multiple time series outputs
            in >> junk >> junk >> junk >> timeSeries[file][i];
            in.ignore(1000,'\n');
            i++;
        }
        in.close();
    }
    return;
}

Description:

This function loops through a set of output files in different folders, looks for the beginning of the needed data, and collects the data in a 2-dimensional array.

The Problem:

Does anybody see anything wrong with this…?

While this seemed to work fine on some HPC clusters, today I ended up spending a couple hours dealing with “Error opening model output file!” messages. Thinking that it might have something to do with the fact that I was running in an MPI environment, I spent a lot of time ensuring that the path and file it was using was correct.  As it turns out, there was nothing wrong with the path at all.  Apparently, (and this doesn’t appear to be universal across systems), when the eof (end-of-file) flag got set after reading through the first file in the loop, this flag doesn’t get reset the second time through the loop, even though we are closing the file stream.  Hence, the “if (!in)” conditional will always test true, even if the file was opened successfully.

The solution:

Place an “in.clear()” statement immediately before the “in.close() in order to reset any error flags that get set on the file stream from the previous run through the loop.

Advertisements