Based on a conversation I had with Eric Simmons, I’d like to make a few comments about some basic terminology you need to think about when starting out in the group. One of the most important things you can learn how to do when starting out in the group is framing “the problem”. This requires a set of terms that we’re fairly precise about, so it’s good to know them in detail. Here goes:
Scope of the Problem
What is it that you are trying to solve? Let’s use the example of acid mine drainage. We all know that acid mine drainage is a “problem” per se, but it’s important to define the scope of the problem when putting it in the context of many objective analysis. Generally speaking, we want to answer:
- What needs to be done?: In our hypothetical example, we may want to prevent future acid mine drainage from occurring, or remediate some environmental damages that have already taken place.
- Who is going to do it?: We must think about the decision makers actually making the decision. For our example, a governmental agency that is funding the work may have a different set of priorities (and a different level of decisions) than an engineering firm hired to carry out the technical work. Define the stakeholders, those who stand to gain or lose in the process. Do the decision makers have to keep the interests of the stakeholders in mind? For example, the governmental body that makes a decision about how to fund an acid mine drainage project may have to think about the people who own property around the affected site.
- What are the limits of the analysis?: In fluid mechanics, we draw a “control volume” around a system, which indicates the area that we’d like to study. Similarly, are there limitations to what we are allowed to do in the system? Perhaps the best option is to move everyone out of the affected area, but this may not be feasible.
The paradigm of many-objective analysis allows us to generate alternatives for a system, evaluate them with respect to multiple performance outcomes, and visualize the results to negotiate a final solution, if needed. But we’re getting ahead of ourselves! When formulating a problem, we have to think about the following analysis components:
Decision variables answer the question: How is it that I am going to do the task at hand? Let’s think about acid mine drainage. In our “scope” section we discussed that we might want to prevent the problem, or remediate a problem that has already occurred. For the purpose of remediation, perhaps we want to design a new system that can pump out the contaminated water, and treat it offsite.
Our actions are encoded in decision variables within our problem. Decision variables for this problem may be the pumping rate, the location of wells that are used to pump the water, or a treatment chemical that can be added to the water. Each of these decision variables should be coded in some way — typically as real or integer-valued numbers that can be modified.
Decision variables are also called levers in the policy analysis literature. It’s a nice way to think of them: if I push a lever, something happens. In the realm of public policy, a lever may be something like the interest rate. The federal reserve changes the interest rate in the hopes of affecting the economy in some way — in an engineering system, you may design the diameter of a pipe in a certain way as to influence the performance of your system.
We’ve thought about what may change in our system or design, but we need ways of measuring the performance of the system. In our acid mine drainage system, it could be the cost of the remediation system, the reduction of contaminants over time, and the disturbance to the landowners around the project area.
These evaluation objectives most likely conflict with each other — the “do nothing” strategy has optimal cost and no disturbance around the project area, but it has poor performance as far as reducing contaminants. In contrast, systems that have great contaminant reduction do so at high costs and high disruption to the landowners. Multiobjective evolutionary algorithms give us high-quality alternatives that are “nondominated” with respect to multiple objectives — solutions that are better in at least one objective than all other solutions considered. In our example, it is the lowest cost at each level of contaminant reduction and land disturbance (or conversely, the highest level of contaminant reduction at each cost and land disturbance level). Finding these objectives is great for the decision makers — he can see good levels of performance, while making a decision about how much to spend or how much disturbance to allow, and so forth.
In the policy analysis literature, objectives are also called “measures” — how do you measure your performance?
Often, we want to set a limit on the values of objectives to consider. For example, there may be a regulatory limit on how poor the performance of a remediation system should be. Or, we may have other targets that need to be met, such as resource limitations. These can be encoded as constraints in our problem.
Simulation, or the Relationship Between Actions and Outcomes
Now that we have our analysis components in place, we need to figure out a way to calculate the actual results. This is something that’s often ignored in great detail in classical formulations.
Let’s think about a linear program for resource allocation. One might say: Maximize the net profit from making widgets A and B, such that resource limitations are met. Here, the relationship between “actions” and “outcomes” is simple: a linear equation maps the actions (the number of A and B that are manufactured) with the outcome (the amount of profit made). There are some clear limitations with this, though:
- Are there economies of scale? That is, is it cheaper to make the 10,000th unit of A than the 1st unit of A?
- Are there non-linear effects for making the products? Maybe the workers get tired after making their 15th unit of A, and the quality suffers.
- The optimal mix of A and B assumes that some resources are “tight” and others have quantities that are unused. Milan Zeleny explored this in great detail, showing that the optimal mix changes depending on modifying your resource constraints.
- Classical optimization solvers can only solve problems that have certain properties — i.e., linear, convex, and so forth
A much better approach is to link a trusted simulation of the system to a competent solver such as a multiobjective evolutionary algorithm. While there isn’t a claim of “global optimality” with these solutions, you can see that the challenges above limit the effectiveness of such solutions in real policy recommendations. Each design is fully evaluated inside a simulation, and you can trust that the fidelity of the objectives or measures that come out of the system match what the simulation would calculate as the outcomes.
In general, the policy analysis literature would call this a “relationship” between actions and outcomes. If we don’t have a good simulation model, it may be a survey-based collection of experts’ opinions about the system. Even in situations like this, though, we can still use this data within a search procedure to recommend non-dominated policies or designs.