Errors in Computational Fluid Dynamics Simulations

As computational fluid dynamics (CFD) solvers have matured, their role in the support of system design and evaluation has expanded considerably. The increasing utilization of unstructured grid methods in particular, with their relatively fast setup process, has contributed to the growing use of CFD in many applications of interest. The design and evaluation of new and emerging air vehicles has increasingly relied on CFD analyses for an improved understanding of the aerodynamic characteristics and to supplement ground test data. These analyses can assist in examining scaling issues in the transition from ground test to flight test, and in addition, offer a means of assessing dynamic loads over time scales greater than those achievable in ground tests.

The sole use of CFD as an analysis tool is not yet an accepted practice, but has instead been employed as a complement to experiments and other analyses. While CFD analysis plays an ever-increasing role in supporting the design and experimental testing of air vehicles, it is wind tunnel data that is generally regarded as the “truth” upon which design decisions are made. The perceived risks associated with using CFD results may be somewhat mitigated if the solution errors are better understood, automatically controlled, and reduced. Such processes would increase the trust in numerical predictions. This is particularly true for simulations of propulsive phenomena where physical modeling is often more complex and plays a vital role. While solution verification is generally considered important, there remains little consensus among the CFD community on how to bound the error or uncertainty in any given calculation.

Solution errors are inherent in any CFD simulation. The numerical simulation and prediction of complex fluid flow phenomena for realistic scenarios of interest are a challenging endeavor, and

computations obtained using CFD codes are often not reliable. There are numerous potential sources of error in any given CFD simulation, many of which are summarized in Table 1.

Specific areas of concern include spatial and temporal accuracy, improper physical modeling, and human factors. With many potential error modes, there is no single path towards establishing solution credibility. Rather, the task of solution verification should be addressed on multiple fronts.

Table 1. Types of errors in CFD simulation.

Spatial Temporal

• Truncation error of scheme

• Resolution of solution features with varying length scales

• Mesh quality considerations

• Poor representation of surface curvature

• Formal accuracy of integration scheme

• Size of time step

• Solution stability constraints

• Convergence of inner iterations

• Competing time scales (fluid vs.

chemical)

Physical Modeling Other Sources

• Choice of governing equations

• Choice of near-wall treatment

• Limitations of turbulence model

• Limitations of reaction set

• Limitations of two-phase model

• Validity of boundary conditions

• Programmatic errors (bugs)

• Usage errors (improper inputs, etc.)

• Geometry errors (CAD model)

• Numerical scheme (artificial

dissipation, upwinding, limiters, etc.)

• Insufficient iterative convergence

Spatial errors are perhaps the most recognizable and readily addressed. In all CFD applications, grid resolution is a fundamental issue. While it is readily accepted that poor mesh resolution has a detrimental impact on solution accuracy, the generation of a suitable mesh often requires inputs from an experienced CFD analyst. One of the challenges in generating meshes for complex flow fields is that solution gradients and regions of critical concern are generally not known a priori.

As a result, computational meshes generated with inadequate point distributions may produce solutions that do not fully reflect reality. Examples include smeared shock structures with reduced downstream pressures, vortices that dissipate prematurely, and shear layers that mix too rapidly due to mesh-induced diffusion. At worst, a poorly resolved mesh may fail to capture critical relevant flow features. In this regard, structured grid solvers possess an inherent advantage over unstructured grid schemes, as it is far easier to implement higher-order schemes on structured meshes. These higher-order stencils naturally serve to minimize mesh-induced diffusion, and more readily lend themselves to applications in acoustics and similar disciplines where high fidelity is required. For unstructured grids, extensions beyond second-order accuracy are prohibitively expensive, as demonstrated by Barth [1]. Therefore, there is a greater need to address spatial accuracy concerns for unstructured solvers, which are generally limited to second-order.

However, the errors present in CFD simulations are not limited to mesh discretization. Errors may be introduced by invalid or incomplete physical modeling. This is particularly true for phenomena that are not well understood, such as ignition chemistry, particle-wall interactions, and boundary layer transition. Often models of complex processes are incorporated in an empirical fashion, and include tunable coefficients based on experimental curve fits, such as turbulence model constants and activation energies in reaction rates. Calibration of these models naturally leads to some

limitations in their applicability, with validity restricted to a particular range in the dependent variables (e.g., Reynolds number or temperature range). Other sources of error include temporal errors, round off and precision errors, programmatic errors, and use of the simulation code. Use of a validated code in no way guarantees credible simulations as the analyst himself can play a large part in establishing simulation credibility [2]. The human element introduces different forms of error in the analysis, as noted by Mehta [3], including making decisions based on insufficient information, errors in interpretation and risk assessment, and subjective personal attachment to the work performed, causing the analyst to perceive the results as reality without sufficient evidence to support the claim.

With so many contributing factors, CFD analysts, particularly those lacking years of experience and insight on which to rely, have a definitive need for tools to assess the various potential sources of solution error. For example, discretization error has well-known consequences in the form of numerical diffusion, and the generation of an appropriate grid is subject to the judgment and experience level of the analyst. Once a solution is obtained, alteration of an initial mesh is again left in the hands of the user. Guidance in where and how to improve subsequent meshes, or automated mesh adaptation would facilitate this task. Systematic grid refinement studies have historically been one means of controlling and reducing spatial errors, however, they are seldom done in practice due to the considerable time and labor costs involved. Any means of expediting, automating, and reducing the man-in-the-loop effort associated with this process is therefore extremely desirable.

The need for assessing solution accuracy has been recognized for some time. Archival publications such as the AIAA Journal and the Journal of Fluids Engineering have sought to enforce standards on the presentation of numerical analyses obtained using CFD, and comparison with experimental data alone is no longer sufficient. In the late 1980s, the American Institute of Aeronautics and Astronautics (AIAA) Committee on Standards for Computational Fluid Dynamics was formed, with the goal of improving the value of solutions obtained from CFD tools. Their work resulted in the AIAA Guide for Verification and Validation (V&V) of CFD Simulations [4]. However, as pointed out by Frietas [5], neither these editorial policies nor the guide define procedures for how numerical uncertainty is to be addressed. The AIAA Guide, for example, sought to formalize definitions and methodology, and does not present suitable techniques. While recent workshops in computational aerodynamics have raised the standard in solution accuracy by stressing mesh refinement studies [6-9], few practical tools are available to CFD analysts for assessing and reducing the errors present in their results.

This is particularly true for the propulsion community. For the past several years, the Joint Army Navy NASA Air Force (JANNAF) Modeling and Simulation Subcommittee have held a series of workshops on V&V with the intention of compiling a set of recommended practices similar to the AIAA Guide, but with an emphasis on the physics and modeling requirements specific to propulsive applications. Several representatives from the CFD community, including the author [10], have presented their perspectives on the subject of simulation credibility.

While the terms V&V are often referred to in tandem, they address separate matters. Verification is concerned with certifying the numerical accuracy of the simulation, namely verifying that the selected governing equations are solved correctly. The process of validation answers the question of how well the selected governing equations and physical models reflect reality. Typically the former is answered by demonstrating grid convergence, while a comparison with test data is generally required for the latter. Only when grid-induced errors, iterative convergence errors, and usage errors are bounded and understood, can physical modeling errors be properly addressed.