Numerical Methods for Engineering Analysis
(Level 5)
Most real engineering equations can't be solved with pencil and paper. Nonlinear differential equations, multi-dimensional integrals, systems with thousands of variables won't have clean symbolic answers. Numerical methods give you the next best thing: approximate answers you can trust, generated by algorithms you can verify. This is computational engineering math. You trade exact closed-form solutions for numerical solutions with quantifiable error bounds. Computers do the grunt work. You set up the problem, choose the method, and interpret the results.
What You'll Learn
Why Engineers Rely on Numerical Methods
Try solving dy/dx = y² exactly. It's doable (separable variables). Now try solving dy/dx = y² + sin(x). Good luck. No closed-form solution exists. Or consider calculating the integral of e^(-x²) from 0 to ∞. You can't write down the antiderivative in elementary functions, but the definite integral has a known numerical value: √π/2. That's where numerical methods come in. They don't solve equations symbolically. They compute approximate answers to arbitrary precision. You decide how accurate you need to be, and the algorithm delivers.
Root-Finding Algorithms for Nonlinear Equations
We need to solve f(x) = 0, but we can't factor, or rearrange, or even guess-and-check to an exact answer. That's when we bring out the root-finding algorithms, which systematically home in on the solution, eventually as precise as you like.
The bisection method for solving the quadratic equation x^2 - 2 = 0 converges to the value of √2, which is approximately 1.41421.... Each iteration of the bisection method doubles the number of correct digits in the answer, or halves the error. Thus, to obtain an answer with 10 decimal places (i.e., an error of less than 10^(-10)), it will take approximately 33 iterations.
The function f(x) = x² - 2 converges much more quickly to √2 with the iterative formula x_n+1 = (x_n)² - 2 starting from x₀ = 1 than it does with the bisection method. Much faster than bisection, however, this method requires you to know the derivative of the function and you can sometimes have it diverge to positive or negative infinity if your initial guess is too far off from the square root of the number for which you are solving.
Why This Matters Later
Did you know that every time you solve a nonlinear equation, you are secretly using a root-finding algorithm. Optimization? That is just a special type of root finding where you are minimizing (or maximizing) a function. The eigenvalues of a matrix are, in fact, the roots of the characteristic polynomial. This video explores several methods for root finding, with examples in MATLAB, Python (using the scipy library), and the Excel Solver.
Numerical Differentiation from Data and Models
There are situations where you don't have a formula, or where the function is too messy to differentiate symbolically. Numerical differentiation refers to the practice of approximating the derivative of a function by using finite differences: i.e., by changing x slightly and seeing what change that produces in the function's value, usually with more than one small change in x to get multiple approximate slopes.
Measure how much up hill it is by taking a small step forward and measuring the slope with a leveling tool, probably introduces an error of the order of h.
Average the forward slope and the backward slope. More accurate (error is proportional to h²) which is why this is generally preferred when you can afford the extra function evaluation.
Smaller h (for the same amount of computations) gives better approximations - not always true; around h ~ √ε (machine precision) is typically optimal due to round-off error (floating-point precision limits).
Why This Matters Later
Finite element analysis typically uses numerical derivatives of displacement or geometry to estimate stress gradients. The experimental data from a test (e.g. force vs. displacement or pressure vs. time) is also differentiated numerically to extract rates and even accelerations. This approach to numerical differentiation is used whenever test data is analyzed.
Numerical Integration of Functions for Engineering Applications
This chapter introduces numerical integration methods that are extensively used in engineering to compute surface area, volume, applied moment, and bending moment for diverse geometric shapes and designs.
Ever get stumped by an antiderivative? Don't worry, you can still approximate the area under a curve using simple shapes such as rectangles, trapezoids, or even parabolas. More complex shapes will give better approximations with fewer function evaluations.
Approximate the integral of f(x) over some interval using one trapezoid. Error will depend on how curved f is. For higher accuracy, subdivide the interval into many trapezoids (composite trapezoidal rule).
Boyer's method is more accurate than the trapezoid rule for the same number of points. The error scales with the fourth derivative of f, so it integrates very accurately for smooth functions.
Why This Matters Later
Numerical integration of functions is a fundamental operation performed by every CAD/CAM system, engineering analysis program, control system, and probabilistic risk assessment computer code. In engineering analysis, areas and curves are weighted and summed to obtain corresponding volumes and centroids. Stress analysis integrates strain energy over an area, volume, or domain. Control system error is integrated over time. Probabilistic risk assessments, on the other hand, integrate a function over a probability distribution. Numerical integration is, therefore, a ubiquitous and indispensable tool of computational engineering.
Numerical Solutions of Differential Equations for Engineering Dynamics
This chapter introduces numerical methods for solving ordinary differential equations relevant to topics covered in the remaining sections on Engineering Dynamics.
Differential equations or 'change equations' describe phenomena that change over time. Numerical "solvers" for so-called ordinary differential equations (ODEs) compute the solution (an approximation is generally sufficient) step by step in time. You tell it the equation for the dy/dt, an initial condition, and then a step size, h, and it marches forward in time, approximately computing y for each t.
Accumulate error by assuming the derivative is constant over an interval at each step. Accumulates error very fast; first-order accurate (error ~ h); useful for teaching but not for real work.
Fourth order accurate (~h⁴). ode45 in MATLAB uses this method with an adaptive step-size using a Runge-Kutta method. It has more function evaluations per step than the methods of order two and higher that are based on Runge-Kutta methods, but it requires less steps for similar accuracy and is therefore the workhorse of ODE simulations.
Why This Matters Later
All dynamic mechanical simulation tools use numerical ODE solvers, whether Simulink, Adams, Abaqus/Explicit, or another package. In fact any simulation which involves time-dependent behavior, such as transient heat transfer or electrical circuit analysis, or optimization problems involving trajectories in time, is solved numerically. As a practitioner who solves physically-based simulations it is crucial for me to understand the inner workings of these algorithms and why they can fail at certain times during a transient simulation.
Error, Stability, and Convergence of Numerical Methods
All numerical results are approximations. The more interesting question is not whether there is error, but rather how large the error is and whether it can be controlled.
Truncation error - This type of error is due to the method used for approximating the solution. For example, Euler's method assumes that the derivative is constant for each step. However, in reality, the derivative is not constant. This type of error is referred to as truncation error.
Round-off error: Numbers on computers have limited precision (double precision ~ 16 decimal digits). When adding up tiny errors, their cumulative effect can be significant.
Consistency: Some algorithms may blow up even when the exact solution exists. A stable method for solving a problem ensures that the errors in the approximate solution are always bounded, i.e., it does not cause the errors to explode exponentially.
Convergence: As you refine discretization (e.g. as h->0, n->infinity), does the numerical solution converge to the true solution? If yes, then your method converges. If no, then you have a problem.
Why This Matters Later
When your finite element simulations are unstable or diverge to nonsensical results, they typically indicate either a stability problem or convergence problem. Understanding these issues can help you track down the source of the trouble, which may require finer meshing, improved boundary conditions, or an alternative solver.
Interpolation and Curve Fitting for Data Analysis
You've got data points. You need a continuous function. Two approaches: interpolation (force the function to pass exactly through every point) or curve fitting (find the best-fit function that doesn't necessarily hit any point exactly but minimizes overall error).
This creates a unique solution for the system of equations that fits the given data best. However, very high-degree polynomials are notorious for having a large amount of wild oscillation between the data points (known as Runge's phenomenon), so use with caution.
Common use: fit experimental data to a theoretical curve (linear regression, exponential fit, etc.). Doesn't have to pass through the points. Much better at averaging out the noise for noisy data.
Why This Matters Later
When creating FEA input, the programs we use to generate geometric models (e.g. GeomCaliber) first interpolate the available data for the material properties (stress-strain curves, thermal expansion coefficients, etc.), then fit experimental measurements to these models to obtain the relevant model parameters. Also, CAD splines are defined by a set of control points and exist by means of interpolating between these points to create a smooth curve. All of us do this all the time, whether or not we realize it.
Probability and Statistics for Engineering Uncertainty
Engineering is not deterministic. loads are unknown, material properties have tolerance, measurements of geometrical properties have error. Therefore, we require methods to quantify error.
Here we look at mean and variance, two important measures of central tendency and spread. The mean gives an idea of what average value to expect, but the variance (or standard deviation) of a distribution is a measure of the spread of the distribution.
Normal: The beloved bell curve. The Central Limit Theorem guarantees that pretty much anything will be approximately Normal in the long run, thanks to the accumulation of small, independent, random effects.
Error propagation: Having determined the error Δx for x, what error Δy is there in y = f(x). A first-order Taylor series expansion gives dy ≈ (df/dx)Δx. It's amazing how small errors can propagate to large errors.
Why This Matters Later
When performing a tolerance stackup, tolerance amounts are combined using statistics to estimate the assembly variations. The reliability engineering field studies the failure rates of parts or systems by modeling them using probability distributions to account for experimental uncertainty. Generally, you don't have to be a statistician, but you need to know some basic concepts related to this field.
Why Engineers Use Numerical Methods
Real engineering problems rarely have analytical solutions. Instead, we get to wrangle with nonlinear differential equations, complex geometries, and problems involving multiple physical phenomena all at the same time. As the problems get harder, symbolic math solutions hit walls very quickly. Numerical methods can still solve our problems with quantifiable accuracy, however. Finite element analysis is a powerful numerical tool used to solve problems described by Partial Differential Equations (PDEs) by discretizing them into much simpler systems of equations that can be solved numerically, one at a time. CFD or Computational Fluid Dynamics analyses are another great example of numerical methods in action as they solve the fluid flow equations to solve problems relevant to engineering design. Optimization algorithms can also be employed to search millions of design permutations until a solution is found that could never be solved by hand. Even simple signal processing tasks, such as filtering, can be more easily accomplished by transforming a time-domain signal into the frequency domain via techniques involving the Fourier and Laplace transforms. This book explores the numerical foundations of the software that enable us to solve some very hard engineering problems that would otherwise be intractable.
Apply Numerical Methods in Specializations
Numerical methods and computational techniques are critical in simulation-heavy specializations:
- Structural Analysis & FEA - Apply finite element methods to solve complex stress and deformation problems
- Fluid Mechanics & CFD - Use numerical integration and differential equation solvers for flow simulations
🎓 Level Complete!
You've completed all five levels of Engineering Math. Ready to explore other engineering fundamentals?
Return to Engineering Math Overview →