Hi there! My name is Parker, welcome to my free blog on Numerical Programming, a topic I am extremely passionate about. What is Numerical Programming you might ask? There are a lot of definitions out there, but I prefer to break it down into five discrete steps.
- Defining an Engineering Problem
- Modeling the World with Approximations
- Algorithm Design
- Software Implementation
- Analysis and Visualization
Engineering problems are all around us. Understanding how different loads and torques impact structures, estimating how vehicles move through space, and an infinite number of other possibilities that you could dream up. While physics is about modeling the absolutes of the world, engineering is about solving real-world problems, while contending with the constraints and limits that physics impose on us. Numerical methods are a tool that any engineer can use to solve messy, real-world problems where a closed-form, elegant chalkboard solution probably does not exist.
The first two steps of using numerical methods involve identifying a problem and constructing a model of the world. Oftentimes, we start with the raw physics and make simplifying assumptions from there. For example, if you wanted to model the kinematics of a projectile flying through the air, you might start by asserting that the law that governs the projectile motion is Newton’s second law, F = ma.
While you could try to model every possible force acting on the projectile (gravity and all its perturbations, air resistance, wind gusts, etc.), this isn’t feasible in practice. If we tried to model every phenomenon, our work would be unending and we would never build anything real! One of the key challenges of the engineer is making decisions about which factors are important to model, and which ones can be ignored. A pragmatic engineer might look at the problem at hand and decide that a simple gravity model and a cannon-ball air resistance model is “close enough” and move along.
Once we have an appropriate model of our world, we can design an algorithm to solve our problem. In the example problem, we want to understand the projectile’s kinematics over time, meaning tracking it’s position and velocity vs time. In practice, the exact quantities you estimate are highly problem specific. Once again, knowing what to estimate is half the job of the engineer! Luckily for us, we have a model that relates the forces on the projectile to the acceleration and we know that velocity and position can be integrated from acceleration given a set of initial conditions. If we can estimate the forces, we can figure out the kinematics!
To design a well-formulated algorithm, we need to not only define our preferred mathematical model, but also discretize it into a form that a computer can solve. For our problem, this might involve setting up a numerical integration scheme, like the trapezoidal rule or a Runge-Kutta algorithm. Once again, the engineer is tasked with selecting the appropriate algorithm (notice how we keep coming back to this??).
Next, with our well-defined problem and a discrete algorithm for solving it, we can turn to software. Oftentimes, higher level scripting or programming languages, like Matlab or Python, are used to prototype a numerical method. Solutions that need to run in real-time on the edge often have higher performance requirements than one can get with these higher level languages, especially for more compute-intensive algorithms. For this reason, our final implementation is often done in a lower level language like C, C++ or Rust.
Numerical programming may also require interaction with special hardware devices, like GPUs to further accelerate processing. The art of designing real time software is a challenging, yet worthy endeavor. It requires balancing a litany of factors like compute/memory constraints, performance and accuracy requirements and more.
Finally, your algorithm runs and you collect a bunch of data. Now what? Analysis and visualization is oftentimes performed offline to evaluate the accuracy and performance (speed) of an algorithm. This work is important not just to you as the engineer to ensure your solution works, but also as a means of communicating with non-technical people. Look, it’s no secret that most engineer’s have some room for improvement in their communication skills. Analysis and visualization is like a Rosetta Stone between engineers and program managers or their customers.
Simple enough right? Numerical methods are a huge field that can often be intimidating, but it doesn’t need to be. My goal with this blog is to write tutorials, with runnable code included, to help make many of these concepts easier to understand. I also hope to inspire the next generation of engineers to enter the field and push the boundaries of scientific computing. I hope you enjoy, make sure to check back soon for more content and happy learning!
