If you’ve taken a comp sci course, you’ve probably heard of vectorization. It refers to the process of taking a loop and trying to express it in a mathematical way so that a computer can interpret it at once. The idea is that computers are fast; very fast. But they only do one thing at a time. What we want to do is tell the computer to do multiple things at once, without having to iterate through each item and perform the same task on each one.
In other words, vectorization is important because it lets us get faster runtimes with less code. That sounds like an amazing deal, right? So why don’t people do it more? Unfortunately, vectorization is hard! Especially when you’re still learning about programming, or your programming language. Fortunately for you though, I’m going to teach you how to vectorize your code.
Let’s start out with the basics: what is vectorization and why should we care? In the simplest terms: vectorization is the act of converting a piece of code from using loops (iteration) into using vectors (mathematical arrays). When you don’t have any experience with vectorized code, it can be very difficult to understand why anyone would want to go through all
Vectorization describes a process in which you take a task and replace the scalar operations with vector operations. For example, if you have a loop that performs basic arithmetic, such as addition or multiplication, on every element in an array, that would be a scalar operation. By vectorizing it, you can replace the loop with some built-in math functions to do the same thing.
In this blog post, I will provide some examples of why vectorization is important and how to use it in your own code.
A little background on vectors
There are several types of vectors in computer science: arrays, matrices, tensors. The simplest vector is an array. An array is a collection of values stored together at contiguous memory locations. They are indexed by integer numbers starting from 0 and each value can be accessed using its index number. These are very similar to Python lists (or NumPy arrays).
Although the advent of computers has led to a much more streamlined life, they have also made it much more difficult to get things done. With so many more options available, we have to make decisions in our lives every day before we know what is going on. We have to figure out what is important and what is not, who will be important in our lives and who will not be, who we should listen to and who we should not listen to, and how we should act in general. It can be exhausting.
But when it comes to the computer, the only thing that matters is how fast you do things. If you do things too slowly, you lose. So when you write code for a computer, you want it to run as quickly as possible, because if it doesn’t then someone else’s program will beat yours or someone else’s computer will run faster than yours. And if you don’t care about speed then there are plenty of other reasons why your code should run as quickly as possible: it makes debugging easier, it makes testing easier, and it just makes your life easier in general.
So what is vectorization? It’s the process of taking a set of data and turning it into a list of numbers (or
Vectorization is important in computer programming because it improves the performance of code.
Vectorization is a programming style that favors operations on whole arrays at once over operations on individual array elements. A vectorized implementation of an algorithm uses primitive constructs (e.g., loops) as little as possible, replacing them with calls to high-level functions that operate on whole arrays. Vectorized code can be more concise and easier to write than non-vectorized code, while also often being faster.
Matlab is a popular language for scientific computing, and the Matlab compiler allows you to compile your Matlab code into executables or C libraries, which can then be used from other languages like C++, Java or Python. However, a major drawback of Matlab is that it is slow when used in a looping construct (for example, a for loop). This means that if you want to implement algorithms involving loops in Matlab, you will need to use vectorization to improve the performance of your code.
Vectorization is very important when it comes to optimizing code. Vectorization allows you to operate on multiple data points at the same time in a single operation, which is much more efficient than operating on one at a time. This can greatly improve the performance of your program if you do it right.
For example, let’s say we are working on a game and want to update all of our gameObjects every frame. In vanilla C, this would look something like:
for (int i = 0; i < numGameObjects; ++i) { GameObject* obj = &gameObjects[i]; obj->transform += obj->velocity * dt;
}
This will work just fine, but we can make it better by vectorizing it with SIMD intrinsics. SIMD stands for Single Instruction Multiple Data, which means that we are able to apply the same instruction to multiple data points at the same time. For example, SSE has 128 bit registers which can store 4 floats or 2 doubles. So if we have an array of floats or doubles we can load them into an SSE register and apply an instruction once to manipulate all 4 floats or 2 doubles instead of doing it individually for each element in the
Vectors, matrices, and arrays are used in a wide range of computer science applications. These data structures can be represented as columns and rows of numbers. Because of their importance in so many aspects of computer science, it is very important for programmers to understand how to work efficiently with these data structures.
A vector is a one-dimensional array of numbers. A matrix is a two-dimensional array of numbers. An array is an n-dimensional array of numbers.*
There are many different types of processors. In this article I am going to describe the general structure of a processor and why vectorization can be important in some scenarios. If you have never used a vectorized code you will find this article very useful. This article is not the end all, but it should give you the motivation to learn more about vectorization.