October 17, 2022
Discover exactly how computers take zeroes and ones and use them to generate convincing 3D spaces.
It is often said that computers “think” in ones and zeroes. This notion is a fact for most conventional computers. Due to the way the circuits are built, the most reliable way to store, retrieve, and process data is by flipping electronic switches called transistors on (1) and off (0).
So how, I once wondered, do computers take those ones and zeroes and manage to create the rich immersive world of Ancient Greece in Assassins Creed: Odyssey, or the fluid acrobatics and web-slinging movements of our friendly neighborhood Spider-man?
In the following sections, we will see exactly how computers take these zeroes and ones and use them to generate convincing 3D spaces.
First, we must understand how a binary number system can represent numbers greater than one. Let’s start by examining the number system we’re already familiar with, decimal. Decimal is called that because it gives us ten numbers to work with. One through nine and zero, which represents no value at all.
When we count in decimal, we can use a single digit to represent a value up to nine. But in order to represent a value of ten, we must use two digits. The one now represents ten, and the zero represents that it is no more than ten. We can think of it like this
With binary, we use the same pattern to represent numbers with more than bit (binary digit). The following table shows how this is done.
Numbers can be used to represent an amount of something.
How many apples do I have?
How much money does this cost?
These questions can be answered with a single number. These values are called scalars in mathematics.
Scalars have magnitude which is a fancy word for how much of something there is.
The important bit here is that scalars are often used to represent distance, or how far one point is from another point. This is important.
If the whole world consisted of one dimension, it would look like a line. Suppose we mark the center of this world with a blue circle. This center point is called the origin of the world.
In this world, describing to someone where you are is very simple. You can say, “I am 8 miles ahead of the origin”. Or if you are on the other side, you can say “I am 8 miles behind the origin”. In mathematics, we can tell ahead and behind as positive or negative numbers.
Since we can always refer to this distance as a single number, one dimensional locations can be represented with a scalar.
If the whole world consisted of two dimensions, it would be flat like a piece of paper. This makes 2D math very convenient when all you have is pencil and paper. The system we use for 2D math is called the Cartesian Coordinate system. This system uses the letters X and Y to represent the two dimensions. In this world, we can represent position with a collection of two numbers called a coordinate.
In this world, we can use these coordinates to describe any position away from the origin using a coordinate. In the case of (2,3) we are two units to the right of the origin, and three units above the origin. When representing position, these are called coordinates. But we can also represent this same value with a vector. A vector is a value with both magnitude (length of the arrow) and direction (the way the arrow is pointing).
Vectors are a handy way to think of coordinates to convey movement away from the origin. This is formally called displacement in mathematics.
At this point, it gets a little difficult to visualize positions because we are modeling this on a 2D screen. Still, we can use the illusion of depth to simulate a third dimension.
In this model, our point (2,3,3) is:
X: 2 units right of the origin.
Y: 3 units above the origin
Z: 3 units in front of the origin.
In this way, we can use numbers to describe the position of an object in our 3 dimensional world.
3D artists use specialized programs like Blender 3D, to create 3D models. In this software, they have the ability to create points in 3D space called vertices. Each vertex is represented by a coordinate or vector that tells the computer where it is relative to the origin of the mesh.
Let’s take Codie here for example:
Each point in the figure above has a coordinate that represents the distance between itself and the origin of the 3D model. The computer has the coordinates of each vertex in its memory which it uses to understand the shape of the model.
The computer then looks at how these vertices are connected. It says “this vertex has lines connecting it to this particular vertex, and this one, and this one…”. The connections between vertices are called edges.
Three or more edges can be joined together to form a face. Faces make up the surface of a 3D model’s geometry. The vertices, edges, and faces of a 3D model are called a mesh. Later, we can tell the computer that those faces are opaque, and form the surface of the model.
Using a technique called UV Mapping, artists use 3D software to take a 2D image and wrap it around a 3D object. Some software allows artists to paint directly onto the model from any angle with a digital brush. The computer then produces and saves the 2D image to use.
A texture by itself, however, just tells the computer the color of each face. By itself, a 3D object with just a texture looks pretty flat. To get a convincing illusion of a 3D object, the object needs a shader.
The shader tells the computer how the surface should behave when it is hit with light. Shaders can control how shiny or dull a surface is, how it casts shadows, if it is transparent or opaque, and more. Modern shaders also use special tricks to smooth out sharp edges to give the illusion of perfect curves, even if the geometry underneath is blocky.
In order to render a realistic 3D model, the computer also needs to know the locations of the light sources, their color, and their intensity. Then it needs a virtual camera that can render a 3D scene with depth by converting it to a 2D image made of pixels of different colors.
For many objects, once they are shaded they are finished. However, some objects require additional data to allow animators to make them come to life. The artist uses a technique called rigging where virtual bones are strategically placed at different parts of the mesh. This way, parts of the geometry can be grouped together and animated instead of having to describe the position of each individual vertex at a different point in the animation. The computer knows the location of the bones in the same way it knows the location of vertices, using coordinates which are just collections of numbers, which are represented by ones and zeroes in the computer.
The process outlined so far is repeated for every object in the game world. Game studios may have dozens or even hundreds of artists working together to create every item that exists in the world. Sometimes artists specialize in a specific type of modeling. In a single art team, there may be sub-departments dedicated to props, characters, foliage, buildings, and even level designers whose job it is to organize all of these models into a level.
Game engines like Unity allow programmers to write code that describes the rules of the world, and how all of these assets work together to create an immersive 3D experience.
Now that we know how all the building blocks are assembled, and how they physically manifest in the computer as ones and zeroes, we can imagine how this process repeated for many scenes and levels can create entire virtual worlds.
At Mastery Coding, we show our students how to create their very own 3D worlds. Concurrently, students explore careers, build portfolios, and work towards obtaining real industry certifications!
.png)
Juan is Mastery Coding's EVP Product. He has contributed to our platform and courses at a deep level and is driven by an innate desire to create quality computer science curriculum that empowers students to create software they can get excited about.
