Today we're going to talk about single view geometry and let's start with a picture. A simple picture like this, a picture of a street scene. We see the lane marker, we see the distance, we see the building. What else we see? We see the lights. We see that the lane markings are getting shorter and shorter. How about the camera man himself? Do we see ourself in the picture? While we don't see ourself physically, but we do see ourself oriented in the street, in the sense that, we are facing straight into the street lane markers. Do we also see, know we are looking down or looking up. You probably can guess this, that we are looking up rather than looking down, because the horizons are up from where we're looking from the street. And the information tells us how we are oriented in this space, just from a single picture itself. So this course will start looking at single pictures and moving to sequence of pictures and videos. From those we will try to infer the geometry of the scene, as well as, how the camera man is looking into the scene. All this started with the simple fact that we live in the three dimensional world. That when we measure the world we tend to measure in x, y, z location, where three numbers are x, y, and z. But when we take a picture of it, that three dimensional location has become a two dimensional plane. And that plane could be the camera, or it could be our eyeballs, but in short, we have only a two dimensional description of the world. We had lost a third dimension in the process. So that's the first fact. The second fact is that when we take a picture it matters how we are oriented to the world. Depends how we orient ourself to the world, the image of that point, project to different points in image plane. So those two facts are what we'll start using for the rest of the classes, is simple facts, but they will become more interesting as we go on. So, let's start thinking about how we take a picture. All of us have a camera now, all of us have a cell phone has a camera, and we often forget how easy it is to take a picture, or how difficult it was to take a picture when we don't have the cell phone camera. Here's a device back in the 18, 1600s called a perspectrograph. And perspectrgraph's made of in mechanical devices helping artist draw a picture of the world as if there's a camera. So the device is made of two rods, a horizontal rod and a vertical rod, the horizontal rod is fixed. And the artist, looking through a scope, and move the vertical rods left and right such that the line of sight between the eye, the little hole and objects line up with the vertical rod. And then what he does is, he read out the position on the horizontal rods, position on the vertical rods. He asks his assistant to write it down where it is on the piece of paper. So it took two people to draw a diagram, and here's a video illustrating this process. This is pretty clumsy as you can imagine this take two person to draw our pictures. In the 1600s there was another device made up of two parallel rods that simplify the process, that requires only a single person to take a picture. Here's a video of it. So the rods are no longer horizontal or vertical. They're made of two parallel rods that attach to a vertical rod. The person looking through the object with line of sight lining up with the top rod. And then painted with the bottom rods on the piece of paper. Yet another device that we have is called a Bi-Dimensional Perspectograph, and that's a much more interesting complicated device. This allow you to draw a picture of a three dimensional shape on the same piece of paper, on the same plane. Here's a video illustrating this process of taking a picture of this rectangular object. And what we're trying to do is draw a picture as if this rectangle is a scene in a vertical play, but we'll have this mechanical device to help us draw it. On a horizontal paper. This device required two linkage straight rods pivoting around a point, and it's actually tracing through a set of geometrical constructions, points of this rectangle on the vertical plane as seen As is through a picture plane. But again we going to imagine this paper plane is placed horizontal on the table. My colleague Costos will actually give you a more precise construction of this geometrically. So now of course we don't have to go through this complicated process of making a picture. We can just take our cell phones, go out. And just click, with a click we can take a picture. And we will start understanding, given this single picture, how do we see the world, and how does the world reflect the geometry of the camera person himself? For example, we would like to know what are the lengths of This vertical post in the building. If I tell you one of the posts is made about two meters tall, and we want to know how tall is the other post? Well we could probably guess, because they're made of horizontal lines of the same length. We probably can guess they're are about two meters as well But how about the person standing between the two posts? How tall is he? We can probably infer a little bit less than two meters, but exactly how tall is he? We won't actually figure this out from just this single picture. Another thing we want to know is exactly the length of the window, the height of the window Is the person taller than the window or shorter than the window. If I will can only infer this from the single picture. This line of work in fact was started by Antonio Criminisi. And was documented in the newspaper in 1999 So fairly reason work. In this work he showed that from the single picture, he's able to reconstruct three-dimensional measurements in the picture. He was able to measure how wide the floor is, how tall the piano is, how big the room is. As a result he can do his three-dimensional simulation of this world seen from a single picture. So how can we do this? Without even thinking about too much about the math, how do we do this? The first idea is imagine we are looking at this picture of a hallway. And we would like to measure distance on the floor. So how will we actually go about measuring this distance on the floor? Any ideas? But what ideas we have is we know this floor is made of tiles. And the tiles are made of rectangular shapes meaning that the two lines on a horizontal direction, they are parallel. And two lines in the vertical direction are Parallel as well and they form in 90 degree angles. So one idea is we match and we can virtually change the single view picture such that the lines which are supposed to be parallel, stay parallel. Or we could do it, just imagine we can deform the picture into so that Lies in the vertical directions. Move slowly, not converging to a point but it starts staying parallel to each other. Once we create this image we can sort of go in to the picture in this virtual view and start measuring distance and angles. That's one idea that requires us imagining how the picture looks like by moving lines, which are supposed to be parallel into parallel lines, and the angle is supposed to be completely rectangular into 90 degrees. Another idea we're going to use Is the idea of vanishing points. Vanishing points are the concept of a point which is at infinity actually can be seen. And this is a little bit strange concept, but in the image we actually see them all the time. For example we have this picture of the building we saw earlier. That we see the roof and the ground planes form a horizontal line. And those set of lines are parallel in physical space because we built the building this way. In fact all the lines on the roof they are stayed parallel to each other. And those parallel lines, if you look at it, they are tilted. On the fact to converge to a single point. And this point is called a vanishing point. Similarly we can see others [INAUDIBLE]. There are planks on the building, and those lines are staying parallel in the physical world. If we're looking under the plank, they're still parallel. But somehow when we take a picture of this The lines start converging, if I could converge the single point, and is also called a vanishing point. In fact there's many, many vanishing point in the world. And, they lies in a particular lie, which we'll call the horizon, called the vanishing line And this concept allow us to reason about the position of the camera man to the scene. As well as reason about the elements in a scene and the lengths of those elements relative to each other. Let's look at this situation more precisely. Here is a image of the plane which is your camera centred in front a camera center which is marked in red and we look it straight on to the world. Imagine there's a ground plane in front of us ans we're tracing a line on the ground plane. As those points move in this physical space They are projected to the vertical image plane, as is shown by straight line projections. As you can see, as we trace our line out in the physical space they project to a line in the image space. As we move further and further out with equidistant steps Those projection points getting closer and closer to each other. And in fact they will converge slowly to a point, which is at infinity as if we were looking straight out in the direction with this line through the camera. And that's it's point of a vanishing point. The interesting thing about vanishing point is first of all there are many physical lines in a physical space, they are not touching each other so long as they're in the same direction they will go out and eventually meet each other at a point of infinity. That's the vanishing point with that set of lines. And this is a very strange concept because we physically have never seen a vanishing point. No matter how big your camera is, how good your lens is, you can never see a point in infinity. But what we see is intersection of those projection lines forming a physical point corresponding to a point in infinity which is many, many light years away. In fact, it's never going to be reaching that point but is a physically situated in the picture. Here's an overhead view In a two dimensional plane. So imagine we have looking camera plane as a vertical plane, and we're looking at a set of lines tracing out. All the lines are physically away from each other, they are not touching each other, but in fact all other points on those lines parallel to each other will converge To a point in infinity where a projected image is marked at the vanishing points. And that's the main concept we will use in the next few lectures. Now it's interesting to turn things around to ask the question, what if we have Lines that appear parallel in the image plane, how do they look like in the ground plane? So, we can imagine the situation. We have a set lines on the ground plane and when they take a picture of it, this picture Actually in fact shows those lines not staying convergent or divergent but those lines stay parallel to each other. So who are those lines? At this point I actually want you to take your cell phone out, and do the following exercise. And the exercise is fairly simple, that you were to draw a set of radiating lines On a piece of paper, single point conversion, reading straight out. I would like you to take you cell phone and simply move your cell phone such that long directions radiations and keep the camera vertical to the paper until you start seeing. Projection your lines start looking vertical as you see in the right. And mark this point of where the camera is rather the conversion point is, we'll come back to that exercise later.
