Usage

Here is short introduction on using ReconstructMe. Please read this from front to back to ensure maximum reconstruction success.

Topics

1. How do I start ReconstructMe?
2. When I start ReconstructMe, a window pops up and closes immediately?
3. How to perform real-time reconstruction?
4. How to scan multiple volumes and stitch them together?
5. How to select a different sensor?
6. How to select a different device to perform computations on?
7. How to change the reconstruction configuration?
8. How to perform offline reconstruction?
9. How to resume a previously started reconstruction?
10. How to calibrate my depth sensor?
11. What coordinate system are used?

How do I start ReconstructMe?

1. If you have not done so already, follow the installation guideline.
2. In the Windows Explorer, go into the ReconstructMe directory. You should see somewhat like this:
   

3. When you scroll down, you will see a few files named “Start ReconstrucMe ….“. These are the files you can doubleclick to start ReconstructMe. Each of these batch files uses a some pre-configured settings to get you started.
4. Now, what batchfile do you start? The most important question is, whether you have a Kinect for Windows, a Xbox Kinect, or an Asus Xtion:
   • Doubleclick Start ReconstructMe Scan MSKinect.bat if you have a Microsoft Kinect
   • Doubleclick Start ReconstructMe Scan OpenNI.bat if you have a Xbox Kinect or Asus Xtion.
5. After starting, you will see a Windows Console window that looks like this:
   

6. To continue, press the key y, then Enter.
7. If you have followed the installation guideline, doubleclicked the correct batch file, and have a supported graphics card, you should now get a GUI window:
   
8. Now you can click that GUI Window (to make sure it is the active window), and press p to switch between play and pause mode to start reconstruction. To quickly restart scanning if something went wrong, press r.
9. After you have finished scanning, press Esc (top left key on your keyboard). This will close the GUI window, and leave you with the Console window where you can save your scan.
10. To save your scan, answer the question Save content? (y/n): by pressing the key y and then Enter.
11. You will now be asked to enter a filename:

    It is important that you enter a name with extension. For example, enter MyScan.obj as above. The extension automatically determines the file format. .obj is usually a good choice because the file will be small and can be quickly opened in e.g. Meshlab. See What output formats does ReconstructMe support? for a list of supported file formats.

12. The resulting file will be written in the same directory as ReconstructMe, so you might have to scroll up and down a bit in the Explorer to find it:
    

13. Now you can open your 3D model in your favorite application. That’s it, you now have performed a 3D scan using ReconstructMe!

When I start ReconstructMe, a window pops up and closes immediately?

If you have started the correct batch file (see How do I start ReconstructMe?), you can find error logs in the directory logs. Go into this directory, and doubleclick the last file:
This will give you the error messages. Have a look at the FAQ, maybe your problem is described here. If you do not find any answer, We have an active Forum where you can search for your problem and get help quickly.

Good luck!

How to perform real-time reconstruction

Make sure you have completed the installation instructions before continuing. ReconstructMe is currently a console application, which is configured via a command line interface. To reconstruct using live sensor data use the --scan command.

[sourcecode language="text"]
reconstructme.exe –scan
[/sourcecode]

The above command will try to open the sensor via the OpenNI backend. If you use Kinect for Windows, make sure you read How to select a different sensor?.

After agreeing to our license terms, the above command will start the reconstruction in paused mode. You can switch between paused and play mode by pressing the ‘p’ key with reconstruction window in the foreground.

Reconstruction happens in a predefined volume. By default the size of this cube to a side-length of one meter. When you press ‘p’ for the first time, the position of the volume will be fixed relative to the current sensor position as illustrated in the following top/side view sketch:

Reconstruction works best when you know about a couple constraints and keep them in mind when performing.

Points of failure

• Violating view point constraints
  Tracking can only succeed when the volume to be reconstructed is in view. Pointing the sensor away from the volume or stretching the distance between the sensor and the volume to more than 2000mm will lead to reconstruction errors. Keep in mind that the Kinect sensor does not see anything closer than 400mm. Objects within that range won’t be seen.
• Violating geometry constraints
  Stable tracking requires geometry that is non-planar and non-symmetric. Make sure that whatever you reconstruct does not entirely consist of a single plane, a cylinder, a sphere or anything else that looks alike from different perspectives. When keeping the sensor fixed and turning an object infront of it, make sure everything inside the volume turns.
• Violating acceleration constraints
  Fast movements of the sensor will lead to blurry data and thus to reconstruction errors.

Avoiding these points will improve your reconstruction success a lot. To aid you in avoiding reconstruction errors, we’ve added a tracking failure detection that attempts to detect points of failure and stops reconstruction until the sensor is moved to a recovery position. The details are shown in the following video

Upon closing the reconstruction window, you will be queried if you like to export the scanned model. Type ‘y’ for yes and then type in a filename. We currently support the following file formats: .stl, .obj, .osg, .3ds and .raw. The correct exporter is chosen based on the filename extension you type in.

How to scan multiple volumes and stitch them together?

ReconstructMe supports a scanning mode that allows the user to scan multiple volumes and stitch them together like a panorama image in 3d. In order to start a new multiple volume project, you need to use the --multiscan command

[sourcecode language="text"]
reconstructme.exe –multiscan
[/sourcecode]

This command is very similar to --scan and shares a lot of available command switches. Since we are about to scan multiple volumes, ReconstructMe will place all volumes belonging to one project inside a folder. If you don’t specify any other this folder will be called default_project. Once ReconstructMe starts press ‘p’ to start scanning. Press ‘n’ to create a new volume at the exact position and orientation of the current sensor position. While scanning the new volume, the previous volume will be downloaded in the background and stored as .raw.

It is important to note, that there should be overlap between the old volume and the new one created to allow the stitching process to refine the position. Also note, that only one volume can be downloaded in the background. Wait for the volume to complete the download (see console to get a feeling), before pressing ‘n’ again. Also note that all volumes committed by pressing the ‘n’ key are part of the project.

Once you have completed scanning volumes, use the --stitch command to convert the .raw parts into matching 3d surfaces.

[sourcecode language="text"]
reconstructme.exe –stitch
[/sourcecode]

By default this command loads all volumes from default_project that haven’t yet been converted, stitches them together and saves them as .ply output. Note that --stitch does not fusion volumes, but re-positions them to fit together.

In case you used a non-default --config with the --multiscan command, you need to pass the same config to the --stitch command.

In case you wan’t to resume at a previously scanned volume (say volume_3.raw) simply invoke

[sourcecode language="text"]
reconstructme.exe –multiscan –resume_raw default_project/volume_3.raw
[/sourcecode]

This allows you refine this particular volume and offers the possibility to create new volume branches using the ‘n’ key. Once you are done, simply invoke

[sourcecode language="text"]
reconstructme.exe –stitch
[/sourcecode]

again. This will only load the modified or new volumes and stitches them. Since stitching is not always perfect, you might need to post-process the volumes in tools such Meshlab to improve the accuracy of the output.

How to select a different sensor?

By default ReconstructMe tries to open the first sensor through the OpenNI backend. In case you want to use the Microsoft Kinect backend or have multiple sensors attached or, you can use the --sensor switch to select the correct driver/sensor. Here are some examples

• --sensor mskinect,0 Instructs ReconstructMe to use the Microsoft Kinect backend and open the first sensor.
• --sensor openni,1 Instructs ReconstructMe to use the OpenNI backend and open the second sensor.
• --sensor mskinect,1 Instructs ReconstructMe to use the Microsoft Kinect backend and open the second sensor.

This switch is simply added to the invocation of ReconstructMe. Here’s an example

[sourcecode language="text"]
reconstructme.exe –scan –sensor mskinect,0
[/sourcecode]

How to select a different device to perform computations on?

By default ReconstructMe selects the ‘best’ OpenCL compliant device found: If no GPU device is found, the first CPU is used. In case you have multiple devices you can use the --device switch to select a different one.
Here are some examples

• --device 0 Use the first device.
• --device 1 Use the second device.

This switch is simply added to the invocation of ReconstructMe. Here’s an example

[sourcecode language="text"]
reconstructme.exe –scan –device 0
[/sourcecode]

Available devices can be listed by calling

[sourcecode language="text"]
reconstructme.exe –list_devices
[/sourcecode]

How to change the reconstruction configuration?

ReconstructMe comes with a set of predefined parameter sets. Those parameter sets are stored as text files inside the cfg sub directory. In order to tell ReconstructMe to use a different configuration, you can use the --config switch.

The following command will load the parameters for scanning a 3m volume using highres mode.

[sourcecode language="text"]
reconstructme.exe –scan –config cfg/volume_3m_highres.txt
[/sourcecode]

You can change and create new parameter sets easily. Just copy an existing one and play around with the settings. Keep in mind that there is no official parameter documentation. A look at the content of volume_1m.txt and reading the coordinate systems section carefully might help.

How to perform offline reconstruction

Your GPU is too slow for real-time reconstruction or you want record for later reconstruction? No problem, we have considered this use case and equipped ReconstructMe with the --record feature. It allows you to record raw sensor data and reconstruct offline using the --replay command.

[sourcecode language="text"]
reconstructme.exe –record
[/sourcecode]

Use ‘p’ to play/pause. Upon exit the output can be reconstructed by

[sourcecode language="text"]
reconstructme.exe –replay –depths reconstructme.depths.gz –colors reconstructme.avi
[/sourcecode]

Upon closing the reconstruction window, you will be queried if you like to export the scanned model. Type ‘y’ for yes and then type in a filename. We currently support the following file formats: .stl, .obj, .osg and .3ds. The correct exporter is chosen based on the filename extension you type in.

How to resume a previously started reconstruction

In order to be able to resume a real-time reconstruction at a later point in time, you need to save the volume with the .raw extension. To resume reconstruction, you simple use the --resume_raw switch and add the previously saved .raw volume filename. Here is an example

[sourcecode language="text"]
reconstructme.exe –scan –resume_raw volume.raw
[/sourcecode]

When ReconstructMe starts, the viewpoint you need to align your sensor with will be shown on the right. Once ReconstructMe has confidence that the viewpoints are aligned, the bar on the bottom will turn green reconstruction continues. Note that ReconstructMe starts in paused mode, so you need to press ‘p’ before ReconstructMe is able to test for alignment.

In theory this process can be repeated any number of times in order to continuously update your world and eventually create a polygonal model from it. A raw volume in standard resolution will take around 5MB of disk space. A raw volume in high resolution around 30 MB.

Note that the configuration settings should be the same across reconstructions. In case you used the --config switch to start a reconstruction, you should use the same configuration file to continue reconstruction.

How to calibrate my depth sensor?

ReconstructMe ships with built in functionality to calibrate your depth sensor. This functionality is only available with the OpenNI backend, because it is the only backend that provides access to the raw infrared stream. To calibrate your depth sensor follow these steps.

First, print a chessboard pattern in either A4 or A3 (recommended). Ready to print patterns are provided inside the patterns sub directory of ReconstructMe. The pattern is best printed on a laser printer with pigments opaque to infra red, as it is discussed in this thread. Here’s an image provided by Francis that shows contrast differences of a ink-jet printed chessboard pattern (A3 background) and a laser printed one (A4 foreground) as observed by the sensors infra red sensor.

Once you have your printed pattern, ensure that the printer did not distort the chessboard pattern (i.e squares in the pdf remain squares when printed). Measure the size of a square in millimeter and save it for a later step.

Next, cover the infrared emitter on your sensor with a paper, your hand or something similar. This is necessary to improve the image quality. Below are two images of a chessboard: on the left side without covering, on the right side with cover.

Start ReconstructMe with

[sourcecode language="text"]
reconstructme.exe –calibrate –square_size 33.875
[/sourcecode]

The --square_size of 33.875 is the default for A3. Exchange this with what you have measured previously.

ReconstructMe will start up and on the left side you will see the current infrared image. On the console you will see an estimation of the factory calibration. Put your chessboard on the ground (it has to lie down flat) and position the camera so it can see the entire chessboard. Press ‘n’ to take an image of the chessboard. If the chessboard is detected correctly, you will see that on the right side of the viewer (see below).

Next, take roughly 10-20 views from the chessboard from different angles and distances. The quality of the calibrations improves when you move as close as possible, so that the chessboard covers the entire chip.

Once you are done, close the viewer. The calibration will start automatically. Once it has finished, it outputs a reprojection error and intrinsic depth calibration values. Here are some typical values.

[sourcecode language="text"]
Reprojection error in pixels 0.297901
Calibration depth intrinsics are
camera_fx: 589.546
camera_fy: 590.936
camera_px: 326.965
camera_py: 249.184
[/sourcecode]

The reprojection error should stay below 1 pixel. If not, repeat the process. Finally, copy those values (fx, fy, px, py) over to your configuration file.

What coordinate system are used?

ReconstructMe uses a couple of right-handed coordinate systems that you might want to be aware of.

For one ReconstructMe uses the sensor coordinate system. This coordinate system is attached to the sensor in the following way

Next, ReconstructMe uses the world coordinate system. This frame aligns with the sensor coordinate system when the reconstruction starts. From then on, the world coordinate system stays fixed. The volume to be reconstructed is expressed in terms of this coordinate system. Below is an example for the world coordinate system

Finally, ReconstructMe uses the CAD coordinate system. This coordinate system aligns with the world coordinate system but is rotated around x+ about 180 degrees.

We use this coordinate system exclusively when exporting the model to common 3D file formats, in order to allow third-party programs to display the content correctly upon opening.