Reconstruction

From NaturalPoint Product Documentation
Jump to: navigation, search
Back To Page References


The Reconstruction Properties pane can be accessed under the View tab in Motive or by clicking Tb12.png icon on the main toolbar.

Reconstruction in motion capture is a process of deriving 3D points from 2D coordinate information obtained from captured images, and the Point Cloud is the core engine that runs the reconstruction process. The Reconstruction pane in Motive contains all of the settings and filter options to modify the engine's parameters, and it can be accessed under the View tab. These settings can be modified to optimize the quality of reconstruction depending on the conditions of the capture and what you're trying to achieve. You will find that this pane is used most often during Live mode, 2D playback, and when reconstructing a take in order to optimize the marker tracking for a given system.

For more information, visit Data Types page.

Click Here to see settings for Motive 1.9.


Point Cloud

Multiple marker rays contributing to the reconstruction of a marker.

The Point Cloud reconstruction engine converts two dimensional point from camera images into coordinates in a three dimensional space through triangulation. All cameras should be calibrated for the engine to function properly (see Calibration). The triangulation of a marker occurs when a minimum of 2 rays intersect. Rays are generated from the objects present on a camera image and they resolve into a 3D point when the conditions defined by the reconstructions settings are met. These rays can be seen from the Perspective View pane when the tracked rays and untracked rays are enabled from the visibility Viewport16.png settings.

Due to inherent errors in marker tracking, rays generally do not converge perfectly on a single point in 3D space, so a tolerance value is defined. This tolerance, called the residual, represents one of the reconstruction constraints. If a ray could be defined as an infinite series of points aligned in a straight line, two or more rays that have points within the defined residual range (in mm) will form a marker.

Enable Point Cloud Reconstruction

Default: ON

This toggles real-time 3D reconstruction on and off.

It is recommended to turn this off if computer resource need to be dedicated to 2D recording. When disabled, you will not be able to see 3D data from the Live mode nor from the recorded 2D data.


Options

Reconstruction pane in Motive.

Maximum Residual (mm)

Default: 10.00 mm

The residual value sets the maximum allowable offset distance (in mm) between rays contributing to a single 3D point.

When the residual value is set too high, unassociated marker rays may contribute to marker reconstruction, and non-existing ghost markers may be reconstructed. When this value is set too low, the contributing rays within a marker could reconstruct multiple markers where there should only be one.


Choosing the Residual Value

Depending on the size of markers used, the contributing rays will converge with a varying tolerable offset. If you are working with smaller markers, set the residual value lower. If you're working with larger markers, set this value higher because the centroid rays will not converge as precisely as the smaller markers. A starting point is to set the residual value to the diameter of the smallest marker and go down from there until you start seeing ghost markers. For example, when 3 mm and 14 mm markers are captured in a same volume, set the residual value to less than 3 mm. The ghost markers can appear on larger markers if this value is set too low.

The residual can also be viewed as the minimum distance between two markers before they begin to merge. If two markers have a separation distance smaller than the defined residual (in mm), the contributing rays for each marker will be merged and only one marker will be reconstructed, which is undesirable. Remember that for a 3D point to be reconstructed, it needs to have at least two rays contributing to a marker depending on the Minimum Rays setting.

If calibration quality is not very good, you may need to set this value higher for increased tolerance. This will work only if your markers are further apart in the 2D views throughout the given marker motion. This is because there is more errors in the system. However, for best results, you should always work with a calibration with minimal error (See Calibration).

  • Multiple tracked rays converging into a reconstructed marker.
  • Average residual value for a selected marker.

  • Maximum Ray Length (m)

    Default: None — the calibration solver will set a suggested distance based on the wanding results, but this can still be adjusted by the user after calibration.

    This sets the maximum distance, in meters, a marker can be from the camera to be considered for 3D reconstruction. In very large volumes with high resolution cameras, this value can be increased for a longer tracking range or to allow contributions from more cameras in the setup. This setting can also be reduced to filter out longer rays from reconstruction. Longer rays generally produce less accurate data than shorter rays.

    When capturing in a large-size volume with a medium-size – 20 ~ 50 cameras – camera system, this setting can be adjusted for better tracking results. Tracking rays from cameras at the far end of the volume may be inaccurate for tracking markers on the opposite end of the volume, and the unstable rays may contribute to ghost marker reconstructions. In this case, lower the maximum ray length to restrict reconstruction contributions from cameras tracking at long distances. For captures vulnerable to frequent marker occlusions, adjusting this constraint is not recommended since more camera coverage is needed for preventing the occlusions. Note that lowering this setting can take a toll on performance at higher camera counts and marker counts because the solver has to perform numerous calculations per second to decide which rays are good.


    Minimum Ray Length (m)

    Default: 0.2 m

    This sets the minimum distance, in meters, between a marker and a camera for the camera to contribute to the reconstruction of the marker. When ghost markers appear close to the camera lens, increase this setting to restrict the unwanted reconstructions in the vicinity. But for close-range tracking applications, this setting must be set low.


    Minimum Ray Count

    Default: 2 rays

    This sets the required minimum number of cameras that must see a marker for it to be reconstructed.

    For a marker to be reconstructed, at least two or more cameras need to see the marker. The minimum rays setting defines the required number of cameras that must see a marker for it to be reconstructed. If you have 4 cameras and set this to 4, all cameras must see the marker; otherwise, the marker will not be reconstructed and the contributing rays will become the untracked rays.

    When more rays are contributing to a marker, more accurate reconstruction can be achieved, but generally, you don't need all cameras in a setup to see a marker. If you have a lot of cameras capturing a marker, you can safely increase this setting to prevent false reconstructions which may come from 2 or 3 rays that happen to connect within the residual range. However, be careful when increasing this setting because a high number of minimum rays requirement may decrease the effective capture volume and increase the frequency of marker occlusions during capture.


    2D Object Filter

    Filter Type

    Default: Size and Roundness

    Toggles 2D object (Size and Roundness) filtering on or off.

    This filter is very useful for filtering out extraneous reflections according to their characteristics (size and roundness) rather than blocking pixels using the masking tool or the Block Visible feature. Turn off this setting only when you want to use every 2D pixels above the brightness threshold from camera views. When there are extraneous or flickering reflections in the view, turn on the filter to specify and consider reflections only from markers. There are multiple filtering parameters to distinguish the marker reflections. Given that there are assumed marker characteristics, filtering parameters can be set. The size parameters can be defined to filter out extra-small or extra-large reflections that are most likely from extraneous sources other than markers. Non-circular reflections can be ignored assuming that all reflective markers have circular shapes. Note that even when applying the size and roundness filter, you should always Block Visible when you calibrate.


    The size filter applied with a maximum size limit of 200 pixels. Any reflections bigger than 200 pixel-size is omitted from the Point Cloud reconstruction calculation.

    Min Thresholded Pixels (pixels)

    Default: 4 pixels

    The minimum pixel size of a 2D object, a collection of pixels grouped together, for it to be included in the Point Cloud reconstruction. All pixels must first meet the brightness threshold defined in the Cameras pane in order to be grouped as a 2D object. This can be used to filter out small reflections that are flickering in the view. The default value for the minimum pixel size is 4, which means that more than 4 pixels need a group of pixels needs to be greater than 4 for a ray to be generated.


    Max Thresholded Pixels (pixels)

    Default: 2000 pixels

    The maximum size of a 2D object, in pixels, in order for it to be included in point cloud reconstruction. Default is 2000 pixels which basically means that all of detected large reflections smaller than 2000 pixel-size will be included as a 2D object. Use this to filter out larger markers in a variable marker capture. For instance, if you have 4 mm markers on an actor's face and 14 mm markers on their body, use this setting to filter out the larger markers if the need arises.


    The circle filter omitting non-circular reflections from a 2D camera view.

    Circularity

    Default: 0.6

    The circularity value sets the 2D object filter sensitivity to the roundness of detected reflections. Valid range is between 0 and 1, with 0 being not round and 1 being perfectly round. This setting is relative to the resolution of the camera you are using. If you have lower resolution cameras, a marker will appear more pixelated and non-circular. If a marker is detected in only 5 pixels in a lower resolution camera at a set distance, the same marker will appear with more pixels from a high resolution camera and the captured reflection will have more circular perimeters.

    When using lower resolution cameras to capture smaller markers at a long distance, you may need to lower the circularity filter value for the reflection to be considered as a 2D object from the camera view. Also, this setting may need to be lowered when tracking non-spherical markers in order to avoid filtering the reflections.


    Intrusion Detection

    Default: Normal

    The intrusion detection feature allows cameras to recognize merged reflections and filter them out from the image. Merged reflections are susceptible to inaccurate centroid calculation and may result in jittering of the reconstructed markers. From 2D object images, both of the intruding and detected marker reflection will be omitted whenever there is an extraneous reflections in the vicinity, within the intrusion boundary. Available detection modes are None, Normal, and Aggressive, and by default, Normal intrusion detection mode is recommended.

    Intrusion Detection Modes

    • None: Intrusion detection is disabled.
    • Normal (Default): Creates a circular boundary, an Intrusion Band, around the detected marker centroids. If any reflection intrudes the boundary, the marker is filtered out.
    • Aggressive: Uses a bigger intrusion boundary. More specifically, the marker is filtered out, whenever there is any extraneous reflection detected when marker reflection cells are extracted for the image mode processing.


    Intrusion Band

    Default: 0.5(pixels)

    Sets the size of the intrusion boundary for the Normal intrusion detection mode. More specifically, the intrusion band value is added to the calculated radius of detected markers to establish a boundary, and any extraneous reflections intruding the boundary is considered as the intrusion.


    Advanced Options

    Reconstruction Bounds

    Ray Ranking

    Default: 4

    This setting enables the Ray Ranking, which calculates quality of each ray to potentially improve the reconstruction. Setting this to zero means that ray ranking is off, while 1 through 4 set the number of the evaluation iterations; 4 being 4 iterations. Setting this value to the max of 4 will slow down the reconstruction process but will produce better results.

    The Ray Ranking increases the stability of the reconstruction but at a heavy performance cost. The ray quality is analyzed by comparing convergence of rays that are contributing to the same marker. An average converging point is calculated, and each ray is ranked starting from the one closest to the converging point. Then, each ray is weighed differently in the Point Cloud reconstruction engine according to the assigned rankings.

    This setting is useful especially when there are multiple rays contributing to a marker reconstruction. If you're working with small to medium marker counts, enabling this will not have an evident improvement on performance. Also, when precise real-time performance is required, disable this setting especially for a setup with numerous cameras.


    Pixel Gutter (pixels)

    Default: 0 pixels

    Establishes a dead zone, in pixels, around the edge of the 2D camera image. Any 2D objects detected within this gutter will be discarded before calculating through the point cloud. In essence, it is a way of getting only the best data of the captured images, because markers seen at the edges of the camera sensor tend to have higher errors.

    This setting can be increased in small amounts in order to accommodate for cases where lens distortions are potentially causing problem with tracking. Another use of the setting for limiting the amount of data going to the reconstruction solver, which may help when you have a lot of markers and/or cameras. Be careful adjusting this setting as the trimmed data can't be reacquired in post-processing pipelines.


    Minimum Angle (degrees)

    Default: 5 degrees

    The minimum allowable angle – in degrees from the marker's point of view – between the rays to consider them valid for marker reconstruction. This separation also represents the minimum distance required between the cameras.

    For best results, cameras should be separated apart in order to capture unique views on the target volume. For example, if there are only two cameras, an ideal reconstruction would occur when the cameras are separated far enough so the rays converge with a 90 degree of an incident angle from the perspective of the reconstructed marker(s).

    If you are working with a small number of cameras that are separated evenly around the capture volume, set this value lower (default 5 degree). On the other hand, if you have a large setup with numerous cameras placed close together, you can set this value a bit higher to ignore contributing rays that are coming from the cameras that are too close together.

    Similar vantages obtained by the cameras in vicinity do not contribute unique positional data to the reconstruction, but they only increases the required amount of computation. Rays coming from very close cameras may increase the error in the reconstruction. Better reconstruction can only be achieved with a good, overall camera coverage (See Camera Placements).


    Reconstructed Markers Only

    Default: False

    This option determines whether the 3D reconstruction data is solely based on point cloud reconstruction. When this is False, the rigid body and skeleton asset definitions will also be used in in conjunction with 2D data and reconstructed 3D data to facilitate reconstruction of additional 3D marker locations.

    More specifically, for rigid body tracking, Motive will utilize untracked rays along with the rigid body asset definition to replace the missing markers in the 3D data. In order to compute these reconstructions, the rigid body must be using the Ray-Based tracking algorithm. For skeleton tracking, only the asset definitions are used to approximate virtual reconstruction at the location where the occluded marker was originally expected according to the corresponding skeleton asset.

    Using the asset definitions in obtaining the 3D data could be especially beneficial for accomplishing stable tracking of the assets in low camera count systems where all of the reconstructions may not always meet the minimum required tracked ray requirements.


    Rigid Body Markers Override

    Default: False


    When the Rigid Body Marker Override is set to True, Motive will replace observed 3D markers with the rigid body's solution for those markers.

    This is applicable for rigid bodies using Ray-Based tracking, and when the Reconstructed Markers Only setting is disabled.

    Trajectorization / Auto-labeler

    Auto-labeler section in the Reconstruction pane.

    After markers have been reconstructed in Motive, they must be labeled. Individual markers can be manually labeled, but the auto-labeler simplifies the process using the Assets. Rigid body and skeleton assets, created in Motive, saves their marker arrangement definitions and use them to auto-label corresponding marker sets within the Take. The auto-labeling, is a process of associating 3D marker reconstructions in multiple captured frames by assigning marker labels within the defined constraints. After the labeling process, each of the labeled markers provides respective 3D trajectories through out the Take.


    Prediction Radius (mm)

    Default: 200 mm

    To identify and label a marker from one frame to the next, a prediction radius must be set. If a marker location in the subsequent frame falls outside of the defined prediction radius, the marker will no longer be identified and become unlabeled. This is not the case when the marker is part of a tracked rigid body or skeleton. The prediction radius has no direct influence on reconstruction and is used only for marker labeling.

    For capturing relatively slow motions with tight marker clusters, limiting the prediction radius will help maintaining precise marker labels throughout the trajectory. Faster motions will have a bigger frame to frame displacement value and the prediction radius should be increased. When capturing in a low frame rate settings, set this value higher since there will be bigger displacements between frames.


    Maximum Marker Label Gap

    Default: 20

    This setting sets the maximum allowable gap size for assigning associated labels. If a marker is hidden, or occluded, longer than the defined number of frames, then the labeled trajectory will be truncated and the marker will become unlabled.


    Minimum Key Frames

    Default: 2

    Sets the required minimum number of frames without occlusion for a tracked markers to be reconstructed into 3D data. This feature will be applied only in the post-processing auto-labeling, and any trajectories with a length less than the minimum key frames will be discarded.


    Real Markers Only

    Default: False

    Track using only detected real markers without using virtual markers. When tracking skeletons and/or a rigid bodies, marker trajectory history is utilized to create a virtual marker to improve the trajectory.


    Create Missing Markers

    Default: False

    This option reconstitutes the use of Untracked Rays – rays that are not contributing to reconstruction of markers. Untracked Rays appear when the ray is contributing to a marker that is not qualified for reconstruction because the defined minimum ray requirement is not satisfied. With this setting enabled, the untracked rays can be utilized to aid solving assets – skeleton or rigid body -- that is susceptible to marker occlusions.

    The markers may be occluded from reconstruction when the minimum ray requirement is set too high. When capturing in a noisy environment and the minimum ray requirements need to be increased, this feature can be enabled to improve tracking of a skeleton and/or a rigid body. A good indicator that you may need this is if markers are missing from your data that are required by tracked assets - such as a flickering chest or finger marker.


    Auto-labeler (Trajectorization) Passes

    Default: 1

    The number of iterations for analyzing detected marker trajectories for maintaining constant marker labels. Increasing this setting can improve the marker auto-labeling, but more iterations will require more time and computation effort to complete the auto-labeling.


    Rigid Body Assisted Labeling


    Rigid Body Assisted Labeling of markers on the hand results in less broken trajectories

    Default: OFF

    Rigid Body Assisted Labeling can be used to optimize the labeling of markers within a region defined by a rigid body. The first step in using this feature is to create a rigid body from markers that are visible and rigidly connected. The example shown in the figure below demonstrates this for hand tracking. Five white markers are selected on the top of the wrist - which is rigidly defined. The black markers on the fingers are not rigidly defined in any fashion but are within the boundary of the Rigid Body Assisted Labeler. Labeling continuity is improved for the markers on the fingers which are given automatic labels.

    Tracking of organic or flexible objects - that do not have a tracking models like the face and hand, are good candidates for Rigid Body Assisted Labeling.


    Rigid Body Volume Radius

    Default: 300 mm

    The rigid body volume radius defines the region of space where the rigid body assisted labeling is applied. Increasing this radius will increase time needed for the auto-labeling so care should be made when setting this property.


    Maximum Assisted Labeling Gap

    Default: 30 frames

    The maximum gap frames property defines the maximum number of frames a marker can be hidden before it is truncated or unlabeled. Increase this value if larger gaps are to be anticipated. Increasing the assisted labeling gap will increase the processing time of reconstruction.


    Prediction Radius (mm)

    Default: 10 mm

    The predication radius defines the size of the bounding region used to label markers. When labeling a marker from one frame to the next, a bounding region, relative to the rigid body, is created around each labeled marker. The labeling continuity is restricted to the bounding region from frame to frame. Increasing this can allow markers to swap if there are occlusions in the data. Decreasing this restricts labeling from frame to frame but may lead to an increase in broken trajectories.


    Discard External Markers

    Default: False

    Discards markers outside of rigid body volume.


    Maximum Gap Frames

    Default: 30 frames

    The maximum gap frames property defines the maximum number of frames a marker can be hidden before it is truncated. Increase this value if larger gaps are to be anticipated. Increasing this setting will require longer processing time of reconstruction and auto-labeling.


    Discard External Markers

    Default: OFF

    Enabling this property will eliminate marker reconstructions outside of the region defined by the Rigid Body Volume.


    Dynamic Translational Constraint

    Default: None

    Prevents the rigid body from moving/rotation more than specified amount per frame.

    Max Translation (mm)

    Default: 100

    Distance for Dynamic Translation Constraint option.

    Max Rotation (deg)

    Default: 30

    Angle for the Dynamic Rotation Constraint option.

    Minimum Tracking Frames

    Default: 20

    Dynamic constraints are enabled when the rigid body is consecutively tracking more than this frame count.


    Marker Filter Diameters

    Default: False

    Markers less than this diameter will not be used for rigid body tracking.

    Minimum Diameter (mm)

    Default: 10 Diameter used for Marker Filter Diameter option.


    Back to Top