Skip to content

Handle scaling and truncation for 32-bit images #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aneesas wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Handle scaling and truncation for 32-bit images #53

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
954ceb8
Handle scaling and truncation for 32-bit images, which o3d won't appl…
aneesas Apr 2, 2026
0424e23
Use effective_scale in both point cloud creations
aneesas Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions roman/map/fastsam_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,27 +335,42 @@ def run(self, t, pose, img, depth_data=None):
depth_obj[mask==0] = 0
logger.debug(f"img_depth type {depth_data.dtype}, shape={depth_data.shape}")

# Open3D won't apply depth_scale argument to 4-byte-per-channel images
# TODO: need to check for anything other than float32 that would fall under this category?
is_float32 = depth_obj.dtype == np.float32

if is_float32:
# Scale manually before heuristic test
if self.depth_scale != 1.0:
depth_obj = depth_obj / self.depth_scale
effective_scale = 1.0
else:
# Open3D will handle the scaling in point cloud generation
effective_scale = self.depth_scale

# Extract point cloud without truncation to heuristically check if enough of the object
# is within the max depth
pcd_test = o3d.geometry.PointCloud.create_from_depth_image(
o3d.geometry.Image(np.ascontiguousarray(depth_obj).astype(np.dtype(depth_obj.dtype).type)),
self.depth_cam_intrinsics,
depth_scale=self.depth_scale,
# depth_trunc=self.max_depth,
depth_scale=effective_scale,
stride=self.pcd_stride,
project_valid_depth_only=True
)
ptcld_test = np.asarray(pcd_test.points)
pre_truncate_len = len(ptcld_test)
ptcld_test = ptcld_test[ptcld_test[:,2] < self.max_depth]
ptcld_test = ptcld_test[(ptcld_test[:,2] < self.max_depth) & (ptcld_test[:,2] > 0.0)]
# require some fraction of the points to be within the max depth
if len(ptcld_test) < self.within_depth_frac*pre_truncate_len:
continue

# Open3D won't apply depth_trunc argument to float32 images, so we manually truncate
if is_float32:
depth_obj[(depth_obj > self.max_depth) | (depth_obj <= 0.0)] = 0.0
pcd = o3d.geometry.PointCloud.create_from_depth_image(
o3d.geometry.Image(np.ascontiguousarray(depth_obj).astype(np.dtype(depth_obj.dtype).type)),
self.depth_cam_intrinsics,
depth_scale=self.depth_scale,
depth_scale=effective_scale,
depth_trunc=self.max_depth,
stride=self.pcd_stride,
project_valid_depth_only=True
Expand Down
Loading