Object Avoidance with PointCloud2 Map


Hi there FlytBase community, we’re building a drone and using a mmWave sensor as a radar to capture the environment as a point cloud, and use it to help the drone in object avoidance. It’s a 2D sensor, but we’ve mounted it on a swivelling servo to capture a 3D point cloud. We can publish this on ROS and visualise it. The points are captured in Cartesian coordinates. We want to use this

However, we are a bit stuck as to how we can proceed. Ideally, all we want to do is avoid collisions with objects based on our point cloud scan, and so we expect to do some path-planning by generating some velocity vectors, and feeding it into our PX4 flight control board. After some reading, I believe we will need to first figure out a way to do a ‘tf-transform’ of our static map as the drone moves around, and then figure out how to output values (velocity vectors) to the drone. The part where we are stuck, is what kind of navigation algorithms can we use in the first place? Do we require SLAM in 3D for this? We don’t really have a direction here.

We are using a PX4. Do you know how one might be able to achieve this with FlytOS? Are there any pre-built APIs or functions we can use that can help us? Thank you! I have attached a photo of our point cloud, visualised on ROS’ RVIZ.

Hoping if anyone has had any experience in this and could offer or lend some insight or advice.



Hello @SamLow

3D point clouds in ROS can be used to generate a complete 3D map of the environment.

If you are trying to do simple obstacle detection and simple response (such as stopping drone 3 meters away from obstacle) then there are very simple ways to do it.

However, for path planning through cluttered environments, there is a lot of active research going on. Typically, you need to build the map of the whole surrounding and then use this map in path planning to navigate. There are multiple projects/packages in ROS for this.
You can take a look at RTABmap. This project provides SLAM solution for your use case. You may need to do some work on converting your makeshift 3D pointcloud to one supported by RTABmap or any other projects. Once you are able to go through all of this you can use FlytOS ROS APIs (Position or Velocity) for drone navigation.

If you want to start with simple approach, convert the pointcloud to opencv image, then apply some density clustering algorithms (Scipy) on all channels to detect obstacles. Once you are able to do that, use FlytOS navigation APIs to send appropriate commands (e.g. Position hold) to the drone.

I hope that gives you some direction. All the best for your project.


Thank you very much for these pointers Dhirajd :slight_smile: