Self navigating drone using zed camera


#1

Hi
We are doing a project in which we need to take the data from ZED camera and feed to pixhawk for self- navigation
our structure is as follows
Flight Controller => Pixhawk-mini
Companion computer=>Nvidia Tx2
Camera=>zed camera.
The zed sdk is installed at Nvidia Tx2 and we are able to see streaming from zed sdk. We are trying to use flytos.
Looking at your forum

we did the following
1.Edited
EKF2_AID_MASK: vision position fusion
EKF2_HGT_MODE: vision

2.provided odom data to /flytos/mavros/vision_pose/pose

3.When we view followme app, we are not able to see streaming and when we try to do take off its giving message
sys_auto_start is not set to zero or sys_companion is not set to 921600

sys_companion is set to 921600.and sys_auto_start gets its value as soon as we select the drone frame

Can you please help if we are doing some mistake in the process.We are new to flytos so help will be widely appreciated


#2

Typo mistake
We are doing a project in which we need to take the data from ZED camera connected to tx2 and give this data to pixhawk for self- navigation


#3

Hi @snehashahi,

Sure we can help you with this.
Please send us the last few Run logs of FlytOS to help us debug the situation better. Check this link to know how to fetch the logs.

In the meantime, let me try to answer your queries inline:

FlytOS has an inbuilt camera driver which expects a camera device at /dev/video0. Do you see a /dev/video0 device being registered in Nvidia TX2 when you connect the Zed camera?

This is correct.

This is correct too. Please make sure you are sending the correct timestamp in the corresponding field and do correct frame transformation from Zed camera frame to NED frame.

Please make sure that FlytOS is indeed receiving data from PX4. Please connect to FlytConsole and confirm that the Connection status is indeed valid. Nevertheless, the run log should help us debug the problem better.


#4

hi @srv-admin
Thanks for the reply.Please find the attached logsflyt_runlogs.log (34.2 KB)
flyt_startup.log (40.4 KB)

I have one question about the following

This is correct too. Please make sure you are sending the correct timestamp in the corresponding field and do correct frame transformation from Zed camera frame to NED frame

We are sending the timestamp and x,y,z coordinates from zed odom. We do not know about the NED frame format in which the pixhawk is expecting data.In QGC widget , we could not see messages for vision pose estimate.Can you please refer us with any reference link ,document or sample code which we can refer and proceed on this.

Thanks


#5

Any update on the issue

Thanks and Regards
Sneha


#6

The runlog suggests that the connection to the autopilot is not correct.

The error messages:

e[33m[ WARN] [1542250692.499963589]: VER: unicast request timeout, retries left 1e[0m
e[31m[ERROR] [1542250696.500218863]: WP:m: request list denied, waiting for boot up services to completee[0m
e[33m[ WARN] [1542250696.753681233]: PR: request list timeout, retries left 2e[0m
e[33m[ WARN] [1542250697.004436767]: PR: request list timeout, retries left 1e[0m
e[33m[ WARN] [1542250697.255046444]: PR: request list timeout, retries left 0e[0m
e[33m[ WARN] [1542250697.506190141]: VER: unicast request timeout, retries left 0e[0m
e[31m[ERROR] [1542250701.500552382]: WP:m: request list denied, waiting for boot up services to completee[0m
e[31m[ERROR] [1542250706.501189840]: WP:m: request list denied, waiting for boot up services to completee[0m

suggests that the TX channel of your TX2 or RX channel of Pixhawk is not working correctly. Please make sure that the hardware connections are fine. Please verify the connectors are fine by using QGC or Mission Planner to connect to the autopilot via the connectors.

Are you using USB-UART module to connect TX2 with the autopilot?
Moreover, what are the values of SYS_AUTOSTART, MAV_SYS_ID and MAV_COMP_ID parameters?


#7

Hi @srv-admin,

We are using PX4 with the PixHawk-mini hardware which has only one TELEM (TELEM1).

We are configuring the same to 921600 baud using /etc/extras/ from SD card and can see in the startup log that:
Detected serial port: /dev/ttyTHS2 at baudrate 921600
MAVLink protocol: 1.0 detected.

SYS_AUTOSTART=4001

MAV_SYS_ID=1

MAV_COMP_ID=1

Regards,
Unnikrishnan

-Sneha’s teammate.


#8

flyt_startup.log (49.5 KB)
flyt_runlogs.log (34.7 KB)

These are the latest logs.
Changed
EKF2_AID_MASK: GPS
EKF2_HGT_MODE: GPS

And even now, when we attempt to take_off:
Error:
-- Nav: Takeoff command received for height:5m above current position
-- Nav: Wait for FlytOS to boot up completely. Retry after 30s. Rejecting Takeoff command

Should we be editing any more parameters to use TELEM1?
As discussed here: https://dev.px4.io/en/companion_computer/pixhawk_companion.html ,

  • we do-not see MAV_1_CONFIG, MAV_1_MODE, or MAV_X_RATE in the list of parameters over QGC.

#9

Hi @snehashahi,

You configurations seem to be correct. I would need to get ssh access to your companion computer nvidia TX2, to understand the problem better. For this you need to make sure your TX2 is connected to the internet.

I am available from 11.30 AM (+5.30 UTC) to 5 PM (+5.30 UTC). Please suggest a suitable time.


#10

Hi @srv-admin,

Thank you for the support.

We will share a zoom-link tonight (10 PM PST -> 11:30 AM IST).

And share the credentials for SSH access there if that sounds ok.

Best Regards,
Unnikrishnan


#11

Hi @srv-admin,

Please let me know how we could share the SSH credentials directly with you?


#12

Hi @unnikrishnankgs,

Sorry that i missed the call, as I missed your message too.
Lets setup the call tomorrow same time. I am sending a calendar invite as well.


#13

Hello guys,
Were you able to get this problem resolved?


#14

Hi @srv-admin,

Thank you so much for the support.
Yes, we were able to resolve the issues - had to re-install Jetpack and FlytOS again and then everything seemed alright.

Suspect apsync packages that were installed on the machine was interfering - but should mention they were all stopped.

Anyway we are now able to use FlytOS onboard API to pitch, and roll.
Working on the yaw with some issues - hope to resolve that too soon.

We can close this thread.

Thank you again!


#15