Comparing two cameras

[clolsonus]

Recently I had a chance to survey an area with two entirely different systems. The first is a DJI Phantom 4 Pro v2 (20 megapixel camera) and the second is a fixed wing system we assembled in-house. (I work for the U of MN AEM UAV Lab.) Our in-house system features a Sony A6000 mirrorless camera with a 30mm prime lens. We have a project hunting for invasive plant species so detail, resolution, and image quality are important for our use-case. This was a quick test, not super scientific, but I thought it was interesting to compare imagery side by side. We flew both systems at the same altitude (200' AGL) and covered the same area. I wrote up a short blog entry so feel free to jump over here for all the gory details.
External link: DJI Phantom 4 Pro camera vs. Sony A6000 – Gallinazo
I'll try to reproduce a bit of it here. You all know what a phantom looks like, so here is our in-house system based on an X-UAV Talon, flying our in-house linux-based C++/python hybrid autopilot (not a pixhawk.)

Here are two example images from my other post. The first is the DJI, the second is the Sony A6000. I don't know how well they reproduce here on this site, so if you can't see clearly, trust me that there is way more subtle variation and rich color in the Sony imagery. You can start to see the pine needles pretty clearly in the Sony image where as with the DJI camera the needles are mostly blurred together. It's not entirely apples to apples ... for a given altitude, the A6000 gets about 2x the pixel resolution in each dimension (so 4x pixel per area) compared to the DJI due to lens and sensor size. In many cases you could fly the DJI lower to pick up the same pixel resolution, but you may still notice differences in sensor quality and image compression artifacts as you begin to zoom in. So here are two sample comparison pictures, first the DJI, then the Sony. (And I know I've just shown that the more expensive camera gets better picture, duhh, but it was interesting to crunch the numbers and see the result.)

Thanks for your time!

Curt.

 

[Outta Control]

What were the relative flight speeds of both aircrafts and agl?

 

[clolsonus]

What were the relative flight speeds of both aircrafts and agl?

(I gave more of these details in the external link)

Both flights were at 200' AGL, Phantom 13 mph, Talon/Sony 30 mph.

Due to the nature of the terrain and trees relative to our launch point, I would not have felt comfortable flying either the quad or the fixed wing any lower than 200' AGL.
The Phantom was flying with the Drone Deploy flight planning app, and that does 13 mph by default. The Talon/Sony flies comfortably about 30 mph (but can do faster or slower if there is a reason ... as can the Phantom.)

Here is the link to the drone deploy stitch from the Phantom images if you want to see some context of the area. We were out there flying with DNR permission surveying areas where oriental bittersweet was spotted. Oriental bittersweet is an invasive vine that will strangle whole areas of trees. We weren't able to spot any clear evidence in the Phantom images:

Externtal link to Drone Deploy Map:

DroneDeploy

Oh, and I used the tail end of one of my DJI batteries to capture a few minute video tour of the survey site:

 

[monkmartinez]

How do you trigger the a6000 to shoot? Can you do it via software in the same way you would with a Phantom?

 

[clolsonus]

How do you trigger the a6000 to shoot? Can you do it via software in the same way you would with a Phantom?

That is a very interesting question. Is it ok if I give the long answer? (But here's the spoiler alert: picoswitch + hacked remote trigger cable.)

Originally we purchased a fairly expensive triggering system from seagull uav. This worked as advertised except I couldn't trigger the camera faster than once every 2-4 seconds. I worked with customer support and they had the same exact camera (sony a6000) on site and said they had no problem triggering at rates faster than once per second. Unfortunately the list of possible obvious things I could have been missing was very long, not to mention the number of camera settings to tweak with. So eventually I just gave up on it and built my own system.

I purchased a picoswitch (which is a relay that plugs into a servo output on your flight controller.) I bought a $10 remote [wired] trigger off amazon and cut off the connector end and wired it to my picoswitch. Then I could plug that into a my receiver and trigger the shutter with a switch on my transmitter ... just to prove to myself the system physically worked correctly.

We are flying our own in-house flight controller (which is a hybrid of python and C++) so I wrote up a quick mission module that knows the fov of the camera as well as the altitude of the airplane. It will trigger the camera automatically to achieve the programmed amount of end-lap (ex: 70%) Then it' is up to our route planner (more on-board python) to generate a route with the correct amount of side-lap. Side note: our ground station software sends up the outline of the survey area and the airplane figures the optimal route considering wind, etc.

Even with all this I was some how missing every other picture. I fiddled around one day and discovered that if I hold the trigger down for 0.3 seconds, the picture is reliably snapped upon trigger *release*. (Probably another thing that is obvious to everyone except myself.) So I modified my python script and the geotagging/logging parts to reflect that. In the end I can sustain 0.5 second trigger rates with very precise timing/geotagging.

Our in-house stitching/fitting software uses a combination of camera pose (location/attitude) plus SRTM terrain map to make an initial guess at the location of all the found/matched features. It then hands this over to an optimizer to minimize the mean reprojection error (i.e. shuffle everything around for a perfect fit.) I am seeing that the initial mean reprojection error (which is a reflection of the accuracy of the initial placement) is on par with images geotagged from our DJI system, possibly a tiny bit more accurate. (But I've only run a couple data sets from both systems through our map stitcher/fitter system.)

It took us a while to work through all the kinks and figure all this out (This is why you should always buy commercial and not think you are going to DIY something yourself for cheaper)

So that's the longer answer. Our in-house DIY system is more complex and more difficult to operate compared to a Phantom (of course!) and probably cost a lot more if you add up all the hours of our time, but it has been consistently giving us better quality data for our use-case (finding needles in the haystack ... more specifically invasive plants.)

 

[Florida Drone Supply]

Its tough to tell if the focus is off on the P4P or if its just the GSD. To get 1cm per pixel on a P4P you have to fly around 115 feet. Being almost double that you will definitely lose some detail. The Sony with a 30mm would have compared to 45mm full frame (P4P is 24 by comparison). At 200 feet with the Sony you would have had right about 1cm per pixel - maybe just a touch more. But if the Phantom was in autofocus you could have had a little focus hunting going on before it snapped the photo. At the speed you were flying it definitely might have struggled - especially when the center point was right over a patch of snow without much detail to lock into.

Sounds like you have some great testing going on. Good luck with it all.

As a side note, I hope you are enjoying the U of M. I am originally from Minneapolis and heard you had a brutal winter up there this year.

Michael

 

[clolsonus]

Its tough to tell if the focus is off on the P4P or if its just the GSD. To get 1cm per pixel on a P4P you have to fly around 115 feet. Being almost double that you will definitely lose some detail. The Sony with a 30mm would have compared to 45mm full frame (P4P is 24 by comparison). At 200 feet with the Sony you would have had right about 1cm per pixel - maybe just a touch more. But if the Phantom was in autofocus you could have had a little focus hunting going on before it snapped the photo. At the speed you were flying it definitely might have struggled - especially when the center point was right over a patch of snow without much detail to lock into.

Sounds like you have some great testing going on. Good luck with it all.

As a side note, I hope you are enjoying the U of M. I am originally from Minneapolis and heard you had a brutal winter up there this year.

Hi Michael,

Thanks for the comments. It's nearly impossible to do a completely fair comparison of two totally different systems, but I thought I'd do my best and share what I learned. I'm still figuring out the DJI (have always done fixed wing flying previously) so I'm sure there are a few tricks and tweaks to improve the images a bit more. There was some wind that day so along with everything else going on, the subject matter is also moving a bit.

Does the phantom have a manual focus mode? I do fly the sony camera in manual focus mode. I have been flying some missions with the Drone Deploy app (we have an educational license for a few more months) and also with the Sentera field agent app. We also have a Sentera 5 band multispectral camera on board, which wasn't activated on these flights due to a power cable issue that is now fixed as of Monday.

The particular area where I was flying had quite a bit of terrain and I wasn't able to launch from the high point of the area, so I set the altitude to 200' above my launch point which was about as low as I felt comfortable.

Winters up here still usually have a few brutal stretches, nothing much has changed with that. We try to fly year round as much as possible, but there was a stretch from mid-January through Feb. where we were totally shut down due to weather. I also had my first icing experience earlier in January. In retrospect, light drizzle plus near freezing temps should have been a tip off to me. After the plane completed an hour flight, it was crazy nose heavy and I had to fly the approach with 1/2 to 3/4 throttle. Landed it safe, but when I went to pick it up, the front 1/3 of the wing was sheeted in ice. Crazy! Thankfully the pitot tube stayed clear and the autonomous survey route was flown just fine, otherwise we might have had to do some hiking.

Next up on my todo list is experimenting with the 5-band multispectral sentera camera ... we are hoping to be able to tease out some vegetation differences, possibly identify some invasive weeds ... we'll see. The grass has made a couple fleeting appearances up here already this spring between snow storms.

 

[Florida Drone Supply]

Hi Michael,

Thanks for the comments. It's nearly impossible to do a completely fair comparison of two totally different systems, but I thought I'd do my best and share what I learned. I'm still figuring out the DJI (have always done fixed wing flying previously) so I'm sure there are a few tricks and tweaks to improve the images a bit more. There was some wind that day so along with everything else going on, the subject matter is also moving a bit.

Does the phantom have a manual focus mode? I do fly the sony camera in manual focus mode. I have been flying some missions with the Drone Deploy app (we have an educational license for a few more months) and also with the Sentera field agent app. We also have a Sentera 5 band multispectral camera on board, which wasn't activated on these flights due to a power cable issue that is now fixed as of Monday.

The particular area where I was flying had quite a bit of terrain and I wasn't able to launch from the high point of the area, so I set the altitude to 200' above my launch point which was about as low as I felt comfortable.

Winters up here still usually have a few brutal stretches, nothing much has changed with that. We try to fly year round as much as possible, but there was a stretch from mid-January through Feb. where we were totally shut down due to weather. I also had my first icing experience earlier in January. In retrospect, light drizzle plus near freezing temps should have been a tip off to me. After the plane completed an hour flight, it was crazy nose heavy and I had to fly the approach with 1/2 to 3/4 throttle. Landed it safe, but when I went to pick it up, the front 1/3 of the wing was sheeted in ice. Crazy! Thankfully the pitot tube stayed clear and the autonomous survey route was flown just fine, otherwise we might have had to do some hiking.

Next up on my todo list is experimenting with the 5-band multispectral sentera camera ... we are hoping to be able to tease out some vegetation differences, possibly identify some invasive weeds ... we'll see. The grass has made a couple fleeting appearances up here already this spring between snow storms.

Thanks for the reply and yes, the Phantom can be put in manual focus which might have helped a little. Its a button in the top right of the app. You can get to your altitude, tap for auto focus to lock in what you want and then switch to manual.

Interesting to hear about the Sentera you also have. We use an AGX710 on the Matrice series for a custom golf course solution we developed. Its basically modified precision agriculture - but with no commercial product meeting our needs we designed an fairly eloborate piece of software to work around the Sentera (or any other multi band camera). We would love to follow your project and there is someone from Sentera here on the forum as well. @Peter Gagliardi is his name. Next time we are back up in Minneapolis we are going to try to swing by Sentera's operation.

If you need any pointers on the DJI stuff just give us a ring. We are happy to help you out any way that we can.

Michael

 

[clolsonus]

Next time we are back up in Minneapolis we are going to try to swing by Sentera's operation. If you need any pointers on the DJI stuff just give us a ring. We are happy to help you out any way that we can.

Thanks! Sentera's operation is really cool, I had a chance to get a detailed tour a few weeks ago. They have really grown since the last time I was there. One of the students out of our UAV lab is working for them this summer and another of our alums from a year or two ago is working for them full time now. I know trips get super busy, but next time you come through Minneapolis, if you have an extra hour or two, you are welcome to swing by the University and I can show you our lab and crazy projects.

 

[Florida Drone Supply]

Thanks! Sentera's operation is really cool, I had a chance to get a detailed tour a few weeks ago. They have really grown since the last time I was there. One of the students out of our UAV lab is working for them this summer and another of our alums from a year or two ago is working for them full time now. I know trips get super busy, but next time you come through Minneapolis, if you have an extra hour or two, you are welcome to swing by the University and I can show you our lab and crazy projects.

I'll definitely take you up on that. We will be up in the next few months sometime I think. We still have family around the lakes in SW Minneapolis so we are very close. Thanks!