Darknet / Yolo Real-time Object Detection

Question

DJ and Team,

Have you seen the TensorFlow / Darknet / YOLO Real-time Object Detection videos? A friend of mine has implemented it on his PC and OMG, it is the fastest and most accurate object identification software that I have ever seen. It is light years beyond Open CV. And the best part is that like Open CV, it is also open source. It is a neural network program which it runs on Windows (and Linux).

TED Talk: https://www.youtube.com/watch?v=Cgxsv1riJhI

Have you ever thought of adding this to ARC?

Thomas Messerschmidt

Object detection/recognition

Sebediah · Answer

This looks absolutely amazing. Thanks for sharing Thomas!

DJ Sures · Answer

Here you go - this is the robot program episode on training and recognizing objects in real-time. Find more activities in the learn section of this website. Locate the top menu and press the LEARN button, then select your ezrobot product. have fun! Full link: https://synthiam.com/Tutorials/Lesson/106

Nink · Answer

Let the YOLO vs SSD vs RCNN games begin. If we are starting a requirements list for Ez-B 5.0 a GPU for Edge AI. AMD or Nvidia on board would be cool.

TMesserschmidt · Answer

I finally got my YOLO2/TensorFlow implementation running! It is amazing in how accurate it is, even in bad light and weird angles. I think they call that "deep learning". YOLO ("You Only Look Once") is a faster way of doing vision processing. It uses TensorFlow They say you can get it to process 20-40 frames per second from live video feed with a GPU. I don't get so many with my old laptop, but it works well enough for my purposes. Wiki says, "TensorFlow is (Google's) open-source software library... It is...used for machine learning applications such as neural networks. It is used for both research and production at Google." It runs kind of slow on my old windows 7 laptop. It really needs a graphics processor unit board (GPU) to do its best work. I'll have to put "a new computer with a fast GPU" on my shopping list. YOLO2 is a real pain to install. A lot of the online tutorials/instructions are out of date or assume you know most of everything that they do. Still, once you get the applications installed (python, Atom, tensorflow, OpenCV), the rest is relatively easy. There were a few "gotchas", but nothing that 10 or 20 hours of combing the internet couldn't solve! LOL! I will be putting up a youtube video in a few days (weeks?) that shows how I did it. I will probably write an ARC skill plugin as well, but I really don't know where to start with the plugin.

Sebediah · Answer

Glad to hear you've got it working! I'm very interested in YOLO and what it could achieve in our robots! Really looking forward to that video of yours, would be great to see how it was done and maybe some samples of YOLO in action.

Mickey666Maus · Answer

@thomasfromla sweet...looking forward to see what you accomplished! Seems like there are a few version of YOLO out there which do have a limited training set...which gives better results if computing power is limited! Good job!

TMesserschmidt · Answer

Thanks! When I make that video, please let me know if I skip a step or get something wrong. It will probably be this weekend. Thanks, Thomas

ptp · Answer

@thomasfromla: Quote: It uses TensorFlow They say you can get it to process 20-40 frames per second from live video feed with a GPU.  Yolo is not based neither is related to the Tensorflow. YOLO: (You only look once) is a system for detecting objects, works on top of the Darknet framework. https://github.com/pjreddie/darknet/wiki/YOLO:-Real-Time-Object-Detection Darknet: Darknet is an open source neural network framework written in C and CUDA. It is fast, easy to install, and supports CPU and GPU computation. https://pjreddie.com/darknet/ Tensorflow: https://en.wikipedia.org/wiki/TensorFlow You can see object detection results using Tensorflow and Yolo: https://synthiam.com/Community/Questions/10279 I've compiled both on Windows: Tensorflow took me 4 hours and a lot of memory and disk. Yolo is straightforward although i get some random crashes.

TMesserschmidt · Answer

Well thank you for that detailed explanation. It is certainly an interesting mix of software. Someone should come up with a name for this. A simple, descriptive name would be a good thing. Tensorflow/Darknet/YOLO is a bit of a mouthful to say. Perhaps "TiDY" or "DitTY" --Thomas

TMesserschmidt · Answer

SUCCESS! I finally have DTY (Darknet/Tensorflow/YOLO2) working with ARC. DTY classifies/identifies an object. I added code to then output what it sees to a text file. Then I read the text into ARC, and finally parse it and assigns each atribute to a variable. The attributes are - OBJECT - CONFIDENCE - TOP - BOTTOM - LEFT - RIGHT - HCENTER - VCENTER - TIMESTAMP Quote:If at first you don't succeed, code, code again!

fxrtst · Answer

@thomas! Great job! I look forward to details and videos !

JustinRatliff · Answer

Congratulations @thomasfromla ! That is no small feat to accomplish! It sounds like you did it a very efficient way as well (write text file/read in - I love that method). I hope you'll get to share more on how your did it, I'd like to follow in your foot steps and experiment. With TensorFlow I assume you used the 'CPU' install instead of the GPU install? I was pricing a multi unit GPU system the other day...PHEEWWW! I was not prepared for those prices!

Mickey666Maus · Answer

@thomasfromla Man this is great...really looking forward to see how you accomplished!:D

ptp · Answer

@thomasfromla: Good job. Compiling Yolo is quick, regarding Tensorflow: How long it took to compile ? Did you enabled AVX2 extensions ? (write text/read in) pointed by Justin follows the KISS principle ! I'm working on a different integration, not so simple .. so no results yet.

TMesserschmidt · Answer

@ptp Thanks. I'm running it from Atom/Python with a few changes. Best of luck on your implementation!

TMesserschmidt · Answer

PS. I've gotten my robot to physically point at requested objects. Me: do you see a dog? Robot: yes. It's on my left. (Points to the dog.)

Perry_S · Answer

This sounds great, is there any way to post up a vid of it operation? There is an immovable implementation over on the MRL site the video looks awesome.

Mickey666Maus · Answer

@thomasfromla this is great progress! We could also use the information perceived to drive a fully IK rigged arm within Unity, which would be my next goal! Do you think it is possible to send the Values from YOLO directly to the Http Server, this would eliminated to read and write a .txt file and speed up things significantly!

fxrtst · Answer

Now this is getting interesting. I look forward to a video!

TMesserschmidt · Answer

OK. I hear you. You want a video (or two) Here is my to do list: 1. Put a demo video up this weekend showing the software working. 2. Put up a video showing the robot running via ARC. 3. Put up a Tensorflow / darkflow / YOLO instructional video with links to all the needed software. 4. Put up a Atom installation video 5. Put up a OpenCV installation video 6. Correct any errors 7. Answer all questions 8. Improve as needed Depending on your internet connection, setting this up should take less than a half of an hour to install and get working. You will need a computer with good "horsepower" under the hood--a strong 4 core processor and at least 8 GB of RAM. And a GPU will get you up to 10-30 images processed per second. I don't have a GPU, so the best I get is about 1 image every 2 or 3 seconds.

TMesserschmidt · Answer

Hi, I put up a demo video with tensorflow/YOLO/Darknet/Darkflow on YouTube:

Mickey666Maus · Answer

@thomasfromla this is a great demo! The end is hilarious! Looking forward to see the setup and it would be great if you could share the ARC Project! Cant wait to try it myself! Btw, I heard there are a few faster version of YOLO out there with a reduced amount of trained objects, can you confirm this?

TMesserschmidt · Answer

There is a mini version, but I could never get it to work right. It is currently unsupported.

TMesserschmidt · Answer

Next video: installation of the free Atom editor. You can run the software without Atom, but it is a whole lot harder. You see, Atom is more than just an editor. It allows you to compile and run code.

Mickey666Maus · Answer

I love Atom...its great!

fxrtst · Answer

Fantastic progress Thomas! Very Impressive.

DJ Sures · Answer

Great video Thomas!

TMesserschmidt · Answer

Well, I started it two years ago, but ptp finished it this year! Thanks ptp for making it work for everyone!

TMesserschmidt

Darknet / Yolo Real-Time Object Detection

Upgrade to ARC Pro