IoT Sharing: Deep learning - Computer vision

Hot

Showing posts with label Deep learning - Computer vision. Show all posts
Showing posts with label Deep learning - Computer vision. Show all posts

Tuesday, March 31, 2020

Demo 50: Bring Tensorflow Lite to ESP32 Arduino - person detection application using deep learning with ESP32 CAM

5:12 AM 1
1. Introduction
Deep learning is hot. It is hotter when you can run it on ESP32 a hot MCU for IoT. I made a demo Demo 47: Deep learning - Computer vision with ESP32 and tensorflow.js It is an interesting demo but it not really run on ESP32. Today I will make another demo that is bring Tensorflow Lite to ESP32 Arduino through person detection application using deep learning with ESP32 CAM.
Figure: Bring Tensorflow Lite to ESP32 Arduino
2. Hardware
I use the ESP32 CAM module
Figure: ESP32 CAM with OV2640 cam
3. Software
I prepared the resources and the code for you.
Steps to install:
- Install libraries Jpeg decoder and Tensorflow lite.
Jpeg decoder: https://github.com/nhatuan84/tensorflow-lite-esp32-person-detection/blob/master/resources/JPEGDecoder-master.zip
Tensorflow lite: https://github.com/nhatuan84/tensorflow-lite-esp32-person-detection/blob/master/resources/tensorflow_lite.zip
- Install zip libraries, choose Sketch > Include Library > Add .Zip Library
- Download Arduino code and open it with Arduino IDE:
Arduino code: https://github.com/nhatuan84/tensorflow-lite-esp32-person-detection/tree/master/Arduino_code/person_detect
- After flashed the code, open the Terminal to see the IP address of the board.
- Open Web browser and type the IP address above and enjoy the result
4. Result
It is not really smooth and slow.


Read More

Monday, July 29, 2019

Demo 47: Deep learning - Computer vision with ESP32 and tensorflow.js

3:58 AM 39
1. Introduction
- Deep learning is a hot topic and esp32 is a hot IoT MCU. Recently many applications related to computer vision are deployed on ESP32 (face detection, face recognition, ...). In this post I will show you a new approach to deploy Deep learning - Computer vision applications on ESP32 such as object classification (SqueezeNet), object detection and recognition (YOLOv3). After reading this post I am sure you can deploy hot network such as YOLOv3 on ESP32.
- My approach is using TensorFlow.js is a library for developing and training ML models in JavaScript, and deploying in browser.
- In this post, I will create a simple Deep learning - Computer vision application that is object classification using SqueezeNet. The esp32 will act as a webserver and when the client connect to it, a slideshow of objects will start and the objects will be classified using SqueezeNet.
You can do similar steps for YOLOv3, but instead of reading pictures from sdcard, you will use esp32-camera module and pass each camera frame to YOLOv3 model created by tensorflow.js.
 Figure: Deep learning - Computer vision with ESP32 and tensorflow.js
2. Hardware
You need a micro sdcard module as in Demo 7: How to use Arduino ESP32 to store data to microsdcard (Software SPI and Hardware SPI)
In this demo, I used Hardware SPI so please connect pins as below:
MICROSD CS    -      ESP32 IO5
MICROSD SCK   -     ESP32 IO18
MICROSD MOSI  -    ESP32 IO23
MICROSD MISO   -   ESP32 IO19
MICROSD Vcc   -      ESP32 3.3V
MICROSD GND   -    ESP32 GND
3. Software
- In order to make this demo, you have to review some demos:
Demo 12: How to turn the Arduino ESP32 into a Web Server
Demo 7: How to use Arduino ESP32 to store data to microsdcard (Software SPI and Hardware SPI)
- Knowledge of Jquery and Javascript.
- Material for deep learning part make by me: https://github.com/nhatuan84/tensorflowjs-squeezenet (or you can use the outputs that I generated)
- Knowledge of Deep learning. If you don't know, just follow me. I had another blog about Machine Leaning. It is here.
- I had to modify the webserver library in Demo 12: How to turn the Arduino ESP32 into a Web Server so that It can be used for this demo.
- Here are the steps:
  + Download all the resources here and unzip it.
  + Reinstall the ESP32WebServer.zip (in resources) for Arduino (you may uninstall old ESP32WebServer library).
  + Copy files: group1-shard1of2.bin, group1-shard2of2.bin, model.json, index.html, 1.jpg, 2.jpg, 3.jpg (in resources) to sdcard.
  + Create an Arduino project with code:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
#include <WiFiClient.h>
#include <ESP32WebServer.h>
#include <WiFi.h>
#include <ESPmDNS.h>
#include "FS.h"
#include <SD.h>
#include <SPI.h>

const char* ssid = "ssid";
const char* password = "pass";

ESP32WebServer server(80);
File root;

void handleRoot() {
  root = SD.open("/index.html");
  if (root) {  
    /* respond the content of file to client by calling streamFile()*/
    size_t sent = server.streamFile(root, "text/html");
    /* close the file */
    root.close();
  } else {
    Serial.println("error opening index");
  }
}

bool loadFromSDCARD(String path){
  path.toLowerCase();
  Serial.println(path);
  String dataType = "text/plain";
  if(path.endsWith("/")) path += "/index.html";
  if(path.endsWith(".src")) path = path.substring(0, path.lastIndexOf("."));
  else if(path.endsWith(".jpg")) dataType = "image/jpeg";
  else if(path.endsWith(".txt")) dataType = "text/plain";
  else if(path.endsWith(".zip")) dataType = "application/zip";  
  if(path == "/favicon.ico")
    return false;
  
  root = SD.open((String("/") + path).c_str());
  if (!root){
    Serial.println("failed to open file");
    return false;
  }

  if (server.streamFile(root, dataType) != root.size()) {
    Serial.println("Sent less data than expected!");
  }

  root.close();
  return true;
}

void handleNotFound(){
  if(loadFromSDCARD(server.uri())) return;
  String message = "SDCARD Not Detected\n\n";
  message += "URI: ";
  message += server.uri();
  message += "\nMethod: ";
  message += (server.method() == HTTP_GET)?"GET":"POST";
  message += "\nArguments: ";
  message += server.args();
  message += "\n";
  for (uint8_t i=0; i<server.args(); i++){
    message += " NAME:"+server.argName(i) + "\n VALUE:" + server.arg(i) + "\n";
  }
  server.send(404, "text/plain", message);
  Serial.println(message);
}

void setup(void){
  Serial.begin(115200);
  WiFi.begin(ssid, password);
  Serial.println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
  
  //use IP or iotsharing.local to access webserver
  if (MDNS.begin("iotsharing")) {
    Serial.println("MDNS responder started");
  }
  if (!SD.begin()) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
  //handle uri  
  server.on("/", handleRoot);
  server.onNotFound(handleNotFound);

  server.begin();
  Serial.println("HTTP server started");
}

void loop(void){
  server.handleClient();
}
  + Open web browser and type the IP address from Terminal, you will see:

Figure: esp32-tensorflowjs-squeezenet prediction

Read More
Thường mất vài phút để quảng cáo xuất hiện trên trang nhưng thỉnh thoảng, việc này có thể mất đến 1 giờ. Hãy xem hướng dẫn triển khai mã của chúng tôi để biết thêm chi tiết. Ðã xong