1. Introduction
- Deep learning is a hot topic and esp32 is a hot IoT MCU. Recently many applications related to computer vision are deployed on ESP32 (face detection, face recognition, ...). In this post I will show you a new approach to deploy Deep learning - Computer vision applications on ESP32 such as object classification (SqueezeNet), object detection and recognition (YOLOv3). After reading this post I am sure you can deploy hot network such as YOLOv3 on ESP32.
- My approach is using TensorFlow.js is a library for developing and training ML models in JavaScript, and deploying in browser.
- In this post, I will create a simple Deep learning - Computer vision application that is object classification using SqueezeNet. The esp32 will act as a webserver and when the client connect to it, a slideshow of objects will start and the objects will be classified using SqueezeNet.
You can do similar steps for YOLOv3, but instead of reading pictures from sdcard, you will use esp32-camera module and pass each camera frame to YOLOv3 model created by tensorflow.js.
You can do similar steps for YOLOv3, but instead of reading pictures from sdcard, you will use esp32-camera module and pass each camera frame to YOLOv3 model created by tensorflow.js.
Figure: Deep learning - Computer vision with ESP32 and tensorflow.js
2. HardwareYou need a micro sdcard module as in Demo 7: How to use Arduino ESP32 to store data to microsdcard (Software SPI and Hardware SPI)
In this demo, I used Hardware SPI so please connect pins as below:
MICROSD CS - ESP32 IO5
MICROSD SCK - ESP32 IO18
MICROSD MOSI - ESP32 IO23
MICROSD MISO - ESP32 IO19
MICROSD Vcc - ESP32 3.3V
MICROSD GND - ESP32 GND
3. Software
- In order to make this demo, you have to review some demos:Demo 12: How to turn the Arduino ESP32 into a Web Server
Demo 7: How to use Arduino ESP32 to store data to microsdcard (Software SPI and Hardware SPI)
- Knowledge of Jquery and Javascript.
- Material for deep learning part make by me: https://github.com/nhatuan84/tensorflowjs-squeezenet (or you can use the outputs that I generated)
- Knowledge of Deep learning. If you don't know, just follow me. I had another blog about Machine Leaning. It is here.
- I had to modify the webserver library in Demo 12: How to turn the Arduino ESP32 into a Web Server so that It can be used for this demo.
- Here are the steps:
+ Download all the resources here and unzip it.
+ Reinstall the ESP32WebServer.zip (in resources) for Arduino (you may uninstall old ESP32WebServer library).
+ Copy files: group1-shard1of2.bin, group1-shard2of2.bin, model.json, index.html, 1.jpg, 2.jpg, 3.jpg (in resources) to sdcard.
+ Create an Arduino project with code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | #include <WiFiClient.h> #include <ESP32WebServer.h> #include <WiFi.h> #include <ESPmDNS.h> #include "FS.h" #include <SD.h> #include <SPI.h> const char* ssid = "ssid"; const char* password = "pass"; ESP32WebServer server(80); File root; void handleRoot() { root = SD.open("/index.html"); if (root) { /* respond the content of file to client by calling streamFile()*/ size_t sent = server.streamFile(root, "text/html"); /* close the file */ root.close(); } else { Serial.println("error opening index"); } } bool loadFromSDCARD(String path){ path.toLowerCase(); Serial.println(path); String dataType = "text/plain"; if(path.endsWith("/")) path += "/index.html"; if(path.endsWith(".src")) path = path.substring(0, path.lastIndexOf(".")); else if(path.endsWith(".jpg")) dataType = "image/jpeg"; else if(path.endsWith(".txt")) dataType = "text/plain"; else if(path.endsWith(".zip")) dataType = "application/zip"; if(path == "/favicon.ico") return false; root = SD.open((String("/") + path).c_str()); if (!root){ Serial.println("failed to open file"); return false; } if (server.streamFile(root, dataType) != root.size()) { Serial.println("Sent less data than expected!"); } root.close(); return true; } void handleNotFound(){ if(loadFromSDCARD(server.uri())) return; String message = "SDCARD Not Detected\n\n"; message += "URI: "; message += server.uri(); message += "\nMethod: "; message += (server.method() == HTTP_GET)?"GET":"POST"; message += "\nArguments: "; message += server.args(); message += "\n"; for (uint8_t i=0; i<server.args(); i++){ message += " NAME:"+server.argName(i) + "\n VALUE:" + server.arg(i) + "\n"; } server.send(404, "text/plain", message); Serial.println(message); } void setup(void){ Serial.begin(115200); WiFi.begin(ssid, password); Serial.println(""); // Wait for connection while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.print("Connected to "); Serial.println(ssid); Serial.print("IP address: "); Serial.println(WiFi.localIP()); //use IP or iotsharing.local to access webserver if (MDNS.begin("iotsharing")) { Serial.println("MDNS responder started"); } if (!SD.begin()) { Serial.println("initialization failed!"); return; } Serial.println("initialization done."); //handle uri server.on("/", handleRoot); server.onNotFound(handleNotFound); server.begin(); Serial.println("HTTP server started"); } void loop(void){ server.handleClient(); } |
Figure: esp32-tensorflowjs-squeezenet prediction
63 Comments
data science course in pune
ExcelR data analytics course in Pune | business analytics course | data scientist course in Pune
data scientist course in mumbai
Data science course in mumbai
ExcelR Analytics Course in Bangalore
ExcelR data science course in mumbai
Deep Learning Final Year Projects for CSE
JavaScript Training in Chennai
Project Centers in Chennai
JavaScript Training in Chennai
Please check this ExcelR Courses in Business Analytics
Artificial Inteligence course in Mysuru
AI Course in Mysuru
Artificial Inteligence course in Mysuru
AI Course in Mysuru
Data Science Training Bangalore
360digitmg data science course in guwahati
Data Science In Banglore With Placements
Data Science Course In Bangalore
Data Science Training In Bangalore
Best Data Science Courses In Bangalore
Data Science Institute In Bangalore
Thank you..
initiatives in the same niche. Blog gave us useful information to work. You
have done an amazing job!
artificial intelligence training in Bangalore
AWS Course Training in Hyderabad
Data Science Course Training in Hyderabad
Course in Data Analytics
Data Science Training In Hyderabad
Best Institute For Data Science In Hyderabad
data scientist training
I enjoyed reading your post, very nice share.data science training"
Thanks for the useful Message.Interesting to read this blog.
what is python used for
highest paying skills
why learn python
hadoop architecture
java interview questions for experienced
data scientist courses
thanks for giving that type of information. I am really happy to visit your blog.Leading Solar company in Andhra Pradesh
ai advantages
.net application
what is hadoop technology
devops deployment tools
selenium interview questions and answers pdf download
digital marketing interview questions and answers for freshers
bay từ hàn quốc về việt nam
>đặt vé máy bay hải phòng hồ chí minh
ve may bay gia re di ha noi
hà nội nha trang vé máy bay
ve may bay gia re di quy nhon
Data Analytics Courses in Bangalore
Data Scientist Course
Digital Marketing Course
Artificial Intelligence Training in Bangalore | Artificial Intelligence Online Training
Python Training in Bangalore | Python Online Training
Data Science Training in Bangalore | Data Science Online Training
Machine Learning Training in Bangalore | Machine Learning Online Training
AWS Training in bangalore | AWS Online Training
UiPath Training in Bangalore | UiPath Online Training
data scientist training in malaysia
AI Patasala Courses in Data Science
cyber security course
business analytics course in varanasi
data scientist course in hyderabad
Data Scientist Course in Bangalore
cyber security training malaysia
business analytics course in pondicherry
Data Science Course in Bangalore
Data Science in Bangalore
Data Science Course in Bangalore with Placement
Data Scientist Course in Delhi
<a href="https://360digitmg.com/india/data-analytics-certification-training-course-in-patna>data analytics course in patna</a>
data science training in borivali
Rise Food Beach Noida Extension
Techooly