Tuesday, September 12, 2017

Demo 32: Simple Machine Learning - Artificial neural network demo using Arduino ESP32

1. Introduction
In this demo, I will make a simple Machine Learning - Artificial neural network system using Arduino ESP32. Certainly, I will not use ESP32 for training process; instead, i will use Python + numpy for training process. After training, I will use the result weights will be used by ESP32 for output calculation based on input.
Artificial neural network (ANN) systems are inspired by the biological neural networks. These systems can learn to do tasks by considering examples, without task-specific programming. One of most famous methods for ANN training is Backpropagation. You can refer here.
In this demo: ESP32 with 2 buttons (B1 and B2) with possible states: pressed (0) or released (1) and there is 1 LED output with possible state state on (1) or off (0). The LED output can be set based on the state of input.
B1 B2 LED
pressed pressed off
pressed released on
released pressed on
released released off
Figure: state of LED based on state of buttons
We will create an ANN to learn the table above. The network has structure:
Figure: ANN structure of demo
2. Hardware
Here we set input pins as INPUT_PULLUP so the schematic is simple and we can re-use the schematic in Demo 21.
 Figure: hardware connections
[ESP32 GIO12 - BUTTON 1 - GND]
[ESP32 GIO13 - BUTTON 2 - GND]
[ESP32 GIO14 - LED - GND]
3. Software
3.1 ANN is implemented by Python + numpy
I used the code here.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import numpy as np
 
epochs = 10000                                  # Number of iterations
inputLayerSize, hiddenLayerSize, outputLayerSize = 2, 3, 1
L = 0.1                                         # learning rate      
 
X = np.array([[0,0], [0,1], [1,0], [1,1]])      # Buttons states array
Y = np.array([ [0],   [1],   [1],   [0]])       # LED states array
 
def sigmoid (x): return 1/(1 + np.exp(-x))      # activation function
                                                # weights on layer inputs
Wh = np.random.uniform(size=(inputLayerSize, hiddenLayerSize))
Wz = np.random.uniform(size=(hiddenLayerSize,outputLayerSize))
 
for i in range(epochs):
 
    H = sigmoid(np.dot(X, Wh))                  # calculate forward part
    Z = np.dot(H,Wz)                            # 
    E = Y - Z                                   # calculate error
    dZ = E * L                                  # delta Z
    Wz +=  H.T.dot(dZ)                          # calculate backpropagation part
    dH = dZ.dot(Wz.T) * sigmoid_deriv(H)        # 
    Wh +=  X.T.dot(dH)                          # update hidden layer weights

print("**************** error ****************") 
print(E)
print("***************** output **************") 
print(Z)   
print("*************** weights ***************") 
print("input to hidden layer weights: ")     
print(Wh)
print("hidden to output layer weights: ")
print(Wz)

Figure: training process and weights output
3.2 Arduino ESP32
The Arduino code just do the forward part to calculate the output when getting the inputs.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
#include <math.h>

#define B1  12
#define B2  13
#define LED 14

int X[1][2]    =   {{1,0}};
/*these matrices was calculated by python */
float W1[2][3] =   {{0.74000854,  4.47769531, -0.98692059}, 
                    {0.83034991,  4.48772758, -0.55733578}};
float W2[3][1] =   {{-6.17234487}, 
                    {4.8835918}, 
                    {1.28875386}};
float Wo1[1][3];
float sum = 0;
float Y = 0;

/*sigmoid function*/
float sigmoid (float x)
{
    return 1/(1 + exp(-x));
}


void setup()
{
  Serial.begin(115200);
  pinMode(B1, INPUT_PULLUP); 
  pinMode(B2, INPUT_PULLUP); 
  pinMode(LED, OUTPUT); 
  digitalWrite(LED, LOW);
}

void loop()
{
  X[0][0] = digitalRead(B1);
  X[0][1] = digitalRead(B2);
  printf("B1 = %d, B2 = %d\n", X[0][0], X[0][1]);
  
  /* calculate forward part based on weights */
  //hidden layer
  for(int i=0; i<1; i++)
  {
      for(int j=0;j <3; j++)
      {
          for(int k=0; k<2; k++)
          {
              sum += X[i][k]*W1[k][j];
          }
          Wo1[i][j] = sigmoid(sum);
          sum = 0;  
      }
  }
  //output layer
  for(int i=0; i<1; i++)
  {
      for(int j=0;j <1; j++)
      {
          for(int k=0; k<3; k++)
          {
              Y += Wo1[i][k]*W2[k][j];
          } 
      }
  }
  printf("Y = %f\n", (Y));
  Y = round(Y);
  digitalWrite(LED, int(Y));
  Y = 0;
  delay(1000);
}
4. Result

2 comments:

ppf said...

sigmoid derivative function is missing

def sigmoid_deriv(x):
f = sigmoid(x)
return f * (1 - f)

source : https://stackoverflow.com/questions/10626134/derivative-of-sigmoid

Great post.
Very helpfull to start experiments.
Lot of your other posts are also well done.
Thx.

iotsharing dotcom said...

Thank friend,

this is feed forward part so you will not see derivative.
I am wrtting new blog about machine learning.
It is http://www.fossreview.com/
Hope you like it :)
Regards