<header class="entry-header">
<h1 class="entry-title">Handwritten Digit Recognition Using Artificial Intelligence on a Low-Cost FPGA Board</h1>
<div class="entry-meta"><span class="posted-on"><a href="https://blog.digilentinc.com/handwritten-digit-recognition-using-artificial-intelligence-on-a-low-cost-fpga-board/" rel="bookmark"><time class="entry-date published" datetime="2019-11-18T12:13:09-08:00">November 18,2019</time></a></span><span class="meta-sep"> - </span><span class="byline">by <span class="author vcard"><a class="url fn n" href="https://blog.digilentinc.com/author/davidh/">David Horn</a></span></span><span class="meta-sep"> - </span><span class="comments-link"><a href="https://blog.digilentinc.com/handwritten-digit-recognition-using-artificial-intelligence-on-a-low-cost-fpga-board/#respond">Leave a Comment</a></span></div>
</header><a class="image-link" href="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-5.png"><img class="attachment-hitmag-featured size-hitmag-featured wp-post-image" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-5-622x400.png" alt="" width="622" height="400" data-wp-pid="27627" /></a>
<div class="entry-content">

In the world of artificial intelligence (AI),the recognition of handwritten digits proves that you got your neurons right and in working condition. This application of AI is already quite old – its breakthrough came in 1989 when a reliable machine-enabled parsing of ZIP codes for postal services was achieved. Soon after,it was proven that multi-layer feed-forward networks can implement any function. Shortly,financial institutions adopted the technique for the automatic parsing of account numbers on remittance slips for wire transfers or bank checks. Today the recognition of handwritten digits using different AI-techniques is also famous in academia for teaching and learning purposes.
<h4 class="has-medium-font-size"><strong>Recognition of Handwritten Digits is Complex</strong></h4>
<figure class="wp-block-image is-resized"><img class="wp-image-27623" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-1.png" alt="" width="543" height="384" /></figure>
While the presentation of the problem itself has not changed over the years,computing power has increased dramatically during recent decades,making it now possible to run the recognition on an <a href="https://store.digilentinc.com/fpga-development-boards-kits-from-digilent/?utm_source=blog&amp;utm_medium=digit-recognition-using-ai&amp;utm_campaign=fpga">FPGA (Field Programmable Gate Array)</a> board suitable for teaching,together with a small digital camera. Nevertheless,it is still one of the harder tasks for an artificial intelligence to tackle.  Handwritten digits differ in size,width,and orientation,and each person’s handwriting has distinct variations. Different cultures even write digits differently. There are,for example,subtle differences on how Americans and Europeans write the digits 1,4,or 7.

Another area raising digit recognition issues are the similarities between some digits: 3 and 8,0 and 8,5 and 6,2 and 7,etc. Last but not least,the quality of the image itself has a large influence. If not properly trained,the AI will incorrectly classify too many digits for the final product to be of any use. Trying to implement such a recognition system using the classical programming approach will get the programmer in a lot of trouble with a lot of details and special cases.
<h4 class="has-medium-font-size"><strong>Implementing a Textbook Version</strong></h4>
But how does the recognition work? Looking at it from a very high-level,it’s straightforward: An image of the digit needs to be captured,processed and analyzed,and the result presented to the user.

Performing a more thorough analysis,it soon becomes obvious that many more tasks have to be executed and problems solved. The hardware for such a system is not the real obstacle. Digilent,for example,implemented a textbook-version of the digit recognition through artificial intelligence as a proof-of-concept using their <a href="https://store.digilentinc.com/pcam-5c-5-mp-fixed-focus-color-camera-module/?utm_source=blog&amp;utm_medium=digit-recognition-using-ai&amp;utm_campaign=fpga">5 megapixels Pcam 5C fixed-focus color camera module</a> for the image capture,their <a href="https://store.digilentinc.com/zybo-z7-zynq-7000-arm-fpga-soc-development-board/?utm_source=blog&amp;utm_medium=digit-recognition-using-ai&amp;utm_campaign=fpga">Zybo Z7 Xilinx Zynq-7000 ARM/FPGA development board</a>,and its <a href="https://store.digilentinc.com/pmod-mtds-multi-touch-display-system/?utm_source=blog&amp;utm_medium=digit-recognition-using-ai&amp;utm_campaign=fpga">Pmod MTDS multi-touch display system</a> as user interface and to display the result of the recognition process (the whole setup is available at a discount in our <a href="http://bit.ly/2pcyvd6">Embedded Vision Bundle</a>).
<figure class="wp-block-image is-resized">
<figure id="attachment_27624" class="wp-caption aligncenter" aria-describedby="caption-attachment-27624"><img class="wp-image-27624" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-2.png" alt="" width="574" height="601" /><figcaption id="caption-attachment-27624" class="wp-caption-text"><em>Example of a low-cost handwritten digit recognition system using different boards from Digilent</em></figcaption></figure>
</figure>
Once the Pcam 5C has captured an image of a digit at a size of 1,000 by 1,000 pixels (which actually is an overkill for this application),it will send it to the Zybo Z7 through the <a href="https://www.mipi.org/">MIPI</a> CSI-2 interface (MIPI Camera Serial Interface 2) and be pre-processed on the FPGA there. Here it will be scaled down to 500x 500 pixels,converted into greyscale,and finally binarized into black-and-white.

Further processing takes place on the 667 MHz dual-core <a href="https://www.arm.com/products/silicon-ip-cpu/cortex-a/cortex-a9?utm_source=google&amp;utm_medium=cpc&amp;utm_campaign=https://www.arm.com/products/silicon-ip-cpu/cortex-a/cortex-a9?utm_source=google&amp;utm_medium=cpc&amp;utm_campaign=2019_brand_cpus_mk30_search_bol&amp;utm_source=google&amp;utm_medium=cpc&amp;gclid=EAIaIQobChMI3LGAts615QIVpBitBh0r5AxZEAAYASAAEgI2UPD_BwE">ARM-Cortex-A9 processor</a> where a bounding box search – the search for the smallest frame containing the digit – is performed,the digit centered in the bounding box  (centering it by the center of mass within a larger window would have been another possibility) and the bounding box scaled to an image size of 28 by 28 pixels. This size was chosen,as the training set for the neural network uses this resolution.
<div class="wp-block-image">
<figure class="aligncenter"><img class="wp-image-27625 aligncenter" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-3.png" sizes="(max-width: 624px) 100vw,624px" srcset="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-3.png 624w,https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-3-600x195.png 600w" alt="" /><figcaption><em>Neurons in the FPGA</em></figcaption></figure>
</div>
From this image,each pixel is used as input to one of the 784 (28 x 28) input neurons of the feedforward artificial neural network (ANN) on the FPGA. Each neuron there uses a fixed-point Q4.11 representation and consists of a multiplier,multiplying the input pixel width the weight derived from the training and one adder adding the bias – also created by the training data – to the multiplication result (see above image). Finally,a sigmoid function is used to create the activation output of the neuron. Equations 1 and 2 give the mathematical details:

<em>a= σ(w∙x+b) </em>(Equation 1)
<em>σ(z)= 1/(1+e^(-z) )</em> (Equation 2)
where w is the weight,b the bias,and the sigmoid function.

The activation output of the input layer is then used as input to all of the 16 neurons of the first hidden layer,with the mysteriously sounding term “hidden layer” meaning that the neurons belonging to that layer are neither input nor output neurons. The output of the hidden layer 1 is then used as input to all neurons of the second hidden layer,also consisting of 16 neurons. With two hidden layers,the system technically qualifies as a deep neural network (deep learning),but it cannot be compared to other systems with several thousand layers.
<h4><strong>Each Output Neuron Corresponds to One Digit</strong></h4>
The output layer finally consists of 10 neurons,one for each digit in the [0…9] range. Each output neuron has an activation value and the neuron with the highest activation value is considered the correct guess. If,for example,the output neuron representing the digit 4 has the highest value,we consider the digit 4 as the best guess of the recognition.
<figure id="attachment_27635" class="wp-caption aligncenter" aria-describedby="caption-attachment-27635"><img class="wp-image-27635 size-full" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-4-1.png" sizes="(max-width: 506px) 100vw,506px" srcset="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-4-1.png 506w,https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-4-1-135x93.png 135w,https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-4-1-135x93@2x.png 270w" alt="" width="506" height="349" data-wp-pid="27635" /><figcaption id="caption-attachment-27635" class="wp-caption-text"><em>Structure of the artificial neural network</em></figcaption></figure>
As the outputs of the neural network are not probabilities per se,the degree of certainty of the result cannot be told. But this could be solved by adding a Softmax layer to the network,which turns numbers into probabilities. The final result of the recognition is then displayed on the Pmod Multi-Touch Display System.
<figure id="attachment_27638" class="wp-caption aligncenter" aria-describedby="caption-attachment-27638"><img class="size-medium wp-image-27638" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-5-1-600x426.png" sizes="(max-width: 600px) 100vw,600px" srcset="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-5-1-600x426.png 600w,https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-5-1.png 622w" alt="" width="600" height="426" data-wp-pid="27638" /><figcaption id="caption-attachment-27638" class="wp-caption-text"><em>This is all it takes to run a handwritten digit recognition powered by artificial intelligence</em></figcaption></figure>
<h4><strong>Training is Most Important</strong></h4>
As with all artificial neural networks,the system needs to be trained before it can be used. In case of our example,the universality theorem was applied to compute the weights and biases from images in the widely used MNIST-(Modified NIST-)data base,which comprises 60,000 training images and 10,000 testing images,both sets with labeled images of handwritten digits in the range from zero to ten. The data base is called MNIST,as it is a subset of the data sets collected by NIST,the United States National Institute of Standards and Technology. The training was supervised and performed on a PC using Python. Each training set is a pair of input and the desired output and the training algorithm was taken from the open-source textbook from Michael A. Nielsen<a href="https://blog.digilentinc.com/handwritten-digit-recognition-using-artificial-intelligence-on-a-low-cost-fpga-board/?utm_source=newsletter&amp;utm_medium=email&amp;utm_campaign=digilent_newsletter&amp;utm_content=november_2019#_ftn1">[1]</a>.
<h4 class="has-medium-font-size"><strong>What was Achieved?</strong></h4>
Once trained,the recognition rate of the trained artificial neural network on the MNIST test images reached 95.2%. Higher rates can be achieved if a convolutional neural network (CNN) is used. The best academical results with an error rate of 0.23% have been achieved with a hierarchy of 35 convolutional neural networks,something which would have been beyond the scope of this proof-of-concept. It also turned out that the live recognition rate highly depends on the digit skew,as automatic de-skewing was not implemented,but manually performed.  The most critical digit turned out to be the digit “9”,which was rarely recognized by the demonstrator. Once turned,it was mostly recognized as “6”,something which points to the fact that retraining the system should help.

Pre-processing took 1664 ms,if done solely on the dual-core ARM-Cortex-A9 processor on the Zybo Z7 development board from Digilent and 999 ms if performed on the ARM-core and the FPGA. Execution time of the artificial neural network was 73 ms on the ARM and 57 ms if done on ARM plus FPGA.
<p class="mceTemp"></p>

<figure id="attachment_27628" class="wp-caption aligncenter" aria-describedby="caption-attachment-27628"><img class="size-full wp-image-27628" src="https://blog.digilentinc.com/wp-content/uploads/2019/10/AI-article-6.png" alt="" width="225" height="377" data-wp-pid="27628" /><figcaption id="caption-attachment-27628" class="wp-caption-text"><em>The resulting guess is displayed on a Pmod MTDS </em></figcaption></figure>
This textbook-implementation of a handwritten digit recognition using a low-cost FPGA-board demonstrated that is it possible to implement such an artificial neural network with deep learning on such a system. The Xilinx Zynq-7000 ARM/FPGA development board,together with the Pcam 5 and the Pmod MTDS display – all from Digilent and available at a discount in the<a href="http://bit.ly/2pcyvd6"> Embedded Vision Bundle</a> –allows to simply take a textbook and to implement a live capture and inference system from scratch.

<a href="https://blog.digilentinc.com/wp-admin/post.php?post=27622&amp;action=edit#_ftnref1">[1]</a> Michael A. Nielsen,„Neural Networks and Deep Learning”,Determination Press. 2015 (<a href="http://neuralnetworksanddeeplearning.com/chap1.html">http://neuralnetworksanddeeplearning.com/chap1.html</a>)

</div>

Handwritten Digit Recognition Using Artificial Intelligence on a Low-Cost FPGA Board

Read articles, tips, and resources to help you learn, grow, and succeed. Explore our latest posts covering education, community, and personal growth

Blog – Insights, Tips & Resources for Learning and Growth

Explore articles, tips, and insights to help you succeed in learning, community building, and personal development.

Handwritten Digit Recognition Using Artificial Intelligence on a Low-Cost FPGA Board

Handwritten Digit Recognition Using Artificial Intelligence on a Low-Cost FPGA Board

Related Posts

Windows 10 on the Raspberry Pi: What you need to know

Why Open Source Matters: Understanding its Meaning and Significance

What’s Next? 7 Key Trends Shaping Industry’s Future

What Vending Machines Make the Most Money for You?

What Open Source Means & Why It’s Important?