Setting up OpenVINO™ in the cloud

9 min readJan 28, 2020

This article was born out of the multiple requests from my course mates in the Intel AI challenge on how to install OpenVINO™ in the cloud. The cloud provider used in this article is AWS. This decision is based on the fact that their free tier servers come with 1GB RAM and this is good because some of the processes are RAM intensive.

Warning: There are lots of images in this article because i believe pictures speak louder than words. Hope you enjoy learning something new.

Step 1: Provision a server on AWS

Create an AWS account. Your credit card details will be needed before you can complete registration. Once registration is done, go to the console home and search for EC2.

Once you get to the EC2 page, you click instances on the menu on your left, then click Launch instances. On the Launch instances page, you’ll see multiple AMIs(Amazon Machine Image), choose Free Tier only. I recommend the Ubuntu Server 18.04 LTS (HVM), SSD Volume Type because it is one of the Operating Systems supported by OpenVINO™.

Please select the 64-bit (x86) on the AMI page because the OpenVINO™ Toolkit runs only on Intel Processors so the ARM processor won’t work for this deployment. I’ll be posting images on the next few steps.

Choose the t2.micro to still be in the free tier. You can see the processor being used at the top as per what I said earlier.

Click Next: Configure instance details to go on to the next page. Leave this page as it is. Don’t modify anything here if you don’t know how they work.

Click Next: Add Storage to go on to the next page. On the free tier, you are allocated 30GB of storage on the EBS(Elastic Block Store). This is the storage device that is attached to your server. I’m using 26GB, this still keeps me under the free tier.

Click Next: Add Tags to go on to the next page. Leave this page as it is. Don’t modify anything here if you don’t know how they work.

Click Next: Configure Security Groups to go on to the next page. A security group is a set of firewall rules that control the traffic for your instance. I’m going to open the HTTP and HTTPS port on 80 and 443 respectively because I’m going to be running a webapp on the server. Click Add Rule and select type to choose.

Click Review and Launch. Click Launch on the next page. Before Launching you’ll be asked to create a new key pair or select an existing one. A key pair consists of a public key that AWS stores, and a private key file that you store. Together, they allow you to connect to your instance securely. You’ll create a new key pair and you’ll be asked to download the private key. Store it in a secure and accessible location. You will not be able to download the file again after it’s created. Please never share this file on a public page like Github e.t.c. Click Launch Instances once you’re done downloading your private key.

Click View Instances to view your Instance. You’ll see your instance running and you’ll be able to connect to it. Connecting to your instance depends on the Operating System that you’re using locally. Visit Here to see how to connect to your instance from any Operating System. Once you’ve connected you should have a terminal connection to the instance. Here’s mine:

Step 2: Create Swap Space

Swap Space is a space in Linux used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space. Swap space can help machines with a small amount of RAM. In case you chose a VM instance with ≤ 1GB RAM, then your memory will get exhausted and you will get a cannot allocate memory error. The reason for this is because you’ll face an issue later on while running the OpenVINO™ Verification Scripts because you’ll exhaust your RAM during the Building of CXX Objects.

Create a swap file to add 6GB (6G) space. You can also increase the amount of space you need. I’m actually overcompensating with 6GB.

$ sudo fallocate -l 6G /swapfile

Now you have a swap file of correct size and can be enabled as swap space. Proceed by setting up correct permissions.

$ sudo chmod 600 /swapfile

You can now mark the file as swap space with the mkswap command.

$ sudo mkswap /swapfile

You will receive the output similar to this.

Output
Setting up swapspace version 1, size = 6 GiB (6442446848 bytes)
no label, UUID=f59595fb-754b-47ae-af6b-8dd6e98654d8

Enable the swap space and allow the system to utilize the space.

$ sudo swapon /swapfile

Verify that the swap is available by typing:

$ sudo swapon --show

You will receive the output similar to this.

Output
NAME      TYPE  SIZE USED PRIO
/swapfile file   6G  0B   -2

You can check the output of the free utility.

$ free -h

You will receive the output similar to this.

Output
           total   used   free  shared   buff/cache   available
Mem:       983M    99M    473M   748K     411M         744M
Swap:      6.0G    0B     6.0G

Now swap space is only created for the current session. If you reboot the server the current changes will be gone. So you can make this setting permanent by adding the swap file to our /etc/fstab file.

$ sudo nano /etc/fstab

Add the following at the end of the file.

/swapfile swap swap defaults 0 0

That’s all about swap space.

Step 3: Install OpenVINO™

To download OpenVINO™ to your instance, visit this Link to get OpenVINO™ for Linux. Click Linux* (supports Ubuntu*, CentOS*, and Yocto Project*) to get to the next page. Click Register & Download. After registering, get the download link of the full package and run this command on your instance.

$ wget http://registrationcenter-download.intel.com/akdlm/irc_nas/16057/l_openvino_toolkit_p_2019.3.376.tgz

The package has been downloaded to the home directory which is /home/ubuntu you can check with pwd . You can also check the contents of the directory using ls .

Unpack the downloaded .tgz file:

$ tar -xvzf l_openvino_toolkit_p_2019.3.376.tgz

Go to the l_openvino_toolkit_p_2019.3.376 directory:

$ cd l_openvino_toolkit_p_2019.3.376

Your instance is not setup to run any GUI operation, so everything has to be done using the command line. Install dependencies of OpenVINO™ by running this script from the package.

$ sudo -E ./install_openvino_dependencies.sh

After it’s done installing the dependencies, run this script to install OpenVINO™:

$ sudo ./install.sh

Follow the prompts from the script to finish the installation.

On the next Page you’ll see the Terms & Conditions Page. Press your Space Bar until you get to the bottom of the page, then type accept and press Enter to continue:

On the next page you’ll be asked for consent to the collection of your information:

Consent to collection of Information Page

After the consent page, a check for prerequisites fulfillment is carried out and some missing optional prerequisites are pointed out to you.

Missing optional prerequisites
-- Intel® Graphics Compute Runtime for OpenCL™ Driver is missing but you will be prompted to install later
-- Missing required libraries or packages. You will be prompted to install them later

The first missing prerequisites is a driver that enables the use of GPU in carrying out inference with the OpenVINO™ Toolkit. We don’t have GPU on this instance so it can be ignored. The second missing prerequisites will be installed later on. Press Enter to accept default choice.

The next page shows you all the components that will be installed by the OpenVINO™ Toolkit. Press Enter to accept default choice.

The missing prerequisites page will come up again with the driver that enables the use of GPU in carrying out inference with the OpenVINO™ Toolkit. Ignore like before. Press Enter to accept default choice.

The OpenVINO™ installation will begin after this.

Once the installation is done, press Enter to exit the installation script.

You must update several environment variables before you can compile and run OpenVINO™ applications. Run the following script to temporarily set your environment variables:

$ source /opt/intel/openvino/bin/setupvars.sh

The OpenVINO™ environment variables are removed when you close the shell. As an option, you can permanently set the environment variables as follows:

Open the .bashrc file in /home/ubuntu :

$ nano /home/ubuntu/.bashrc

Add this line to the end of the file:

$ source /opt/intel/openvino/bin/setupvars.sh

Save and close the file: press ctrl+x, type y and press Enter.

To test your change, open a new terminal. You will see [setupvars.sh] OpenVINO environment initialized.

That’s all about installing the OpenVINO™ Toolkit.

Step 4: Configure the OpenVINO™ Model Optimizer

Go to the Model Optimizer prerequisites directory:

$ cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites

Run the script to configure the Model Optimizer for Caffe, TensorFlow, MXNet, Kaldi*, and ONNX:

$ sudo ./install_prerequisites.sh

Step 5: Run the Verification Scripts to Verify Installation

Go to the Inference Engine demo directory:

$ cd /opt/intel/openvino/deployment_tools/demo

Run the Image Classification verification script:

$ ./demo_squeezenet_download_convert_run.sh

This verification script downloads a SqueezeNet model, uses the Model Optimizer to convert the model to the .bin and .xml Intermediate Representation (IR) files. The Inference Engine requires this model conversion so it can use the IR as input and achieve optimum performance on Intel hardware. This verification script builds the Image Classification Sample Async application and run it with the car.png image located in the demo directory. When the verification script completes, you will have the label and confidence for the top-10 categories:

Run the Inference Pipeline verification script:

$ ./demo_security_barrier_camera.sh

This script downloads three pre-trained model IRs, builds the Security Barrier Camera Demo application, and runs it with the downloaded models and the car_1.bmp image from the demo directory to show an inference pipeline. The verification script uses vehicle recognition in which vehicle attributes build on each other to narrow in on a specific attribute. First, an object is identified as a vehicle. This identification is used as input to the next model, which identifies specific vehicle attributes, including the license plate. Finally, the attributes identified as the license plate are used as input to the third model, which recognizes specific characters in the license plate. Normally when the verification script completes and you have a GUI interface, you will see an image that displays the resulting frame with detections rendered as bounding boxes and text but since we don’t have a GUI interface and all our processes are being carried out in the terminal, you’ll get an error:

Congrats!!!

You have completed all required installation, configuration and build steps in this guide to use your Server to work with OpenVINO™ Toolkit. You can run any OpenVINO™ project on this server now.

You can refer to this article on how to Deploy Django/Flask applications on a VPS using Nginx, Gunicorn and Supervisor.

REFERENCE Links:

1) Accessing your Instances : https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html
2) Swap Space : https://www.cloudbooklet.com/how-to-add-swap-space-on-ubuntu-18-04-google-cloud/
3) OpenVINO™ Installation guide For Linux: http://docs.openvinotoolkit.org/2019_R3.1/_docs_install_guides_installing_openvino_linux.html

Setting up OpenVINO™ in the cloud

Congrats!!!

REFERENCE Links:

Written by Ahiwe Onyebuchi Valentine