This post is still a draft

Writing a custom image recognition website in an hour.

In this post I will show you how to write a custom image recognition website. The end result will be a website which can distinguish multiple type of objects.
The software we will be using:

Server side Client side
R server R client
NodeJS R studio


  1. Train model localy
  2. Push model to server
  3. consumer model via web page

Train model

Setup R studio

We need to setup R studio to use R server (which you installed in the R client). in R studio go to: Tools -> global options it should look like this if you have installed R client in the default location.
R studio config

Load training data

The data set I use is the Caltech 101 [1] dataset this dataset has 101 classes of different pictures to keep training times low I will use only 5 classes. You can download the entire data set here, you can also just download the classes which I am training here(way smaller download).

# Name the folders you want to include in the training set
names =c('airplanes','car_side','helicopter','cannon','wrench');

# Create an empty data frame
trainData <- data.frame(path= c(),label=c())

#loop through the different object types.
for(i in names){
  folder = paste('C:/PATH TO FOLDERS/',i,sep = "")
  list = list.files(path = folder,full.names = TRUE)
  data <- data.frame(Path = list, 
                      Label = c(rep(i,length(list))), 
                      stringsAsFactors = FALSE)
  trainData <- rbind(trainData,data)

# Check the resulting data frame.

Train the model

Now that we have the training data ready it is time to build a model which is able to classify the different types of objects, we will use the package MicrosoftML to do this for us.

  • Install and load the package
  • Train the model, in this case I will be using a Logistic regression as this gives good result without any parameter tuning. The function might look a little unfamiliar for people who have not worked with ScaleR functions before.
model <- rxLogisticRegression(
  Label ~ Features, # Basic function formula
  data = trainData, # The dataset with the path and object type we created 

  # List of transforms before the data is put into the model.
  # This is necessary as the model does not take a path to the model, it needs the features of the picture
  mlTransforms = list(     
    loadImage(vars = list(Features = "Path")), # Load the images
    resizeImage(vars = "Features", width = 224, height = 224), #resize images
    extractPixels(vars = "Features"),
    # Pass the image through a pre trained Resnet18 to get the image features
    featurizeImage(var = "Features")
  mlTransformVars = "Path",
  # Because we have more then 2 classes we need a multi class logic regression

Congratulations!!! you have just build your own image classification model now lets take it for a test drive. We are going to use the same training data to test the model.

# Predict the model using the same training data and the model we created
outcome = rxPredict(model, data = trainData)

# Make a data frame with values to let us know if the prediction was correct
y = sapply(1:nrow(outcome), function(x){
 if(outcome$PredictedLabel[x] == trainData$Label[x]){

# Check if the model preformed well, in my case it did very well.

next we need to wrap this prediction into a function, which we will use later to push to our server. Making the prediction into a function is very simple just like this:

getObject <- function(name){
  # Create a similar data frame like the training data set
  data <- data.frame(Path=c(name),Label=c(''),stringsAsFactors = FALSE)
  print(rxPredict(model, data = data))
# Test the function with the path to a object picture. 
getObject('PATH TO A PICTURE')

You are completely ready to setup a server and publish the model.

Setup the server

login to Azure here, if you do not have an account you can just create it freely and get some free credits. Once logged on create a R server like below, follow the wizard, for size choose something like DS2_v2 it does not have to be big. Once you are finished with this demo you can simply destroy the machine again so it will not nearly cost as much as the price states ;).

Create R server

Once this is created we need to a open the firewall to allow people to access the website we will be making.

Go to the network security gorup.

Open port 12800 for the R server and port 80 for the web server.

When the firewall is setup Remote Desktop into the machine like this:
Connect to vm

Once you are in the machine there are a few things that need to happen.

  • Firstly go to the 'control panel' --> 'programs and features'. Find R server and press the 'Change' button, you will be prompted with a wizard check the 'Install pre trained models' and finish the wizard.
  • Boot R server, from the start menu go to 'Microsoft R server' --> 'R server utils'. Select 'Configure R for operationalization' --> Select 'One-Box', for ease of use, take the same password as your machine. Your R server should be setup now.
  • Download and install NodeJS as this is going to be our web server.
  • Once NodeJS is installed allow it to make internet connection like this: 'control panel' --> 'Windows Firewall' --> 'Allow an app ..... Firewall' find and add NodeJS and save.
  • Download the web server code here, put this is an easy place like Documents, if you have downloaded it via zip extract the files.
  • Open the server.js file in a editor and add your login information. This is the same username and password you have chosen for the server it self.
  • R server needs access to the folder where the pictures will be uploaded so right click on the root folder (props to Joris ;)).
  • Open Powershell and go the the place where you have downloaded the code like: cd ./PATH TO CODE.
  • Install all the packages with a simple command npm install
  • Run npm start to start the server.
  • Take a cup of coffee you deserve it.

Push model to the server

As last step before we can use the model we need to push the function which we created in R to the server. Before we can push code the server from our client we need to login to the server.

remoteLogin("http://IP OF YOUR R SERVER:12800", 
            username = "admin", 
            password = "YOUR PASSWORD",
            session = FALSE)

Once logged-in we can push the model with this simple single command.

api <- updateService(
  # Name of our api
  # Name of the function we have created
  code = getObject,
  # Model we have created
  model = model,
  inputs = list(name = "character"),
  outputs = list(answer = "data.frame"),
  v = "v1.0.0"

Use your model

Everything should be running and working now. To test your model simply go the the ip address of your server which is found in the azure portal, you should be greeted by this beautiful home page. Select a picture and press upload, the result should appear quickly on the page.

If You have any questions just leave them in the comments.

ps. sorry for the long post

  1. L. Fei-Fei, R. Fergus and P. Perona. Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. IEEE. CVPR 2004, Workshop on Generative-Model Based Vision. 2004 ↩︎