As a technology solution provider, our focus is to help schools reducing the drop out rate using technology.  For this purpose, we will be using Machine Learning (AI) to predict the student churn and identify dropout students at an early stage.

We will be walking-through an experiment on Azure Machine Learning Studio that will help us to predict future student churn predictions.

In order to achieve our goal, we will be using Two Class Decision Forest algorithm for our solution that comes with Azure Machine Learning Studio. We will be using sample data for this experiment.

So, in the first step we will upload the data on Azure Machine Learning Studio in the CSV file format, where it contains historical data composed of attributes on various parameters such as:

• School Id
• Student Id
• Gender
• Subject marks
• Students performance
• Guardian
• Internet
• Number of absences
• Lack of awareness
• School performance
• Establishment year
• Health protected
• Continue or Drop (Target feature)

• First, open Azure Machine Learning home page and Click +NEW at the bottom of the window
• Select DATASET
• Select FROM LOCAL FILE

In the Upload a new dataset dialog, click Browse, and find the studentChurnPrediction.csv file you created.

Now in the next step we will create an experiment in Machine Learning Studio that uses the data you uploaded. So, click +NEW at the bottom of the window and Select EXPERIMENT, and then select “Blank Experiment”.

Select the default experiment name at the top and rename it to KAISPE Student Churn Prediction and in the module palette to the left of the experiment page, expand Saved Datasets. Find the dataset you created under My Datasets and drag it onto the canvas.

Now let’s prepare and clean the data by using Select Columns in Dataset and Clean Missing Data module which is useful if you want to clean and reduce the size of the data by deleting unwanted columns.

In addition, we will be using Edit metadata module which will help us to select categorical column in our dataset.

• Search and drag Edit Metadata and in the Properties pane to the right page, click Launch column selector and select the following categorical columns and make categorical.

Now, we’ll use our data for both training the model and testing it by splitting the data into separate training and testing datasets.

• Search and drag Split Data onto the canvas and connect to the last Edit Metadata module.
• Click Split Data and in the Properties pane to the right of the canvas and set it to 0.75. In this way, we will use 75% of the data to train the model and 25% of the data for testing.

In the next step we will be applying Two Class Decision Forest machine learning algorithm which is most suitable for our KAISPE Student Churn Prediction experiment.

So, in the next step we will find and drag the Train Model module to the experiment canvas.

• Connect the output of the Two Class Decision Forest algorithm to the left input of the Train Model module
• Connect the training data output (left port) of the Split Data module to the right input of the Train Model module.

After successfully we have trained our model with 75% of the data, we can use it to score and evaluate model to check the other 25% of the data to understand the function of our model.

Now, we will be Running our experiment to Visualize the output of our Score and Evaluate the model.

I hope you found this blog post helpful. If you have any questions, please feel free to contact info@kaispe.com.

The post Student Churn Prediction using Microsoft Azure Machine Learning appeared first on KAISPE.

In today’s blog we will discover some useful insights about using a created model on Azure Machine Learning Studio and call that model in our power bi so we have data in power bi and use that API as a tool for machine learning.

Before we move any further let me remind you that there are various options to consume machine learning in power bi but for this blog post we will only be covering couple of them.

So, first we will be covering Microsoft Standard to invoke an Azure Machine Learning Studio (classic) model into Power BI. Here the following steps you can follow to invoke a machine learning model into power bi:
1- create an Azure ML model if you don’t already have a model and publish it.
2- Next we have to access  Azure Machine Learning model from power bi, To do that we have to get a Reader role from Azure subscription.
3- Now we have to create a Dataflow in power bi whom you granted access to Azure Machine Learning Model. After you signed in:
– create a workspace and navigate to a workspace on your dedicated capacity that has
the AI preview enabled and Select Add new entities.
– Upload the dataset Text/CSV File as our data source
4- In the last step we will apply insights from Azure Machine Learning model, navigate to AI Insights button in the ribbon, and from Azure Machine Learning Models folder navigate to the Azure ML models to which you’ve been granted access are listed as Power Query functions with a prefix AzureML.

To invoke an Azure Machine Learning model, we will specify our input parameters such as (timestamp, air pressure, angular speed, piston speed and piston vibration). In last step, select Invoke to view the preview of the Azure ML model’s output as a new column in the entity table.

Now, we can also consume a machine learning webservice in power bi which is much easier to integrate for free. To achieve the task, we have to follow these steps:
1- You should have an Azure Machine Learning model and deployed as a webservice.
2- Import the dataset from your local computer and navigate to the query editor
3- next we need to navigate to the Run R script where we connect our Azure Machine Learning model with Power BI. For the following step you have to have the following credentials from Azure Machine Learning Studio workspace: (1) workspace Id (2) Authentication token (3) Service name. Next hit the OK button to view the result in output column.

I hope you found this blog post helpful. If you have any questions, please feel free to contact info@kaispe.com.

The post Microsoft Power BI and Azure Machine Learning appeared first on KAISPE.

At KAISPE, we pride ourselves on our innovation, expertise, and our continued commitment to ensuring delivery of the highest quality products to our customers. As a Microsoft Certified Partner, we provide solutions backed by industry best practices and standards. We are Microsoft Co-Sell partner and have strong presence on Microsoft marketplaces with our remarkable apps and services.

To learn more, visit https://testing.kaispe.com or call (315) 791-4472. Connect with us on Twitter, LinkedIn, and Facebook.

The post KAISPE Azure Machine Learning PoC Offering is now available on Microsoft Marketplace appeared first on KAISPE.

As we all know water irrigation is the foundation of global agricultural productivity, enabling farmers to buffer crops to cope with the risks of rainfall variability and drought. However, freshwater supplies to irrigated agriculture are increasingly constrained by increased demand for water from other sectors, changes in water supply due to climate change and increased awareness of the impact of irrigation on the environment.

To address these challenges, Machine learning technology plays an important role in water irrigation system. According to study by applying machine learning in agriculture it enhances the water irrigation system which helps us to use the water in efficient manner and reduces water waste.

So, In the last blog we created a model that predicts the future water requirements based on a given data using Two Class Decision Forest. Now, in today’s experiment we will try to improve our solution to get better insights of Water Irrigation system Using Microsoft Azure Machine Learning.

First, we need to collect data to build a training dataset. We choose the attributes that have significant influence on crop water usage and the data for which are available throughout the whole cropping seasons. Please note that the data we collected for this experiment was through KAISPE Agriculture Remote Monitoring solution using Azure IoT.

Here we have the example of a training dataset:

Our training dataset contains historical data composed of attributes on various weather parameters Temperature, Humidity, Rainfall, Wind speed, Soil Moisture, phLevel combined with Crop Type and Water Usage.

Next, the method or technique we use to predict water usage in this experiment is Decision Forest Regression. So, what is Decision Forest Regression and how decision forests work in regression tasks, lets take a look into it:

Building a number of trees called a forest, instead of a single tree. It does so in order to extract more patterns and logic rules. It basically uses the same approach of a single tree building algorithm.

Decision trees have these advantages:

• They are very effective in both calculation and memory usage during training and prediction.
• They can represent non-linear decision boundaries.
• They perform integrated feature selection and classification, and are resilient in the presence of noisy features.

The regression model consists of a set of decision trees. Each tree in the regression decision forest outputs a Gaussian distribution as a prediction. Perform aggregation on the collection of trees to find the Gaussian distribution of the combined distribution of all the trees closest to the model. For more information: https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/decision-forest-regression

Here we have the example of a multiple decision trees from our dataset:

Now, in order to evaluate the performance of our data pre-processing techniques we build training dataset which we divide into two parts training(70%) and testing(30%). A Decision Forest Regression algorithm is applied on the training dataset, it is then applied on testing dataset to check the prediction accuracy of unseen records. Before we evaluate our model, let see the score model to see what our model has scored.

Finally, the performance of the model on the dataset is evaluated by prediction accuracy. Here we have the result that has performed the best with the accuracy of 95%.

So, if you can see the given output has Coefficient of Determination (which also known as R square) is 0.95, Mean Absolute Error is 0.07, Root Mean Squared Error is 0.28, Relative Absolute Error is 0.07 and in last Relative Squared Error which is 0.04.

For better understanding of the result lets take a look into it:

Coefficient of Determination -> It can directly indicate the excellent performance of our model. In more technical terms, we can define it as a “determination coefficient” which is a measure of the variance of the response variable “y”, which can be predicted using the predictor “x”.

Mean Absolute Error -> It is usually used when the performance is measured on continuous variable data. It gives a linear value, which averages the weighted individual differences equally. The lower the value, better is the model’s performance.

Root Mean Squared Error -> In this metric also, lower the value, better is the performance of the model.

I hope you found this blog post helpful. If you have any questions, please feel free to contact me muhammad.ahmad@kaispe.com

The post Water Irrigation Experiment Using Microsoft Azure Machine Learning appeared first on KAISPE.

To improve water management practices and maximize water productivity, In the current situation, models to predict future water requirements based on data mining techniques can be useful.

In this section we will be walking-through an experiment in Azure Machine Learning Studio that will help us to predict future water requirements based on data mining technique.

In order to achieve our goal, we will be using Two Class Decision Forest  for our solution that comes with Azure Machine Learning Studio. Please note that the data we collected for this experiment was through KAISPE Agriculture Remote Monitoring solution using Azure IoT.

So, in the first step we will upload the dataset on Azure Machine Learning Studio in the CSV file format, where it contains historical data composed of attributes on various weather parameters such as Temperature, Humidity, Soil Moisture, phLevel, Rainfall and Wind speed combined with Crop Type and Water Usage.

First, open Azure Machine Learning home page and Click +NEW at the bottom of the window

• Select DATASET .
• Select FROM LOCAL FILE.

In the Upload a new dataset dialog, click Browse, and find the KSP-Irrigation-Data.csv file you created.

Now in the next step we will create an experiment in Machine Learning Studio that uses the dataset you uploaded. So, click +NEW at the bottom of the window and Select EXPERIMENT, and then select “Blank Experiment”.

Select the default experiment name at the top and rename it to Predictive Water Irrigation Experiment and in the module palette to the left of the experiment page, expand Saved Datasets. Find the dataset you created under My Datasets and drag it onto the canvas.

Now let’s prepare the data by using Select Columns in Dataset which is useful if you want to reduce the size of the dataset by deleting unwanted columns.

• Search and drag Select Columns in Dataset, and in the Properties pane to the right page, click Launch column selector and select the following columns:

In addition, we will be using Filter Based Feature Selection which will help us to identifies the column with the strongest predictive power in the input dataset.

• Search and drag Filter Based Feature Selection and in the Properties pane to the right page, click Launch column selector and select the following target column and feature scoring method:

Now, we’ll use our data for both training the model and testing it by splitting the data into separate training and testing datasets.

• Search and drag Split Data onto the canvas and connect to the last Select Columns in Dataset.
• Click Split Data and in the Properties pane to the right of the canvas and set it to 0.75. In this way, we will use 75% of the data to train the model and 25% of the data for testing.

In the next step we will be applying Two Class Decision Forest machine learning algorithm which is most suitable for our Predictive Water Irrigation Experiment.

• Expand the Machine Learning category in the module palette on the left side of the canvas
• Expand Initialize Model. This shows several types of modules that can be used to initialize machine learning algorithms.
• For this experiment, we are selecting the Two Class Decision Forest under the Classification and drag it to the experimental canvas.

So, in the next step we will find and drag the Train Model module to the experiment canvas.

• Connect the output of the Two Class Decision Forest module to the left input of the Train Model module
• Connect the training data output (left port) of the Split Data module to the right input of the Train Model module.

Click the Train Model module, click Launch column selector in the Properties pane, and then select the waterUsage column. waterUsage is the value that our model is going to predict.

Now let’s Run the experiment so, we can have a trained model that can be used to score new Irrigation data to make Water Usage predictions.

Now that we have trained the model with 75% of the data, we can use it to score the other 25% of the data to understand the function of our model.

• Search and drag the Score Model module to the experiment canvas.
• Connect the output of the Train Model module to the left input port of Score Model.
• Connect the test data output (right port) of the Split Data module to the right input port of Score Model.

Let’s Run the experiment and view the output of the Score Model module by clicking on the output port of the Score Model, then select Visualize. The output shows the predicted value of the price and the known value in the test data.

Now as we are on the final stage to test the quality of the results.  We will select the Evaluate Model module and drag it to the experimental canvas, then connect the output of the Score Model module to the left input of the Evaluate Model. The final experiment should look like this:

In the next step we will be Running our experiment to Visualize the output of our Evaluate Model.

In addition, we will be using Cross-Validate Model which is an important technique commonly used in machine learning to assess the variability of data sets and the reliability of any model trained using that data.

• Search and drag Cross-Validate Model and in the Properties pane to the right page, click Launch column selector and select the following label column:

Now Run the experiment to See the results for a description of the report.

I hope you found this blog post helpful. If you have any questions, please feel free to contact me muhammad.ahmad@kaispe.com

The post Agriculture Water Irrigation – Predictive Analytics Using Microsoft Azure ML appeared first on KAISPE.