Classification

10. Two-Class Boosted Decision Tree - Build Bank Telemarketing Prediction

Hello and welcome. In the previous lectures, we saw what a decision tree is and what ensemble learning is, along with the two most commonly used ensemble methods, which are boosting and bagging. Today we are going to build a model based on two class boosted decision trees. But before we do that, let's first try to understand the business problem that we are going to solve today. The data here is related to direct marketing campaigns of a Portuguese banking institution. The marketing campaigns were based on phone calls, and often more than one contact with the same client was required. This was done to find out if the product would be subscribed to or not. The classification goal is to predict if a client will subscribe to a term deposit.

So this is a supervised learning problem and it is of type two class classification. There are various features of this data set, and we will go through the details when we visualize the data set in Azaria. You can also find the data set on the website of UCI, and it is publicly available. You can search for it as "bank marketing data set office" and it should provide you with among the first three links. In the previous lecture, we saw what is Boosting is and how it ensembles the results. Boosted decision trees are based on this boosting method of ensemble learning. That means the next tree corrects for the errors of the previous tree, and predictions are based on the entire ensemble of trees. This is among the easiest methods to get a top performance. However, as it builds hundreds of trees, it is very very memory intensive. In fact, the current implementation holds everything in memory, so it may not be suitable for very large data sets. All right, let's go to the Azure ML studio and build our first boosted decision tree model. Here I am and I have already uploaded the data set as well as dragged and dropped it onto the canvas.

So let's visualise the data set. Let's try to understand all these variables one by one. Sometimes it's worth spending some time analysing and visualising the data, as many patterns as well as the quality of the data can be seen. The first column we have is the age of the customer. It is a numeric feature and does not have any missing values. great. The second feature is job, which is a stringfeature and has twelve unique values, such as management technician, blue collar, and so on. It also does not have any missing values, and that does help us. The next one is the Marital Column, and it is also a string-based feature with four distinct levels of education with eight unique values. Column default indicates if the credit is in default or not. As a result, it contains only yes values, no unknown values, and so on. You can go through every column or you can also get the bank names text file from the UCI website to understand the columns and their values. Alright, let me close this first, and because we do not have any missing values, we are going to straight away jump to the split module. Let me search for it and drag and drop it here. Provide the connections and let's do a 70/30 split. So our split fraction will be 0.7. Let the random seed be one, two, or three, and let's do the stratified split on the column y. So let's launch the column selector, select column y, and click okay. As the outcome here is binary, that is yes or no. We will select a two-class model. So let's now apply the two-class boosteddecision tree model to this data set. So let me search for two class-boosted decision trees. There it is and I am going to drag and drop it onto the canvas. Let's look at the various parameters it accepts. I hope you recall the concepts from our first lecture on what a decision tree is. If you remember them, then understanding these parameters will not be difficult at all. As with previous models, it asks for trainer mode and we are going to continue with single parameter mode. Next is the maximum number of leaves we want per tree.

As you know, leaves, or terminal nodes, are the nodes that we cannot or do not want to split further. By increasing this value, you potentially increase the size of the tree and get better precision. The better precision here comes at the risk of overfitting and longer training time. The minimum number of samples per leaf node indicates the number of cases or observations required to create any terminal node or leaf in a tree. By increasing this value, you raise the threshold for creating new rules. For example, with the value of y, the training data will have to contain at least five cases that meet the same conditions before we can split it further. All right. The learning rate determines how fast or slowly the learner converges on the optimal solution. If the step size is too big,you might overshoot the optimal solution. And if the step size is too small, training may take longer to arrive at the best solution. The number of trees constructed indicates the total number of decision trees to be created. By creating more decision trees, you can potentially get better coverage, but the training time will increase. We already know what the random number seed is, and we specify one, two, or three there. Then we check to allow unknown categorical values.

Our decision tree model has now been set. Let's first train this model using the Train model module. All right. Let's also score this model on the data set, making sure that at every step we have all the right connections. Let's also add the evaluation model so that we have everything ready. There is our evaluation model, and we are ready to run it with the right connections made. Believe me, it may take some time depending on which region you are running it in. I assume you have followed me and are now ready to run your first two class boosted decision trees. So get yourself a cup of coffee and also pause the video while it runs.

All right, all the steps have been successfully completed, and let's now visualize the are now read congratulations. Because we have just achieved an AUC of more than 0.9% and our accuracy is also more than 90%. You should know these terms by now. If you are still not clear about how to interpret these results, I suggest you go through the class on understanding the results, where we have explained this in great detail. That concludes our session on two class boosted decision trees. In this class, we used the bank's telemarketing data and predicted it with very high accuracy. And what did we predict? We predicted whether the prospect would pay the term deposit or not. In the next class, we will cover the two class decision forests and try to predict the same outcome as in this class. So I'll see you in the next class and enjoy your time.

11. Decision Forest - Parameters Explained

Hi, The next lab we will do is on decision forest. But before we start working on the experiments, let's try to understand the various parameters required for this module. The Create Trainer mode, number of decision trees, minimum number of samples per leaf node, as well as unknown categories are exactly the same as we have seen in the lectures on logistic regression parameters and boosted decision tree parameters. Let's see, what do we mean by the resampling method?

There are two options for Bagging and Replicating. Over here, we have seen what is bagging? During the lecture on ensemble learning, It simply creates random samples of the data set and creates and trains different trees. Finally, all the trees vote to give a final prediction. whereas in the case of a replicate, it creates only one random sample of the data set. It then trains all the different trees using this sample. The rest of the process remains exactly the same. You may ask, why do we need this? Well, if the original data set is huge, it may take longer training time using the bagging method. Replicate usually requires less time than bagging because we are working only on one sample set. However, if we do not have a large number of records, we will prefer bagging as the resampling method. Okay, I hope that explains why the two resampling methods and what they are next is the maximum depth of the decision tree. Well, it's nothing, but how far down can you go before the decision tree buildup stops? So, in this example, the depth of the decision tree is three.

As we split them one after another, I hope it's clear what we mean by depth. It's pretty straightforward. Lastly, let's see what we mean by the number of random splits per node. Let's say we have this data set of marks studied versus the grid obtained. Now, the biggest dilemma we will have while constructing the decision tree or splitting the nodeis how do we split this? Should I split it on the number of hours studied? more than 30, 40, or 50? Well, there are various methods which are used to get the best split on such nodes. However, before it can determine the best split, it must divide it based on the different values of a feature. This parameter simply specifies what should be the maximum number of splits within a node before we select an optimal split value. I hope that clarifies what the various parameters required for decision forest are. So, see you in the next lecture, where we'll build a decision forest model. Until then, enjoy your time.

12. Two Class Decision Forest - Adult Census Income Prediction

Hello and welcome. Today we are going to cover one of the most interesting and very popular models called decision forest. Decision forests are also called random forests. They are based on ensemble learning methods, and as we have seen in the previous lecture, in the case of bagging, the algorithm works by building multiple decision trees and then voting on the most popular output class. When we test the models, we will get different outputs and, depending upon the probability of outcome, we may assign different weights to each of these models. Then voting is performed to come up with the most popular output class. All right, let's try to solve a business problem using decision forest. The problem at hand is to predict whether a particular person earns more than 50,000 per year or not. This is going to use one of the sample data sets named Adult Sensors Data Set, and the data has been extracted from the 1994 Sensors database.

A set of reasonably clean records were extracted. So let's go to the Azure Studio and visualize the data set. There is our data set. Let's try to visualize this data set. As you can see, it has more than 32,000 records and 15 columns or features. Let's go through them one by one. The first column is the age of the individual. It's a numeric feature and has got no missing values. A work class is a type of employment such as private, self-employed, state or local government, or federal government without pay and so on. It has eight unique values and 1836 missing values. So we have got some work to do here. SNL wet is nothing but the number of people the census takers believe that observation represents. Those may not have any significant impact, and we are going to assume that these records are individual and hence will ignore this column. Education is the highest level of education of the individual. The education number basically represents the numerical form of the highest level of education.

It's the same as the education column, and we can consider one of the two as they represent the same data in two different formats. "Marital" is the marital status of the individual. The Occupation column is the occupation of the individual and has some missing values. Relationship denotes the family relationship that this person represents during the census. The Race column is the description of the individual's race, such as Asian, white, or black. Column six is the gender of the individual. Capital gain basically means whether the capital gains were recorded, if any, and capital loss means the capital losses which were recorded for that individual. Hours worked per week is the number of hours worked per week by the individual. That person's native country is nothing but their country of origin, and also has some missing values. Finally, the column income represents the value of whether or not the person makes more than $50,000 per annum. It has only two categorical values: more than 50,000 and less than or equal to 20,000. All right, let's first replace the missing values from our data set. Let me close this and we will come back to our Azure ML Studio.

Let's search for the missing data module and drag and drop it here. Connect the adult sensors data to this module and let's select the columns that have missing values. So let's launch the column selector. All right, we have missing values for work class, occupation, and native country. So we select them and click OK. I'm going to keep the minimum and maximum missing value ratio. sass it is, and let's replace them with more options. We are ready to run this module. It has run successfully, and let's visualize the dataset and check the columns with missing values.

All right, we have successfully replaced the missing values with mode. However, there are a few more things we need to do before we can apply the model. As you know, a couple of columns such as F and Lag and Education in the numeric form are of no use to us. So we are going to drop them from the experiment and select only the remaining ones. Let me close this for now. What is the name of the module that allows us to select the columns of interest? Well, the name is there in the sentence I mentioned. It's select columns from the data set module. So let's search for it and drag and drop it here. Connect the previous output to this module and let's launch the column selector. We are going to select all the columns except these two and click okay. You can run it if you want. I am going to proceed with the rest of the experiment, and we will run it all at once. As we have seen, all the nonnumeric columns of this data set are strings.

However, all of them represent only categorical values and should be changed to the same. So let's get the Edit metadata module and change the variables to the categorical one. I am going to search for it and drag and drop it here. Let's make the right connections. All right, and let's now launch the the column selector and let's select the columns work class, education, marital status, occupation, relationship, race, sex, and native country along with the predicted column, which is income. Let me click on okay and let's run this.

All right, it has run successfully along with the previous modules, and let's visualize the output. As you can see, all the columns we processed are now categorical features. All right, let me close this okay, and let's split the data for training and testing purposes. And let me get the split data module. Let me search for it and drag and drop it here. Make the right connections. Let's select the split ratio as 70/30,so the split fraction will be zero and the random seed will be one, two, or three. And we also do a stratified split on column income. So let's launch the column selector and select the column name. Click okay. We are now ready to run it.

All right, it has been successfully and let's apply the two-class decision forest module. Let me search for it and drag and drop it here. Alright, let's see the parameters it requires. It has got two replication methods: bagging and replicating. In the Bagging method, each tree is grown on a new data set created by randomly sampling the original data set with replacement until you have a data set as the size of the original and the output of the models are combined by voting, which is a form of aggregation. In the case of replicate, each tree is based on exactly the same input data. The determination of which split predicate is used for each tree node remains random, and the trees will be of diverse nature. We have seen the trainer mode, so we will remain with the single trainer mode until we have seen the tuned model hyper parameters.

The next parameter is the number of decision trees. We want to build the maximum depth of the decision tree, which is nothing but how many levels below the root node. Increasing the depth of the tree might increase precision, but at the risk of some overfitting and increased training time, the number of random splits per node is nothing but the number of splits we want when building each node of the tree. A split here means that features in each level of the tree are randomly divided into a minimum number of samples per node. We specify and keep it as is. For now, let's keep them at the default values for now. And we need to train this model on the training dataset. So let's get the train model and drag and drop it here. Make the required connections from the untrained model to this node and the training data set to this one. Let's launch the column selector and select column income and click OK.

Well, the rest of the steps are exactly the same. That is, get the score model and evaluate the model. So there we have the score model going to drag and drop it here and there is our evaluating model. Let's make the right connections from this data set to the score model and then to the evaluation model. And now we are ready to run the model. So let's go and run it, selected it successfully, and let's visualize the output. Wow, the AUC Roc curve looks very good. Let me go down and check the various ratios. Well, an accuracy of 0–85 and an AUC above 0–9 means our model is a very good model. We can still improve this result and can also reduce the training time by doing some data processing. We will do that in one such lecture. If you have followed and done this along with me, I must say congratulations on running your two class decision forests successfully with very good accuracy and AUC. That brings us to the end of this lecture, where we learnt about the two-class decision forest using the adult sensors data and successfully predicted with very high accuracy the income level for a particular observation. With this, we conclude the lecture on the two class decision forests, and in the next lecture, let's build multiclass boosted decision trees and multiclass decision forests. Thank you so much for participating in this, and I hope to see you in the next one.

13. Decision Tree - Multi Class Decision Forest IRIS Data

Hello and welcome to the course on Azure Machine Learning. So far, we have seen what a decision tree is and how a decision tree is constructed. We have also seen what is called ensemble learning. Along with boosting and bagging, we have also constructed two class boosted decision trees and two class decision forests. Today we are going to COVID multi-class decision forest using Irish data, which remains one of the most popular data sets on UCI. This is perhaps the best known database to be found in the pattern recognition literature. It was introduced by Ronald Fisher and Fisher's paper is a classic in the field and is referenced frequently till today. The dataset contains three classes of 50 instances each, where each class refers to a type of Irish plant. One class is linearly separable from the other two. The latter are not linearly separable from each other. It can be downloaded from the UCI website, or alternatively, we can simply import it using our import module. Alright, let's go and build our multiclass-decision forest model using the Iris data.

So let's use the import data module and drag and drop it here. Let's now go to the UCI website where the data has been stored. All right, click on the data folder and then on the Irish data. Let's copy this link and come back to the studio. Okay, so in the data source, let's change it to web URL via Http and paste this link in the data source URL. Our data format is CSV and it does not have a header. Let's also check the Use cached results so that it does not fish it from the source all the time, and let's run it. Alright, our import has run successfully, so let's visualize the data set. But before we do that, because we know there are no headers, let's use the Edit metadata to add some column names so that it makes sense to view and analyse the data if needed. So let's find the edit metadata, drag and drop it onto the canvas, and make the right connection. And let's simply change the column names. So we launch the column selector, select all the columns and click OK.

And let's change the names here. They are the sample length, the sample width, the petal length, and the petal width, all in centimeters. And the final one is the class of the Iris plant. All right, and we are ready to run it now. It has finished running and let's go and visualize the data set. It looks great, except that there is one missing value in the column class. It's probably the last row with all missing values as it is supposed to have only 150 rows as it is a predicted column. Let's remove that rule by using the Clean missing data module. So let's drag and drop it here, make the connection, and launch the column selector. Let's use the column class and click OK.

We are going to now choose the option to remove the entire row. The next task is to split the data for training and testing purposes. So we search for split, drag, and drop the module here, making the right connections. And let's also specify our split ratio at 80%. We want to do a randomized split, so we keep this checkbox as it is and the random seed as one, two, or three. And let's make it a stratified split on column class. So let's launch the column selector and select the class. Let's press OK. And our next task is to choose our model, train it, and score it before we evaluate it. All right, let's do that now.

So we are going to apply multiclass decision forest for this as we have three classes to be predicted and parameters are not very different than the two-class decision forest. So we leave them as default and you can try changing those values and compare the results. Let's now bring in the train module and drag and drop it here. Launch the column selector, select class, clicklock, and let's get the score model. Okay, connect it to the train model and also to our test data set, which is coming from the second node from the split module. One last thing we need to do is evaluate the performance. So we bring in the evaluation model and connect it to the score model.

All right, we are now ready to run our multi-class decision forest model on one of the most popular data sets in the UCI library. So let's hit the ground and run intuit should be fairly quick as the data set is very small. All right, it has run successfully and let's go and visualize the output. Wow, that's like a dream. We have successfully classified all the results into the correct class. Irish data does provide high accuracy anyway, and with an advanced algorithm like decision forest, it is bound to give such high accuracy. All right, that brings us to the end of this lecture on multicolor decision forest. I hope you have enjoyed building the multiclass decision forest with me and must be happy to see a result of 100% accuracy. Thank you so much for joining me in this class, and I'll see you in the next one. And bye then. Have a great time.

Study with ExamSnap to prepare for Microsoft Certified: Azure Data Scientist Associate Practice Test Questions and Answers, Study Guide, and a comprehensive Video Training Course. Powered by the popular VCE format, Microsoft Certified: Azure Data Scientist Associate Certification Exam Dumps compiled by the industry experts to make sure that you get verified answers. Our Product team ensures that our exams provide Microsoft Certified: Azure Data Scientist Associate Practice Test Questions & Exam Dumps that are up-to-date.