Day 18

When I began today, I had all of my writing so far done, which meant I did not have anything on which to work while my classifier experiment code ran, so Ron introduced me to my next experiment, which is evaluating postprocessing methods. I started by documenting the already-written code for the postprocessors, which were a Markov Random Field (MRF) and a Conditional Random Field (CRF). It remains a challenge for me to document code written by others. I usually document code that I write when I am writing it so that I do not have to go back to it and try to figure out what it does. After I finished documenting the two postprocessors, I began writing the code for my postprocessor experiment, but I will likely have to alter it later because I do not yet know which classifier is the most accurate, because my classifier experiment is not over yet. For now I have the postprocessing code use the SVM-RBF classifier, which is the best one so far. Finally my trials on the MLP classifier with the MinMaxScaler preprocessor finished running. The MLP gave me a lot of problems, so I was glad to see that the results were finally reasonable. However, I still had to test the MLP with the StandardScaler preprocessor to see which way is better. I ran this test, and the results were nearly identical to the MinMaxScaler results. I will have to figure out which I should use tomorrow. It took all day for these two MLP tests to run, which is another reason the MLP classifier annoys me. Fortunately, more than one program can be run at a time in Spyder, the Python environment I use. (I found that out today.) Knowing that, I had begun to run parameter tests for the Random Forest classifier. First, I had to edit the Random Forest code, which was cool because I have just been documenting and testing the classifier code, not writing it until now. I added a way to choose whether the classifier refits the data, as it had just done it automatically before and I needed it to not refit the data. In the experiment, I had to figure out which preprocessor and which number of trees are the best for Random Forest. ("Trees" are a kind of decision tree used by Random Forest.) I had a lot of results for this by the end of the day, but I will decide if I need more tomorrow.

Comments

Post a Comment

Popular posts from this blog

Day 6

I began today by finishing the documentation I had been doing of the code for my experiment. After that, I began working on new code that would test the performance of data preprocessing methods that were already written. I am using the Pavia University scene, which is a set of labeled hyperspectral data. I began to write code to run the experiment, which will include testing various preprocessing methods each with thirty trials. In the process, I used the code from my first tutorial as a reference, and I found an error in that code that had been making it run too slowly. It had been using too much data to train. I fixed this issue, making that code run more quickly, and continued with the code for my experiment. Eventually, in my experiment code, I ran into a problem: I was shaping the data into different dimensions to split it into training and testing groups, but it had to be in its original dimensions to be used by the preprocessing methods. I am still working on this issue, and I ...
Read more

Day 14

Today I continued testing different parameters for the MLP classifier. While I ran tests, I finished documenting the classifiers that I am testing. I also wrote an introduction to my descriptions of the preprocessing methods, finished citations for the descriptions, and wrote a summary of my preprocessing experiment in an article being written in Overleaf about the machine learning pipeline. Although I am not yet ready to test the next classifier, RandomForest, I began writing code to test for the optimal preprocessing methods and parameters to use with it. Throughout the day, I collected data on the metrics of different combinations of parameters for MLP. I looked through this data and ran a larger number of trials on the most promising parameters, but it still appears that the default parameters are the most accurate, even though the overall accuracy resulting from the default parameters averages at about seventy percent, which is not very good compared to the other two classifiers I...
Read more

Day 3

Today I began by implementing the changes in the tutorial that Ron and I had previously discussed. I used arrays in my code instead of multilevel lists, because the machine learning program requires arrays. I then used two different machine learning programs, one linear and one non-linear, to process the training data and predict the second array of testing data based on the first. The linear learning program (a Linear Support Vector Machine) was more successful and now only takes about two minutes to train and make its predictions. I ran metrics to test the accuracy of the linear program, such as overall accuracy, mean-class accuracy, and kappa statistic, and it proved to be fairly accurate. Later in the tutorial, the accuracy will be improved with further measures. However, the non-linear program (a Support Vector Machine with Radial Basis Function kernel) is still too slow and inefficient. I am not certain why this is, but tomorrow I hope to fix this issue.
Read more