RIT CIS Internship

Today I continued tests for my classifier experiment. I worked on finding the optimal parameters for the Multi-Layer Perceptron (MLP) classifier. I was trying to determine the best hidden layer shape and weight decay. Hidden layer shape is how many dimensions (layers) of neurons (nodes used for machine learning) there are, and how many neurons are in each dimension. I only began with the knowledge that there would be a maximum of three layers in the hidden layer shape. Weight decay is a value that decreases the weight values after each training iteration to prevent overfitting to the training data. I knew that the best value for weight decay would be 1e-4, 5e-4, or 0. I wrote a program to test different combinations of hidden layer shapes and weight decay values. While these tests were running, I documented some of the classifiers with which I was working, including the MLP classifier. This process helped me to better understand MLP and the other classifiers because it made me work thr

Today, while I finished running my tests on PCA without whitening, I wrote my outline for the presentation on the research I am doing. In the outline, I divided my presentation into two sections: the first is Reason/Explanation and the second is Methods and Results. I went into much detail describing the reasoning and background behind my project, while I was more general about the methods and results of each of my experiments. This is because I have only finished one experiment so far, so I have limited results to share. (However, I included the table of results from my first experiment, evaluating preprocessors, in the outline.) After I finished writing my outline, I wrote code to produce line graphs of the data from the PCA tests, which were finally done running. The graph shows mean-class accuracy in relation to the number of components. Unfortunately, the graph did not show any obvious pattern, but it did show that PCA with whitening is significantly different than PCA with whiten

This is the outline for the presentation I will be giving on my research project: Presentation Outline Reason/Explanation: Explain Project: My project is to research different parts/steps of a machine learning pipeline. Explain Pipeline: The pipeline is a framework that uses previously classified hyperspectral imaging (HSI) data to train itself to classify other HSI data.   Explain HSI: Hyperspectral imaging is collecting image data on objects (and in this case landscapes) that is not limited to visible light, detecting a broader range of the electromagnetic spectrum including ultraviolet and infrared light. Explain Labels/Classification: Once HSI data is collected, different portions of the data must be classified. In a hyperspectral image of a landscape, each pixel must be labeled as a part of what it makes up. For example, if a pixel is part of a car or a tree in the image, it must have the car or tree label. Labels are known as classes. Give Reason for Project: It

This post is about yesterday (7/27/2017). I am writing it the day after because we returned from the field trip late last night. Yesterday I continued running the trials on PCA with different numbers of components, because I had not been able to run it over the previous night. When I got to the lab, Ron helped me set up the software so that I would be able to run programs overnight in the future. At the lab, I finished the trials on PCA with whitening, and I began the trials on PCA without whitening. Meanwhile, I continued working on descriptions of each preprocessing method and created a chart with the results from my research on each method. Since I was nearing the completion of my preprocessing experiments, Ron introduced me to my next experiment: I would be doing research on each of five classifiers, two of which (Linear SVM and SVM-RBF) I was already familiar with. Yesterday we (the Imaging Science interns) also went on a field trip. We went by bus to the Mees Observatory t

Today I finished running trials on the remaining two preprocessing methods. I recorded the means and standard deviations of the three metrics of each preprocessing method in a chart. Also, for one of the methods, called "PCA" or Principal Component Analysis, Ron had me begin a separate experiment: I was to find the mean-class accuracy when using PCA with different numbers of components. I would do this first using PCA with whitening, which is a linear data transformation, and then without whitening, and compare the two groups of results. I added to my code so that it could run thirty trials with each number of components I was to test. I had to make adjustments in my code for PCA because it is used differently than the other preprocessing methods I am testing. I began to run the trials on PCA with each number of components (of which I am testing eleven). I ran trials and recorded the results from using PCA with whitening with nine different numbers of components today. I plan

Today I began by completing the change to my code that I had begun the previous day, allowing me to finally run a trial on the first preprocessing method I was to test, "MinMaxScaler." The metrics looked strange at first, and it turned out that while the mean-class accuracy of the first trial was fairly good, the overall accuracy was too low. I talked to Ron about this issue, and after much searching for an error, we found that there was a problem with shuffling the data. After that was solved, I began to successfully run trials on the preprocessing methods. Today I ran thirty trials on each of five out of the seven preprocessing methods which I am testing. I plan to run trials on the remaining two methods tomorrow. Each group of thirty trials took about thirty minutes, so meanwhile I worked on creating a diagram of the machine learning pipeline and writing a description of what each preprocessing method I am testing does. I will continue these descriptions and perform the re

Today was my first day back in the lab after being out of town for a week. I began the day by refreshing my memory on what I had been doing before I left: I had been working on creating a training array. I finished that part relatively quickly because I was already almost done with it. I then used my index array to create an array of training data, which was the data left over from training. The training array only includes fifty samples from each of nine classes, so the testing data array is much larger than it. Once I had my arrays set up, I passed them to the machine learning pipeline so as to later be able to test each preprocessing method. I will test these methods by finding and recording the overall accuracy, mean-class accuracy, and kappa statistic of each method, similar to what I did during the tutorial. I have already set up the code to run these metrics. I am almost ready to run the trials on each preprocessing method, but Ron and I discussed a change I have to make in my p

Today, I started by writing my abstract, which is a short summary of the project on which I am working this summer. (The abstract is included in my last post.) To write my abstract, I needed to develop a clear idea of my project. I found that summarizing it in writing actually helped me to better understand what I am doing as a whole. After completing and posting my abstract, I returned to my experiment. I first set out to solve the problem of incorrect dimensions in my data, which I had run into on the previous day. I took out the code that reshaped the data, because the data had to be in their original dimensions for the preprocessing methods to function correctly. However, this meant that I had to rewrite the code that split the data by labels into training and testing categories. I had to use an entirely different strategy than I had been using before. Instead of directly splitting the data into two arrays, I am now creating an array of the indices of the data values in the origina

This is my project abstract, which is a brief description of what I will be doing during my internship at the Center for Imaging Science: HSI Classification Research Project In my Visual Perception and Machine Vision research project, I will be evaluating various aspects of a machine learning pipeline. This pipeline is a framework designed to process hyperspectral imaging (HSI) data, using a limited amount of labeled data to train itself to classify larger amounts of data. The purpose of this is to allow machines, with minimal information, to be able to recognize and map out hyperspectral images, in this case of a landscape. I will be testing numerous parts of the pipeline such as data preprocessing, feature extraction, and postprocessing, researching which methods of each are most effective.

I began today by finishing the documentation I had been doing of the code for my experiment. After that, I began working on new code that would test the performance of data preprocessing methods that were already written. I am using the Pavia University scene, which is a set of labeled hyperspectral data. I began to write code to run the experiment, which will include testing various preprocessing methods each with thirty trials. In the process, I used the code from my first tutorial as a reference, and I found an error in that code that had been making it run too slowly. It had been using too much data to train. I fixed this issue, making that code run more quickly, and continued with the code for my experiment. Eventually, in my experiment code, I ran into a problem: I was shaping the data into different dimensions to split it into training and testing groups, but it had to be in its original dimensions to be used by the preprocessing methods. I am still working on this issue, and I

I began the first experiment of my project today. But first, I prepared by doing some reading on the topic of hyperspectral image classification. I read a paper written by Ron and Dr. Kanan about self-taught feature learning. It discussed two self-taught learning frameworks: the multiscale ICA and the stacked convolutional autoencoder. Much of it was difficult for me to understand because I am new to this topic, but I learned more about self-taught machine learning programs. After I finished reading, I began my first experiment. The first step was to document already written machine learning code and to figure out what that code was doing. I read about the standard Python documentation style, then I began to document the code. This proved to be challenging, because this code implemented many methods from outside of itself, which meant I had to understand those methods as well. Today I learned that understanding and following code that was written by someone else can be just as challeng

I finished both of the machine learning tutorials today. In the morning, I continued work on the first tutorial. I sped up the non-linear Support Vector Machine (SVM) that had been a problem the previous day by running the data through it without performing parameter searches, as I had been attempting to do before. This made the process faster because in order to do parameter searches, the program had to test multiple parameters using the SVM. However, I was able to perform both grid parameter searches and randomized parameter searches on the linear SVM, which is significantly faster than the non-linear SVM. (I am now also able to perform a search using the non-linear SVM, but it takes about half an hour.) I then used MinMaxScaler and StandardScalar to transform the data, making the predictions of the SVMs more accurate. After that, I applied a Principal Component Analysis (PCA) and an Independent Component Analysis (ICA) to the data. These transform the data into lower dimensions to i

Today I began by implementing the changes in the tutorial that Ron and I had previously discussed. I used arrays in my code instead of multilevel lists, because the machine learning program requires arrays. I then used two different machine learning programs, one linear and one non-linear, to process the training data and predict the second array of testing data based on the first. The linear learning program (a Linear Support Vector Machine) was more successful and now only takes about two minutes to train and make its predictions. I ran metrics to test the accuracy of the linear program, such as overall accuracy, mean-class accuracy, and kappa statistic, and it proved to be fairly accurate. Later in the tutorial, the accuracy will be improved with further measures. However, the non-linear program (a Support Vector Machine with Radial Basis Function kernel) is still too slow and inefficient. I am not certain why this is, but tomorrow I hope to fix this issue.

Today I continued work on the tutorial on machine learning basics. In the tutorial, I am attempting to shape data from digital images by altering the number of array dimensions into which the data are organized, and divide the shaped data into training and testing groups. The machine learning programs will use the training data to predict the results of the testing data. So far, this process has been gradual and challenging for me, as I have limited prior experience in machine learning. Ryan, who is working on the same tutorial, has more background in machine learning than I do, and he has assisted me along the way. Also, toward the end of my work today, I encountered a flaw in my programming: in order to be compatible with the machine learning programs we are using in the tutorial, the data must be organized using a different type of structure than I had been using. I discussed this issue with Ron, and he helped me through it. Next time, I plan to implement the changes that we conside

Today was the first day of the internship program. We (all of the interns) met as a group and discussed how the summer was going to go. Then we went to a barn and did team-building exercises, such as trying to figure out how to flip over a carpet while standing on it and cross the floor without touching it. These challenges were a little strange but they helped me improve my ability to work as a team. We then had lunch and proceeded to our smaller research groups. Mine is Visual Perception and Machine Vision. Ryan and I met with Dr. Kanan and Ron, a PhD student, both of whom will be assisting us in our research. We discussed research, college, and the program in which we will be participating. We were introduced to our research projects and we signed onto the server. Tomorrow I will be working on the first tutorial that will introduce me to the programs I will be using.