When Jason Lin ’21 joined the Geospatial Evaluation and Observation Lab (geoLab) in the fall of 2019, the newly-created roadRunner project consisted of an uncompleted app and an idea to evaluate road quality from satellite imagery. Nearly a year later, the roadRunner project has transformed into one of the lab’s most promising ventures with over thirty college students across the state of Virginia competing to create the best convolution neural network (CNN) model.
The project began last year with Professor Peter Kemper and Eric Nubbe ’20, who were working to create an app that automatically evaluates road quality based on the smart phone’s built-in accelerometer. The pair began collaborating with the geoLab in hopes of it aiding with data collection. The project was passed to the geoDev team–one of four core teams in the lab–which began developing a CNN model that could come to rely solely on satellite imagery. The app was completed in the spring of 2020.
Lin, along with helping on the app, started working on building a CNN last spring, which uses artificial intelligence to analyze visual imagery. He believes that this technology could help indicate road quality in nations that struggle to routinely survey the roads.
“When we train this convolution neural network, the app is sort of helpful on its own, but having the neural network and being able to analyze real quality from satellite imagery…the implications of that are huge,” Lin said. “Within the US, we can definitely afford for people to evaluate roads. But in areas where it’s not feasible, like Honduras, the roads don’t really get analyzed that much. We don’t know which roads are good. But, you can point the satellite at it and say ‘what’s the road quality look like here?’ And [the CNN] will automatically determine: this road is terrible, I need to repair it.”
Currently overseeing the project with aid from project leader Lin is Ph.D. student Ethan Brewer, who is studying applied sciences at William and Mary. Brewer joined the roadRunner Project in June 2020, when he switched Ph.D. advisors to Dr. Dan Runfola (who runs the geoLab) after taking interest in Runfola’s work.
“[Runfola] handles the money side of things, and he has kind of an overall vision,” Brewer said. “He teaches me some things in that way. But in the end, I’m managing the whole project. I’m leading the project. I’m giving the team leaders directions….I sort of have a personal stake in it because I’m writing a chapter of my dissertation on the results of this project.”
Over the summer, the roadRunner Project received funding from the Commonwealth Cyber Initiative, a Virginia program that looks to support world-class research in data and cyber-physical systems. Through this funding, the project will sponsor 15 William and Mary students and 15 non-William and Mary students to come together in a year-long competition to develop a CNN model.
There will be six teams made up of students from different Virginian universities: William and Mary, Christopher Newport University, Virginia Tech, Virginia Commonwealth University, George Mason University, and the University of Richmond. Each team is made up of five members and has a William and Mary student as its leader.
Lin will serve as the team leader for the University of Richmond, while Brewer will be managing and overseeing the entire competition.
“We’ve sort of identified people from William and Mary who really have good, strong backgrounds in machine learning, data science, programming, and they are going to be the team leads,” Lin said. “So with these six teams, we’re going to be competing in a cattle competition. Our main focus is going to be improving the accuracy of the neural net when we look at satellite imagery”
The William and Mary students will receive a $1000 grant, and the non-William and Mary students will receive a $3000 grant. The $2000 discrepancy was supposed to cover travel costs, but due to the COVID-19 pandemic, the project will be conducted remotely for the fall semester.
“We have a total of basically 30 of the best computer science students in the state,” Brewer said. “I like working with the teams and seeing them evolve. I like teaching them. I like seeing their progress. I like seeing them learn.”
For the fall semester, the six teams will work to collect as much data as possible and integrate that information into a CNN model. Then, in the spring semester, each team will attempt to feed the other teams’ models bad data in the “data poisoning” phase.
“Data poisoning is where one team tries to feed bad data to the other teams’ models to see what effect it has on it,” Lin said. “Basically, one team tries to break the other teams’ models, and we want to make our own models resilient. One team is defending; one team is attacking. I want to make my model resilient against whatever you throw at it. I want to make it robust.”
The team that creates the overall best model by the end of the spring semester will be offered a guaranteed internship from the William and Mary Initiative for Computational Societal and Security Research (ICSS) for Summer 2021. Any student that makes an above-and-beyond contribution will be featured as a co-author of any relevant academic publication regarding the project.
“From the technical side of things, I mean, building the models is interesting, but I’m addressing the overall sort of can we do this type of thing?” Brewer said. “Because there are a lot of moving parts, and there are a lot of places where things can go wrong or where they can become difficult. So I’m sort of eager to see how things just work generally, whether the techniques that we’re using will actually lead to some accurate results.”
