Rebooting the Harvard Computer Society

William Xiao ’16 just needed to send a quick email before settling into a problem set around 10 p.m. one Monday in early September.

But as he sat in his Eliot House dorm room, keeping an eye on his inbox, the email did not come through. Xiao, who also serves as president of the Harvard Computer Society—the group that manages more than 7,000 student mailing lists—quickly realized his night was not going to turn out as planned.

As Xiao began to receive text and Facebook messages from friends concerned about why their emails were not sending, HCS board member Roger Zou ’17 trekked to 1414 Mass. Ave, where the society keeps its physical servers, tucked above Bank of America and CVS. He aimed to restart Virgil, the filer that stores all the data for HCS-hosted lists. At that point Virgil had been down for several hours, quieting hundreds if not thousands of active lists and websites on campus.

When Zou reached 1414 Mass. Ave, which primarily houses Harvard University Information Technology servers, he faced a series of “mission impossible” obstacles: a locked entrance, a terminal computer that needed a passcode, and a mesh cage that housed the servers themselves. Receiving instructions via a video call with Xiao, Xiao’s predecessor Saagar Deshpande ’14, and Deshpande’s predecessor, Zou identified the correct metal gray machine, and together they restarted the filer around 1 a.m..

The Harvard Computer Society maintains servers in a facility located above Bank of America on 1414 Mass Ave. These servers used to be located at the group's headquarters, located at the SOCH. By Christopher J Magnani

“When the filer is running, fans are going off, and you can hear this constant buzzing,” Zou said. “Everything, everything went silent for two seconds. And then it started running up again, and a lot of lights began flashing.”

As the incident, the first in recent memory that blacked out of all of HCS’s services, suggests, the stakes for Harvard Computer Society are high. Run by five College students, the society is the only undergraduate-driven organization in the Ivy League that serves as the predominant web hosting and emailing list provider for other student groups. For Harvard students, HCS’s services are a cornerstone of daily communication.

As more clubs demand an online presence, HCS members say that its now-antiquated infrastructure can hardly support those requests—leading to a series of headaches like the early September blackout. With those headaches in mind, the more than 30-year-old club is finalizing a transition from its on-campus servers to the cloud and readying for the next phase of its service.

A MASSIVE REACH

Although most widely known for the algorithmic “DataMatch”—a personalized list of romantic potentials distributed on Valentine’s Day—Harvard Computer Society impacts almost every undergraduate’s life on a daily basis.

Among the thousands of mailing lists under HCS’s helm, including “quadlings,” “ghungroo-production-14,” “mather-official,” “culinary-discuss,” and “cabot-thesis,” lies the means of reaching those belonging to a certain theater production, academic field, House, or even those with a penchant for Japanese anime. The club also hosts at least dozens of active club websites.

“Most people don’t even know that HCS runs a lot of this in the background,” Deshpande, last year’s HCS president, said. “They assume, ‘Oh, there’s a House mailing list, and Harvard runs it,’ but they don’t connect the dots that actually a group of students are doing a lot of it.”

By Christopher J Magnani

According to Deshpande, at its founding in 1983, HCS intended to teach its peers about early technology, publishing a magazine that would review the latest gadgets. That changed about 15 years ago, when the group transitioned from providing students a venue for discussion to providing actual services on club-owned servers.

While other means of web hosting have emerged over the years, HCS offers one of the only ways for a Harvard student group to host a website free of charge and avoid third-party fees that can approach $100 per year.

Zachary Hamed ’14, a former HCS member, delved into the archives of “HCS-jobs”—a mailing list for technical recruiters to advertise employment opportunities and students to find partners for side projects—and said that the oldest emails he could find were from 2000, around the year group members estimate was the starting point of HCS’s mailing list administration.

Today, HCS’s database holds more than 160,000 unique email addresses, though that count includes addresses for graduated students and fake or incorrect addresses that have not been pruned out, according to Xiao.

“We touch everything and have a massive reach at Harvard,” Xiao said.

According to James H. Waldo, Harvard’s chief technology officer and HCS’s faculty advisor, HUIT has no “official” relationship with the club, but does house its servers and pays for the “minimal” cost of providing electricity to the machines. As an advisor, Waldo said he connects the undergraduates with faculty members who have appropriate expertise when the group seeks guidance, but has no role in how they manage or administer the club.

“They are a non-professional organization doing a professional job,” Waldo said. He added that in his personal view, he appreciates that a student organization could put up its website without University oversight or bureaucracy.

While the scale of the operation is a constant challenge, board members say that the experience of administering student lists and websites mimics a professional setting that cannot be found elsewhere on campus.

“HUIT has given us a fantastic opportunity. There would be no other way for students to interact with such a wide-ranging systems base except through HCS,” Zou said. “I have really appreciated the learning experience.”

Deshpande echoed Zou’s sentiment, saying that his time working with the student organization was “an experience pretty difficult to get outside of working at a company.”

NEAR CAPACITY

As the number of student lists has proliferated over the years and email volume has increased, HCS members say their system’s current hardware and software iteration has been under increasing stress. That has translated into more issues for the members of the five-man management board to address.

For example, Xiao said that the help request mechanism for users to notify the group of a malfunctioning service or for assistance on a technical problem has an estimated three-month backlog.

“We are getting near the capacity of what we can handle in terms of clubs,” Xiao said. “I could imagine one to two full-time people just working on this, or another full-time person whose main job is to manage servers for Harvard’s clubs.”

Part of the problem is that the club is still running six-year-old software, which likely contributed to Virgil’s failure, according to Deshpande.

“It’s been increasingly difficult to maintain it especially since it’s getting more costly to provide replacement parts,” Deshpande said. “It’s like buying an old car. People also no longer send out patches to fix bugs in the software.”

In the meantime, the aspiring engineers have found ways to mitigate the capacity problem until they can completely rebuild the database from the ground up. According to Xiao, the current database reaches 95 percent of capacity at the end of each day, so the club runs a set of instructions which delete junk data. This high volume puts the database at risk of crashing if hackers clog one of its hosted sites.

“Every once in a while, it doesn’t clear in time before it hits that 100 percent, and everything crashes,” Xiao said. “Things are going to break, and that’s something we accept as a student organization. There’s not much we can do.”

TRANSITIONING TO THE CLOUD

This convergence of database and hardware problems led HCS leaders last spring to conclude the organization would be best served by transferring all its data from physical servers on campus to the cloud.

The group expects to transfer all services over to Amazon Web Services, a cloud computing platform, by the end of October. Former HCS presidents Deshpande and Salvatore Rinchiera ’14 spearheaded the project.

“The big push is for us to get to a brand new set of systems and update all of the software and hardware. For a lot of work moving forward in the next 10 years, people are very heavily going to be using cloud,” Deshpande said. “It’s a necessity, but it’s also the most practical thing to do.”

Waldo, for his part, said that he particularly aimed to push the club away from its current machinery, which he called “not so current.” According to Waldo, HUIT will continue to fund HCS by paying for its cloud services, but he expected the cost to be the same if not lower than what it is now.

For the five-person board, moving to the cloud will mean relief from the responsibility of fixing physical breaks and coordinating with HUIT to enter Harvard’s server room.

The initiative will also allow the group to easily upgrade and update its services, have minimal server downtime, and potentially lower costs.

“The goal of this transition, this migration, is for the inners of HCS to change and become more updated and for everyone else to feel like things are the same,” Deshpande said. He noted that a new feature will allow users to login to club accounts with their Harvard ID rather than an old unique password given to them at sign-up.

“With the new setup, when we are running an event like DataMatch, we can pay for a little more capacity. But during summer or Christmas break when people aren’t using our services, we can turn off some of the servers,” Deshpande said. “As prices go down on the cloud, it will be a lot easier and simpler to maintain these things in the long run.”

—Staff writer Amna H. Hashmi can be reached at amna.hashmi@thecrimson.com. Follow her on Twitter @amna_hashmi.