Science & Tech

‘There will be cascading failures that get fixed on the fly’

View from above of city network of lights.

Anastasia Dulgier/Unsplash

long read

Harvard’s Waldo says the public flight to remote work will stress-test the internet — and some parts will need repair

This is part of our Coronavirus Update series in which Harvard specialists in epidemiology, infectious disease, economics, politics, and other disciplines offer insights into what the latest developments in the COVID-19 outbreak may bring.

With offices across the country shuttered and workers being asked to work remotely when they can, the nation is relying on the robustness of the internet and technology infrastructure as never before. To understand the issues in play, the Gazette spoke with Jim Waldo, chief technology officer for the Harvard John A. Paulson School of Engineering and Applied Sciences, professor of the practice of computer science there, and professor of technology policy at the Harvard Kennedy School. Waldo, who spent three decades in the tech industry, discussed the likelihood that parts of the all-important internet will fail, and the equal likelihood that engineers will make repairs on the fly to keep people working.

Q&A

Jim Waldo

GAZETTE:  Are we better prepared technologically than we might have been 10 years ago for this sort of a shift to remote work?

WALDO: Oh, my goodness, yes. The ubiquity of the internet and the tools that have been built around it make this kind of shift at least possible to think of. Ten years ago, I don’t think it would have been. We did not have applications like Zoom that allow fairly high-quality video conferencing and the recording of video and even audio conferencing at fairly compressed bandwidth. Then, that wasn’t possible. We did not have creations like Canvas that allow us to do things in the classroom that older systems didn’t allow us to do. Back then, we had webpages, and now we have a set of interactive tools that allows discussions to happen online, that allows the distribution of content in various ways, that allows us to run quizzes and exams. All of this has been made possible by changes made in recent years, both in the technology of networking in general and the way that we are using technology at Harvard.

GAZETTE: I’ve heard that in addition to this being a nationwide crisis — one that touches us here on campus — there’s also opportunity. You hear about Zoom stock being up; perhaps, amid the damage, this shift will be beneficial to some of these companies and part of this industry.

WALDO: Zoom may be the only company that is having that experience, but maybe some of the network vendors are seeing an uptick. I actually look at this as a very interesting experiment. We built the internet years years ago to handle the kind of traffic that researchers needed, and it has grown to be part of the fabric of everyday life. Now we are going to stress-test it in a way it has never been before. And the internet was always built to be sort of reliably crummy.

GAZETTE: Can you explain that?

WALDO: Well, the internet is built on the notion that packets will get lost, that pieces of it will go out of service, that it can do adaptive routing, that bad things will happen in the network. And the internet is built to deal with those kinds of problems. But I don’t think we have ever stress-tested it on the scale that we are going to over the next couple of weeks.

“One of the miraculous things about the internet is how well it has scaled up. So that’s why I don’t worry, but I do think this is an interesting experiment.”

GAZETTE:  How is the current situation, with so many people home, different from what happens when everyone is in the office? Is the traffic different? Is it more? I know the infrastructure here in my house is not as robust as Harvard’s; is it going through different equipment?

WALDO: It’s not so much going through different equipment as it is going through a lot more stuff. At Harvard, we have a fairly sophisticated network. That keeps a lot of the traffic that is being sent from Harvard inside its network, where we know what the bandwidth constraints are, we know how many switches it’s going to be going through. For the most part, it doesn’t get out of a Harvard-controlled network. Now that everybody is at home — and home may be anywhere from Boston to China for some of our students — it is going via a much more complicated route and a much more varied set of equipment, and that’s going to stress everything. This is going on for every other company as well.

I’m not worried so much about the core of the internet; that’s not going to change all that much. But that last mile — or in some cases that last 200 yards — between you and the major backbones of the internet are not built for the kind of stress that we have been putting on them over the last week. The internet itself was not built for any of the kinds of stresses that we’re putting on it right now. One of the miraculous things about the internet is how well it has scaled up. So that’s why I don’t worry, but I do think this is an interesting experiment.

GAZETTE: In real time, with fairly high stakes.

WALDO: Well, yes. And sometimes when an experiment fails, it truly sucks, to use a technical term. But I think there are various ways we can think of adapting. We are starting off by saying, “All right, let’s try running our classes on Zoom.” Everybody has their video on, and you can look at people. Well, to ease bandwidth, it may be that we need to actually run our classes on Zoom with the video off, so it’s more like a large-scale teleconference. I also think instructors are going to have to start thinking about how they design their courses for asynchronous attendance because not all your students are going to be online at the same time. They’re in different time zones, in different places, and then maybe at endpoints that have different amounts of connectivity. Are they going to be watching on a laptop, on a tablet, or on a phone? That’s going to require that we adapt the ways that we have traditionally taught.

GAZETTE:  Have you been advising faculty? Have folks come to you and said, “How do we do this?”

WALDO: There are much better resources for that than me. The Bok Center and the vice provost for advances in learning have both put up really excellent sites. I’ve noticed a lot of departmental sites where instructors are talking over how they can teach their particular subjects in different ways. We’re all learning really rapidly, because we have to. Generally, designing a course — if you do it in a thoughtful fashion — takes somewhere between a couple of months and a semester, but we’re all being asked to redesign our courses in thoughtful ways in a week.

GAZETTE:  How have you redesigned your course? You mentioned that was part of what you’re going to be doing this week.

WALDO: Well, fortunately, the week is not over. But as I said before, I’m looking at various ways we can deliver content in an asynchronous fashion rather than assuming that everybody is going to be online at the same time. The course I’m teaching has around 65 people in it. We’re looking at ways of breaking that up into smaller discussion groups that could be online. We plan on making heavy use of a couple of discussion forums that we already have connected into our Canvas site. And we’ve already set up online office hours. We will adapt as we go along.

GAZETTE:  Does asynchronous mean one of the first steps is to make sure your lecture is recorded?

WALDO: Certainly that helps. I hope to never give another lecture — certainly not another lecture in this class this semester. So, if there are things that I’m going to record, I hope it’s going to be much more like the Khan Academy approach, where it’s more like you are talking to the student one-on-one over a computer screen. And even then we’d have that segment be short, 10 minutes or so, interspersed with things to think about and to put on the class discussion board. That’s my thought right now, but this has been changing day by day.

GAZETTE:  What is the ability to adapt as the semester goes on and as people begin to learn what works and what doesn’t? Can you change on the fly, so a class that’s being taught when exams begin the first week of May looks different from what it looks like next week?

WALDO: I would be amazed if they didn’t. I’m co-teaching my course right now with Mike Smith, and one of the things we emphasized during our last in-person meeting with our students was that this was going to be a joint effort. It will be up to them to give us feedback on what is working and what is not. This is a great adventure, and it’s not going to come top-down, it’s going to come sideways, and from the bottom up, and everybody has to participate, which in some ways makes it exciting. I look at this as an opportunity to really examine how we can teach differently.

GAZETTE:  Do you expect enduring things to come out of this? Innovations?

WALDO: I think there may be some. I have been meaning to try a more thorough flipping of my classroom for some time, and this may be a good excuse to try that. But I don’t think there is, quite frankly, anything that we are going to be able to do in this situation that is going to compete with the richness of the face-to-face interactions that we have — or even more to the point, to compete with the most powerful teaching tool we have at Harvard: the students being co-located and talking to each other. I’ve been known to say that that the true learning environment at Harvard is the fact that we bring in the students that we do, we pack them together, and we give them just enough interesting ideas and enough adult supervision to keep the “Lord of the Flies” thing from happening. And then we let them teach each other. That’s what’s really going to be lacking here. Seeing what we can do online to get the students teaching each other so they can learn in that sort of rich discussion environment is the real challenge.

GAZETTE:  What should readers think about going forward, both folks who don’t work at Harvard and are going through this experience with company X, and people who are looking at the end of spring break and the resumption of classes online?

WALDO: I think everybody has to understand that this really is an experiment on many, many different levels. It is an experiment in how we can teach a different way, without being face-to-face with our students, but still give them an educational experience that is worthy of being called “Harvard.” It’s an experiment in how to use technology in better ways not only to teach, but to connect and keep the community together. And it’s an experiment in how robust the technology is that we’ve all come to depend on and that now we’re depending on even more. So, in every layer, this is an experiment and we will learn as we go.

Don’t think that we actually have a good idea of what we’re doing. This is, in a sense, pure research, where we have no idea what we’re doing. We’re going to try things, and some of them will work, and some of them won’t work. And we will all learn from it, and that will be good.

GAZETTE:  The tech industry draws a lot of bright minds with a lot of good ideas. If something fails the stress test, do you see those minds as able to fix it on the fly? Or do you anticipate the opposite — cascading failures?

WALDO: I think we’ll get both: There will be cascading failures that will get fixed on the fly. I spent more than 30 years in industry, and one of the major ways you learn in industry is when things fail. In all of engineering, it’s that way. So failure is an opportunity to make things better. We just have to be sure we know why we failed and make sure we don’t fail the same way twice.

GAZETTE:  Is there anything you’d like to add?

WALDO: Just that it’s a weird feeling — both of elation and being scared to death — that this is giving to a lot of us because it is such an opportunity to learn and experiment.

This interview has been lightly edited.