Welcome to the History of Computing Podcast, where we explore the history of information technology. Because by understanding the past prepares us for the innovations of the future! Today we’re going to look at the emergence of the cloud. As with everything evil, the origin of the cloud began with McCarthyism. From 1950 to 1954 Joe McCarthy waged a war against communism. Wait, wrong McCarthyism. Crap. After Joe McCarthy was condemned and run out of Washington, **John** McCarthy made the world a better place in 1955 with a somewhat communistic approach to computing. The 1950s were the peak of the military industrial complex. The SAGE air defense system needed to process data coming in from radars and perform actions based on that data. This is when McCarthy stepped in. John, not Joe. He proposed things like allocating memory automatically between programs, quote “Programming techniques can be encouraged which make destruction of other programs unlikely” and modifying FORTRAN to trap programs into specified areas of the storage. When a person loading cards or debugging code, the computer could be doing other things. To use his words: “The only way quick response can be provided at a bearable cost is by time-sharing. That is, the computer must attend to other customers while one customer is reacting to some output.” He posited that this could go from a 3 hour to day and a half turnaround to seconds. Remember, back then these things were huge and expensive. So people worked shifts and ran them continuously. McCarthy had been at MIT and Professor Fernando Corbato from there actually built it between 1961 and 1963. But at about the same time, Professor Jack Dennis from MIT started doing about the same thing with a PDP-1 from DEC - he’s actually probably one of the most influential people many I talk to have never heard of. He called this APEX and hooked up multiple terminals on TX-2. Remember John McCarthy? He and some students actually did the same thing in 1962 after moving on to become a professor at Stanford. 1965 saw Alan Kotok sell a similar solution for the PDP-6 and then as the 60s rolled on and people in the Bay Area got really creative and free lovey, Cobato, Jack Dennis of MIT, a team from GE, and another from Bell labs started to work on Multics, or Multiplexed Information and Computing Service for short, for the GE-645 mainframe. Bell Labs pulled out and Multics was finished by MIT and GE, who then sold their computer business to Honeywell so they wouldn’t be out there competing with some of their customers. Honeywell sold Multics until 1985 and it included symmetric multiprocessing, paging, a supervisor program, command programs, and a lot of the things we now take for granted in Linux, Unix, and macOS command lines. But we’re not done with the 60s yet. ARPAnet gave us a standardized communications platform and distributed computing started in the 60s and then became a branch of computer science later in the late 1970s. This is really a software system that has components stored on different networked computers. Oh, and Telnet came at the tail end of 1969 in RFC 15, allowing us to remotely connect to those teletypes. People wanted Time Sharing Systems. Which led Project Genie at Berkely, TOPS-10 for the PDP-10 and IBM’s failed TSS/360 for the System 360. To close out the 70s, Ken Thompson, Dennis Ritchie, Doug McIllroy, Mike Lesk, Joe Assana, and of course Brian Kernighan at Bell Labs hid a project to throw out the fluff from Multics and build a simpler system. This became Unix. Unix was originally developed in Assembly but Ritchie would write C in 72 and the team would eventually refactor Unix in C. Pretty sure management wasn’t at all pissed when they found out. Pretty sure the Uniplexed Information and Computing Services, or eunuchs for short wasn’t punny enough for the Multics team to notice. BSD would come shortly thereafter. Over the coming years you could create multiple users and design permissions in a way that users couldn’t step on each others toes (or more specifically delete each others files). IBM did something interesting in 1972 as well. They invented the Virtual Machine, which allowed them to run an operating system inside an operating system. At this point, time sharing options were becoming common place in mainframes. Enter Moore’s Law. Computers got cheaper and smaller. Altair and hobbyists became a thing. Bill Joy ported BSD to Sun workstations in 77. Computers kept getting smaller. CP/M shows up on early microcomputers at about the same time up until 1983. Apple arrives on the scene. Microsoft DOS appears in 1981 and and In 1983, with all this software you have to pay for really starting to harsh his calm, Richard Stallman famously set out to make software free. Maybe this was in response to Gates’ 1976 Open Letter to Hobbyists asking pc hobbyists to actually pay for software. Maybe they forgot they wrote most of Microsoft BASIC on DARPA gear. Given that computers were so cheap for a bit, we forgot about multi-user operating systems for awhile. By 1991, Linus Torvalds, who also believed in free software, by then known as open source, developed a Unix-like operating system he called Linux. Computers continued to get cheaper and smaller. Now you could have them on multiple desks in an office. Companies like Novell brought us utility computers we now refer to as servers. You had one computer to just host all the files so users could edit them. CERN gave us the first web server in 1990. The University of Minnesota gave us Gopher in 1991. NTP 3 came in 1992. The 90s also saw the rise of virtual private networks and client-server networks. You might load a Delphi-based app on every computer in your office and connect that fat client with a shared database on a server to, for example, have a shared system to enter accounting information into, or access customer information to do sales activities and report on them. Napster had mainstreamed distributed file sharing. Those same techniques were being used in clusters of servers that were all controlled by a central IT administration team. Remember those virtual machines IBM gave us: you could now cluster and virtualize workloads and have applications that were served from a large number of distributed computing systems. But as workloads grew, the fault tolerance and performance necessary to support them became more and more expensive. By the mid-2000s it was becoming more acceptable to move to a web-client architecture, which meant large companies wouldn’t have to bundle up software and automate the delivery of that software and could instead use an intranet to direct users to a series of web pages that allowed them to perform business tasks. Salesforce was started in 1999. They are the poster child for software as a service and founder/CEO Marc Benioff coined the term platform as a service, allowing customers to build their own applications using the Salesforce development environment. But it wasn’t until we started breaking web applications up and developed methods to authenticate and authorize parts of them to one another using technologies like SAML, introduced in 2002) and OAuth (2006) that we were able to move into a more micro-service oriented paradigm for programming. Amazon and Google had been experiencing massive growth and in 2006 Amazon created Amazon Web Services and offered virtual machines on demand to customers, using a service called Elastic Compute Cloud. Google launched G Suite in 2006, providing cloud-based mail, calendar, contacts, documents, and spreadsheets. Google then offered a cloud offering to help developers with apps in 2008 with Google App Engine. In both cases, the companies had invested heavily in developing infrastructure to support their own workloads and renting some of that out to customers just… made sense. Microsoft, seeing the emergence of Google as not just a search engine, but a formidable opponent on multiple fronts also joined into the Infrastructure as a Service as offering virtual machines for pennies per minute of compute time also joined the party in 2008. Google, Microsoft, and Amazon still account for a large percentage of cloud services offered to software developers. Over the past 10 years the technologies have evolved. Mostly just by incrementing a number, like OAuth 2.0 or HTML 5. Web applications have gotten broken up in smaller and smaller parts due to mythical programmer months meaning you need smaller teams who have contracts with other teams that their service, or micro-service, can specific tasks. Amazon, Google, and Microsoft see these services and build more workload specific services, like database as a service or putting a REST front-end on a database, or data lakes as a service. Standards like OAuth even allow vendors to provide Identity as a service, linking up all the things. The cloud, as we’ve come to call hosting services, has been maturing for 55 years, from shared compute time on mainframes to shared file storage space on a server to very small shared services like payment processing using Stripe. Consumers love paying a small monthly fee for access to an online portal or app rather than having to deploy large amounts of capital to bring in an old-school JDS Uniphase style tool to automate tasks in a company. Software developers love importing an SDK or calling a service to get a process for free, allowing developers to go to market much faster and look like magicians in the process. And we don’t have teams at startups running around with fire extinguishers to keep gear humming along. This reduces the barrier to build new software and apps and democratizes software development. App stores and search engines then make it easier than ever to put those web apps and apps in front of people to make money. In 1959, John McCarthy had said “The cooperation of IBM is very important but it should be to their advantage to develop this new way of using a computer.” Like many new philosophies, it takes time to set in and evolve. And it takes a combination of advances to make something so truly disruptive possible. The time-sharing philosophy gave us Unix and Linux, which today are the operating systems running on a lot of these cloud servers. But we don’t know or care about those because the web provides a layer on top of them that obfuscates the workload. Much as the operating system obfuscated the workload of the components of the system. Today those clouds obfuscate various layers of the stack so you can enter at any part of the stack you want whether it’s a virtual computer or a service or just to consume a web app. And this has lead to an explosion of diverse and innovative ideas. Apple famously said “there’s an app for that” but without the cloud there certainly wouldn’t be. And without you, my dear listeners, there wouldn’t be a podcast. So thank you so very much for tuning into another episode of the History of Computing Podcast. We’re lucky to have you. Have a great day!
In the current day Linux is the most widely used UNIX-like operating system. It's rise to prominence has been an amazing success story. From it's humble beginnings Linux has grown to power everything from super computers to car stereos. But it's not the first UNIX clone. A much earlier system existed, called Coherent. And as it turns out both Linux and Coherent share a lot of similarities. The biggest difference being that Coherent was closed source.
Like the show? Then why not head over and support me on Patreon. Perks include early access to future episodes, and stickers: https://www.patreon.com/adventofcomputing
Important dates in this episode:
1973: AT&T UNIX V4 Goes Public
1949: DOJ Sues AT&T Over Antitrust Violations
1975: AT&T UNIX V6 Released
1977: First Version of BSD Circulates
1977: XYBASIC Released by Mark Williams Company
1980: Coherent Released for PDP/11
1983: Coherent Comes to the IBM PC/XT
1995: Mark Williams Company Closes
Many people have never even heard of Unix, an operating system first released in the early 1970s. But that doesn't change the fact that all of the internet, and nearly every computer or smart device you interact with is based on some variant of Unix. So, how was such an important project created, and how did it revolutionize computing?
Today we will dive into the story leading up to Unix: time-sharing computers in the 1960s. This is really just the background for part 2 where we will discuss the creation and rise of Unix itself. However, the history of early multi-user computers is itself deeply interesting and impactful on the evolution of computing.
Now, as the name suggests this is the second part of a series on the history of UNIX. Part 1 mainly covers the background leading up to UNIX. If you haven't listened to it yet, I strongly suggest you go do that now. A lot of what was covered in part 1 provides needed context for our discussion today.
Just as a quick recap, last time I told you about CTSS and Multics, two of the earliest time-sharing operating systems. Today, we are going to be picking up where we left off: Bell Labs just left Project MAC and decided to start their own time-sharing project. What they didn't realize was that this new project, called UNIX, would soon outshine all of its predecessors. But when this all started, in 1969 on a spare mainframe at Bell Labs, there was no hint at it's amazing future.
Even after nearly 50 years C remains a force in the programming world. Anytime you brows the web, or even log into a computer, C is somewhere in the background. This episode I wrap up my series on C by looking at it's early development and spread. We will get into the 1st and 2nd C compilers ever written, and take a look at how a banned book lead to generations of avid C programmers.
Like the show? Then why not head over and support me on Patreon. Perks include early access to future episodes, and bonus content: https://www.patreon.com/adventofcomputing
DTSS, or The Dartmouth Time Sharing System, began at Dartmouth College in 1963. That was the same year Project MAC started at MIT, which is where we got Multics, which inspired Unix. Both contributed in their own way to the rise of the Time Sharing movement, an era in computing when people logged into computers over teletype devices and ran computing tasks - treating the large mainframes of the era like a utility.
The notion had been kicking around in 1959 but then John McCarthy at MIT started a project on an IBM 704 mainframe. And PLATO was doing something similar over at the University of Illinois, Champaign-Urbana. 1959 is also when John Kemeny and Thomas Kurtz at Dartmouth College bought Librascope General Purpose computer, then being made in partnership with the Royal Typewriter Company and Librascope - whichwould later be sold off to Lockheed Martin.
Librascope had Stan Frankel - who had worked on both the Manhattan Project and the ENIAC. And he architected the LGP-30 in 1956, which ended up at Dartmouth. At this point, the computer looked like a desk with a built-in typewriter.
Kurtz had four students that were trying to program in ALGOL 58. And they ended up writing a language called DOPE in the early 60s. But they wanted everyone on campus to have access to computing - and John McCarthy said why not try this new time sharing concept. So they went to the National Science Foundation and got funding for a new computer, which to the chagrin of the local IBM salesman, ended up being a GE-225.
This baby was transistorized. It sported 10,0000 transistors and double that number of diodes. It could do floating-point arithmetic, used a 20-bit word, and came with 186,000 magnetic cores for memory. It was so space aged that one of the developers, Arnold Spielberg, would father one of the greatest film directors of all time. Likely straight out of those diodes.
Dartmouth also picked up a front-end processor called a DATANET-30 from GE. This only had an 18-bit word size but could do 4k to 16k words and supported hooking up 128 terminals that could transfer data to and from the system at 3,000 bits a second using the Bell 103 modem. Security wasn’t a thing yet, so these things had direct memory access to the 225, which was a 235 by the time they received the computer.
They got to work in 1963, installing the equipment and writing the code. The DATANET-30 received commands from the terminals and routed them to the mainframe. They scanned for commands 110 times per second from the terminals and ran them when the return key was pressed on a terminal. If the return key was a command they queued it up to run, taking into account routine tasks the computer might be doing in the background.
Keep in mind, the actual CPU was only doing one task at a time, but it seemed like it was multi-tasking! Another aspect of democratizing computing across campus was to write a language that was more approachable than a language like Algol. And so they released BASIC in 1964, picking up where DOPE left off, and picking up a more marketable name.
Here we saw a dozen undergraduates develop a language that was as approachable as the name implies. Some of the students went to Phoenix, where the GE computers were built. And the powers at GE saw the future.
After seeing what Dartmouth had done, GE ended up packaging the DATANET-30 and GE-235 as one machine, which they marketed as the GE-265 the next year. And here we got the first commercially viable time-sharing system, which started a movement. One so successful that GE decided to get out of making computers and focus instead on selling access to time sharing systems. By 1968 they actually ended up shooting up to 40% of the market of the day.
Dartmouth picked up a GE Mark II in 1966 and got to work on DTSS version 2. Here, they added some of the concepts coming out of the Multics project that was part of Project MAC at MIT and built on previous experiences. They added pipes and communication files to promote inter-process communications - thus getting closer to the multiple user conferencing like what was being done on PLATO with Notes.
Things got more efficient and they could handle more and more concurrent sessions. This is when they went from just wanting to offer computing as a basic right on campus to opening up to schools in the area. Nearby Hanover High School started first and by 1967 they had over a dozen. Using further grants from NSF they added another dozen schools to what by then they were calling the Kiewit Network. Then added other smaller colleges and by 1971 supported a whopping 30,000 users. And by 73 supported leased line connections all the way to Ohio, Michigan, New York, and even Montreal.
The system continued on in one form or another, allowing students to code in FORTRAN, COBOL, LISP, and yes… BASIC. It became less of a thing as Personal Computers started to show up here and there. But BASIC didn’t. Every computer needed a BASIC. But people still liked to connect on the system and share information. At least, until the project was finally shut down in 1999. Turns out we didn’t need time sharing once the Internet came along.
Following the early work done by pioneers, companies like Tymshare and CompuServe were born. Tymshare came out of two of the GE team, Thomas O’Rourke and David Schmidt. They ran on SDS hardware and by 1970 had over 100 people, focused on time sharing with their Tymnet system and spreading into Europe by the mid-70s, selling time on their systems until the cost of personal computing caught up and they were acquired by McDonnell Douglas in 1984.
CompuServe began on a PDP-10 and began similarly but by the time they were acquired by H&R Block had successfully pivoted into a dial-up online services company and over time focused on selling access to the Internet. And they survived through to an era when they migrated their own proprietary tooling to HTML in the late 90s - although they were eventually merged into AOL and are now a part of Verizon media. So the pivot bought them an extra decade or so.
Time sharing and BASIC proliferated across the country and then the world from Dartmouth. Much of this - and a lot of personal stories from the people involved can be found in Dr Joy Rankin’s “A People’s History of Computing in the United States.” Published in 2018, it’s a fantastic read that digs in deep on the ways that many of these systems evolved. There are other works, but she does a phenomenal job tying events into one another.
One consistent point across her book is around societal impact. These pioneers democratized access to computing. Many of those who built businesses around time sharing missed the rapidly falling price of chips and the ready access to personal computers that were coming. They also missed that BASIC would be monetized by companies like Microsoft. But they brought computing to high schools in the area, established blueprints for teaching that are used through to this day, and as Grace Hopper did a generation before - made us think of even more ways to make programming more accessible to a new generation with BASIC.
One other author of note here is John Kemeny. His book “Man and the computer” is a must read. He didn’t have the knowledge of the upcoming personal computing - but far more prophetic than not around cloud operations as we get back to a time sharing-esque model of computing. And we do owe him, Kurtz, and everyone else involved a huge debt for their work. Many others pushed the boundaries of what was possible with computers. They pushed the boundaries of what was possible with accessibility. And now we have ubiquity.
So when we see something complicated. Something that doesn’t seem all that approachable. Maybe we should just wonder if - by some stretch - we can make it a bit more BASIC. Like they did.
One of my first jobs out of college was ripping Banyan VINES out of a company and replacing it with LAN Manager. Banyan VINES was a network operating system for Unix systems. It came along in 1984. This was a time when minicomputers running Unix were running at most every University and when Unix offered far more features that the alternatives.
Sharing files was as old as the Internet. Telnet was created in 1969. FTP came along in 1971. SMB in 1983. Networking computers together had evolved from just the ARPANET to local protocols like ALOHAnet, which inspired Bob Metcalfe to start work on the PARC Universal Packet protocol with David Boggs, which evolved into the Xerox Network Systems, or XNS, suite of networking protocols that were developed to network the Xerox Alto. Along the way the two of them co-invented Ethernet. But there were developments happening in various locations in silos. For example, TCP was more of an ARPANET then NSFNET project so wasn’t used for computers on their own networks to communicate yet.
Data General was founded in 1968 when Edson de Castro, the project manager for the PDP-8 at Digital Equipment Corporation, grew frustrated that the PDP wasn’t evolving fast enough. He, Henry Burkhardt, and Richard Sogge of Digital would be joined by Herbert Richman, who did sales for Fairchild Semiconductor. They were proud of the PDP-8. It was a beautiful machine. But they wanted to go even further. And they didn’t feel like they could do so at Digital. A few computers later,
Within a year, they shipped the next generation machine, which they called the Nova. They released more computers but then came the explosion of computers that was the personal computing market. Microcomputers showed up in offices around the world and on multiple desks. And it didn’t take long before people started wondering if it wouldn’t be faster to run a cable between computers than it was to save a file to a floppy and get on an elevator.
By the 1970s, Data General had been writing software for customers, mostly for the rising tide of UNIX System V implementations. But just giving customers a TCP/IP stack or an application that could open a socket over an X.25 network, which was later replaced with Frame Relay networks run by phone systems and for legacy support on those X.25 was streamed over TCP/IP. Some of the people from those projects at Data General saw an opportunity to build a company that focused on a common need, moving files back and forth between the microcomputers that were also being connected to these networks.
David Mahoney was a manager at Data General who saw what customers were asking for. And he saw an increasing under of those microcomputers needed a few common services to connect to. So he left to form Banyan Systems in 1983, bringing Anand Jagannathan and Larry Floryan with him. They built Banyan VINES (Virtual Integrated NEtwork Service) in 1984, releasing version 1. Their client software could run on DOS and connect to X.25, Token Ring (which IBM introduced in 1984), or the Ethernet networks Bob Metcalfe from Xerox and then 3Com was a proponent of. After all, much of their work resembled the Xerox Network Systems protocols, which Metcalfe had helped develop.
They used a 32-bit address. They developed an Address Resolution Protocol (or ARP) and Routing Table Protocol (RTP) that used tables on a server. And they created a file services application, print services application, and directory service they called StreetTalk. To help, they brought in Jim Allchin, who eventually did much of the heavy lifting. It was similar enough to TCP/IP, but different. Yet as TCP/IP became the standard, they added that at a cost.
The whole thing came in at $17,000 and ran on less bandwidth than other services, and so they won a few contracts with the US State Deparment, US Marine Corps, and other government agencies. Many embassies used 300 baud phone lines with older modems and the new VINES service allowed them to do file sharing, print sharing, and even instant messaging throughout the late 80s and early 90s. The Marine Corp used it during the Gulf War and in an early form of a buying tornado, they went public in 1992, raising $28 million through NASDAQ.
They grew to 410 employees and peaked at around $75 million in sales, spread across 7000 customers. They’d grown through word of mouth and other companies with strong marketing and sales arms were waiting in the wings. Novel was founded in 1983 in Utah and they developed the IPX network protocol. Netware would eventually become one of the most dominant network operating systems for Windows 3 and then Windows 95 computers.
Yet, with incumbents like Banyan VINES and Novel Netware, this is another one of those times when Microsoft saw an opening for something better and just willed it into existence. And the story is similar to that of dozens of other companies including Novell, Lotus, VisiCalc, Netscape, Digital Research, and the list goes on and on and on.
This kept happening because of a number of reasons. The field of computing had been comprised of former academics, many of whom weren’t aggressive in business. Microsoft ended up owning the operating system and so had selling power when it came to cornering adjacent markets because they could provide the cleanest possible user experience. People seemed to underestimate Microsoft until it was too late. Inertia. Oh, and Microsoft could outspend on top talent and offer them the biggest impact for their work. Whatever the motivators, Microsoft won in nearly every nook and cranny in the IT field that they pursued for decades.
The damaging part for Banyan was when they teamed up with IBM to ship LAN Manager, which ultimately shipped under the name of each company. Microsoft ended up recruiting Jim Allchin away and with network interface cards falling below $1,000 it became clear that the local area network was really just in its infancy. He inherited LAN Manager and then NT from Dave Cutler and the next thing we knew, Windows NT Server was born, complete with file services, print services, and a domain, which wasn’t a fully qualified domain name until the release of Active Directory. Microsoft added Windsock in 1993 and released their own protocols. They supported protocols like IPX/SPX and DECnet but slowly moved customers to their own protocols.
Banyan released the last version of Banyan VINES, 7.0, in 1997. StreetTalk eventually became an NT to LDAP bridge before being cancelled in the end. The dot com bubble was firmly here, though, so all was not lost. They changed their name in 1999 to ePresence, shifting their focus to identity management and security, officially pulling out of the VINES market. But the dot com bubble burst, so they were acquired in 2003 by Unisys.
There were other companies in different networking niches along the way. Phil Karn wrote KA9Q NOS to connect CP/M and then DOS to TCP/IP in 1985. He wrote it on a Xerox 820, but by then Xerox was putting Zilog chips in computers and running CP/M, seemingly with little of the flair the Alto could have had. But with KA9Q NOS any of the personal computers on the market could get on the Internet and that software helped host many a commercial dialup connection and would go on to be used for years in small embedded devices that needed IP connectivity. Those turned out to be markets overtaken by Banyan who was overtaken by Novel, who was overtaken by Microsoft when they added WinSock.
There are a few things to take away from this journey. The first is that when IBM and Microsoft team up to develop a competing product, it’s time to pivot when there’s plenty of money left in the bank. The second is that there was an era of closed systems that was short lived when vendors wanted to increasingly embrace open standards. Open standards like TCP/IP. We also want to keep our most talented team in place. Jim Allchin was responsible for those initial Windows Server implementations. Then SQL Server. He was the kind of person who’s a game changer on a team. We also don’t want to pivot to the new hotness because it’s the new hotness. Customers pay vendors to solve problems. Putting an e in front of the name of a company seemed really cool in 1998. But surveying customers and thinking more deeply about problems they face - that’s where magic can happen. Provided we have the right talent to make it happen.