'unix' Episodes

The Apache Web Server

     10/29/2019

Welcome to the History of Computing Podcast, where we explore the history of information technology. Because understanding the past prepares us for the innovations of the future! Today we’re going to cover one of the most important and widely distributed server platforms ever: The Apache Web Server. Today, Apache servers account for around 44% of the 1.7 Billion web sites on the Internet. But at one point it was zero. And this is crazy, it’s down from over 70% in 2010. Tim Berners-Lee had put the first website up in 1991 and what we now know as the web was slowly growing. In 1994 and begins with the National Center for Supercomputing Applications, University of Illinois, Urbana-Champaign. Yup, NCSA is also the organization that gave us telnet and Mosaic, the web browser that would evolve into Netscape. After Rob leaves NCSA, the HTTPdaemon goes a little, um, dormant in development. The distress had forked and the extensions and bug fixes needed to get merged into a common distribution. Apache is a free and open source web server that was initially created by Robert McCool and written in C in 1995, the same year Berners-Lee coined the term World Wide Web. You can’t make that name up. I’d always pictured him as a cheetah wearing sunglasses. Who knew that he’d build a tool that would host half of the web sites in the world. A tool that would go on to be built into plenty of computers so they can spin up sharing services. Times have changed since 1995. Originally the name was supposedly a cute name referring to a Patchy server, given that it was based on lots of existing patches of craptostic code from NCSA. So it was initially based on NCSA HTTPd is still alive and well all the way up to the configuration files. For example, on a Mac these are stored at /private/etc/apache2/httpd.conf. The original Apache group consisted of * Brian Behlendorf * Roy T. Fielding * Rob Hartill * David Robinson * Cliff Skolnick * Randy Terbush * Robert S. Thau * Andrew Wilson And there were additional contributions from Eric Hagberg, Frank Peters, and Nicolas Pioch. Within a year of that first shipping, Apache had become the most popular web server on the internet. The distributions and sites continued to grow to the point that they formed the Apache Software Foundation that would give financial, legal, and organizational support for Apache. They even started bringing other open source projects under that umbrella. Projects like Tomcat. And the distributions of Apache grew. Mod_ssl, which brought the first SSL functionality to Apache 1.17, was released in 1998. And it grew. The Apache Foundation came in 1999 to make sure the project outlived the participants and bring other tools under the umbrella. The first conference, ApacheCon came in 2000. Douglas Adams was there. I was not. There were 17 million web sites at the time. The number of web sites hosted on Apache servers continued to rise. Apache 2 was released in 2004. The number of web sites hosted on Apache servers continued to rise. By 2009, Apache was hosting over 100 million websites. By 2013 Apache had added that it was named “out of a respect for the Native American Indian tribe of Apache”. The history isn’t the only thing that was rewritten. Apache itself was rewritten and is now distributed as Apache 2.0. there were over 670 million web sites by then. And we hit 1 billion sites in 2014. I can’t help but wonder what percentage collections of fart jokes. Probably not nearly enough. But an estimated 75% are inactive sites. The job of a web server is to serve web pages on the internet. Those were initially flat HTML files but have gone on to include CGI, PHP, Python, Java, Javascript, and others. A web browser is then used to interpret those files. They access the .html or .htm (or other one of the other many file types that now exist) file and it opens a page and then loads the text, images, included files, and processes any scripts. Both use the http protocol; thus the URL begins with http or https if the site is being hosted over ssl. Apache is responsible for providing the access to those pages over that protocol. The way the scripts are interpreted is through Mods. These include mod_php, mod_python, mod_perl, etc. The modular nature of Apache makes it infinitely extensible. OK, maybe not infinitely. Nothing’s really infinite. But the Loadable Dynamic Modules do make the system more extensible. For example, you can easily get TLS/SSL using mod_ssl. The great thing about Apache and its mods are that anyone can adapt the server for generic uses and they allow you to get into some pretty really specific needs. And the server as well as each of those mods has its source code available on the Interwebs. So if it doesn’t do exactly what you want, you can conform the server to your specific needs. For example, if you wanna’ hate life, there’s a mod for FTP. Out of the box, Apache logs connections, includes a generic expression parser, supports webdav and cgi, can support Embedded Perl, PHP and Lua scripting, can be configured for public_html per-user web-page, supports htaccess to limit access to various directories as one of a few authorization access controls and allows for very in depth custom logging and log rotation. Those logs include things like the name and IP address of a host as well as geolocations. Can rewrite headers, URLs, and content. It’s also simple to enable proxies Apache, along with MySQL, PHP and Linux became so popular that the term LAMP was coined, short for those products. The prevalence allowed the web development community to build hundreds or thousands of tools on top of Apache through the 90s and 2000s, including popular Content Management Systems, or CMS for short, such as Wordpress, Mamba, and Joomla. * Auto-indexing and content negotiation * Reverse proxy with caching * Multiple load balancing mechanisms * Fault tolerance and Failover with automatic recovery * WebSocket, FastCGI, SCGI, AJP and uWSGI support with caching * Dynamic configuration * Name- and IP address-based virtual servers * gzip compression and decompression * Server Side Includes * User and Session tracking * Generic expression parser * Real-time status views * XML support Today we have several web servers to choose from. Engine-X, spelled Nginx, is a newer web server that was initially released in 2004. Apache uses a thread per connection and so can only process the number of threads available; by default 10,000 in Linux and macOS. NGINX doesn’t use threads so can scale differently, and is used by companies like AirBNB, Hulu, Netflix, and Pinterest. That 10,000 limit is easily controlled using concurrent connection limiting, request processing rate limiting, or bandwidth throttling. You can also scale with some serious load balancing and in-band health checks or with one of the many load balancing options. Having said that, Baidu.com, Apple.com, Adobe.com, and PayPal.com - all Apache. We also have other web servers provided by cloud services like Cloudflare and Google slowly increasing in popularity. Tomcat is another web server. But Tomcat is almost exclusively used to run various Java servers, servelets, EL, webscokets, etc. Today, each of the open source projects under the Apache Foundation has a Project Management committee. These provide direction and management of the projects. New members are added when someone who contributes a lot to the project get nominated to be a contributor and then a vote is held requiring unanimous support. Commits require three yes votes with no no votes. It’s all ridiculously efficient in a very open source hacker kinda’ way. The Apache server’s impact on the open-source software community has been profound. It iis partly explained by the unique license from the Apache Software Foundation. The license was in fact written to protect the creators of Apache while giving access to the source code for others to hack away at it. The Apache License 1.1 was approved in 2000 and removed the requirement to attribute the use of the license in advertisements of software. Version two of the license came in 2004, which made the license easier for projects that weren’t from the Apache Foundation. This made it easier for GPL compatibility, and using a reference for the whole project rather than attributing software in every file. The open source nature of Apache was critical to the growth of the web as we know it today. There were other projects to build web servers for sure. Heck, there were other protocols, like Gopher. But many died because of stringent licensing policies. Gopher did great until the University of Minnesota decided to charge for it. Then everyone realized it didn’t have nearly as good of graphics as other web servers. Today the web is one of the single largest growth engines of the global economy. And much of that is owed to Apache. So thanks Apache, for helping us to alleviate a little of the suffering of the human condition for all creatures of the world. By the way, did you know you can buy hamster wheels on the web. Or cat food. Or flea meds for the dog. Speaking of which, I better get back to my chores. Thanks for taking time out of your busy schedule to listen! You probably get to your chores as well though. Sorry if I got you in trouble. But hey, thanks for tuning in to another episode of the History of Computing Podcast. We’re lucky to have you. Have a great day!


The Advent Of The Cloud

     9/5/2019

Welcome to the History of Computing Podcast, where we explore the history of information technology. Because by understanding the past prepares us for the innovations of the future! Today we’re going to look at the emergence of the cloud. As with everything evil, the origin of the cloud began with McCarthyism. From 1950 to 1954 Joe McCarthy waged a war against communism. Wait, wrong McCarthyism. Crap. After Joe McCarthy was condemned and run out of Washington, **John** McCarthy made the world a better place in 1955 with a somewhat communistic approach to computing. The 1950s were the peak of the military industrial complex. The SAGE air defense system needed to process data coming in from radars and perform actions based on that data. This is when McCarthy stepped in. John, not Joe. He proposed things like allocating memory automatically between programs, quote “Programming techniques can be encouraged which make destruction of other programs unlikely” and modifying FORTRAN to trap programs into specified areas of the storage. When a person loading cards or debugging code, the computer could be doing other things. To use his words: “The only way quick response can be provided at a bearable cost is by time-sharing. That is, the computer must attend to other customers while one customer is reacting to some output.” He posited that this could go from a 3 hour to day and a half turnaround to seconds. Remember, back then these things were huge and expensive. So people worked shifts and ran them continuously. McCarthy had been at MIT and Professor Fernando Corbato from there actually built it between 1961 and 1963. But at about the same time, Professor Jack Dennis from MIT started doing about the same thing with a PDP-1 from DEC - he’s actually probably one of the most influential people many I talk to have never heard of. He called this APEX and hooked up multiple terminals on TX-2. Remember John McCarthy? He and some students actually did the same thing in 1962 after moving on to become a professor at Stanford. 1965 saw Alan Kotok sell a similar solution for the PDP-6 and then as the 60s rolled on and people in the Bay Area got really creative and free lovey, Cobato, Jack Dennis of MIT, a team from GE, and another from Bell labs started to work on Multics, or Multiplexed Information and Computing Service for short, for the GE-645 mainframe. Bell Labs pulled out and Multics was finished by MIT and GE, who then sold their computer business to Honeywell so they wouldn’t be out there competing with some of their customers. Honeywell sold Multics until 1985 and it included symmetric multiprocessing, paging, a supervisor program, command programs, and a lot of the things we now take for granted in Linux, Unix, and macOS command lines. But we’re not done with the 60s yet. ARPAnet gave us a standardized communications platform and distributed computing started in the 60s and then became a branch of computer science later in the late 1970s. This is really a software system that has components stored on different networked computers. Oh, and Telnet came at the tail end of 1969 in RFC 15, allowing us to remotely connect to those teletypes. People wanted Time Sharing Systems. Which led Project Genie at Berkely, TOPS-10 for the PDP-10 and IBM’s failed TSS/360 for the System 360. To close out the 70s, Ken Thompson, Dennis Ritchie, Doug McIllroy, Mike Lesk, Joe Assana, and of course Brian Kernighan at Bell Labs hid a project to throw out the fluff from Multics and build a simpler system. This became Unix. Unix was originally developed in Assembly but Ritchie would write C in 72 and the team would eventually refactor Unix in C. Pretty sure management wasn’t at all pissed when they found out. Pretty sure the Uniplexed Information and Computing Services, or eunuchs for short wasn’t punny enough for the Multics team to notice. BSD would come shortly thereafter. Over the coming years you could create multiple users and design permissions in a way that users couldn’t step on each others toes (or more specifically delete each others files). IBM did something interesting in 1972 as well. They invented the Virtual Machine, which allowed them to run an operating system inside an operating system. At this point, time sharing options were becoming common place in mainframes. Enter Moore’s Law. Computers got cheaper and smaller. Altair and hobbyists became a thing. Bill Joy ported BSD to Sun workstations in 77. Computers kept getting smaller. CP/M shows up on early microcomputers at about the same time up until 1983. Apple arrives on the scene. Microsoft DOS appears in 1981 and and In 1983, with all this software you have to pay for really starting to harsh his calm, Richard Stallman famously set out to make software free. Maybe this was in response to Gates’ 1976 Open Letter to Hobbyists asking pc hobbyists to actually pay for software. Maybe they forgot they wrote most of Microsoft BASIC on DARPA gear. Given that computers were so cheap for a bit, we forgot about multi-user operating systems for awhile. By 1991, Linus Torvalds, who also believed in free software, by then known as open source, developed a Unix-like operating system he called Linux. Computers continued to get cheaper and smaller. Now you could have them on multiple desks in an office. Companies like Novell brought us utility computers we now refer to as servers. You had one computer to just host all the files so users could edit them. CERN gave us the first web server in 1990. The University of Minnesota gave us Gopher in 1991. NTP 3 came in 1992. The 90s also saw the rise of virtual private networks and client-server networks. You might load a Delphi-based app on every computer in your office and connect that fat client with a shared database on a server to, for example, have a shared system to enter accounting information into, or access customer information to do sales activities and report on them. Napster had mainstreamed distributed file sharing. Those same techniques were being used in clusters of servers that were all controlled by a central IT administration team. Remember those virtual machines IBM gave us: you could now cluster and virtualize workloads and have applications that were served from a large number of distributed computing systems. But as workloads grew, the fault tolerance and performance necessary to support them became more and more expensive. By the mid-2000s it was becoming more acceptable to move to a web-client architecture, which meant large companies wouldn’t have to bundle up software and automate the delivery of that software and could instead use an intranet to direct users to a series of web pages that allowed them to perform business tasks. Salesforce was started in 1999. They are the poster child for software as a service and founder/CEO Marc Benioff coined the term platform as a service, allowing customers to build their own applications using the Salesforce development environment. But it wasn’t until we started breaking web applications up and developed methods to authenticate and authorize parts of them to one another using technologies like SAML, introduced in 2002) and OAuth (2006) that we were able to move into a more micro-service oriented paradigm for programming. Amazon and Google had been experiencing massive growth and in 2006 Amazon created Amazon Web Services and offered virtual machines on demand to customers, using a service called Elastic Compute Cloud. Google launched G Suite in 2006, providing cloud-based mail, calendar, contacts, documents, and spreadsheets. Google then offered a cloud offering to help developers with apps in 2008 with Google App Engine. In both cases, the companies had invested heavily in developing infrastructure to support their own workloads and renting some of that out to customers just… made sense. Microsoft, seeing the emergence of Google as not just a search engine, but a formidable opponent on multiple fronts also joined into the Infrastructure as a Service as offering virtual machines for pennies per minute of compute time also joined the party in 2008. Google, Microsoft, and Amazon still account for a large percentage of cloud services offered to software developers. Over the past 10 years the technologies have evolved. Mostly just by incrementing a number, like OAuth 2.0 or HTML 5. Web applications have gotten broken up in smaller and smaller parts due to mythical programmer months meaning you need smaller teams who have contracts with other teams that their service, or micro-service, can specific tasks. Amazon, Google, and Microsoft see these services and build more workload specific services, like database as a service or putting a REST front-end on a database, or data lakes as a service. Standards like OAuth even allow vendors to provide Identity as a service, linking up all the things. The cloud, as we’ve come to call hosting services, has been maturing for 55 years, from shared compute time on mainframes to shared file storage space on a server to very small shared services like payment processing using Stripe. Consumers love paying a small monthly fee for access to an online portal or app rather than having to deploy large amounts of capital to bring in an old-school JDS Uniphase style tool to automate tasks in a company. Software developers love importing an SDK or calling a service to get a process for free, allowing developers to go to market much faster and look like magicians in the process. And we don’t have teams at startups running around with fire extinguishers to keep gear humming along. This reduces the barrier to build new software and apps and democratizes software development. App stores and search engines then make it easier than ever to put those web apps and apps in front of people to make money. In 1959, John McCarthy had said “The cooperation of IBM is very important but it should be to their advantage to develop this new way of using a computer.” Like many new philosophies, it takes time to set in and evolve. And it takes a combination of advances to make something so truly disruptive possible. The time-sharing philosophy gave us Unix and Linux, which today are the operating systems running on a lot of these cloud servers. But we don’t know or care about those because the web provides a layer on top of them that obfuscates the workload. Much as the operating system obfuscated the workload of the components of the system. Today those clouds obfuscate various layers of the stack so you can enter at any part of the stack you want whether it’s a virtual computer or a service or just to consume a web app. And this has lead to an explosion of diverse and innovative ideas. Apple famously said “there’s an app for that” but without the cloud there certainly wouldn’t be. And without you, my dear listeners, there wouldn’t be a podcast. So thank you so very much for tuning into another episode of the History of Computing Podcast. We’re lucky to have you. Have a great day!


Coherent Is Not UNIX!

     5/17/2020

In the current day Linux is the most widely used UNIX-like operating system. It's rise to prominence has been an amazing success story. From it's humble beginnings Linux has grown to power everything from super computers to car stereos. But it's not the first UNIX clone. A much earlier system existed, called Coherent. And as it turns out both Linux and Coherent share a lot of similarities. The biggest difference being that Coherent was closed source.

Like the show? Then why not head over and support me on Patreon. Perks include early access to future episodes, and stickers: https://www.patreon.com/adventofcomputing

Important dates in this episode:

1973: AT&T UNIX V4 Goes Public
1949: DOJ Sues AT&T Over Antitrust Violations
1975: AT&T UNIX V6 Released
1977: First Version of BSD Circulates
1977: XYBASIC Released by Mark Williams Company
1980: Coherent Released for PDP/11
1983: Coherent Comes to the IBM PC/XT
1995: Mark Williams Company Closes


Unix for the People, Part 1

     5/20/2019

Many people have never even heard of Unix, an operating system first released in the early 1970s. But that doesn't change the fact that all of the internet, and nearly every computer or smart device you interact with is based on some variant of Unix. So, how was such an important project created, and how did it revolutionize computing?

Today we will dive into the story leading up to Unix: time-sharing computers in the 1960s. This is really just the background for part 2 where we will discuss the creation and rise of Unix itself. However, the history of early multi-user computers is itself deeply interesting and impactful on the evolution of computing.


Unix for the People, Part 2

     6/2/2019

Now, as the name suggests this is the second part of a series on the history of UNIX. Part 1 mainly covers the background leading up to UNIX. If you haven't listened to it yet, I strongly suggest you go do that now. A lot of what was covered in part 1 provides needed context for our discussion today.

 

Just as a quick recap, last time I told you about CTSS and Multics, two of the earliest time-sharing operating systems. Today, we are going to be picking up where we left off: Bell Labs just left Project MAC and decided to start their own time-sharing project. What they didn't realize was that this new project, called UNIX, would soon outshine all of its predecessors. But when this all started, in 1969 on a spare mainframe at Bell Labs, there was no hint at it's amazing future.


C Level, Part II

     4/18/2021

Even after nearly 50 years C remains a force in the programming world. Anytime you brows the web, or even log into a computer, C is somewhere in the background. This episode I wrap up my series on C by looking at it's early development and spread. We will get into the 1st and 2nd C compilers ever written, and take a look at how a banned book lead to generations of avid C programmers.

Like the show? Then why not head over and support me on Patreon. Perks include early access to future episodes, and bonus content: https://www.patreon.com/adventofcomputing


The Dartmouth Time Sharing System and Time Sharing

     10/14/2021

DTSS, or The Dartmouth Time Sharing System, began at Dartmouth College in 1963. That was the same year Project MAC started at MIT, which is where we got Multics, which inspired Unix. Both contributed in their own way to the rise of the Time Sharing movement, an era in computing when people logged into computers over teletype devices and ran computing tasks - treating the large mainframes of the era like a utility.

The notion had been kicking around in 1959 but then John McCarthy at MIT started a project on an IBM 704 mainframe. And PLATO was doing something similar over at the University of Illinois, Champaign-Urbana. 1959 is also when John Kemeny and Thomas Kurtz at Dartmouth College bought Librascope General Purpose computer, then being made in partnership with the Royal Typewriter Company and Librascope - whichwould later be sold off to Lockheed Martin.

Librascope had Stan Frankel - who had worked on both the Manhattan Project and the ENIAC. And he architected the LGP-30 in 1956, which ended up at Dartmouth. At this point, the computer looked like a desk with a built-in typewriter.

Kurtz had four students that were trying to program in ALGOL 58. And they ended up writing a language called DOPE in the early 60s. But they wanted everyone on campus to have access to computing - and John McCarthy said why not try this new time sharing concept. So they went to the National Science Foundation and got funding for a new computer, which to the chagrin of the local IBM salesman, ended up being a GE-225.

This baby was transistorized. It sported 10,0000 transistors and double that number of diodes. It could do floating-point arithmetic, used a 20-bit word, and came with 186,000 magnetic cores for memory. It was so space aged that one of the developers, Arnold Spielberg, would father one of the greatest film directors of all time. Likely straight out of those diodes.

Dartmouth also picked up a front-end processor called a DATANET-30 from GE. This only had an 18-bit word size but could do 4k to 16k words and supported hooking up 128 terminals that could transfer data to and from the system at 3,000 bits a second using the Bell 103 modem. Security wasn’t a thing yet, so these things had direct memory access to the 225, which was a 235 by the time they received the computer.

They got to work in 1963, installing the equipment and writing the code. The DATANET-30 received commands from the terminals and routed them to the mainframe. They scanned for commands 110 times per second from the terminals and ran them when the return key was pressed on a terminal. If the return key was a command they queued it up to run, taking into account routine tasks the computer might be doing in the background.

Keep in mind, the actual CPU was only doing one task at a time, but it seemed like it was multi-tasking! Another aspect of democratizing computing across campus was to write a language that was more approachable than a language like Algol. And so they released BASIC in 1964, picking up where DOPE left off, and picking up a more marketable name.

Here we saw a dozen undergraduates develop a language that was as approachable as the name implies. Some of the students went to Phoenix, where the GE computers were built. And the powers at GE saw the future.

After seeing what Dartmouth had done, GE ended up packaging the DATANET-30 and GE-235 as one machine, which they marketed as the GE-265 the next year. And here we got the first commercially viable time-sharing system, which started a movement. One so successful that GE decided to get out of making computers and focus instead on selling access to time sharing systems. By 1968 they actually ended up shooting up to 40% of the market of the day.

Dartmouth picked up a GE Mark II in 1966 and got to work on DTSS version 2. Here, they added some of the concepts coming out of the Multics project that was part of Project MAC at MIT and built on previous experiences. They added pipes and communication files to promote inter-process communications - thus getting closer to the multiple user conferencing like what was being done on PLATO with Notes.

Things got more efficient and they could handle more and more concurrent sessions. This is when they went from just wanting to offer computing as a basic right on campus to opening up to schools in the area. Nearby Hanover High School started first and by 1967 they had over a dozen. Using further grants from NSF they added another dozen schools to what by then they were calling the Kiewit Network. Then added other smaller colleges and by 1971 supported a whopping 30,000 users. And by 73 supported leased line connections all the way to Ohio, Michigan, New York, and even Montreal.

The system continued on in one form or another, allowing students to code in FORTRAN, COBOL, LISP, and yes… BASIC. It became less of a thing as Personal Computers started to show up here and there. But BASIC didn’t. Every computer needed a BASIC. But people still liked to connect on the system and share information. At least, until the project was finally shut down in 1999. Turns out we didn’t need time sharing once the Internet came along.

Following the early work done by pioneers, companies like Tymshare and CompuServe were born. Tymshare came out of two of the GE team, Thomas O’Rourke and David Schmidt. They ran on SDS hardware and by 1970 had over 100 people, focused on time sharing with their Tymnet system and spreading into Europe by the mid-70s, selling time on their systems until the cost of personal computing caught up and they were acquired by McDonnell Douglas in 1984.

CompuServe began on a PDP-10 and began similarly but by the time they were acquired by H&R Block had successfully pivoted into a dial-up online services company and over time focused on selling access to the Internet. And they survived through to an era when they migrated their own proprietary tooling to HTML in the late 90s - although they were eventually merged into AOL and are now a part of Verizon media. So the pivot bought them an extra decade or so.

Time sharing and BASIC proliferated across the country and then the world from Dartmouth. Much of this - and a lot of personal stories from the people involved can be found in Dr Joy Rankin’s “A People’s History of Computing in the United States.” Published in 2018, it’s a fantastic read that digs in deep on the ways that many of these systems evolved. There are other works, but she does a phenomenal job tying events into one another.

One consistent point across her book is around societal impact. These pioneers democratized access to computing. Many of those who built businesses around time sharing missed the rapidly falling price of chips and the ready access to personal computers that were coming. They also missed that BASIC would be monetized by companies like Microsoft. But they brought computing to high schools in the area, established blueprints for teaching that are used through to this day, and as Grace Hopper did a generation before - made us think of even more ways to make programming more accessible to a new generation with BASIC.

One other author of note here is John Kemeny. His book “Man and the computer” is a must read. He didn’t have the knowledge of the upcoming personal computing - but far more prophetic than not around cloud operations as we get back to a time sharing-esque model of computing. And we do owe him, Kurtz, and everyone else involved a huge debt for their work. Many others pushed the boundaries of what was possible with computers. They pushed the boundaries of what was possible with accessibility. And now we have ubiquity.

So when we see something complicated. Something that doesn’t seem all that approachable. Maybe we should just wonder if - by some stretch - we can make it a bit more BASIC. Like they did.


Banyan Vines and the Emerging Local Area Network

     2/27/2022

One of my first jobs out of college was ripping Banyan VINES out of a company and replacing it with LAN Manager. Banyan VINES was a network operating system for Unix systems. It came along in 1984. This was a time when minicomputers running Unix were running at most every University and when Unix offered far more features that the alternatives.

Sharing files was as old as the Internet. Telnet was created in 1969. FTP came along in 1971. SMB in 1983. Networking computers together had evolved from just the ARPANET to local protocols like ALOHAnet, which inspired Bob Metcalfe to start work on the PARC Universal Packet protocol with David Boggs, which evolved into the Xerox Network Systems, or XNS, suite of networking protocols that were developed to network the Xerox Alto. Along the way the two of them co-invented Ethernet. But there were developments happening in various locations in silos. For example, TCP was more of an ARPANET then NSFNET project so wasn’t used for computers on their own networks to communicate yet.

Data General was founded in 1968 when Edson de Castro, the project manager for the PDP-8 at Digital Equipment Corporation, grew frustrated that the PDP wasn’t evolving fast enough. He, Henry Burkhardt, and Richard Sogge of Digital would be joined by Herbert Richman, who did sales for Fairchild Semiconductor. They were proud of the PDP-8. It was a beautiful machine. But they wanted to go even further. And they didn’t feel like they could do so at Digital. A few computers later,

Within a year, they shipped the next generation machine, which they called the Nova. They released more computers but then came the explosion of computers that was the personal computing market. Microcomputers showed up in offices around the world and on multiple desks. And it didn’t take long before people started wondering if it wouldn’t be faster to run a cable between computers than it was to save a file to a floppy and get on an elevator.

By the 1970s, Data General had been writing software for customers, mostly for the rising tide of UNIX System V implementations. But just giving customers a TCP/IP stack or an application that could open a socket over an X.25 network, which was later replaced with Frame Relay networks run by phone systems and for legacy support on those X.25 was streamed over TCP/IP. Some of the people from those projects at Data General saw an opportunity to build a company that focused on a common need, moving files back and forth between the microcomputers that were also being connected to these networks.

David Mahoney was a manager at Data General who saw what customers were asking for. And he saw an increasing under of those microcomputers needed a few common services to connect to. So he left to form Banyan Systems in 1983, bringing Anand Jagannathan and Larry Floryan with him. They built Banyan VINES (Virtual Integrated NEtwork Service) in 1984, releasing version 1. Their client software could run on DOS and connect to X.25, Token Ring (which IBM introduced in 1984), or the Ethernet networks Bob Metcalfe from Xerox and then 3Com was a proponent of. After all, much of their work resembled the Xerox Network Systems protocols, which Metcalfe had helped develop.

They used a 32-bit address. They developed an Address Resolution Protocol (or ARP) and Routing Table Protocol (RTP) that used tables on a server. And they created a file services application, print services application, and directory service they called StreetTalk. To help, they brought in Jim Allchin, who eventually did much of the heavy lifting. It was similar enough to TCP/IP, but different. Yet as TCP/IP became the standard, they added that at a cost.

The whole thing came in at $17,000 and ran on less bandwidth than other services, and so they won a few contracts with the US State Deparment, US Marine Corps, and other government agencies. Many embassies used 300 baud phone lines with older modems and the new VINES service allowed them to do file sharing, print sharing, and even instant messaging throughout the late 80s and early 90s. The Marine Corp used it during the Gulf War and in an early form of a buying tornado, they went public in 1992, raising $28 million through NASDAQ.

They grew to 410 employees and peaked at around $75 million in sales, spread across 7000 customers. They’d grown through word of mouth and other companies with strong marketing and sales arms were waiting in the wings. Novel was founded in 1983 in Utah and they developed the IPX network protocol. Netware would eventually become one of the most dominant network operating systems for Windows 3 and then Windows 95 computers.

Yet, with incumbents like Banyan VINES and Novel Netware, this is another one of those times when Microsoft saw an opening for something better and just willed it into existence. And the story is similar to that of dozens of other companies including Novell, Lotus, VisiCalc, Netscape, Digital Research, and the list goes on and on and on.

This kept happening because of a number of reasons. The field of computing had been comprised of former academics, many of whom weren’t aggressive in business. Microsoft ended up owning the operating system and so had selling power when it came to cornering adjacent markets because they could provide the cleanest possible user experience. People seemed to underestimate Microsoft until it was too late. Inertia. Oh, and Microsoft could outspend on top talent and offer them the biggest impact for their work. Whatever the motivators, Microsoft won in nearly every nook and cranny in the IT field that they pursued for decades.

The damaging part for Banyan was when they teamed up with IBM to ship LAN Manager, which ultimately shipped under the name of each company. Microsoft ended up recruiting Jim Allchin away and with network interface cards falling below $1,000 it became clear that the local area network was really just in its infancy. He inherited LAN Manager and then NT from Dave Cutler and the next thing we knew, Windows NT Server was born, complete with file services, print services, and a domain, which wasn’t a fully qualified domain name until the release of Active Directory. Microsoft added Windsock in 1993 and released their own protocols. They supported protocols like IPX/SPX and DECnet but slowly moved customers to their own protocols.

Banyan released the last version of Banyan VINES, 7.0, in 1997. StreetTalk eventually became an NT to LDAP bridge before being cancelled in the end. The dot com bubble was firmly here, though, so all was not lost. They changed their name in 1999 to ePresence, shifting their focus to identity management and security, officially pulling out of the VINES market. But the dot com bubble burst, so they were acquired in 2003 by Unisys.

There were other companies in different networking niches along the way. Phil Karn wrote KA9Q NOS to connect CP/M and then DOS to TCP/IP in 1985. He wrote it on a Xerox 820, but by then Xerox was putting Zilog chips in computers and running CP/M, seemingly with little of the flair the Alto could have had. But with KA9Q NOS any of the personal computers on the market could get on the Internet and that software helped host many a commercial dialup connection and would go on to be used for years in small embedded devices that needed IP connectivity. Those turned out to be markets overtaken by Banyan who was overtaken by Novel, who was overtaken by Microsoft when they added WinSock.

There are a few things to take away from this journey. The first is that when IBM and Microsoft team up to develop a competing product, it’s time to pivot when there’s plenty of money left in the bank. The second is that there was an era of closed systems that was short lived when vendors wanted to increasingly embrace open standards. Open standards like TCP/IP. We also want to keep our most talented team in place. Jim Allchin was responsible for those initial Windows Server implementations. Then SQL Server. He was the kind of person who’s a game changer on a team. We also don’t want to pivot to the new hotness because it’s the new hotness. Customers pay vendors to solve problems. Putting an e in front of the name of a company seemed really cool in 1998. But surveying customers and thinking more deeply about problems they face - that’s where magic can happen. Provided we have the right talent to make it happen.


A Brief History Of Time

     7/11/2019

Welcome to the History of computing podcast. Today we’re going to review A Brief History of Time - no, not that brief history of time. But instead how time has evolved in computing. 

We love old things still being used on this podcast. Time is important; so important that it’s epic! Or epoch more specifically. The epoch is a date and time from which a computer measures the time on the system. Most operating systems derive their time from the number of seconds that have passed since January 1st, 1970 when the clock struck midnight, when time began - likely the Catch-22 that the movie was made based on, later that year. This queue is taken from Unix Epoch time. 

Different systems use different time as their epoch. MATLAB uses January 0, 1BC - which is all you need to know about Matlab developers, really. COBOL used January 1, 1601, likely indicating that was the year Cobol was written. OK so it isn’t - but I’m guessing it’s when many of the philosophies of the language were first conceived. Time must seem like it started on January first 2001 to Apple’s Cocoa framework, which began epoch then. My least favorite would be AmigaOS, which started Epoch time on January first 1978 - Nothing good happened in 1978. Jaws 2 and Halloween were released that year. Yuck. Well, Animal House was pretty good. But I could do without Boogie Oogie Oogie. And I could do without Andy Gib’s Shadow Dancing. Disco died the next year. As did the soul of anyone that had to use an Amiga. 

Due to how many modern encryption protocols work, you want to keep time in sync between computers. A skew, or offset in that time, by even microseconds can impact the ability to decrypt data. This lead to the Network Time Protocol, or NTP for short. NTP NTP was designed by David L. Mills of the University of Delaware. It is a networking protocol that provides for clock synchronization between computer systems over standard data networks. NTP has been running since 1985, making it one of the oldest Internet protocols still in use today, with the most updated specs defined in RFC 958. 

`date +%s`

NTP has had a number of updates over the years, although they have slowed as it became more popular. NTP 0 was released in 1985, the same year as the Goonies, Pale Rider, the Breakfast Club and ironically Back to the Future. Given that NTP was free, it’s also ironic that Dire Straits released Money for Nothing the same year it was released. Simple Minds, Aha, and Tears for Fears ruled the airwaves that year, with Tears for Fears proving that Everyone wants to rule the world, but despite being free, NTP is the one on all computers, thus outlasting the rest and being the one that ended up ruling the world. 

Version 1 came in 1988, 2 in 1989, , 3 in 1992, and NTPv4 was drafted in 2010 but has not yet been published given how dependent we as an IT industry now is on NTP. To better understand how dependent we are, let’s look at the three main platforms:

  • In Windows, you can just “Double-click the system clock and then click on the Internet Time tab.” 
  • On Mac, open System Preferences > Date & Time which configures the /usr/libexec/timed launchdaemon
  • And on Linux, open System > Admin >Time and Date. These screens allow you to enter an NTP Server. NTP is short for Network Time Protocol. 

NIST Internet Time Service (ITS) provides 24 names of Network Time Servers, and each vendor often operates their own, such as time.apple.com. Each machine then operates a time zone offset. You know Apple’s time servers because you can read them plain as day by default if you cat /private/etc/ntp.conf - it just outputs server time.apple.com. I’d tell you how to do it in Windows but it would blow your mind. OK, I’ll do it anyways: Just reg query HKLM\SYSTEM\CurrentControlSet\Services\W32Time\Parameters and then read the value of the NtpServer field in the output. OK, not mind blowing. But what is mind blowing?

The Big Bang is mind blowing. Not the TV show; that’s not.

NTP uses 64-bit timestamps. Those consist of a 32-bit portion used for seconds and a 32-bit portion used for a fraction of a second, meaning that it rolls every 232 seconds, which is 136 years. NTPv4 will still need to get ratified by February 7th, 2036, if only to cover the first rollover. NTPv4 moving to a 128-bit date format should take us until the next Big Bang when this stuff won’t matter any more. 

Mills was an interesting cat. He got his PhD in Computer Sciences from the University of Michigan in 1971, where he worked on ARPA projects wrote terminal software that provided connections to the IBM360 mainframe. He also worked on the Exterior Gateway Protocol. He initially invented NTP in 1981 and was a professor in computer science at the University of Delaware from 1986 to 2008. He’s still an emeritus professor at the University of Delaware. 

In 1610 (a few years after the COBOL epoch), the English naval officer Samuel Argall named the Delaware River and Delaware Bay after then governor of Virginia, Thomas West. West happened to be the 12th Baron De La Warr. Did you know that Delaware was the first state to ratify the constitution on December 7th 1787? Delaware is the diamond state, and the second smallest state in the Union. The state insect is a lady bug. Ryan Phillippe is probably more famous than NTP, even though he killed disco with his awful acting in Studio 54. Henry Heimlich is from Delaware. Hopefully you don’t need to use his infamous maneuver as often as NTP gets updated. Elisabeth Shue is also from Delaware. The Karate Kid was awesome. But that’s it. No one else of note. Joe Biden, Senator from Delaware from 1973-2009 and Vice President from 2009 to 2017 - he’s not from Delaware, he’s from Scranton. In case you’re curious, that’s not in Delaware. 

Following the retirement of Mills from the University of Delaware, the reference implementation is currently maintained as an open source project led by Harlan Stenn, who has submitted bug fixes and portability improvements to the NTP codebase since the 1980s. He’s been allowed to focus on time because of the Network Time Foundation, which can be found at https://www.nwtime.org. What’s next for NTP? For one, ratifying NTS. 

Network Time Security (NTS), draft RFC 7384, lets users or servers authenticate to the Network Time Protocol (NTP). This involves a key exchange over TLS that protects against man in the middle attacks, using standard PKI as well as a TLS handshake that then allows time synchronization via extension fields. 


(OldComputerPods) ©Sean Haas, 2020