When you click on a link or type a website address into your browser's address field the invisible gears of the Internet get to work. When everything is working hunky dunky you probably give no thought to how it all works. But, if you're like most people, you become lost and helpless when a single piece of this massive orchestration fails. My goal is to help clear that up for you, at least a bit.
In order for everything to work the following three pieces must work together.
- domain registration - a domain has to be properly registered
- initial domain lookup - your computer or device needs to know where the web server is that will process your request
- web server processing - the web server has to understand what you want
It all starts with registering a domain. Notice I wrote "registering" and not "buying". You don't own a domain, you register the use of a domain for 1-10 years at a time. So the next time someone says, "Oh, I own that domain," you can pretentiously correct them and say "Wrong sir, you registered the domain. You do not own the domain." Not a good way to make friends but you can sound really smart.
In reality, the quasi-owner of the domain is ICANN (Internet Corporation for Assigned Names and Numbers). They are the organization that allows other companies to become a top level domain registry, who can then sell domains using the licensed top level domain.
A top level domain (or TLD) is the .com, .net, .us, or .edu part of the domain.
A couple years ago there were only a handful of TLDs, such as, .com, .org, .net, .us, .cc, .gov, .edu, etc. Now there are hundreds of top level domains and more arriving regularly. You can register yourdomain.me, yourstore.blackfriday, or even आपकी साइट.भारत (Hindi for yoursite.bharat).
Here's how it works:
- A user, or registrant, visits a registrar like GoDaddy, Network Solutions, enom, or one of the other thousand ICANN accredited registrars around the world.
- Once the registrar knows what domain you want it then contacts the TLD's registry operator. For .com TLDs the registry operator is Verisign. The registry then checks if the TLD is available and returns the status back to the registrar.
- If the domain is available, the registrar notifies the user and the user pays the fee for a registration of 1-10 years. Some domains require additional information and the registrar is responsible for obtaining that information or verifying proper intended use of the domain. An example is the .us domain. Organizations that register a .us domain must be a citizen, resident, organization, or foreign entity with a US presence.
Also during the final registration process, or during a subsequent update to the domain, the user can the name servers that will handle its requests (more on this later).
After registration, the registrar (or an associated registrar reseller) is responsible for notifying customers when domains are about to expire and also for maintaining current contact information for the domain.
If you, or someone in your organization is not responsible for keeping your domain registered then you need to do this ASAP. We've had several customers who ignored expiration notices from their registrar and lost their domain. One even had a foreign entity register the domain as soon as it was available, create an exact copy of their website, and change all links on the site to point to an e-commerce site for Gucci watches. And they were prepared to do this because they copied their entire website before the domain expired. I can only imagine what important emails were also being routed to them!
A domain is the friendly representation of a more complicated numbering system called an IP address (or network address). You should be quite familiar with domains. They are used to find websites, send emails, and connect computers to each other for processing all sorts of data (credit cards, file storage, video streaming, etc). marketpath.com is our domain that translates into the IP address 18.104.22.168. Rather than typing in that long number I simply type marketpath.com. That's when the magic begins.
- The user types marketpath.com into the browser's address field. The computer contacts its assigned domain name system (DNS) to get the IP address of the web server hosting Marketpath's website.
- The DNS server, if it hasn't looked up the domain recently, will reach out to the Internet's root name server to figure out where to go next. The root name servers holds the registry locations of all the TLD's, such as, .com, .net, and .org. Basically, the root name server tells the DNS server where to search next, or for .com domains, who has all the information about the .com domains.
- The DNS server then contacts the registry server for the .com domain. This server will tell the user's DNS server where marketpath.com holds all their records. Verisign is curerntly the contracted agency in charge of maintaining the .com TLD.
- The DNS server then goes to the marketpath.com name server. This server answers all requests for marketpath.com and it's sub-domains (e.g. www.marketpath.com, mail.marketpath.com, etc). We use a reliable, hosted DNS provider called Zerigo to handle all of these requests. If Zerigo finds the domain or sub-domain, it will then send the destination IP address for marketpath.com - i.e. 22.214.171.124.
- The DNS server relays the resulting IP address back to the user's browser. The browser stops communication with the DNS server for that domain and redirects its attention to the destination web server for marketpath.com.
All of this happens in less than a second and once the first lookup takes place your computer and local DNS server will store the results in a cache so it doesn't have to perform the lookup every time.
This is often where delays occur when seeing a newly launched website. Your website agency or team says "Hey, the new site is live! Congrats!" but you may not see it for a while because your old website's IP address is cached in your browser, on your computer or in the DNS chain somewhere down the line. When your web development team tells you it could be between 1 - 48 hours before you see your new site they really mean it. Generally, there's a time-to-live (TTL) value stored with each DNS record that tells devices how long to maintain the cached value before looking it up again. Not all DNS servers adhere to that, however.
Web Server Request
Finally, we get to the web server which will eventually send you a web page with text and pretty pictures. If you're still here reading, then kudos to you. You've survived the onslaught of technical drabble that summarizes how we get to the final web server that has the website for marketpath.com. In reality, there are many more layers to this entire process where something can go wrong and cause it all to fail. Redundancy is the magic word and well thought out systems, including websites, have built-in redundancies and fail-overs that help mitigate any single point of failure.
Most experienced website developers build sites with some sort of dynamic capability. There are few websites with just static elements. Instead, sites more often use a content management system, like Wordpress, Marketpath CMS, Joomla, or Sitefinity, to name just a few. These systems provide easy-to-use tools for editing and publishing content to the website. Typically there is no need to know code. But the systems themselves are still built on top of a general web server architecture like Microsoft's IIS (Internet Information Services), Apache, or Node.js. They all answer and process web requests in similar ways. One common process is shown below.
- The user's browser sends the initial request to the web server asking for a single resource. If I type in "marketpath.com" I will implicitly be requesting the home page (i.e. http://www.marketpath.com/home).
- The web server has a service that is always listening for requests and captures that request.
- The service uses its route handler to decide if it needs to return a static file or send this to a script (dynamic code) for further processing.
- If the user is requesting a static file, it will grab the file and return it. If we're asking for the home page it is likely to be dynamic, and if so, it will move down the chain and begin processing the script.
- The script often uses common object libraries to retrieve information. For example, an e-commerce site may use it's store library and specifically request a product object.
- The requested object may then connect to the database where all the data is stored and return information for that product.
- Eventually, the completed request will send the data, whether static or dynamic, back to the web service.
- The web server will send the content back to the user's web browser and begin rendering the html content. Starting at the top, it will look for additional resources that need to be loaded.
- Requests back to the web server are made for each additional resource.
And that's it! Easy, right? Ok, I'll admit, there's a lot going on here. And this is just one example. Web server processing methods can vary dramatically from this flow chart. However, this is a fairly common flow. Just keep in mind that when your site has problems and your developer or IT crew says "We're looking into it" you should be patient and understand there are a ton of places where something can go awry, many often out of your team's direct control.