Learn about Network and HTTP in JavaScript

Starting from this lesson, we are going to start exploring how JavaScript works in the backend. Before we start, you must familiarize yourself with some basic concepts.

Server and client

First of all, you need to know what is a server and what is a client.

Both the server and the client are computers. When you open a browser and visit a website, your computer will be the client.

The client will send a request to the server requesting certain data and resources. If the server is OK with that request, it will send an OK response back to the client and then start transmitting the requested resources.

If the server is not OK with that request or encounters a problem, an error response will be returned. There are several different types of error responses, each indicating a different type of error. We will delve further into them later.

Sometimes, the client will submit new data to the server, such as when you submit a new form. The server will take that data, perform some tasks, and maybe return new data back to the client.

The interconnected network of servers and clients forms what we call the Internet today.

Different network protocols

When requests and responses flow between clients and servers, they have to follow a specific format so both ends know how to process them. These formats are called network protocols.

For instance, there are different protocols for sending and receiving emails, sharing files, and even controlling another computer over the Internet.

We can't cover all of them in one lesson, so instead, let's focus on HTTP and HTTPS, which are the two protocols used by web applications, as well as TCP, which is the ancestor for many other protocols, so we have to include it here.

TCP

The TCP protocol is one of the most commonly used Internet communication protocols, and a lot of other protocols are created on top of it. This protocol demands that one computer must always be listening, waiting for other computers to start talking to it.

This computer has different "listeners", and they can listen for different kinds of communications at the same time, to make sure these listeners don't interrupt each other, each of them will take up one position (port) on that computer.

For example, when you receive emails, that email is sent to you using the SMTP protocol, which is created based on TCP. By default, our computer will always be listening on port 25 for emails.

For another computer to send data to the listening computer, it needs to "talk" to the listening computer through the correct port. A connection will be established if the listening computer can be reached and is listening on that port, and then the data transfer can begin.

In this case, the computer that is listening is the client, and the computer that is talking is the server.

HTTP and HTTPS

The Hypertext Transfer Protocol (HTTP) is used to retrieve named resources. This means that the client will first make a request to the server, asking for some resources, which are usually web pages, images, or static files.

If the server is OK with that request, it will return a 200 OK message back to the client, and start transferring the files. The HTTP request sent by the client has the following format:

text

1GET /index.html HTTP/1.1
2
3Host: example.com
4Accept-Language: en

The request starts with the HTTP method, which we are going to cover later, and then followed by the name of the resource, and the protocol version.

After that, you can include some extra information in the request body, such as the Host, Accept-Language, and so on.

Upon receiving this request, if the server is OK with it, it will return a response that looks like this:

text

1HTTP/1.1 200 OK
2
3Date: Sat, 09 Oct 2010 14:28:02 GMT
4Server: Apache
5Last-Modified: Tue, 01 Dec 2009 20:18:22 GMT
6ETag: "51142bc1-7449-479b075b2891b"
7Accept-Ranges: bytes
8Content-Length: 29769
9Content-Type: text/html
10
11<!DOCTYPE html... (here come the 29769 bytes of the requested web page)

The response starts with the protocol version, and followed by the response status. The response will also contain some extra information such as Date, Server, Content-Type, and so on. And finally, start transferring the requested resource.

Of course, in practice, you never actually had to do this manually. The browser does everything automatically for you when you type in the URL, which specifies the protocol, host, and path to the resource you want.

text

1http://example.com/2020/03/16/13_browser.html
2|     |           |                         |
3protocol   server             path

The HTTPS protocol works exactly the same, except it is encrypted. It uses something called the transport layer security (TLS) protocol to make sure that the communication between the client and the server is secure. The server has a private key and the client has a public key, the connection could only be established if the two keys match each other.

HTTP request methods

Since we are focusing on web development, we’ll only discuss the HTTP protocol in detail. Notice from our previous example, that our HTTP request has the keyword GET, which is called an HTTP method. There are several other methods besides GET, and each of them serves a different purpose.

The GET Method

The GET method is used to request data and resources from the server. When you send a GET request, the query parameters are embedded in the URL in name/value pairs like this:

text

1http://example.com/index.html?name1=value1&name2=value2

Note that the question mark (?) marks the beginning of a list of parameters. Each parameter forms a key/value pair (name=value), and the ampersand (&) is used to divide two different parameters.

The POST Method

The POST method is used to send data to the server, either adding a new resource or updating an existing resource. The parameters are stored in the body of the HTTP request.

text

1POST /index.html HTTP/1.1
2Host: example.com
3name1=value1&name2=value2

The DELETE Method

This one is very intuitive. It deletes a resource from the server.

The HEAD Method

The HEAD method works just like GET. Except the HTTP response sent from the server will only contain the head but not the body. Meaning if the server is OK with the request, it will give you a 200 OK response but not the resource you requested. You can only retrieve the resource with the GET method.

This is very useful when you are testing if the server works. Sometimes it takes a long time for the resource to be transmitted, and when you are only testing, you really only need a 200 OK response.

THE PUT Method

The PUT method is used to update existing resources, and it is similar to the POST method, with one small difference.

When you make a PUT request to a resource that already exists, it will update that resource. And making multiple identical PUT requests will have the same effect as making it once.

The POST method, however, will duplicate that resource every time you make the request.

Sending HTTP requests with HTML forms

Now that we know what an HTTP request would look like, it is time to talk about how to send one. The most common way of doing that is through HTML forms. It allows the user to fill out information and submit them as parameters in an HTTP request. Here is an example:

html

1<form method="GET" action="/program">
2  <div>Name: <input type="text" name="name" /></div>
3  <div>Message:<br /><textarea name="message"></textarea></div>
4  <div><button type="submit">Submit</button></div>
5</form>

Let's first look at the <form> tag. The method attribute specifies the HTTP method we are going to use, which is GET in this case. That means the parameters will be embedded inside the URL when you click Submit. The action points to the program that will be processing the submitted data.

If you look inside the <form> element, notice that the user input elements (both <input> and <textarea>) have name attribute, which defines the name of the parameter. Remember that the parameter is a key/value pair, and the corresponding value is the user input.

When you push the Submit button, the HTTP request will look like this:

text

1GET /program?name=Jean&message=Yes%3F HTTP/1.1

Sending HTTP requests with JavaScript

Besides the HTML forms, JavaScript can also be used to send HTTP requests. It can be done using the fetch() method like this:

javascript

1fetch("path/to/resource").then((response) => {
2  // Get the returned response status (200 OK)
3  console.log(response.status);
4  // Get the header of the response
5  console.log(response.headers.get("Content-Type"));
6});

By default, the fetch() method uses the GET method to make the request, but you can change that by specifying a custom method.

javascript

1fetch("path/to/resource", {method: "POST"}).then(...);

Or add extra information in the header, and add parameters in the body like this:

javascript

1fetch("path/to/resource", {
2  method: "POST",
3  headers: {
4    "Content-type": "application/x-www-form-urlencoded; charset=UTF-8",
5  },
6  body: "name1=val1&name2=val2",
7}).then(...);

However, using JavaScript to make HTTP requests does raise some security concerns. Because the user and the programmer aren't usually the same person, they might not have the same interests in mind. Obviously, you don't want a random web page to access your bank with credentials stored in your browser. This is why most browsers forbid JavaScript from making HTTP requests by default.

This can be very annoying because it is possible that the JavaScript code wants to access another domain for a legitimate reason. To solve this problem, the servers can include in the response saying that it is OK for the request to come from another domain.

text

1Access-Control-Allow-Origin: *

HTTP responses

After the server has received the request, it will return something back to the client, which is called an HTTP response. A typical response looks like this:

text

1HTTP/1.1 200 OK
2Date: Fri, 01 Apr 2024 12:00:00 GMT
3Server: ExampleServer/1.0
4Content-Type: text/html
5Content-Length: 1234
6
7<!DOCTYPE html>
8<html>
9. . .

200 is an HTTP response code, indicating the server is OK with the request, and will be transferring the requested resources immediately.

There are three categories of response codes in total. The 2xx responses indicate that an action is successful:

200 OK: The server is OK with the request, and will be transferring the resources.
201 Created: The request to create new resources has been fulfilled. This response usually contains a URL that points to the newly created resource.
204 No Content: The requested action has been successfully completed, but no content will be returned. This response is often for requests that do not need a response body, such as DELETE.

The 4xx responses indicate that a mistake happened on the client side:

400 Bad Request: An error occurred in the request, and the server could not understand it.
401 Unauthorized: The user should be authenticated before requesting the resources.
403 Forbidden: The user is authenticated but doesn't have permission to access the resources.
404 Not Found: The user is requesting a resource that does not exist.

The 5xx responses indicate that a mistake happened on the server side:

500 Internal Server Error: This is a generic error response, indicating something went wrong on the server side.
503 Service Unavailable: The server is not able to handle the request due to temporary overloading or maintenance.

Viewing HTTP requests and responses

Lastly, you can check the HTTP response directly in your browser. Go to a random webpage, and open the Developer Tools. Head to the Network tab, and you should see a list of resources that the browser had to retrieve to render the webpage. Click on one item, and should see the corresponding request and response.