Majority of the traffic over the internet is HTTP Traffic. There is a HTTP Client which wants some data from HTTP Server, so it creates a HTTP Request Message in the protocol understandable by the server and sends it. Server reads the message, understands it, acts accordingly and replies back with HTTP Response.
This complete process is abstracted by the tools like curl, requests libraries and utilities like Postman. Instead of using these tools and utilities, we shall go by the hard way and see HTTP messages in action.
The Webserver
For experimentation purpose let’s create a very basic webserver in Python Flask framework that exposes a trivial Hello World end point.
Python webserver script
from flask import Flask
app = Flask(__name__)
@app.route('/hello')
def hello():
return "Hello, World!"
app.run(port=3000)
Installing requirements
pip install flask
Start the webserver
python hello.py
The server listens on port 3000 . If you hit from the browser http://localhost:3000/hello, you should see Hello, World! rendered.
The HTTP Request Message
A HTTP Client talks to HTTP Server via a common protocol that is understandable by the two parties. A sample HTTP request message looks something like
GET /hello.html HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.sample-server.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
To understand more about HTTP Request messages, see references at the end of this article.
The HTTP Communication happens over a TCP Connection. So we create a TCP connection with the server and try to get response from it. To get a TCP connection I will use netcat.
Netcat
netcat is the utility that is used for just about anything under the sun involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6.
The webserver that was created above is listening on port 3000 . Lets create a TCP Connection and connect to it using netcat.
netcat localhost 3000
The command along with creating a TCP connection, will also open a STDIN. Anything passed in that input stream will reach the server via the connection. Lets see what happens when we provide This is a sample as input.
The input message given is not a valid HTTP message hence server responded with a status code of 400 which is for Bad Request. And if you closely observe the server logs on flask application, you will see an entry of our last request.
Since the server is a HTTP Server, so it understands HTTP request. Let’s create one to hit our exposed API endpoint /hello .
The HTTP request message for this request looks something like this
GET /hello HTTP/1.1
And you should see output like this
The HTTP Server understands the message sent from the client and it responded back as directed by the source code.
Complex Requests and HTTP Request Messages
GET method with query params and headers
Following method exposes an endpoint which accepts a query parameter named name, and returns a response with name in it.
from flask import request
@app.route('/user')
def get_user():
name = request.args.get('name')
return "Requested for name = %s" % name
HTTP Request Message
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
GET /user?name=arpit HTTP/1.1
Output
Basic POST Method example
Following method accepts form data via HTTP POST method and returns a dummy response with username and password in it.
from flask import request
@app.route('/login', methods=['POST'])
def login():
username = request.form.get('username')
password = request.form.get('password')
return "Login successful for %s:%s" % (username, password)
HTTP Request Message
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
POST /login HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 32
username=arpit&password=welcome
Output
POST Method with JSON Request Body
Following method accepts JSON data that contains a field id with integer value via HTTP POST method and returns a dummy response with id in it.
from flask import request
@app.route('/save', methods=['POST'])
def save_user():
user_data = request.json
return 'Saving user with id = %d' % (user_data.get('id'))
HTTP Request Message
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
POST /save HTTP/1.1
Content-Type: application/json
Content-Length: 30
{"id": 1092, "name": "Arpit"}
Output
Conclusion
The hard way to hit REST endpoints was not hard at all ;-) Stay curious and dive deep.
