Menu Close

Python Requests Module Tutorial

Admin
Python requests module

In this article, you will learn all about the Python requests module along with examples. Request module in Python is one of the most popular modules to send HTTP requests to the server or API call. This is a highly downloaded package of Python and is also used by most Python developers.

If we want to learn requests module from scratch, Then you are on right place because in this guide we will see all the basic of requests module, who full fill your basic requirement.

In this article we will see what is requests module is and how to send HTTP methods using the requests module, By the end of this article, you will be able to work with the requests module.

Before starting requests implementation, let see a brief description of some HTTP and its methods.

HTTP Methods

HTTP stands for ( HyperText Transfer Protocol ). HTTP works as a request-response protocol between client and server. A client ( browser ) requests to the server and then the server process that requests and sends a response back to the client.

The process of sending requests is done by some methods which are described below.

  • GET:- GET request is used to access information or resources from the server using given URL.
  • POST:- POST request is used to send information or resources to the server.
  • PUT:- PUT request is used to update existing information or resources to the server.
  • DELETE:- DELETE request is used to delete the information or resources from the server.

Status Code

status code is a unique number that returns along with the response.
for example status code 200 represents a request that is successfully sent.
There is various status code available.

Why learn Requests Module in Python?

There are various reasons to use the requests module in Python.

  • Python request module is the Apache Licensed HTTP library.That allow http request using Python.
  • Python request module play most major role in REST API and Web Scrapping.
  • You can send various kinds of request to the server along with headers and request body.
  • This will very helpful When you are going to work with APIs.

What is Python Requests Module?

Requests module is an external python module that provides a facility to send all kinds of HTTP requests like GET, POST, DELETE, PUT, etc.
Python requests module is one of the most downloaded modules from the python external library.

Requests module is easy to use the library with a lot of features for example we can pass parameters in URLs, Custom headers as well as SSL verification.

Requests allow us to send HTTP requests, You can add headers, form data, multipart files, and parameters with a simple python dictionary and access the response data in the same way.

Installing requests

To work with the python requests module we have downloaded it because it is an external library. To download the requests module use the following command.

pip install requests

GET Request

HTTP GET request is used to getting information from the server.GET method is done by requests.get() method.

Syntax

requests.get(url, params = {key:value}, args)

Example: Send GET request to the server using requests.get() method.


import requests
url = 'https://httpbin.org/get'

# send get request
x = requests.get(url)

#print status code
print(x.status_code)

# print response text
print(x.text)

# access url
print(x.url)

#access content type
print(x.headers['Content-Type'])

# access cookies
print(x.cookies)

#json response content
print(x.json())


#pass parameters with the help of url
mydata = { 'key1': 'value1', 'key2': 'value2'}
x = requests.get(url, params = mydata)
print(x.url)


#or 

mydata = { 'key1': 'value1', 'key2': ['value2','value3']}
x = requests.get(url, params = mydata)
print(x.url)

Output


200
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.25.1",
    "X-Amzn-Trace-Id": "Root=1-60fcdf30-29ab522148cf13dd03b30f47"
  },
  "origin": "47.31.136.246",
  "url": "https://httpbin.org/get"
}

https://httpbin.org/get
application/json
<RequestsCookieJar[]>
{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.25.1', 'X-Amzn-Trace-Id': 'Root=1-60fcdf30-29ab522148cf13dd03b30f47'}, 'origin': '47.31.136.246', 'url': 'https://httpbin.org/get'}
https://httpbin.org/get?key1=value1&key2=value2
https://httpbin.org/get?key1=value1&key2=value2&key2=value3

In above x is the response object where all the information is stored.

The requests.get() the method has the following parameters.

  • URL:- Required
  • params:- Optional A distionary or list of tuples which is send as query string.By default it is none.
  • auth_redirects:- Optional A Boolean to enable/disable redirection. Default True
  • cookies:- Optional: A dictionary of cookies to send the specific URL. By default it is none.
  • headers:- Optional A dictionary of headers to send the specific URL.
  • timeout:- Optional: A number or tuple, indicating how many seconds to wait for the client to make a connection or send a response.
  • Verify:- Optional. A Boolean or a String indication to verify the servers TLS certificate or not.Default True
  • auth:- Optional A tuple to enable certain HTTP authentication.
  • proxies:- Optional. A dictionary of the protocol to the proxy URL. By default it is None
  • cert:- A String or Tuple specefy a cert file or key.
  • Stream:- Optional A boolean indication if the response should be immediately downloaded (False) or streamed ( True ).

POST Request

An HTTP POST request is used to send information or resources to the server.POST is done by requests.post() method.

Syntax

requests.post(url, data = {key: value}, json = {key: value}, args)

Example


import requests
mydata = { 'key1': 'value1', 'key2': 'value2'}
url = 'https://httpbin.org/post'
x = requests.post(url, data = mydata)
print(x.json())

Output


{
   "args":{
      
   },
   "data":"",
   "files":{
      
   },
   "form":{
      "key1":"value1",
      "key2":"value2"
   },
   "headers":{
      "Accept":"*/*",
      "Accept-Encoding":"gzip, deflate",
      "Content-Length":"23",
      "Content-Type":"application/x-www-form-urlencoded",
      "Host":"httpbin.org",
      "User-Agent":"python-requests/2.25.1",
      "X-Amzn-Trace-Id":"Root=1-60fce2ea-57eca3584cc7be9909b6a6c4"
   },
   "json":"None",
   "origin":"47.31.136.246",
   "url":"https://httpbin.org/post"
}

Sometimes data arguments have multiple values for each key. This can be done by making tuples or dictionaries.

Example: Send multiple values using tuple


import requests
mydata = [('key1', 'value1'),('key1': 'value2')]
url = 'https://httpbin.org/post'
x = requests.post(url, data = mydata)
print(x.text)
print(x.json())

Example: Send multiple values using a dictionary


import requests
mydata = {'key1': ['value1', 'value2']}
url = 'https://httpbin.org/post'
x = requests.post(url, data = mydata)
print(x.text)
print(x.json())

POST Multipart-Encoded File

Requests make it simple to upload Multipart-encoded files.

Example: Send multipart-encoded file using requests.post()


import requests
url = 'https://httpbin.org/post'
data = open('C:/Users/Vishvajit/Desktop/Course.txt', 'rb')
mydata = {'key1': data}
x = requests.post(url, data = mydata)
print(x.text)
print(x.json())

PUT Request

A PUT request is used to update the existing information or resources on the server. A POST request is sent by the requests.put() method.

Example: Send PUT request using requests.put()


import requests
url = 'https://httpbin.org/put'
r = requests.put(url, data = {'key':'value'})
print(r.json())

Output


{
   "args":{
      
   },
   "data":"",
   "files":{
      
   },
   "form":{
      "key":"value"
   },
   "headers":{
      "Accept":"*/*",
      "Accept-Encoding":"gzip, deflate",
      "Content-Length":"9",
      "Content-Type":"application/x-www-form-urlencoded",
      "Host":"httpbin.org",
      "User-Agent":"python-requests/2.25.1",
      "X-Amzn-Trace-Id":"Root=1-60fce73a-34e4b2e670d6d5390f58a3f6"
   },
   "json":"None",
   "origin":"47.31.136.246",
   "url":"https://httpbin.org/put"
}

DELETE Requests

DELETE request is used to delete the information or resource from the server. This is done by requests.delete() method.

Example: Send DELETE requests to the server


import requests
url = 'https://httpbin.org/delete'
x = requests.delete(url)
print(x.json())

Output


{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "application/json",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-US,en;q=0.9",
    "Host": "httpbin.org",
    "Origin": "https://httpbin.org",
    "Referer": "https://httpbin.org/",
    "Sec-Ch-Ua": "\" Not;A Brand\";v=\"99\", \"Google Chrome\";v=\"91\", \"Chromium\";v=\"91\"",
    "Sec-Ch-Ua-Mobile": "?0",
    "Sec-Fetch-Dest": "empty",
    "Sec-Fetch-Mode": "cors",
    "Sec-Fetch-Site": "same-origin",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36",
    "X-Amzn-Trace-Id": "Root=1-60fce91e-1a10794b5c2186a56864a455"
  },
  "json": null,
  "origin": "47.31.136.246",
  "url": "https://httpbin.org/delete"
}

Response Headers

Server response can be access by using the headers property. It returns response headers in the form of Python Dictionary.

Example: Access response headers using headers


import requests
url = 'https://httpbin.org/put'
r = requests.put(url, data = {'key':'value'})
print(r.headers)

Output


{'Date': 'Sun, 25 Jul 2021 04:39:15 GMT', 'Content-Type': 'application/json', 'Content-Length': '478', 'Connection': 'keep-alive', 'Server': 'gunicorn/19.9.0', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Credentials': 'true'}

You can check the value of the particular key.

Example


import requests
url = 'https://httpbin.org/put'
r = requests.put(url, data = {'key':'value'})
print(r.headers['Content-Type'])

Cookies

If a response contains some cookies, You can access them easily by using the cookies property.

Example: Access cookies using cookies property


import requests
url = 'https://httpbin.org/get'

# send get request
x = requests.get(url)
print(x.cookies)

To send your own cookies you can pass using the cookies parameter.

Example: Send your own cookies.


import requests
url = 'https://httpbin.org/get'
cookies = dict(cookies_are =  'ProgrammingFunda')

# send get request
x = requests.get(url, cookies = cookies)
print(x.json())

Output

{
   "args":{
      
   },
   "headers":{
      "Accept":"*/*",
      "Accept-Encoding":"gzip, deflate",
      "Cookie":"cookies_are=ProgrammingFunda",
      "Host":"httpbin.org",
      "User-Agent":"python-requests/2.25.1",
      "X-Amzn-Trace-Id":"Root=1-60fcee12-3e04180a33a949484d8370ec"
   },
   "origin":"47.31.136.246",
   "url":"https://httpbin.org/get"
}

Redirection and History

If you want to track redirection then you can use the history property of the response object. For example, techno2hindi redirects all HTTP requests to HTTPS.

Example: 


import requests
url = 'http://www.techno2hindi.in/'
# send get request
x = requests.get(url)
print(x.status_code)
print(x.history)

If you are using GET, PUT, POST, OPTIONS, DELETE, or PATCH then you can disable redirection by passing the allow_redirects parameter.

Example


import requests
url = 'http://www.techno2hindi.in/'
# send get request
x = requests.get(url, allow_redirects = False)
print(x.status_code)
print(x.history)

Timeouts

You can tell the request stop waiting for a response after a given number of seconds with the timeout parameter:

Example


import requests
url = 'https://github.com/'
r = requests.get(url, timeout = 0.01)
print(r.text)

Authentication

Authentication is the process of recognizing a user’s identity. It is the process of associating the incoming requests with a set of credentials. The provided credentials are compared to those files store in the database. If the user is verified by credentials, Gives permission to do what the user wants to do.

Here we will see different-2 types of authentication. Many web services require authentication with different-2 types.

Basic Authentication

HTTP Basic authentication is the process of identifying user identity. This is the simplest way to identify user identity. In this authentication, user provide their credentials (username, password)
along with HTTP requests.

Let’s see how can I do that using the request module in Python.

Example: Basic Authentication using requests


from requests.auth import HTTPBasicAuth
import requests
response = requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
print(response)

In the above example, replace the user with your user and pass it to your password.

Digest Authentication

This is the other most popular authentication. In Digest Authentication, a client authenticates then identity by providing their username and password.

Example: Digest Authentication using requests


from requests.auth import HTTPDigestAuth
import requests

url = 'https://httpbin.org/digest-auth/auth/user/pass'
response = requests.get(url, auth=HTTPDigestAuth('user', 'pass'))
print(response)

In the above example, replace the user with your username and pass it to your password.

Conclusion

So this is all about the Python requests library. The requests module in Python is the most loved module by most Python programmers because it is easy to use the library. Python requests module is capable of sending kinds of requests to the server along with headers and request body.

If you are going to work with API in your Python project, then believe me you can’t ignore this one.

I hope this tutorial will have helped you. If you like this article, please share and keep visiting for further Python tutorials.

Reference:- Click Here

◀Higher-Order Function in Python
How to Access a Column in DataFrame▶

About the Author: Admin

Programming Funda aims to provide the best programming tutorials to all programmers. All tutorials are designed for beginners as well as professionals.
Programming Funda explains any programming article well with easy examples so that you programmer can easily understand what is really going on here.

View all posts by Admin | Website

Related Posts