Cryptocurrency Historical Data Scrapping in Python!

In this short blog, I will show you how to build a small scrapper in Python by using beautifulsoup and selenium in order to extract historical data of your favorite cryptocurrency from coinmarketcap.com.

Why Scrappying?

Scrapping historical data for a specific cryptocurrency is very useful for designing trading bots, for instance, by training a predictive model for a trading bot. If you create your model correctly, you will have a good chance at predicting future crypto prices!

Installing Beautifulsoup & Selenium

In order to extract the data, firstly we need to install "beautifulsoup," and "selenium". To do this, enter the command "pip install beautifulsoup4" in the terminal (needless to say you need to have PIP already installed). After that, you should execute ‘pip install requests’. Next, you enter `pip install selenium` then make sure you have already downloaded "chromedriver" (according to http://chromedriver.chromium.org/downloads)

from selenium import webdriver
import requests
import bs4
import csv


dateList=[]
highList=[]
marketCapList=[]

r = requests.get('https://coinmarketcap.com/currencies/bitcoin/historical-data/?start=20130428&end=20190423')
soup = bs4.BeautifulSoup(r.text,"lxml")

tr = soup.find_all('tr',{'class':'text-right'})
for item in tr:

    dateList.append(item.find('td',{'class':'text-left'}).text)
    highList.append(item.find_all('td')[2].text)
    marketCapList.append(item.find_all('td')[6].text)


row0=['date','high','market cap']
rows=zip(dateList,highList,marketCapList)
with open('coinmarketExample2.csv','w',encoding='utf-8',newline='')as csvfile:
    links_writer=csv.writer(csvfile)
    links_writer.writerow(row0)
    for item in rows:
        links_writer.writerow(item)