Real Estate Market in Colombia

Click Here to see Code

Project description: With this code you will be abble to compare market prices for sellig or renting properties (Apartments/Houses/Offices) from diferent neighboorhoods or cities in Colombia, and get the information from each property publication (Rooms, bathrooms, M2, House Expenses, etc.)

This algorithm scrapes data from Mercadolibre Colombia and creates a database with property items and their Google geogrpahical location.

1. Define locations

Enter the link locations inside the “start_request” function by filtering in the web page

def start_requests(self):
        yield scrapy.Request(
            'https://listado.mercadolibre.com.co/inmuebles/apartamentos/venta/bogota-dc/suba/acacias/_DisplayType_LF',
            self.parse)

2. Run Spider

Run the script and automatically the file will be save in the folder where the script is. Modify the file’s name in the “custom_settings”:

 custom_settings = {
        'FEED_URI': 'inmuebles_' + str(datetime.today().strftime('%Y-%m-%d')) + '.csv',
        'FEED_FORMAT': 'csv',
        'FEED_EXPORTERS': {
            'json': 'scrapy.exporters.CsvItemExporter',
        }

Once finished the process, you’ll receive and email with the stats:

Global Stats
start_time	2020-01-05 23:44:09.213644
item_scraped_count	329311
elapsed_time_seconds	3853.932806
finish_time	2020-01-06 10:28:23.146450
finish_reason	finished

3. Final Table

Click Here to view an example that shows the structured data